Anthropic Co-founder: Building Claude Code, Lessons From GPT-3 & LLM System Design

Tom Brown, co-founder of Anthropic and lead author of the GPT-3 paper, shares his unconventional path into AI and the lessons learned building some of the most influential language models.

Early Career: The Wolf Mindset

Background

• Studied computer science at MIT
• First job was at a friend’s YC startup as employee #1
• Learned the crucial difference between “wolf mindset” and “dog mindset”
  • Dog mindset: Wait for someone to tell you what to do
  • Wolf mindset: Figure out what needs to happen and make it happen
• This entrepreneurial thinking became foundational for his career

YC Startup Experience

• Worked at Linked Language (language learning startup)
• Later joined Grouper (social dating app)
• These experiences taught him to be resourceful and self-directed

Self-Teaching AI

The Six-Month Journey

• Decided to transition into AI/ML around 2015-2016
• Self-study approach:
  • Andrew Ng’s Coursera machine learning course
  • Kaggle competitions for practical experience
  • Linear algebra fundamentals
  • Reading research papers
• Key insight: You don’t need a PhD to work in AI

Breaking Into the Field

• Applied to Google Brain but was rejected
• Connected with Greg Brockman through a friend
• Joined OpenAI in its early days

OpenAI and GPT-3

Early OpenAI Work

• Worked on Starcraft environment for reinforcement learning
• Transitioned to language models as the field evolved

The GPT-3 Project

• Lead author on the GPT-3 paper
• Key technical challenges:
  • Training on TPUs initially, then switching to GPUs
  • Managing massive distributed training runs
  • Debugging at scale (finding bugs in 1000+ GPU clusters)

Scaling Laws Discovery

• One of the most important insights: more compute = more intelligence, predictably
• This wasn’t obvious at the time - many thought there would be diminishing returns
• The scaling laws gave confidence to invest heavily in larger models

Infrastructure Lessons

• TPU to GPU transition taught flexibility in hardware choices
• Software stack matters enormously (PyTorch vs TensorFlow)
• Debugging distributed systems requires new tools and approaches

Founding Anthropic

Why Leave OpenAI?

• Group of people who took scaling laws very seriously
• Wanted to focus on AI safety alongside capabilities
• Left during COVID period (2020-2021)
• Founded with several OpenAI colleagues including Dario and Daniela Amodei

Early Anthropic Days

• Started with Constitutional AI research
• Focus on making AI systems more helpful, harmless, and honest
• Built infrastructure from scratch

Claude’s Success Story

The Turning Point: Claude 3.5 Sonnet

• Claude 3.5 Sonnet was the breakthrough moment
• Particularly strong at coding tasks
• First time Anthropic had a clearly leading product in a major category

Why Coding Worked

• Coding is a well-defined task with clear success metrics
• Models can be evaluated objectively (does the code run?)
• Rapid iteration possible based on user feedback

Building Claude Code

Origin Story

• Built initially for internal use at Anthropic
• Engineers wanted a better way to interact with Claude for coding
• Surprised when it became the best product in the market

Product Philosophy

• Empathy for Claude users - the team uses Claude Code daily
• Focus on developer experience
• Iterate based on real usage patterns

Key Features

• Terminal-based interface for developers
• Deep integration with development workflows
• Designed for extended coding sessions

Infrastructure Strategy

Multi-Chip Approach

• Anthropic uses three different chip manufacturers:
  • NVIDIA GPUs
  • Google TPUs
  • AWS Trainium
• This diversity provides:
  • Supply chain resilience
  • Negotiating leverage
  • Ability to optimize for different workloads

Software Challenges

• Each chip requires different software stack
• Significant engineering investment to support multiple platforms
• Trade-off between flexibility and engineering complexity

Advice for Young People

Career Philosophy

• Take more risks when you’re young
• Work on things that would make the idealized version of yourself proud
• Don’t optimize for prestige or conventional success metrics

On AI Careers

• The field is still accessible to self-taught practitioners
• Practical experience (Kaggle, open source) matters more than credentials
• Find problems you’re genuinely curious about

The Wolf Mindset Revisited

• Don’t wait for permission or instructions
• Identify what needs to happen and make it happen
• This applies whether you’re at a startup or a large company

Looking Forward

AI Progress

• Scaling laws suggest continued rapid improvement
• Coding is just the beginning - many domains will be transformed
• The gap between AI capabilities and human integration is the current bottleneck

Anthropic’s Mission

• Balance capability advancement with safety research
• Make AI systems that are genuinely helpful
• Build tools that augment human capabilities rather than replace them