Andrej Karpathy — “We’re summoning ghosts, not building animals”

Summary

• Karpathy’s headline call: agents are a decade-long project, and “decade of agents” is the correction to industry over-prediction. Agents like Claude and Codex are impressive daily tools, but the test is whether you’d hire one as an employee — “they just don’t work”: no continual learning, weak multimodality, “cognitively lacking.” His decade estimate comes from 15 years of watching reputable people get timelines wrong, and problems that are “tractable… surmountable, but they’re still difficult.”
• The recursive self-improvement thesis takes direct fire from the person who just tried it. Building nanochat, coding agents were “not net useful” — they bloat, misunderstand custom code, and are “not very good at code that has never been written before,” which is exactly what AI research is. Karpathy concedes this is “why my timelines are a bit longer” than AI-2027-style forecasts, and calls current agent hype “slop… maybe they’re trying to fundraise.”
• RL as practiced is broken: “you’re sucking supervision through a straw.” Outcome reward upweights every token of a lucky trajectory — “It’s just stupid and crazy. A human would never do this” — and process supervision fails because LLM judges are gameable: a nonsense string like “dhdhdhdh” got 100% reward. He expects “three or four or five more” major algorithmic updates in that realm.
• Against the explosive-growth trade: AI won’t show up in GDP as a kink. “We’re in an intelligence explosion already and have been for decades” — computers and the iPhone are invisible in GDP because diffusion averages everything into the same exponential. Dwarkesh’s 20%-growth, AGI-is-labor pushback gets strong pushback: “We have God in a box… it just won’t look like that.”
• Self-driving is the deployment template: a “march of nines” where every nine is constant work. He saw a perfect Waymo drive in 2014; Waymo remains uneconomical, and Karpathy says human involvement may be greater than expected — “we haven’t actually removed the person, we’ve moved them to somewhere where you can’t see them” — and he’s on record that Tesla’s approach is more scalable. Production software has self-driving-grade failure costs, so expect the same slog.
• Despite all that, no overbuild call on compute: even as the current buildout would 10x available compute in a year or two and more than 100x it by the end of the decade, he says, “I don’t know that there’s overbuilding. I think we’re going to be able to gobble up what… is being built” — Claude Code and Codex didn’t exist a year ago. He acknowledges the railroads/telecom-bubble precedent but is “actually optimistic,” objecting only to timelines with “geopolitical ramifications” if miscalibrated.
• Model-size contrarian take: in 20 years, the “cognitive core” could be ~1B parameters because internet pretraining data is “total garbage” and today’s trillion-parameter models are mostly memory, not cognition — he wants less memory and more cognition. Frontier progress stays incremental across all fronts: “nothing dominates. Everything plus 20%.”
• His next act, Eureka, is a bet that post-AGI humans still self-improve: “pre-AGI education is useful. Post-AGI education is fun” — school becomes the gym, and the alternative (WALL-E/Idiocracy disempowerment) means “I don’t even care if there are Dyson spheres.”

Deep dive

Not yet available upstream; scheduled sync will retry.