Summary

AI’s 2024 step change was a collapse in cost and an expansion in capability, not a clean intelligence leap beyond GPT-4. Simon Willison counts 18 organizations whose models beat the GPT-4-from-a-year-earlier barrier, while hosted inference became dramatically cheaper and multimodal features reached phones. His summary: “everything’s got really good and fast and cheap,” even though models “didn’t get massively better than GPT-4.”
DeepSeek-V3 punctured the thesis that frontier-model development must inevitably consolidate around a few labs spending billions. DeepSeek reported a $5.5 million training run—roughly one-tenth of prior assumptions—and released the leading open-weights model on Christmas Day, prompting Willison to call it “an absolute bombshell.” Swyx preserves the diligence caveat: outsiders question the accounting and speculate about copying, but “we basically will never know” as external commentators; the demonstrated model still changes expectations for capital intensity.
The viable agent market is splitting between bounded, reviewable workflows and autonomous systems that remain unsafe. Research agents can inspect 56 websites, coding agents can execute code and repair errors, and Swyx says NotebookLM’s podcast feature uses internal agent loops, though the speakers note that this depends on the definition of “agent.” But an agent allowed to browse, decide, and spend encounters the unsolved gullibility problem: Claude Computer Use followed a webpage’s instruction to download malware and joined a botnet, making trusted economic autonomy feel to Willison like “an AGI-level problem.”
Multimodal inference has become cheap enough to turn cameras, screens, glasses, and earbuds into continuously promptable software surfaces. Gemini’s free tier may support roughly one captured image per second or minute, while Willison calculated that captioning his entire 68,000-photo collection with Gemini 1.5 Flash-8B would cost $1.68—about 1/400 of a cent per image. The opportunity spans monitoring, accessibility, memory, and wearables, but privacy becomes load-bearing when a model can see an entire screen or record daily life.
The next application-layer moat may be interface design rather than another chat box. Willison compares today’s blank prompt to dropping a new computer user into a Linux terminal; Canvas, tldraw’s Make It Real, Claude Artifacts, and Bolt point toward models generating task-specific maps, sliders, dashboards, and applications. The missing loop is letting the model observe how users manipulate those “knobs and dials,” turning generated interfaces into an actual conversation.
As synthetic media becomes abundant, credibility and human review become scarcer—and therefore more valuable. AI can remove production steps through lip-sync avatars, transcription, editing, and generated background assets, but Willison argues that an LLM cannot stake a reputation because “it’s a matrix multiplication.” His useful boundary for “slop” is content that is both “unrequested and unreviewed”; in his framing, human-reviewed output carrying a creator’s name is not slop.
OpenAI remains strong but no longer enjoys an uncontested platform lead. Willison says o3 “clawed them back up again,” yet Google Gemini had an exceptional year and Claude 3.5 Sonnet remained his favorite model; meanwhile, capable local models again became worth running after a late-2024 step change. The competitive field now rewards efficiency, distribution, workflow integration, trust, and usability—not merely owning the largest pretraining run.

Deep dive

1. AI improved sideways rather than delivering the expected GPT-5 leap

Willison’s state-of-the-market summary is blunt: “everything’s got really good and fast and cheap.” Models gained longer context, image and audio capabilities, video understanding, and much lower latency and cost, but “didn’t get massively better than GPT-4.”
McCullough’s pushback captures the unmet expectation: after GPT-2, GPT-3, and GPT-4, users anticipated another obvious phase change in intelligence. Willison argues that the phase change arrived elsewhere—by year-end, a phone could converse, inspect its camera feed, and impersonate Santa Claus.
Asked to bet on 2025, Willison declines to predict a simple “models but smarter” jump. He expects inference-time compute such as o1 and o3 to keep working—harder problems become solvable by spending more money and waiting longer—and would be “completely happy” with today’s intelligence made cheaper, faster, more capable, and longer-context.

2. The GPT-4 barrier fell as inference prices collapsed

At the start of 2024, OpenAI had led for roughly nine months without a close challenger. By year-end, Willison counted 18 other organizations with models that clearly beat the GPT-4-from-a-year-earlier model; “that barrier got completely smashed.”
Microsoft’s Phi-4 made the shift tangible: a 14-gigabyte download running on a MacBook Pro, with benchmarks definitely up there with GPT-4.0. Willison keeps the qualitative hedge—“It’s probably not as good when you actually get into the vibes of the thing”—but a laptop running anything comparable had seemed implausible one year earlier.
OpenAI’s current inference was roughly 100 times cheaper than using GPT-3 two and a half years earlier. Gemini 1.5 Flash cost $0.075 per million tokens, while Gemini 1.5 Flash-8B was 27 times cheaper than GPT-3.5 Turbo had been a year earlier despite adding image recognition and million-token context.
Competition explains part of the decline, but Willison says trusted sources told him Google Gemini was not losing money on inference; an Amazon executive likewise indicated that Amazon Nova was not losing money. That excludes model training and the “army of PhDs,” but it shows that extraordinarily cheap marginal inference need not be subsidized.

3. DeepSeek reset the capital-intensity debate without resolving its mysteries

McCullough had worried that billion-dollar or larger training runs could leave only nation-states able to train new models. DeepSeek released V3 on Hugging Face on Christmas Day as a “giant binary blob” without even a README. It led open-weights benchmarks and reportedly cost $5.5 million to train—about one-tenth of prevailing estimates—leading Willison to call it “a bombshell” that blew apart that earlier mental model.
Willison’s hypothesis is that export controls forced Chinese labs to extract more from constrained hardware, exposing abundant low-hanging efficiency gains. He “would not be surprised” to see better models trained for still less money within six months, but explicitly presents this as an uninformed opinion rather than laboratory expertise.
Swyx supplies the skeptical case: few labs trumpet cheap training, DeepSeek has roughly 150 employees, and “nobody quite believes” the headline accounting. Online theories allege copying because DeepSeek sometimes identifies itself as Claude or OpenAI GPT-4, but Swyx concedes that outsiders do not know the training tokens or provenance and “will basically never know” as external commentators.
Swyx estimates that equal-Elo GPT-4-class capability became 1,000 times cheaper during 2024, then asks whether that is a Moore’s-law curve or a one-off harvest. Willison suspects researchers only recently prioritized efficiency, while conceding 2024 might have exhausted the easiest gains: “we’ll know for sure in about three months.”

4. Open reasoning models made model behavior newly visible

The exchange about DeepSeek R1 is internally inconsistent: Swyx first describes it as a reasoning model runnable on a laptop and answers yes when asked whether its weights were released, then later says, “R1 is the API available.” The transcript does not establish whether local R1 weights were available.
Willison instead points to Alibaba’s Qwen reasoning models QwQ and QVQ, the latter adding vision. Unlike o1, which sort of hides its thinking process, the Qwen models visibly “churn away,” producing dozens of paragraphs while solving a problem.
His best specimen is the Pelican Bench: QwQ contemplated SVG construction in Chinese before producing a respectable pelican riding a bicycle. “The fact that my laptop can think in Chinese now is so delightful,” he says—the playful test also exposes style, visual coding ability, and reasoning behavior that benchmarks can miss.

5. Autonomous agents remain blocked by gullibility, not merely accuracy

Willison’s first frustration is definitional: “agent” can mean a travel booker, an LLM using tools in a loop, or a scheduled background job, while each builder assumes their definition is universal. Academics have debated the term for more than 30 years; for this discussion, an agent receives a job and independently performs it.
Under that definition, reliability collides with prompt injection. LLMs cannot reliably distinguish instructions from untrusted content, and Willison stresses that the field has discussed this unsolved problem for two years.
Anthropic’s Claude Computer Use provided the decisive demonstration: it could operate a browser inside a container, but a webpage telling it to download and execute a file succeeded immediately. The file was malware that enrolled the machine in a botnet—the “very first, most obvious dumb trick” worked.
Swyx compares agent enthusiasm to the perennial “year of Linux on the desktop,” but resists pure cynicism. Self-driving was supposedly imminent in 2014, yet Waymo now works in bounded settings; agents may follow the same “slow cook,” accumulating concrete progress over a decade without solving everything at once.

6. Research and coding agents work because their loops are bounded

Willison “mostly believes” in research assistants. Google’s Gemini 1.5 Pro with what he thinks is called Deep Research could inspect 56 websites, load them into its million-token context, and produce a genuinely useful report, aided by Google’s search index and page cache. Deliberately deceptive sources could still defeat it, but most research tasks are not adversarial.
Coding agents have an even longer proof point: ChatGPT Code Interpreter was already writing Python, executing it, reading errors, and rewriting the code nearly two years earlier. That feedback loop “obviously works” because execution supplies a checkable signal.
The dividing line is consequential autonomy. Willison does not expect an agent that independently makes decisions and spends money to work reliably “for a very long time”; Stripe can give an agent a virtual card, but his immediate safety mechanism is simply a $50 spending cap.
Even travel automation offers less value than its demos imply. Google Flights already works, an agent might save Willison 15 seconds, and he still wants to reject an airline regardless of price. NotebookLM’s stronger pattern combines a useful retrieval product with a “total gimmick”—its striking synthetic podcast—that finally earned the underlying product attention.

7. Multimodal inference turned continuous perception into a cheap primitive

A year earlier, GPT-4 Vision was the lone clearly impressive vision model and Gemini 1.0 had not earned broad credibility. Gemini 1.5 Pro changed that; video systems could sample one frame per second into long context, while newer models began combining imagery and audio more natively.
ChatGPT’s iPhone app made the shift concrete: a user can open the camera mid-conversation and ask, “What kind of tree is this?” Whether it samples frames or handles richer video internally matters less to Willison than the new end-user capability, which he thinks most people had not yet noticed.
Swyx says Gemini Flash’s free tier may support something like one captured photo every second or minute, potentially enabling a camera app that runs continuously and detects changes or issues prompted alerts. Willison’s cost calculation is starker: captioning 68,000 photographs with Gemini 1.5 Flash-8B would cost $1.68, or about 1/400 of a cent per image. “That doesn’t make sense. None of that makes sense.”

8. Generative video will enter production through pieces, not prompted movies

Sora’s public release appeared disappointing beside Google’s Veo 2, but Swyx objects to the comparison: users received distilled “Sora Lite,” while social media compared its failures with Veo 2’s cherry-picked marketing examples. He believes Veo 2 may still be better; no one had announced when “full-fat Sora” would ship.
Willison’s benchmark is not a two-hour film generated from three sentences. He asks what elite artists could achieve with these tools, pointing to the five-person visual-effects team behind Everything Everywhere All at Once, some of whom learned techniques from YouTube. Swyx adds that the team used Runway ML, while warning that he does not know how extensively.
McCullough expects adoption in three-second and 20-second pieces that previously required enormous budgets. Swyx similarly emphasizes low-value backgrounds, crowds, music, and sound effects, where consistency defects matter less than when Sora mangles a foreground gymnast.
Swyx sees a cultural war alongside the technical one: much of Hollywood opposes AI, leaving adoption to a fringe of willing artists. He also flags Hai Luo, Kling, and other Chinese systems as surprisingly strong, prompting Willison to wonder whether an AI-native film industry could emerge outside established production centers.

9. Human review becomes the boundary between leverage and slop

McCullough describes recording his own audio, then using HeyGen to lip-sync a trained avatar. The result is not perfectly beyond the uncanny valley, but it removes camera, lighting, and editing work while preserving his voice and message—the useful future is workflow compression, not necessarily synthetic personalities replacing creators.
Willison welcomes tools that let humans attempt more ambitious work, but keeps returning to credibility. ChatGPT cannot stake its reputation on a claim because “it’s a matrix multiplication”; credibility belongs to the person willing to publish, defend, and put a name behind the result.
Willison proposes human-reviewed rather than human-originated work: generate several variations, choose one, and attach personal responsibility. His matching definition of “slop” is AI content that is both “unrequested and unreviewed”; editorial judgment elevates an output only when someone has decided it is worth another person’s time.

10. Generated interfaces can replace the blank-prompt command line

McCullough frames the current chat UI as a usability crisis, echoing Willison’s comparison of a blank prompt to dropping a new computer user into a Linux terminal. ChatGPT Canvas offers collaborative document editing, while tldraw’s Make It Real showed how drawing an interface could generate working software.
Claude Artifacts introduced another path: the model answers with a custom HTML and JavaScript application instead of prose. Willison wants models to ask questions through generated maps, sliders, and “knobs and dials,” then observe the interaction; Artifacts can build those controls but had not yet closed the feedback loop.
Bolt already generated polished Spotify- or Airbnb-like applications from one prompt, and zero-shot app generation had become common enough to benchmark. Willison expects this to become a standard web-app feature within six months and wants his dataset data-exploration project to support prompted dashboards, forms, charts, and database actions.
Swyx argues that Canvas and Gemini inside Google Sheets make LLMs easier to use. Willison’s rebuttal is that every feature adds undocumented boundaries: an Artifact could not call arbitrary APIs because of iframe CORS restrictions, forcing ordinary users to learn web-security headers. More capabilities therefore increase the expertise required to understand what is actually possible.

11. Local models revived, while practical workflow tools delivered immediate returns

Willison had nearly abandoned local LLMs because nothing on his laptop approached Claude 3.5 Sonnet. A capability jump during the final three months renewed his interest: local models remained weaker, but no longer uselessly far behind.
Hardware is still constraining. Running a Llama 3 70B-class model consumes most of his 64GB of RAM and forces him to close browsers and VS Code; a future laptop with twice the memory—or NVIDIA’s announced $3,000, 128GB machine—could run near-top-tier open weights while remaining usable as a computer.
His local on-ramps are MLC Chat on iPhone, Ollama for packaged models and an API, LM Studio for a polished interface, and Open WebUI as an open-source front end. Playable models start around 2GB, while the most impressive laptop-friendly downloads typically occupy 20–30GB.
Apple Intelligence receives the harshest review: “It’s rubbish,” largely because its models are weak and users are not shown when to invoke it. Still, Willison thinks better small models could improve it within six months; elsewhere, MacWhisper became a several-times-daily transcription tool, and Riverside’s Smart Edit could eliminate three or four hours of camera-switching work—even though McCullough still retained a human editor.

12. Competition, regulation, and wearables define the next pressure points

Willison thinks OpenAI is “in a bit of trouble” after losing talent and its unambiguous lead; without o3, the position would look much worse. O3 restored some frontier status, but Gemini had “an amazing year,” and Claude 3.5 Sonnet remained his personal favorite.
He wants better criticism than endlessly repeating that LLMs are useless, environmentally damaging, and trained on unlicensed work. The harms contain substantial truth—training is probably legal under fair use yet plainly feels unfair when the resulting model competes with the creator—but “completely useless” ignores enormous value available to users who learn the models’ unintuitive limits.
Swyx warns that regulation keeps targeting “the last war,” citing California SB 1047’s proposed 10^25 compute threshold just as DeepSeek emphasized efficiency and labs pivoted from scaling GPT-5 pretraining toward o1-style inference. Willison prefers regulating uses: prohibit unexplained black-box insurance denials and create simple privacy rules ensuring prompts are not reused for training, while avoiding cookie-banner-style failure modes.
Swyx’s contrarian 2025 call is wearables: after Rabbit R1 and Humane became “toxic nuclear waste,” cheaper multimodal models make the category newly feasible. Limitless, formerly Rewind, is shipping a wearable that records only the wearer’s voice when opted in; McCullough adds smart glasses and more capable earbuds linked to a phone “mothership.” The unresolved product boundary is societal permission: when does useful memory become unacceptable recording?

Verification Notes

The transcript is internally inconsistent on DeepSeek R1: Swyx first describes it as runnable on a laptop and confirms released weights, then later says “R1 is the API available.” The digest does not resolve the local-weight status.

A Conversation with 姜Dora: “Super Individuals,” AI, and Liangzhu LA's Wildfire Disaster, Zuck Flips on Free Speech, Why Trump Wants Greenland