No Priors Ep. 100 | With Sarah and Elad
Summary
- DeepSeek was a real open-source reasoning advance, not a $5.5 million repudiation of frontier compute. Elad said comparable final runs already cost roughly $5–10 million and suspected hundreds of millions preceded DeepSeek’s distilled result, making NVIDIA’s roughly 20% selloff “a bit unwarranted.”
- The larger investment signal is a 180x decline in GPT-4-equivalent inference cost per token over 18 months. Benchmark gaps are narrowing, yet Sarah stressed that even a multiple of DeepSeek’s reported cost remains far below a multibillion-dollar or Stargate-sized entry price—a genuine “narrative violation.”
- Frontier leadership still buys distribution, workflow lock-in and a possible recursive research advantage. Better models can generate synthetic data, label data and write code for successors; whether that produces “liftoff” is uncertain, while widely available base models could instead become “a big leveler.”
- OpenAI’s Deep Research immediately raises the analyst bar, but its outputs cannot be treated as authority. Elad said he would compare median analyst or intern work against it because “the comp is hard”; Sarah found it especially useful for surveying unfamiliar domains, but in domains she knows well, users are “really going to have to audit the outputs.”
- AI’s emerging control over knowledge makes model plurality and open source strategically important beyond economics. Elad compared AI’s blind spots to Gell-Mann Amnesia: people spot errors in familiar fields, then trust the same system elsewhere; Sarah warned that an AI combining search, social networks and media into “one single device that you interrogate” creates censorship and propaganda risk.
- Stargate reflects uncertainty about scaling returns, not evidence that capital has stopped mattering. Elad said he could not imagine an AGI lab not wanting “the biggest cluster they could have” if it were free or financeable, although he considered continued pretraining gains likely to become less efficient.
- Their 2025 map favors foundation-model consolidation, vertical AI, agents and autonomy, with robotics still at the proof stage. Harvey, Decagon, Sierra, Cognition, Tesla, Waymo and Applied Intuition illustrate the opportunity set; consumer resurgence may come from cheap low-latency models, while biology, materials and health could benefit from smarter domain-specific data generation.
Deep dive
1. DeepSeek advanced the frontier without rewriting compute economics
Elad’s read was that DeepSeek mattered but remained “roughly on trend”: it delivered a state-of-the-art Chinese, open-source reasoning model and genuinely novel reinforcement-learning techniques that other laboratories were beginning to adapt.
The reported $5.5 million covered a final run, not the entire program. Elad said knowledgeable practitioners put comparable runs around $5–10 million and believed DeepSeek likely spent hundreds of millions on tooling, data, experimentation, pretraining and post-training; NVIDIA’s roughly 20% decline therefore looked “a bit unwarranted.”
Sarah emphasized the release sequence: V3 appeared in December without crashing NVIDIA, while R1—a reasoning counterpart to OpenAI o1—produced the “narrative violation.” Post-training made the model useful, and a Chinese laboratory’s rapid catch-up challenged a 20-year US-versus-China technology-dominance story. She also noted that DeepSeek’s mobile app briefly became a top App Store contender; Elad thought the attention was more likely curiosity about the leading Chinese model than proof that consumers choose the cheapest capable model.
2. Falling costs commoditize capability but do not erase frontier value
Sarah’s pushback—worth keeping—was that even a sizable multiple of $6 million is not a multibillion-dollar or Stargate-sized entry price. Elad answered that the direction was already obvious: GPT-4-equivalent inference cost per token had fallen “180x, not 180%” in 18 months.
Elad pointed to Artificial Analysis’s independently rerun benchmarks: across reasoning, knowledge, math, coding, multilinguality and cost, leading models were getting “closer and closer” rather than separating. A new breakthrough might temporarily leapfrog the field, but convergence was the prevailing trend.
Frontier leadership still offers market share, optimized-prompt and tooling stickiness, plus better synthetic data, labeling and coding for the next model. Elad hedged the stronger “liftoff” claim; Sarah added that broadly available high-quality base models could instead be “a big leveler” for self-improvement.
3. Deep Research raises the analyst bar while weakening epistemic visibility
Elad said Deep Research immediately raises the bar for knowledge work and that he would compare the work of a median analyst or intern against it: “the comp is hard.” Sarah found it especially useful for surveying unfamiliar domains, building a comprehensive view and identifying experts.
Her reservation was its implicit authority ranking: when deciding which web claims and ideas were good, it often required users to “audit the outputs.” It could orient you, but its conclusions could not simply be taken as given.
Elad invoked Gell-Mann Amnesia: readers recognize errors when a newspaper covers their specialty, then trust its next page on an unfamiliar subject. He said AI could become a definitive information source while making its sources less evident. Sarah warned that a system combining search, social networks and media into “one single device that you interrogate” creates propaganda and censorship risk; she viewed a multi-AI, multi-company world as an offset and open source as especially important for civil liberties.
4. Stargate exposes uncertainty around scaling returns
Elad framed infrastructure demand as “uncertainty rather than risk.” Algorithmic efficiency, synthetic data and test-time scaling make capability gains difficult to forecast, but he could not imagine a serious AGI laboratory declining “the biggest cluster that they could have if it was free.”
That revealed his underlying call: pretraining should continue producing gains, though probably less efficiently. He offered no strong conclusion on capital-market depth or sovereign involvement, preserving those as open questions rather than treating Stargate’s scale as self-validating.
5. Vertical AI and agents headline the 2025 map
Elad expected partial foundation-model consolidation—especially in image, video, voice and secondary models—if the FTC became friendlier, alongside new races in physics, biology and materials. His central application call was an “era of vertical ops,” citing Harvey, Decagon, Sierra and increasingly agentic products such as Cognition.
Sarah defined agents pragmatically as systems that “do multi-step tasks successfully,” manage state and act beyond content generation. Security, support, SRE and coding already supplied examples; copilots should naturally expand by taking on more work and getting better at handling failures.
Elad expected self-driving to command attention through Tesla and Waymo, with Applied Intuition as his “dark horse.” Sarah expected technical proof of breakthroughs in robotics and generalization this year, “though not deployments”; Elad similarly anticipated only a “glimmer of how this thing will work versus the whole thing.”
Elad predicted a consumer resurgence, while Sarah said small, low-latency models could unlock free experiences, whether local or web-based in the browser; edge compute mattered only when it was transparent to the user. Elad also argued that reasoning may improve reliability as much as task complexity, so apparent technical failures deserve repeated reassessment. As innovation diffuses beyond the tip of the spear, he expects smarter domain-specific data-generation and data-capture strategies—potentially including biotech innovation—in biology, materials and perhaps health.