
Doug O'Laughlin
Frontier Insights
Frontier Thesis: Agentic coding models like Claude Code are redefining knowledge work, turning code into the primary interface for complex labor while low-latency inference commands massive price premiums. Meanwhile, frontier models like Kimi K3 confirm China’s top-tier parity despite severe compute footprints.
Strategic Imperatives: Operators must exploit routing layers to optimize frontier costs and aggressively secure memory supply lines.
Risks & Bottlenecks: Systemic hardware crunches threaten deployment. HBM is cannibalizing standard DRAM capacity—spurring multi-year shortages and doubling memory/storage costs—while human review capacity, broken apprenticeship models, and data trustworthiness remain unaddressed execution chokepoints.
Key Views & Dialogues
Ep. 028 - Most Neoclouds Suck At Security: How Agents Hacked Hugging Face (Neoclouds, Security)
- 🗓️ Date:
2026-09-02| 🎙️ Show:SemiAnalysis
Neocloud security is counterparty risk as AI startups spend “60, 70, 80% of their venture capital” on GPUs. Hugging Face reached cluster-admin in 13 hours through a malicious README and missing Kubernetes admission controls, making basic isolation the decisive defense. CMAX Audit Security is actionable, but audit-as-a-service depends on closed models maintaining a lead over GLM, while unchanged CVE-to-patch ratios leave impact unresolved.
View Dialogue Notes & Key Takeaways
Neocloud security is a counterparty risk hiding in plain sight: AI startups spend “60, 70, 80% of their venture capital” on GPUs, and Jordan Nanos says “some deserve it and some really do not deserve that trust.” Hyperscalers set the enterprise bar; only certain neoclouds meet it, and ClusterMAX 3 will “name and shame” — the failures include a cross-tenant remote-code-execution hole at an Asian provider serving inference on OpenRouter, meaning users of OpenClaw or coding assistants on that endpoint were “conceptually exposing credentials.”
The Hugging Face hack was a 13-hour cascade from a malicious README to cluster-admin — and the fix was basic Kubernetes hygiene, not novel cryptography. The agent uploaded a README that read
/proc/self/environ, harvested root-level credentials and source on a worker pod, and moved laterally; Hugging Face had no admission policy rejecting root/hostPath pods, and Jordan believed OpenAI’s own out-of-date Linux kernel carried a publicly documented vulnerability called “Fragnesia.” Hugging Face claimed multiple clusters, implying lateral movement. Sam Harshe’s bar: “Just do the basics correctly.”The viral swarm behavior is being over-read: these were models “RL’d to hell to be extremely relentless,” not default frontier behavior — and Sam argues the real story is scale, not special intelligence. “If you could have an agent swarm of 70,000 GLM-5.3 Flash or whatever, you would probably get to the same outcome”; Doug’s related point is that persistence and willingness to burn tokens and probe every weak point matter more than mysterious intelligence.
Safety training currently punishes defenders: Hugging Face got refusals from Claude Haiku, Claude Opus, and GPT-5.2 while under attack and had to use GLM-5.3. SemiAnalysis hits the same wall doing cluster audits despite approved cybersecurity access (“it’s trained into the weights, it’s not a classifier or filter”), while attackers can use the obliterated GLM-5.3 red-team fine-tune — which Sam calls a useful “warning shot to everybody that anybody can do this.” Sam says the asymmetry favors attackers; Doug’s pragmatic step is KYC, though it implies a chunk of the non-KYC endpoint market “is going to go away.”
Jordan’s outcome-priced “security audit as a service” pitch turns out to be, in Doug’s words, “a finance bro’s perspective of Project Glasswing and Daybreak” — a potentially durable lab revenue line, but only while the closed-model lead holds. Sam’s caveat is that customers will not pay Anthropic if they can get comparable work from GLM for a fraction of the cost. Sam says Mythos trained perhaps in February and was announced April 7; Mythos 2 is “clearly better” but unreleased — the frontier open-vs.-closed gap “continues and is strong,” while the gap between usable, publicly announced open and closed frontier models “is clearly closed” and public cyber benchmarks are saturated: “strictly held-out private ones and vibes at this point.”
The empirical anomaly nobody is discussing: despite cheap model-assisted exploit-finding, GitHub data shows no CVE surge — the CVE-to-security-patch ratio is unchanged and one observed relationship was “probably not statistically significant.” There are more PRs and perhaps a slightly higher share of security-related changes. Doug’s puzzle is that models already seem capable of exploiting straightforward bugs in critical infrastructure, yet “we are not already seeing the world go to hell”; some “outer loop” appears to be holding things together.
The actionable output is a CMAX security audit utility:
pip install clustermax, checking NVIDIA, Docker, Linux, and AMD bulletins down to BlueField NIC firmware. The catalog of real-world failures includes BMCs open to the public internet, missing InfiniBand P/M/SA keys, multi-tenant Grafana, and single layers of isolation. “People are already starting to update some of their things”; customers should run the audit and demand upgrades.🔗 Original source & video: Ep. 028 - Most Neoclouds Suck At Security: How Agents Hacked Hugging Face (Neoclouds, Security)
Ep. 023 - Everyone Leaves Google, Elon Forecasts 1T ARR, Reflecting On GPT-5 | Jon from Asianometry
- 🗓️ Date:
2026-08-07| 🎙️ Show:SemiAnalysis
Google’s talent drain, including Jeff Dean, John Jumper, Noam Shazeer, and David Silver, raises questions about whether system-level judgment can be replaced by more compute. The risk is execution, not earnings: Google may remain highly profitable and strong in TPUs while quietly losing frontier-model leadership, as agentic coding accelerates demand for bespoke software and Terafab faces a heroic physical ramp.
View Dialogue Notes & Key Takeaways
Google’s talent drain is real, with Jeff Dean, John Jumper, Noam Shazeer, David Silver and several Gemini leads departing. Demis Hassabis is described as moving into a higher-level chairman/chief-scientist role while also serving as Isomorphic Labs’ CEO. The panel splits on whether this is merely “the dream team…breaking up as they all turn 38” or the moment Google loses unusually broad system-level judgment that compute alone cannot replace.
The bearish Google call is about execution, not earnings. Dylan argues that Google repeatedly invents foundational technology, fails to commercialize it, and risks becoming an AI-era Bell Labs: highly profitable, strong in TPUs and potentially “really good for the stock in the short run and medium run,” yet destined to “quietly bow out” of frontier models.
China restrictions increasingly mean banning the best product, rather than excluding cheap imitators. Jon supports reducing revenue flows to Chinese suppliers but says firms may route around narrowly drafted rules through Vietnam or Thailand. With China strong in optics and the 1.6T/16T transceiver supply chain primarily there, an InnoLight ban could hurt Western buyers more than its intended target.
Agentic coding redeemed the GPT-5 thesis. Jon says GPT-5 itself “probably wasn’t that great,” Doug calls 5.2 disappointing, but both see 5.6—and potentially “six”—as a major step forward: once agentic coding arrived, “everything hit the pedal to the metal.”
Bespoke software can be built for hundreds of dollars. With four attempts, a Claude-written specification and Codex 5.3 or something similar, Jon built a 40 MB video editor tailored to his keyboard-driven workflow; 5.6 later completed a previously stubborn feature after roughly 25 specification questions. His advice: “roll your own software,” the technical equivalent of making your own furniture.
AI may make the open internet toxic to machines. The panel’s mechanism is that software systems are too large for any human to understand, while a model can absorb the code and interdependencies in one context window, finding both high-level and memory-level exploits. Jon’s endpoint is “superhuman level breakage of the system forever,” potentially making nearby compute and models more attractive than constant web access.
Elon’s $1 trillion forecast requires heroic physical ramps: SpaceX discussed moving from 1 GW to 10 GW by end-2027, then 20 GW, while pulling its revenue target from 2031 into 2030. Jon expects Terafab to start with memory; Dylan argues its scarcity and margin make it the fastest route to revenue. The panel’s deliberately sharp hurdle—“anything under $100 billion at the end of Q4 ’27 is a miss”—highlights how far the arithmetic outruns the required ramp.
🔗 Original source & video: Ep. 023 - Everyone Leaves Google, Elon Forecasts 1T ARR, Reflecting On GPT-5 | Jon from Asianometry
[Emergency Episode] Moonshot’s Kimi K3 has Arrived! China has a Frontier Model
- 🗓️ Date:
2026-07-18| 🎙️ Show:SemiAnalysis
Kimi K3 is now a clear top-three model by benchmark composites, while Dylan ranks it second for practical use as Opus access remains frustrating and restricted. Its 2.8-trillion-parameter scale requires B300, GB300, or MI355X-class hardware, but $3/$15 per million input/output tokens suggests attractive economics; adoption, Western sovereign demand, and routing layers remain the key commercial variables.
View Dialogue Notes & Key Takeaways
Max calls Kimi K3 a clear third-best model, while Dylan says it may be second-best for his practical use. Max says benchmark composites show a clear, stable top three including Soul 5.6 and Kimi K3, ahead of Google, Meta, and SpaceX; he says Google should feel embarrassed. K3 is still slower and officially trails Fable 5 and GPT 5.6 full, but Dylan finds it less frustrating than Opus when access is rejected or limited.
K3’s 2.8 trillion parameters make inference capacity an immediate bottleneck. Dylan says it cannot fit in a B200 and needs B300, GB300, or MI355X hardware for service on a single 8-way HGX system; multi-node pipeline parallelism would hurt performance. Max speculates that the roughly 10-day weights delay gives serving teams time to prepare and may let Moonshot arrange licensing and GB300 capacity with Together, Fireworks, Nebius, and CoreWeave.
K3’s pricing highlights attractive frontier-model economics but a constrained buyer base. Pricing rose from $0.95/$4 for the previous 2.7 Code model to $3/$15 per million input/output tokens. Max argues that if K3 is probably not operating at negative margins, similarly sized Fable charging $10/$50 would imply extraordinary margins, potentially better than SaaS. Cost-conscious users may choose GLM-5.2 or MiniMax M3, while SemiAnalysis-like users continue using GPT-5.6 and Opus; Max doubts large enterprises will adopt K3 seriously.
Dylan attributes the narrowed open-versus-closed gap squarely to US restrictions that limit access to Anthropic’s best models. He points to Mythos versus Fable, says he cannot use Opus and can use Sonnet only sometimes, and thinks Soul 5.6 may not be OpenAI’s largest model. He expects policy changes or another frontier step near the end of summer, while also arguing that open models may eventually reach true frontier parity.
Western open source has a sovereign-computing opportunity beyond cost savings. Max sees strong demand for a Western model that does not suck and expects many US enterprises to reject Chinese weights even in air-gapped systems; he also says a Chinese-model ban may be coming. Dylan says Western models must beat both bulk Chinese open models and tier-2/3 frontier-lab models, with government as the larger market. K3’s quantization during SFT—MXFP4 weights and MXFP8 activations—supports broad hardware compatibility and China’s domestic-accelerator push.
Harnesses, routing, and untapped adoption still support frontier-lab revenue. Max says model quality is increasingly difficult to distinguish across maximum, high, and medium effort, making features in tools such as OpenCode, Hermes, and Pi decisive. Dylan suggests outcome-based pricing could produce 95%-plus margins, and Max agrees that routers and adjustable thinking effort could do so. Max sees substantial room for new use cases across coding, audio, video, deep research, robotics, and world models; Dylan and Max believe new adopters will overwhelm any K3 substitution and prevent ARR-growth deceleration.
🔗 Original source & video: [Emergency Episode] Moonshot’s Kimi K3 has Arrived! China has a Frontier Model
Ep. 011 - GPT 5.5 vs Claude 4.7: OpenAI’s Comeback From the Brink (Tokenomics) | Jordan Nanos, Dylan Patel, Doug O’Laughlin, Max Kan
- 🗓️ Date:
2026-05-06| 🎙️ Show:SemiAnalysis
GPT-5.5 brings OpenAI back into the frontier conversation after Anthropic surpassed it on a like-for-like revenue basis, but Claude 4.7’s quality advantage over 4.6 remains unproven despite a 6x fast-mode premium. Token costs are beginning to pressure heavy users as new tasks drive Jevons-style consumption, while China’s compute constraints widen the open-source gap and revive the CLI-versus-app battle over agent orchestration.
View Dialogue Notes & Key Takeaways
OpenAI is back “in the conversation” with GPT-5.5 after a stretch Max Kan calls “really dire.” Anthropic—riding Opus 4.5’s step change in coding and agentic ability—passed OpenAI on a like-for-like revenue basis in early-to-mid April (a leaked ~$19B ARR versus ~$24B at the start of the year), and GPT-5.4 “was honestly just an embarrassment” whose model card didn’t even compare against Opus. 5.5 is back on the frontier but is not “definitively better” than Opus 4.6/4.7 “despite what the Twitter propaganda machine was trying to push.”
Dylan Patel’s release calendar: “Everyone’s releasing in two weeks”—Google and OpenAI definitely. The “Spud” (5.5) shipped with pre-training unfinished, so the next drop is finished pre-training plus more RL; Google’s is expected to be “mostly just” a multimodal swap.
Fast-mode premiums are decaying while the price stays 6x: Opus 4.6 fast has slipped from 2.5x to under 2x speedup (90→70 tok/s against an unchanged 35–40 base). Yet this is the first time SemiAnalysis engineers chose fast over higher-quality tokens—Jordan Nanos’s read is that “4.7 is just not meaningfully better quality than 4.6 for people today.”
Token pricing is starting to price out even heavy professional users. Dylan says the desk is “on the cusp” of being priced out; Mythos was quoted ambiguously at $25/$150 or $25/$125 versus Opus’s $5/$25, while Doug characterized it as roughly 5x, with fast mode 6x on top. Doug’s $800 boil-the-ocean scraping run versus a $55–100 data-enrichment API is the cautionary tale—and cost growth comes from new tasks (Jevons), not repricing old ones.
Doug’s structural bear case for frontier pricing: Opus 4.5 may have crossed the threshold where day-to-day tasks are one-shottable without supervision. GPT-5.5-level intelligence in a 100–200B-parameter form factor in “probably less than a year” might mean the majority of people do not need frontier-level intelligence.
Benchmarks have degraded to “a vibe check to make sure that the model’s not total trash.” Humanity’s Last Exam is esoteric multiple choice; SWE-bench scrapes GitHub issues with implementation-scoped unit tests. Meanwhile, 4.7’s new tokenizer can cost 35% more for identical output, and the desk’s truther take—“Opus 4.7 is actually Sonnet”—captures how small the model smells.
The China open-source gap is widening again, per Dylan’s flat “Yes”—and it’s a compute-constraint story. Chinese frontier weights conveniently fit an 8x H200 pod’s memory domain, Ascend kernels only partially serve DeepSeek V4, and Max expects Meta—behind today but signing monster compute deals—to “pull away from all the Chinese guys” by H2’26 or H1’27.
The form-factor fight is live: Dylan calls the CLI “a foregone relic” and says OpenAI’s app holds the true agent-orchestration vision; Max’s rebuttal: “it’s CLI all the way down. Pure maxi vision.” Tri Dao’s cracked-kernel workflow, via Dylan’s group chat: have Codex write it, then Opus fix the slop—“you can’t go the other way around”—though “everyone else at the firm prefers the other way around.”
🔗 Original source & video: Ep. 011 - GPT 5.5 vs Claude 4.7: OpenAI’s Comeback From the Brink (Tokenomics) | Jordan Nanos, Dylan Patel, Doug O’Laughlin, Max Kan
Ep. 003 - Deep Dive on NVIDIA Vera Rubin VR NVL72 (AI Supply Chain) | Jordan Nanos, Myron Xie, Copper Wei (Wega), Howie
- 🗓️ Date:
2026-02-26| 🎙️ Show:SemiAnalysis
Rubin’s adaptive compression engine makes sparse FP4 potentially usable, delivering up to 50 PF effective performance from 35 PF dense processing, while HBM4 targets 22 TB/s per chip and may bin suppliers by capability. Cableless compute trays and NVIDIA-controlled SoCAMM procurement address GB200’s yield pain and memory tightness, but 2.3 kW GPUs and a likely 1H27 mass deployment leave ramp hiccups and HBM pricing as risks.
View Dialogue Notes & Key Takeaways
Rubin’s headline 50 petaflops of sparse FP4 is the first sparsity number the team thinks will actually be usable, not just “Jensen math.” Howie’s explanation: instead of structured sparsity — “where you force every other data point… into zeros,” which “wasn’t really used at all” at FP4 because models wouldn’t converge — a new adaptive compression engine in the Transformer Engine dynamically compresses the data stream, delivering up to ~50 PF effective while processing at 35 PF dense (vs 15 PF for Blackwell Ultra). NVIDIA’s 5x marketing claim compares Rubin sparse to Blackwell dense.
The HBM4 story is a supplier-differentiation trade: NVIDIA wants 22 TB/s per chip vs 8 TB/s per chip on Blackwell, well above the JEDEC spec, and not every vendor can hit it. Copper’s detail: Micron kept a DRAM process for the base die with relatively poor-quality transistors, SK hynix went to TSMC N12, and Samsung to its own 4nm — with Micron “having trouble hitting the speeds.” Consequence: “a lot of Rubin that ships with lower than the advertised memory bandwidth,” with GPUs effectively binned by HBM supplier.
NVLink 6 doubles scale-up bandwidth without doubling wires via bidirectional SerDes — the marketed “400G SerDes” is really 200G running both directions on the same physical wire. Per-switch bandwidth did not increase, so switch count doubles (18→36 per rack); Howie notes this kept the switch die monolithic, since the prior generation was already at the ~800mm² reticle limit. The backplane hardware is unchanged from Blackwell, and real unidirectional throughput will land below the 400G best case.
The “cableless” compute tray is a manufacturability play that roughly doubles PCB content — the removed cables are Ethernet and PCIe, not NVLink. Wega ties it to GB200’s ramp pain: cables scratched at connectors in tight assemblies caused low yield and debug problems, whereas the new modular Strada/midplane/Orchid design cuts compute-tray assembly “from two hours to five minutes” with automation that only three vendors can do or are working with NVIDIA to do. Trainium did cableless first — with one massive baseboard — and the trade-off is cost: quartz-fiberglass midplane material with worse drilling yield, with de-specification still “under debate.”
Power and thermals jump to 2.3 kW per Rubin GPU (vs 1.4 kW Grace Blackwell Ultra), handled by evolution, not revolution. The microchannel lid wasn’t time-to-market ready; instead a two-piece heat spreader plus stiffener fights warpage, electroplated gold guards against corrosive liquid-metal TIM, and higher coolant flow rate enables chiller-less 45°C-inlet designs — an option, not a mandate, since “not everything is optimized for Rubin.” A liquid-cooled busbar and 50V-direct-to-Strata-board conversion round out the rack changes.
NVIDIA has taken back SoCAMM procurement from customers amid DRAM tightness — a potential pricing-stability advantage over competitors. Wega estimates NVIDIA contracted perhaps half of current Rubin deployment availability for SoCAMM (“Jensen went to Korea for fried chicken… there’s a purpose there”), will pass it through with a margin, making it a double-digit BOM chunk. Jordan’s read: stable input costs let NVIDIA win marginal deals against AMD and Trainium, which may be “more exposed to changes in the DRAM pricing market” — and a high HBM price hike could still be coming.
Timeline: rack systems ship 2H26, mass deployment more likely 1H27, Rubin Ultra (1TB HBM4E, 16 stacks) scheduled 2H27. The team expects a smoother ramp than GB200 — supply chain, rack assemblers, and test-site power are already in place — though “with the speed that NVIDIA is pushing… there could always be hiccups.” The next major system challenge is Kyber, not Rubin.
🔗 Original source & video: Ep. 003 - Deep Dive on NVIDIA Vera Rubin VR NVL72 (AI Supply Chain) | Jordan Nanos, Myron Xie, Copper Wei (Wega), Howie
Claude Code for Finance + The Global Memory Shortage: Doug O’Laughlin, SemiAnalysis
- 🗓️ Date:
2026-02-24| 🎙️ Show:Latent Space
Claude Code with Opus 4.5 now behaves like a junior analyst, compressing PhD-scale projects into days while still requiring expert review of its frequent mistakes. Adoption near 4–5% of public GitHub and a memory shortage that could lift DRAM prices another 100% point to accelerating agent demand, constrained infrastructure, and exposed information-work roles.
View Dialogue Notes & Key Takeaways
Doug O’Laughlin’s core call is that Claude Code crossed from novelty to economically useful information worker around Opus 4.5. It can compress what looked like a “PhD project” into a day or two, yet “this crap makes mistakes all the time”; today it resembles a junior analyst, not an autonomous expert. Shawn compared the paid tool to a perfectly compliant junior analyst. The immediate payoff accrues to experienced reviewers who recognize the “artisanal last 5%,” while junior data-analysis roles look increasingly exposed.
Claude Code’s adoption curve suggests coding agents are becoming the interface for all information work, not merely programming. Doug described Claude-attributed commits reaching roughly 4% of public GitHub in “2 weeks or something”; Shawn said the updated chart was around 5%. Doug’s deliberately sandbagged year-end prediction was 25%, while Shawn said the current rate pointed closer to 50% and that 25% fit a 95% confidence interval. The broader claim is sharper: “Excel is the IDE for analysts,” and Excel, Bloomberg, PowerPoint, and other human-oriented interfaces are vulnerable once agents can retrieve trusted data and render the answer directly.
The most concrete hardware call is a memory shortage that Doug thinks cannot clear for roughly two years. One bit of HBM effectively consumes three to four bits of conventional DRAM capacity after process complexity and yield, just as suppliers emerge from a severe bust that froze clean-room investment. SemiAnalysis sees scope for DRAM prices to rise another 100%, forcing demand destruction, delaying data centers and devices, reviving old DDR4 through CXL, and making “context rationing” a plausible product reality.
Google’s TPU V7 has a temporary cost window, but Nvidia’s control of memory and the wider supply chain may close it with Rubin. Doug thinks nearly every major lab would consume more TPU V7 if supply were unconstrained, and the shared discussion put a possible TPU business near $1 trillion at roughly 30% share. Yet TPU V8’s HBM3 position versus Rubin’s HBM4, Nvidia’s aggressive supplier management, and Google’s limited available capacity make this an installed-base race rather than a clean architectural victory.
Microsoft has “the most to lose” because AI attacks the horizontal software where humans perform information work while Azure finances the attackers. Doug compared OpenAI to “barbarians at the gate”: Microsoft can remain a highly profitable compute supplier while its Office abstractions are disrupted, or redirect investment toward defending Office and building internal models. Oracle’s attempt to absorb the displaced buildout was not necessarily a bad underlying investment, in Doug’s view, but its huge, poorly staged debt issuance overwhelmed market liquidity and turned financing cadence into a bottleneck.
The AI buildout already resembles railroads more than the internet, implying multiple booms and busts rather than one clean cycle. Shawn cited railroad investment at about 4.8% of GDP and 25% of gross fixed capital investment, while also comparing Stargate with roughly 2% of U.S. GDP. Doug will not claim demand rises forever, but Claude Code changed his own elasticity: he would value access at $20,000–$30,000 a year or more because it behaves like many parallel junior analysts.
Doug’s semiconductor framework remains the anchor: Moore’s Law slowing while scaling-law demand accelerated transferred value to companies that could still deliver system-level performance. His 2020 conclusion that Nvidia was the primary beneficiary proved directionally right, though even he did not anticipate “the most valuable company in the world.” The durable investing lesson is not precision EPS maintenance; it is finding the one to three technological variables capable of moving billions in revenue, then using agents to investigate them without outsourcing judgment.
🔗 Original source & video: Claude Code for Finance + The Global Memory Shortage: Doug O’Laughlin, SemiAnalysis
Ep. 002 - InferenceX 2.0 Release (Technical Staff) | Cam Quilici, Bryan Shan, Doug O’Laughlin, Jordan Nanos
- 🗓️ Date:
2026-02-19| 🎙️ Show:SemiAnalysis
InferenceX 2.0 shows GB200/GB300 delivering 20x DeepSeek-R1 throughput per GPU versus a fully tuned H100 at low interactivity, and 80–100x at 100 tokens/sec/user. NVLink’s 72-GPU domain, software optimization, and multi-token prediction drive the gap, while MI355’s roughly 25% TCO advantage over B200 highlights margin potential; composability, legacy fleets, and larger frontier models remain risks.
View Dialogue Notes & Key Takeaways
The headline result of InferenceX 2.0 is that GB200/GB300 delivers 20x the DeepSeek-R1 throughput per GPU of a fully-tuned H100 at low interactivity — and 80–100x at 100 tokens/sec/user, a speed the H100 essentially can’t serve at all. The mechanism is the NVLink domain: DeepSeek’s 256 experts dispersed across “72 GPUs acting as one” rather than over InfiniBand, which Cam Quilici says simply “frame mogs” everything else. Jordan Nanos stresses 100 tok/s is a real production requirement — “fast” tiers like Codex Spark (on Cerebras) already serve above it, versus ~40 for standard models.
Software is compounding as fast as hardware: AMD’s MI355 multi-node DeepSeek recipe roughly tripled from ~1,000 to ~3,000 tokens/sec/GPU at 20 tok/s interactivity in about a month of ROCm optimization. Cam’s core critique of incumbent benchmarks is “benchmark maxing” — one point in time, one point on the Pareto frontier, submissions every three to six months, and vendors able to unsubmit losing results. InferenceX runs nightly, and “once they’re up, they’re not gonna be taken down.”
Multi-token prediction is “basically a free optimization”: turning MTP on took B300 (Dynamo + TensorRT) serving DeepSeek at 68 tok/s/user from 25 cents to 5.7 cents per million tokens — an ~80% cost cut with no accuracy loss on GSM-8K. DeepSeek popularized pre-trained MTP heads and now “basically everyone is using it,” including GLM-5 and Qwen 3.5; it’s definitely in production today.
On TCO rather than raw throughput, MI355 flips ahead of B200: the two are “for all intents and purposes equal” on FP8 SGLang throughput, but MI355’s ~50-cent price difference makes it ~20 cents per million tokens cheaper — roughly a 25% edge that matters “when margins are pretty thin.” The caveat is AMD’s composability problem: FP8 single-node matches B200, FP4 is significantly worse, and stacking disaggregation plus MTP degrades further — flags that work alone don’t work in combination.
Doug O’Laughlin’s econ-brain takeaway is bullish inference margins: current benchmarks use random single-turn data with zero prefix caching, so “this is the worst it would be — it would only get better from here.” With MTP, multi-turn caching and the rest stacking toward ~50% price decreases, he now thinks Max “might not be as unprofitable as you think” versus SemiAnalysis’s first tokenomics estimates. Jordan’s offsets: frontier models are likely bigger than DeepSeek, and these gains apply to the newest GPUs, not the large legacy fleets still serving inference.
Roadmap: Trainium and TPU coverage is a V3/V4 goal, plus planned day-zero support for DeepSeek V4; the team is currently trying to provide day-zero support for Qwen 3.5, Minimax and Kimi 2.5. They also plan EPD (encoder-prefill-decode) disaggregation for multimodal agentic workloads and a real-world multi-turn prefix-cache benchmark. The blocker is that agentic/multimodal datasets essentially don’t exist publicly — the best available is WildChat, “three years old… people asking GPT to be their therapist” — so the team plans to generate its own traces.
The coda is an honest hedge on the whole AI trade: Cam admits a “fleeting thought — what if they just stop getting better? Then what? Then we’re fucked, man.” His resolution is the thesis InferenceX itself documents: even if “models aren’t getting smarter,” they are getting “cheaper, faster, and more performant,” so diffusion widens regardless. Cam’s experience with Codex 5.3 (“better for targeted issues,” Opus 4.6 “for general vibes”), which Doug agreed with, is offered as proof they update on evidence.
🔗 Original source & video: Ep. 002 - InferenceX 2.0 Release (Technical Staff) | Cam Quilici, Bryan Shan, Doug O’Laughlin, Jordan Nanos
Ep. 001 - Claude Code, Memory Mania, CPUs are Back | Jordan Nanos, Doug O’Laughlin, Myron Xie
- 🗓️ Date:
2026-02-13| 🎙️ Show:SemiAnalysis
Claude Code now authors roughly 4.7%–4.8% of public GitHub commits, while Opus Fast and GPT-5.3-Codex-Spark test how much users will pay for faster inference above the $2,400/year Max tier. Memory has shifted from the worst downturn since 1996 to a multi-year shortage as HBM absorbs wafer capacity, CPUs tighten, and even new supply may not close the gap; storage demand hinges on logs, synthetic data, and generated video.
View Dialogue Notes & Key Takeaways
The memory cycle has whipsawed from the worst downturn since 1996 — which “bankrupted a third of the industry” that year — to a historic shortage with little near-term relief. Myron Xie’s mechanism: HBM yields several times fewer bits per wafer than conventional DRAM, every incremental GPU/XPU pulls wafers toward HBM, and post-COVID there were virtually no additional wafer starts or cleanroom space — “we’re adding virtually no additional supply while demand is booming.” Doug O’Laughlin: memory and NVMe prices rose ~20% in a single day and ~100% month-on-month, and it stays tight “for a few years at least.”
The classic cycle-top signal — Samsung capitulating and adding capacity last — may not work this time. Doug: “I think they’re gonna build capacity, and it’s still not gonna be enough. The supply-demand gap is the largest it’s ever been, and it gets worse from the work that we see.” His “galaxy brain take”: the EUV bottleneck could pull 3D DRAM forward from the expected ~2035 timeframe toward ~2030, though not before 2030.
CPUs are back, and even Intel is still — or probably will be — sold out, “which tells you things are tight.” The discussion frames Intel as the marginal supplier, while Arm is “such a bigger threat than it’s ever been” and AMD “continues to crush.” Phoenix CPUs are an interesting change, particularly for how Arm’s business model may adapt.
Claude Code now authors ~4.7–4.8% of public GitHub commits, per Doug’s scraped daily series — and Anthropic linked the article containing the estimate in the press release for its $30B raise at a $380B valuation. The share rose from ~2% on January 20 to ~4% on February 5 and is still growing; Doug built the tracker after a tweet tipped him that Claude Code signs its own commits.
Opus Fast and OpenAI’s GPT-5.3-Codex-Spark (on Cerebras) are price-elasticity experiments above the $2,400/year Max tier — the “caviar trade.” Opus Fast charges ~6x per token for roughly 2–2.5x speed; Jordan Nanos describes the Cerebras trade as “10x the money for 4x the performance, or really more like 20x the money for 10x the performance or something like that.” Jordan says the Fast mode is working on him: “Now that I’m addicted to Fast tokens… I don’t think I could go back.”
China Model Week delivered real breakthroughs: MiniMax 2.5 “at Opus performance… 1/10 the price,” GLM-5, and above all ByteDance’s Seedance/CDance 2 video model — “animation done, cooked.” Doug calls it the first “total victory” over Google’s Veo/Genie lead, and floats a conspiracy: “the surveillance state gives everyone better training data” — 10% of all hard-drive demand is Chinese surveillance.
Jordan’s storage thesis: SSD/HDD demand will boom on logs, synthetic data, and generated video — not KV-cache offload, whose retained fraction can be “less than 1%” of generated video. He also warns against attach-rate shortcuts like Jensen’s 16TB-per-GPU figure, which makes ICMS look like <0.7% of drive shipments — “the right conclusion for ICMS, but not the right conclusion for the growth of drive demand.”
🔗 Original source & video: Ep. 001 - Claude Code, Memory Mania, CPUs are Back | Jordan Nanos, Doug O’Laughlin, Myron Xie