Summary

Neil Movva is building Sola Research as a “token factory”—an inference company optimized for one thing: the lowest cost per token in the industry, “by a mile,” for long-running background agents rather than chatbots. His core forecast: workloads end this year “maybe 50/50 background and real-time,” but go “to 90/10 in favor of background,” because background token demand is unbounded while human-in-the-loop demand is capped by attention span. “Whenever you make something ten times cheaper, it’s a new product category.”
The load-bearing technical claim is that the latency-throughput tradeoff in GPU systems is foundational—and everyone chose latency because Cursor pulled the market that way. GPUs are throughput machines happiest with big batches (his analogy: a bus versus private transit); NVLink and tensor parallelism buy minimum latency at sublinear scaling (8× hardware for maybe 4–5× speed). If you stop caring about latency, chips that beat NVIDIA on FLOPS-per-dollar but lack interconnect suddenly have a home—the arbitrage his stack exploits.
The supply-side strategy is deliberate scavenging: “there’s no bad chips, there’s only bad pricing,” extended to power and data centers no one else will touch. He’ll buy “any chip anywhere in the world for any duration of time,” deploy in one-megawatt data centers (eight refrigerator-sized liquid-cooled racks) with single power feeds and no redundancy—“I will buy 95% uptime”—and could potentially tolerate even 80% uptime at the right price, including intermittent solar/wind power, because failed requests migrate and background agents can absorb added latency. “Mini-mills, not large monolithic steel plants.”
His answer to the bubble worry (semis at ~20% of the S&P 500 versus a historical 2–4%): unlike dot-com networking or the 2023–24 Hopper crunch, today’s spend isn’t speculative. Training was inherently speculative; inference is bought because it’s immediately valuable—“You don’t hoard tokens, you use them immediately… There’s no speculation on inference”—and he expects inference spend to increase monotonically.
Two contrarian hardware takes: NVIDIA’s like-for-like bfloat16 performance-per-watt “hasn’t improved all that much” from Hopper to Blackwell to Rubin, and losing TSMC “wouldn’t be that bad”—Intel is at worst maybe 2× behind on perf/watt, far smaller than the chip-war dialogue implies. The bottleneck to attack is HBM, because “the boys in Boise don’t love huge CapEx for cyclical”—he thinks iPhones may cut memory and rise in price rather than new memory fabs arrive quickly.
On frontier labs versus open source: “the labs pay an immense premium to be three to six months ahead,” and he doubts that premium lasts—enterprises are still on Opus 4.6/4.7, and “latent distillation” through AI-generated GitHub repos makes capability diffusion “fundamentally impossible” to prevent. “It gets easier every day to train a frontier-class model.”
The biggest inefficiencies he’d stack-rank: KV-cache memory (“not earning its keep,” off by an order of magnitude or two) and idle compute. NVIDIA is producing five million Blackwells this year, but he doubts they are all used continuously and says idle utilization across the rest of the world is worse than the xAI clusters people mock. The end-state ambition is a trillion tokens per day, which at OpenAI pricing for 5.5 or 5.6 is at least ~$5 million—meaning he’s asking for three to six orders of magnitude of cost improvement.

Deep dive

1. Sola Research is a token factory betting on abundance

Movva’s literal description: an API serving open-source LLMs “at a price that is unbeatable in the market,” plus “Sola Boxes”—long-running agent VMs “designed for agents that run for hours, days, or weeks.” The theme is abundance: “we think it’s so profound that the machine can think, and now our job is to make as many machines as possible in the world work toward thinking.”
Tokens are today’s North Star but not the final unit—the direction is outcomes, with agents that “self-administer a token budget” rather than companies capping engineer spend, and token count becoming a dependent variable of the task.
Two tailwinds he cites: the rise of open source (customers “care about owning intelligence… you always have the weights”), and the incumbents’ positioning—Baseten, Fireworks, Together AI all optimized low-latency inference because of “one very important customer, Cursor.” Right choice a year ago; “as of six months ago, it started to look like maybe low latency wasn’t the only thing you wanted.”

2. The core bet: background agents eat real-time workloads

Patrick’s challenge—“I want everything as fast as possible”—gets Movva’s signature reframe: “the best latency is no latency at all. When you wake up in the morning, the work’s already been done overnight. You didn’t even have to ask for it.” Humans in the loop are the bottleneck; agents should run on human management timescales—you check on colleagues weekly, not every five minutes.
The evidence: test-time compute scaling became bettable “late last year with Claude Opus 4.5”—“the first agent that was at all suitable for longer-horizon tasks. It was pretty mediocre when it first came out”—and now hour-long runs are “quite suitable.” A few points on that curve and “you draw out the exponential.”
The sizing logic: “I love this market because it’s unbounded.” He personally can’t consume 10× more tokens in Codex or Claude Code—he’s already saturated. Background consumption has no such ceiling: 50/50 by year-end, 90/10 background long term.

3. What abundant background inference unlocks: 10,000-source research, security as proof of work

Best current examples: deep research over “not 100 sources, not 1,000 sources, but 10,000 sources or more”—customer Parallel Web Systems wants to index and monitor the entire internet in real time, “the kind of crazy exabyte-scale task” needing a different scale of intelligence.
Cybersecurity as told: when Fable or Mythos first came out, the community pushed to run them against every line of code ever written, in 20 different ways, until “security has become proof of work”—how secure your software is equals “how many dollars you spent on Anthropic’s APIs trying to break into your software?” And the frontier is jagged: Haiku finds bugs Fable misses and vice versa, encouraging diverse sampling.
The dreamy version: proactive Siri with “a much more encyclopedic view of your life,” surfacing your next action when you open your phone. He says they are not quite there yet. The unlock is that “you have to be willing to spend tokens without any promise of return.” Verifiable problems—software, formal math, and potentially scientific discovery—are dropping from millions to thousands of dollars, maybe tens. His caveat on non-verifiable tasks: “we have not solved human taste yet, and I don’t know that it fundamentally can.”

4. NVIDIA formation: tensor cores earned their die area, and “speed of light” is the religion

Movva’s first job was NVIDIA in the mid-2010s, when managers’ lab notebooks recorded grad students at ICML/NeurIPS training on gaming GPUs—“we should double-click on this.” Tensor cores (matrix-multiply accelerators) had to fight the graphics teams for “maybe like five, ten percent” of silicon area, initially for computer-vision convolutions.
The cultural inheritance: “they always chase the speed of light for any piece of hardware”—the edge of what’s possible. His version today: “we’re chasing 100% speed of light. I don’t care about relative numbers versus the competition. I only care about absolute numbers.”
The texture detail worth keeping: NVIDIA’s frugality ran to no free milk—engineers paid a dollar a month to a “milk club” stocking Costco milk. “We don’t do that at Sola.”

5. The latency-throughput tradeoff is foundational—and NVLink is a latency technology

The mechanism: GPUs want large batches of many users’ work, but every request in a big batch “spends a longer time on the GPU being carried with other people’s traffic.” His analogy: the bus versus private transit downtown—the bus serves everyone and stops constantly; the car takes the direct path. “Narrow and fast or wide and slow is a classic trade-off in all computer science.”
NVLink lets you shard a matrix multiply across up to eight GPUs to cut latency—but scaling is sublinear: “you’ll use eight times more hardware… you might get like four to five X the speed.” Patrick’s clarifying question lands the strategic point: NVLink is primarily a latency technology.
The consequence: since Sola doesn’t care about low latency, a chip that multiplies matrices well but can’t communicate across peers “may be a really, really good compute per dollar option.” Movva optimizes FLOPS per dollar of ownership, uses expert or pipeline parallelism instead of tensor parallelism, and hides communication latency in ways a low-latency service can’t.

6. Cerebras and Groq explained from physics up—and why KV cache is their thorn

The SRAM/DRAM tour: SRAM (6T cells on the logic die) is blisteringly fast but area-hungry—an 800mm² Blackwell-sized die made entirely of SRAM would hold single-digit gigabytes; NVIDIA’s logic die carries maybe 500MB. DRAM (Micron, SK Hynix, Samsung) writes charge to capacitors that must be refreshed every ~50ms—“constantly juggling billions of balls in the air”—but stacked as HBM gives Blackwell 288GB. Cerebras takes the whole wafer, ~50GB of SRAM at 21 petabytes per second versus ~10TB/s for HBM, and gets “a path to 1,000 tokens per second.”
The thorn: KV cache—the stored representation of every token in the conversation, “frequently larger than the weights themselves”—grows dynamically with usage and is difficult to handle on the SRAM-focused architecture. “The KV cache is gonna be a big thorn in your side.”
His prediction: hybridization. “The original sin of transformers is that you’ve taken this fundamentally memory-bound layer [attention] and juxtaposed it right next to a compute-bound layer [MLP].” Put the weights/MLP on Cerebras-style SRAM and attention on GPUs with capacity—“I believe this is what’s happening with NVIDIA and Groq.” Think of wafer-scale chips as accelerators paired with GPUs, not replacements.

7. Transformers are sponges; the next data block is RL gyms

Why transformers won: they scaled—“transformers are the link to go from millions to trillions of parameters”—and attention “represents any pairwise relationship that you want,” so “if there’s any relationship in the sequence at all, you’re gonna find it.” The Karpathy lesson he carries: first overfit to prove the relationship exists; “once you can overfit, then you can compress, and the compression is how you get generalization.”
On data: “the internet was a one-time subsidy”—roughly 30 trillion high-quality tokens, 300 trillion on a wider view, “and we’ve basically looked at it all already.” Even random ChatGPT interaction data no longer provides much value: “the signal you get from random human preference, unconditioned human preference, is not actually worth anything anymore.” He says expert human preference is what matters now.
The successor: RL environment gyms with verifiable tasks and self-grading—“the environment becomes the data.” His stronger claim: AGI arrives by “stacking specialized intelligences until you have no more gaps to fill,” provided every task is verifiable—and labs’ spending has shifted from data to RL environments.

8. Kernels are written on the whiteboard now—and software edge erodes

Kernels aren’t done: any model deviation, such as a new RoPE variant, breaks the FlashAttention happy path. But the craft has changed: “we write kernels on the whiteboard”—humans do conceptual design, describe the dispatch strategy in natural language, and models implement. Crediting Tri Dao: “you shouldn’t write kernels by hand anymore necessarily.”
His honest concession under Patrick’s probing: kernel skill isn’t a moat. “The rising tide of something like Mythos or GPT-5.6 Sol, that lifts all boats… we’re all downstream of the frontier in terms of this capability.” Patrick’s coal analogy gets a nuanced answer: big-batch matmuls already run at 70–80% of peak—limited by power throttling, not software; NVIDIA’s quoted peak FLOPS are “a little optimistic.” The real frontier is programming the whole NVL72 rack—GB300 ships as a rack of 72 units—“an open race to figure out who can program the whole rack-scale computer.”

9. “There’s no bad chips, only bad pricing”—the scavenger chip book

On Blackwell scarcity: NVIDIA could raise prices to clear the market, but Movva says it considers the long-term risk of letting the deepest pockets accrue too much power; “they understand that compute is power today.” Access runs on relationships or “incredible financial backing.”
On AMD: “great chips overall” whose problem is that people can’t program them, while other vendors do less out-of-the-box kernel work than NVIDIA. “Other people have this perception that AMD is not as good as NVIDIA. That’s music to my ears. I’m very happy for them to sleep on this chip”—though he concedes it’s “not actually super true anymore,” with Meta and OpenAI buying heavily. He also watches Etched, SambaNova, and d-Matrix; their constraint is TSMC wafer allocation.
The posture: “we will buy any chip anywhere in the world for any duration of time”—TPUs, Trainium, whatever—“every chip has a comparative advantage. We have to find that advantage and then squeeze it in that direction.”

10. The bubble question: “I’m less a student of history, more a member of history”

Patrick’s setup is precise: semiconductors went from 2–4% of the S&P 500 historically to 19–21%, and investors in Micron and SK Hynix fear mean reversion because “long term, compute’s a commodity.”
Movva’s answer runs through biography—born in 1997, mother at Intel through the dot-com run-up, remembers Cisco as the world’s most valuable company. The difference: dot-com networking spend anticipated demand that never came, and even the 2023–24 Hopper crunch was training-oriented, “inherently speculative.” Today “people buy tokens because they’re immediately valuable… You don’t hoard tokens, you use them immediately”—companies are instituting spend caps on Claude Code. “I do think inference spend monotonically increases. There’s no speculation on inference.”

11. One-megawatt data centers, 95% uptime, and solar nobody else will take

The market structure: data centers were built for training, a superset workload, and there’s a diseconomy of scale—gigawatt sites are basically impossible in the US, 100MW very hard, 10MW on the edge, and “one megawatt I would argue is plentiful.” Aggregate power exists everywhere; concentrated power doesn’t. Liquid cooling makes 1MW roughly eight refrigerator-sized racks.
The radical part: strip redundancy—no diesel backup, one trenched fiber line, no failover SLAs. “I won’t be surprised if some of them get down to like ninety-five percent uptime… You’d have basically zero buyers for a data center that has ninety-five percent uptime. I’m that first buyer.” Failures migrate via a robust control plane; a background agent’s turn takes an extra minute or several, even ten, and “my customers don’t care because their agent was running for hours.” The explicit deal: competitive average throughput, uncontrolled P99, unbeatable economics.
Power follows the same logic: “could I even take 80% uptime at the right price? Probably.” Solar and wind’s intermittency—the worst-case, days- or weeks-long outage—is “in fact highly predictable”; model the weather and move the workload. Patrick’s label sticks: a scavenger strategy. “I do not wanna be bidding against Anthropic or OpenAI for compute… We’re trying to build the best steel factory in the world, but it will come through mini mills, not large monolithic steel plants.”
On vertical integration, the founder/CEO split as told: “the founder in me wants to do everything… I will never stop until I have built the most efficient system from soup to nuts” (Patrick: “real-life Factorio”); the CEO starts with software’s leverage, prefers power-purchase agreements to owning generation, and earns the right to pull layers in-house.

12. Stack-ranking the waste: KV cache and idle silicon

His inefficiency ledger: compute scaling is already efficient—frontier MoEs activate closer to 1% of experts than 10%. Attention’s memory use is not: “the KV cache is quite uncompressed… it’s not earning its keep,” with kilobytes per token “probably off by an order of magnitude or two.” DeepSeek has published work compressing it further and making substantial progress.
The bigger scandal is orchestration. NVIDIA is pumping out five million Blackwells this year—“are they all being used all the time? I certainly doubt it”—but he says a ton of GPUs sit idle in warehouses or private pools. “We all make fun of xAI for having challenges with total FLOP utilization on its clusters. The reality for the rest of the world is it’s far worse.”
On fabs: don’t expect a snap-your-fingers fix—“you make 20% more chips, then you have another bottleneck immediately.” His creative angle: TSMC spends heavily tightening process corners so the worst chip matches the best; he’d happily take more rejects if the tradeoff were exposed, expanding US power supply so marginal chips “earn their place in a data center.” His hiring corollary: he screens for “100% curiosity” and love of performance engineering—CUDA experience is “a huge red herring.”

13. Open vs. closed, the distillation heresy, and why Jensen won’t sell tokens

The frame: “the labs pay an immense premium to be three to six months ahead… that’s probably still worth it”—but the premium’s durability is suspect, since enterprises are still running Opus 4.6 or 4.7. On distillation-as-theft, his alternative is that AI-generated artifacts saturate the commons—“what percentage of repos created in the last year do we think were created by Claude Code?… We’re gonna have latent distillation for a long time. I don’t think it’s fundamentally possible to prevent the diffusion of information or model capabilities.” And open source can’t die: “if one leader steps out, a new leader will step in… It gets easier every day to train a frontier-class model.”
The contrarian pair: like-for-like bfloat16 perf/watt barely moves Hopper→Blackwell→Rubin, or across TSMC 5/4/3/2nm—so losing TSMC access “wouldn’t be that bad. Supply would take a shock for sure,” but Intel is “at worst maybe 2x worse performance per watt.” For hardware founders: master the four bottlenecks—TSMC wafers, HBM, advanced packaging, power—and “be spiky”; HBM is the one to attack, because “the boys in Boise don’t love huge CapEx for cyclical. They’ve been burned many times.” Consequence: iPhones may cut memory and go up in price.
Why NVIDIA stops short of tokens: “Jensen is really good at making his friends billionaires”—a diverse ecosystem of hungry neoclouds means a provider that defects to AMD or vertical integration can be replaced by others. And the scale of the demanded future: a trillion tokens per day is at least ~$5 million at OpenAI pricing for 5.5 or 5.6—three to six orders of magnitude of cost reduction required, though “for some size of model, we are approaching a trillion tokens being measured in tens of thousands of dollars.” Asked about demand for such abundant intelligence, Movva says, “I never will believe in that. There is always demand for intelligence in the world.”

How to win a stock pitch competition | lessons from an Ira Sohn winner SOEs Buy In at Peak WRC: Bubble Squeezed, Embodied AI Goes Real-World