Google DeepMind's Ed Chi: AI's Endgame Is Capability Over Cost

Summary

• Ed Chi’s view is that model capabilities may gradually commoditize, but not all capabilities will. Google is a full-stack company spanning models, applications, hyperscaling, token factories, chips and energy. He says Google can deliver tokens at the lowest cost, which “could become the most important moat going forward.” “All economic questions ultimately come down to capability over cost. There’s no question about it.”
• As gaps in model capabilities narrow, value creation may shift toward applications and token factories—a directional call for founders. On the application side, the moat is user lock-in, stickiness and experience. On the hyperscaler side, the questions are about cost: energy consumption, chip design, the relationship between memory and computation, and how much runs on-device. “These are all open questions, and they’re very exciting.”
• General-purpose assistants are still at a very early stage, and he expects the interface between them and AI Mode Search to become clearer in 6 months to 1 year. Products such as Instinct and Muse Spark aim to move beyond Q&A into getting things done—booking flights, buying things—but run up against permissions and reliability. People worry that if they hand a general-purpose assistant their credit card, it will go on a spending spree; he says there is still no real new data on the confirmation layer. His “Grandma Test” is that the era of general-purpose assistants will have truly arrived when Grandma points at a robot and says, “Why are you so stupid?”
• One reason open models caught up with closed models is a Chain of Thought paper he co-authored: open models use CoT to surface and replicate reasoning methods, while major labs are trying to hide chain of thought to make copying harder. He says it is too early to call this settled: distillation has made some capabilities more commoditized, but there is no sign yet that capabilities such as multimodality will commoditize.
• DeepMind’s recent small models are mainly for applications, while larger models remain in development; Google abandoning model development is “completely impossible.” Users expect Google to have its own technology, with something of a technological-sovereignty dimension to it. Resource allocation between small and large models “is a difficult problem even inside large companies.” He describes Bay Area “AGI-pilled” tech bros as believers that software took over the world and that good coding can take over the world—which explains their love of ultra-size models. He personally does not want applications to depend entirely on ultra-size models, given the energy required to run them.
• Memory is a bottleneck that researchers have yet to fully understand: Transformer context buffers can currently reach around 1 million or 2 million tokens, which he considers nowhere near enough. His core framework is that memory is not just storage and retrieval; it is also a ranking problem—when you think of Paris, which memories come to mind is a matter of ranking, rather than every related memory flooding in at once. From a cognitive perspective, memory is also a program for programming AGI machinery. Asked about possible shortages of DRAM, NAND and HDD, he says memory and computation can substitute for each other; companies with long-term investment plans are less likely to worry about a short-term memory shortage.
• The bubble is localized, not market-wide: expectations for physical intelligence may be unrealistic, and many models’ visual reasoning capability is “terrible.” If you believe that agents solving software means solving everything, much of what he has said does not apply; if you do not, “there’s still plenty of room to play.” In 3–4 years, reasoning has gone from elementary-school math problems—models had just 33% accuracy then—to solving Navier–Stokes equations. For the next exciting monetization opportunity, he is bullish on consumer: consumer services have always been a Google strength, and that is why he stayed at Google.

Deep dive

1. Reasoning has gone from elementary-school math to Navier–Stokes in 3–4 years, but remains concentrated in specific domains

• In his panel recap, Ed Chi said that over the past 30 years, particularly the first 20, the focus was largely on recommendation and search. Over the past decade, neural networks’ next-token prediction and Chain of Thought have helped uncover machines’ basic cognitive and reasoning capabilities. The next academic questions include more complex reasoning, memory and RSI (recursive self-improvement).
• His timeline is striking: 3–4 years ago, when he wrote the CoT paper, the task was elementary-school math—“You buy 5 apples, then buy 6 more”—and accuracy was just 33%. This year, models are solving difficult physics or math problems such as Navier–Stokes equations. But reasoning remains concentrated in specific domains. “I’m a golf enthusiast. How much can it reason about clubs, balls and equipment? That’s not the same as math.” Applications need to start by understanding the model’s reasoning level across different domains.

2. General-purpose assistants: the Grandma Test and a 6-to-12-month fight over the interface

• He sees the Instinct and Muse Spark wave this way: Google’s AI Mode Search has evolved from search into search or Q&A, while these products aim to move from Q&A to general-purpose assistants that book flights, reserve hotels and buy things. The sticking points are permissions and reliability: people worry that if they hand an assistant their credit card, it will buy things all over the place. As for confirmation—whether it bought the right thing or the wrong thing—“we haven’t really seen new data yet.” He expects the relationship and interface between general-purpose assistants and AI Mode Search to become clearer over the next 6 months to 1 year.
• The moat is tied to the user experience and how much error users will tolerate: one serious mistake can change a user’s view of the product. It is similar to the tolerance people have for mistakes by a human personal assistant. His Grandma Test: “One day, when your grandma points at the robot and says, ‘Why are you so stupid?’, you’ll know the era of general-purpose assistants has really arrived.” Grandma’s expectations for a general-purpose assistant are high; both the product and the science are still exploring that boundary.

3. The ecosystem is wide open at both ends: no settled answer on consumer hardware or the user entry point

• The consumer ecosystem has a hardware side and a software side. On hardware, the big question is how on-device and off-device models interact. Apple has talked about hybrid intelligence, but the future of consumer hardware remains a wide-open question in his view. On software, the conversation is now about services, not downloadable apps, and the entry point is unsettled: search, a phone, smart glasses or another device? In the era of general-purpose assistants, people should not still have to download apps onto a robot all day—that was a concern 15 years ago.
• Asked whether a breakout product will come from Google, Meta, Apple or a startup, he said “where it comes from may not be that important.” What matters is how it reaches mass adoption and what the right human-computer interface is. A startup’s product could also be acquired by a large company; investors and developers want a healthy ecosystem. He noted that after 4 or 5 years, it suddenly seemed like nobody was making smart speakers anymore. But for a personal general-purpose assistant, putting a screen on a refrigerator and connecting it to the Internet is much easier now than it was 7 or 8 years ago. Consumer hardware remains exciting.

4. DeepMind’s small-model cadence, AGI-pilled believers and capability over cost

• Asked about personnel changes and the release cadence, he stressed that Google has not stopped and is still developing models. Most releases in recent months have been small models because applications such as general-purpose assistants and personalization need them. Larger models are also in development: “Stay tuned.” Allocating resources between small and large models “is a difficult problem even inside large companies,” and people will disagree. His description of Bay Area “AGI-pilled” tech bros: they believe software took over the world, and that good coding and good software can take over the world—“that’s why they love ultra-size models.” He personally does not want future applications to depend entirely on ultra-size models, because they require too much energy to run.
• Google abandoning model development is out of the question: users expect it to have its own technology, rather than use someone else’s or an open-weights model. That may amount to technological sovereignty. Google’s advantage is its full stack, from model development to applications and services, then down to hyperscaling, token factories, chip development and energy use and control. He says Google can deliver tokens at the lowest cost, which could become the most important moat going forward. If model capabilities commoditize, cost control becomes critical: “All economic questions ultimately come down to capability over cost.”
• Asked about concerns over CapEx, cash flow and AI application revenue, he distinguished short-term from long-term thinking. In the short term, companies focus on quality and accuracy; companies investing with a 5- or 10-year horizon also need to think about what metrics to optimize next—not just capability, but capability over cost. Google’s approach is to invest where it can and double down across the board; models, applications, chips and the full stack are all being innovated.

5. Memory is an ill-understood bottleneck: it is both ranking and program

• Asked about potential shortages of DRAM, NAND and HDD, Ed’s framework is that memory and computation can substitute for each other. “With a very long-term investment plan, you worry less about a short-term memory shortage, because you can use computation instead.” Research questions include how much memory and bandwidth different computing methods require, and whether neural network designs should reduce memory requirements. He cited the Delta Attention Model, which uses memory differently from full attention (O(N²)) and sparse attention. There are also questions around personalized memory and KV cache.
• He sees 2 bottlenecks. First, Transformer context buffers can currently reach around 1 million or 2 million tokens, which he considers nowhere near enough. Second is the cognitive question: people still treat memory as a retrieval mechanism for storage and recall, but “from the perspective of human cognition, memory is also a program… a program for programming AGI machinery.” Researchers have not yet figured memory out.
• The host noted that the human brain can actively forget, learn new things and compress old information. Ed added that many people trained in computer science assume machines do not need to forget. But memory is also a ranking problem: “For example, when you think of Paris, you might recall memories from a trip there. That is a ranking problem, not every single thing related to Paris flooding in at once.” People in hardware are also exploring new ways to combine memory and computation; there could be developments over the next few years.

6. Value is shifting toward both ends: the limits of open-model distillation, localized bubbles and a consumer resurgence

• If he were starting a company today, what direction would he choose? He sees some signs of model capabilities beginning to commoditize, though not completely. As the gap between the best and worst models narrows, value creation may shift toward applications and token factories. Applications compete on user lock-in, stickiness and experience; the hyperscaler side is about energy, chip design, the relationship between memory and computation, and how much will eventually run on-device.
• One reason the gap between open and closed models has narrowed is that open models used methods from the CoT paper he co-authored to uncover chains of thought in models and replicate their reasoning. Large labs are now doing a lot of work to hide reasoning chains and make them harder to copy. He cautioned against treating this as settled: distillation has made some capabilities more commoditized, but there are capabilities that show no sign of commoditizing. He cited multimodality as one example.
• He rejected a blanket judgment on bubbles: he does not think the entire market is a bubble, but there may be bubbles in specific areas. Expectations in physical intelligence may be unrealistic, and many models’ visual reasoning capability is “terrible.” The disagreement depends on what you believe. If you think agents solving software means solving everything, much of what he has said does not apply; if not, “there’s still plenty of room to play.”
• He is bullish on a consumer resurgence, noting that much of Anthropic and OpenAI’s revenue is driven by enterprise subscription revenue. Consumer services have always been one of Google’s strengths, and that is why he stayed at Google.
• His closing AlphaGo analogy was an answer to whether children should still study math. After AlphaGo played a chess champion 10 years ago, many people said humans would stop playing chess; instead, more people took it up. Math is similar. Some people used to pursue it to explore the boundaries of knowledge; in the future, people may also do it for understanding, entertainment or enjoyment, and more math research may be driven by curiosity. Likewise, after large language models emerged, some thought search engines would no longer be as relevant, but Google has seen search applications increase. He does not deny that AI will change the existing structure, but believes it will also create new possibilities.