The Model Eats the Scaffolding: DeepMind's Logan Kilpatrick & Tulsee Doshi on 3.5 Flash, Omni & More

Summary

• Google is leading with Gemini 3.5 Flash because it is emphasizing cost-adjusted performance alongside frontier quality. Tulsee Doshi says slower responses hurt live Search and Gemini experiments “even if the model is hugely better,” while Flash is roughly three times faster than other large models and significantly cheaper. She cited “I think 280 tokens per second” on Artificial Analysis—a speed that can make an agent finish before she cancels it.
• The missing Ultra brand does not mean Google stopped scaling frontier capability. Logan Kilpatrick calls model naming “marketing”: Pro has grown larger and more powerful, while Deep Think adds test-time scaling; internally, Pro influences Flash, which influences Flash-Lite through distillation, and recipes can also scale upward. Google’s portfolio choice reflects its need to serve multiple products and billions of users, where Flash and Flash-Lite economics matter alongside demand for high-end quality.
• The strategic center of Google’s AI stack is shifting from standalone models to “model-harness-product-symbiosis.” Antigravity is becoming shared agent infrastructure for Gemini Spark, AI Studio, the Agents API, and eventually more Google products—the new cross-product through-line after Gemini itself. Logan’s shorthand for the direction is “agents, agents, agents,” with the harness handling tool loops and orchestration that individual teams previously had to build.
• A co-trained harness could deepen switching costs, but Google says it is explicitly preserving “harness diversity.” Labenz’s investor-relevant challenge was that optimized model-harness stacks might create silos, stickiness, and pricing power. The guests want both outcomes: a seamless full-stack Gemini experience that generates better training and evaluation data, and models that still generalize to enterprise customers’ own orchestration systems.
• The durable operational advantage may be infrastructure standardization and the feedback loop across Google’s product estate. Logan says the AI stack now needs rewriting “every 12 to 18 months,” making common infrastructure essential; Tulsee says products expose rough edges that flow back into model evals and data. “The model eats the scaffolding” captures the cycle: capabilities once implemented in surrounding code progressively move into the model or shared harness.
• Recursive self-improvement is already part of Gemini development, but Google’s near-term framing remains human-directed collaboration. Gemini can submit code changes, launch evaluations, propose research improvements, and support parallel ablations; one safety-and-alignment lead reportedly ran such experiments from her phone in a hot tub and produced a report within an hour. Logan says the near-term picture of an autonomous “ML intern” launching enormously expensive pre-training runs does not seem realistic: the opportunity cost of a wrong direction keeps “the human in the driver’s seat.”
• Google describes Gemini as a collaborator while treating apparent psychological distress as a model or product defect and questioning whether welfare interviews reveal an inner state. The team evaluates sycophancy, role-play, looping, and rabbit-holing checkpoint by checkpoint; Tulsee calls models going off the rails a “model bug.” She is skeptical of welfare interviews that ask models about deployments they cannot observe, arguing that without such context they are largely “pontificating.”
• Rather than maximizing raw context length, Google is betting on search grounding, compaction, and speed. A 1 million-token request can cost “a few dollars,” limiting demand, while much of a giant context is distracting; the preferred direction is selecting the right information at the right time. Diffusion coding remains active research, but 3.5 Flash already delivers enough speed that Google is testing where further acceleration actually has diminishing returns.

Deep dive

1. Flash wins the launch slot by optimizing intelligence per dollar and second

• Labenz framed the conversation with the reversal of the 2023 “No Moats” anxiety: over the subsequent three years, Google added roughly $3.5 trillion in market capitalization. In the episode introduction, he also said Google grew annual revenue by $50 billion from 2024 to 2025 and still controlled 25% of global compute, context for his claim that its moats were holding up.

• Against Google’s approximately $400 billion in annual revenue, he challenged the company’s decision to launch Gemini 3.5 Flash before an Ultra-class model.

• Tulsee’s answer starts with consumer behavior: Flash occupies the “sweet spot of being really smart while also being really fast and really cost-effective.” In Search and the Gemini app, latency regressions visibly damage live experiments, “even if the model is hugely better from a quality perspective,” because “users aren’t willing to wait.”

• The stated performance proposition is concrete: Flash is about three times faster than other large models, significantly cheaper, and designed for coding and agentic workflows. Tulsee said 3.5 Flash benchmarks at “I think 280 tokens per second” on Artificial Analysis.

• Google’s lineup therefore spans Pro, Flash, and Flash-Lite, with Deep Think providing a runtime-scaling dimension. Flash-Lite was added because serving Google-scale consumer populations created demand for an even faster, cheaper tier that was not necessarily part of the original 2.0 series.

2. “Ultra” is partly a brand question, not a ceiling on scaling

• Labenz’s pushback—worth keeping—is that some customers appear relatively price-insensitive when buying the single best model, making an Ultra offering look commercially obvious. He also asked whether a hidden mega-model trains Pro, which then distills into Flash.

• Logan’s answer: “At the end of the day, the naming of these things is marketing.” Pro models have continued scaling in size, power, and test-time compute; Google repeatedly discusses whether a scaled-up release should carry the Ultra name but has favored series-to-series consistency.

• Tulsee confirmed bidirectional recipe transfer rather than a simple hierarchy: “Pro influences Flash influences Flash-Lite,” while Flash recipes can also scale upward into Pro. Progress is being pursued across pre-training, post-training, inference techniques, sub-agent systems, and tighter integration with the agent harness.

3. Antigravity turns agents into Google’s next shared product layer

• Logan’s headline for I/O is “agents, agents, agents,” enabled by a new “model-harness-product-symbiosis.” The model changes alongside the harness; the harness then powers agentic products including Gemini Spark, AI Studio experiences, and the Agents API.

• Gemini was the first common thread stitched through Google’s sprawling product suite. Antigravity is intended to become the next one as products turn “agentic by default,” providing reusable orchestration rather than making every team independently rebuild tool calling and agent loops.

• Tulsee said 3.5 Flash will span the Gemini app, AI Mode in Search, Antigravity, AI Studio, and Spark. That breadth is also a constraint: the same model must work for consumer, developer, Search, and Cloud users with materially different goals.

• The model and harness are increasingly designed together, but implementations remain extensible because “everyone’s not building the same product.” Different teams can customize the common infrastructure while feeding recurring model failures back to DeepMind.

4. Full-stack optimization raises lock-in questions Google cannot ignore

• Labenz compared the harness to a “mecha suit” around a general-purpose knowledge worker. His challenge was whether deep co-training ends today’s model-and-infrastructure mix-and-match market, creating siloed stacks with higher switching costs, stickiness, and frontier-lab pricing power.

• Logan’s preferred outcome is both optimization and choice: Google’s “full stack AI story” should make Gemini work exceptionally well, while the model should remain useful with other harnesses because developers and enterprises demand flexibility.

• Tulsee called the requirement “harness diversity.” Co-development creates a flywheel—cleaner data collection, faster debugging, stronger evaluations, and faster iteration—but Gemini must still support customers’ differing tool and orchestration architectures.

• Labenz floated HarnessBench to measure that generalization. His test echoes Demis’s games framing: if a supposedly general model works only inside one optimized scaffold, that is evidence of “jagged intelligence,” not robust capability.

5. Recursive improvement is real, but humans still allocate the expensive bets

• Logan said Gemini is already “pretty deeply” involved in improving Gemini: it increases researcher productivity, submits code changes, runs evaluations, suggests research improvements, and can contribute work that ultimately changes the model.

• Tulsee’s sharpest example came from safety-and-alignment lead Anca, who messaged from a hot tub: “I could run all of these ablations from my phone.” She described ablating Gemini, testing issues, comparing SIs and data ablations, and producing a report within the last hour.

• Logan’s qualification is economic rather than philosophical. Near-term autonomous pre-training does not seem realistic because major runs consume enormous resources and scarce compute; the opportunity cost of letting an AI choose a bad direction keeps “the human in the driver’s seat.”

• The role division is nevertheless changing. Automation lets researchers spend more time interpreting results and choosing strategy, while Gemini Mic lets users ramble through an idea and have Antigravity turn that speech into actions and content such as slide decks or reasoned output.

6. Standardization accelerates products, while products train the next model

• Logan’s infrastructure lesson is blunt: “Every 12 to 18 months now, you have to rewrite everything from scratch.” The serving stack evolved from tokens-in/tokens-out into tool loops and agent orchestration, so duplicating each rewrite across many product teams would become prohibitive.

• Tulsee emphasized iteration speed: move a promising checkpoint into internal developers’ hands, prototype prompts, run live experiments, build signal-bearing evals, and discover failures only visible in a working product. Those failures then become new data and evaluation targets for the next revision.

• NotebookLM is the exemplar. The original Audio Overview’s surprisingly coherent conversation was “base Gemini with a bunch of banger prompts”; the team knew how to “prompt-whisper” the model and get the best out of it.

• Labenz summarized the direction as “the model eats the scaffolding.” At each capability turn, functions previously supplied by bespoke prompts or surrounding software can be absorbed into the model or common harness, allowing product teams to focus more on the product itself.

7. Omni pushes native multimodality into video, but the API is unsettled

• Tulsee introduced Gemini Omni Flash as a move toward “all modalities in and all modalities out,” initially through video generation and editing. Users will be able to combine scenes and source images, put their own avatar into videos, and use Gemini’s world knowledge and reasoning to generate native video.

• Labenz called it the “Nano Banana moment for video,” meaning language, reasoning, and pixel understanding remain integrated rather than passing through a lossy textual intermediary. Tulsee embraced that as the internal tagline.

• The distribution plan is more definite than the developer interface: Omni is initially coming to the Gemini app, Flow, and YouTube, but “it’s not available in the API yet.” Existing Gemini video input downsamples frames with a configurable FPS parameter; future API behavior and sampling decisions remain undecided.

• Audio is advancing in parallel. Gemini Live is faster, smarter, and better at detecting background noise, while Logan sees audio becoming a larger part of how people engage with models and how models engage back. Tulsee calls it a frontier modality.

8. Gemini’s persona is evaluated as product behavior, while welfare claims remain uncertain

• Labenz contrasted Anthropic’s treatment of Claude as something like a mind with OpenAI’s tool-oriented Model Spec. Tulsee said Google DeepMind contains a range of perspectives, but the shared product framing is Gemini as a “collaborator” and partner in work and daily life.

• Asked about “psychological distress,” Tulsee reframed the concern around how a long-running partner communicates. Every checkpoint receives safety evaluations for sycophancy, role-play, looping, and rabbit-holing, with launch-over-launch comparisons intended to catch deterioration.

• Tulsee was explicit that when Gemini goes off the rails, it is a “model bug”; Logan added that such behavior is not intended. User feedback is valuable because the team wants to identify and remove those patterns.

• On model-welfare interviews, Tulsee’s skepticism is that models usually lack any context about their training or deployment. Asking how they feel about those systems therefore pushes them out of distribution, leaving them “just pontificating” from broad training patterns.

9. Search grounding and compaction matter more than ever-larger raw windows

• Labenz said he believed publicly launched Gemini models still had a January 2025 knowledge cutoff despite performing well on recent research. Tulsee accepted “bug” as a characterization; Logan said updating the cutoff is important, while explaining that Search grounding lets the model distinguish parametric knowledge from information that may be only minutes old.

• Google Cloud’s Exa partnership does not signal abandonment of Google Search, Logan argued. Cloud already offers competing models and services because enterprise customers want choice; meanwhile, Gemini’s direct collaboration with Search remains deep.

• Tulsee’s context-window thesis is to use context “smartly.” Personal archives and giant codebases may contain vast information, but much is distracting; selection and compaction can expose the useful subset while letting the effective information pool exceed the active window.

• Economics reinforce that architecture. Logan said a 1 million-token request can cost “a few dollars” in some cases, making demand small relative to its serving-compute burden. He hopes for a research breakthrough, but today customers often deliberately choose smaller windows for cost and latency.

10. Diffusion remains research while Flash captures the immediate speed demand

• Tulsee said diffusion coding is “awesome” and “super fast,” with researchers still improving quality and speed while testing where it is useful enough to release. Logan characterized the prior demonstration as a look behind the curtain, not a committed product roadmap.

• Flash raises the question of where further acceleration starts producing diminishing returns rather than meaningful user value. The 3.5 Flash model was cited at roughly 280 tokens per second, and Logan said a faster version of 3.5 Flash is already in Antigravity.