⚡ Inside Google Labs: Building The Gemini Coding Agent — Jed Borovik, Jules + AIE CODE Preview
Summary
Jules is built for a future in which coding agents run for hours or days on dedicated cloud computers, rather than merely assisting inside an IDE. Borovik’s defining phrase is that “autonomous means, like, it has its own computer”; Google is designing for agents that can be multiplied, triggered remotely, and become ambient infrastructure.
Borovik described an advantage in the feedback loop between Labs’ product team, DeepMind’s models, internal developer tooling, and usage data. He described an end-to-end system spanning “pixels on the page” through infrastructure, models, and training, while noting that coding matters both “from an AGI perspective” and “from a dollars perspective.”
Improving foundation models are steadily making elaborate agent scaffolding less necessary. Early Gemini versions required complex subagents and orchestration; now Borovik’s conclusion is “less is more,” with scaffolding increasingly resembling a crutch for model weaknesses. On RAG, the host argued that semantic search should remain one tool among others rather than the system’s only organizing principle; Borovik agreed that embedding-based RAG is not the only approach and said Google is actively researching the area.
Jules is becoming a durable product, while a separate 30-day session anecdote revealed how long users may sustain a coding-agent workstream. Jules’s reception after its May announcement following I/O became a turning point, and its appearance in GitHub’s keynote reinforced Borovik’s message that it is “certainly here to stay.” Separately, the host’s team assumed nobody would sustain one session for 30 days—until the first user hit the session-data limit and became upset when it locked.
Distribution spans the API, CLI, GitHub workflow, and eventually the wider Gemini surface. One user triggers Jules to update a site and uses a GitHub Action to merge its pull requests automatically, illustrating how an agent can disappear into workflow infrastructure. Borovik said the recently launched Jules CLI would be integrated with Gemini CLI by the episode’s publication.
The host sees vertical agent businesses as economically easier than generic agent infrastructure, while Borovik sees software demand expanding rather than employment simply collapsing. The host said agent companies appear to be “shooting fish in a barrel,” ramping ARR faster with better margins, though he explicitly allowed that exceptional infrastructure companies can win. Borovik’s counter to labor pessimism is elastic demand: if engineers become 50%, 100%, or even 10x more productive, more software projects may justify investment.
Context management, not raw context-window size, becomes critical when agents work for weeks and consume files plus enormous command outputs. Even a context window of up to 2 million tokens may be insufficient; summarization, external notes, and subagent handoffs all create information-transfer challenges. The host’s emerging answer for local-to-cloud continuity was not brain-state transfer but artifacts: “I don’t transfer my brain state to you. I just tell you what I did.”
The successor to careless vibe coding must solve both specification and verification. The host framed the two problems as specifying what is wanted and verifying that the result matches it. Spec-driven development can improve prompts but risks becoming a 50-page waterfall exercise; interactive planning admits that users often discover what they want through feedback. Borovik described images as another input mode, the host requested video for time-dependent UI bugs, and both discussed computer use as an interface beyond chat.
Deep dive
1. Stable Diffusion turned an experienced Googler toward coding agents
After nine years at Google, much of it on Search freshness and news, Borovik traces his AI turn to Stable Diffusion—not ChatGPT. Artists split between “this is stealing my art” and treating it as “a tool to create better art,” a choice he soon recognized in programming.
When people began advising college students against computer science, Borovik’s reaction was personal: “This is my Stable Diffusion moment.” AI could take “my art, my craft,” or become a tool for making better software; he deliberately chose the second path and eventually joined Jules.
Google Labs exists to build “new, innovative products that the rest of Google isn’t well positioned for.” Although fundamentally a product organization, it works closely with DeepMind across the complete loop—from interface pixels and infrastructure to model training.
Google had already published work on coding assistance and built internal capabilities around Cider, its internal IDE. Borovik emphasized the advantage of combining in-house models with unusually rich usage data, allowing sophisticated systems that had previously remained internal.
2. Jules assumes autonomous agents will leave the IDE
The core question was not how to copy existing coding assistants, but where agents go as models become more capable: “Let’s build for that future.” If an agent can tackle complicated work for hours or days, Borovik argues it should have its own environment rather than occupy the developer’s local workspace.
Autonomy is only the first layer; the team also wants Jules to become “ambient.” Its API lets events initiate work without a developer opening a chat, including one workflow where Jules updates a site and a GitHub Action automatically merges the resulting pull requests.
The product is spreading across developers’ existing surfaces. Jules had launched a dedicated CLI a few weeks before recording, while Borovik said Gemini CLI integration would arrive by publication; APIs cover remote triggers, while CLIs cover developers already working locally.
3. Stronger models make thinner agent harnesses viable
Borovik’s clearest engineering lesson is that model quality changes what can be built and how much orchestration it needs. Early Gemini models required “incredibly complex” scaffolding; as the models improved, that architecture simplified because “the scaffolding is almost a crutch” for capabilities the model lacks.
The team experimented with separate agents for edits, subproblems, integrations, reviewing, and other roles. Borovik did not claim the modern harness is unsophisticated, but found that fewer rigid components are easier to maintain and improve through both ordinary engineering and machine learning: “Less is more.”
Embedding-based RAG produced a parallel rethink. The host’s pushback was unusually categorical: arbitrary chunk boundaries mean retrieval “will never be good” as the sole mechanism because a chunk capturing one semantic relationship misses another; large-context attention can avoid hard-coding that bottleneck.
Neither speaker discarded semantic retrieval entirely. The host still favors it as one tool alongside file access, grep, and the web, rather than “the only way to do things”; Borovik agreed and said Google continued researching the area. The host also said that after pointing people toward several code-embedding models, his side found it did not need them.
4. Jules crossed from experiment to product as adoption and visibility grew
Borovik said Jules was always intended to become real, but its May announcement following I/O was the decisive milestone: strong reception triggered a push to turn it into a durable product. Its mention in the GitHub keynote reinforced the internal posture—“it’s certainly here to stay.”
The host offered a separate market read: too many capable developers build frameworks because developer tools feel comfortable, without conviction about a customer or vertical. He argued vertical agents appear to ramp ARR faster and carry better margins—“shooting fish in a barrel”—while qualifying that outstanding infrastructure companies can still succeed.
Their discussion of AI Engineer Code framed conferences as industry-neutral gathering points during the shift from academic papers and open research toward products, startups, and closed models. The host estimated roughly one acceptance per 23 applicants for the upcoming summit and said the “hallway track” matters most because the talks are published free.
That hallway track extends beyond the official program: the previous New York event generated about 15 participant-organized dinners and meetups. The host’s advice was to arrive with a concise calling card—what you do, what you can help with, and what help you need—then create focused gatherings around niches such as voice, code-review agents, finance, or biology.
5. A thirty-day session limit turned context into a research frontier
The host shared a coding-agent anecdote in which his team stored session data for only 30 days. They assumed, “There’s no way anyone’s going to be using a single session for 30 days”; the first user who hit that limit became upset when the session locked, revealing how long a coherent agent workstream can persist.
Coding agents stress context unusually hard because they ingest files, command outputs, and many turns while running for weeks. Even an available window of up to 2 million tokens does not remove the problem, making coding simultaneously a high-impact product category and a demanding research environment.
Available techniques include summarization, externalizing state into notes, and handing bounded work to subagents that do not need to carry all the parent thread’s context. Each creates a fidelity problem—“how do you make sure enough information is going back and forth?”—and the speakers noted that the best approach may change as models change.
The host had not read papers comparing which methods work best, while Borovik described the topic as active research he could not detail. The lack of settled practice matters because long-lived coding-agent sessions are already challenging the original assumptions behind their design.
6. The durable handoff may be a pull request, not a conversation
The emerging product surface is becoming recognizable: IDE, VS Code extension, CLI, and cloud agent. The unresolved question is how those modes interoperate when a developer starts locally, delegates asynchronously, and later needs to resume deep work without replaying an entire agent history.
The host initially imagined “teleporting” a conversation between tools, then revised his own answer. The transferable unit might instead be a Linear ticket, GitHub pull request, or concise status update: “I don’t transfer my brain state to you. I just tell you what I did”—and omissions are discovered later, as in human collaboration.
Borovik wants the industry to articulate the experience it is creating, not treat coding agents as a zero-sum race. His question for peers is “what is this beautiful future we’re creating?”—including how local and cloud work fit together and how software engineering becomes a better craft rather than merely a faster production line.
7. Software demand may expand, but careless vibe coding has hit its limit
Borovik cautioned that AI-tool adoption remains less diffused across companies than he sometimes expects, yet personally finds returning to unaided coding unimaginable: “That sounds horrible.” He rejects the simple claim that improved agents necessarily mean companies stop hiring software engineers.
His mechanism is demand elasticity. Farming employment collapsed because food consumption has a ceiling, while cheaper electricity induced more consumption; software, he argued, has highly elastic demand.
If one engineer can deliver 50%, 100%, or 10x more, projects previously unable to justify investment may become viable. The host’s counterweight was qualitative: agents can absorb commoditized work, ideally leaving engineers more room for strategic, novel, high-risk, and deeply focused work.
The host argued “vibe coding” had become permission to YOLO prompts, produce “the worst code imaginable,” and leave cleanup to others. “Agentic coding” was the leading replacement term, but lacked vibe coding’s joy; the desired practice must recover care, craft, and reliability.
Spec-driven development offers one answer because “if you prompt sloppily, you’re going to get slop,” yet a three-day, 50-page specification merely recreates waterfall. The host favored interactive planning: users often do not know what they want until the agent produces something they can correct.
The host framed trustworthy agent work as two paired problems: specifying the intended result and verifying that the result matches it. Jules runs existing repository tests when they are present, but the host noted that it does not force every change to introduce new tests; the engineer should control that output.
Specification should also move beyond chat. Jules already accepts images, the host requested video for time-dependent UI bugs, and both saw computer use as the next layer—agents rendering and operating a VM or browser, provided the system can achieve sufficient precision, speed, and affordable cost.
Borovik invited both enthusiastic users and skeptics to share their workflows, failures, and reasons for avoiding these tools. He is also hiring and said he would be happy to talk with people interested in working on coding agents.