Code Context is King: Augment’s AI Assistant for Professional Software Engineers, with Guy Gur-Ari
Summary
Augment’s core wager is that enterprise coding is a context-infrastructure problem, not a prompt-to-app problem. Its target customers have hundreds or thousands of developers working across millions of lines of evolving code; even a 1 million-token window holds only about 100,000 lines at the stated 10-tokens-per-line ratio. The product therefore makes “full codebase understanding” foundational to completions, chat, and agents.
The technical moat is a retrieval stack optimized simultaneously for relevance, branch-level freshness, security, and roughly 300-millisecond completion latency. Augment sends a request on every keystroke, combining proprietary retrievers, static analysis, multiple models, and user steering; it also built a vector database because existing products could not provide real-time, isolated views of each developer’s branch without duplicating the repository. Guy Gur-Ari’s qualification is important: sophisticated chunking helps with tail cases, but “if you have strong retrievers and strong models, chunking shouldn’t be a blocker.”
The highest-leverage AI-development practice discussed is almost comically modest: start with 10 to 20 examples you understand deeply. Establish an off-the-shelf baseline, compare iterations manually if necessary, and expand the evaluation only after it saturates; then supplement it with dogfooding and user feedback because no single metric captures the prompt and expectation distributions of real users. The goal is to turn “philosophical questions” into experiments.
Modern RAG should optimize recall before precision, provided latency and cost permit it. Nathan Labenz’s rule is to “turn your hyperparameters up,” and Gur-Ari agrees: models can now sift through 50 or 100 retrieved chunks far better than older systems, while forcing the answer into the top five or 10 makes retrieval exponentially harder. The exception is instructions, where adding more context still causes models to skip steps or ignore requirements.
Augment’s free tier creates a potentially valuable data flywheel, while enterprise data remains segregated and inaccessible for training. IDE activity can reveal what code the developer ultimately chose, enabling reinforcement learning from better-versus-worse outputs across correctness, style, and codebase conventions. Gur-Ari calls real user data the “holy grail” because it minimizes the gap between training tasks and actual production use.
Coding agents create obvious customer value but threaten conventional SaaS economics. One instruction may generate 10 or more large model calls, and Gur-Ari says Augment’s agent has written all his code for several months; meanwhile, some users could already justify $1,000 per month against today’s $30 and $60 plans. With roughly $250 million raised, Augment can absorb experimentation, but Gur-Ari expects consumption-oriented pricing to become more relevant as agents run for hours, overnight, or without direct user triggers.
AI may soon generate most new lines of code without eliminating the developer who decides, steers, and supervises the work. Gur-Ari thinks a three-to-six-month adoption forecast is probably too fast, but also doubts the transition takes three years: “If you took away the human, nothing good would happen.” Augment still hires excellent junior engineers, yet Gur-Ari cannot confidently forecast software employment three to six years out and advises his own older child to consider fields tied more closely to the physical world.
Deep dive
1. Enterprise software work is constrained by inherited context
Augment was founded in 2022, before ChatGPT existed, when autocomplete was the visible product signal. Gur-Ari’s team believed models were crossing a usefulness threshold, but chose the harder market: professional developers maintaining and extending existing systems rather than generating isolated greenfield apps.
The representative customer has hundreds of developers; some have thousands. Repositories may be monorepos or split by microservice, but the code generally begins at millions of lines, making feature development inseparable from historical architecture, internal APIs, conventions, and obsolete patterns that remain in the tree.
Nathan’s personal workflow—concatenate a small project and paste it into a model—breaks numerically at enterprise scale. Gur-Ari estimates roughly 10 tokens per line, so even a 1 million-token context window covers only around 100,000 lines, “which in industry is still considered a small project.”
Context is not merely finding a function signature. The model must choose among several viable APIs, supply correct parameters, follow the organization’s preferred calling pattern, and distinguish current examples from outdated ones; without that, it produces bad completions or bad chat answers despite understanding the local task.
2. Every keystroke triggers an intent-and-retrieval problem
Augment explored three approaches to codebase understanding, each through a multi-month research project, before landing on a retrieval-heavy architecture. It uploads and indexes the repository with custom-trained retriever models, then selects relevant code for completions, chat requests, and agent work.
Every keystroke can initiate retrieval and generation, yet Gur-Ari wants the entire completion path to finish on the order of 300 milliseconds. That budget includes identifying relevant code across a large repository and generating the answer, making speed “a super important feature” alongside end-to-end quality.
Completion is harder than question answering because the user supplies no clean query. From a cursor, partial code, and perhaps a comment, Augment must infer the developer’s intention before deciding whether the model needs a function signature, current usage examples, related implementation, or counterexamples showing what not to do.
The production system mixes RAG, static analysis, several models, and explicit steering through selected files or directories. Gur-Ari protects the details as “secret sauce,” but demotes one common obsession: better chunking addresses tail problems, while “if you have strong retrievers and strong models, chunking shouldn’t be a blocker.”
3. Developers are moving away from code at different speeds
Augment sees two broad, overlapping camps. Some developers remain close to the editor and rely heavily on completions; others work predominantly through chat, assigning tasks and reviewing proposed changes rather than writing each function themselves.
“Next Edit” preserves the code-centered workflow while extending autocomplete: it can operate away from the cursor, delete or revise existing code, and assist beyond simply appending text. It is assistance without requiring the developer to surrender editorial control.
Agent mode takes another step back by editing several files, running tests, executing commands, and returning a larger unit of work. In big codebases, however, developers still frequently inspect and repair code themselves; Gur-Ari expects autonomous enterprise workflows to arrive later than zero-to-one app generation.
Augment supports VS Code, JetBrains, and Vim rather than forcing everyone into a VS Code fork. A fork offers deeper UI control but imposes update and security-patch maintenance—especially consequential in enterprise sales—and asks JetBrains users to make a substantial workflow change.
4. Real-time branch isolation forced Augment to build its own database
Gur-Ari’s freshness requirement is concrete: after a developer or chat creates a function, the next test-writing request should immediately retrieve it. The model therefore needs something close to a real-time view of every individual developer’s current code, not merely a static repository index.
Those views must remain isolated because two developers’ feature branches cannot leak into each other, including for security reasons. At the same time, Augment wants one deduplicated representation of shared repository content rather than copying an enormous index for every team member.
Conventional approximate-vector techniques often cluster embeddings to make queries affordable. Frequent updates make those structures expensive to maintain, while filtered views can cause a statistical search to miss the relevant subset entirely—requirements that Gur-Ari says no available product they knew of simultaneously met when Augment began.
Building the database was difficult, and larger customer repositories continue to create active scaling work. Erik suggests it could become a standalone product; Gur-Ari agrees in principle but says AI labs repeatedly encounter infrastructure that could be commercialized, making focus on “the best AI assistant” the harder strategic discipline.
5. Ten examples are enough to begin serious AI research
Gur-Ari’s default starting point for a retrieval or model project is only 10 to 20 carefully hand-labeled samples. Run the easiest off-the-shelf baseline and ask whether it solves 20%, 80%, or some intermediate share; once the set saturates, add harder and more diverse cases.
The small set’s advantage is intimacy, not statistical grandeur: researchers can hold every case in their heads and understand why a change helped or hurt. Ground-truth labels can even wait; run the old and new systems side by side and judge all 10 manually when the task is inherently subjective.
Nathan connects that practice to creative work, where two reviewers may disagree about the best script, voice-over, or image selection. Even there, agreeing on 10 specimens of “a job well done” exposes hidden team disagreements and gives experiments a stable starting surface.
No evaluation stays fully aligned with user experience. Prompt distributions, expectations, and quality dimensions resist compression into one number, so Gur-Ari insists on dogfooding and direct user feedback once a system exists: the evaluation remains essential, but cannot substitute for observing real use.
6. Modern RAG wins by retrieving more, not by pretending precision is free
Real-world retrieval is “almost never a single thing.” Embeddings can be combined with structured filters and any other useful signal, while the model receives enough material to ensure the right evidence is present; for modern models, recall is generally more important than pristine precision.
Nathan’s rule is to “turn your hyperparameters up”: retrieve more chunks and expand farther around each hit, accepting some latency and cost. Gur-Ari fully endorses the principle when those trade-offs are affordable because models have become much better at ignoring distracting information.
Landing the answer somewhere in the top 50 or 100 chunks is achievable; forcing it into the top five or 10 makes the research problem “exponentially harder.” Gur-Ari expects long-context retrieval to keep improving because transformer attention is designed to sift through noise and focus on relevant material.
The caveat is instruction density. Models may tolerate large bodies of retrieved evidence while still failing to follow long lists of commands, skipping steps or silently ignoring requirements. Retrieval-context scaling and instruction-following are therefore separate problems with different current limits.
7. Custom training must earn its complexity through faster experiments
Gur-Ari sees a recurring tendency to reach for complicated research too early. His prescription is to bias toward simplicity, trust a task-specific evaluation, exhaust readily available models and retrieval methods, and fine-tune only after experiments show that simpler approaches remain insufficient.
Iteration speed affects whether a solution is found, not merely when. Cutting an experiment from hours to minutes may enable 100 attempts instead of 10—enough to discover that an unexpected open-source model, SQL-style filter, embedding, or combination happens to fit the task.
Augment itself does not pre-train foundation models. It bet before Llama, when open models were weak, that published techniques and available expertise would rapidly improve them; it now post-trains strong open-source bases for retrieval, completion, and next-edit generation.
Nathan proposes continued pre-training on proprietary company knowledge, but Gur-Ari objects that such datasets are simultaneously too large for prompts and too small for efficient training. Multiple epochs risk memorization, updates create friction, and per-user email models are operationally awkward; he would first deploy an agent that searches repeatedly until it thinks it has found the answer.
8. Developer behavior creates a rare training-data flywheel
Augment draws a firm data boundary: enterprise customer data is not inspected and remains behind audited access controls. Its free community tier is explicitly available to users comfortable with Augment examining their activity and using it to improve models, whether or not their code is open source.
An IDE supplies a signal that ordinary chat lacks. A chat system sees a question and an answer but rarely knows the desired ground truth; by following an editing session, Augment can eventually infer what the developer was actually trying to do because the IDE is where the work happens.
Reinforcement learning then uses contrast rather than a binary correct answer: one output can be better than another because of algorithmic correctness, developer preference, style, or conformity with the surrounding repository. Augment first applied this “reinforcement learning from developer behaviors” to produce a better completion model.
Gur-Ari views internet data as more or less exhausted, leaving synthetic generation, paid human labeling, and product usage. User data is the “holy grail” because it matches deployment tasks, but exceeding human capability ultimately needs external rewards: executable code and scientific experiments qualify; judging a poem autonomously does not, absent an already-superior evaluator.
9. Agent usage breaks the economics of conventional SaaS
Augment has raised roughly $250 million, but salaries are not the only meaningful expense. Serving retrieval and inference on every keystroke is costly, training adds more, and Augment leases GPUs rather than operating data centers; falling unit prices have not kept pace with usage growth or demand for the newest models.
The workload—and therefore the bill—rises with agents: a single instruction can trigger 10 or more model calls, including large file edits, shell commands, and interpretation of their outputs. The value is also substantial: Gur-Ari says he has “not written a line of code in several months,” while the agent has written a great deal.
Today’s $30 and $60 monthly plans coexist with a wide usage distribution; some users could justify a $1,000 price even now. Augment already sells credits consumed only when a developer uses the product, aligning payment more closely with activity than a conventional seat purchased regardless of adoption.
The episode offers no definitive pricing answer: enterprise customers value predictability, while usage-linked costs and increasingly autonomous agents pull toward consumption-based models. As agents run for hours, overnight, or over days—and initiate code reviews through APIs rather than direct human prompts—per-developer subscriptions may fit poorly.
10. AI will write the code before it can own the engineering
Gur-Ari interprets Dario Amodei’s near-term superhuman-coder claim narrowly: inspect a newly created line and it will increasingly have been generated by a model. Three to six months is probably too fast given adoption, but he also doubts the change requires three years.
Generated code is not autonomous software development. Humans still decide what to build, steer the agent, and supervise results; in Gur-Ari’s blunt formulation, “If you took away the human, nothing good would happen.” Models cannot yet take product requirements and reliably build the finished system alone.
Longer-term labor demand is much less predictable: enterprises will still need people who understand systems because “if you’re just vibe coding your enterprise software you will run into trouble,” but nobody knows how many developers greater productivity requires. Gur-Ari cannot confidently project three to six years ahead.
Augment still hires both junior and senior engineers, selecting for excellence, and Gur-Ari expects skill in extracting value from agents to matter during the transition. Yet his advice to his 14-year-old is to consider mechanical engineering, robotics, or another field tied to the physical world, where disruption may take longer.