Summary

Agent economics are governed less by the tool-calling loop than by what re-enters the model on every turn. Martin notes that a typical Manus task uses roughly 50 tool calls and Anthropic production agents may use hundreds; his naive deep researcher consumed 500,000 tokens and cost $1–$2 per run. The operational mandate is “feeding an LM just the right context for the next step,” because long histories both exhaust the window and suffer “context rot.”
Offloading is a strong way to lower token consumption without permanently deleting evidence. Martin stores raw tool output in a file system or agent state, then returns a high-recall summary, URL, or pointer that lets the agent retrieve details on demand. Compaction can be lossy, especially when irreversible; Alessio reported Jeff’s meetup claim that curated compaction was about 2× better than automated approaches, so the raw material should remain recoverable.
Multi-agent gains depend on workload shape, not the number of agents deployed. Parallel subagents work well for read-only research collection followed by one centralized writing pass; they become dangerous when several coding agents make interdependent writes and “implicitly make decisions” that conflict. Coordination quality and task decomposition matter more than a multi-agent label.
Simple agentic retrieval can be competitive with elaborate indexing stacks. Against Windsurf’s chunking, embeddings, knowledge graphs, grep, and reranking, Claude Code uses basic file tools without indexing; in Martin’s 20-question LangGraph test over roughly 3 million documentation tokens, an llms.txt catalog with good descriptions and on-demand fetching worked extremely well. “The descriptions in your llms.txt matter a lot,” making document packaging, prompts, and MCP resources part of the retrieval product.
Caching improves latency and unit economics but does not repair degraded reasoning over long context. The hosts suggested that the Responses API and newer Anthropic and Gemini behavior may cache stable prefixes automatically, while Martin had previously used Anthropic’s explicit caching setup. Yet “caching doesn’t solve the long context problem”: 100,000 cached tokens remain 100,000 tokens the model is using, while provider-specific caching can deepen platform lock-in.
Memory is context engineering stretched across time, with automation creating both the utility and the failure modes. Claude Code’s explicit model—load CLAUDE.md, write only when asked—is predictable; the host cited Simon’s example of ChatGPT retrieving an irrelevant memory by inserting a user’s location into an image. Martin’s preferred use is narrower: convert human corrections to an ambient agent into updated preferences so it “gets smart over time.”
The durable agent architecture is the one that can be dismantled as foundation models improve. Martin cites compute becoming 10× more available every five years at the same cost and rebuilt Open Deep Research twice after once-reasonable structure became a bottleneck; the lesson is to “add structure” for today’s capability, but “remember to remove it later.” That favors low-level, rewritable orchestration over opaque agent abstractions.

Deep dive

1. Tool output turns prompt engineering into context engineering

Martin’s definition starts with why the term caught on: successful buzzwords capture a shared experience. Developers were told an agent was merely “tool calling in a loop,” then discovered that making the loop reliable required “feeding an LM just the right context for the next step.” Prompt engineering remains a subset, but no longer describes the whole input surface.
In chat, the human message is the primary variable; in an agent, context also arrives through every tool response across the trajectory. System instructions, user intent, intermediate observations, errors, and retrieved documents all accumulate, so the builder must decide what the model sees at each step rather than merely polish the opening prompt.
Scale makes the distinction concrete. Manus described a typical task as roughly 50 tool calls, while Anthropic described production agents that may make hundreds. Martin’s first naive Open Deep Research loop passed back token-heavy research results until a run consumed about 500,000 tokens and cost $1–$2.
The context window is only the obvious ceiling. Martin also points to Chroma’s “context rot” work: performance can degrade idiosyncratically as context length grows even before the formal limit is reached. A larger window therefore permits more accumulation, but does not remove the need to select and structure what enters it.

2. Offloading preserves evidence while compaction risks deleting it

Manus’s core move is to treat the file system as “externalized memory.” Instead of returning every raw tool result to the message history, an agent can store it on disk or in runtime state and pass back a summary, URL, or reference. The model knows the artifact exists and can fetch it later without repeatedly paying for its full contents.
Asked what metadata survives the offload, Martin cautioned against a casual one-line summary. For Open Deep Research, he carefully prompts a model to produce an exhaustive, high-recall set of key bullets: enough compression to save tokens, but enough coverage for the agent to decide whether the original document is relevant. Cognition goes further, discussing fine-tuned summarizers at agent boundaries.
Reduction can also occur at tool boundaries or near the window limit—Claude Code’s familiar compaction around 95% is the intuitive example. Hugging Face’s research agent leaves token-heavy code execution inside the environment and returns limited context; Anthropic’s multi-agent researcher summarizes findings before passing them onward.
The warning is irreversibility. Manus keeps raw observations available because pruning and summarization can lose a detail that becomes important later. Alessio reported Jeff’s meetup claim that curated compaction was about 2× better than automated compaction. Coding is especially sensitive: the agent may need not just the current implementation, but why earlier decisions were made.

3. Parallel agents work best when they gather rather than jointly build

Cognition’s objection to subagents is not that parallelism never works; it is that every agent makes implicit decisions. If several coding agents write related components, their assumptions can conflict, and summaries may not carry enough history to reconcile them. The host’s testing example sharpened the problem: a test-writing agent may need to understand both how and why the implementation changed.
Martin’s preferred boundary is read versus write. Research collection is naturally parallelizable because subagents gather independent evidence; once that shared context exists, one agent performs the final report-writing pass. Both his system and Anthropic’s research architecture follow this pattern. “What you apply multi-agents to matters tremendously.”
Claude Code’s support for subagents suggests coordinated coding can at least be attempted, but Martin keeps the hedge: coding tasks that require tightly coordinated writes remain much harder. The apparent disagreement between Cognition’s “don’t do subagents” and Anthropic’s successful research system dissolves once the workload and communication burden are specified.

4. Agentic retrieval can sometimes avoid an elaborate indexing stack

Windsurf represents the classical retrieval pipeline Martin described: chunk code along semantic boundaries, embed it, combine semantic search with grep and knowledge-graph signals, then rerank the results. Claude Code takes the opposite approach—no indexing, just agentic exploration with basic file tools—and still works “extremely well.”
Martin tested the trade-off on 20 LangGraph coding questions using roughly 3 million tokens of documentation. He compared vector-store retrieval, an llms.txt file listing documentation URLs and descriptions for on-demand loading, and stuffing all documentation into context. The simple llms.txt route let the coding agent repeatedly identify and fetch only the pages required.
In that April snapshot, Claude Code outperformed Cursor on Martin’s particular test, though he repeatedly cautioned that “these things are always changing.” His durable finding was narrower: agentic fetching worked extremely well when each file had a high-quality description, and those descriptions could themselves be generated cheaply by an LLM crawling the documentation.
MCP can package this context more deliberately. Martin uses project-specific servers containing documentation, resources, and prompts that teach the model how to use the server. The host also described a case where leaving usage instructions in a README led to apparent “skill issue” failures; the prompt worked better when placed in the server. The hosts emphasized that MCP is not merely tool integration—prompts, resources, and sampling can move extraction work outside the main context.

5. Caching cuts repeated inference but leaves context rot intact

Manus recommends caching prior message history because every agent turn otherwise resends the same prefix. Martin had used Anthropic’s explicit caching header, while the hosts suggested increasingly automatic behavior through the Responses API and implicit caching from Anthropic and Gemini. Where histories remain stable, those cache hits can reduce both cost and latency significantly.
Martin’s caveat is decisive: a cached 100,000-token history still presents 100,000 tokens for the model to use. He said the context-rot characterization he cited applies whether or not caching is enabled, so caching solves an economic problem rather than a reasoning problem. The hosts also flagged lock-in: self-hosted open models offer direct control over caches that provider APIs only approximate.

6. Memory is retrieval governed by read and write policies

Martin maps memory across two decisions—when to write it and when to read it—and the degree of automation on each. Claude Code occupies the simple corner: it loads CLAUDE.md every session, while the user explicitly asks it to save new information. Martin likes this “00” approach because both operations remain legible.
ChatGPT illustrates the automated extreme: the system chooses both what to save and what to retrieve. The host cited Simon’s failure example, in which an image request unexpectedly incorporated the user’s location. Martin’s broader framing is that reading memory at scale converges with retrieval: it is another potentially complex RAG pipeline, only over past conversations rather than documentation or the public web.
For ambient agents, Martin finds a clean write signal in human intervention. His email assistant pauses before sending, allows him to change tone or modify a tool call, then uses an LLM to reflect on the correction and update prior instructions. “Memory pairs really well with human in the loop” because explicit edits reveal user preferences over time rather than requiring the system to guess them.

7. The bitter lesson punishes scaffolding that outlives its model

Martin’s cited formulation is that compute improves by 10× every five years at the same cost, and general methods using more data and compute eventually beat hand-built assumptions. Some structure may be necessary at today’s capability level; the discipline is to “remember to remove it later,” before it becomes the ceiling on further gains.
His 2024 research workflow encoded strong assumptions: avoid unreliable tool calling, divide a report into predefined sections, and write those sections in parallel. It was initially more reliable than an agent, but rapidly improving tool use reversed the comparison. The structure then blocked capabilities such as MCP and prevented the model from choosing its own research path.
Martin’s first reduction of structure still retained a hidden mistake: each research subagent wrote its own report section. Because the agents could not coordinate decisions, the combined report was disjoint. He removed independent writing, retained parallel research gathering, and moved to one-shot composition at the end.
The resulting Open Deep Research was, by Martin’s hedge, the best-performing open-source deep-research agent on Deep Research Bench “at least on that particular benchmark,” though not as good as OpenAI’s end-to-end-RL system. He also mentioned strong GPT-5 results and argued that the open-source assistant can ride ongoing model improvements, reinforcing the point that applications benefit from model gains only when their architecture does not block them.

8. Rewritable orchestration ages better than opaque abstractions

The hosts extended the bitter lesson to products: an incumbent can improve an existing workflow with AI, yet an AI-native design may overtake it once models become capable enough. A less-structured product can look inferior for a while, then cross the capability threshold—as the episode’s Cursor example did when Claude 3.5 “hits”—and suddenly escape the structured product’s ceiling.
Martin separates frameworks from abstractions. LangGraph exposes low-level nodes, edges, and state, plus practical checkpointing and state management; he could use those pieces to replace his research workflow with an agent and then rebuild it again. An abstraction such as “from framework import agent” is riskier because its hidden assumptions may be difficult to inspect or unwind.
Shopify’s internal Roast system supplied the organizational argument for low-level orchestration. When many teams independently build agents and workflows, review and coordination become difficult; a common library of composable primitives reduces cognitive load without requiring one opaque agent design. Martin calls himself sympathetic to anti-framework critiques when they are really critiques of abstraction.
The same practicality drove MCP. Once Anthropic’s tool calling became effective around mid-2024, teams created incompatible integrations and “complete chaos” followed; a standard protocol made tools easier to access and review, while reducing cognitive load. The episode’s closing principle is therefore not zero structure, but structure whose assumptions remain visible, replaceable, and removable.

Inside OpenAI Enterprise: Forward Deployed Engineering, GPT-5, and More | BG2 Guest Interview OpenAI’s 姚顺雨: A 3-Hour Interview on 6 Years of Agent Research