Summary

GPT-5’s strategic purpose is to make reasoning the default rather than force users to choose between instant GPT models and the slower o-series. OpenAI is researching how much thinking each prompt deserves, aiming to remove that product friction while delivering more agentic behavior “by default.” For investors, the competition is shifting toward adaptive thinking budgets, latency, reliability, and usable autonomy.
OpenAI considers many familiar evals effectively saturated and is moving toward benchmarks based on genuine discovery and economic relevance. Reinforcement learning can create narrow domain experts, so improving from 96% to 98% may say less about generalization than it once did. AtCoder and IMO remain credible markers because leading researchers passed through them, but the next milestone is “actual movement on things that are economically relevant.”
The central research roadmap is an “automated researcher” capable of discovering new ideas in machine learning and other sciences. Jakub Pachocki estimates that reaching near-mastery of high-school competitions would correspond to roughly “one to five hours of reasoning”; progress now requires longer planning, persistent memory, recovery from failed approaches, and autonomous operation measured over expanding time horizons.
Reinforcement learning keeps producing gains because language-model pretraining supplied the rich environment that earlier RL systems lacked. Natural-language modeling gave models a nuanced understanding of human language, after which researchers could explore many objectives and domains. Mark Chen expects reward design to become simpler, while Pachocki says learning should move toward something more humanlike and warns enterprises “not to assume that what is now will be forever.”
GPT-5-Codex shows that deployment quality depends on allocating intelligence and time correctly, not merely maximizing it. The previous generation spent too little time on the hardest tasks and too much on easy ones; the new work targets lower latency for simple jobs and deeper reasoning for difficult, messy coding environments. Jakub Pachocki, who said he had mostly used Vim, said a 30-file refactor can now be completed “pretty much perfectly in 15 minutes,” although the tools remain in an “uncanny valley” short of a coworker.
“Vibe researching” is a hoped-for future, but the guests argue that taste, persistence, and honest failure analysis remain load-bearing. Research means attempting something “that will most likely fail,” maintaining conviction without going out of one’s way “to prove that it works,” and recognizing both software bugs and flawed conceptual frames. The human research contribution still includes choosing important, hard problems and learning when to persist or pivot.
OpenAI’s organizational approach combines protected fundamental research, deliberate prioritization, and still-scarce compute. The lab resists chasing every competing release, maintains distinct mandates for algorithmic advances and product-oriented research, and Jakub’s interrupted answer to an extra-10%-resources question was “compute.” The risk of diffuse investment is ending up “second place at everything,” while longer-term constraints broaden from compute to energy, robotics, and the physical world.

Deep dive

1. GPT-5 makes reasoning the default rather than a separate mode

Mark Chen framed GPT-5 as an attempt to “bring reasoning into the mainstream.” Earlier users had to choose between the instant GPT-2/3/4 lineage and an o-series model that thought much longer; GPT-5 instead tries to infer the appropriate reasoning budget for each prompt.
The product thesis is as important as the raw capability: users should not be “puzzled” about which mode to select. OpenAI wants to take that decision away while making reasoning and agentic behavior available “by default,” even though the model also improves broadly over o3.
The guests described different personal thresholds. Chen said o3 made mathematical formulas and derivations “fairly trustworthy” for daily work; Pachocki said GPT-5 then pushed further into hard sciences, where physicists and mathematicians repeatedly found it could produce non-trivial mathematics that earlier models could not.

2. Static benchmarks are yielding to discovery as the real eval

Pachocki’s critique of legacy evals: moving from 96% to 98% on a nearly saturated test is no longer decisive. During the GPT-2-to-GPT-4 era, one pretraining recipe could be judged by broad generalization; reinforcement learning can now deliberately make a model expert in one domain, weakening the inference from a high specialized score to broad generalization.
Competition results still matter because they are not merely artificial tests. Pachocki called AtCoder and IMO “real-world markers for success in future research,” noting that many exceptional researchers developed through those contests; OpenAI had already reached number two in the AtCoder competition.
The more consequential evidence came from working scientists. Pachocki described a “light-bulb moment” when GPT-5 automated work that might have taken a student months, while qualifying that the mathematics was nontrivial but not necessarily very complicated.
OpenAI therefore sees itself in an “evaluation deficit.” The replacement target is not another saturated question set but evidence that a model can discover something new and produce “actual movement” on economically relevant work.

3. The automated researcher is a horizon-and-memory problem

Pachocki stated the roadmap plainly: “The big thing that we are targeting with our research is producing an automated researcher.” Automating OpenAI’s own machine-learning research is an obvious test, but the team also wants models to advance other sciences rather than measure itself only through a self-referential loop.
Pachocki estimated that reaching near-mastery of high-school competitions would correspond to roughly “one to five hours of reasoning.” The next jump requires planning over much longer horizons and retaining memory, making the duration of reliable autonomous operation a particularly important eval.
The hosts’ pushback is worth keeping: every additional tool call or planning hop can create quality regressions or another opportunity for the system to drift. Pachocki’s answer was that they are fundamentally the same problem: maintaining depth means remaining consistent, and reasoning models have already lengthened the period before they go off track.
Chen illustrated the required robustness with mathematical work: try an approach, absorb hard feedback, diagnose the mistake, choose another route, and repeat. Reasoning is this extended recovery loop, not simply producing more intermediate steps.

4. Long-horizon research collapses the divide between verifiable and open-ended work

Pachocki rejected a clean boundary between formal and open-ended domains once tasks last months or years. An hour-long constraint problem may have a finite idea space; proving a Millennium Prize problem would require choosing relevant fields, seeking inspiration from physics, and designing an entire research program.
Even a seemingly crisp objective such as reducing modeling loss becomes open-ended at research scale: the harder question is whether the team is “actually asking the right questions.” Longer horizons therefore make problem formulation, taste, and exploration part of otherwise verifiable work.
Anjney Midha noted that OpenAI also considers the limits of “open-ended,” citing Sam’s earlier tweet about making models write more creatively. The claim was not that every domain now has an objective reward, but that research must eventually span both hard feedback and less explicit judgments.

5. Language pretraining turned RL into a compounding research platform

Chen traced RL’s earlier bottleneck to the environment: researchers had a powerful learning method but struggled to anchor agents to reality, even considering simulated islands where they might collaborate and compete. Large-scale natural-language modeling supplied an environment with an “incredibly nuanced understanding of human language.”
Once reinforcement learning operates on that pretrained base, researchers can pursue many objectives inside a robust, information-rich environment. Chen called this perhaps OpenAI’s “most exciting period” in several years because numerous new directions appeared promising at once, while conceding that the team is still learning how they combine.
Asked how a biologist, physicist, or enterprise should construct rewards, Chen expected the process to evolve rapidly and become simpler. Pachocki said the field is still moving beyond the way it thought about fine-tuning datasets two years ago and toward more humanlike learning; his key warning was, “Do not assume that what is now will be forever.”

6. Codex reallocates thinking time and makes manual coding feel optional

Chen described the Codex team’s job as converting raw reasoning intelligence into useful real-world coding. Coding environments are messy, and quality includes softer behavior — style, proactivity, and whether the model is “lazy” — so the team is effectively defining a behavioral specification as well as improving task performance.
A central correction was time and latency allocation per task. The prior generation spent too little time on the hardest problems and too much on easy ones; GPT-5-Codex targets lower latency when a problem is simple and higher latency when deeper work materially improves the result.
Pachocki said he had mostly used Vim and described his conversion: “You can do a 30-file refactor pretty much perfectly in 15 minutes — you kind of have to use it.” Yet he still placed the experience in an “uncanny valley,” highly useful but “not quite as good as a coworker.”
The competitive-programming frontier is not completely closed: IMO Problem 6 and the hardest programming problems still leave “a little bit of headway.” But Anjney’s encounter with high-schoolers suggested the cultural transition has arrived — writing mechanics from scratch seemed strange to them because “the default way to code is vibe coding.” His hoped-for sequel: “the future hopefully will be vibe researching.”

7. Vibe researching still depends on persistence, taste, and failure

Pachocki’s definition begins with uncertainty: research creates or learns something not already known to work, meaning “you’re always trying something that will most likely fail.” The researcher needs clear hypotheses, readiness to learn, and enough honesty not to distort evidence merely to prove the favored idea works.
The hosts pressed on whether conviction and truth-seeking are inherently in tension. Pachocki said no: someone can believe strongly and persist while results remain negative, provided they accurately track progress and update from failures rather than confusing commitment with confirmation.
His problem-selection heuristic is to pursue questions that are both important and widely considered not tractable, then ask what specifically blocks the next step. Caring deeply about the outcome supplies the motivation needed to continue for years.
Chen emphasized that experience teaches the right problem horizon and the emotional management of repeated failure. Papers and colleagues help develop “interestingness,” but there are “very few shortcuts”; sometimes the correct choice is perseverance, and sometimes it is switching problems.
One recurring breakthrough mechanism is finding bugs. These may be literal software defects that quietly invalidate months of experiments, or “bugs” in a mental frame that produce wrong assumptions; both the first reasoning models and the larger pretrained models involved issues that required rethinking assumptions and frames.

8. OpenAI protects frontier research while compute remains scarce

Chen said the recruiting proposition is fundamental discovery, not asking what “company X” or “company Y” just released: “We really don’t like copying.” Hiring favors people who have solved hard problems in physics, computer science, finance, or elsewhere, coupled with technical fundamentals, ambition, and the willingness to stick with difficult work. Chen also cited OpenAI’s resident program as a way to bring people from other fields into AI through hands-on implementation of core results.
Research talent need not fit one engineering archetype. Some researchers generate “so much alpha” through a stream of ideas; others rigorously map the experimental space around one idea. The organization aims to combine those styles rather than require every contributor to express impact identically.
Both leaders called it essential to “protect fundamental research” from constant product demands and competitive releases. Product-oriented researchers should have explicit accountability, while another group receives room to ask what models could look like in one or two years; the automated-researcher objective supplies a coarse compass without prescribing every project.
Nano Banana illustrated the portfolio tension: compelling media tools can unlock everyday creativity, and OpenAI does not discourage researchers excited by those products. The answer is consistent prioritization and separate protection for algorithmic advances, with compute moved dynamically between core and product research as monthly needs change.
Asked where an additional 10% of resources would go, Pachocki began answering “compute” before the exchange moved on. He also said he had not bought the claim that AI would soon become primarily data-constrained; anyone doubting the scarcity should “step into my job for a week.” Without prioritization, the danger is becoming “second place at everything.”
Pachocki’s durable priors extend beyond compute to energy and other physical constraints, with robotics likely to become a major focus “not too far” away. On intelligence itself, he would make few assumptions.
The lab’s speed rests partly on avoiding a learning plateau: Chen said new results emerge fast enough that staying current is a full-time job, while Pachocki described repeatedly reconfiguring around new constraints and possibilities. Their own partnership grew from an unpopular early reasoning effort — Pachocki attacking technical problems deeply, Chen turning scattered contributors into a coherent team.

Amp: The Emperor Has No Clothes September 2025 Random Ramblings