Summary

Marblestone’s central bet is that the missing ingredient may be the loss functions — evolution may have encoded a developmental curriculum rather than much pretrained knowledge in the weights. “I think evolution may have built a lot of complexity into the loss functions actually, many different loss functions for different areas turned on at different stages of development. A lot of Python code, basically.” This helps explain the 3-gigabyte-genome puzzle: Dwarkesh’s Python example makes the compactness concrete — a reward function is one line of Python, so a thousand of them cost almost nothing, and it reframes sample efficiency as partly a cost-function problem.
His timeline is the investable number: “my timelines might be more in the 10-year-ish range,” with the transformative single event “still more than five years away or something.” The crux isn’t a claim that LLMs can’t get there — “I don’t have a super strong claim that LLMs can’t do it” — it’s that the current paradigm “is weirdly different from all this brain stuff and I’m kind of waiting for the thing that starts to look more like brain stuff.” He concedes his research is irrelevant in an AI 2027 world.
The cheapest asymmetric bet he names: comprehensive neuroscience for “low billions-dollar scale funding” against trillions in GPU capex. E11 Bio is trying to push one mouse connectome from the Wellcome Trust’s several-billion-dollar estimate down to “low tens of millions,” following the genomics cost curve his PhD advisor George Church watched drop a million-fold in ten years. Dwarkesh’s pitch to labs: “Give me one one-hundredth of your projected budget in 2030.”
A central criticism of current RL: no value functions. Citing Ilya, “You use the dumbest form of RL basically… even compared to what was being done 10 years ago” — Atari-era Q-learning had temporal difference learning, and neuroscience evidence indicates dopamine carries a reward-prediction-error signal consistent with value functions. “The fact that we don’t have value functions at all in the LLMs is crazy. I think because Ilya said it, I can say it.”
Near-term and already priced: formal math is a promising RLVR problem — “I think that this problem of RLVRing the crap out of math proving is going to work,” with “at least one billion-dollar valuation company, Harmonic,” and AlphaProof on the same thesis. What isn’t solved is conjecturing: is there a loss function for a good explanation, something like the Kolmogorov complexity of the rest of math given this theorem? The bigger commercial prize is formally verified software, blocked not by proving power but by the specification problem.
Dwarkesh’s read, which Marblestone half-accepts: the mocked camp may have been right. “All these people that folks make fun of on Twitter, Yann LeCun and Beff Jezos and whatever, maybe they got it right” — met with “That is actually one read of it,” plus the reminder that “that’s what all the neuroscientists and all the AI people thought until 2021 or something.” Test-time compute, on this view, is the sampling that amortization threw away.
The alignment stake sits largely in the reward machinery: channeling Steve Byrnes, the minimum viable Steering Subsystem for capability is far smaller than the one for ethics. “It probably is possible to make super powerful model-based RL optimizing systems… that don’t have most of what we have in the human brain reward functions and as a consequence might want to maximize paperclips. And that’s a concern.”

Deep dive

1. The neglected variable is the cost function, and evolution may encode many of them

Marblestone breaks the brain question into the machine-learning primitives — architecture, hyperparameters, learning algorithm, initialization, cost functions — and plants his flag on the last one: “My personal hunch within that framework is that the field has neglected the role of these very specific loss functions.” Machine learning, by contrast, “tends to like mathematically simple loss functions. Predict the next token, cross-entropy, these simple computer scientist loss functions.”
The signature framing, worth keeping verbatim: evolution built “many different loss functions for different areas turned on at different stages of development. A lot of Python code, basically, generating a specific curriculum for what different parts of the brain need to learn.” The proposed mechanism is: “evolution has seen many times what was successful and unsuccessful, and evolution could encode the knowledge of the learning curriculum.”
He is explicit that this is a hunch inside an admitted vacuum. On the cortex’s supposed universal learning algorithm: “This is a huge question and we don’t know.” And the meta-position he keeps returning to: “we have to empower the field of neuroscience to just make neuroscience a more powerful field technologically… to actually be able to crack a question like this.”

2. The cortex may do omnidirectional inference; an LLM computes one conditional probability

The alternative to next-token prediction, as he states it: any area of cortex can “learn to predict any subset of all the variables it sees from any other subset. Omnidirectional inference.” Whereas an LLM “is just seeing everything in the context window and then it computes a very particular conditional probability.”
The example as told: “it would be weird for a large language model to say ’the quick brown fox blank blank the lazy dog’ and fill in the middle versus doing the next token, if it’s doing just forward. It can learn how to do that stuff at this emergent level of the context window and everything, but natively it’s just predicting the next token.”
He flags the lineage himself — “a lot of the things I’m saying, by the way, are extremely similar to what Yann LeCun would say” — and spells out the energy-based-model mechanic: clamp an arbitrary subset of variables, sample any other subset, then pick a totally different subset and do it again. The payoff is at test time: a network trained to fill one blank “doesn’t know how to do that” for a blank you never trained, because that prediction was never “amortized into the network.”
Dwarkesh’s naive pushback — why not just drop the causal mask and train every token against every token, or add cross-modal labels? “I mean, that may be the way. It’s not clear to me.” Adam’s own leaning is that the brain runs “crappy versions of backprop to predict through a few layers” but does it “in an extremely flexible way.”

3. Byrnes’ Learning/Steering split may answer Ilya’s question about where desires come from

The setup is a quote Dwarkesh got from Ilya: “I’m not aware of any good theory of how evolution encodes high-level desires or intentions.” Marblestone thinks Steve Byrnes — a former physicist and AI safety researcher at Astera synthesizing the literature — “has an answer to Ilya’s question essentially.”
The example, delivered on himself: if he misdescribes energy-based models and LeCun objects, “That’s going to activate in me innate embarrassment and shame, and I’m going to want to go hide” — but “evolution has never seen Yann LeCun or known about energy-based models or known what an important scientist or a podcast is.” So the brain must wire learned world-model features to innate reward circuits it could never have anticipated. Dwarkesh: “The French AI researchers are coming for you, Adam.”
The mechanism: the Steering Subsystem (hypothalamus, brainstem, amygdala) has its own primitive sensory apparatus — the superior colliculus, a subcortical visual system with innate face and threat detection — and the cortex trains a predictor for each of its variables. “Am I about to flinch? Am I talking to a friend? Should I laugh now? Is the friend high status?… Am I about to taste salt?” The neurons that matter for social status are simply “the ones that predict those innate heuristics.”
Dwarkesh presses on generalization: the innate system labels a picture of a spider, but nothing supervised “somebody’s telling me the spider’s on your back.” The resolution is that the predictor’s inputs are the world model’s abstract features, so the word “spider” fires it — demonstrated live: “Now I’m activating your Steering Subsystem… based on these very abstract concepts in the conversation.” Dwarkesh: “If you keep going, I’m going to put in a trigger warning.”

4. Cell-type atlases suggest bespoke machinery below the cortex

The empirical hook comes from single-cell atlases by Fei Chen, Evan Macosko and others under the BRAIN Initiative: sequence RNA, count cell types by region. The result — “there are a lot more weird and diverse and bespoke cell types in the Steering Subsystem, basically, than there are in the Learning Subsystem.” Cortex has “enough to build a learning algorithm up there and specify some hyperparameters”; the Steering Subsystem has “a gazillion, thousands of really weird cells.”
Why each reward function might need its own cell type: genetic wiring needs molecular addresses. “Those cells express different receptors and proteins that say, ‘Okay, when this one comes in contact with this one, let’s form a synapse.’” The contrast: “the amount of Python code needed to make an eight-layer transformer is not that different from one that makes a three-layer transformer. You’re just replicating.” The reward side is “a bunch of bespoke, species-specific, situation-specific crap. The cortex doesn’t know about spiders, it just knows about layers.”
Dwarkesh supplies the payoff for the genome puzzle: 3 gigabytes total, a small fraction coding the brain, and the old story was that evolution found hyperparameters. If instead evolution found the loss functions that aid learning, “it actually makes sense how you can build an intelligence with so little information. Because the reward function, in Python the reward function is literally a line.”
Dwarkesh adds that the compactness compounds, because the reward function inherits the world model’s generalization for free: “It doesn’t have to anticipate everything about the future of the reward function. It just has to anticipate what variables are relevant and what are heuristics for finding what those variables are.”

5. The hominid brain expanded cheaply because social learning raised the returns on cortex

Dwarkesh’s proposed causal chain: social learning raised sample efficiency — “Instead of having to go and kill the boar yourself…, you can just be like, ‘The elder told me this is how you make a spear’” — which raised the incentive for a bigger cortex. Adam agrees and notes it’s affordable: “that can be done with a relatively few genes, because it’s really replicating what the mouse already has, making more of it.”
Asked whether cortex changed or merely got scaled and rewired, he leans toward the latter — “I think a mouse has a lot of similarity in terms of cortex as a human” — while conceding Suzana Herculano-Houzel’s finding that neuron count scales better with weight in primate than rodent brains. “Maybe, maybe. I’m not super deep on this… So I’m not saying there isn’t something special about humans in the architecture of the Learning Subsystem at all.”
On language he keeps the hedge tight: possibly “macro-wiring where you need to wire auditory regions into memory regions… and into some of these social instincts,” pointing at Broca’s and Wernicke’s areas and their prefrontal and hippocampal connections. “So there’s like some small number of genes maybe for enabling humans to really properly do language. That could be a big one.”
His reinterpretation of a Beren Millidge post is the sharpest disagreement in this stretch. Millidge noted that visual and auditory cortex scaled disproportionately versus olfactory between primates and humans, and attributed it to odor data having worse scaling-law properties. Adam’s alternative: the social reward functions needed those channels — “in order to make sense of these cues which guide learning, you needed to activate the vision and audio more than odor.” Even the pupil-and-white design of the human eye exists so we “establish relationships based on joint eye contact.”

6. Test-time compute is the sampling that amortization threw away — and neurons are free samplers

Dwarkesh lays out amortized inference: real inference means scoring candidate causes against observations, intractable, so you Monte Carlo it — “the original Boltzmann machines and stuff” — while a feedforward net jumps straight from observation to the best cause. Adam’s addition: “One way to think about it might be that test-time compute, inference-time compute is actually doing this sampling again. You literally read its chain of thought.”
The generalization Dwarkesh draws, and the interesting open question in it: capabilities elicited by inference-time compute get distilled back into weights, so “digital minds which can be copied, have different tradeoffs” — “it should make sense to amortize more things because you can literally copy the amortization.” What, then, “will make sense to amortize into these minds, which evolution did not think was worth amortizing into biological minds?”
The probabilistic camp’s rebuttal, as he steelmans it: “No, it’s inherently probabilistic and amortizing it in this way just doesn’t make sense,” and the brain’s stochastic sampling neurons are their exhibit. But he won’t let them have it clean — “it’s also strange how perception can work in just milliseconds or whatever. It doesn’t seem like it uses that much sampling.”
Applied to evolution, this is the punchline of the whole framework: the Learning Subsystem got almost no pretraining. “It has a certain architecture, but then within lifetime it learns. Then evolution didn’t actually amortize that much into that network. It amortized it instead into a set of innate behaviors in a set of these bootstrapping cost functions.”

7. Current LLM RL is cruder than Atari-era RL, and the brain runs a stack of model-free and model-based layers

On value functions he defers to Ilya to make the criticism sayable: “conceptually it’s a really dumb form of RL, even compared to what was being done 10 years ago. Even the Atari game-playing stuff was using Q-learning.” He is careful to grant the engineering — “these people are incredibly smart and they’re optimizing for how to do it on GPUs” — before the line: “I know one one-hundredth of what he does about AI, but it’s kind of crazy that this is working.”
The brain’s division of labor as he describes it: striatum and basal ganglia do something like model-free RL over “a certain finite relatively small action space” — motor commands, or cognitive moves like “tell the thalamus to allow this part of the cortex to talk to this other part.” Above that, Peter Dayan’s line of work and the dopamine reward-prediction-error evidence, which “is consistent with learning these value functions” and is part of why DeepMind pursued temporal difference learning at all.
The elegant consequence of omnidirectional inference: you get planning for free by clamping reward. “There’s this idea of RL as inference. You could say, ‘Well, conditional on my having a high reward, sample a plan that I would have had to get there.’” Dwarkesh reaches for a value head; Adam offers “a value input” — reward is just one more sensory variable to predict.
Dwarkesh’s tangent, and the correction that lands: Joe Henrich’s cultural evolution — the bean that poisons you unless you run a precise ten-step process — looks “very much like model-free RL happening at a civilizational level.” Adam: “No, not exactly. Evolution is the simplest algorithm in some sense… the outer loop can be extremely not foresighted.” They land on a stack: “evolution: model-free. Basal ganglia: model-free. Cortex: model-based. Culture: model-free potentially.”

8. Describe the brain like a training run, not a circuit diagram — and expect it to be ugly

Dwarkesh’s challenge is the strongest in the episode: we have full read access to neural network weights and still can’t explain why LLMs are intelligent, so why would a connectome help? Adam disputes the premise: we do have a description of an LLM — “I have an architecture and I have a learning rule and I have hyperparameters and I have initialization and I have training data.” Dwarkesh’s counter: “those are things we learned because we built them, not because we interpreted them from seeing the weights.”
The framing he takes from Konrad Kording and Tim Lillicrap’s “What does it mean to understand a neural network?”: train a net on the digits of pi or on stock prices and interpretability will never fully capture the internals — “but we can still say that the way it got that way is that it had an architecture and we gave it this training data and it had this loss function. So I want to describe the brain in the same way. I don’t need to understand how it computes ‘Golden Gate Bridge.’”
The dissent he refuses to dismiss is György Buzsáki’s The Brain from the Inside Out: our psychology and AI vocabulary “is just made-up stuff,” and “we have to start with the brain and make new vocabulary.” Adam wants both in the portfolio — a fully bottom-up simulation of a worm or zebrafish, plus AI-vocabulary reverse engineering — while betting on the latter: “My guess is that the reverse engineering one is actually going to work-ish or something.”
His hunch on how the brain represents its world model is the honest low point: “it’s going to be a huge mess and we should look at the architecture, the loss functions, and the learning rules. I don’t expect it to be pretty in there” — probably not symbolic, though “other people think very differently.” On conscious experience he simply folds: “I’m pretty much at a loss on this one… I don’t think anybody has any idea. It might even involve new physics.”

9. Connectomics is a Human-Genome-Project-shaped bet at low-billions scale

The cost curve is the whole thesis. Wellcome Trust put the first mouse brain connectome at “several billion dollars”; E11 Bio and the surrounding field are trying to get “a single mouse connectome down to low tens of millions of dollars.” A human brain is roughly a thousand times bigger, so naive scaling still means billions — but he doesn’t want every human neuron: “an entire mouse brain and a human Steering Subsystem and the entire brains of several different mammals with different social instincts.”
The precedent, from his PhD advisor George Church: the first genome cost about $3 billion, “roughly $1 per base pair,” then the National Human Genome Research Institute “structured the funding process right,” companies competed, the paradigm shifted from macroscopic chemistry to imaging individual DNA clusters, and “the cost dropped like a million-fold in 10 years.” E11’s analogous move is electron microscopy to optical, which yields a “molecularly annotated connectome” — not just who connects to whom, but which molecules sit at the synapse and what cell type it is.
Funding is unresolved and he says so: philanthropy so far, an NSF “Tech Labs” call that is “somewhat FRO-inspired,” and “some rumors going around of connectomics-related companies potentially forming.” He tried the AI labs “seven or eight years ago and there was not a lot of interest. Maybe now there would be.” The pitch is a rounding error: “in the grand scheme of trillions of dollars of GPUs and stuff, it actually makes sense to do that investment.”
The alignment case for spending it, channeling Byrnes: “the minimum viable things in the Steering Subsystem that you need to get something smart is way less than the minimum viable set of things you need for it to have human-like social instincts and ethics.” A competent paperclip maximizer still needs curiosity and some interest in others, “but that’s pretty minimal I think. And that’s true for humans, but it might be less true for something that’s already pretrained as an LLM.”

10. Lean and RLVR may eat mechanical math; the Gap Map says the gaps could cost a few billion

Lean’s investable property is that correctness becomes a reward: “if Lean says it’s correct, it’s just correct,” which makes formalized proving “a perfect RLVR task.” His call is strongly optimistic about the mechanical half — “I think that this problem of RLVRing the crap out of math proving is going to work,” with at least one billion-dollar-valuation company, Harmonic, based on this, and AlphaProof on the same thesis. What remains is conjecturing, “the conceptual organization of math of what is interesting.”
The open research question, pointing at a Bengio paper: is there a loss function for a good conjecture? A great theorem “compresses lots of information and has lots of implications for lots of other theorems” — Dwarkesh names it, “the Kolmogorov complexity of this statement,” and Adam extends it: “if you add this, how does it affect the complexity of the rest of the network of proofs?”
The commercial spillover he is actively incubating an FRO around is formally verified software — provably unhackable memory boundaries, of the kind DARPA demonstrated with a provably secure quadcopter — and the blocker is not proving power but the specification problem: “The power grid engineers just made this thing, but they don’t necessarily know how to lift the formal spec from that.” Dwarkesh pushes the sci-fi version, citing Terence Tao’s tweet that “these models are like automated cleverness but not automated intelligence”: if you can filter cleverness with proofs, make everything provable so billions of AIs can build on each other and check “are you trying to socially influence me?” — the davidad/ARIA safeguarded-AI design. Adam: “that’s an interesting vision. I don’t know in the next 10 years whether that will be the vision that plays out.”
The Gap Map — the inventory behind Convergent Research’s FROs — is his closing surprise, and it’s a sizing argument. It’s “a few hundred fundamental capabilities.” Even if each were a Series A, “It’s not like a trillion dollars to solve these gaps.” The unexpected entry was math: “mathematicians I thought just needed whiteboards, but they actually need Lean.” Some gaps show up even in well-funded astronomy, because above a critical project size you inherit “a more bureaucratic process with the federal agencies.”
The one idea he offers for AI labs, credited to a Gwern post: auxiliary loss on brain activity. Alongside the “cat” or “dog” label, predict “what is my neural activity pattern when I see a cat” — regularization that forces representations to be consistent with the brain’s geometry, possibly buying robustness to adversarial examples. The blocker is purely instrumental: “If every iPhone was also a brain scanner, you would not have this problem and we would be training AI with the brain signals. It’s just the order in which technology has developed is that we got GPUs before we got portable brain scanners.”

What's The Trade For 2026?[State of RL/Reasoning] IMO/IOI Gold, OpenAI o3/GPT-5, and Cursor Composer — Ashvin Nair, Cursor