Summary

Scarfe’s central call is that successful brain models are useful simplifications, not revelations of what brains literally are. Hydraulic pumps, telegraphs, telephone switchboards, computers and now free-energy minimizers each mirror their era’s most impressive technology. The recurring error is Whitehead’s “fallacy of misplaced concreteness”: forgetting that the map was built for a purpose and promoting it into the territory. Chirimuuta adds that, for applied science, there is no in-principle problem with oversimplification if it achieves the technological goal.
The commercial power of today’s AI does not prove that AGI is inevitable or that cognition is computation. Scarfe says Claude Code has produced more interesting software-development progress in six months than the prior 20 years, yet calls it “automation technology” whose value remains bounded by the user’s ability to specify, supervise and delegate. He frames confidence in biological-like AI as potentially a “cultural-historical illusion” inherited from mechanistic thinking, rather than a conclusion about how minds work.
Prediction and control can work while understanding—and therefore failure boundaries—remain unresolved. John Jumper separates prediction, control and understanding: the last is almost achieved when a small, human-communicable collection of facts fits “on an index card.” His examples expand the searchable field from 200,000 experimental structures to 200 million predicted ones, but “it doesn’t do the act of understanding for us”; black boxes may work until they break without warning.
Joscha Bach’s claim that “software is spirit” exposes the episode’s deepest disagreement over AI substrates. Bach treats algorithms and money as causally powerful patterns invariant across physical implementations, with the brain acting like a Minecraft-running computer that insulates imagined worlds from their surroundings. Scarfe’s pushback is that humans identify the supposed sameness: different chips perform physically different events, while money’s power resides in social agreement rather than paper or electrons. Scarfe also invokes Anna Ciaunica’s mountain analogy against functionalism: the path and physical instantiation may matter, not only the final output.
The internet and foundation models do not provide a perspective-free repository of knowledge. Chirimuuta argues that knowledge becomes achievable only when finite communities narrow their questions, tools and possibilities; an LLM’s “every-person voice” lacks the situated socialization needed for an honest, trustworthy perspective. Claude Opus 4.5 may appear authoritative, but Scarfe’s framing is that its knowing was “ours all along”—compressed and reflected back through silicon.
The free energy principle is valuable precisely if users resist treating its elegance as literal ontology. Friston presents it as an “almost logically simple” principle of least action over conditional probability densities; Scarfe calls it the ultimate “spherical cow.” The disciplined question is not whether it finally captures the brain, but “what does this help us do,” what does it illuminate and what does it leave dark?

Deep dive

1. Simplicity buys leverage, not metaphysical truth

Scarfe begins with young Carl Friston watching woodlice slow in sunlight and accelerate in shade, an observation that eventually fed the free energy principle: one mathematical quantity meant to encompass perception, action, learning and behavior. Friston describes it as “almost logically simple,” essentially a principle of least action governing conditional-density dynamics.
The provocation is that this may be the ultimate “spherical cow”: an intentionally emaciated account of self-organization whose very generality approaches tautology. Scientists must omit detail because human working memory, attention and lifespans are finite; the unresolved question is why those omissions work and what success licenses us to claim about reality. Chirimuuta distinguishes this from curiosity-driven science: if a simplification achieves an applied technological goal, she sees no in-principle problem with oversimplifying.
Marta Halina gives science a humanistic purpose: making the universe intelligible and meaningful to us, rather than essentially controlling, predicting or exploiting it, even though science can enable all three.
Scarfe stages the dispute as Simplicius versus Ignorantio. Simplicius reads elegant laws as evidence that nature is fundamentally orderly; Ignorantio sees models as purpose-built approximations and embraces Nicholas of Cusa’s “learned ignorance”—knowledge that includes awareness of what remains unknown.
François Chollet’s kaleidoscope hypothesis becomes the clean specimen: surface complexity may arise through repetition and composition of a few “atoms of meaning” that intelligence extracts as abstractions. Chirimuuta does not call it wrong; she calls it a philosophical bet, akin to Plato’s wager that messy appearances conceal a neat, mathematically decomposable reality.

2. The computer metaphor hardened into an ontology

Descartes compared the nervous system to hydraulic automata; later eras reached for telegraphs and telephone switchboards. McCulloch and Pitts used logic gates as a functional analogy for neurons, but contemporary neuroscience often drops the “like” and says the brain is a computer—the metaphor becoming “the thing itself.”
Bach’s strongest case rests on causal invariance. Money persists through paper, coins, gold and digital ledgers; software can run across chips and perhaps neurons. A computer is a “causal insulator,” allowing Minecraft’s world to ignore casing color, voltage and CPU, while a brain similarly hosts memories and possible futures independent of the immediate present. Bach accepts physical causal closure but treats abstract and physical descriptions as two real, irreducible views of the same causal structure. Hence his categorical conclusion: “Software is spirit.”
Scarfe’s objection is that cross-substrate sameness may exist in human description rather than nature. Money only acts through interpretive social practices, and different hardware entails genuinely different physical events. The temperature analogy adds a constraint: knowledge need not be a separate substance, but like temperature it always requires physical embodiment—“it’s not storing it in nothingness.”
Anna Ciaunica’s mountain analogy supplies a further challenge to functionalism: reaching the summit does not make the first steps irrelevant. Scarfe uses it in his debate with Mike Israel to argue that the path and physical instantiation may matter, not only intelligent-looking outputs; a helicopter can climb a mountain better without abstract reasoning and planning.

3. Models answer purposes, not absolute questions

Luciano Floridi distinguishes reality as a system from ontology as a model of that system: “the music of the radio is not about the radio, but there is a radio.” Digital technology can re-ontologize the experienced world without revealing its metaphysical foundation.
His building example makes the relational point concrete. It is the same building for giving directions, but not if its function changed from school to hospital; a reconstructed Ship of Theseus remains the same to the tax collector and becomes worthless to a collector.
Accordingly, “is the universe a gigantic computer?” is meaningless as an absolute question but useful when modeling digital life. Floridi calls humans “informational organisms” for a 21st-century purpose—not as final metaphysics. A defensible answer requires the question, its purpose and the chosen model or level of abstraction.
That distinction punctures AGI inevitability without denying AI’s utility. Scarfe calls Claude Code “genuinely amazing” yet still automation, not intelligence. He links the sense that biological-like AI is preordained to a long mechanistic history; if that mechanistic hypothesis fails, inevitability loses its foundation.

4. Prediction can outrun understanding

Jumper’s tripartite test is operational: prediction forecasts a future value; control makes that value come out 17; understanding is almost having a small collection of facts one human can communicate to another in compact form. Machines can predict and perhaps control, while humans must still derive the compact explanation.
Modern predictive models create a real scientific tension. Jumper’s example expands the searchable field from 200 million predicted structures rather than only 200,000 experimental structures, while LLMs and neural response models sacrifice the mathematical legibility earlier scientists sought. Scarfe notes that GPT-5.2 had apparently solved one problem from Terence Tao’s website, but performance alone does not settle understanding.
Chomsky’s counterexample is deliberately brutal: a two-word theory—“Anything goes”—accommodates every known and future law yet explains nothing. A theory must answer both “Why are things this way?” and “Why are things not that way?” On that standard, he says, “GPT-3 has done nothing.”

5. Knowledge is embodied, situated and cognitively bounded

Marta Chirimuuta rejects knowledge as a detachable object: a book is an archival record of ideas, not knowledge itself. Throwing engineering manuals and cement into a gorge will not produce a bridge; knowledge “can only go to work when it’s embodied” in teams, organizations and communities.
Chirimuuta extends that argument against a universal, perspective-free internet. Inquiry succeeds by narrowing possibilities from a particular place and community. LLMs aspire to an “every-person voice,” but their lack of finite socialization makes it difficult to locate an honest, trustworthy standpoint.
Her “haptic realism” casts scientific knowledge as more touch-like than detached vision: scientists run into, manipulate and change what they study. For Scarfe, the patterns that emerge are real yet partly shaped by experimental contact. Nature resembles Proteus—temporarily pinned down to answer one question, then opening onto other perspectives.
Chomsky’s “cognitive horizon” supplies the limit: a rat can learn complex mazes but never “turn right at every prime number” because it lacks the concept. Humans may face analogous walls. Scarfe therefore closes without nihilism: use Friston’s framework and foundation models, but remember that the brain is not a hydraulic pump, computer or telephone network, and is probably not a literal free-energy minimizer either.

Why AI Has a Plato Problem — Mazviita Chirimuuta A Call to Action for TSMC's AI Customers, Plus Netflix Anxiety