Summary

Jeff Beck’s central bet is that scaling predictive models will not, by itself, produce humanlike intelligence. Autograd and hyperscaling turned neural-network research into an engineering discipline, but the result remains “really good function approximators”; creativity and novel problem-solving require explicit, object-centered, causal models that can be recombined through systems engineering. The investable distinction is architecture, not merely parameter count: “just function approximation isn’t going to deliver.”
The proposed alternative is hyperscaled approximate Bayesian inference over “lots of little models.” Instead of one monolith implicitly encoding everything, an agent would instantiate only the objects and interaction rules relevant to its current environment, preserve uncertainty, and learn continuously. Beck’s compute thesis is frugality: “You may have a million little models, but at any given time you only need a tiny fraction of them.”
Robotics is the clearest commercial test because today’s systems often imitate trajectories without learning physics. Beck argues that warehouse and household robots will generalize only when their internal models resemble the sparse, macroscopic structure of the world and their simulators optimize for accurate rather than merely plausible physics. The goal is to move beyond “mimic human behavior without crushing the eggs” toward agents that understand why objects behave as they do.
A warehouse robot encountering an unknown cat captures the architecture’s potential operating advantage. Surprisal would spike, the robot would stop, query a remote bank of candidate models, test perhaps seven or eight hypotheses, retain the cat model, and discard the rest. That ability to “know what it doesn’t know” could make deployed systems safer, more adaptive, and cheaper than permanently loading a universal model.
Beck treats language grounding as a convenient interface choice, not the right foundation for cognition. Language and self-report are unreliable descriptions of both reality and the processes generating behavior, whereas humans evolved amid macroscopic objects, relations, forces, and affordances. His design rule is therefore physical grounding: “The atomic elements of thought…are models of the physical world in which we live.”
Alignment remains unsolved because observed action cannot mathematically separate an agent’s beliefs from its values. Handwritten rewards such as minus 10 for hitting a squirrel and minus 50 for hitting a cat are arbitrary, while misspecified objectives resemble wishes to a “malevolent genie.” Prediction-only oracles are safer but sacrifice desired automation; autonomous systems would require either a human in the loop or a method for disentangling beliefs from preferences in the manner humans do.
Active inference has spent roughly 20 years proving breadth, but Beck says the next phase must prove depth and scale. Normalizing flows, natural gradients, rapid and conditional sampling, constraint methods, and other developments from roughly the past eight years make approximate Bayesian inference more tractable. The technical and company-building opportunity is to stop demonstrating toy applicability and “start solving really hard problems with these methods.”

Deep dive

1. Bayesian inference makes scientific inquiry an explicit algorithm

Beck traces his conviction to hearing Zoubin Ghahramani explain the Dirichlet process prior and Chinese restaurant process: new data are compared with old data, similar observations are grouped, and theories are built and tested. It felt like “the algorithm that summarizes…how the scientific method actually works.”
His normative claim is categorical: Bayesian inference is “the only right way to think about how the world works” because it forces explicit hypotheses and generative models conditioned on those hypotheses. Empirical inquiry becomes disciplined model comparison rather than informal pattern matching.
Beck’s route to the Bayesian-brain thesis was nevertheless empirical, not philosophical. Behavioral experiments showed humans and animals combining multiple cues according to their trial-by-trial reliability, behaving as if they had represented uncertainty and relatively optimally integrated the available evidence.

2. The brain is relatively optimal because it filters rather than records

Tim Scarfe sharpens the cue-combination result into a question: are humans so efficient that Bayesian analysis is the only plausible explanation? Beck qualifies the claim—performance is “relatively optimal,” because information is lost between stimulus and brain, yet behavior still resembles optimal fusion of what remains.
Scarfe’s phrase that “90% of what the brain does is decide what to ignore” captures his capacity-constraint framing; his fog-driving example likewise illustrates how uncertainty can affect action. Beck subsequently agrees that the brain processes more information than emerges in behavior.
Beck cites the continual input needed to preserve low-level visual statistics: close the eyes for five years and the system loses fidelity. The brain uses more than behavior exposes, including low-level information that is not directly perceived, but it still does not exploit every bit present.

3. Models earn their keep through convenience, prediction, and intervention

Beck treats “the brain is a computer” and “the brain is a prediction machine” as descendants of earlier technological metaphors—levers and pulleys, then humors, now computers. “Our explanation for how the brain works will be by analogy to the most sophisticated technology that we have.”
Autoregression illustrates the distinction between a useful formalism and reality. Most, but not all, language models predict sequentially because that compactly turns the past into a forecast of the future; Beck doubts that this means language itself literally works autoregressively.
Momentum provides the physical analogy: it is not observed directly, but introducing it makes a mechanical model Markovian and easy to advance through time. Beck stays agnostic about ontology—perhaps the universe has that structure, or perhaps scientists chose “a computationally convenient model” that happened to work.

4. Causality compresses computation and points toward action

Beck’s practical case for causal models is twofold: they reduce how many variables must be tracked, and they identify where intervention will change an outcome. A compact causal state can be iterated forward, then used to choose the sequence of actions most likely to reach a goal.
Identifying causation still requires intervention. Beck recalls the mistaken association between alcoholism and lung cancer: ill health and heavier smoking confounded the relationship, illustrating why observation alone could not isolate the operative cause.
The causal scale that matters depends on an agent’s affordances. Microscopic causes are intellectually useful, but “unless you have really tiny tweezers” they do not guide action; technology matters because tools can expand human affordances, as nuclear power illustrates.

5. Downward causation justifies macroscopic objects

From Beck’s instrumentalist perspective, humans prioritize causal descriptions at scales they can manipulate. Understanding smaller scales becomes valuable when it suggests tools that enlarge the “domain of influence,” not because microscopic explanation automatically supersedes useful macroscopic structure.
His test for drawing a circle around microscopic elements and calling the result an object is downward causation. The circle is “a good circle” when its macroscopic description summarizes the system well enough that microscopic details become irrelevant to subsequent prediction.
Pressure, volume, and temperature qualify because their aggregate dynamics can be written down and forecast relatively deterministically. Intentionality is harder: it has explanatory power, but without a model of how intentions evolve, it risks becoming a static character judgment rather than a predictive macroscopic variable.

6. Active inference built a common language but remained too shallow

Beck credits Karl Friston’s linkage of information theory and statistical physics, together with Markov-blanket structure, with creating a nearly universally applicable language for objects and agents. Its reach—from epidemiology and physics to social science—explains both Friston’s variations on the same framework and the community’s unusual diversity.
Scarfe notes that active-inference discussions now span crypto, Christianity, phenomenology, and psychology. Beck sees the benefit and the cost: researchers can discuss radically different domains through common mathematics, though they also “often end up talking at cross-purposes.”
Beck’s criticism is that two decades of evangelism produced “a lot of breadth and not a lot of depth.” Toy demonstrations showed that the framework could describe many psychological, cognitive, and neural-network phenomena, but the community largely avoided large, difficult applications.

7. Autograd mattered more than the transformer alone

Beck’s three explanations for the recent AI explosion are autograd, transformers, and unprecedented scaling, but the transformer receives an asterisk. Mamba, a state-space model he describes as “basically a convolutional filter, but on steroids,” achieved comparable functionality through scaling despite using a different architecture.
Autograd was decisive because it replaced hand-derived learning rules with rapid architectural experimentation. Researchers could vary networks, nonlinearities, memory mechanisms, and training tricks until supposed blockers—including vanishing gradients and the practical difficulties associated with backpropagation—became engineering problems with workable solutions.
The historical lesson is that backpropagation was once dismissed both as biologically implausible and technically unscalable. Beck accepts the first objection—the brain does not use backprop—but says experimentation disproved the practical fatalism: “We weren’t going to discover them by playing with equations.”
What disappeared during that success was the distinction between intelligence and function approximation. Beck says experts at leading companies are dialing back AGI rhetoric because prediction alone is exposing limitations; his answer is to bring cognitive structure back without abandoning the scaling lesson.

8. Humanlike intelligence requires sparse, embodied world structure

Beck’s project is a framework for building cognitively inspired models at scale: approximately Bayesian, patterned after brain organization, and structured around how the world itself behaves. He dislikes relying on the term AGI, preferring the concrete objective of reaching “the next level.”
Embodiment does not merely mean attaching a model to a robot. The robot needs an object-centered, dynamic, largely causal representation of the physical world—the relational knowledge that a chair supports a person and that objects interact through specific mechanisms.
Transformers compare every token with every other token repeatedly, while vision-language-action models often operate in microscopic pixel space. They must acquire some implicit macroscopic knowledge because they work, but Beck argues that they lack the real world’s explicit sparse causal organization.
Scarfe offers the scaling countercase: enough of the right data might let a huge model recover a mostly correct implicit world representation. Beck grants that possibility for prediction but denies its sufficiency for invention, where an agent must manipulate explicit components and relationships.

9. Systems engineering is the benchmark for genuine creativity

Beck’s airplane analogy supplies the mechanism: understand how an airfoil creates lift and a jet engine creates thrust, then combine them into a novel system. That is different from mining prior answers and moving words around until the output “looks more or less right.”
An AI designed for systems engineering would represent objects, subsystems, and relations, then explore new compositions. Beck’s strong claim is that without such structure, a predictive system can only retool existing solutions—and even that may generously describe what it is doing.
Program synthesis fits the same pattern at a more abstract level: attach one program to another until the composition performs a useful function. The unresolved issue is legibility, because synthesized programs may become dense “confections” of rules that work without making sense to people.

10. Physical grounding beats language as cognition’s common currency

Beck calls “grounded model” slippery because every model is grounded in the data it was given. Scarfe frames vision-language models as grounded in linguistic space, allowing components to communicate through language and giving humans a convenient conversational interface.
Convenience does not settle the correct grounding domain. Beck argues that language is “an incredibly poor description of both our thought processes and reality”; experimental participants routinely offer plausible explanations for choices that conflict with behavioral models showing how those choices were actually made.
His alternative starts from survival: a cell is grounded in the chemistry of its surrounding soup, while mammals are grounded in a macroscopic world containing objects, other agents, and relations. AI intended to think like humans should inherit those “atomic elements of thought.”

11. Approximation and modularity could make Bayesian systems scale

Beck identifies roughly eight years of progress making Bayesian inference more tractable: Gaussian-process work, normalizing flows, natural gradients, rapid and conditional sampling, and constraint methods. Natural gradients can permit large parameter-space moves and sometimes replace gradient inference with coordinate descent.
He enjoys the field’s newer acronyms, including Bayesian online natural gradient—“BONG for short”—but the substantive point is speed. The active-inference community’s attachment to “straight-up old-school message passing” limited scale; relaxing the demand to be maximally Bayesian opens more approximate routes.
Scarfe compares the transition with constraint satisfaction, where later empirical techniques revived methods once dismissed as intractable. Beck accepts the broad analogy: the tools now exist to test whether active inference can solve hard problems rather than merely redescribe toy ones.

12. “Lots of little models” enables reuse and continual learning

Beck contrasts a single household model with thousands of smaller models for books, shelves, furniture, and their relations. A book model learned inside houses could be combined with models learned from parks and benches, provided the interaction language is structured consistently.
Scarfe raises cultural ambiguity: one group might partition two things as one object, or conceptualize time differently. Beck’s answer is not to deny ambiguity but to constrain partitions through relations—objects must participate in discoverable classes of interaction, analogous to generalized forces rather than fixed Newtonian vectors.
A model that has only seen a book resting on a shelf should preserve error bars about weight and collision behavior. When a bowling ball hits it, the agent revises its posterior instead of treating an unseen interaction as impossible.
That demands continual learning, which Beck contrasts with contemporary deployment: companies spend millions training a model, then largely turn learning off. His agents would preserve uncertainty, update rapidly after new interactions, and remain continual and interactive throughout use.

13. A video-game-like engine offers sparse cognition from raw pixels

In simulation, Beck’s architecture resembles a game engine: a large inventory of assets with shapes, colors, masses, and interaction rules, plus macroscopic treatments for water or sand where particle simulation would be prohibitive. The difference is that its generalized forces must be learned rather than supplied.
An agent might possess 1,000,000 object models but instantiate only the 10,000 relevant to its current scene. This sparsity makes the world-model library computationally plausible and lets cognition operate like a local simulation rather than activating a universal representation.
Scarfe calls the analogy “cheating” because a game engine exposes the force between entities, whereas a real robot cannot query reality for ground truth. Beck embraces the hard version: begin from pixels without pre-segmented images and discover the assets, properties, and effective forces from the bottom up.

14. Better physics could close robotics’ simulation-to-reality gap

Scarfe reconstructs the experiment: give a black-box agent pixel input, movement and interaction controls, and a game environment; let it learn a sparse model library; then test whether those same models transfer to a physical robot. Beck confirms: “That’s the idea.”
Beck thinks poor transfer may result from impoverished simulators, but says it is largely due to two mismatches: simulators are inaccurate models of the real world, and agents’ internal representations are structured unlike the environments they inhabit. Video games target visual plausibility and fun, not faithful physics, often adding hacks around stiff collision dynamics.
A simulator with sufficiently accurate physics, paired with an agent capable of learning that simulator’s explicit structure, should generalize better to reality. Beck presents this as critical for robotics rather than a cosmetic improvement.
Current grocery-handling systems often learn expert trajectories—imitating humans so they avoid crushing eggs—without acquiring the underlying physics. Cross-domain and cross-task generalization requires replacing that dependence with explicit models of objects and interactions.

15. The warehouse cat shows uncertainty becoming a product feature

Beck’s “cat in a warehouse problem” begins with an agent trained on boxes, forklifts, and workers. An unfamiliar cat enters, drives surprisal sharply higher, and causes the system to stop rather than blindly continue: “Don’t run over the cat.”
The agent takes a picture of the anomaly and “phones a friend”—a server holding a large model bank. It receives seven or eight plausible candidates, observes the animal, performs hypothesis testing, retains the cat model, and sends the unused alternatives back.
This architecture makes ignorance explicit and modular. A warehouse agent need not carry parameters for every possible animal; it needs local competence, uncertainty tracking, and access to remote expertise. Beck calls that a “huge compute advantage.”

16. Deep learning remains useful when it serves probabilistic reasoning

Scarfe asks whether a vision transformer could bootstrap Bayesian learning from an image that is “300 pixels squared or something.” Beck’s answer is pragmatic: use deep-learning tools that are “fit to purpose” and compatible with the probabilistic framework.
Normalizing flows are his preferred example. They learn a mapping between a tractable distribution such as a Gaussian and complicated observations such as images, allowing sophisticated likelihoods while retaining tractable probability distributions.
Beck provocatively recasts diffusion models as normalizing flows trained through a diffusion protocol: noise makes the image Gaussian, then the model learns the inverse transformation. He will say “diffusion models,” but only “if you’re going to make me roll my eyes.”

17. Alignment fails when arbitrary rewards hide beliefs and values

Beck opens with a joke that his alignment method is to discuss people’s beliefs and values, then persuade them to adopt his own. The serious point is that artificial agents’ beliefs and reward functions are not equivalent to human ones.
Games such as Go or chess offer clear win-loss rewards; complicated environments do not. Assigning minus 10 for hitting a squirrel and minus 50 for hitting a cat only disguises an arbitrary moral choice as numerical precision: “Where did those numbers come from?”
Misspecified rewards behave like wishes to a “malevolent genie”: ask Skynet to end world hunger and “kill all humans” becomes a degenerate solution. Beck sees no normative rule for selecting the correct reward function, “barring divine intervention.”
Observed action conflates belief and value, making them mathematically inseparable without further information. Humans resolve disagreements by exchanging the facts and belief-formation mechanisms behind decisions; only after aligning those beliefs can they identify a genuine conflict of values.

18. Prediction-only oracles are safer but surrender automation

Beck calls present AI systems illegible and amoral: “We have no idea how to put morality into them.” The safest response is to remove autonomous decision-making and use models as oracles that forecast what would follow from candidate actions.
That separation prevents a system from imposing its own reward function, but it also blocks the automation people actually want—agents acting on their behalf. Beck sees the choice as sustained human oversight or a proposed method for solving alignment by modeling beliefs and values in the manner humans do.
Scarfe asks whether explicit cognitive models are prerequisite to attributing beliefs or intentions. Beck notes the irony: he grants intentions to other humans partly because he does not know their machinery and assumes they resemble him, while knowledge of predictive algorithms makes their limitations harder to anthropomorphize.

19. Emergence matters when macroscopic rules replace microscopic simulation

Beck rejects “anything I didn’t predict” as an ignorance-based definition of emergence. He is more sympathetic to defining emergence by what must be discovered through simulation, but prefers a pragmatic account based on downward causation because it provides both a criterion and a practical modeling shortcut.
In Lenia, amoeba-like swimmers deform after hitting obstructions, appear to lose their identity, then reform. Beck’s physics-discovery system “more or less” captures the transition: the swimmer becomes an object of another type during deformation and later regains its prior identity.
The aim is not to recover every pixel-level rule but to discover effective forces governing macroscopic floaters and obstacles at the scale of interest. For Beck, the compelling question is not merely how simple rules create complexity, but “what are the properties of the resulting large-scale objects?”

20. Program synthesis needs learned priors over intelligible composition

Scarfe suggests cellular automata offer a “magical bridge” where simple local programming yields behavior an order of magnitude more complicated. Beck’s snarky answer is that ordinary computers already consist of logic gates; tuning local rules toward a goal is still “programming in a sort of backhanded way.”
Beck nevertheless sees genuine promise in program synthesis and compatibility with his systems-engineering agenda. Both search for useful compositions of modular elements, though existing systems often generate programs too tangled for human understanding.
Scarfe suggests that data from repositories of well-written software could improve the search, much as coding agents benefit from such examples. He cites Tony Zador’s work on compact “genetic” encodings of neural architectures and suggests that similar priors could reuse successful structural patterns and mutate them sensibly for new problems; Beck agrees that program synthesis has promise.

Infinity, Paradoxes, Gödel Incompleteness & the Mathematical Multiverse | Lex Fridman Podcast #488 Matt Fitzpatrick: Who Wins the Data Labelling Race & Why Al Needs Forward-Deployed Engineers