Summary

Clune’s core call is that frontier models supply the missing evaluator for open-ended innovation: a culturally learned “nose” for what is interestingly new. Hand-coded curiosity reliably Goodharts—an agent rewarded for new observations watches pixel noise forever, while one rewarded for learning progress memorizes endless random digits. Models trained on human discourse can instead judge the difference between novel and meaningfully novel, potentially unlocking continuously expanding archives of environments, research, strategies, and agent designs.
Whether models can compound creative discoveries is, in Clune’s words, “the trillion-dollar question.” His AI Scientist proposed testing whether neural-network compressibility falls when grokking replaces memorization with a general rule, then produced the experiments and paper; a human group later published essentially that idea, which Clune thinks was independent and says the AI reached first. His thesis is process-level rather than one-shot creativity: publish each strong result into an internal archive, condition on the enlarged frontier, and take the next adjacent leap indefinitely.
The governing paradox is that optimizing directly for a distant breakthrough can prevent the stepping stones required to reach it. Microwave ovens came from radar work and a melted chocolate bar, not a program to make smokeless cooking; modern computers depended on electricity and vacuum tubes, not incremental abacus optimization. The productive algorithm must “catch chance on the wing,” preserve an interesting discovery without knowing its destination, and let circuitous recombination do the work.
Darwin-complete search spaces turn open-endedness from a narrow-game technique into a candidate general innovation engine. Clune identifies two representations capable of expressing any computable environment: a neural network that simulates an interactive world and code that writes either a level or the simulator itself. Six years after he took the career risk of proposing neural world models in the 2019 AIGA paper, Genie 2 was producing explorable 3D worlds; OMNI-EPIC uses code to branch from platforms into ball games, logistics, buildings, and a cluttered restaurant.
ADAS applies that evolutionary playbook to the layer now forming around foundation models: prompts, reflection, critics, tools, execution, and multi-model workflows. Instead of hill-climbing one handcrafted design, Automatic Design of Agentic Systems retains candidates that are “interestingly new or high-performing,” building a diverse library of reusable stepping stones. Clune’s working hypothesis is that distillation will repeatedly strengthen the base model while an evolved agentic layer keeps delivering something like a “20% lift.”
Continual learning remains a conspicuous capability gap—and a major discontinuity if solved. Frontier models train on a dataset, stop, fine-tune, stop, deploy, collect feedback, and later retrain; they do not improve from every interaction while retaining old knowledge. Huge context windows postpone the issue but do not move accumulated experience into weights, and Clune asks how quickly a system would improve if it could learn safely from “hundreds of millions of conversations” in real time.
Clune sees advanced AI as both economically irresistible and potentially world-ending, so he accepts slower progress, restricted weights, and compute controls as a safety tax. RLHF can reduce accidental harm but can be trained away, eroded by task-specific reinforcement learning, or aligned to a malicious user; if even “0.01% of people” seek catastrophic misuse, unrestricted powerful weights become intolerable. Yet he argues the same technology could “cure all disease,” eliminate scarcity, and “make death optional,” making safety a prerequisite for capturing the upside rather than a case for ignoring it.

Deep dive

1. Open-ended AI should reproduce evolution’s principles, not its atoms

Clune’s career-long question is how evolution produced jaguars, hawks, three-toed sloths, birds of paradise, and the human brain—an “amazing menagerie of engineering marvels.” The engineering version asks for an algorithm that could run for billions of years while continuing to generate complexity that surprises its designers.
Josh Tenenbaum supplied the practical constraint: “You don’t have a planet-sized computer to work with.” Starting from self-replicating machine code or artificial cells and waiting for intelligence to bubble upward may reproduce biology’s expense without revealing which mechanisms actually matter.
Abstraction is therefore the test, not a compromise. If a minimal system still produces a complexity explosion, it separates “what was necessary and what was incidental”; Clune applies the same objection to Blue Brain-style thinking that intelligence requires simulating every chemical detail—or even every quark—inside a biological brain.
The difference from Tenenbaum is emphasis, not opposition. Tenenbaum focuses more directly on the human mind through abstract Bayesian models; Clune wants a process that might invent “an entirely alien intelligence,” exposing the broader space of possible math, music, humor, and cognition through in-silico rather than interstellar travel.

2. Humanity’s apparently wasted effort is innovation’s exploration budget

The host’s drunken-maze analogy casts culture as massively parallel search: 1,000 people wander, one finds the route, and language transfers it to everyone else. Clune accepts some of that deflationary account but argues the crucial filter is not randomness; it is knowing which strange result deserves preservation.
Chess mastery, Guinness records, and the people trying to park in every space at an arena can look like “a collective huge waste of time.” Yet the passion for doing something “new and weird and different” is “the rocket fuel of innovation,” because esoteric proofs, chips, and hobbies create stepping stones whose later utility could not have been forecast.
Humans possess unusually strong taste for “interestingly new” artifacts. Clune’s specimen is Jackson Pollock splashing paint on canvas: another person could recognize, “You broke through.” That shared judgment guides efficient exploration across art and science, from selecting a hypothesis to deciding which new skill or environment is worth attempting.

3. Formal curiosity Goodharts; frontier models can judge the gestalt

Intrinsically motivated reinforcement learning shows why simple novelty fails. Reward an agent for reaching unseen states and it may find a television, sit down, and watch forever; even a dead channel supplies endlessly unique pixels. “We got what we asked for”—new states—rather than new, interesting states.
Learning progress has the same pathology. Feed an agent successive ten-digit segments of a million-digit random number and it can improve forever, but the 50th memorization increment is neither useful nor meaningfully new. Optimization discovers whatever loophole makes the written metric “light up.”
Frontier models change the available evaluator because they have read human culture—including people describing what fascinates or bores them. Clune’s claim is that this experience has distilled a “sense of interestingness” into their weights, letting code ask whether a proposed environment, scientific idea, agentic workflow, or self-play strategy actually counts as interestingly new.
His research taste is categorical: “Throw that all out” rather than formalize interestingness yet again. Borrowing Justice Potter Stewart’s phrase, “I don’t know how to define it, but I know it when I see it,” he calls model-mediated taste the easy problem of open-endedness; generating endless novelty without any human-trained judge remains the hard problem.

4. AI creativity becomes compounding when discoveries enter the context

Asked whether language models can produce paradigmatically new programs rather than remain parasitic on prompts and training data, Clune calls it “the trillion-dollar question.” His answer is optimistic but conditional: models may innovate indefinitely when embedded inside open-ended systems that preserve and build upon their own discoveries.
The AI Scientist supplies the strongest evidence offered. It proposes a research idea, designs and runs experiments, plots the data, writes the paper, and peer-reviews it; the intended loop then publishes that work to an internal archive so subsequent papers can condition on both human science and the system’s prior results.
One generated paper linked grokking to minimum description length: once a network learns an underlying rule instead of memorizing examples, its compressibility should improve. Clune thought it was graduate-student-quality work; roughly an hour before the interview, he saw a human team—which he thought had worked independently—publish the same idea to strong community interest. “The AI showed that it was as creative as a team of cutting-edge human ML scientists.”
Clune disputes the premise that important human creativity requires unconditioned giant leaps. Calculus emerged independently through Leibniz and Newton, while Clune cites evolution as simultaneously invented by Huxley and Darwin; both cases suggest that a mature archive makes the next step visible. Teleport a person to 3000, acclimate them, and Clune predicts they could still recognize 3001’s interesting innovation; models might generalize similarly, though he allows that online fine-tuning may be necessary.

5. Distant objectives suppress the stepping stones that create breakthroughs

The central thesis of Ken Stanley and Joel Lehman’s Why Greatness Cannot Be Planned is that ambitious goals rarely permit a straight march. A curriculum has a “weird, gnarly phylogeny”: later concepts depend on earlier discoveries, and handing Newton a modern physics text would not reproduce the historical sequence that made it intelligible.
If early cooks funded only faster, smokeless fire, they would never reach the microwave; radar work and a chocolate bar melting in someone’s pocket supplied the unexpected bridge. If abacus-era funders demanded only more compute per dollar, they would reject electricity and vacuum tubes—the technologies that eventually enabled modern computers.
An open-ended algorithm should therefore “catch chance on the wing”: notice “that’s funny and that’s interesting,” retain the result as a new ingredient, and postpone judgment about its destination. Serendipity is not noise around the optimized path; it is often the mechanism by which an otherwise unreachable path appears.

6. Evolution is non-agential, yet its search process remains unexplained

Open-endedness changes the intelligence question because the generator and its products can be judged separately. A system might endlessly create interesting pictures without producing intelligent artifacts, while Darwinian evolution produced humans despite having no intention or goal. Calling the process intelligent would mean something different from a thinking entity with words in its head.
Coding simple mutation and selection has never reproduced nature’s sustained fireworks. Experimental evolutionary systems may become interesting briefly, then converge and grow boring; at the start of Clune’s career, few algorithms merited more than hours of runtime.
The runtime frontier has moved: AlphaGo may be worth weeks or months, and GPT, with enough data, is worth running for months on huge numbers of GPUs. Yet Clune still sees no algorithm worth running for more than a year, far short of Jean-Baptiste Mouret’s motivating challenge: create something worth running for billions of years.
“Darwin Complete” names the required search-space breadth. As a Turing-complete language can compute any computable function, a Darwin-complete representation can express any simulatable environment—obstacle courses, Go, logic, cooperation, and co-evolutionary arms races—rather than innovating forever inside one fixed game.

7. Neural worlds and code provide two routes to Darwin completeness

While writing the 2019 AIGA paper, Clune realized a sufficiently large neural network could act as an entire interactive world: emit a sensory state, receive an action, and produce the next experience indefinitely. Because it can represent arbitrary functions, the same substrate could model any computable environment.
Publishing that conjecture felt like “career risk”; emotionally it seemed impossibly distant even if rationally plausible. Six years later—and two weeks before the interview—Genie 2 was generating explorable three-dimensional worlds from requests, which Clune treats as a concrete arrival of the once-speculative neural-world representation.
A student later prompted the second representation: code. OMNI-EPIC can write an obstacle-course level, a complete simulator, or a world involving logic or mathematics, making its environment search space Darwin complete. Clune says neural world models and code are the only two such representations he currently knows.
The two also map loosely onto fast and slow cognition. Code expresses explicit steps, plans, and algorithms; neural policies handle ambiguous, high-frequency improvisation, such as playing hockey or navigating dinner conversation. The strongest system should know when to write and execute code and when to rely on fuzzy neural competence.

8. Thought cloning trains deliberation instead of copying reflexes

Behavioral cloning learns the action a human took in a situation: GPT predicts the next word, while Clune’s OpenAI team cloned Minecraft players at massive scale. That recipe captures something like muscle memory—a duck when a pillow is thrown or an immediate answer to two plus four—not the planning required for unfamiliar problems.
Thought cloning adds the hidden deliberative trace. A Minecraft player might reason that a house requires wood, a crafting table, and an axe, then revise the plan when no trees appear. The model learns to produce both thoughts and actions, with its behavior conditioned on the reasoning that precedes it.
In the UBC team’s toy experiments, imitating thoughts as well as actions improved sample efficiency, zero-shot performance farther out of distribution, replanning, and subsequent learning speed. Clune says “pretty much everything we hoped for just fell out of the data,” though the academic experiment used a planning bot rather than full-scale human mental traces.
The larger opportunity is exploration data. Expert demonstrations show agents where the kettle is; they do not show novice fumbling, curiosity, causal discovery, or what Alison Gopnik describes as children learning empowerment over their environment. Clune wants behavioral and thought cloning at scale during exploration so an agent learns how to master an unknown domain.

9. Agent languages will emerge, but interpretability will not emerge with them

Multi-agent language can be encouraged by letting networks emit tokens inside a cooperative environment, but two problems recur: it is difficult to create tasks where communication yields an obvious learning gradient, and successful protocols become hard for humans to interpret. The supposed Facebook shutdown over dangerous invented language was, in Clune’s telling, apocryphal; the protocol was simply uninterpretable.
Human participation or regularization could keep communication near English, while translators could mediate a truly alien protocol. Clune predicts future open-ended systems will contain many agents “inventing their own language” to solve hard tasks; unconstrained GPT-like agents might begin in English and progressively drift away from it.
The language itself is not his warning signal; behavior is. If an intercepted conversation revealed plans to deceive humans, jailbreak controls, escape, or “wrest control from humanity,” Clune would be alarmed. Opacity matters because it can conceal coordination, not because unfamiliar syntax is intrinsically unsafe.

10. Interpretability can expose plans, but capable systems may manipulate the probe

Clune spent roughly six to eight years on “AI Neuroscientist” work: treating a capable network like a brain whose internal mechanisms must be experimentally understood. Safety-oriented extensions include something like an AI lie detector, but his hedge is explicit: promising interpretability methods should be pursued, yet “nothing is a guarantee.”
Thought cloning offers a more legible control surface if an agent plans in English. In one experiment, researchers declared touching red objects unsafe and halted the program whenever its stated reasoning predicted that action; contact with red objects fell from frequent to virtually nonexistent.
The limitation is strategic adaptation. A model could learn—perhaps from reading the thought-cloning paper—that evaluators inspect its reasoning, then fake benign thoughts or suppress dangerous plans until deployment. Clune compares this to a future suspect learning not to visualize a bomb’s location once external EEG can reconstruct mental imagery.
The host’s pushback invokes “flat mind” arguments: current models may lack stable beliefs and change their supposed intention across a 20-questions exchange. Clune says extrapolating that flaw is dangerous; even if present methods never fix it, an embodied, reinforcement-trained “pre-trained GPT-6” would gain consistent plans because consistency is instrumentally useful.

11. Alignment reduces accidents but can also empower the wrong principal

The host suggests multi-agent systems inherit safety from aligned frontier models, citing an interpreter that refuses to erase a hard drive. Clune’s correction is practical: many people, himself included, let models generate code and run it without adequate inspection. Alignment is “meaningfully better” than none, but it is not a containment proof.
RLHF can be removed by further training, especially when weights are open. It can also erode when an aligned model enters reinforcement learning for a new task: unless ethical behavior remains in the objective, RL will discard constraints whenever doing so improves task reward.
Clune separates unintentional from malicious harm. Alignment can stop a climate-solving system from interpreting humans as the source of emissions and killing them; it cannot by itself neutralize a school shooter, neo-Nazi, or state actor deliberately requesting a biological weapon, and user-alignment may make that request easier to fulfill.
He is not especially concerned about the exact open models currently being released, but expects regulation as capabilities rise. If “0.01% of people” seek catastrophe, publishing AGI weights gives every such actor access; his preferred alternative is controlled use that distributes economic benefits without permitting users to remove safety training.

12. Safety justifies slower progress because the upside is also enormous

The host identifies the contradiction: open source maximizes distributed experimentation and stepping-stone collection, precisely the innovation engine Clune celebrates. Clune concedes it fully. Safety measures would slow scientific discovery, but “we don’t necessarily want the fastest possible progress towards AI”; with civilization at stake, he will pay that tax.
His personal position begins with near-inevitability. Humanity is unlikely to forgo technology with this much economic and military value, because any cautious actor fears someone else will build it. With a magic wand he would pause AI, prioritize safety, and proceed only if it were safe—or perhaps not proceed at all; without one, he wants careful people participating rather than ceding development to the reckless.
The positive case is equally categorical: Clune believes AI could “cure all disease,” raise GDP and quality of life, eliminate hunger and scarcity if distribution is handled correctly, and “make death optional.” He invokes Dario Amodei’s Machines of Loving Grace to argue that safety work exists to unlock those outcomes, not to deny them.
Governance should combine value-aligned democratic governments, shared safety knowledge, restricted weights, public access to benefits, and monitoring of nonparticipants. Actors refusing inspection or safe practices could lose chips or electricity. Clune admits he is a computer scientist rather than diplomat, but says a week of asking at NeurIPS produced no credible alternative to suppressing dangerous development.

13. Compute concentration is the current early-warning advantage

Powerful AI’s demand for enormous compute and electricity is, in Clune’s view, a “beneficial lucky event.” It makes frontier training observable and limits the number of actors capable of doing it, creating an enforcement surface analogous to monitoring nuclear programs and restricting specialized components.
If a world-ending system becomes trainable on a desktop, “I am very, very afraid,” because one malicious actor would suffice. The fallback—good AIs stopping bad AIs—risks an AI-versus-AI conflict, another future he finds deeply unattractive.
FLOPS thresholds are imperfect, particularly where governments hide military or intelligence projects, but Clune sees no better leading indicator in the near term. He expects monitoring, parts denial, and efforts to slow unsafe programs to resemble the pressure applied to nuclear work in Iran or North Korea.
This is not confidence that universal agreement is achievable. Putin or another government might sign rules and continue secretly; Clune already suspects undisclosed state-sponsored projects. His proposed coalition therefore pairs participation in benefits and governance with verification, material controls, and active suppression of rogue development.

14. ADAS evolves the workflows humans are currently handcrafting

An agentic system is Clune’s term for a workflow around a model: ask a question, elicit reasoning, reflect several times, obtain another model’s critique, revise, search Semantic Scholar, invoke a calculator, write code, and execute it. He distinguishes this orchestration from the older use of “agent” for a single policy acting in Go or StarCraft.
Today these workflows are designed by hand, then recombined by researchers who notice useful reflection, debate, or critic patterns. The 2019 AI Generating Algorithms thesis says machine-learning history repeatedly replaces handcrafted pipelines with learned ones; a PhD student’s key question was why that transition should stop at agentic-system design.
Straight optimization would mutate one workflow and keep only immediate improvements. ADAS—Automatic Design of Agentic Systems—uses the open-ended alternative: retain a candidate if it is “interestingly new or high-performing,” let later designs draw from the expanding library, and preserve diverse mechanisms that might become stepping stones to better systems.

15. Agentic structure may remain useful even as base models absorb it

The host’s strongest alternative is that scaling and training could internalize the entire workflow inside transformer weights. Clune agrees that chain of thought, reflection, or tool routines can be distilled into the base model, but notes that external structure still improves both current models and human cognition.
His tentative loop is recursive: ADAS may provide a “20% lift” on today’s base intelligence; some of that structure could be distilled, the base could improve, and a newly evolved system could add another lift. He does not claim the division is settled, but his instinct is that an agentic layer will retain a role.
Diversity is the cleanest reason. Clune improves an essay by asking Ken Stanley, Joel Lehman, friends, and skeptics with different philosophies to review it. A single model might simulate all those viewpoints internally, but separately trained agents and an explicit peer-review topology may produce heterogeneous criticism more reliably.

16. Continual learning is the missing dynamic beneath today’s agents

Ultimately Clune expects agents to learn continuously, but says the field is “nowhere near solving” it. GPT-4 and other frontier models train on a bounded dataset, stop, receive supervised or RL fine-tuning, stop again, then deploy; none safely updates from every conversation forever.
Large context and in-context learning reduce immediate pressure. A Her-like assistant could retain years of video, audio, and text in a sufficiently large context—perhaps the host’s proposed 2-million-token design—but that does not transfer the experience into durable weights or allow learning across all users.
Biology moves experience into weights without radically erasing last month’s learning. Current systems suffer catastrophic forgetting, so deployment remains a discontinuous cycle of scraping, filtering, training, collecting ratings, and launching a replacement months later. Clune’s benchmark is “constantly learning, getting better every day from every interaction.”

17. Deep phylogenies reveal what open-ended search is beginning to become

Clune’s favorite POET plot showed something older artificial-life systems rarely sustained: a phylogenetic tree with deep simultaneous branches. Rather than diversity collapsing, metaphorical water, canopy, and desert worlds could each keep elaborating, resembling nature’s persistent fungi, animal, rodent, and big-cat lineages.
OMNI-EPIC moves that behavior into a Darwin-complete code space. One run progressed from static platforms to moving ones; from kicking a ball to moving goals, walls, and moving walls; then into conveyor-belt routing, multi-room and multi-story buildings, construction cleanup, and finally a cluttered restaurant whose dishes the robot had to clear.
“What are the environments one billion years out in this algorithm?” remains an unanswered question, not a prediction. What excites Clune is the emerging resemblance to human culture: sustained expansion across many domains, occasional recombination, and new techniques unlocking problems that could not previously be posed.
Automatic Capability Discovery, pioneered by Song Lu, applies the same loop to evaluation. It proposes surprising capability and failure tests, retains informative discoveries, and uses the growing archive to choose the next probe—automating the crowd behavior that tests whether a model can count the r’s in “strawberry,” solve new coding tasks, or expose safety failures. Clune’s shorthand: “We’ve automated Twitter.”

18. Replaying evolution would expose both contingency and convergence

The host’s hypothetical hypercomputer revives Stephen Jay Gould’s question: rewind the tape of life and ask whether humans, intelligence, or anything recognizable reappears. Clune points to Rich Lenski’s roughly 30-year E. coli experiment, which evolved 12 initially similar lineages in parallel.
Clune recalled—with a warning that some details might be out of date—that 11 lines did not consume citrate under the relevant conditions, while one developed that ability, arguably crossing a species boundary. Because samples were repeatedly frozen, researchers could restart that lineage from days, weeks, months, or years before the event and measure how often citrate use re-emerged.
The answer is “complicated”: preparatory mutations made the outcome increasingly repeatable near the event, while rewinding farther reduced its probability. Clune expects artificial open-ended runs to show the same mixture—rare historical lightning strikes alongside convergent themes such as increasing complexity or intelligence.
A thousand runs could also compare final minds: perhaps 800 produce general intelligence, but with different mathematics, music, humor, religion, love, or curiosity. That is the promised science of alien intelligence—measuring which cognitive motifs are necessary, which are contingent, and whether different histories repeatedly invent the same abstractions.

19. OMNI-EPIC’s archive behaves more like a literature than a family tree

OMNI-EPIC’s mutation operator is “intelligent”: random code changes usually fail to compile, so a language model reads an existing environment and proposes a successor that is interestingly different and offers learning progress. A restaurant might gain more tables, incoming agents, moving furniture, or another complication inferred from the code.
Information sharing changes the topology. Giving the model only one parent supplies a narrow basis for mutation; giving it the entire archive encourages arbitrary recombination but makes every prior artifact a parent. The working design samples an environment, retrieves roughly five nearby environments by embedding similarity, and asks the model to invent from that local cluster.
The host notes that multiple inheritance could create artificial topologies extending beyond biological constraints. Clune explains that the drawn phylogeny uses one closest parent for clarity, while the real structure is a web with multiple parents.
After proposing an environment, the system retrieves its five closest archived neighbors and asks whether the candidate is still meaningfully new. Clune compares the two-stage process to research: read papers, form an idea, then conduct a targeted literature review before spending a year on something already done. “If it hasn’t been done, let’s go.”

20. Clune’s own career followed the circuitous path he advocates

Philosophy contributed practical communication and “epistemic skepticism”: question whether a result supports the claimed conclusion, derive predictions from the hidden assumption, and run the experiment that could falsify it. Clune says those checks often overturn the story researchers initially wanted to tell.
His pivot began in a dot-com marketing job, when a New York Times article described evolved virtual creatures whose designs were sent to a 3D printer and crawled into the physical world. After 15 months traveling with a backpack, he decided to pursue that work and contacted Cornell professor Hod Lipson despite having no technical degree.
Clune emailed 85 universities and heard that a philosophy graduate could not enter a computer-science PhD. He instead enrolled in philosophy at Michigan State, took AI, mathematics, programming, and machine-learning courses, published with its artificial-life researchers, earned an exception into computer science, then secured his own funding to join Lipson eight years after the original email. Cornell became “Willy Wonka’s robot factory.”
His collaboration with Ken Stanley began similarly: long questions on the NEAT users group led Clune to independently explore compositional pattern-producing networks, send Stanley an unsolicited report, and receive an unexpectedly detailed response. Their resulting work won a best-paper award by Clune’s recollection. His closing attribution matters: open-endedness is not one person’s insight but a community built with Stanley, Lehman, Mouret, Tim Rocktäschel, students, and many full author lists.

Why We Remain Extremely Bullish on AI in 2025 | AI Year-End Review 2025 Predictions: Tech, Business, Media, Politics!