Pioneers Insight Method Research Author
Building an AI Physicist: ChatGPT Co-Creator’s Next Venture
Back to Episodes

Building an AI Physicist: ChatGPT Co-Creator’s Next Venture

Summary

  • Periodic Labs’ central wager is that nature should become the next reward function for AI. Early ChatGPT learned helpfulness from human preferences and later gained mathematical and coding correctness through verifiable graders; advancing physics requires the same optimization pressure against real experiments. Cubuk’s formulation is the thesis in one line: “Nature is our RL environment.”
  • More compute will improve models, but it cannot manufacture missing scientific knowledge or efficient out-of-domain learning. Fedus accepts that scaling laws continue to hold, then asks, “What is this y-axis?” A coding model can recursively improve at passing unit tests, but “that model is not going to then cure cancer”; Cubuk adds that an out-of-domain power law may have such a shallow slope that progress would take “centuries.”
  • Periodic is building the data engine that the scientific literature cannot provide. Published measurements can span orders of magnitude, negative results are rarely reported, and synthesis or superconductivity datasets may have noise floors too high to train predictive models. Because “these systems aren’t magic,” experiment must collapse uncertainty and continually move the training distribution toward the target.
  • High-temperature superconductivity is both a falsifiable benchmark and a forcing function for the full autonomous-science stack. The ambient-pressure mark cited is roughly 135 Kelvin; the founders say exceeding it would likely require autonomous synthesis, characterization, simulation, and experimental iteration. A hypothetical 200 Kelvin superconductor would matter even before commercialization because observing quantum effects at that temperature would be “such an update to people’s view of how they see the universe.”
  • The commercial wedge is an intelligence layer for engineers and researchers in space, defense, semiconductors, and advanced manufacturing. Periodic wants systems that automate simulations, connect design pipelines, and reduce physical R&D iteration time across “massive R&D budgets.” Fedus explicitly links mission and economics: “Technology and capital are intertwined,” so the lab plans a scoped “land and expand” motion rather than attempting to transform a production line on day one.
  • Mid-training—not retrieval alone—is how Periodic expects to turn general models into physics and chemistry experts. It plans to continue pre-training on crystal structures, synthesis recipes, simulations, experiments, and customer knowledge, then use high-compute reinforcement learning and specialized tools. The organizational design mirrors this composition: a roughly 30-person “N of one” team spanning LLMs, experiments, simulations, automation, and theory, reinforced by academic advisers and grants.

Deep dive

1. Nature supplies the reward that digital graders cannot

  • Periodic’s origin began eight years earlier at Google Brain, where Fedus helped Cubuk flip a tire neither could move alone. Their later conversations kept returning to quantum mechanics and superconductivity; as LLMs became useful for recalling science and writing simulations, improvements in reasoning, high-compute RL, physical simulation, and experimental scaling began to look like “the same kind of principles at play in ML.”

  • Fedus traces early ChatGPT to a standard RLHF pipeline: supervised examples converted a raw autocomplete model into an assistant, while a reward model learned that humans preferred completion A over B. That reward encoded friendliness, not whether mathematics or code was correct. Midha notes that later gains involved reasoning and “much better, more precise reward functions.”

  • Periodic aims to replace math and code graders with a physically grounded loop: agents read, calculate, simulate, propose experiments, observe results, and iterate. Simulators remain tools rather than truth—experiments error-correct their deficiencies because “the ground truth is the experiment.” Fedus’s broader premise is equally important: even brilliant humans discover little without opportunities to act, fail, and revise.

2. Scaling works, but its target distribution decides what it buys

  • Midha’s pushback—worth keeping: if predictable scaling produced emergent capabilities from general pre-training, why wouldn’t frontier labs eventually crack physics without a dedicated physical-verification stack? Fedus accepts the empirical scaling law but disputes the implied destination: internet-distribution performance and successful experimental science occupy very different y-axes.

  • Fedus’s coding analogy makes the boundary concrete. A model can write a pull request, receive unit-test feedback, reinforce successful code, and perhaps accelerate its own software development because code is abundant and digitally verifiable. Yet “that model is not going to then cure cancer”; it may assist a cancer researcher, but it has not acquired the missing data or iterated against that environment.

  • Cubuk separates in-domain from out-of-domain scaling: both may improve as power laws, but the latter’s slope deteriorates with distance from the training distribution. “You might need to spend centuries” to reach the desired result. Periodic’s answer is to make the target increasingly in-domain by changing the training set through experiments.

  • The existing corpus is not merely incomplete but structurally distorted. One reported physical property spanned orders of magnitude; formation-energy labels can be too noisy for predicting the next synthesis; superconductivity datasets may not help at all. Positive-result publication also removes a crucial learning signal: a valid negative result reveals what fails, while an unrecorded one teaches the model nothing.

3. Superconductivity makes the AI-scientist ambition measurable

  • Periodic’s first planned lab probes the quantum-mechanical scale where chemistry, biology, and materials operate. Its powder-synthesis system will mix existing powders and heat them into new materials—a process Fedus compares mechanically to a coffee-making robot at the San Francisco airport, yet one capable of exploring superconductors, magnets, and other technologically important materials.

  • Progress has an unusually clear scorecard: “What is the highest-temperature superconductor we synthesized?” Cubuk cites roughly 135 Kelvin as the best ambient-pressure number. Applied evaluations are similarly grounded—measure ductility, toughness, strength, or another requested property directly—making the signal much harder to hack than conventional model-training benchmarks.

  • A hypothetical 200 Kelvin superconductor is valuable even before a product exists. Cubuk argues that quantum behavior at such a high temperature would reshape understanding of the universe; technically, superconductivity is also a phase transition whose temperature is usually more dominated by fundamental crystal properties than by defects or microstructure that present simulators cannot yet capture.

  • Midha challenges whether superconductivity and magnetism are stepping-stones or domain-specific off-ramps. The founders’ answer is operational: the north star forces autonomous synthesis, characterization, theoretical calculation, simulation, and tool use into one complete loop. After proving it, Periodic wants a repeatable route across domains while measuring how systems generalize—from superconductivity to magnetism, and perhaps differently to fluid mechanics.

4. Physical-R&D systems are the commercial bridge

  • Midha compares Periodic’s path with frontier AI labs: an AI researcher may be the north star, but programming became the commercially valuable intermediate product. She frames Periodic’s possible equivalent as copilots for engineers and researchers in advanced industries. Fedus’s formulation is an “intelligence layer” for space, defense, semiconductors, and manufacturing, where physical iteration remains central and companies often lack good tools.

  • The ambition is to shorten iteration, find better material or process solutions, and preserve expertise as senior engineers leave. Fedus acknowledges the financing logic directly: “Technology and capital are intertwined.” A “wildly successful commercial entity” would fund greater scientific acceleration while addressing enormous physical-R&D budgets.

  • Deployment starts with a promoter, a critical scoped problem, and an agreed evaluation—not “transform your fab line on day one.” One prospective customer emphasized automating simulations that currently require substantial employee training, matching formats, feeding results into the design pipeline, and treating fragmented data together. Periodic would enter where customer urgency intersects with its strongest capability, then follow a “land and expand type method.”

5. Mid-training turns general models into scientific specialists

  • Fedus defines mid-training as continued pre-training that inserts missing knowledge before conventional post-training. For Periodic, that ranges from low-level crystal structures to semantic accounts of how material XYZ was made, plus simulation and experimental data. The goal is not a pile of disconnected corpora: adding distribution C should improve performance on A and B, demonstrating useful cross-domain generalization.

  • Retrieval is useful because access privileges can mirror the employee making a request, but it does not encode knowledge into model weights. Training can create a richer understanding while introducing a harder governance problem: if some documents are CEO-only, Periodic must bucket knowledge and potentially create distinct systems rather than indiscriminately pre-train one model on everything.

  • Periodic expects to build on improving base LLMs, open simulation methods, and neural tools rather than recreate every component. Agents can call specialized neural nets where those systems excel; language is useful for literature and synthesis recipes, while equivariant graph neural networks and geometric diffusion models can provide geometric tools for representing atoms and design geometries.

  • Academia contributes another missing layer: expert reasoning strategies. One physicist rejected a model’s reasoning strategy because it should reason at a higher level, “in terms of symmetries.” Academic collaborators can identify such primitives and analyses, while Periodic designs reinforcement environments that actually reward them.

6. The organization mirrors the multidisciplinary model it wants to train

  • The roughly 30-person team spans LLM research, experiments, simulation, solid-state chemistry and physics, automation, facilities, theory, mid-training, RL, and infrastructure—expertise Cubuk describes as “fractal.” Weekly teaching sessions transfer RL and data-cleaning knowledge one way and science and scientific history the other; hybrid researchers translate between groups by asking API-like questions about inputs, outputs, and targets.

  • An advanced physics or chemistry degree is not required. Cubuk reverses the basketball line about proximity to LeBron James: even the strongest physicist’s unknown territory is much larger than what they know. Modern science has become too broad for one person to master the chemistry, synthesis, characterization, and physics required for a breakthrough, making collaboration—and eventually the integrative model itself—essential.

  • The academic strategy has two arms: an advisory board spanning superconductivity, solid-state chemistry, physics, synthesis, and high-throughput DFT, and a grant program for work better housed at universities. The transcript names prospective contributors whose rendered names are unclear; the proposed grants focus on LLM agents, synthesis, materials discovery, and physics modeling.

  • Hiring prioritizes deep curiosity, pragmatism, world-class ability along at least one dimension, and a desire to “make contact with reality.” Cubuk adds the decisive tempo requirement: “We want these technologies not in 10 years”—the goal is for LLMs to improve physical science as soon as possible.