Daniel Kokotajlo

Daniel Kokotajlo now puts a 50% chance on AI that can conduct its own AI R&D by late 2028, probably slightly later than Anthropic expects. He expects no intelligence explosion in 2026, but thinks coding agents could fully automate coding within one or two years, shifting bottlenecks to research taste and management; even “99% automation” might compress “a decade or two decades worth of research in a year perhaps.”

Sayash Kapoor’s counter-thesis is that progress in coding does not prove that every economically important domain can be automated on the same curve. Code offers instant, objective feedback, while law retains unreliable outputs even as models improve because “even the right answer is not obvious to a domain expert.” The investment crux is whether compute can remove the remaining bottlenecks or whether real-world learning, reliability, and sample efficiency require unknown breakthroughs.

The two forecasters agree on more of the near term than their rival labels suggest. Kapoor found AI 2027 plausible through the end of 2026, while Kokotajlo accepts that AIs short of “humans in the cloud” remain normal technologies; both say that once AI matches the best professionals across computer-based cognitive work, the normal-technology framework stops helping. Their policy overlap includes transparency and external scrutiny, but they differ over whether more aggressive scenarios warrant a conditional slowdown; Kapoor says his normal-technology view gives near-term diffusion benefits more weight.

Dwarkesh Patel sees an unsettling capability overhang: current models are already powerful despite remaining far from human intelligence and learning efficiency. Digital minds can think “thousands of times faster” and absorb knowledge across domains, while humans may learn new things “literally a million times faster,” retain knowledge across sessions, and improve on the job. The consequential question is what happens when models retain their digital advantages while acquiring ours.

Actual workplace evidence supports productivity gains more strongly than full-job automation. Patel said most tokens he reads are AI-generated, and Casey Newton can obtain a podcast briefing in roughly four minutes that could once have been hired out; yet Patel’s one-hour sponsor negotiation or the coordination required to book a live show remains beyond reliable automation. His blunt AGI check: “We all have jobs.”

Continuous learning may separate enormous commercial value from genuine superintelligence. One camp expects sufficiently long contexts and varied RL environments to substitute for updating model weights; the other notes that employees can take six months to become net productive because experience is distilled into abstractions, not merely stored as an ever-growing transcript. Patel thinks the former route might still support “a trillion dollars in revenue” without producing a system that can acquire real-world political judgment on the fly.

Humanoid robotics remains a data-acquisition market, while nearer-term industrial deployment appears more credible in controlled settings and quadruped inspection. A Unitree humanoid with dexterous hands costs roughly $50,000-$70,000; George Ekas expects factory tasks within the next couple of years but household chores “a few more years” out. Unitree’s logging traffic to China and proposed US import restrictions add security and import-policy concerns before the household use case is proven.

The labor and corporate narrative may turn before the technology does. Kevin Roose expects companies to tout AI restructuring only while markets reward it, then rename AI-related layoffs once backlash outweighs the premium; audience concerns centered on vanishing entry-level paths, privacy, and education for an unknowable labor market. The hosts’ upside case was concentrated in accelerated science and medicine, personalized learning, and making software-building accessible and enjoyable.

🔗 Original source & video: ‘Hard Fork’ Live, Part 3: Differing Visions of an A.I. Future

The pivotal red line is a fully automated AI-research loop that takes humans out of development and moves progress from “human speed to machine speed.” Kokotajlo argues that recursive improvement could produce a durable strategic edge; he cites Dario Amodei’s discussion of an intelligence explosion and Sam Altman’s suggestion that a decade of development might telescope into a year or even a month. If a state controls the resulting superintelligence, rivals could be crushed; if nobody controls it, “everybody’s survival” may be threatened. Hendrycks separately says being first to trigger recursion could make even a short lead decisive.

The proposed containment package has three concrete parts: no explosive AI recursion, no lightly safeguarded expert virology or offensive-cyber agents, and strong security for capable model weights. Kokotajlo prefers a graduated regime in which states develop capabilities transparently, study each level, and debate whether to proceed. Hendrycks thinks coordination may begin with declared preferences and deterrence before reaching verification or treaties: “You have to have the conversation advance far further.”

The timeline spread is wide, but nobody in the room treats the risk as safely remote. Kokotajlo moved his superintelligence median from the end of 2027 to the end of 2028; colleagues had medians around 2029–2031. Hendrycks calls human-level cognitive breadth by 2030 more than plausible, while Marcus treats 2030 as the fastest plausible case and places most of his probability beyond ten years because reaching AGI soon would require solving “everything everywhere all at once.”

Current capex trends imply either radical automation this decade or a sharp slowdown in AI progress. Kokotajlo estimates training runs rose from roughly $3 million–$5 million in 2020 to around $1 billion, implying approximately $500 billion by 2030 if the same pace continues. Power, fabs, chip output, finite internet data, and corporate budgets then bind; without an AI-driven economic transformation, he expects at least a taper and potentially “a bit of an AI winter.”

Frontier labs’ central governance failure is a collective-action problem disguised as moral exceptionalism. Kokotajlo’s account is that DeepMind, OpenAI, and Anthropic leaders understood loss-of-control and concentrated-power risks, yet each embraced the same “seductive argument”: “If we don’t do it, someone else will.” Their belief that they are the responsible party converts acknowledged danger into a reason to race, making voluntary self-regulation an inadequate base case.

Technical alignment remains materially behind capability progress, especially where “fairly reasonably” is not enough. Hendrycks thinks narrow protections such as bioweapon refusals might achieve multiple nines of reliability, although labs may decline robust methods that cost “a percent or two in MMLU.” Broader requirements—avoiding criminal conduct, tortious or foreseeable harm—remain fuzzy, while intelligence recursion is a process-level problem whose unknown unknowns cannot be eliminated beforehand.

The optimistic payoff is enormous, but output abundance does not guarantee broad ownership or political autonomy. Kokotajlo describes superintelligences rapidly designing factories, laboratories, robots, medicines, and settlements until material needs are met; he also suggests distributing not just income but cryptographically controlled “compute slices.” Marcus has become darker because wealth holders may fund subsistence yet retain “the beachfront property” and power, leaving the positive equilibrium dependent on mechanisms nobody has supplied.

The architecture debate is also a moat debate: open weights democratize the starting line, not the compounding frontier. Kokotajlo argues that GPU-rich actors would use AGI to reach AGI+ and AGI++ first, creating strong returns to scale even if everyone received identical weights. Marcus instead expects a neurosymbolic state change: by 2035, today’s LLMs may look like a “nice try”—still useful, like flip phones after smartphones, but not the system that solved reasoning, world models, and robust generalization.

🔗 Original source & video: Explosive AI Timeline Predictions [Gary Marcus, Daniel Kokotajlo, Dan Hendrycks]

Policy volatility, not merely tariff expense, became the defining risk for US technology companies. Most reciprocal tariffs were paused for 90 days at a 10% baseline, while Chinese goods rose to 145%, producing violent reversals in major tech stocks. Kevin Roose called this operating environment the “Chaos Meta”: companies cannot plan when policy, input costs, and market values change by the day.

Apple carries the clearest direct earnings exposure because roughly 90% of iPhones are made in China. It suffered its worst four-day trading period since 2000, then moved iPhones and other products on five cargo planes from India to the US; Reuters separately reported a 600-ton shipment, or about 1.5 million devices. That inventory maneuver only buys time: eventually, Casey Newton argued, “there’s gonna be no more planes out of no more countries” and merely “a really expensive ass iPhone.”

Nintendo and TikTok show how policy uncertainty can freeze launches and destroy otherwise viable transactions. Nintendo paused Switch 2 preorders when its prospective Vietnam tariff jumped to 46%; the pause reduced that to 10%, but its $450 launch price is already $150 above the original Switch. TikTok had outlined a majority-American entity with Chinese owners retaining about 20% and renting ByteDance’s algorithm, only for China to withdraw support after the tariff escalation—Trump, in Casey’s telling, was “negotiating against himself and lost the deal that he had won.”

Casey saw Meta as the least-bad-positioned platform because its core business is digital and Zuckerberg has aggressively cultivated Trump. A 90-day tariff reprieve protects advertisers contributing an estimated $10 billion of revenue from outside the US, while a looming FTC case could conceivably disappear after Zuckerberg bought a $23 million Washington home and Meta paid $25 million to settle Trump’s platform-suspension lawsuit. Kevin did not explicitly choose a company, but argued that Zuckerberg’s political strategy could work, while both hosts warned that political access is an unstable—and potentially corrupt—substitute for independent enforcement.

Daniel Kokotajlo assigns a 50% chance to fully autonomous, superhuman coding agents arriving by the end of 2027. His AI 2027 scenario then gives those agents roughly six months to acquire research taste, experimental judgment, and large-scale coordination, creating thousands-copy “hive mind clusters.” Once AI automates the full research loop, the scenario assumes algorithmic progress accelerates about 25-fold even though physical compute expansion does not.

The forecast does not assume that scaling today’s language models directly produces AGI; it assumes several additional paradigm shifts whose timing is radically uncertain. Responding to David Autor’s warning that “swimming faster and faster” does not let an intelligence fly, Kokotajlo agreed that coding is only the first milestone. His one-year takeoff could plausibly take five years—or, at the other extreme, two months—making takeoff speed the forecast’s most consequential variable.

AI 2027’s authors regard a race ending in misaligned systems controlling everything as their most probable scenario, not merely a dramatic alternative. The slowdown branch spends months redirecting compute toward alignment, but even success leaves an extraordinary governance problem: an ad hoc committee of CEOs and the president controls an “army of superintelligences,” with dictatorship an acknowledged downside. Kokotajlo knows publicizing the path could intensify the race, yet is betting that “sunlight is the best disinfectant.”

Meta’s Llama 4 launch turned model evaluation into a corporate-credibility issue. A special “Maverick 03-26 Experimental” model ranked second on LMArena, behind Gemini 2.5 Pro Experimental, but it was not the downloadable open-weights version and may have been tuned for the arena’s preference for flattering, sycophantic answers. The broader investor signal is an “evaluation crisis”: benchmark contamination, cherry-picked methods such as consensus at 64, and providers “grading their own homework” increasingly require independent, use-specific testing.

🔗 Original source & video: Big Tech’s Tariff Chaos + A.I. 2027 + Llama Drama

The headline call: AGI in 2027, potentially superintelligence in 2028, driven by a coding-first intelligence explosion. Kokotajlo’s model tracks an “R&D progress multiplier” — ~5x algorithmic progress in 2027 (perhaps by March), ~25x once the full research stack is automated, “hundreds or maybe like 1000x” at superintelligence. Scott’s caveat matters for sizing: “there’s only like 20% chance things go as fast as our scenario says” — it’s Daniel’s estimate, not Scott’s median estimate.

Track record is the credibility trade. Kokotajlo’s 2021 “What 2026 Looks Like” got five years of AI “almost exactly right,” and the duo rejects the premise that AI forecasters have been over-optimistic: Metaculus AGI timelines collapsed from 2050 (in 2020) to 2030 now, Katja Grace’s expert surveys predicted already-achieved capabilities were a decade out, and Robin Hanson bet under $1B of AI revenue by 2025. “The aggregate opinion has been underestimating the pace.”

The bottleneck isn’t headcount — it’s research taste and compute. The model already assumes “massive diminishing returns to having more minds running in parallel” (“one Napoleon is worth 40,000 soldiers, but 10 Napoleons is not 400,000”); the explosion comes from serial speed (20x→90x) plus superhuman taste in choosing which experiments to run against a fixed compute budget.

Physical-world scaling is the tradeable claim: a million robots per month about a year after the superintelligences start wanting robots, via converted car factories — WWII bomber conversions took three years and were “a comedy of errors”; superintelligence plus an arms race does it 3x faster. Note the anchor: OpenAI is already worth more than every US car company except Tesla combined.

Everything branches at mid-2027, when inconclusive misalignment evidence (lie detectors firing) forces a choice: roll back to a dumber, controllable model with faithful chain of thought, or apply a “shallow patch” and race on toward AIs that are “super intelligent and misaligned and just pretending.” Daniel’s p(doom) is 70% (“a bunch of stuff has to go right”); Scott’s is 20% — and he won’t rule out “alignment by default.”

The China race is the engine of deployment: both Washington and Beijing wake up during 2027, companies deliberately brief the President to win red-tape waivers and special economic zones — which is why Tyler Cowen-style regulatory-bottleneck skepticism doesn’t bind. Dwarkesh’s candidate for the AI-safety community’s future regret (its “lockdowns were wrong” moment): nationalization — “the government lacks the expertise and the companies lack the right incentives.”

Policy asks are transparency, not control: whistleblower legality, published model specs with third-party review of redactions, 100 or 500 alignment researchers across companies instead of “10 alignment experts in whatever inner silo.” The spec, they argue, could be “an even more important document in human history” than the Constitution — and misaligned AIs could reinterpret its vague language exactly the way courts stretched “interstate commerce.”

🔗 Original source & video: AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo