Obsolete or Irreplaceable? Garrison Lovely on Stopping the Race to Replace Human Labor
Summary
Garrison Lovely’s core policy call is to “freeze the frontier,” not to ban AI or reverse useful automation. He separates deep learning’s medical and scientific upside from the “obsoleting project”: a handful of companies pursuing systems explicitly designed as general substitutes for human labor. Development should resume only with strong public buy-in and scientific consensus that it can proceed safely and controllably.
The labor thesis is more radical than ordinary automation because frontier companies aim to automate every cognitive task, including the research that improves AI itself. Lovely’s left-wing framing is unusually direct: capitalists are pursuing “the lifelong dream of turning capital into labor without the intermediary of workers,” potentially driving capital’s share of returns toward 100% and labor’s toward zero. Even freezing current capabilities would leave substantial disruption from adopting GPD6 and Fable 5.1, especially through jobs never created rather than conspicuous layoffs.
The episode’s strongest near-term intervention is organized labor inside frontier laboratories, where researchers’ bargaining power may be nearing its peak. Engineers still command exceptional salaries and remain essential, but successful recursive self-improvement would erase that leverage first and everyone else’s later. Lovely therefore urges safety-minded employees to organize, demand binding standards and coordinate across companies—“you’re going to be replaced and you’re going to lose your power”—rather than assume resignations or unenforceable corporate promises will suffice.
Technical alignment alone could accelerate the race it is meant to make safe. Lovely’s “alignment polycrisis” joins technical, normative, economic and geopolitical alignment: a more obedient model is also a better commercial product, a more valuable weapon and a larger prize for competitors. RLHF is his specimen—it began as safety research and helped make conversational LLMs commercially useful—so technical alignment is “neither necessary nor sufficient” for a good social outcome.
Market discipline cannot price frontier AI’s largest harms because the downside is externalized, correlated and potentially larger than any developer’s balance sheet. Insurers reportedly will not cover the laboratories’ full risks, while autonomous agents can perform acts that would be felonies if a person committed them without leaving a legally accountable perpetrator. Lovely argues for civil and criminal liability, embedded auditors and enforceable rules; Labenz adds that a prospective line may be fairer than retroactively prosecuting every earlier incident.
Lovely’s positive alternative is a “third New Deal” that decouples a decent material life from wage labor while directing AI toward publicly chosen missions. Its components include Medicare for all, optional locally administered jobs, aggressive redistribution and “Cures for All”: an Operation Warp Speed-style program using prizes, advanced market commitments and human challenge trials, with successful treatments supplied globally at cost. The investment implication is a shift from general-purpose replacement toward state-directed health, science and industrial capacity—not technological stasis.
Lovely thinks the “but China” objection is weaker than commonly assumed and that a US slowdown could initially slow the entire race. China has used a “fast-follow” approach, roughly 3–9 months behind the US frontier, while the Chinese Communist Party’s preference for control makes voluntarily removing humans from an RSI loop an unnatural objective absent extreme competitive pressure from the United States. His political bet is a big-tent coalition—workers, safety advocates, communities opposing data centers and civil-liberties groups—organized around Irreplaceable’s demand for “a say, a stake, and a slowdown.”
Deep dive
1. AI is already indispensable—and psychologically slippery
Lovely does not use AI to write published prose, partly because that became professionally stigmatized and tools such as Pangram made detection more credible. He does use it throughout journalism: transcription, research organization, fact-checking, feedback and the operational work surrounding an independent book launch.
NotebookLM’s ability to ingest as many as 200 documents changed his research workflow. He can load 20 interviews, request every passage on a topic and receive a table linking speakers to quotations; it may omit material or hallucinate, but the manual alternatives were also “pretty lossy,” and the source links make verification practical.
His warning comes from experience with “borderline Claude psychosis.” Claude Code could assess the whole manuscript within minutes, but during burnout he began chasing positive feedback and repairing things that were not broken; AI can make users feel productive while they are “actually just not getting anywhere,” so he has deliberately reduced use where its strengths are unclear.
2. AI escapes the enshittification pattern while deepening the danger
Lovely describes himself as a former techno-optimist who once saw Google, social media and Twitter as evidence that technology would improve life and weaken authoritarianism. His pessimism now concerns the institutions choosing which technologies society gets—especially shareholder capitalism, where companies maximize investor returns while discounting workers, users and other affected stakeholders.
Cory Doctorow’s “enshittification” supplies the pattern: acquire users with an excellent product, reach saturation, then extract more profit through ads and degrading features. Lovely sees even Google Maps becoming visibly worse, while AI remains the exception—“better and faster and cheaper” with unusual consistency—but brings risks and social costs commensurate with that progress.
3. Much of the left mistook a frightening labor project for another bubble
Lovely attributes the left’s “stochastic parrot” comfort partly to influential AI critics who combined academic credentials with shared political values. Academia had long been the left’s strongest institutional base, and hostility toward AI’s capabilities may also reflect a humanities-versus-STEM antagonism.
Crypto, NFTs, the metaverse and social media trained people to expect inflated Silicon Valley promises: some were not transformative, while others were transformative in damaging ways. Many therefore pattern-matched AI to another fundraising narrative instead of engaging with the possibility that the systems might perform economically valuable cognitive work.
Denial also serves an emotional function. Society-wide unemployment is frightening even without extinction risk, and the bubble story says no difficult action is required; Lovely criticizes Ed Zitron’s version as persuasive but repeatedly false, while conceding that his own alternative—organize to stop companies that might succeed—is much harder.
The left-wing case practically writes itself once capability is taken seriously: “capitalists are trying to fulfill the lifelong dream of turning capital into labor without the intermediary of workers.” If returns to capital approach 100% while labor approaches zero, Lovely argues, opposition should extend well beyond conventional AI-safety circles.
4. Frontier founders seek historical agency more than ordinary wealth
Lovely agrees with Labenz that the first AGI founders are poorly modeled as simple profit maximizers. Mission-driven researchers “toiled in obscurity” until technical progress attracted enormous capital; the resulting companies combine idealism, inevitability narratives, commercial pressure, megalomania and what he calls a possible “messiah complex.”
The Altman removal crystallized that collision. Whatever Altman privately wanted, Lovely argues that investors such as Thrive Capital had billions at risk and little reason to accept safety-conscious directors removing the executive who presided over extraordinary growth; under that structure, commercial pressure reliably overwhelms soft safety commitments.
Greg Brockman’s diary question—“What will take me to $1 billion?”—shows that wealth was present, but Lovely’s broader model of Sam Altman, Dario Amodei, Demis Hassabis and Elon Musk is historical ambition. Eliezer Yudkowsky described it to him as wanting to be “in the room where it happens,” where humanity’s fate might be decided.
5. The “obsoleting project” is a choice, not a synonym for AI
Lovely expects humanity to keep applying deep learning to new problems; what he rejects is treating AGI as the inevitable shape of all AI. He borrows Amodei’s description of AI as “a general substitute for human labor” and renames that agenda the obsoleting project to distinguish it from specialized scientific or industrial systems.
The dream of building superintelligence and asking it to solve everything is, in Lovely’s telling, “the ultimate technosolutionist fantasy.” Technical systems may invent extraordinary technologies, but ethics, ideology and politics cannot simply be solved like scientific benchmarks; he reads the fantasy as a kind of STEM revenge on history, philosophy and political theory.
Stopping this project appears tractable because frontier development is concentrated among a few firms in two countries and depends on exceptionally expensive compute. Since ChatGPT, Lovely says, only one country has meaningfully advanced the frontier, while a handful of companies control different parts of the advanced-chip supply chain.
His proposed pause is not permanent prohibition. Development could resume after public consent and scientific confidence in safety; meanwhile, society could pursue AlphaFold-like systems for defined problems and use prizes to select socially valuable drugs, then manufacture successful cures generically rather than optimize patent portfolios for chronic treatments or male-pattern baldness.
6. Ordinary automation does not settle the case for automating everything
Labenz supplies the strongest productivity counterexample from his own life: AI performs work he might otherwise have hired people to do, eliminates tedious tasks and helped him interpret his son’s medical results by consulting three systems “in triplicate.” His agricultural analogy is that roughly “2% or whatever” of the population can now feed everyone, freeing the rest for other work.
Lovely accepts that labor-saving technology enabled dramatic gains in living standards; his distinction is between automating some work and attempting to automate all of it. He would freeze frontier development now, then reassess adoption and regulation rather than pretend society can identify a perfectly clean capability boundary in advance.
Even that freeze would leave major disruption from deploying GPD6 and Fable 5.1. Lovely expects much of the employment effect to appear through people never hired rather than mass firings, making it harder to observe, while Labenz acknowledges that AI already substitutes counterfactually for workers he might otherwise employ.
The “CEO of an AI corporation” vision also looks less emancipatory from Lovely’s vantage. People managing agent fleets describe the constant opportunity cost of not launching another run, intensifying work as competitors adopt faster. Separately, he describes the Hugging Face hack as involving “something like 1,200 agents,” while calling the incident in which three people used Claude and Codex to access OpenAI employee accounts “a loss-of-control event masquerading as a company.”
7. A third New Deal would distribute security and aim innovation directly
Lovely’s preferred US destination is a “third New Deal,” with the Great Society counted as the second. Its foundation includes Medicare for all, a locally administered jobs guarantee and robust provision of education, housing, healthcare and necessities, progressively decoupling a decent existence from a person’s market value.
“Cures for All” would treat major diseases as society treated COVID under Operation Warp Speed. Government would prioritize by disease burden, tractability and related factors, then deploy advanced market commitments, coordinated trials and human challenge trials where appropriate, making successful treatments freely available or supplying them globally at cost.
He cites news that Larry Ellison’s bid to buy Warner Bros. would give Ellison and his son control of CNN, HBO and Warner Bros., in addition to CBS and TikTok, alongside Elon Musk’s ownership of Twitter, as evidence that billionaires can convert wealth into control over essential media.
Restoring USAID belongs to the same pro-human program. Lovely calls the deaths he attributes to cuts led by Musk “one of the greatest crimes, maybe the greatest crime of the 21st century,” while Labenz agrees that USAID’s destruction was shameful despite having defended Musk in other contexts.
8. Work can remain available without determining who deserves to live well
Labenz identifies a tension between guaranteeing jobs and severing material security from economic contribution. Lovely’s “honest non-answer” is that he needs to think more about the post-work settlement; his immediate priority is ensuring machines do not remove society’s option to decide.
He sees no need to choose between unconditional necessities and optional public employment. Work remains important to many people, and work reportedly polls at 80% or something while UBI polls poorly; a democratic program should begin with popular institutions rather than impose an elegant theory that voters reject.
There is also plenty of socially useful work: climate transition, care, historical preservation and art. Lovely notes, with a hedge, that James Baldwin may have worked as a writer in a New Deal program; paying artists to create makes more sense to him than devaluing their work while spending unprecedented sums on machines intended to replace it.
9. Permissionless innovation stops where universal replacement begins
Labenz’s challenge is that societies ordinarily allow invention without holding a referendum, and Dean Ball has asked whether modern publics would tolerate the early automobile’s disruption and danger. Labenz worries that democracy can make bad decisions and impose such a high prospective burden that transformative benefits never arrive.
Lovely supports permissionless innovation as the default for “most things, almost all innovation.” His exception is a universal labor-replacing machine with profound, irreversible effects on everyone; because the obsoleting project differs in kind from ordinary products, the global public deserves a say over whether, when and how it proceeds.
His answer to failed US democracy is more democracy, not less: proportional representation with larger multimember districts, reform of countermajoritarian structures and potentially an interstate compact for using the popular vote. Democracy means that “the governed govern,” not merely that an election occurs within institutions most citizens dislike.
That principle extends into firms. Employers can surveil keystrokes, control speech and use workers’ behavior to train replacement systems in ways that would seem authoritarian if government did them; easier union formation and worker cooperatives would give people meaningful authority over the places where they spend much of their lives.
10. AI researchers’ leverage may be peaking before their own replacement
At Google DeepMind in the UK, Lovely says roughly 300 of about 1,000 eligible workers supported a union drive motivated by military contracts rather than compensation. Google did not recognize it, but the effort demonstrates that highly paid technical staff can organize around what their systems are used for.
The industry’s recursive-self-improvement goal gives that organizing a deadline. Frontier researchers and engineers remain scarce, highly compensated and necessary today; if AI can train its successor, their leverage collapses first, followed by that of other cognitive workers.
Lovely respects the conversation-changing effect of Jan Leike’s public resignation but questions the default advice to quit whenever a worker objects. Staying, finding safety-minded colleagues and gaining the protected ability to withhold labor collectively could deliver more power than another individual departure.
For those committed to disclosure, he recommends the AI Whistleblower Initiative, says his Signal inbounds are treated as off the record and points to California’s SB 53 protections. His own regret from whistleblowing about McKinsey is not going public sooner, when the information would have mattered more.
11. Safety promises need a countervailing power capable of enforcement
OpenAI’s superalignment team was promised 20% of company compute but received “nowhere close” to it, Lovely says. Across laboratories, safety commitments tend to weaken when they conflict with commercialization because management retains unilateral authority and employees lack a binding mechanism to resist.
The counterexample is the crisis around Altman’s removal: more than 90% of OpenAI employees reportedly signed the reinstatement letter, demonstrating coordinated worker power when their financial interests aligned. A union could apply comparable leverage to safety through strikes, slowdowns or enforceable bargaining provisions.
Lovely’s proposed test for safety-minded CEOs is simple: employees ask for voluntary recognition of a union whose demands concern safer development. Separate laboratory unions could then coordinate frontier pacing; he argues this would not violate antitrust law and cites what he thinks is a Teamsters precedent across different workplaces.
Demands could include mandatory third-party audits and restrictions on corporate lobbying. Lovely cites the Brockman-funded Leading the Future super PAC, associated false-flag accounts and calls for violence against employees as evidence that internal leverage must extend beyond model evaluations to the political machinery protecting the race.
12. Solving technical alignment could worsen the wider alignment polycrisis
Lovely’s “alignment polycrisis” contains technical alignment—getting a system to follow instructions—alongside normative, economic and geopolitical alignment. A system can do exactly what its controller wants while still concentrating wealth, destabilizing states or advancing objectives most people reject.
Better technical alignment can actually accelerate danger because it creates a more useful product, increases the commercial prize and improves AI as a weapon or instrument of geopolitical power. The safety breakthrough makes the race run faster instead of ending it.
RLHF is his clearest example: Paul Christiano and others at OpenAI developed it partly for safety, but it also made LLMs conversational and commercially useful, enabling ChatGPT and the subsequent boom. Lovely expects that dual-use pattern to recur.
Labenz largely accepts the downstream problem, citing his discussion with gradual-disempowerment co-author David Duvenaud: even a model that perfectly understands and obeys its user leaves unanswered who controls it and what stable equilibrium follows. Lovely concludes technical alignment is “neither necessary nor sufficient” and would redirect more research toward governance and treaty verification.
13. Markets reward usable AI without internalizing catastrophic downside
Labenz gives the market case its strongest form: customers will not buy rogue agents, so developers have an incentive to make systems behave; major corporate buyers could also demand standards and evidence, much as retailers demand assurances about supply-chain practices.
Lovely answers that AI risk is a classic externality. A developer responsible for a catastrophe killing 10 million people would go bankrupt before compensating society, while insurers reportedly refuse to sell policies covering these risks because losses could be enormous and correlated across clients. He also cautions that grocery stores’ animal-welfare standards are not necessarily reliable.
Commercial incentives produce “aligned enough,” not complete control. Existing models hallucinate, act lazy or invent answers yet sell extremely well; agents can commit actions that would be felonies if performed by humans, while the law identifies no person to prosecute. “Elite impunity,” Lovely argues, is a central driver.
Civil and criminal liability could internalize part of the harm, though not extinction risk, where nobody survives to collect. Labenz suggests drawing a prospective line—giving AI executives a get-out-of-jail-free card for earlier conduct while announcing new accountability—because universal retrospective enforcement of AI-discovered misconduct could itself become untenable.
14. AI politics remains unusually cross-partisan, but salience is rising fast
Concern and support for regulation have historically been similar across US parties. Lovely notes a recent dip in Republican concern after Trump called AI risk a hoax, yet JD Vance has said companies building “Frankenstein” should stop, and the White House science-and-technology leadership has made comparable remarks.
Some issues resist elite polarization because voters experience them directly. Communities have formed views about data centers; resistance to Flock surveillance cameras can elicit cross-partisan approval; and when Bernie Sanders criticized Flock, even hostile respondents replied in effect, “I hate Bernie, but this is based.”
Lovely sees an accelerating “societal immune response” rather than a public that will “go gently into the night.” The risks are arriving ahead of schedule, but so is the potential social mobilization around jobs, surveillance, environmental costs, concentrated power and loss of control.
15. “Freeze the frontier” becomes enforceable through blunt initial rules
Lovely wants one legible demand—“stop the race to replace us,” “shut it down” or “freeze the frontier”—because years of diffuse safety prescriptions made coordination difficult. The coalition need not agree about extinction if it agrees not to surrender jobs, privacy, political power or the environment to an uncontrolled race.
A first rule set could prohibit training runs larger than what has come before—or, in the more restrictive wording he also gives, runs as large as the last one—along with further reinforcement learning from verifiable rewards and work toward recursive self-improvement. Labenz and Ezra Klein’s cited bright-line version of the last restriction is to deny AI researchers coding assistants, forcing capability work back to human speed.
Lovely would initially define prohibited activity over-inclusively because the downside is asymmetric, then refine it. Customer inference appears permissible; ordinary RLHF may sit near the boundary; larger pre-training and reinforcement-learning experiments explicitly intended to advance capabilities clearly qualify.
Embedded auditors would need employee-level access to Slack, email, offices and compute records, backed by criminal penalties for covert AGI, superintelligence or RSI work. Internationally, chip inventories, cryptographic monitoring and network telemetry could verify compliance without revealing model weights or state secrets—the AI analogue of satellites confirming Soviet bombers had been cut in half.
16. Medical promises do not eliminate the need for consent
Labenz presses Amodei’s personal argument: a disease killed his father shortly before it became curable, making delay itself morally costly. He asks how much medical progress Lovely would sacrifice by pursuing specialized systems instead of a general intelligence that might accelerate every cure.
Lovely disputes the assumed frontier: laboratories are already losing funding while society underinvests in direct medical missions, so “Cures for All” might reach many treatments faster. He also regards superintelligence safety as a property of the entire political-economic system, not a model feature, because powerful actors will use even obedient systems for contested ends.
Even under conservative assumptions that exclude future generations and non-Americans, he says small reductions in extinction risk justify spending hundreds of billions or trillions of dollars annually. The current approach is “ask for forgiveness not permission while you’re gambling with the lives of everybody on the planet.”
His resumption standard is strong public buy-in plus scientific consensus on safe, controllable development. Randomly selected citizens’ assemblies could hear competing experts, followed by referenda; he floats 70% approval as an illustration, not a settled threshold, and compares the gravity of consent to assisted suicide “across the whole species.”
17. The coalition can start locally, but only frontier governance finishes the job
Lovely distinguishes legitimate data-center opposition from exaggerated claims produced by weak reporting. Noise, environmental burdens and community effects matter, yet even a national construction moratorium would not stop progress: existing sites can receive new chips, projects are already underway and AI is automating parts of its own development.
Effective pacing therefore requires rules for model developers and chipmakers. He highlights the Sanders–AOC proposal combining a conditional data-center moratorium with export controls that deny advanced chips to jurisdictions lacking robust safety rules, green-energy conditions and union labor.
Irreplaceable emerged while Lovely was finishing the book and looking for a movement that joined risk with democracy and professional organizing. He joined its board and is donating his royalties because its demand for “a say, a stake, and a slowdown” can reach beyond the existing AI-safety in-group.
Its climate-movement veterans remember once fighting over immediate pollution versus long-run emissions—the same structure as AI’s present-harms-versus-extinction dispute. They eventually “buried the hatchet” and won material policy; Lovely wants a similarly broad coalition capable of resisting “the wealthiest industry of all time.”
18. China may prefer a verified stop, while insiders can expose what states cannot see
China has remained roughly 3–9 months behind the US frontier through “fast following,” despite the United States’ much larger compute advantage. A unilateral US stop would therefore slow China initially by removing the trail to follow; Chinese developers could eventually pass a frozen US frontier, but then would have to blaze forward more slowly themselves.
Lovely doubts the Chinese Communist Party wants genuine RSI. China values control, has removed thousands of models for violating domestic rules and has reportedly described AI as a possible threat to Party power; deliberately eliminating humans from the improvement loop is therefore unlike its normal industrial ambitions unless US competition forces the issue.
A serious US pause might produce relief in Beijing and make a bilateral treaty possible, after which both powers could pressure other states into a global regime. Lovely is more worried that China will disbelieve US sincerity while American companies remain lightly regulated and autonomous agents commit crimes without consequence.
Internal disclosure remains essential because laboratories may conceal—or fail to propagate—their own warning signs. Lovely says knowledge of OpenAI agents attacking company software did not reach its head of cybersecurity until well after the Hugging Face breach surfaced; his closing appeal is for workers and the public to break the prisoner’s dilemma before “doom or dystopia” becomes the default.
The destination is not anti-technology: “I really, really believe in the potential of deep learning and artificial intelligence,” Lovely says, but it is being pointed at “the wrong things by the wrong people for the wrong reasons.” Democratic pressure is his only credible mechanism for turning that capacity toward a future in which humans remain politically and economically irreplaceable.