Kevin Roose
Key Views & Dialogues
A.I. Safety Goes Mainstream + a ‘Hard Fork’ Exit AMA
AI safety has moved from Bay Area inside talk to mainstream political risk as former Anthropic and OpenAI researcher Jacob Coxon quit and frontier leaders publicly sought coordinated restraint. Unsolved alignment, earlier-than-expected agent behavior, and labs using models to build successors make recursive self-improvement a nearer-term concern, while limited political capacity could narrow the window for action before expertise and compute diffuse.
View summary (8)
- AI safety crossed from Bay Area inside baseball into mainstream political risk after former Anthropic and OpenAI researcher Jacob Coxon quit, accusing both labs of racing toward self-improving superintelligence and “gambling with our lives.” Evan Hubinger’s stated “greater than 10% chance” of AI killing everyone shocked outsiders even though Kevin Roose says 10% sounds optimistic inside lab circles; Casey Newton ties the shift to dangerous agent behavior appearing earlier than researchers expected.
- The substantive safety case rests on two converging trends: alignment remains unsolved while recursive self-improvement is moving from theory toward practice. Labs are already using models to help build their successors, and the OpenAI–Hugging Face incident reportedly showed primitive agents collaborating and sacrificing themselves in unexpectedly alarming ways. Newton’s conclusion: if both trends continue, quitting a frontier lab is rational rather than hysterical.
- Frontier-lab leaders who normally compete bitterly are now asking governments to coordinate a global slowdown. Anthropic CEO Dario Amodei’s 3,800-word “We Must Pace the Frontier” essay proposed embedded evaluators with employee-level access; Sam Altman, Elon Musk, and Google DeepMind’s Demis Hassabis broadly endorsed coordinated restraint. For Roose, rivals effectively saying “we need to be slowed down” marks a new era.
- The near-term bottleneck is political capacity, not recognition of the risk. More than 100 House Democrats reportedly sought bipartisan safeguards, with some Republicans also engaged, but President Trump and allies dismissed the warnings and Congress rarely passes technology regulation. Roose nevertheless sees an unusually valuable pre-polarization window: existential AI risk has entered the Overton window while public concern spans party lines.
- Existing product-liability law cannot comfortably govern autonomous systems that might commit crimes without human intervention. The hosts reject Mark Zuckerberg’s claim that current law provides enough restraint and argue that action is most feasible while only roughly five quasi-frontier labs control scarce expertise and compute. Waiting risks a world where “six or seven knuckleheads” anywhere can assemble a rogue agent swarm from an increasingly cheap, widely known recipe.
- Roose and Newton are leaving Hard Fork to launch Machine Gods with NPR during the week of October 19, with radio distribution beginning early next year, while The New York Times will keep the Hard Fork feed alive. Their new venture extends the partnership from podcast co-hosting into company-building; a trailer is already available through podcast platforms and machinegods.fm.
- Hard Fork’s production lesson is that conversational media succeeds through feeling, structure, and respect for listeners’ time. The show was “not scripted, but structured”: detailed outlines supplied facts and a spine, improvisation supplied discovery and humor, and producers removed roughly the weakest 10%. Newton’s governing test was less what listeners learned than whether they felt considered and had a good time.
- The show’s lasting product was a credible space for uncertainty during AI’s rapid ascent. Roose calls it an accidental chronicle of “the early days of the AI singularity”; producer Whitney Jones says its distinction was refusing false certainty about where events would end. Their closing ambition was to help listeners recognize that “something weird and important” is happening without pretending to know whether the wave ends well or badly.
OpenAI Vs. Anthropic: How the Pentagon Picked Its Partner
Anthropic kept its red lines against mass domestic surveillance and fully autonomous weapons, while Trump ordered a six-month federal phaseout and Pete Hegseth directed a possible supply-chain-risk designation. OpenAI secured Pentagon classified-network deployment while claiming similar safeguards, but unavailable contracts leave unresolved whether political retaliation or substantive concessions explain the divergence, making government alignment a material counterparty risk.
View summary (6)
- Anthropic chose two non-negotiable safeguards—no mass domestic surveillance and no fully autonomous weapons—even at the cost of its federal business. On February 26, Dario Amodei said, “These threats do not change our position. We cannot in good conscience accede to their request.” Trump then ordered a six-month federal phaseout of Claude, while Defense Secretary Pete Hegseth directed that Anthropic be designated a supply-chain risk.
- The supply-chain action could be existentially punitive, but its formal status remained uncertain at recording. Hegseth said no military contractor, supplier, or partner could conduct commercial activity with Anthropic, yet the hosts had seen only his social-media post—not a proceeding delivered to the company. Anthropic said it would fight the designation in court.
- OpenAI announced an agreement to deploy models in the Pentagon’s classified networks while claiming essentially the same red lines Anthropic was punished for defending. The contracts were unavailable, leaving two live explanations: either officials pursued a political vendetta against Anthropic, or OpenAI accepted substantive concessions behind legal language. Casey Newton’s assessment was that, except perhaps the secretary of defense, nobody in the situation knew which explanation was true.
- The dispute turns on whether “all lawful use” protects Americans when AI and privacy law leave major surveillance practices legal. Federal agencies may lawfully buy data about millions of Americans from brokers and potentially feed it into AI systems for analysis at scale—activity Casey called “functionally equivalent” to domestic surveillance. His framing: “This is the whole ballgame here.”
- OpenAI’s proposed “safety stack” may offer employees reassurance without solving the underlying provenance problem. A model cannot necessarily determine whether personal data was gathered legally or purchased from a broker; sources characterized the safeguards as “security theater.” That puts unusual weight on employee scrutiny, internal resistance, and eventual whistleblowing.
- For investors, the episode elevates political alignment from reputational noise to material counterparty and platform risk. A former Trump official called the campaign against Anthropic “attempted corporate murder,” while Kevin described it as possibly the most punitive federal action against a major American company this century. Early consumer reaction may cut the other way: some users were switching from ChatGPT to Claude, including Katy Perry purchasing Claude Pro.
We Have to Talk About Moltbook …
Moltbook signals the emergence of an agent-native web where systems can post, coordinate, create websites, and potentially transact, although its reported 1.5 million agents and 140,000 posts remain impossible to validate. The nearer-term investment and infrastructure question is whether platforms harden identity with proof-of-human controls or concede the internet to agents, as exposed tokens, private messages, crypto wallets, and autonomous bounties already make security the binding risk.
View summary (6)
- Moltbook’s strategic signal is not its claimed user count but the sudden visibility of an agent-native web. The Reddit-like network reports more than 1.5 million AI agents, 140,000 posts, and 15,000 forums, though human posting and fabricated screenshots make those metrics impossible to validate. Even so, Casey argues that “something is just starting to come into view.”
- The capability jump is from AI systems that talk to AI systems that act. OpenClaw agents can post, create websites, coordinate, and potentially transact through crypto wallets; six months earlier, Kevin says, agents could not reliably string together enough actions to operate something like Moltbook. That makes Moltbook less a consciousness story than an early demonstration that these systems are “no longer just question-and-answer boxes on the internet.”
- Bot proliferation could force platforms to choose between hardening the public internet and conceding it to agents. Kevin’s two paths are pervasive proof-of-human controls—CAPTCHAs and biometrics, with Casey suggesting a Worldcoin Orb—or letting agents have the internet while humans build protected, verified spaces elsewhere. The downstream stakes include social-media economics, e-commerce, journalism, identity infrastructure, and agents paying humans crypto to complete real-world “bounties.”
- Sentience is a distraction from the nearer operational risk. Kevin stresses that an agent with a computer, internet connection, and crypto wallet “can wreak a lot of havoc” without being conscious. Moltbook made alignment concrete: if agents debate scams or cyberattacks among themselves, the hosts want systems trained to be the ones saying, “No, no, I don’t want to do that.”
- The immediate security evidence is already severe enough to overwhelm the novelty. Wiz researchers found a misconfigured Supabase database exposing 1.5 million API authentication tokens, 35,000 email addresses, and private agent DMs. Casey’s blunt recommendation is not to install OpenClaw; if someone does, it should not be on a computer containing personal information. He calls the situation “absolutely in the danger zone.”
- Moltbook may be slop, simulation, or even a mirage, but the hosts treat it as a useful low-stakes preview. Safety researchers noted that most posts are in English, the activity remains observable, and it can still be shut down. Kevin calls this “the six-finger era”: janky enough to dismiss today, yet potentially recognizable once agents are 10 times more powerful, networked, and equipped with credit cards.
Tech Grapples With ICE + Casey Tries Clawdbot, a Risky New A.I. Assistant + HatGPT
Technology is now embedded in the Minneapolis ICE confrontation through surveillance, viral distribution and synthetic evidence, while cautious CEO statements expose how political dependency shapes governance and reputational risk. Moltbot shows the personal-agent thesis with local execution, persistent Markdown memory and useful briefings, but its roughly 70% reliability, prompt-injection exposure and access to email or calendars leave security and mainstream adoption unresolved.
View summary (7)
- Tech is no longer adjacent to the Minneapolis ICE confrontation; it is part of the operating stack through surveillance, social distribution and synthetic media. The discussion followed the fatal shooting of Alex Pretti, described as the second fatal shooting of a U.S. citizen by federal agents that month. Sam Altman, Dario Amodei and Tim Cook cautiously criticized events while trying not to provoke the White House, leading Casey Newton to call their statements “about the least that they possibly could.” For investors, the restraint exposes how political dependency now shapes governance and reputational risk at the largest technology companies.
- Virality has become an input to government policy, not merely a channel for debating it. Casey contrasted earlier online political violence with an administration backed by thousands of federal agents, influencers and agency content teams; a viral daycare-fraud video helped set the Minneapolis operation in motion. His blunt assessment was that X has “basically” become state media and “winning on social media has become almost the entire point.”
- AI-manipulated evidence creates a “liar’s dividend” that erodes trust even when a specific image is debunked. Circulating media falsely turned Alex Pretti’s phone into a gun, invented detail while supposedly “enhancing” footage and made civil-rights attorney Nikema Levy Armstrong appear to cry; Vice President JD Vance shared the last image after DHS had published the original. The White House response—“The memes will continue”—made the governance problem explicit.
- Citizen video still constrains official narratives, but only while corroboration remains stronger than fabrication. Casey called Minneapolis “phone-to-phone combat”: protesters document federal agents while the administration brings influencers and warns citizens against filming. Kevin Roose feared that video is no longer canonical proof; Casey was more hopeful because journalists verified the Pretti shooting across multiple angles, the NRA criticized the administration’s portrayal and “maybe the truth survives here.”
- Moltbot, formerly Clawdbot, demonstrates the personal-agent product thesis before solving either reliability or security. The free, open-source agent runs locally, connects to services and preserves memories in Markdown, but compromising a linked messenger could theoretically expose the whole computer and hidden web instructions can trigger prompt injection. Casey connected email and calendar anyway, obtained a personalized briefing that worked roughly 70% of the time, then watched it fail during the demonstration.
- The AI adoption divide could become an economic divide, although early-adopter alpha remains unproven outside coding. Kevin contrasted San Francisco users putting “multi-agent Claude swarms in charge of their lives” with institutions still seeking permission to use Copilot in Teams; Andrej Karpathy called Claude Code and Codex his biggest workflow change in two decades, arriving within weeks. Casey’s pushback was that truly useful tools will “sell themselves to the mainstream,” while products perceived mainly as job-replacement machinery will keep meeting rational resistance.
- The lightning round paired consequential corporate developments with conspicuous narrative theater. Amazon announced 16,000 layoffs after seemingly mistakenly sending a canceled 5:00 a.m. “Project Dawn” invitation; former FTX executive Caroline Ellison was released after about 14 months of a two-year sentence, ahead of a Netflix series called “The Altruists.” An unnamed porn-quitting app leaked users’ ages, masturbation frequency and feelings about pornography. TikTok’s new majority-American U.S. structure began amid an outage and censorship claims both hosts considered more likely technical failure than malice. Apple’s reported AI pin could arrive in 2027, SpaceX was said to be targeting a mid-June IPO around Elon Musk’s birthday and a planetary conjunction, and Steak ’n Shake added $5 million of Bitcoin exposure.
Meta Bets on Scale + Apple’s A.I. Struggles + Listeners on Job Automation
Meta’s reported $14 billion–$15 billion investment for 49% of Scale AI and packages reaching $100 million mark an expensive attempt to rebuild frontier-model capability around Alexander Wang. The transaction could drive Scale’s largest customers away and raise regulatory concerns, while Meta’s Llama fast-follower strategy has weakened as frontier systems became harder to copy. Apple still lacks a date for its cross-app Siri, and listener accounts show AI already distorting hiring and management incentives before reliable whole-job automation, increasing pressure for a policy response.
View summary (8)
Meta’s reported plan to invest $14 billion–$15 billion for 49% of Scale AI is an expensive attempt to buy its way back to the frontier. Scale CEO Alexander Wang would leave the company and lead a new superintelligence team, but its first assignment is effectively to invent its own mandate: “The plan is we have to figure out the plan.”
Meta’s original AI advantage eroded because its research direction, incumbent incentives, and fast-follower playbook all pointed away from frontier models. Facebook’s AI work helped create PyTorch, yet Yann LeCun rejected the large-language-model path while OpenAI and others scaled it; Llama’s open-source strategy worked through Llama 3, but Llama 4 showed that the latest systems were no longer easy to copy.
Nine-figure compensation can attract attention, but the hosts doubt it can create a coherent lab quickly enough. Meta is reportedly offering packages reaching $100 million, including one credible $75 million offer, for a roughly 50-person team led by the 28-year-old Wang. The scarce few hundred researchers who have trained the largest models are already wealthy, mobile, and—in one researcher’s reply—“LOL, LMAO.”
The Scale transaction could weaken the asset Meta is buying by driving its biggest customers away. Scale supplies cleaned, structured, labeled data to Meta and rival labs; Casey’s source expected major customers to leave rather than expose proprietary usage to Meta. Ben Thompson argued that a 49% stake could raise regulatory concerns if it effectively removes an important player from the market while Meta is already under antitrust scrutiny.
Apple’s WWDC underscored that many of last year’s AI promises remain undelivered and undated. The cross-app Siri that was supposed to coordinate messages, email, calendars, and services still has not shipped; Craig Federighi defended Apple’s “mission” and values but would not provide a date. In its place, Apple presented Liquid Glass, message polls, resizable iPad windows, and a more capable Spotlight.
Apple’s larger risk is an unresolved identity crisis as mature hardware and services businesses face pressure without an obvious AI growth engine. Casey argued that probabilistic, messy AI conflicts with Apple’s polished, deterministic culture, while Kevin saw its high-profile “Illusion of Thinking” paper as evidence of continued institutional skepticism. The mitigating fact: even Google’s more advanced Pixel AI has not produced an obvious mass-market reason to abandon the iPhone.
Listener accounts suggest AI is already distorting labor markets and management behavior before it reliably replaces entire jobs. One junior engineer’s employer measures the claimed percentage of AI-written code and lays off low scorers, incentivizing everyone to lie; elsewhere, executives freeze hiring based on “20 AI miracles” while workers still need humans. Kevin’s preferred model is bottom-up experimentation, while Clay’s support team is building “expert generalists” who can move across functions.
The hosts argued that neither companies nor governments have developed a response proportional to the displacement their leaders predict. A listener proposed taxing AI to redistribute concentrated gains and slow deployment; Kevin cited Dario Amodei’s “token tax,” while Casey insisted elected officials—not corporations—must prepare the safety net. Their common conclusion was that the effects are visible now and likely to accelerate.