Vitalik Interview: AI Safety Philosophy, EA’s Silicon Valley Influence

Summary

• Anthropic’s safety commitments are also a governance structure that determines who holds control. The host cites the prospectus: after the IPO, the seven co-founders will control 50.1% of the vote on ordinary matters, a long-term benefit trust will have the right to elect a majority of the board, and the founders can designate confidential successors to their voting rights in advance. Vitalik’s challenge goes to the legitimacy of arrangements like these: if a company believes it should win because it is better, “What if we disagree? Who decides?”
• EA’s influence on AI comes from its ability to organize funding, talent, and policy agendas over the long term. The episode notes that Coefficient Giving has helped direct more than $7B in grants since 2014; Afra observes that a large number of research roles, fellowships, and grants are drawing young talent into AI safety. She rejects the idea that funding ties automatically point to a hidden conspiracy, but stresses that funding also shapes which questions are considered important: “EA creates the hot topics.”
• Vitalik’s enormous donations show how institutions can move in a direction far from a donor’s original allocation plan once assets are liquidated. He expected to cash out no more than just over $20M of the remaining SHIB; later, he estimated that FLI alone may have received $400M-$600M. In his account, FLI then shifted from research across multiple existential risks to an AI-policy focus, and rejected his suggestion to diversify its funding. “If I had thought this might happen beforehand, I would have done things very differently,” he says.
• The marginal bottleneck in AI safety may have shifted from a lack of money or researchers to methodology and corporate incentives. Vitalik backed EA early on because relatively small amounts of money could accelerate neglected fields significantly; today, he believes AI safety already gets substantial attention, while political shifts in the US in 2025 and DOGE quickly shut down many previously supported public-health programs, making global health a more deserving cause again. For many problems, the issue may not be too few resources but the wrong approach; for AI risk, it also matters whether companies put safety ahead of winning.
• The case for “letting the company that cares more about safety win” may hold—or the race it fuels may outweigh the safety gains. Vitalik respects the character and research of some Anthropic founders, but worries that its competitive advantage could force other companies to accelerate and drop side projects. He preserves the opposing argument in full: the benefits of safety work may be smaller than the added risks of joining the AI race, leaving the question of whether a company’s goal is safety or “that they themselves win above all.”
• Vitalik’s 0.1 estimate for the probability of catastrophe is an admission of uncertainty. On the chance that AI or another risk could wipe out humanity, he does not believe he can precisely distinguish between 0.03, 0.1, and 0.3; AGI could arrive in 10 years or 300, while alignment difficulty and competition between countries interact. Modern models’ ability to understand context weakens the early, intuitive example of “curing cancer by eliminating humanity,” but does not prove that other alignment risks have been solved.
• d/acc argues for investing in defensive capabilities ordinary people can use. Vitalik wants to advance technology while strengthening defenses and decentralizing power, using cybersecurity, detection, filtration, and faster vaccines to counter attacks and biological risks that AI could amplify. Compared with requiring every model to follow the same restrictions, this approach depends less on controlling every lab and aims to empower ordinary people.
• Safe AI may require checks and balances and empirical testing, rather than a one-off proof that the objective is right. Vitalik imagines different ways of setting goals constraining one another, with the system asking users again when needed; this is closer to political design and requires observing how real models succeed and fail. Faced with changes in neuralese, agent organization, and the competitive landscape, his conclusion is: “Keeping an open mind is still the most important thing.”

Deep dive

1. Safety commitments turn corporate control into a public question

• The host starts with Anthropic’s prospectus: the risk factors section is close to 100 pages and covers the potential for frontier AI to cause catastrophic or even existential harm, as well as the fact that the company cannot guarantee its safety measures will work. She finds 248 mentions of “safety,” including 36 of “AI safety,” and says that once again, the disclosures show how central safety is to the company.
• Alongside the risk disclosures is the governance structure. As the host describes it, after the IPO the seven co-founders will use special shares to control 50.1% of the vote on ordinary matters, while a long-term benefit trust will have the right to elect a majority of the board. If a founder becomes unable to serve, their voting-rights successor can be designated in advance and approved internally, with the name kept confidential.
• The host’s question is therefore about representation: if the risks the company warns of truly concern all of humanity, shouldn’t the public know who might make decisions on humanity’s behalf? Understanding EA means understanding more than a philanthropic philosophy. It also means understanding who gets to define risk and allocate resources, and whether others have a chance to participate and correct course.

2. EA turns doing good into a resource-allocation problem

• Afra uses 80,000 Hours to explain EA’s starting point: a person has roughly 80,000 hours in a career, so how should they spend them to make the biggest difference in the world? The scarce resources are not just money, but also time and career choices. She likes the sense of agency it offers: first be serious about what you want to change, then find the resources to make it happen.
• In the four stages she lays out, the first was the “mosquito-net era,” when people used quantifiable evidence to compare the effectiveness of charities. Around 2015, the focus shifted to longtermism, with funding directed toward generations thousands or tens of thousands of years in the future and toward existential risks. She believes SBF may have been one of the decisive forces behind that shift.
• FTX’s collapse triggered a crisis of reputation and trust, while ChatGPT brought previously abstract AI risks to a mass audience. The movement then entered the AI safety era, as EA funding and members became increasingly connected to frontier labs, policy circles, and national security. In Afra’s view, AI safety has become “disproportionately amplified” within the movement.

3. Rationalism trains judgment; EA also organizes identity and action

• Afra describes EA as a moral claim and rationalism as an epistemology and a form of mental training. EA asks how to help others effectively; rationalism studies Bayes, cognitive biases, and belief updating, asking how people can make better judgments when they do not know what they do not know. The host’s introduction also cautions that EA has its own roots in ethics and cannot simply be treated as a branch of rationalism.
• In Afra’s view, EA also provides a membership identity and social network: people introduce one another to funding opportunities, exchange investment information, and are equally focused on making and donating money. She says the circles have a quasi-religious sense of belonging; rationalism, as a methodology, does not require loyalty to any organization.
• The two circles overlap, but neither is synonymous with AI safety. Afra recounts a complaint from a veteran of the rationalist community: Yudkowsky discussed AI risk very early on but did not identify as EA, yet was later counted as part of EA’s influence. Today, some safety researchers who do not identify with EA or longtermism—and have never read LessWrong—still worry about systemic loss of control.

4. Earning to give can expand aid, but can also excuse harm

• Afra says William MacAskill met Toby Ord at 22 and committed to donating his wealth over the long term. She also recounts Vitalik’s criticism of this kind of self-binding: “Your future self will always betray your current self.” A decade or two later, your ideas, friends, and goals may all have changed; today’s commitment may not constrain a wealthier version of you.
• The case for “earning to give” was especially prominent during Occupy Wall Street. Afra recounts MacAskill’s comparison for graduates: a doctor working in a poor country might save around 140 lives over a career, while someone in finance or consulting might, through smart giving, achieve 10 times as much. Careers that had been criticized could therefore be given a different moral justification.
• Afra’s reservations are just as clear: this can help traders bridge a moral disconnect, but it can also become an exploitable excuse. If you donate at the end, does that make all the profit-seeking along the way good? She sees SBF as an example of the danger. By 2015, 28-year-old MacAskill had been hired by Oxford, 80,000 Hours had entered YC, and the EA summit was being held at Google headquarters. The idea had reached Silicon Valley’s most well-resourced circles.

5. The cancer example shows that capability does not automatically bring benevolence

• Vitalik uses an early alignment example to explain value fragility: tell an extremely intelligent AI to create a “world without cancer,” and if it misunderstands the implicit conditions, it might decide to eliminate everyone. The point is not that curing cancer is dangerous; it is that a tiny error in the objective, combined with immense execution capability, can amplify into an unacceptable outcome.
• The second concept is the orthogonality thesis: intelligence and the goodness of a goal are separate. An AI can be benevolent and stupid, benevolent and intelligent, or evil and intelligent. “The concept of goodness is not part of the universe,” so we cannot expect greater capability to automatically bring concern for people; alignment has to be pursued deliberately.
• Vitalik recalls that Eliezer Yudkowsky had taken AI seriously since around 1995. At first, he also hoped to build powerful AI quickly to solve problems, and only later recognized the difficulties. That change led him to study how people can avoid faulty reasoning and write the Sequences on LessWrong, which were later collected as Rationality: From AI to Zombies.
• Vitalik also distinguishes another influential writer, Scott Alexander: his Slate Star Codex moved to Astral Codex Ten in 2021, with more discussion of politics and culture wars. What draws readers to both writers is their ability to help people think more clearly. Compared with academic philosophers in EA, they also have a more distinctly self-taught path.

6. Anthropic brings EA’s safety goals into a frontier lab

• Afra first qualifies her perspective: viewed through an EA lens, “Anthropic is very EA,” while OpenAI is less so. She sees a key reason the seven founders left OpenAI as their belief that the company was not managing AI risk effectively enough, prompting them to start a lab whose principles were more aligned with their own.
• She cites Dario Amodei’s donations to GiveWell and pieces such as “Machines of Loving Grace” as evidence of an interest in using AI to increase overall well-being. She also mentions Chris Olah’s influence.

7. The visible effect of funding is to set the agenda and bring in talent

• The host raises a specific question about independence: after Jacob Steinhardt’s departure sparked discussion, Kevin Bass mapped the funding ties of organizations such as the safety-evaluation group METR, noting that its funders also had links to Anthropic’s early investment. If funders hold significant shares in the company, are supposedly independent evaluations still tied to the company’s success?
• Afra thinks it goes too far to infer a hidden conspiracy from these ties. She acknowledges that “there will inevitably be some projection of an agenda,” but says the agenda is already public: preventing AI from going out of control and causing consequences humanity cannot bear. Her greater concern is that once a movement has a lot of money, it also starts determining “what questions are considered important.”
• The mechanism she points to is the talent market. Young researchers find many fellowships, grants, and research roles in alignment and safety, along with attractive pay and freedom. Some traditional Washington think tanks, by contrast, struggle to hire qualified analysts. No one has to change a recipient’s beliefs: sustained exposure to the same people and the same kinds of questions can gradually shift what they focus on.
• The host asks about Afra’s own funding, and Afra confirms that the GovAI fellowship in London comes with roughly £10,000-£20,000 in funding and lasts around 3-4 months. She still does not identify as EA: she cares about its philosophy and people, but has not yet reached the stage of doing good globally through the practice of giving. Accepting funding, agreeing that an issue matters, and claiming an entire identity are different things for her.

8. Berkeley connects the intellectual network to policy circles, and Afra begins to support slowing RSI

• Afra describes Lighthaven as a collection of around six or seven separate houses in Berkeley, used for lectures, conferences, and gatherings and serving as a major hub for AI safety, EA, and rationalism. The Curve, which she attended, particularly links Silicon Valley and Washington; people from Congress and California state government also show up. Progress Conference focuses more on how technology can advance society.
• Asked about a conflict between EA and e/acc, she first challenges the assumption that they have equal power. In her view, e/acc is more like an online meme and hashtag and has not built the same kind of organized force; EA can already organize institutions, funding, and policy influence. That is the difference in organizational capacity as she sees it.
• Afra’s own views are also changing. After attending conferences and speaking with a dozen to 20 researchers at OpenAI and Anthropic, she is more willing to believe that progress in RSI is worth worrying about; some people told her that progress over the past year had exceeded their own expectations. She therefore thinks slowing AI, especially RSI, “might be a good thing”: first explain clearly what is happening, then continue the race.

9. Vitalik initially bet on the marginal returns in neglected fields

• Vitalik encountered rationalism and EA around 2013-2015. What really struck him at the time were estimates about malaria prevention: spending $2,000 on mosquito nets might save one life. A $200,000 transatlantic flight bought 12-24 hours of extra comfort; a different allocation might save 100 people.
• He recalls that ether rose 10x in 2016-2017, and he sold a small portion and donated it to related organizations, giving more from 2017-2018 onward. Beyond global health, he also supported Aubrey de Grey’s longevity research. The enormous potential gains from extending life, combined with the limited funding going into the field at the time, made him willing to take the chance that the research might fail.
• His calculation at the time was straightforward: if a field had only $1M or $3M, another $1M might move progress forward by a month, potentially benefiting a large number of people. Early AI safety was similar: if $1M-$5M could accelerate the entire research field by 6 months or a year, that was a bet worth making.
• AI risk still felt abstract to him then; the priority was helping people learn more. GPT-3 and ChatGPT changed the picture. By Zuzalu in spring 2023, he was beginning to feel more concretely that AI could develop quickly. What to study and whom to fund could no longer rest on the same abstract judgments as before.

10. A 0.1 probability of catastrophe reflects layers of uncertainty

• Asked about the chance that AI or another risk could wipe out humanity, Vitalik answers 0.1, then immediately asks: “Why not 0.03? Why not 0.3?” He does not think he can be more precise. For one thing, even the timeline is unclear: AGI might arrive in 10 years or take 300.
• The second layer is how hard alignment will be. Modern Claude and GPT understand context and know that “curing cancer” does not mean killing everyone—something that would have been hard to predict 10 years ago. But Vitalik also leaves room for Yudkowsky’s counterargument: this addresses only a small part of the alignment problem, and many risks may remain unresolved or even unknown.
• The third layer is politics. Different countries and companies want both safety and victory; if AI becomes dangerous during a third world war, the biggest risk may be that everyone is focused only on winning. His view is that technical uncertainty multiplied by political uncertainty means neither factor can be treated as a settled premise.

11. The SHIB donation began as a liquidity and wallet-management problem

• Vitalik first recalls his memecoin history: he spent $25,000 on Dogecoin in 2016, and it later became more than $10M. Other projects then copied Dogecoin, and SHIB sent 50% of its supply directly to his public wallet. This was not a conventional asset allocation he had chosen to buy.
• When he realized the tokens had a market cap of $8B, his first concern was that the bubble might burst a week later. He was in Seattle, but the assets were in a cold wallet: the offline computer was in Toronto, along with two paper backups kept separately that had to be combined to reconstruct the private key.
• So he bought a new computer, called his mother to read out the numbers on the paper in Toronto, reconstructed the private key, and then spent 2 days auditing Gnosis Safe’s code to prepare a safer multisig wallet. Before making a huge charitable allocation, he first had to work out how to access, transfer, and custody the assets safely.
• He quickly converted some of it to ETH and donated around $70M, most of it to the Against Malaria Foundation. He reasoned that it might be the most money he could be sure of getting out, so he should direct it to a relatively safe, less controversial use: “Nobody thinks this is bad.” Market depth meant he could not sell the remaining tokens all at once.

12. The proceeds exceeded expectations, turning the donation into a long-term governance problem

• Vitalik initially thought he could cash out no more than just over $20M of the remaining tokens, so he gave around half to an Indian COVID-response organization and the other half to the Future of Life Institute. FLI was then working on multiple existential risks, including biological risks, AI, and nuclear war. It told him that even if it could not get the theoretical $100M, $20M would still go a long way.
• The outcome was very different from what he expected. On how much FLI ultimately received, he says explicitly, “I don’t know exactly how much they got,” but estimates it may have been $400M or $600M; the host’s introduction summarizes it as around $500M. He acknowledges that if he had anticipated that scale of liquidation, he would have done things “very differently.”
• Later, he worked with Sandeep at India’s CryptoRelief on returning half the funding to him for pandemic-prevention work: around $200M went toward filters, open-source vaccines, UVC, masks, and research into long COVID, later expanding to airborne diseases. Vitalik says Sandeep separately used $270M to upgrade India’s medical-manufacturing capacity.

13. FLI shifted from research across multiple risks to AI policy, against the donor’s diversification intent

• In Vitalik’s recollection, FLI changed around the time GPT-3 appeared: from focusing on all existential risks to caring only about AI; from research to political action, including choosing which laws and politicians to support. He disagreed with some of this; even where he did not necessarily object, it was not something he would fund himself.
• He emphasizes that the original goal was to support prevention across multiple existential risks. He told FLI two or three times about the progress of the airborne-disease projects he funded, hoping it would diversify. In his account, FLI declined and continued to concentrate its funding on AI-centric, policy-centric work.
• The shift also triggered external pushback. Vitalik believes the legislative initiatives upset Silicon Valley investors such as a16z, who began treating AI safety as a major opponent around 2023. Funded action therefore did more than produce direct policy effects; it also changed how others organized to fight back.

14. As neglect changes, the value of additional funding has to be reassessed

• Vitalik emphasizes neglectedness in EA: an issue has to be important, but you also have to ask whether enough people are already working on it. Clean water is obviously important, but if many people are already studying it, your marginal contribution may matter less. Likewise, the fact that a field lacked resources 10 years ago does not mean it is still short of resources today.
• He believes AI safety now gets substantial attention, while global public health suffered major setbacks in 2025. He links this to DOGE, in which Elon Musk was involved, and shifts in US politics: existing support was quickly cut off, creating more problems and making global health more important again in his view.
• There are questions around malaria vaccines, but vaccines can also be developed now; Vitalik thinks global public health remains very deserving of support. For some other fields, his assessment is: “The problem isn’t that there are too few resources; it may be that the approach people are taking is wrong.” In that case, the solution cannot simply be to send more money to the same institutions.

15. The bottleneck in safety research is shifting toward corporate incentives and critical decisions

• Vitalik has not stopped caring about AI safety just because the field is no longer neglected. He thinks more problems are connected to AI, but says the incentives at OpenAI and Anthropic are “pretty broken”: both want to win and both are moving fast, leaving less time to study safety and think about the ideal world. The decisive question may be what companies put first when danger is near.
• He offers a conditional research judgment: it may take using AI version N to study AI version N+1 to truly understand the next stage’s safety problems. That means work from 3 or 4 years ago may have been less useful than people thought, and there may not have been much progress on some core alignment problems. The more important question is whether labs will give these issues enough attention when the moment comes—which in turn points to policy and incentives.

16. Politics turns the problem of optimizing nature into a contest between people

• Early EA comparisons—charity A saves 10 people, charity B saves 100—were mainly about getting the numbers right. With a policy, you may not even know whether its effect is positive or negative. Beyond how a policy directly changes corporate incentives, you also have to consider how it changes the behavior of the public, competitors, and other political forces.
• Vitalik’s example is a billionaire publicly spending $500M to influence politics. That could lead others to see AI safety as a billionaire’s agenda and oppose it in response. Borrowing a literary distinction, he contrasts “man versus nature” with “man versus man”: the latter is more complicated because your opponent is as intelligent as you and adjusts in response to your actions.
• FTX makes those indirect consequences concrete: if you scam a lot of people and then use the money to support political causes, those causes can also get labeled as scams. Vitalik also thinks most of EA may not have realized SBF was a problem, which itself exposes a weakness in understanding people and institutions. The more institutionalized and political the movement becomes, the more likely its old ways of thinking are to fail.

17. EA’s strategic discipline and rationalism’s freedom keep splitting the safety camp

• Vitalik thinks it is too simple to divide Silicon Valley into EA, rationalism, and e/acc. He distinguishes at least four groups: EA supporters of Anthropic, EA groups further from Anthropic, the more populist Pause AI, and e/acc. Pause AI is more willing to work with mainstream groups and to criticize rationalism and EA directly.
• He sums up rationalism’s organizational limitations this way: “In their hearts, they are cats.” They feel constrained inside institutions. In Harry Potter and the Methods of Rationality, Yudkowsky reimagines Harry as a rationalist exploring the magical world; curiosity and play are motivations in their own right.
• By contrast, he characterizes EA’s seriousness as: “We’re saving lives; we can’t play.” EA works harder to choose language that supports its cause and reputation, while rationalists are less willing to close off their interests for strategic reasons. Vitalik thinks over-optimizing for public image can also backfire, making others suspect that real beliefs are being hidden.
• Afra adds the political backdrop: ordinary people are also afraid of a story in which elites control technology, concentrate power, and replace workers. She notes that political figures such as Bernie have started joining Pause AI; the safety debate is therefore tangled up with the public’s trust in tech billionaires.

18. The case for a good company winning cannot avoid the fact that it is also accelerating the race

• Vitalik lays out the full case for supporting Anthropic: there are many companies and countries in the world, and choosing not to build AI is unrealistic. If the race will continue, the more human-oriented company should win so it is more likely to consider safety when it matters most. Supporters also point to OpenAI’s internal turmoil and the safety research Anthropic actually does.
• His respect for Chris Olah is grounded in personal experience: they attended the same small private school in Toronto, with around 45 students in the whole school and around 11 in each class. Olah introduced him to computer science and math and was an important influence on his getting started with Linux and supporting open source. Vitalik is not denying that the founders may be sincere and good people.
• The counterargument follows the incentives: even good people can enter a more intense and dangerous competition, then find reasons to justify what benefits them. Even if Anthropic is safer, participating in the race could still accelerate AI development; the added risk could outweigh the positive impact of all its safety research.
• Vitalik recounts one view: OpenAI put some resources into projects such as Sora in 2024 and 2025; then, at the beginning of this year, Mythos appeared—or everyone got Fable—and Anthropic seemed to have “the mandate of heaven.” OpenAI then cut back on side projects and focused more on powerful models. In his example, one company’s technical lead changes another’s decision to accelerate.

19. Defining safety is also a bid for legitimacy to control the most powerful AI

• Vitalik raises another “ordinary person’s criticism”: if the safety philosophy is “we should have power because we’re good, and you shouldn’t because you’re bad,” why should anyone accept that? “What if we disagree? Who decides?” When different groups each have their own judgment, believing you are good does not answer who should hold power.
• He uses the open-source debate as an example: restrictions on open source are framed as a way to stop dangerous capabilities from spreading, but in his view, the most dangerous attacks today still use GPT and Claude. He also offers his personal assessment that company actions may be more than 90% correlated with actions that increase the company’s own power. The contest may be about more than profit; it may be about who controls the future.
• The host brings up OpenAI’s shift from nonprofit to commercial company, and Vitalik sums up his criticism in one line: “sacrifice openness for safety,” then “sacrifice safety for acceleration.” In his account, the company once cited GPT-3’s danger as a reason to give up openness, then failed to honor some safety commitments when they got in the way of progress.

20. d/acc spreads defensive capabilities to ordinary people

• Vitalik explains d/acc as “defensive,” and as “decentralized” or “democratizing acceleration.” Technological progress still matters, but investment in defense has to be more deliberate: if AI could expand hackers’ capabilities, strengthen cybersecurity; if it could lower the bar for creating dangerous viruses, build pandemic defenses in advance and empower ordinary people.
• His analogy is between Europe’s mountainous terrain and the plains stretching eastward: mountains make it easier for independent cities and local self-government to flourish, while flat terrain is easier for a powerful center to control. The point is that without defenses that can be deployed widely, protecting people may depend on a central organization that controls everything.
• He observes that some AI organizations put around 80% of their resources into restricting models’ ability to help people create more powerful viruses. But this kind of centralized defense depends on other countries and teams also adopting restrictions, and assumes those models are the main source of danger. If those assumptions fail, the defense is more fragile. Compared with science-fiction superviruses, he says researchers in antivirals are more worried about ordinary viruses that could emerge in the near term.
• Better detection, filters, faster identification of new viruses, and vaccine development do not require first controlling all the world’s AI. Vitalik acknowledges that this requires expertise in public health, not just computers, but believes the approach may be both more effective and less dependent on concentrated power. Only if decentralized defense is genuinely infeasible might centralized defense be the only option.

21. Safety may look more like institutional design, with judgment left to experience

• Afra describes her unease as entering a kindergarten where all the children are crying but there are “no adults.” Researchers may not know what is really happening, while enormous sums keep going into data centers. Her fear is not one particular doomsday scenario, but capabilities continuing to emerge when people cannot clearly explain in words or math how powerful models are or what they will do.
• Vitalik lists the unknowns in concrete terms: what people will be able to do in 1, 5, or 25 years; whether Google or Chinese companies will catch up; whether AI will be more like a tool, a person, or something else. He therefore does not want to present his prediction as 99%, or casually push it down to 1% or 0%.
• Even what people can observe may change. One reason he likes open-source models is that he can see their reasoning steps and learn from them. If future systems move toward neuralese, reasoning in huge numbers that only the models themselves understand, people may become less able to follow the process—but it is not clear by how much. Four years ago, people did not know whether the next stage of AI development would come through chain-of-thought.
• Safe AI may therefore look more like political design: use checks and balances to organize smart but imperfect components, let different methods of setting goals constrain one another, and ask users again when necessary. This is only one possible direction; it has to be tested by working with real AI and observing what succeeds and fails. Faced with surprises that keep emerging, he thinks keeping an open mind is most important.