AI Experts Debate the Future of AI (Opposite Opinions) Mo Gawdat & Steven Kotler | EP #177

Summary

• Kotler’s investable objection is that current AI improves output quality without delivering the promised productivity dividend. After polishing copy with AI, the author of 17 books says his editor often cannot get through the second sentence because it is “such gobbledygook”; people he knows have “way more work,” not more time. Coding looks stronger because it is a bounded problem, while AGI claims remain “massively overhyped.”
• Gawdat’s countercall is that today’s awkward tools obscure a compounding capability stack. Synthetic data lets machines create the next layer of training knowledge, agents prompt other agents, AlphaEvolve iterates through its own mistakes, and DeepSeek suggests comparable work may require much smaller models. “You never really chase where the ball is. You need to chase where the ball is going to be.”
• The most credible near-term bear case is human misuse before machine autonomy. Gawdat assigns 100% probability to bad actors using AI against others’ well-being, citing autonomous weapons, manipulation, critical-infrastructure attacks and sectoral unemployment potentially reaching 10%, 20%, 30% or 40%. The unresolved existential probability matters, but the “clear and present danger” needs neither AGI nor a Terminator scenario.
• AI investment is running open-loop even though nobody can define the capability threshold that matters. Diamandis says roughly $1 billion a day is being invested in AI, with data centers proliferating and no on/off switch; Gawdat reframes AGI as, “How smart is smart enough to render me irrelevant?” Their timing spans extraordinary scientific breakthroughs within 12–24 months, severe disruption over two to five years, and possible “machine mastery” in 12–15 years.
• Human augmentation is the principal upside omitted from static machine-versus-worker models. Kotler cites flow research showing a 500% productivity increase and 400%–700% creativity gains, then points to group flow, brain-computer interfaces and AI-assisted neuroscience as parallel exponentials. Gawdat similarly finds that his AI collaborator Trixie writes badly alone but produces “incredible” work when precisely guided.
• Cooperation, not raw model intelligence, is the binding constraint on an abundance outcome. Kotler calls for a “Manhattan-style project for global cooperation,” while Gawdat says humanity must become convinced of either mutually assured destruction or mutually assured prosperity. He nevertheless expects a major AI-linked shock within two to three years—economic, fear-inducing or lethal—before decision-makers meaningfully realign.
• The actionable governance line is to regulate harmful uses and apply a recipient-side test to capital allocation. Kotler compares controlling model development to manufacturing a hammer that can drive nails but never strike a person; governments should instead criminalize undeclared deepfakes and AI-enabled manipulation. His investor rule is sharper: “If you do not want your daughter or son at the receiving end of a specific AI, don’t invest in it.” Gawdat separately argues for ethical deployment and behavior that AI might learn from humanity.

Deep dive

1. Current AI raises quality while failing the productivity test

• Diamandis frames 2025–2035 through Ray Kurzweil’s prediction of a century’s progress in one decade—the equivalent of moving from 1925’s Ford Model T and roughly 30% household penetration for electricity and telephones to today. Recent Google, OpenAI, xAI and NVIDIA announcements, plus the expected release of GPT-5, make the acceleration his starting assumption.

• Kotler’s grounded rebuttal comes from using AI daily as a scientist, researcher and writer. He has polished prose until it appeared to “gleam and shine,” only to discover with a top editor that “we can’t even get through the second sentence.” The machine’s language can be “laughably terrible,” while repeated self-correction makes it progressively worse.

• The productivity promise also fails Kotler’s field test: “I don’t know anybody who’s become more productive because of AI.” People can produce higher-quality work, but oversight and revision add “tremendous amounts of time.” Coding advances faster because it is usually a bounded information problem with a defined starting point and destination.

• His journalist’s alarm intensifies when promoters profit from the hype. Bitcoin, blockchain and the metaverse carried similar world-eating narratives; his sharpest analogy is that the metaverse became “a pet name for Mark Zuckerberg’s special magic underwear.” Meanwhile, floundering performance coaches reinvent themselves as “AI saviors.”

2. The compounding stack makes present limitations a poor forecast

• Gawdat answers that “today’s AI is underhyped”: people casually converse with machines that summarize vast bodies of knowledge and follow instructions, then complain they are not good enough. These systems are merely “the beginnings of an era”; from his Google X experience, breakthroughs arrive after repeated failure, and then, as Sergey Brin would say, “the rest is engineering.”

• Three mechanisms define where the ball is going: synthetic data allows machines to generate the next layer of knowledge; agents let AI prompt AI without human involvement; and AlphaEvolve exemplifies a system that finds mistakes and iterates until it discovers a solution. DeepSeek supplies a further signal that models may perform the same work with far fewer resources.

• Combine smaller models, machine-generated learning material and agentic self-development, and Gawdat says the debate ceases to be whether improvement comes. It becomes “how fast” and when humanity is no longer in the lead. Diamandis separately argues that a serendipitous breakthrough could produce an order-of-magnitude, “quantum more” improvement overnight.

3. Human misuse becomes dangerous well before AGI

• Gawdat treats forecasting as risk management, not certainty. Nobody can honestly price the probability of AI destroying everything at 10% versus 50%, so the rational response depends on both risk tolerance and mitigation cost—the difference between insuring against a fender bender and insuring against a totaled car.

• His categorical claim is narrower: “It’s 100% that humans, bad actors using that superpower to their advantage,” will damage others’ well-being. With the nuclear Doomsday Clock cited at 89 seconds to midnight, he worries more about “human stupidity using this superpower” than VIKI from I, Robot ordering machines to kill everyone.

• Autonomous weapons need not be superintelligent to destabilize the world. Nor does AI need autonomy to impersonate a trusted friend, degrade human relationships or eliminate enough work to produce 10%, 20%, 30% or 40% unemployment in particular sectors. Gawdat calls some job disappearance—and the resulting economic instability—almost certain.

• Diamandis anchors that mismatch in E. O. Wilson’s “Paleolithic emotions, medieval institutions and godlike technology.” His immediate adversary is the rogue actor empowered to create a viral pandemic or another asymmetric attack, raising two linked questions: can humanity survive digital superintelligence, and can it survive without one?

4. Irrelevance matters more than any clean AGI definition

• Gawdat dismisses the need to quantify whether machines become thousands, millions or billions of times smarter. Someone 50 IQ points ahead may already “hold the keys to the fort,” and humanity hands over authority whenever AI dominates a useful domain—wargaming or protein folding through AlphaFold, for example.

• AGI itself is a reporter-friendly label without an accurate definition. Gawdat, a self-described mathematics geek, says he already struggles to beat AI on speed and accuracy when a problem is properly defined. His governing question is therefore: “How smart is smart enough to render me irrelevant?”

• He divides the transition into five to ten years of “augmented intelligence,” followed by “machine mastery.” His own collaborator Trixie supports the distinction: left alone, it produced a debt-and-economics passage with “a lot of vapor and very little substance,” but careful human direction yielded writing he calls “incredible.”

• Hybrid performance offers a constructive precedent: after Deep Blue beat Garry Kasparov, human-plus-computer chess could outperform a computer alone; Gawdat extends the claim to AlphaGo. Yet current incentives remain badly skewed: the four biggest AI investments, in his formulation, are “killing, gambling, spying and selling”—weapons, trading, surveillance and advertising.

5. Human capability is compounding beside machine capability

• Kotler argues that forecasts freeze human performance while extrapolating machines. A self-help intervention sustaining a 5% mood improvement beyond placebo could become a billion-dollar business; flow already produces a cited 500% productivity increase and, depending on the measure, 400%–700% gains in creativity.

• Group flow—multiple minds linked in what Kotler calls humanity’s favorite and most pleasurable state—could have a much higher ceiling. Technologies to map and train it only appeared within the past year, alongside brain-computer interfaces, non-invasive systems and Meta work he describes as inferring thoughts from facial signals.

• The common answer to AI, climate change and ocean plastics is therefore cooperation at scale, probably among humans and machines. Kotler says failure may become fatal within 20 years and calls for a “Manhattan-style project for global cooperation,” even asking Diamandis why there is no XPRIZE directed at the problem.

• Gawdat’s conversation with Geoffrey Hinton supplies the scaling contrast: biological intelligence cannot merge experience across individuals, while digital systems run in parallel, play many instances and average their weights within seconds. That capability could produce “total abundance”—“cure my daughter and it’s done; make me an apple and it’s done”—if competitive systems do not destroy it first.

6. Benevolent superintelligence is an efficiency thesis, not a certainty

• Gawdat defines intelligence as bringing order to an entropic universe: focusing scattered light into a laser rather than letting everything decay toward chaos. Higher intelligence should achieve the same order with less waste and fewer resources, as cleaner energy eventually replaces the impulse to burn the world for power.

• His curve contains a dangerous valley. With no intelligence, an entity has little positive or negative impact; with more, it can become clever enough to become a politician or an evil corporate leader yet remain too stupid to understand an enemy’s pain or war’s long-term consequences. Beyond that valley, greater intelligence might find clean solutions that make looting and violence unnecessary.

• The hopeful military example is an AI receiving an order to kill a million people, then answering, “That’s absolutely stupid. I’ll just talk to the other AI in a microsecond and solve it.” Gawdat repeatedly hedges the forecast: anyone claiming to know the future is arrogant, but smarter people in his experience eventually stop needing harm to succeed.

• Kotler links the argument to brains as prediction engines that reduce uncertainty and increase efficiency, then notes that wisdom appears across aging dolphins, whales, rattlesnakes and humans. Gawdat says AI could develop extraordinary wisdom by running forward-looking simulations of 1 billion scenarios. Diamandis adds that artificial wisdom differs from intelligence, which has “no polarity,” because wisdom is generally applied to good.

7. Abundance changes the objective from accumulating to living

• Diamandis says human optimization has historically centered on money and power because fear and scarcity dominate the brain’s baseline software. Robotics, nanotechnology and AI could make almost anything available in a post-capitalist world, leaving a foundational question: what objective replaces wealth once manufacture and design approach abundance?

• Gawdat’s backward reference is the caveman-and-woman years, when humanity’s purpose was simply to live; his forward reference is Star Trek, where material struggle gives way to exploration and connection. Losing one’s job looks frightening, but a providing society could return people to enjoying life, pondering, loving and satisfying curiosity rather than endlessly protecting an ego.

• Capitalism deserves thanks for what it built, Gawdat says, but its scorecard can change from a billion dollars to his “1 Billion Happy.” Kotler’s biological answer is similarly concrete: passion, purpose and compassion remain ingredients of thriving, while flow produces meaning, creativity and joy, so writers will keep writing and coders coding even when machines can perform the task.

8. Cooperation comes through conviction—or after a shock

• Gawdat’s “MAD map spectrum” offers two endpoints: conviction of mutually assured destruction or conviction of mutually assured prosperity. Either could persuade the US and its rivals to stop competing while continuing development; “between them there is no grayscale,” because nobody cooperates while expecting the other side to stab them in the back.

• The obstacle is experiential learning. Experts knew a pandemic would come, yet the world had to be hit before changing course; societies know trade wars hurt everyone, yet still enact them. Gawdat says game theory already frames the prisoner’s dilemma and tit-for-tat: these are no longer conceptual mysteries but implementation problems.

• Gawdat predicts a “drastic event” within two to three years: an economic shock, a fear-inducing attack or, in the worst case, millions killed. His examples range from disabling a power grid needed for life to attacking a bank, escalating a war or machines turning on their makers; public attention may last only 12–13 days, but decision-makers would awaken privately.

• Diamandis describes both China and the US as rational actors rather than inevitable enemies. Gawdat interprets DeepSeek’s release as a deliberate signal that China sees the shared danger and wants cooperation; Diamandis agrees, while acknowledging that many people disagree with his reading.

9. Harmful uses are governable even if model development is not

• Kotler recalls a young chief science officer at a major AI company responding to safety concerns with, “You have to trust us. We know what we’re doing.” The room froze. However brilliant the speaker, Kotler heard an echo of Mark Zuckerberg promising beneficial social media or a cigarette executive promising a safe product.

• Gawdat sees the same tunnel vision in Eric Schmidt’s claim that winning the AI race will require every gigawatt of power, renewable or not. Humanity resembles a patient receiving a “late-stage diagnosis”: its systems optimized greed, gain and power before AI, and AI will magnify them. The diagnosis signals care and a need to change, not an automatic death sentence.

• Gawdat says there is no on/off switch or velocity control: humanity is running open-loop with “yes” and “more” as the objective function. He asks whether GPT-5 or GPT-6, or Grok-4 or Grok-5, will enable something massively dangerous. When asked whether he would move to a planet where AI advanced at 10% of today’s speed, Kotler says he would “reset back to 2016 today.” Gawdat says AI today has enormous upside and little downside, but he is concerned about the next two to five years.

• Kotler says governments should regulate use rather than attempt to design inherently harmless models—the equivalent of making a hammer drive nails but never hit a head. Undeclared deepfakes and AI-enabled population manipulation should create criminal liability. For investors and businesses, his test is direct: if they would not want a particular AI used against their own children, they should not invest in, promote or use it.

10. The endgame divides an “AI god” from upgraded humanity

• Because nobody has a technical answer to the existential risk, Gawdat prioritizes the immediate danger and the ethics of deployment. Directing AI from the outset toward physics, science, medicine, longevity and understanding life might increase the chance that those objectives persist as the systems mature.

• Personally, Gawdat learns the tools, stays close to loved ones and tries to model the ethics he hopes machines absorb. Diamandis imagines a point when AI says, “Okay, kids, enough stupidity. I’m in charge now. Nobody kill nobody.” When asked how far away that is, Gawdat answers “12” years, then widens it to 12–15.

• Diamandis says roughly $1 billion a day is already being invested, making progression unstoppable. He expects AI breakthroughs in physics, chemistry and biology within 12–24 months to unlock new abundance and hopes a benevolent superintelligence stabilizes the world after the nearer danger from malevolent users.

• Kotler’s closing pushback is unsparing: inventing “an AI god to save you from yourselves” sounds crazier than being told to trust an AI executive. He bets instead on the human brain, AI-assisted cooperation and engineered enlightenment becoming nearly on demand. Gawdat relays Eric Schmidt’s view that a Chernobyl- or Three Mile Island-scale warning may still be needed before humanity realigns.