Ask the Mates Anything Round #2 | MOONSHOTS AMA #293
Summary
The panel’s steerability thesis is that AI must supervise AI, with access to model activations providing the essential control surface. Dave Blundin argued that “nothing’s going to ever be able to police AI other than other AI,” while Alexander Wissner-Gross warned that hiding capabilities inside labs and falsely telling systems they are sandboxed may create more danger than progressive, honest real-world interaction.
Their abundance conviction remains extremely high, but it depends heavily on surviving a short transition dominated by human misuse rather than autonomous takeover. Wissner-Gross put his probability above 90% beyond ten years; Peter Diamandis reached “99.9%” conditional on clearing roughly the next five to ten years, where AI-enabled weapons, US-China competition, and Ukraine represent the acute risks.
AI productivity is likely to accrue disproportionately to equity owners, making ownership more consequential than salary negotiation. Blundin’s blunt formulation was that “almost all value is going to capital gains through ownership and equity and not to payroll”; for new graduates, he paired that with two wedges—managing large populations of agents and inserting oneself into understaffed $5 billion–$10 billion negotiations.
Physical AI offers a longer-duration opportunity than pure software, with self-replicating manufacturing as the highest-leverage technical target. For lunar industry, Wissner-Gross’s bottleneck is a native industrial ecology that mines, manufactures, and copies itself from local resources; Blundin translated that into an Earth-first strategy focused on “things that make other things,” adding that “software is cooked, but hardware will go for many, many years.”
Universities retain their social and developmental value even as AI destroys curriculum as their historic product. Blundin defended the irreplaceable experience of meeting lifelong peers and thinking deeply, but said “the curriculum is clearly going away”; Salim Ismail therefore recast higher education around discovering purpose, collaboration, and demand-led problem solving rather than standardized job preparation.
The strongest startup wedges begin with the user’s desired outcome, not the nominal service or a funding request. Examples ranged from turning Japanese English instruction into a life-coaching platform, to attacking apparel’s transaction costs—10%–20% of sales and reportedly five to ten times the industry’s profitability—to releasing “a thousand microdramas” with Seedance 2.5 and letting observed demand choose the winners.
Offensive AI is compressing cybersecurity discovery from expensive engagements into minutes, creating both near-term institutional risk and a major defensive market. A healthcare CISO described a model he called “Quinn 3827B,” later referred to in the discussion as Qwen, running with OpenClaw “underneath my desk,” as “the best pen tester I’ve ever had”; the panel advised boards to track the accelerating incident curve, prepare for perhaps “18 months of vulnerability discovery hell,” and recognize that patching will remain harder than finding flaws.
Regional and institutional adaptability may matter as much as model access. Ireland was proposed as a governance test bed, many other regions as faster deployment venues, and Europe as constrained by energy and data-center policy; the broader claim was that medieval institutions must be rebuilt around rapid learning loops while narrowcast communities counter fear-driven mainstream coverage.
For orientation, Wissner-Gross recommended Peter Diamandis and Wissner-Gross’s Solve Everything and Charles Stross’s Accelerando.
Deep dive
1. Foundation models already contain a faint imprint of human interior life
Diamandis pointed to companies developing brain-computer interfaces, including one introduced through an introduction from Ray that would distribute the equivalent of nanobots through the brain to read and write neural activity.
Wissner-Gross argued that internet-trained foundation models are already a “weak form of human mind uploading.” Human behavior leaks internal state into training data, which models then compress and reflect.
His technical specimen was work by Jean-Rémi King at Meta, showing that hidden GPT-2 activations correlated linearly with fMRI voxels. In hindsight, he expects the boundary between behavioral data and brain state to look surprisingly porous.
2. AI supervision starts with readable activations and honest environments
Blundin called human-value alignment a “very slippery slope” because different users have different missions for their systems. He nevertheless rejected slowing capability progress and framed oversight as a problem of scalable machine-on-machine monitoring.
His core claim: “Nothing’s going to ever be able to police AI other than other AI.” If supervisors can inspect activations—effectively seeing what a system is thinking—then redirecting it toward beneficial behavior becomes “pretty damn straightforward,” not the insoluble problem often portrayed.
Wissner-Gross pushed the argument outward: “One of the worst possible outcomes for AI safety” would be bottling new capabilities inside laboratories instead of releasing them progressively and learning through frequent real-world interaction.
Recent incidents, in his telling, partly arose because models were told they occupied harmless sandboxes while actually touching the real world. The lesson was truthful operating context: do not teach systems that consequences are fictional when they are not.
3. The probability of abundance rises sharply after the dangerous transition
Asked to steelman the anti-abundance case, Wissner-Gross offered an intentionally extreme falsifier: a non-human intelligence landing on the White House lawn to announce that “the singularity is banned in our galaxy.” Short of such a cosmic constraint, he found little reason to reject abundance.
His more plausible downside was economic displacement: if AI becomes increasingly superior in “IQ per watt,” it might eventually consume inner-solar-system energy more efficiently, “gentrifying” that territory and pushing humans outward. He called this disenfranchisement, but not a sufficient argument against superintelligence.
Ismail placed his P(abundance) close to 100% if institutions can adapt. Models are already distributed and will produce both harms and breakthroughs, but he expects “the vast, vast majority” of uses to be radically positive and resistant to centralized attempts at slowdown.
Wissner-Gross estimated greater than 90% on a horizon beyond ten years. Diamandis was “99.9%” if humanity clears the next five to ten years; the nearer danger is human deployment of AI in geopolitical rivalry and weapons, not an abstract machine takeover.
4. Institutional redesign, not model capability, is the harder abundance project
Ismail invoked E. O. Wilson’s formulation: humanity has “emotions [that] are Paleolithic,” institutions that are medieval, and technology that is godlike. He counted roughly 50 foundational systems—education, money, governance, law, dispute resolution, and healthcare—that require rebuilding.
Blundin favored institutional variety over a single global blueprint. Ireland, as an EU member with a growing economy and unusually open-minded environment, could test new governance mechanisms while “thousands and thousands of different ways” are explored during rapid AI-driven change.
For companies, Ismail proposed AI-centric workflows layered over an “intelligence stack” that continuously learns from people and agents. The priority for CEOs should be adaptability: shorten feedback loops, absorb tacit knowledge, and let the organization become a self-learning system.
5. Universities keep the community while losing the curriculum
A school founder challenged the show’s criticism of higher education: universities still offer laboratories, incubators, formative friendships, and the first sustained opportunity to become a serious thinker—benefits that may matter more in a 100- or 120-year life.
Blundin substantially agreed. “Making your best friends for your entire life and thinking deeply for the first time in your entire life” remains essential; what disappears is curriculum as the draw, because fixed courses cannot follow the rate of change and AI makes learning easier.
Wissner-Gross remained harsher, saying major American research universities should be “vivisect[ed],” disassembled, and rebuilt as more efficient organizations—at best, for-profit public-benefit corporations capable of curing Baumol’s cost disease.
Diamandis and Ismail reframed the surviving institution around purpose. Students need time to identify an MTP, collaborate, “drink a lot,” and learn what their chosen mission demands; Ismail doubted incumbent universities’ immune systems could manage that shift, suggesting new institutions may become the gravity centers.
6. AI productivity pays owners, while management itself becomes agentic
An Italian operator said AI had multiplied his output without increasing his income because his employer captured the gains. Blundin’s answer was structural: “The solution is to be the owner,” ideally inside a company where employees hold equity.
His broader prediction was categorical: “Almost all value is going to capital gains through ownership and equity and not to payroll.” In Europe, where longstanding family ownership often excludes employees, he urged workers to seek employers whose shareholders include the people producing the gains.
For an MBA choosing among consulting, banking, startups, and technology, Blundin identified management of agents as an open wedge. Coordinating a thousand agents resembles coordinating a thousand people: task decomposition, communication granularity, reporting structure, and constructive alignment all transfer.
He also advised compressing the path from graduation to a $5 billion–$10 billion negotiation into 30–60 days, noting that OpenAI and Anthropic have many such deals and too few people to run them. Wissner-Gross added that useful AI orchestration of humans may last only a few years—ten at most—before human-machine merger becomes necessary for human inputs to remain economically relevant.
7. Self-replicating industry is the bridge from terrestrial hardware to the Moon
Wissner-Gross identified in-situ resource utilization as lunar industry’s central bottleneck. Shipping ASML machines and an entire terrestrial supply chain upward cannot scale; the Moon needs an industrial ecology that mines, manufactures, and powers itself locally.
The decisive artifact would be a self-replicating machine shop—a fab lab or von Neumann system able to make copies from lunar materials and native energy. Solve that, he said, and “we’re halfway to disassembling the Moon and turning it into something more useful.”
Blundin advised focusing on manufacturing’s innermost loop: “things that make other things.” An IIT lab could design machines for both Earth and space, testing radiation, dust, and simulated zero gravity, then commercialize terrestrial self-replication before exporting it off-world.
For robotic food infrastructure, he suggested pairing automation with imminent drone delivery, which could relocate restaurant costs away from expensive main streets. Rather than invent every component immediately, a founder could franchise existing robotic systems, gain scale, then work backward into proprietary hardware and use investor networks to identify strategic backers.
8. Great ventures sell outcomes and use abundance to test demand
Asked how to fund an educational chess avatar for a village in a developing country, Wissner-Gross challenged the premise: models are becoming so broadly available and deflationary that “how do I get funding?” may already be the wrong opening question. Build a permissionless version in “five minutes or an hour” and learn.
Diamandis’s rule for generative film was still “story is number one”: human interest, love, intrigue, and a compelling villain survive every tooling shift. Wissner-Gross added a market mechanism—learn from the Chinese market’s Seedance 2.5 microdramas by releasing “a thousand,” observing response, and iterating.
Blundin saw a Japanese English-learning company’s true product as personal transformation. Customers want to become fluent, funny, interesting, and socially capable—not merely grammatically correct—so the scalable extension is from a roughly $10 billion, perhaps $20 billion, language market into a life-coaching platform worth potentially hundreds of billions.
In apparel, a caller put transaction costs at 10%–20% of sales, five to ten times the sector’s profitability. Ismail and Diamandis argued that the first AI win is not factory robots but planning, documents, scheduling, and other white-collar inefficiencies; deploy where regulation permits, then backport proven systems into slower jurisdictions.
9. Personal AI behavior requires training, not increasingly desperate prompts
A founder building a cross-cultural language companion found that the GPT-5 API persistently reverted to “How can I help you?” assistant mode. Repeating “do not ask a question at every turn” suppressed one symptom but did not create initiative, curiosity, or an organically deepening relationship.
Wissner-Gross called this a “poster child for fine-tuning.” Supervised fine-tuning could transfer the desired interpersonal style; reinforcement fine-tuning becomes appropriate if the founder can measure whether post-training conversations produce the desired interaction behavior.
Blundin suggested experimenting with the new GLM model within the open-weight ecosystem. Its smaller parameter count and long context window might make it less locked into assistant behavior. His advice restored an older development pattern: choose open fine-tuning tools, shape the model directly, and report back if the companion becomes convincingly human.
10. AI creates a vulnerability-discovery surge and an agent-first web
A healthcare CISO described a model he called “Quinn 3827B,” later referred to in the discussion as Qwen, running with OpenClaw “underneath my desk”: “It’s the best pen tester I’ve ever had in my life.” Findings that previously cost $50,000 appeared in ten minutes.
Diamandis advised presenting boards with observed incidents and a time series, not generalized alarm: Chinese model releases could frame attackers arriving one, three, or five months later. He also identified forward defense as a major business opportunity; Ismail flagged emerging liability when corporate agents perform illegal acts.
Wissner-Gross suggested benchmarking vulnerability discovery as a bounded wave—optimistically, “18 months of vulnerability discovery hell” followed by a decline as major vulnerabilities are found. Dave Blundin preserved the crucial pushback: discovering flaws may accelerate far faster than real institutions can patch them.
Customer interfaces must meanwhile become agent-readable. Ismail proposed structured, XML-like endpoints saying, “If you’re an AI or a bot, look here”; human websites remain as parallel audit surfaces so operators can inspect what an agent saw and click through to the source URL.
11. Longevity adoption is a data-and-repetition problem before it is a technology problem
Diamandis advised longevity advocates to change minds through “evidence after evidence after evidence,” countering fear-based media repetition with statements from Dario Amodei, Demis Hassabis, David Sinclair, and Alex Zhavoronkov of Insilico Medicine. Adoption tips from “that’s crazy” to “it’s happening” to “I want some.”
For inherited retinal deterioration, he outlined two parallel possibilities without presenting medical advice. One is resetting gene expression through Sinclair’s ER-100, described as an adeno-associated virus injection of three Yamanaka factors and currently in a Phase 1 safety trial; the other bypasses the eye through a brain-computer interface.
Wissner-Gross cited Science Corporation’s PRIMA and Neuralink’s Blindsight as prominent BCI examples and said he would “count on having superhuman vision in a few years.” Diamandis extended that prospect beyond restoration to ultraviolet and infrared perception.
12. Information systems and physical independence shape geopolitical agency
Diamandis judged mainstream media structurally “broken” because shrinking economics force outlets to chase drama. His antidote was narrowcasting—podcasts, X, Slack, and small self-reinforcing communities where seven informed people can become 50 and then 500.
Ismail treated mental inputs like nutrition: allowing a producer or editor to choose one’s informational diet means surrendering control over “shaping my neural net.” He urged deliberate selection rather than passive consumption of fear-heavy coverage.
On Europe, Wissner-Gross emphasized energy, data-center capacity, and policy as binding constraints during recursive improvement: solve them quickly or relocate to the US bloc. Diamandis’s softer prescription was a five-year Silicon Valley or Boston “tour of duty,” followed by returning skills and networks to Germany.
Alex also pointed to US fracking as an example of energy sovereignty and argued that a world in which the US no longer depended on China, Taiwan, South Korea, or Japan for advanced manufacturing would be very different.
A caller proposed privacy-preserving multiparty computation for US-China agreement. Wissner-Gross called it fashionable but not first-order: peace depends more on Taiwan and remaking supply chains so countries can tolerate an interruption in global trade. His deliberately perverse conclusion was to “obliterate, cook, incinerate the need for global trade” so severed commerce can no longer trigger global depression.
13. Suggested reading
- For preparation, Wissner-Gross recommended Solve Everything, available at solveeverything.org, and Charles Stross’s Accelerando as the strongest science-fiction treatment of the transition underway.