Underwriting Superintelligence

The AI underwriting company’s central thesis is that stronger safeguards should accelerate AI deployment rather than restrain it. Kvist compares safety infrastructure to the helmet and seatbelt that let a race driver corner faster: “security and progress are mutually reinforcing.” Labenz frames the downside as a nuclear-style outcome in which weaponization proceeds while practical benefits are frozen out.

Insurance, standards, and independent audits form a single incentive system that none of the three can provide alone. Standards define best practice, audits establish whether developers actually follow it, and insurance pays when failures still occur. Because insurers want premium growth but bear losses from lax controls, Dattani argues this offers a market-based middle ground between voluntary promises and inflexible top-down regulation.

Existing policies leave both enterprises and insurers exposed to unresolved AI coverage ambiguity. Cyber and other policies may cover familiar categories of harm, but often never mention an AI-induced cause; meanwhile, insurers have not priced a risk they do not yet know how to price. Dattani suspects a replay of early cyber insurance, when years of litigation eventually separated computer-related losses into differently structured products.

Red teaming could help compensate for the historical loss data that conventional underwriting lacks. An EY study cited in the conversation found enterprises already experiencing losses of $1 million, while smaller companies absorb thousands or tens of thousands that insurers may never see. Repeated evaluations can generate synthetic frequency and severity data, while parametric triggers could produce pre-agreed payouts without years of litigation.

Private markets may insure routine AI failures, but the largest tail risks probably require a government backstop. “No one knows what the kind of tail risk distribution looks like because we have not yet seen it,” Kvist cautions. The proposed analogue is US nuclear insurance: operators carry strict liability and mandatory coverage, while damages above $15 billion move to the government balance sheet so private insurers can still perform the governance function.

AIUC-1 packages enterprise AI concerns into one auditable procurement language built from more than 500 industry conversations. It covers data and privacy, security, safety, reliability, accountability, and societal risks, emphasizing disclosure where hospitals and retailers may reasonably choose different thresholds. Initial red teams sometimes find attack-specific failure rates as high as 25%; after safeguards, the same rates can fall by 90%, though the founders stress that “there are no panaceas.”

The application layer is the AI underwriting company’s first commercial wedge because agent vendors make unusually specific promises without foundation-model-scale balance sheets. Customer-support agents promise to resolve a stated percentage of tickets while implicitly promising not to cause brand disasters: “the thicker the promises,” the thicker the assurance required. The longer-term ladder runs from million-dollar risks through hundreds of millions and low billions to tens of billions, spanning applications, foundation-model systems, and data centers.

The company is structuring its economics to avoid the issuer-paid race to the bottom that damaged credit ratings before 2008. As a managing general agent, its compensation will depend on insurers’ underwriting results; large losses mean lower or no compensation, lost carrier relationships, and potentially, “We’ll go out of business.” Labenz disclosed that he invested in the company’s seed round alongside Nat Friedman, Emergence, and Terrain. Founding technical contributors include Cognition, Ada, Intercom, and Recraft; the enterprise consortium includes JPMorgan Chase, Confluent, and Anthropic’s deputy CISO.

🔗 Original source & video: Underwriting Superintelligence: AIUC’s Insurance, Standards & Audits to Accelerate AI Adoption