Ep. 035 - Tech DD’s, Performance Projections, Supply Chain, Investment Thesis (Consulting)
Summary
SemiAnalysis Consulting bridges AI infrastructure’s technical and financial languages through bespoke engagements lasting from 3–4 weeks to a year. Abhilash Jain calls this the “co-design of technology and finance”: institutional investors may need diligence on a $1–2 billion chip, cloud, or data-center investment, while hyperscalers, AI labs, and semiconductor companies need product roadmaps, site selection, or go-to-market strategy.
One engagement modeled global token supply by linking the installed chip base, inference-versus-training allocation, per-GPU throughput, and dollars per million tokens. The difficult part was simulating unreleased chips and models—including potential 10, 15, or 20 trillion-parameter systems—then exposing assumptions through a dynamic dashboard where “literally every number changes,” rather than forcing so many permutations into static Excel.
Custom projects can become scalable research products when clients reveal an unmet need that can be standardized. The data-center model and cloud-AI total-cost-of-ownership work originated in consulting, while the inference simulator moved from internal research to one client, then several, and is now becoming a standalone product: customers effectively identify problems worth standardizing.
NeoCloud diligence increasingly determines whether unfamiliar operators can obtain financing and survive contractual downside. Investors examine SLAs, OEM warranties, cluster performance, capex, and opex; Jordan highlights 99.9% expected availability versus below 90% potentially triggering credits or termination. Abhilash’s blunt conclusion: “Without reliable SLAs you won’t get funding. That’s it, period.”
SLA risk compounds because operators may hold 15–20-year data-center leases against five-year GPU contracts, while a single power, cooling, rack, or software failure can still trigger the full customer remedy. Insurance can absorb penalties for an efficiently run operator paying a basis-point premium, but “if you have poor performance, you won’t even get insurance.”
Build, buy, or partner is a conditional decision driven by time-to-market, compute premiums, technical capability, and access to capital. SemiAnalysis advised one client needing 100 MW within six months to lease capacity; a client willing to wait until 2028–2029 received the opposite recommendation.
The consulting team is converting repeated diligence into operating leverage while expanding its technical reach. NeoCloud checks have become “muscle memory,” with known requests and interviews scheduled by days two, four, six, and eight; Abhilash jokes that conclusions are often predictable. Dylan has asked Abhilash to hire 20 more people within six months, likely consuming “50% of my day hiring.”
Deep dive
1. SemiAnalysis sells a bridge between capital and AI infrastructure
SemiAnalysis had always consulted, but created a dedicated team after its newsletter, roughly 13–14 institutional models, and open research proved too standardized for clients seeking deeper engagement. Every assignment is bespoke, with a dedicated team working anywhere from 3–4 weeks to a year.
Abhilash’s framing is “the co-design of technology and finance”: teach technologists finance and financiers technology. Private-equity and hedge-fund clients need diligence on $1–2 billion investments or theses on “the next big wave”; industry clients need product roadmaps, data-center site selection, and go-to-market strategy.
The industrial scope spans hyperscalers, AI labs, chipmakers, and semiconductor companies. A recurring question is causal and long-dated: if LLMs evolve in a particular direction, “what products should we be thinking about in 3, 4 or 5 years?”
2. The token-supply project became an interactive forecasting engine
A hedge fund asked SemiAnalysis to estimate global token supply from the installed chip base. The equation joined three uncertain variables: five-year accelerator growth and training-versus-inference allocation, daily throughput per GPU, and the resulting dollars per million tokens.
Throughput was hardest because future hardware and models do not yet exist. The team built a simulator for questions such as how NVIDIA specifications, memory reductions, or an OpenAI model with 10, 15, or 20 trillion parameters might affect performance.
Static Excel could not contain the permutations, so the deliverable became a dynamic dashboard. Clients could vary AMD, TPU, and NVIDIA expectations; input-output ratios; sequence and context lengths; data types; and caching at 80%, 85%, or 90%, then watch “literally every number” respond.
3. Consulting demand becomes product strategy—and eventually product
One memory company uses simulated traces across model-chip combinations to decide what memory products to develop, specifically testing how bandwidth-versus-capacity trade-offs affect production inference. The engagement turns workload forecasting directly into an engineering roadmap.
For a semiconductor vendor facing roughly 100 possible buyers and a 3–4-month qualification cycle, SemiAnalysis narrowed the field to 10–15 priority customers. That required using SemiAnalysis’ own network—not expert agencies—to speak directly with 20, 30, or 40 market participants about technical requirements, sales positioning, and the customer journey.
Repeated questions create a research flywheel. The data-center model began as consulting; Jordan clarified that the cloud-AI total-cost-of-ownership model definitely did too. The inference simulator progressed from internal research to one client, then several, and is now becoming a standalone product.
4. NeoCloud diligence is ultimately a test of contractual survivability
A typical diligence combines technical review—SLAs, OEM support, warranties, contracts, and cluster benchmarking—with commercial comparison of capex, opex, utilization, and productivity. Testing can produce a concrete 180-day improvement plan rather than a binary pass or fail.
Jordan distinguishes service-delivery terms from broader contract-termination rights: late delivery or downtime may lead to credits or termination. Customers may expect 99.9% availability; below 90% means losing access roughly one day in ten while still paying for the GPUs.
Abhilash argues SLAs matter most “for bad times.” Operators may carry 15–20-year data-center leases and five-year GPU contracts; failure can destroy lender confidence, trigger penalties, and prevent the second deal or third lease needed to service those obligations.
Abhilash says several companies offer uptime SLA insurance for a basis-point premium, but poor performance means no insurance. It is protection for an efficient operator, not a substitute for reliability.
Risk assessment therefore moves from contracts into physical readiness and bare-metal testing: L3, L4, and L5 commissioning, rack arrival, power and cooling integration, failure simulation, monitoring, and recovery. One root cause—whether power, cooling, racks, or software—can still make the operator liable for all SLA credits.
5. Capacity urgency decides build versus buy
Colocation, chips, and skilled labor remain constrained, so owning a cluster is not always the realistic near-term answer. SemiAnalysis weighs time-to-market, willingness to pay a compute premium, operating expertise, and the client’s ability to finance infrastructure independently.
The contrast is explicit: a company needing 100 MW within six months was told to lease because it could not build fast enough; willingness to wait until 2028–2029 produced “the recommendation diametrically opposite.”
Abhilash finds acquisition work most intellectually interesting because it exposes him to LLM architectures, kernels, GPU optimization, and fast-versus-slow inference. His favorite work, however, is iterative diligence that has become “muscle memory,” with requests and interviews scheduled by days two, four, six, and eight. Jordan jokes when Abhilash says conclusions are not predetermined; Abhilash responds, “You’d be surprised how often they are.”
The team now covers far more than it did six months earlier and plans another episode about lessons learned. First comes scaling: Dylan instructed Abhilash to hire 20 people within six months, turning rapid technical learning and recruitment into parallel operating challenges.