Silicon Valley Coordinates x Innolight's 于让尘: The AI Optical Interconnect Supercycle

Summary

• AI optical interconnects are in a supercycle driven by a 10x gap: compute has grown roughly 300x since 2022, while total optical-connection bandwidth has grown only about 30x. TeraHop Vice President 于让尘 compares compute with gray matter and connectivity with white matter: the two occupy nearly a 1:1 ratio in the human brain, while the white-matter-to-gray-matter ratio in cats and dogs is roughly 10%-20%; “we may still be at the mouse-and-kitten stage.” Optical interconnects currently account for only a single-digit percentage of data-center capex, but he argues that connectivity investment must grow faster than compute: “this gap has to be filled, and the entire industry has to work on it together.”
• Scale up is the industry’s biggest structural opportunity: bandwidth is roughly 10x scale out, while copper is nearing its physical limits. Nvidia’s all-copper NVL72 rack “is already constraining the brain’s development”; the industry is moving from the 576-chip systems now coming online, and 1,000-chip systems, toward Google’s interconnection of 9,000 TPUs. Copper can currently run only 1.5-2 meters; in the 400G/lane era, “even 1 meter will be a stretch,” pointing to a long-term shift toward optical replacing copper.
• The economics need to be calculated backwards: compare output per token, not the cost of optics versus copper. 于让尘 cites the industry rule to “use copper when you can, use optics when you must,” but stresses that end customers care about returns per unit of compute and per token. GPUs cost $30K-$50K, versus several thousand dollars for an optical module; “if you save on interconnect, you are sacrificing your compute.”
• The technology roadmap is not a religious either-or: “Internet companies are practical; they do not make religious choices.” A single company may choose pluggable optics, NPO and CPO at the same time—“adults don’t do multiple-choice questions” (“成人不做选择题”). Google has said publicly that “CPO is always two years away, and every year it is still two years away.” The major cloud companies are all developing proprietary chips—TPU, MTIA, Trainium, Maia, OpenAI Titan and xAI—which makes them receptive to an open optical-interconnect ecosystem. CPO’s failure to reach scaled commercial deployment is partly because customers dislike vertical integration and suppliers gain too much bargaining power, while the technical bottlenecks remain formidable.
• TeraHop is positioning itself around open standards: its 12.8T XPO and Open CPO-X multi-source protocols can deliver 4-8x bandwidth growth, with about 100 companies already participating. The case for the technology rests on silicon-photonics maturity: more than 1,000 eight-channel products are already in use, making 16, 32 and 64 channels a natural progression. Nvidia’s $4B investment in Lumentum and Coherent in early March was interpreted by 于让尘 as concern that the supply chain could not keep up with 10x growth—“locking down the supply chain with real money.”
• OCS is a greenfield market Google began developing roughly 10 years ago, while other major players remain at zero or very low adoption—but electrical switching is not going away. Optical switching cuts power, latency and cost, but it switches slowly and is suited only to predictable workflows. 于让尘 is explicit in his caution: it “cannot completely replace” nanosecond-scale electrical switching over the next 5 years; “never say never,” and the two technologies will coexist over the long term. OCS’s 2-3dB loss makes coherent light a good pairing, and Google’s strategic investment arm has backed the coherent-light startup Celero.
• The industry’s biggest bottleneck is that semiconductor integration remains early-stage, while the value chain will soon become a trillion-dollar question. Lasers are still made on 3-inch wafers and are working toward 6-inch, versus 12-inch wafers for silicon photonics; quantum-dot lasers could eventually put the light source directly on silicon. Over the next 5 and 10 years, the direction is semiconductor-based optical interconnects and deep optoelectronic integration, including TSMC’s COUPE platform. Interconnects should be treated as part of compute itself and ultimately become an equally important component.

Deep dive

1. Core thesis: compute is up 300x, optical interconnect only 30x—the 10x gap is the supercycle itself

• 于让尘’s opening judgment is that optical interconnects are “not just an accessory”; they have become an indispensable part of compute. Since 2022, compute has grown roughly 300x, while AI connectivity bandwidth has grown only about 30x: “there is a 10x gap here. This gap has to be filled, and the entire industry has to work on it together.”
• In terms of current scale, optical interconnects account for only a single-digit percentage of total capex. If compute investment is assigned a value of 10, he estimates connectivity investment could be somewhere between 10 and 20. His rule is that for every 1 unit invested in compute, connectivity investment should grow by more than 1; otherwise, compute is wasted.
• The central analogy of the discussion is gray matter versus white matter: gray matter represents compute and white matter connectivity. The two occupy nearly a 1:1 ratio in the human brain, although another estimate puts the split at roughly 40:60; when cats and dogs hunt mice, their white-matter-to-gray-matter ratio is around 10%-20%. “We may still be at the mouse-and-kitten stage today.” As the brain gets larger, connectivity must grow faster than compute for the overall system to become more optimized.

2. Scale up / out / across: from one rack to all of Manhattan

• Scale out is the connection between brains: training trillion-parameter models requires 10K-GPU-scale clusters connected across several layers of networking. Scale up makes a single brain larger, with roughly 10x the bandwidth of scale out. Nvidia’s NVL72—72 chips in one rack—currently uses copper throughout: “bandwidth is limited, distance is limited, and it is already constraining the brain’s development.”
• The baton is moving up the scale curve. Nvidia has begun moving into 576-chip interconnects, more aggressive internet companies are deploying 1,000-chip systems, and Google TPU can interconnect 9,000 TPUs—using a hybrid architecture of copper inside the rack and OCS plus optical interconnects outside it.
• The distance ladder runs from a few meters to dozens of meters for scale up—8 to 10 racks, with Google potentially reaching dozens of racks—to roughly 1-2 kilometers across a building for scale out, and from several kilometers to dozens of kilometers between data centers for scale across. Meta has unveiled large data-center campuses reaching “Manhattan size.” Scale up “is a low-base opportunity from an optical-connectivity perspective, but it is a 10x opportunity.”

3. The economics of optics replacing copper: calculate token output, not component cost

• 于让尘 cites the industry maxim, “use copper when you can, use optics when you must.” Copper’s limitations are physical and cannot be engineered away; copper should remain in the rack where practical, but making the brain stronger and larger will ultimately require optics.
• The calculation has to be reversed: end customers care about “output per unit of compute, or output per unit of token.” Even if optical interconnects cost more, customers will pay as long as the resulting token output per unit of energy and per unit of cost is better optimized.
• Host 曹卿云’s quantitative framing was confirmed: a GPU costs $30K-$50K, while an optical module costs several thousand dollars. Optical interconnects drive the utilization of clusters containing thousands of GPUs. “Don’t think you are saving money by saving on interconnect. What you save on interconnect is sacrificed compute—and ultimately, your efficiency in producing tokens for customers.”

4. Pluggable / NPO / CPO: the real answer to the roadmap debate is that adults don’t do multiple-choice questions

• The three architectures answer the same fundamental question: how close the optical interface sits to the main chip. Pluggable optics are the most flexible, with the most open supply chain but relatively higher power consumption; CPO minimizes the electrical path and maximizes bandwidth density, but concentrates heat and deeply couples the system, meaning a failure could affect the entire board; NPO sits in between. 于让尘 says this “is not a multiple-choice question, but a question of balance”—power, cost, bandwidth density, serviceability, and reliability all matter to customers.
• He rejects the market shorthand that Google uses pluggables, Meta and Amazon are choosing NPO, and Nvidia is betting on CPO. “People may write all kinds of little stories in public forums … but internet companies are practical. They don’t make religious choices.” A single company may choose all 3: “adults don’t do multiple-choice questions” (“成人不做选择题”).
• Google’s public line is: “CPO is always two years away, and every year it is still two years away.” The interview did not make a definitive claim about whether Google privately keeps every option in its toolkit. 于让尘 said different compute fabrics and different AI networks may call for different choices.
• The underlying force behind the open ecosystem is the rise of proprietary chips: Google TPU, Meta MTIA, Amazon Trainium, Microsoft’s Maia, OpenAI Titan and Elon Musk’s xAI. Each company is buying Nvidia’s vertically integrated systems while also “voting with its wallet” to build its own compute. “Building their own chips is no longer an option; it is something they have to do,” which makes an open optical-interconnect ecosystem attractive.

5. The bandwidth roadmap: 400G/lane, explosive channel counts and the long-term case for “slow and wide”

• Near-term, 200G/lane 1.6T products begin mass production this year and will become a major growth vector over the next 2-3 years. 400G/lane is the industry’s current focus and could arrive within the next 2-3 years. Channel counts are expanding in parallel: the traditional move was from 8 lanes to 64 lanes; NPO/CPO are now at 32 lanes, with 128 lanes or more possible next.
• Copper’s long-term boundary is clear. It currently runs only 1.5-2 meters; “at 400G it may not even manage 1 meter,” and the limit could ultimately fall to half a meter. When copper finally exits, the interface architecture will be “rebuilt from scratch.” Speeds may not continue rising and could instead move lower. The industry is exploring “slow and wide”: each vehicle carries less, but many smaller vehicles move forward at once—simplifying the interface, cutting power and sharply increasing channel counts.
• Coherent light may not enter scale up immediately, but in specific configurations across scale out and scale across, it “will become a more efficient approach.”

6. 12.8T XPO and Nvidia’s $4B: both demand and feasibility have arrived

• Why can bandwidth jump directly to 12.8T? Demand and feasibility are both required. Demand comes from the 10x gap and copper reaching its limit inside a single rack. Feasibility comes from silicon photonics: as process development, yield and reliability mature, “there are already more than 1,000 eight-channel products; moving naturally to 16, 32 and 64 channels” becomes possible. TeraHop has launched 12.8T XPO and 6.4T NPO/CPO products.
• Nvidia’s $4B combined investment in Lumentum and Coherent in early March was interpreted by 于让尘 as concern that the supply chain could not keep pace with 10x growth—“locking down its supply chain with real money.” The core objective was to secure the lasers required for CPO. Jensen Huang has called optical interconnects a critical part of the compute buildout twice at GTC. “It is a small episode within a much larger picture, but a very clear public confirmation.”

7. Open standards versus vertical integration: why CPO has yet to scale commercially

• The strongest signal from this year’s OFC was the launch of multiple multi-source agreements, with TeraHop as a major driver. Its 12.8T XPO and Open CPO-X protocols can deliver 4-8x growth in optical-interconnect bandwidth, and “about 100 companies are willing to participate,” spanning internet companies, chipmakers and module vendors.
• The critique of vertical integration is unambiguous. Solutions from 1 or 2 major customers, large manufacturers and major suppliers are “quite proprietary,” which is “not what end customers want to see.” Once bundled and sold as a package, suppliers’ “bargaining power becomes extremely strong.” That is one reason CPO has been discussed for so long without reaching scaled commercial deployment: customers do not particularly like it, and the technical bottlenecks created by vertical integration are highly challenging.
• One physical bottleneck is easy to overlook: the fiber itself. Hair-thin fiber must carry enormous capacity from the rack across the data center, and density is already becoming inadequate. The Open CPO-X SDM-MCF multi-core-fiber protocol allows a single fiber to carry 4 channels.

8. OCS: Google’s roughly 10-year head start and a greenfield market elsewhere

• OCS, or optical circuit switching, is a key component of Google’s 9,000-TPU “super brain.” In the right workloads, replacing electrical switching with OCS cuts power, latency—because the optical path is transparent and requires no complex processing—and cost. It is not a wholesale replacement, however: Google uses a hybrid architecture in which optical and electrical switching complement each other. Electrical switching remains widely deployed; only the mix is beginning to change.
• The trade-offs are clear. OCS switches slowly and is “basically a tool for replanning routes,” making it suitable only for stable, predictable workflows. Its other advantage is that a failure can be addressed quickly by rescheduling data flows.
• Google began laying the groundwork roughly 10 years ago and is “furthest ahead, and has gone the farthest.” OCS adoption at other internet companies may be zero or very low, with little actual deployment. Everyone is experimenting, and each company may eventually need to develop its own scheduling algorithms. Rising adoption would represent a strong greenfield opportunity for suppliers.

9. OCS’s cost pushes coherent light into the spotlight; the 400G roadmap is still being tested

• OCS adds 2-3dB of loss, raising the link-budget requirements for optical modules. Coherent light is more expensive, more power-hungry and more complex, but its intrinsic amplification effect gives it a higher link budget. “Coherent light and OCS are a good pairing,” and coherent technology could become a major trend over the next 1 or 2 generations of speed upgrades. Google’s strategic investment arm backed coherent-light startup Celero to broaden and secure the supply chain. The challenges include DSPs customized for low power and low latency, as well as more complex silicon photonics for modulation, polarization handling and balanced reception.
• The 400G/lane roadmap is still being contested and validated. Silicon photonics, thin-film lithium niobate, and EML and other III-V-based approaches are all in the running. “It is too early to say which will be mainstream; everyone is still in the PK stage,” with early validation demos from all 3 at this year’s OFC. TeraHop favors integrable approaches and is developing a hybrid platform that grafts thin-film lithium niobate onto silicon photonics, but “all 3 horses are running … we are giving up none of them.”

10. A trillion-dollar market, laser bottlenecks and the 5- to 10-year outlook: interconnect is compute

• The restructuring of the value chain “will quickly become a trillion-dollar issue.” The combined market for switches and optical interconnects is already approaching $1T. The directional call is clear: semiconductor-based optical interconnects are inevitable, with optical and electrical components becoming more deeply integrated. TeraHop’s positioning is: “we are not a pluggable-module supplier; we are an optical-interconnect supplier.” How value is ultimately distributed across the different form factors remains to be determined.
• The industry’s biggest bottleneck is that semiconductor integration is still in its early stages. Silicon photonics is already using 12-inch (300mm) wafers, while laser manufacturers remain on 3-inch wafers and are working toward 6-inch. Startups are developing quantum-dot lasers that, if successful, could eventually manufacture light sources directly on silicon and move from today’s 3-inch or 4-inch wafers to 12-inch. On optoelectronic packaging integration, TSMC’s COUPE platform—silicon photonics combined with electrical chips—is a leading example, and other fabs will follow.
• The 5- and 10-year conclusion is a correction to the old idea that optical interconnects simply save money that can be redeployed to compute. “Interconnect should be viewed as part of compute itself.” These are not 2 resources competing for the same budget, but a symbiotic ecosystem optimized jointly. Ultimately, like the ratio of white matter to gray matter in the human brain, interconnect will become an equally important part of the system.