张金涛 of ShengShu: Vidu S1, Inference Acceleration, Real-Time Video

Summary

• ShengShu Technology has launched Vidu S1: no previous model could use voice for real-time control while generating indefinitely long, uninterrupted streaming video—with 540p resolution and 25-42 FPS, running in real time on consumer-grade GPUs. 张金涛 is blunt about the priority stack: “speed is the top priority”; at the current stage, “we can only say the image quality should come down a little further.” Monetization is live: the web version and APP are free, while the API costs “only 2 or 3 yuan per minute” with no duration limit.
• The demand-side thesis starts with an Elon Musk-style reverse calculation from compute: if everything the human eye sees is generated in the future, multiply each person’s pixels per second by billions of people to estimate global compute and data-center demand. 张金涛’s key distinction is that offline video has a “view count” over which costs can be amortized, while online interactive visual entertainment is different because “everyone wants to see something different” and “everyone needs an independent session.” He sees the direction as “unstoppable”: “in the future, the visual-entertainment signals we consume will be replaced by AI-generated content.”
• 张金涛’s breakout work, C Attention—rendered as “C plus Attention” in multiple places in the original subtitles—has already become a de facto industry standard. NVIDIA, AMD, Huawei Ascend, Moore Threads, and MetaX have each implemented and deployed their own versions, while ByteDance, Tencent, Google, and other multimodal companies are using it. The technical distinction matters: language-model attention is memory-bound, while attention in video-generation models is “heavily compute-bound”—explaining why low-bit acceleration becomes more valuable in the multimodal era.
• His structural call on inference acceleration is that “the future belongs to the layers closer to the hardware and the layers closer to the application.” The operator layer—pushing individual kernels to hardware limits—will converge as AI coding improves. The opportunities are in chips customized for a class of models, with the compute-to-memory-traffic balance tuned accordingly, and in algorithms that reduce complexity—“the human brain actually runs on only 25 watts.” The startup lesson: “just doing the algorithm layer doesn’t offer a particularly strong moat”; good inference acceleration requires hardware-algorithm co-design.
• His explanation for the China-U.S. split in AI video is that “once you’ve polished the modeling, it is no longer fundamental; data has become the fundamental issue.” He says U.S. video-generation companies are likely behind Chinese firms across data volume, quality, human-preference alignment, and data construction. China’s stronger Douyin, Xiaohongshu, livestreaming, and e-commerce ecosystems have produced more data and a stronger incentive to improve video generation.
• On the professor and PhD startup wave, he says “AI is not a bubble” and that no single company can do the whole thing. The boom is rational, but “not everyone is suited to entrepreneurship.” He treats 朱军’s GPT-3 lesson as a methodology: know the world’s leading approach and execute every link to the limit to become world-leading—many companies may not lack the right understanding; “they may simply have 2 lines of buggy code in the repository.”

Deep dive

1. Opening Thought Experiment: How Much Compute Does a Real-Time Generated World Require

• Koji relays an observation brought back by a friend who had visited Elon Musk: if everything the human eye sees is generated in the future, the math is simple—take how many pixels a person’s 2 eyes see per second, how many hours a day those eyes are open, and multiply by billions of people to reverse-engineer the global compute and data-center capacity required to support a real-time generated world.
• That sets up the episode’s central theme: beyond stacking hardware, inference acceleration at the software and algorithm layers is an equally important problem. The guest is 张金涛, a 26-year-old Tsinghua PhD student under 朱军, an NTJ and Aquarius, who leads the full streaming-video-generation stack at ShengShu Technology, along with inference acceleration and cluster deployment for all models.

2. The Starting Point for C Attention: Flash Attention Was Not the Endpoint

• The origin story begins after he moved from a master’s program into a PhD. His advisor, 陈建飞, told him that “inference acceleration is going to be an extremely important direction,” pulling him into the field. At the time, the industry broadly viewed Flash Attention as the optimal kernel. But the team found that “the hardware clearly still had faster compute units”: low-bit compute units were already widely used to accelerate linear layers, yet no one had applied them to attention.
• He breaks down the 3 reasons. First, before early 2024, attention in language models was memory-bound, so accelerating tensor computation offered limited gains; multimodal models, especially video-generation models, are “heavily compute-bound,” making compute speed directly determinative of generation speed. Second, attention acceleration requires hand-written GPU kernels, while Web Coding had not yet developed low-level GPU programming capabilities. Third was precision: quantizing everything to 8-bit produced “very poor” outputs. Solving the latter 2 challenges led to C Attention, which the original subtitles often rendered as “C plus Attention.”

3. From Kernels to Clusters: 3 Layers of Acceleration and Turbo Diffusion

• He groups the inference acceleration work from C Attention to Vidu S1 into 3 layers: the operator layer, which pushes computation toward the theoretical limit on specific hardware; the model layer, which reduces computational complexity; and the engineering layer, covering cluster deployment, model parallelism, and user-request scheduling. Examples at the model layer include distilling diffusion denoising from 50 steps to 4, cutting complexity by more than 10x, as well as sparse attention.
• Turbo Diffusion addresses the second layer. It was built at school under compute constraints, using an open-source video-generation model and a small amount of fine-tuning data. It now has more than 3,600 GitHub stars.
• What he is proudest of is C Attention’s breadth of adoption: it “has already become a de facto industry standard.” NVIDIA, AMD, Huawei Ascend, Moore Threads, and MetaX have all implemented their own versions and deployed them on their respective GPUs. Virtually every multimodal company—including ShengShu Technology, ByteDance, Tencent, and Google—is using it.

4. Vidu S1: Streaming Generation Cannot Be Reduced to “Fast”

• The paradigm shift is the key difference. A conventional video model takes a prompt, runs for a long time, and returns a 5-10 second video. A real-time model generates every frame live, so “the generation speed needs to exceed the playback speed.” Duration is the bigger challenge: “the most advanced foundation models today can only generate 30 seconds at most.” S1 is designed to generate indefinitely—to “chat for days and nights on end”—using a fundamentally different diffusion-training approach built around the attention computation method and noise-strength control, preventing long-run drift and collapse.
• The specifications and trade-offs are explicit: 540p, which essentially no other real-time-generation product on the market reaches; 25-42 FPS; and operation on consumer-grade GPUs. With too many objectives to optimize, the priority is clear: “speed is the top priority”（“快是第一目标”）. Everything else is optimized as far as possible under that constraint. Image quality ranks last: “at the current stage, we can only say the image quality should come down a little further.”
• Monetization is straightforward: the web version and APP are free, while the API costs roughly 2 or 3 yuan per minute with no time limit.

5. Why Real-Time Interactive Video Is the “Inevitably Right” Direction

• He divides visual entertainment into 2 categories: offline generation, such as films and short videos, and online real-time interaction. Koji uses today’s face-to-face conversation as an example; 张金涛 adds dating and gaming. Before the digital revolution, almost all human visual entertainment belonged to the latter category, which is why “the demand for online real-time interactive visual entertainment is greater than the demand for offline, pre-produced content.”
• The investment point is embedded in the contrast. Offline video has a view-count concept over which production can be amortized. Online interaction is different: “everyone wants to see something different,” and “everyone needs an independent session.” That is the condition behind the compute extrapolation in the opening thought experiment. His conclusion is unqualified: “technologically, there is currently no real barrier and nothing impossible about it”—the direction is “unstoppable.” “I firmly believe that in the future, the visual-entertainment signals we consume will be replaced by AI-generated content.”

6. Day-One Use Cases: From Household Dogs to a “Programmer Cheerleader”

• Less than a day after launch, examples with more than 1,000 combined likes and saves had already appeared. One user uploaded their own dog and chatted with it interactively. On Bilibili, another user sent a live game screen to an anime character inside S1: the game took up 3/4 of the frame on the left, while the remaining 1/4 showed an extra companion “playing the game” alongside the user. The character could actually see what was happening in the game and interact in depth.
• S1 supports video understanding, camera input, and arbitrary character customization. Koji’s live thought experiment: stream a coding interface to it and “it becomes a programmer cheerleader.” 张金涛 says “of course,” because it can see both the words being typed and the screen. Another user uploaded the persona and image of “峰哥” and had him perform a real-time spinning blessing dance. His testable prediction: “over the next 2 weeks, there will definitely be many highly entertaining examples built on Vidu S1.”

7. The Inference-Acceleration Map: The Future Belongs to the Lower and Upper Layers

• He first corrects a common misconception: inference acceleration is not just about running an operator at the hardware’s theoretical maximum. The other half is reducing the model’s computational complexity by removing redundant calculations hidden inside the black box—through sparse attention, MoE, model distillation, and sampling-step distillation. He cites TensorRT, vLLM for scheduling and management, ShengShu’s TurboServe for cluster scheduling and deployment of streaming video generation, DSA—likely DeepSeek Sparse Attention—and his own SLA, or Sparse Linear Attention. Multimodal leads at Tencent and ByteDance have told him they use SLA, though he volunteers that the open-source version still has a few minor flaws, mainly in its choice of linear-attention mechanism.
• His structural call is clear: “the future still belongs to the layers closer to the hardware and the layers closer to the application”（“未来属于更底层和更上层”）. The operator layer in the middle will converge as Web Coding improves: given an operator, determining whether it can run quickly on hardware will eventually converge. The bottom layer is chips. Companies can either build the kind of highly general-purpose powerhouse chip 老黄 builds or pursue semi-custom chips for a class of models, tuning the balance between compute and memory traffic according to whether the workload is memory-bound or compute-bound. But the Tesla-style approach of hard-coding one specific model into a chip is wasteful: “once the model updates, the chip is obsolete.”
• The upper layer is algorithmic complexity reduction: “the human brain actually runs on only 25 watts,” and given a little time, it can solve many complex tasks. Asked whether the lower layer is easier for startups, his answer is useful for investors: good inference acceleration “must be hardware-algorithm co-design.” A company can focus on the bottom layer, but “just doing the algorithm layer doesn’t offer a particularly strong moat”; it is difficult to secure validation from the underlying layer.

8. China-U.S. Divergence in Video Models: Once Modeling Converges, Data Is Fundamental

• On why Chinese video models have firmly remained in the top tier, his framework is that after Transformer, “no matter how much you polish the modeling, it is no longer particularly fundamental”（“建模再怎么雕花，其实不是特别本质了”）. The fundamentals have shifted to data: high quality, alignment with human preferences, and strong learnability. Reasoning, in his framing, is the construction of a chain of thought that turns an A-to-Z problem the model cannot learn into something it can. His analogy is vivid: the gap between a good explanation and a bad one is enormous. “Take the Science Space curated by 苏建林—I can watch it for 10 minutes and get more out of it than 3 hours of garbage blogs, and those 3 hours would also contaminate my learning.”
• His conclusion is direct: U.S. video-generation companies are likely behind Chinese companies in data volume, quality, human-preference alignment, and how the data is constructed. He says the gap may partly reflect policy factors, but also culture: China’s Douyin, Xiaohongshu, livestreaming, and e-commerce ecosystems are more developed, producing more accumulated data and a stronger incentive to make video generation better.

9. The Startup Wave, Management Lessons, and a Standout Line of Code

• On the 2026 wave of professor and PhD startups, he says “AI really is not a bubble.” It can satisfy all of humanity’s needs, he argues, making the startup boom rational and capable of lasting a long time. The opportunity is also too large for one company to capture: “one company cannot finish the whole thing.” But he is equally candid about the other side: “not everyone is suited to entrepreneurship.” Some people are technically excellent and write great papers but cannot manage. “Putting it slightly less modestly, I feel I’m actually quite well suited to starting a company.”
• His methodology comes from 朱军’s view of GPT-3: it was not an unreachable innovation; it simply “pushed every link to the limit.” Know the world’s leading method, implement it correctly, and make every line of code correct, and you can become world-leading. The implication is painfully concrete: “many companies don’t fail because they don’t know what the leading approach is; it may simply be that there are 2 lines of buggy code in the repository.” His management principle is to understand people: whether someone wants salary, credit, or a sense of achievement can be determined after a few minutes of simple conversation.
• His career highlight came in May 2024, when he formed the Vidu inference team and replaced FlashAttention with his own C Attention. “It was just 1 line: the generated video was pixel-for-pixel identical to before, but end-to-end inference speed immediately became more than 2x faster.” It was “the first work in the world to accelerate attention computation with low-bit arithmetic.” His self-assessment retains the original edge: “Calling it great would be a little self-congratulatory, but it really was a fairly remarkable piece of work.”
• The closing exchange concerns writing and intelligence: “the boundary of intelligence and the boundary of language are sometimes equivalent; whether you can explain something clearly reveals whether you truly understand it.” His addition is more demanding: explaining one thing clearly is not difficult; the hard part is explaining something very complex with equal clarity.