Summary

Distillation is not copying answers in the literal sense; it is “copying homework with a barrier to entry.” The standard approach is to use a powerful model such as Fable or Claude—its answers, chain of thought and even full Agent task trajectories—to train a student model to reproduce its output behavior, which requires compute, data pipelines and know-how. Even where a practice does not qualify as conventional distillation, 洪灏 notes that using a frontier model’s output to improve a competing model violates the broad user agreements of Anthropic, OpenAI and Google DeepMind; legally, “it is currently a gray area.”
Two technical milestones improved distillation’s return on investment, and the scale expanded sharply from 2025 onward. The September 2024 release of o1 showed that large-scale reinforcement learning in post-training could teach models reasoning strategies, while chain of thought became “better raw material for distillation”; in January 2025, DeepSeek R1 shipped 6 distilled models at once—4 based on Qwen 2.5 and 2 on Llama 3. Combined with open-source models such as K3 closing in on US closed-source models, distillation has moved to the center of the China-US technology race.
张一鸣’s ban on Seed distillation was a real strategic bet, with clear logic and costs. He believes distillation “can improve model performance in the short term, but is fundamentally copying Claude’s existing capabilities,” meaning it can at most approach, not surpass, the original; Manqi believes Seed 2.1, released in June, likely used some distillation before an internal debate led to a decision to stop. Meanwhile, reporting says ByteDance began reorganizing its data team in June and plans to create a first-level department, “AI Data and Safety,” parallel to Seed and Flow. The sharpest argument is about the organizational cost: “张一鸣 may not be the most technically knowledgeable person, but he is probably the person who understands human nature best.”
The escalation from naming names to countermeasures is turning distillation into a compliance tail risk. Anthropic named DeepSeek, Kimi and MiniMax in February; in a June letter to the US Congress, it called Alibaba Qwen’s 28.8M interactions “the largest-scale distillation attack ever uncovered,” and said it had deployed traffic classifiers and behavioral fingerprinting systems capable of detecting cross-account coordination and chain-of-thought extraction. Locally deploying an open-source model approaching 3T parameters for distillation also requires substantial compute and electricity, which Manqi says makes the activity relatively easy to monitor. Inclusion on the Entity List is the potential cost 洪灏 poses as a hypothetical.
Whether a student can surpass its teacher remains technically open, but staying ahead over time depends on innovation. Multi-teacher distillation can “take the best from multiple sources,” and a student can surpass a teacher on a specific task; but “right now, everyone leads for a few months,” and if self-evolution works as assumed, the first model to reach a given state will accelerate faster and “throw the people behind it even farther back.” Staying close through distillation is not enough reason to declare victory.
The hypocrisy debate leaves behind a useful case-law boomerang. Anthropic was sued in a class action by writers for downloading books from pirated sites and reached a $1.5B settlement, while a California court precedent held that training on paid-for material was not infringement. 洪灏 asks whether, under the same logic, using Anthropic’s output to improve another company’s model would also be non-infringing. “This is a pretty interesting case.”
The bigger variable than distillation may be a mismatch between intelligence supply and demand. On the “kill line” chart, every model below and to the right of V4 and Grok 4.6 is eliminated; one productivity-app founder says users can have “8 out of 10 tasks solved by DeepSeek V4 Flash,” raising questions about both the scale and pace of white-collar demand for more intelligence: “We used to say coding was a bigger market than video generation—what if we were wrong?” That would shake the commercial logic of frontier investment more deeply than short-term valuation volatility.

Deep dive

1. The right definition of distillation: not copying answers, but “copying homework with a barrier to entry”

Manqi’s simplified description: Start with a large batch of questions for a powerful model such as Fable or Claude. Collect a large set of question-answer pairs, then train your own student model “to make the student’s output behavior approach that of the teacher model you want to learn from, shrinking the difference between the two.” Once reasoning models enter the picture, the material available for distillation expands from static questions and answers to reasoning processes and then to the complete process an Agent uses to finish a task.
When 洪灏 asked, “Can this be understood as copying answers?”, Manqi’s correction was worth preserving. Distillation is used mainly in post-training and mid-training; it requires compute and the ability to put the data into model training. “It’s not like copying homework, where someone writes A and you write A. This requires methods and techniques, with your own intelligence and labor in the process.” 洪灏’s summary: “It is copying homework, but it is copying homework with a barrier to entry.”

2. Agent-era trajectory distillation and the boundary of “not distillation, but still a breach”

Agent trajectory distillation, which the article did not cover but the podcast added, works like this: A powerful model generates tasks by rewriting and expanding real-world tasks, builds an environment consisting of a codebase, terminal, compiler, unit tests and the rest of the system needed to run the task, then generates a trajectory by having Claude complete the task in that environment. The student model is trained to approximate the entire process behaviorally.
Manqi’s technical boundary: If you only use a stronger model to evaluate how your own model performs a task in an environment, that may not be distillation. It may instead be “using the data and output of some of the strongest models to help you do reinforcement learning for an Agent.” 洪灏 points out that Anthropic, OpenAI and Google DeepMind may view even that—using their output to improve a competing model—as a violation of their broadly written user agreements.
Do user agreements have legal force? Manqi declines to settle the question: “From a common-sense perspective, it is currently a gray area.”

3. Distillation began as compression: from HaoMo’s automotive chips to R1’s 6 small models

One of distillation’s original mainstream purposes was to compress a large model’s capabilities into a small one: lower costs, higher speed and deployment in low-compute, low-latency settings such as phones, cars and eventually robots. Manqi’s first extended in-person discussion of distillation came while interviewing 顾维灏, CEO of HaoMo: a large model trained in the cloud has to fit on a compute-constrained automotive chip such as Nvidia Orin, using distillation alongside pruning and quantization to reduce its size. Qwen’s practice of open-sourcing multiple model sizes falls into the same category.
Two data points anchor the timeline. In an article published this February, Google wrote that it had “observed a sharp increase since last year in activities such as distillation that steal IP.” Manqi’s own information is that “the scale has grown much larger from 2025 to the present.” Compression and capability gains are two sides of the same process: “When I compress a large model’s capabilities into a small model, I am also making the small model stronger.”

4. o1 and R1: the 2 milestones that improved distillation economics

The September 2024 release of o1 brought 2 changes. First, it showed that scaling reinforcement learning in post-training could teach a model reasoning strategies; since distillation is mainly used in post-training, “your return on investment goes up.” Second, the longer chains of thought generated by test-time compute are outputs from the model’s usage phase and theoretically visible to users—if hidden, they can still be recovered through various techniques—so they became “better raw material for distillation.”
The January 2025 R1 technical report shipped 6 distilled small models at the same time: 4 built on Qwen 2.5 and 2 on Llama 3, all using R1 as the teacher. The apparent purpose was to test whether this method could improve the capabilities of smaller models.
Why has the debate become especially heated in 2025-2026? The 3 US closed-source labs began publishing more papers and letters to the US government naming Chinese companies, while China’s open-source models moved noticeably closer to closed-source leaders. “K3 was a milestone,” triggering extensive discussion and controversy in the US. Those factors converged to push distillation into the spotlight.

5. 张一鸣’s ban: accept a short-term lag rather than take the shortcut

张一鸣 explicitly banned distillation at an internal Seed meeting for 3 reasons. Distillation “can improve model performance in the short term, but is fundamentally copying Claude’s existing capabilities”; if continued, it “can at most approach the other side, not surpass it.” He wants Seed to build its AGI moat from more fundamental dimensions, and made a strong commitment that even if avoiding distillation meant temporarily falling behind domestic competitors, Seed would not take the shortcut. Manqi also speculates that ByteDance’s global footprint may have prompted greater concern about compliance and how overseas companies view it.
The insider detail, in Manqi’s judgment, is that Seed 2.1, released in June, probably used some distillation. It should then have gone through a period of reflection and internal debate before the final decision to stop. In the more than 2 years before that—roughly 2-3 years—ByteDance genuinely did not do much distillation. Whether avoiding distillation explains why Seed failed to reach China’s SOTA is “one fairly important reason,” but “you can’t attribute everything to this one thing.”
The talent cost of that stance is real. Several people did want to leave Seed, for a highly personal reason: “Your career is finite.” They wanted to spend their most valuable years producing work they could be proud of, and to join a team that had trained at least China’s strongest model. Manqi points out the opposite effect may also exist: some people may be drawn to Seed precisely because of the stance.

6. Can a student surpass its teacher? Technically open, organizationally costly

Technically, it is not impossible. Multi-teacher distillation can “take the best from multiple sources”—though “if you train it badly, you may just be taking the worst from multiple sources”—and has already been used in post-training to combine capabilities from experts in different directions. Prior research has also suggested that a student may surpass its teacher on a specific task, without implying greater overall generalization or versatility. Manqi calls it “an open question that can be studied.”
The harder constraint is organizational. Distillation is “relatively cheap and relatively certain.” If an organization centers on that certainty, longer-term and more uncertain projects—and the people working on them—“may not receive enough resources and recognition.” Top researchers need an environment that encourages and rewards innovative work. The article’s analogy is that an athlete can theoretically take performance-enhancing drugs while training; “it’s just that once you take them, you will inevitably develop some complacency and inertia.”
One comment from a technically minded listener stayed with Manqi: “张一鸣 may not be the most technically knowledgeable person, but he is probably the person who understands human nature best.” Once both technology and organization are considered, Manqi says, 张一鸣’s claim that distillation may struggle to produce breakthroughs “may actually be true.” There is also a technical side effect: the student may inherit the teacher’s errors and undesirable behaviors.

7. ByteDance’s alternative path: elevating data to a top-level department and using third-party data as a workaround

The move is already under way. A report by 智能涌现 says ByteDance began reorganizing its data team in June and is setting up a first-level AI department, “AI Data and Safety,” parallel to Seed and Flow. Its head, 王银磊 (Adam Wang), previously ran livestreaming at TikTok and reported to 文佳 and Alex. 洪灏 infers that ByteDance needs to build the ability to obtain data without relying on OpenAI, Anthropic and similar companies.
洪灏 also raises a reported practice with a real-world analogue: Some companies may claim not to distill while achieving the same result “outside the model” by buying data from third-party data companies. Manqi’s distinction is straightforward: Technically, if you make your own model approach the output and behavior of a set of stronger models, “I would call that distillation.” Whether the practice can be kept outside the reach of law and user agreements leaves “a lot of room for maneuver”; if the US imposes tougher restrictions, it will no longer be merely a technical question.

8. The named list and the thought experiment of “what if nobody distilled?”

It is impossible to say definitively who is distilling. The public record is that Anthropic named DeepSeek, Kimi and MiniMax in February; in its June letter to the US Congress, it referred to Alibaba’s Qwen and 28.8M interactions, calling them “the largest-scale distillation attack ever uncovered.” Zhipu, notably, has not been publicly named by a US company.
After Alibaba was named, the entire group banned Claude Code. Manqi believes “the two events may not have a direct, necessary connection.” The move was more likely a compliance signal—Chinese users theoretically were not supposed to use Claude in the first place—combined with data-security concerns, though it clearly occurred against the broader backdrop of China-US model competition.
The thought experiment: What would happen if all leading players stopped distilling? Manqi’s answer is that everyone would be on equal footing on this particular issue, so the outcome would still depend on other capabilities and “might not change much.” 洪灏 immediately pushed back: “Then Kimi would still be SOTA and Seed would still be behind.” He was caught presupposing that everyone else was distilling while ByteDance was not.

9. A model calling itself GPT is not evidence of distillation

A November 2024 study tested 27 models with 77 questions designed to examine identity confusion. GPT-4 said it was GPT-3, Gemini Pro said it was Baidu’s Wenxin, and Seed said it was GPT. The conclusion was that identity confusion alone makes it difficult to determine whose training data a model used, and certainly does not establish distillation. Better tests examine whether output distributions are similar and whether overall behavior matches—including speaking style, output format, refusal patterns and coding style. “You need a very large sample”; asking once or twice and finding the models similar proves nothing.
Manqi asks what the purpose of the judgment is. For users, it does not affect usability. For an investor, secondary-market buyer or job candidate, however, the value lies in forming a technical judgment: “Is distillation, over the long term, actually good for a model R&D team?” The next question is whether the team really did it.

10. The real difficulty: accounts, intermediaries and data pipelines—and someone who distilled itself

The first hurdle is account access and user operations: How do you obtain stable, high-frequency access to powerful models? One practice mentioned by industry participants is to build numerous intermediary sites, allowing users who have genuine needs and can ask high-quality questions—STEM students, researchers and senior programmers—to access leading GPT or Claude models through them. Those real questions become the seed data, which is then expanded and rewritten.
A cautionary case provides one of the episode’s best lines. When working with a third-party intermediary, the provider may mix in models you never intended to use while connecting to a leading model: “I know of one model that used an intermediary site, only to discover that what it had actually distilled was itself—it was distilling itself.”
The second hurdle is a complete data pipeline: generating tasks; filtering, screening and deduplicating real-world data; using models to expand and rewrite it; correcting errors; choosing data formats and mixtures; and managing the cost and stability of the whole system. “Overall, it is a fairly complex systems-engineering project,” with real experience and know-how involved. Back to the copying-homework question: “If you insist on calling it copying homework, the process of copying is so complex that, in substance, it is no longer copying.”

11. Distillation leaves traces: electricity bills, behavioral fingerprints and Entity List risk

What if a company bypasses the API, downloads an open-source model and distills hundreds of millions of data points locally? Huang Renxun was asked about this in an interview on a July date in the 20s. K3 is close to 3T parameters and V4 is 1.6T; deploying models that large requires substantial compute, and continuously running data through them means “the electricity bill should also be pretty high.” The bottom line is that the activity “would be relatively easy to monitor.” If a government or anyone else views it as a risk, a government should have little difficulty detecting it.
Anthropic’s countermeasures look for distillation during the process rather than inferring it from the result. The company has built classifiers for identifying distillation traffic and behavioral fingerprinting systems that can detect cross-account coordination, repeated questioning and chain-of-thought extraction. It has also tightened identity verification for education, research and startup accounts—directly overlapping with the practice of recruiting students at universities to generate usage. Multi-teacher distillation may make any one teacher’s fingerprints harder to identify, but Manqi does not believe it can conceal the activity from closed-source model providers.
洪灏 spells out the concern: What if Kimi or Qwen wants to distill without ending up on the US Entity List? Manqi’s answer: “That is another cost of distillation—you do create compliance risks. If you want nobody to know, don’t do it.” Distillation is constrained by both cost and countermeasures; it is “not very likely to expand indefinitely or on a massive scale.”

12. The original-sin debate, the case-law boomerang and a 3-year forecast

“Original sin” is too strong a term. Manqi’s characterization is a gray area: Distillation may violate a closed-source company’s user agreement, but “violating a user agreement does not necessarily constitute legal infringement.” Her red line is clear: Hacking, directly penetrating another company’s systems or breaking into its chain of thought “is not really acceptable.” From an industry perspective, distillation is only one method among many—imagine a very large control panel, with distillation as just one knob.
The hypocrisy debate follows a familiar pattern: US companies accuse Chinese companies of distillation, while Chinese companies accuse the US of a double standard. Manqi concedes that “overall, it is definitely a double standard,” while noting that the conduct is not identical. Anthropic downloaded books from pirated sites without paying and was sued by US writers, ultimately reaching a $1.5B settlement; a California court precedent, by contrast, held that buying books and using them for training did not constitute infringement. 洪灏 turns the argument back around: “Would other companies using Anthropic’s output to improve their own models constitute infringement?” Manqi replies: “This is a pretty interesting case.”
A 3-year forecast is rejected on the spot: “This industry may not be able to see 3 years ahead. Thirty-six months from now, it feels like the world will already have changed.” What can be predicted is continued strategic iteration—you have a method, I counter it, then you find a new method. Distillation faces many practical constraints, so it is “not very likely to spread recklessly,” and everyone will weigh the costs.

13. The bigger question than distillation: has intelligence supply already outrun demand below the kill line?

On the frontier business model, Manqi warns against looking at only one slice. Short-term market expectations may be that Anthropic and OpenAI face valuation pressure, with some even arguing that their IPO timelines could be affected. But after Grok 4.6 launched, Musk replied to another company’s post that Grok 4.7 would surpass all current models, while adding that “Anthropic’s next model could be very strong.” Closed-source companies “may be holding cards they have not shown”; “right now, everyone leads for a few months.” Over a 6-month horizon, Anthropic and OpenAI have each held the lead for several consecutive months.
Is staying close by distilling enough for a follower? Manqi offers one hypothesis: If models reach a certain level of self-evolution and use more AI to optimize AI, their acceleration will keep increasing and “may throw the people behind them even farther back.” A gap of several weeks or months would then become much harder to close. “You can’t be satisfied with saying, I’m keeping up closely, so everything is fine.”
洪灏 believes the commercial logic has in fact been hit. On the famous “kill line” chart, the models below and to the right of V4 and Grok 4.6 are all eliminated, which should at minimum change pricing strategy. The founder of a productivity app told 洪灏 that users can have “8 out of 10 tasks solved by DeepSeek V4 Flash.” Pro had not officially launched at the time; Flash was cheaper and faster.
The discussion ends with a question larger than distillation: the scale and pace of demand for intelligence. An investor friend was asking, “We used to say coding was bigger than video generation—what if we were wrong?” For much white-collar work, once intelligence crosses a sufficient threshold, more is unnecessary: “Why use a butcher knife to kill a chicken?” or bring out a dragon-slaying sword? Manqi’s own view has shifted. After using ChatGPT Codex, she briefly became convinced it could do many things she previously could not; after cooling off, she found herself building tools by hand less often—and her demand for more complex tasks declining as well.

Uber President MacDonald: AI budgets, existential autonomy, DoorDash Excellence Is the Capacity To Take Pain | Travis Kalanick, Founder of Uber