91. A Paper-by-Paper Walkthrough of DeepSeek's 9 Key Papers and Their Innovations — “The Game of the Brave”

Summary

• DeepSeek’s core trajectory is not a single model, but a technology roadmap that keeps branching around efficiency. From more experts and shared experts in DeepSeek-MoE to MLA, GRPO, FP8 training, and multi-token prediction, each step was aimed at lowering costs and improving efficiency rather than innovating for innovation’s sake, gradually diverging from the mainstream path.
• R1’s key breakthrough was converging from reward models to rule-based rewards. 何俊贤’s team and others had tried using reward models for reasoning RL, with disappointing results; checking the final answer in math and running unit tests in coding is often simpler and more stable than having another model score the response. Open-ended tasks may still require reward models.
• V2 and V3 showed that model scale and cost can be decoupled. V2 has 236B total parameters and 21B activated parameters, uses 42.5% less training compute than DeepSeek 67B, cuts KV Cache by more than 90%, and generates 5.76x faster; V3 trained on 2,048 H800s over 14.8T tokens for about $5.57M, with no loss spike or rollback.
• DeepSeek LLM’s value was not merely surpassing Llama 2, but treating reproduction as science. It revisited learning-rate schedules, hyperparameter scaling laws, attention’s computational overhead, and data quality, and used controlled C-Eval experiments to publicly show how leaderboard gaming could lift scores from 47 to 71.
• GRPO is a key building block in DeepSeek’s effort to reduce RL costs. It removes PPO’s value model and uses the mean reward across a group of responses to the same prompt as the baseline; DeepSeek-Math also found online RL superior to offline methods, while candidly noting that RL did not lift Pass@K in tandem and may simply have moved answers the model could already produce to the front.
• DeepSeek’s organizational signature is low-profile, open, detailed, and willing to absorb the cost of trial and error. 何俊贤 sees it as resembling a university lab: its papers disclose extensive detail, and the team is not in a hurry to promote itself or chase leaderboards; he also acknowledges that public papers mainly show successful results, with unpublished failures necessarily behind them.
• The base model still has room to scale, but high-quality data may be approaching saturation. Larger models and more tokens may continue to help, while the next phase could shift toward synthetic data, RL scaling, and test-time scaling; R1 is itself an example of extending pretraining scaling into RL scaling.

Deep dive

1. R1-like work preceded R1; the real convergence was on simple rules

• 何俊贤’s team began related work about two months before DeepSeek-R1 was released and published its own results about five days after R1. The scale was smaller and the method similar, but he stresses that the more accurate description is “R1-like,” not a straightforward reproduction of R1.
• They had previously spent more than six months trying to use a reward model for reasoning RL, with consistently poor results. They eventually found that the simplest approach worked best: check only the final answer in math, and use unit tests for coding rather than asking another model to score the response.
• He believes Kimi 1.5, DeepSeek, and many other teams ultimately converged on similar methods. The community may have taken a detour by following OpenAI’s work on process-supervision reward models; 何俊贤 also believes the approach described in OpenAI’s papers may not have been fully identical to the way it built o1.

2. 5,000 A100s put 幻方 on 何俊贤’s radar before DeepSeek existed

• In late 2022, before ChatGPT had appeared and before the industry had formed a concept of foundation models, 幻方 began publicizing that it had around 5,000 Nvidia A100s. For a quantitative firm that may have had only 100-plus employees, both the scale of the compute and its intended use were highly unusual.
• 幻方 turned temporarily idle compute into a cluster, built systems for scheduling, allocation, and usage, and opened it free of charge to academic researchers. 何俊贤 initially suspected it might be marketing or PR, but after using the system himself, he found it highly mature.
• The experience gave him the impression that 幻方, despite its small headcount, could build and maintain a supercomputing system in-house; it hired selectively against a high bar and paid relatively well.

3. DeepSeek was not an early entrant and was initially underestimated

• In the first half of 2023, before DeepSeek had been formally named, 何俊贤 had already heard that 幻方 might incubate a company modeled on OpenAI to explore AGI.
• China was in the middle of a “hundred-model race,” and internet giants already had accumulated AI capabilities, while 幻方 was coming from quantitative finance and was late to foundation models. Many people, including 何俊贤, were initially unconvinced and saw it as just another startup entering a crowded market.
• What changed his view was the first DeepSeek LLM paper: both the model and its technical details were made public, the paper was high quality, and its writing resembled that of a university lab rather than a company releasing only product results.

4. A low-profile, open, relatively low-pressure research culture

• 何俊贤 considers DeepSeek unique: it is not only open source, but also unusually honest in its papers, disclosing extensive detail. Its overall character feels less like a typical commercial company and more like a pure research lab.
• From the second half of 2023 through early 2024, the industry repeatedly promoted models as surpassing GPT, ChatGPT, or GPT-4, while DeepSeek rarely joined in. Its products and consumer-facing moves also came relatively late.
• He speculates that the team did not appear to be operating under extreme pressure to surpass a particular competitor within a few months, which left it willing to invest in unproven architectures. This is his impression from the papers and the team’s behavior, not an organizational fact publicly confirmed by DeepSeek.

5. Two continuous technology tracks

• The base-model track began with DeepSeek LLM and continued through DeepSeek-MoE, DeepSeek-V2, and DeepSeek-V3. 何俊贤 believes V2 had already delivered many of the key innovations, while V3 was more about large-scale training and engineering on top of that foundation.
• The reasoning track began with DeepSeek-Coder and DeepSeek-Math, continued through DeepSeek-Prover, and ultimately led to DeepSeek-R1.
• He focused primarily on the text modality and did not cover the multimodal papers from the past year in depth, because multimodality was not DeepSeek’s main area of emphasis at this stage.

6. DeepSeek LLM: a rigorous reproduction of Llama 2

• DeepSeek LLM came in 7B and 67B sizes, used roughly 2T tokens, and was followed by SFT and DPO. Its architecture and main training methods largely followed Llama 2, while its data placed greater emphasis on Chinese-English bilingual coverage and quality.
• Its surpassing Llama 2 70B did not surprise 何俊贤: later entrants could improve data quality, and other Chinese models had also surpassed Llama 2 70B by then.
• What really mattered was that DeepSeek did not simply copy the configuration. It continued studying the underlying regularities of training and treated model development as a scientific problem.

7. Multi-step schedules and scaling laws

• The commonly used cosine learning-rate schedule required the total token count to be fixed before training began. If new data were added mid-training, the original curve was difficult to adjust.
• DeepSeek instead adopted a multi-step learning rate: hold the rate constant at first, cut it sharply at stage boundaries, then continue at a new constant. Experiments showed that final performance was close to that of a cosine schedule.
• It also studied the scaling laws for learning rate, batch size, and other hyperparameters, using small-model experiments to extrapolate settings for larger models. Earlier scaling-law work focused more on model size, data size, and configurations under fixed compute; DeepSeek went further by considering the computational overhead introduced by attention.
• The paper also discussed how data quality affects the optimal balance between model size and data scale. 何俊贤 sees this as evidence that the team was not simply throwing resources at the problem, but trying to understand the science underneath.

8. Using small-model experiments to predict large-model performance

• The paper fit the relationship between performance and FLOPs using models trained with relatively small compute budgets, then predicted results at larger compute scales; the actual results for the 7B and 67B models were reasonably close to the predictions.
• The value of this approach is that it allows researchers to estimate the likely outcome of large-scale training without first fully training a large model. 何俊贤 linked it to similar prediction charts in the GPT-4 paper.

9. C-Eval exposed the mechanics of leaderboard gaming

• C-Eval was a Chinese benchmark released by 何俊贤’s team in May 2023. Soon after its launch, leaderboard gaming emerged: scores kept rising, but real-world capability did not necessarily improve in tandem, producing a pattern of “high scores, low ability.”
• DeepSeek ran a controlled experiment and found that adding large quantities of similar multiple-choice training questions could lift the C-Eval score from about 47 to 71, an increase of more than 20 points.
• What 何俊贤 respected most was that DeepSeek did not publish only the post-gaming high score. It reported the pre-gaming result and explained how the benchmark could be gamed and how far the score could be pushed.
• 何俊贤’s team later evaluated multiple base models from China and abroad and concluded that the base model released by DeepSeek showed no obvious leaderboard gaming. He contrasted it with a number of high-scoring models.

10. MoE separates total parameters from per-token computation

• In a dense model, every input passes through all parameters. MoE splits the Transformer feedforward network into multiple experts, with a router activating only a small subset, which is why it is also called a sparse model.
• For example, a model with 100B total parameters divided among 10 experts might use only about 10B parameters for a given input. The model can store more knowledge while reducing the compute required for each training and inference step.
• DeepSeek was not the first team to study MoE: Google had developed GShard earlier, and the industry had speculated that GPT-3.5 or GPT-4 used MoE. DeepSeek’s difference was that it began pushing the efficiency-oriented MoE path forward relatively early and continued doing so.

11. DeepSeek-MoE: more experts and shared experts

• DeepSeek-MoE was more of a study than a fully mature product model. Most experiments were conducted at smaller scales, and the final 145B model was not trained to completion.
• Its two core designs were increasing the number of routed experts and adding shared experts. The paper used 64 routed experts and 2 shared experts; the 2B and 16B models had roughly 2.8B activated parameters.
• Traditional MoE systems commonly used 8 or 16 experts. DeepSeek argued that this partitioning was too coarse, and that more experts would allow knowledge to be divided more finely.
• Shared experts handled general capabilities that could be needed by any input, such as language understanding and common sense; routed experts remained input-dependent.
• The paper concluded that using roughly 40% of the original computation could deliver performance close to that of a 7B dense model. Although the 145B model was trained on only 200B-plus tokens, far fewer than the 2T tokens used for the first model, its benchmarks with about 22.2B activated parameters were broadly comparable to those of a 67B model.

12. V2 expanded the MoE design into a complete model

• DeepSeek-V2 is an MoE model with 236B total parameters and 21B activated parameters. It was trained on 8.1T tokens and supports a 128K context.
• Compared with DeepSeek 67B, V2 uses 42.5% less training compute, cuts KV Cache by more than 90%, and generates 5.76x faster. Deployment also uses low-precision processing such as FP8, along with quantization of the KV Cache.
• V2 continued the shared-expert and large-routed-expert design, using 2 shared experts and 160 routed experts. Compared with Mixtral, it has more total parameters but fewer activated parameters, resulting in lower deployment costs.
• V2 triggered a domestic API price war in May 2024. 何俊贤 heard that DeepSeek was not losing money deploying it, but that margins were thin; he also noted that some people called it “Pinduoduo for foundation models.”

13. MLA addresses the cost of KV Cache

• Standard Multi-head Attention stores an independent key and value for every head. During autoregressive generation, each new token must attend to the preceding context, so historical K and V are stored in the KV Cache to avoid recomputing them.
• GQA lets multiple heads share a set of K and V, while MQA lets all query heads share one set. The cache becomes smaller, but expressive power may decline, creating a tradeoff between cost and performance.
• MLA does not store high-dimensional K and V directly. It first compresses them into low-dimensional latent vectors and maps them back when needed. If the original representation has 1,000 dimensions and the latent vector has only 100, storage requirements can fall sharply.
• 何俊贤 considers MLA an architectural innovation proposed by DeepSeek itself. At a compression level equivalent to roughly 2.25 GQA groups, it performs better than simply reducing the number of groups, and the design was carried forward into V3.

14. MoE’s real efficiency depends on engineering balance

• MoE cannot allow all tokens to concentrate in a small number of experts, or the remaining experts and their GPUs will sit idle.
• The system therefore has to balance expert load, device load, and communication across GPUs so that the cluster maintains high utilization.
• These issues may be less eye-catching than MLA, but they directly determine whether a sparse architecture can translate into real cost advantages. On an H800 cluster, V2 used about 300K GPU hours per 1T tokens, further reducing training costs versus the previous generation.

15. V3: a 671B model trained at low cost

• DeepSeek-V3 was released in December 2024, with 671B total parameters. It was trained on 14.8T tokens using 2,048 H800s at a cost of about $5.57M.
• V3 retained V2’s MLA, shared experts, and large number of routed experts, but its training run experienced neither a loss spike nor a rollback. 何俊贤 sees this as evidence of strong engineering and infra capabilities behind the model.
• The V3 paper devotes extensive attention to training stability, communications, precision, and cluster engineering, not just algorithm design. For a 671B model, 2,048 GPUs is not a large fleet, which was a major reason the result drew attention.

16. V3’s succession of innovations

• V3 uses 1 shared expert and 256 routed experts, continuing the steady increase in expert count from DeepSeek-MoE to V2 and then V3.
• It adopts loss-free balancing: instead of adding a balancing loss, it monitors how frequently each expert is selected. When an expert is selected too often, the system adjusts a constant so that subsequent inputs are less likely to route to it.
• V3 also uses multi-token prediction. DeepSeek did not originate the technique, but it was the first to apply it at such a large training scale and validate its impact through ablation studies.
• The technique provides a denser training signal and may also support speculative decoding during inference. 何俊贤 is unsure whether DeepSeek actually used it in product deployment.
• V3 also successfully carried out large-scale FP8 mixed-precision training. Some intermediate variables still need to remain at higher precision, or training can become unstable and performance can deteriorate; this was an important engineering optimization.

17. V3’s performance and post-training

• In comparisons with Llama 3.1 405B, Llama remained a dense model with nearly 405B activated parameters, while V3 had 671B total parameters but only about 30B activated parameters, putting its deployment cost more than 10x lower.
• V3 was competitive with Llama 3 in English, while substantially outperforming Llama 3 405B in reasoning, code, math, and Chinese.
• V3’s SFT used only about 1.5M examples. Its reasoning data was distilled from an internal DeepSeek-R1; R1 had not yet been released, but already existed internally as a model built on V3, and later helped generate the data needed for V3 chat.
• The team also used DeepSeek-V2.5 to generate long-reasoning data and ran RL. Math and coding used verifiable rule-based feedback, while open-ended question answering may have used a reward model.
• 何俊贤 believes V3’s post-training was relatively shallow and did not show an obsessive focus on leaderboard optimization, but he explicitly says he is not 100% certain and is inferring this from the paper and the model’s behavior.

18. DeepSeek-Coder was an early gateway to overseas awareness

• The first DeepSeek-Coder was a dense model ranging from 1.3B to 33B, with an architecture that largely continued the first-generation DeepSeek LLM and training data focused mainly on code.
• DeepSeek-Coder-V1.5 used continued pretraining: it continued training from the existing DeepSeek LLM 7B for about 2T tokens, roughly 70% of which was code.
• 何俊贤 believes overseas users’ early impression of DeepSeek came largely from Coder. General-purpose models had many alternatives, while the total parameter counts of models after V2 made them difficult for ordinary developers to deploy; Coder’s small and mid-sized versions were easier to use.
• Coding was also one of the earliest clear use cases in which foundation models delivered real-world productivity gains for programmers.

19. Why Coder V2 still used a reward model

• DeepSeek-Coder-V2 was based on a DeepSeek-V2 checkpoint and continued training for about 6T tokens, making it essentially V2’s code model.
• It continued to use a reward model for coding rewards because unit-test coverage could be incomplete: incorrect code might occasionally pass the tests, making a simple 0/1 rule-based signal noisy.
• 何俊贤 does not see this as a rejection of rule-based rewards. With broader unit-test coverage, rules can remain consistent; reward models may work only on part of the data distribution and expose generalization problems as the scale expands.
• At the time, DeepSeek and the broader community generally believed coding required a reward model. Math and reasoning tasks only gradually converged on rule-based rewards later.

20. DeepSeek-Math and GRPO

• DeepSeek-Math is a 7B math model based on DeepSeek-Coder-Base-V1.5 7B, with about 120B additional math tokens. At the time, it was one of the strongest open-source math bases.
• The paper introduced GRPO. PPO typically requires a policy model, reward model, reference model, and value model; GRPO removes the value model, samples multiple responses to the same problem, and uses the group’s mean reward as the baseline to calculate each response’s advantage relative to that mean.
• This removes the memory and compute burden of one large model. DeepSeek later continued using GRPO in V2, V3, and R1, and it became widely used in open-source RL frameworks after R1 was released.

21. Online RL and the limitations of Pass@K

• DeepSeek-Math studied online reinforcement learning. Offline methods generate a batch of data first and then train on that fixed batch; iterative methods regenerate data after each generation of training; online methods continuously generate new data as the model updates.
• The paper found online methods better than offline methods, but more expensive, less stable, and harder to tune. At the time, the open-source community also lacked mature online-RL codebases.
• The paper also analyzed Pass@K: sample K responses to the same problem, and count the problem as solved if any one is correct. After RL, the K=1 benchmark score rose sharply, but as K increased, the gap between pre- and post-RL results narrowed; in some cases, the model without RL was even better.
• 何俊贤 believes this may mean RL merely moved answers the model could already generate to the front, without fundamentally improving its ability. DeepSeek included this unfavorable conclusion in its own paper, which he sees as part of its scientific honesty.
• Rule-based rewards generalize stably: a math problem with a standard answer can be judged correct or incorrect regardless of difficulty. Their weakness is sparse rewards. Process rewards theoretically have a higher ceiling, but are currently difficult to make accurate enough and robust across domains.

22. DeepSeek-Prover: external verifiers provide rule-based feedback

• DeepSeek-Prover handles mathematical theorem proving and uses Lean to determine whether a proof is valid in formalized mathematical language. The model first converts a natural-language problem into a formal language, then submits it to Lean for verification.
• The first stage uses iterative self-improvement: generate proofs, verify them with Lean, discard incorrect results, retain correct ones, and continue training on the resulting data.
• Starting with Prover-V1.5, the system used RL and GRPO. Because the 0/1 reward for theorem proving is extremely sparse, the team did not retrain a reward model; instead, it removed overly difficult data for which correct proofs were almost never generated.
• Later work also tried variants of MCTS. But 何俊贤 believes R1 ultimately used a simpler approach, with neither MCTS nor complex decoding.

23. R1-Zero and R1

• R1’s rewards are mainly accuracy reward and format reward: the former checks whether the answer is correct, while the latter checks whether the required format is followed, such as outputting think before the answer. Both are rule-based rewards.
• R1-Zero applied RL directly to the base model without preliminary SFT, distillation, a complex reward model, or MCTS. On math tasks such as AIME, performance rose from roughly 0.2–0.3 to roughly 0.7–0.8; the chain of thought also grew longer over time, producing the so-called “aha moment.”
• R1 first underwent cold-start SFT using long-sequence data and then RL; the DeepSeek-R1 that users work with today is primarily this version.
• The R1 paper is relatively concise because the V3 base, GRPO, rule-based rewards, and earlier exploration of multiple failed paths had already been completed. R1 did not appear out of nowhere; it was a concentrated combination of the successes and failures that came before it.
• On why R1’s prose is elegant and emotionally resonant, 何俊贤 offers only two hypotheses: DeepSeek may have accumulated high-quality Chinese and humanities data, or the logic and reflection skills developed through long-reasoning training may have transferred to liberal-arts tasks. He stresses that he cannot tell which mattered more.

24. Culture, resources, and the next phase

• 何俊贤 does not know DeepSeek’s ultimate commercial motivation. V3 is open-sourced under the MIT License and permits free commercial use, but that alone cannot establish whether the team cares more about commercialization, making money, or simply exploring AGI.
• He believes DeepSeek publicly disclosed roughly 10,000 older A100s in its early phase, while V3 used 2,048 H800s. Its cost advantage came from MoE, MLA, GRPO, and engineering optimizations in communications, load balancing, and precision handling.
• He also cautions that papers publish successful work, so DeepSeek must have additional failed experiments that have not been made public.
• The base model still has room to scale, but high-quality internet data may be approaching a ceiling. Larger models and more data may continue to work, though marginal returns could be saturating; synthetic data, RL scaling, and test-time scaling may become the focus of the next phase.
• 何俊贤’s summary of DeepSeek is that it neither simply follows authoritative paths such as Llama and Mistral nor innovates for innovation’s sake. It keeps making different choices in pursuit of lower costs and greater efficiency. The accumulated differences—GRPO, MLA, more experts, shared experts, FP8, and multi-token prediction—have ultimately produced a technology roadmap increasingly distinct from those of other models.