Who Sets, Sells and Grades Large Models' Exams? AI Data's Wild Growth

Summary

• AI data valuations are expanding far faster than the underlying fundamentals can be understood: AfterQuery, founded by 3 people in their 20s, was valued at $300M in its Series A this April, then $3.2B in a new round in September—up 10x in less than six months and the fastest a YC company has gone from launch to unicorn; Meta’s investment in Scale AI implied a valuation above $29B, while Mercor reached a $10B financing valuation last October. Later in the episode, Yiwen argues that data vendors typically have less valuation upside than traditional platforms that can monetize an established user base and product over time; 孙一铀 likewise believes data vendors will struggle to sustain this degree of leverage.
• The industry’s center of gravity is shifting from human labeling to rubrics and RL environments. When 何允中 joined Scale AI, rubric data was still new; now, “rubric feels outdated—everyone is talking about RL environments.” He still stresses that pre-training matters. The core logic is weak-to-strong supervision: experts write down the grading rules, and a weaker model can supervise a stronger one by “holding the professor’s grading rubric,” while this year’s focus has shifted to coding and complete environments that let agents take action.
• The 2 guests openly disagree on whether rubrics have peaked, making the dispute a key signal for how much runway remains in the market. 何允中 believes the more accessible expert rubrics have largely been collected, especially as demand for static, single-turn or multi-turn text outputs shrinks; Berkeley’s 孙一铀 pushes back that “the rubric market is still far from saturated”—there is no mature quantitative evaluation system for expert judgments such as whether a game is fun or a corporate decision is good, which could become the next major demand pool.
• Benchmarks and data sales cannot be conflated. 孙一铀 says he has heard privately of data companies selling the same data they use for evaluation, and warns that commercial incentives cannot be allowed to compromise benchmark authority; his framework is that “benchmarks face the future, while data serves the present.” As long as a leaderboard tracks real user experience, “bench-maxxing is not necessarily a pejorative.”
• In vertical markets, procurement is becoming a major bottleneck—and an opening for founders. A MOBA game with only a few thousand players can command RMB2M–3M for its source code; chip design has to solve commercial-software licensing, and one company is said to have built a full AWS simulator. ALE V2 expanded Nature’s subfield taxonomy to the third level, covering roughly 3,000 nodes, and found that existing biology benchmarks may cover “less than 10%, perhaps even less than 5%”; in sensitive verticals such as healthcare, “whoever can obtain large volumes of case data will have a major advantage.”
• 何允中的 contrarian take is that frontier labs may not have the right organizational DNA to solve data problems, and that a data vendor’s real moat is R&D. If a vendor both places an FDE inside a company and collects that company’s data, it may create a conflict of interest: enterprises “will not be entirely comfortable handing their data to you.” The right data vendor should operate like a software company, invest in R&D ahead of time, understand the industry’s problems, and then sell the result to labs working against hard deadlines; in the end, “the competition is still about R&D,” and the winner will be whoever turns that capability into a flywheel.
• Fabricated data is harder to detect than contaminated data, and incentive design remains a vast unsolved research problem. Public-data contamination is relatively easy to identify, but an expert’s fabricated chemical simulation may take months to uncover; even the expert doing the verification may cut corners. “The reason crowdsourced benchmarks are so hard is that human nature is extremely complicated.” OpenAI stopped reporting SWE-bench Verified scores this February; 何允中 recalls the leaderboard rising only from 74% to 80% over the previous 6 months, suggesting that many remaining issues may lie in the questions or evaluation scripts.
• The next playbook is reaching private equity: buy a traditional company, use AI to improve efficiency, cut staff and restructure the business, raise its value—and then monetize the data as well. 何允中’s extreme scenario, which he calls the AI doomist view, is that if the current pace holds for 5–10 years and data from every industry is distilled, “data and large models could become the 2 biggest industries in the world.”

Deep dive

1. Valuation mania: AfterQuery’s 10x jump in less than six months and data vendors at $10B-plus

• Yiwen opened with the relevant coordinates: AfterQuery was valued at $300M in its Series A this April, then reached $3.2B in September, “more than 10x higher in less than six months”—the fastest a YC company has gone from launch to unicorn; Meta’s investment valued Scale AI at more than $29B, while Mercor reached a $10B financing valuation last October.
• The episode’s central question is the one investors most want answered: what exactly are these companies selling, and why are model companies willing to pay so much—from rubrics to RL environments and what leaderboard scores actually say about capability.
• The 2 guests bring complementary perspectives: 何允中 leads post-training and evaluation research at Scale AI and describes the company as “a kind of outsourced data research team for the entire industry” while noting that the views are his own; 孙一铀 is a UC Berkeley postdoctoral researcher and an investigator on Agents’ Last Exam (ALE). ALE is led jointly by Berkeley RDI and 300+ industry experts, with 150+ public tasks spanning 50+ industries, and aims to release 1,000+ tasks in V2 over the next few months.

2. The data-vendor map: recruiting, crowdsourcing, synthetic data and brokers

• 何允中 classifies vendors by “genetics”: recruiting-origin Mercor, with an automated AI interview product designed to find enough experts quickly; Scale AI, which came out of a crowdsourcing network and represents the traditional data vendor; new companies focused on synthetic data; research and engineering newcomers such as BigCode and Snorkel AI, which he suspects may mainly produce code or tool-call environments and capture this year’s theme; and pure brokers—“it could be someone in Hollywood with a huge amount of copyrighted data who knows how to sell it.”
• One interesting edge case is embodied AI: one company has not yet made money, but has invested heavily in producing data and has started selling that data on the side.
• Every data vendor faces the same trade-off: full-time experts are more capable but may offer less flexibility and capacity, while crowdsourcing is flexible but “makes it harder to guarantee each person’s quality.” One of Scale AI’s advantages is that it has experience with both data-quality control and labor-quality control.

3. From pre-training labels to RL: the rubric is really weak-to-strong supervision

• 何允中 revisited the industry’s pivot: after the o-series models arrived, “people realized RL worked, and suddenly everyone was interested in RL.” RL needs verifiable signals; math and coding have natural answer keys, but medicine, finance and instruction-following do not. Their answers are “less structured”—like answering a history exam with a standard answer key even though every student may phrase the answer differently, making pattern matching difficult.
• The core of a rubric is to turn an expert’s tacit knowledge into grading rules: “It’s like asking a TA to grade you using the professor’s rubric—the TA doesn’t need to be particularly smart.” That is weak-to-strong supervision in model training: a weaker model supervises a stronger one.
• But this year’s shift is that “everyone needs to take action”: validation is moving from chat outputs to agent settings involving tools, sandboxes, checking whether environment variables were executed and whether databases were written to. Validation can involve programmatic checks, rubrics or preferences, and “there is not yet much consensus.” Terminal-Bench, for example, may use no rubric at all and rely entirely on programmatic checks; different companies still want different things.

4. The fault line: have rubrics peaked? ALE wants to move from vibe coding to vibe everything

• 孙一铀 put the disagreement plainly: “I differ somewhat from 何允中—I think the rubric area, especially questions involving expert subjective judgment, is still far from sufficient.” 何允中 clarified that only “static, single-turn or multi-turn conversations that output a piece of text and then use a rubric to evaluate it” appear to be reaching a limit; training across industries is nowhere near finished.
• ALE began with a simple premise: vibe coding progressed quickly because coding has many benchmarks and is easy to verify. “If you are climbing a leaderboard using useful scores and meaningful questions, the resulting model improvement translates into real gains in people’s daily work.” The goal is to extend that mechanism to vibe everything, with an end state where solving a benchmark essentially means solving genuinely useful problems for society in a given domain.
• Engineering output is hard to evaluate because there is no single correct answer. A model can be asked to generate a game, and even some mathematical solutions are non-unique; forcing every output against ground truth creates inconsistencies. How to make rubrics compatible with diverse outputs remains an open benchmark-design problem.

5. The frontier of subjective judgment: a self-reinforcing loop and the “cute cat” hybrid

• 何允中’s framework is a self-reinforcing loop: as the model improves, the verifier improves; as the verifier improves, rubrics can become vaguer and delegate more knowledge to the verifier. The constraint is that “the knowledge and capability contained in the rubric plus the verification must exceed that of the model being trained, otherwise the training objective cannot be achieved.”
• Image generation makes the hybrid approach intuitive. Instruction following can use a rubric—“there needs to be a cat and a dog in the image; I can even run OCR to extract the text.” But “the cat needs to be cute” is deeply subjective and may require a reward model trained on human subjective experience, combined with a rubric.
• Yiwen suggested that whether a game is fun might be partially quantified using judgments from introductory game-design materials—for example, whether the reward curve is smooth or follows the brain’s dopamine-feedback mechanisms. But “there is still no particularly complete public evaluation system” because it requires deeper expert involvement.

6. Humans decide, agents execute: decision data is the hardest thing to distill

• 孙一铀’s task taxonomy runs from the Cursor era, when humans participated in execution, toward a model in which “humans make decisions and agents execute.” The clearer the task and the more unique the output, the better a deterministic rubric works. But if the decision itself is to be delegated to an agent, the problem changes: “For a decision by the leadership of a listed company, how do you train a reward? The task changes every day, and it is hard to collect enough data.”
• 何允中 echoed the point: at a company such as Meta, user behavior may provide enough feedback to train a model on whether a cat is cute, “but corporate decisions are much harder for me to collect data on.” ALE asks experts to submit projects and decisions they have made in the past, but the broader challenge is designing incentives that make people willing to disclose important work and allow those decisions to be reconstructed later.

7. Benchmarks versus data sales: the red line, the temptation and “future versus present”

• 何允中 explained why the business exists: data that improves model capability is tightly linked to benchmark methodology, and people who build benchmarks tend to understand the field deeply. That naturally gives them some authority to sell data—and the business emerges from there.
• 孙一铀 says the ecosystem is still developing in a rough-and-ready way, and warned that the industry has to distinguish scientific research from bench-maxxing. He has “heard privately that some data companies crossed this red line by selling the data they use for evaluation.” Commercial activity cannot be allowed to contaminate a benchmark’s authority; otherwise a vendor destroys not only its own benchmark but other people’s models as well.
• His framework is simple: benchmarks face the future, while data serves the present. When ALE started early this year, the team worried that an average score of only 10%–20%, with most questions receiving zero, would make the project irrelevant. “In fact, there was no need to worry, because a benchmark is meant for future models.” If a leaderboard stays close to real user experience, “bench-maxxing is not necessarily a pejorative.”

8. The 2 standards for a good benchmark—and doubts about Artificial Analysis’s weighting

• 何允中’s benchmark-selection logic starts with the highest-ROI source of capability gains: capability itself. “If these capabilities improve, the gains should benefit every benchmark.” If only one benchmark rises while the others do not, that is dangerous; solving an obscure, strange IQ puzzle does not necessarily mean cognition has improved. The second kind of good benchmark maps to a real use case, such as a personal assistant, and “may not even measure the model itself—it may also be measuring your harness.”
• 孙一铀 questioned Artificial Analysis’s methodology, which assigns different weights to different benchmarks and aggregates them into a score. “Are the weights in that distribution really the weights everyone needs?” The criticism is aimed less at the model than at the leaderboard itself. That is why leading labs such as Anthropic and OpenAI tend to build their own evaluation systems, while third-party evaluations make their external claims “more persuasive.”
• The ideal weighting should approximate user experience, somewhat like LMArena. But in the agent era, a long-horizon task can easily take several hours, and users do not have the time to run multiple agents and score them every time. Real ground truth is therefore difficult to obtain. “If any company finds a way to solve this, it will be extremely impressive.”

9. Randomness in the leaderboard ecosystem: SWE-Atlas, specialized ARC and Claude Code’s subjective win

• 何允中 shared the backstory of SWE-Atlas. Scale built 3 styles of SWE-Test—code-repository Q&A, code refactoring and test writing—and planned to release them in 3 stages. “Somehow, the Q&A version was the first to be picked up by Artificial Analysis, and suddenly demand for that area grew.” Q&A was actually the simplest setting; the more complex benchmarks had not yet launched or been incorporated. What appears on a leaderboard and what labs use contain “some random factors—the ecosystem is quite deep and complicated.”
• The lesson from ARC Challenge-style mini-game benchmarks is that the original goal was to measure human intelligence, “but no one expected that this intelligence could be made so specialized that even a small model could solve it.”
• 孙一铀 used Claude Code and Codex to show how subjective user evaluation can be. Claude Code has consistently led on LMArena because its output format is “closer to what people expect to see.” Codex “executes a bit more carefully,” but its output is harder to read and “lost a little on that front.” He personally prefers OpenAI because it is more meticulous and makes fewer errors. 何允中 added that online A/B tests and evaluation results often diverge: factual content, clickbait and telling users what they want to hear can produce entirely different feedback.

10. The endgame resembles search and recommendation, but scenarios are multiplying faster than users accumulate

• Drawing on years working on search and recommendation at Meta, 何允中 said that once a product is mature and has enough users, “what you ultimately care about is user growth.” The task is to identify which metrics correlate most closely with user growth, then move toward monetization.
• Large models differ from traditional search and recommendation because new scenarios keep appearing. “Everyone is now focused on new directions such as scientific discovery; I cannot possibly have enough scientists to produce significant A/B-test results.” The pace at which scenarios emerge is faster than users can accumulate in the field, so benchmarks will remain in demand as evaluation standards. Once a field matures, it can move toward online evaluation and metrics correlated with DAU.

11. Procurement is a major bottleneck: source code costing RMB2M–3M and a biology map with less than 10% coverage

• 何允中 divides post-training data into procurement—buying the raw material—and processing, including labeling and converting it into a training environment through an agent pipeline. “Procurement is becoming increasingly complicated in these verticals.” Small teams have focused on coding this year because software is highly open and has abundant public material. Moving into verticals creates harder problems: chip design must solve commercial-software licensing, while in DevOps “I know of a company that built a complete AWS simulator.” Whoever solves procurement gains a major advantage.
• 孙一铀 offered a striking price anchor. ALE V2 also tried to collect games from Steam and game-development tasks. One obscure MOBA with only a few thousand players reportedly wanted RMB2M–3M for its source code. “I don’t think any data company or benchmark builder has the financial capacity to do this.”
• The size of the gap is equally stark. ALE V2 expanded biology using Nature’s subfield taxonomy to the third level, producing roughly 3,000 nodes, and found that existing life-science and biology benchmarks cover “perhaps 10%, maybe less than 5%,” with no mature, comprehensive benchmark available. 何允中 singled out healthcare as a sensitive, scarce-data vertical. There are also industries outsiders barely know exist—for example, drug developers trade molecules, a single molecule can change hands several times, and each transaction requires DD.

12. 何允中的 contrarian take: labs may lack the data gene, while R&D is the data vendor’s moat

• 孙一铀 asked whether the rising ability of model companies to generate their own data would push the bar too high for non-leading vendors. In math, the industry once only needed to reproduce olympiad textbooks to generate data at scale, but “that data is basically worthless now.” 何允中 acknowledged that “if you focus only on the work in front of me, it will definitely get harder,” and margins will keep falling.
• His first-principles view is that the amount of data not yet distilled out of the world represents a huge opportunity. “Labs may not have the right organizational DNA to solve data problems.” Synthetic data should, in theory, be part of a lab’s post-training research, yet this year’s growth has come from people with industry expertise synthesizing the data externally and selling it back. Procurement can also create a conflict of interest: if one team sends an FDE into an enterprise while another collects that enterprise’s data, “as a company, I would not be entirely comfortable handing my data to you.”
• The real job of a data vendor is R&D: identify important industries in advance, buy the raw material, invest in R&D to understand the sector’s problems, turn the work into a benchmark or dataset, prove that it works, and then sell it to labs operating under hard deadlines. “In the long run, the competition is still about R&D.” The winner will be whoever turns R&D into a flywheel and builds a pipeline for continuously finding new opportunities. Buying out data—paying more for exclusivity—is the standard answer to the problem of selling the same dataset to multiple customers.

13. Training recipes, RSI and fabrication: incentives are the biggest unresolved research area

• Data delivery comes with different training recipes: simple data goes directly into SFT, while rubric data goes through RL. This year’s buzzword is RSI, or recursive self-improvement—“AI developing AI.” A run may take several days to a week and may require a GPU for a model to train another model, making the training regime “somewhat blurry” with little consensus. Customization can include having a lab use an unreleased model to calibrate difficulty—“if the questions are too hard, it cannot produce anything”—collecting trajectories, and finding ways for humans to generate solvable trajectories for SFT. Preventing reward hacking requires agents to red-team the system, and the reward may need to account for budget as well: “I may also care how many resources it consumes.”
• On contamination, Yiwen noted that OpenAI stopped reporting SWE-bench Verified scores this February, citing both test flaws and data contamination. 孙一铀 added that the leaderboard had risen only from 74% to 80% over the previous 6 months. 何允中 believes the remaining issues may include overly strict scripts, intrinsically unsolvable questions or other defects. If the remaining 20% is concentrated in such problems, the benchmark itself may lose its value as a comparison.
• Fabrication is worse than contamination. In ALE, some people seeking authorship “called an agent to synthesize a pile of nonsensical data,” and there were also noncompliant submissions such as court evidence. “The hardest thing is fabricated data—for example, asking someone to run a chemical simulation when they never ran it and simply made up something plausible. You cannot find out for months.” Hiring another expert to verify the result is expensive, since the work itself may take 1–2 weeks, and the verifier may also cut corners. “The reason crowdsourced benchmarks are so hard is that human nature is extremely complicated.”
• 何允中 sees incentive design as the way out, calling it a “huge research area.” One option is proof of work: “Require the person to record the entire workflow, sit there and work honestly for 12 hours, and submit every step. They may then have less incentive to fabricate the data.” His adjacent industry view is that once there are enough MCP and API calls, “more than 95% of so-called real-workflow data can be classified as coding,” because “code is the agent’s hands and feet.”

14. The new PE play, the valuation discount for data vendors and ALE 2.0’s incentive economics

• 何允中 described a new direction in private equity: the flip strategy now has another layer. Buy a company, use AI to improve efficiency, cut some staff and raise its value, then sell it—but “I can also extract its data, and that data can generate revenue.” Both data vendors and traditional PE are watching. His extreme scenario, which he calls the AI doomist view, is that if the current pace holds for 5–10 years and data from every industry is extracted, “this and large models could become the 2 biggest industries in the world.”
• 孙一铀 offered a sober valuation view: “Data vendors generally have less valuation upside than traditional companies because it is hard for them to sustain the leverage.” Unlike Facebook and Google, which could accumulate users and then “lie there and keep making money,” model iteration is moving at a terrifying pace, forcing data-vendor leaders to keep thinking about what comes next.
• ALE 2.0’s hardest problem reinforces the importance of incentives. Academic projects can often attract experts at this level to contribute data for free—not purely for money, but because they want to improve the project together or value authorship. Mercor’s published rates, by contrast, put manufacturing engineers at roughly $100–200 per hour. “That price makes it impossible to recruit particularly senior engineers. Different reward systems attract different people, which means your upstream is different.”