The Internet's Data is Running Dry—AI's Next Mine is Your Workflow
Deep thoughts on AI and aspirations —— ByteDance Deep Thinking Circle
The data business keeps turning pages. During the autonomous driving and early large model era, Scale AI made its first fortune from data annotation; when training shifted toward expert data, companies like Mercor and Surge rose with it; now demand has changed shape again. Model labs are starting to circle around something new: data generated within real enterprise workflows. Public internet data has been panned over repeatedly—this mine has barely been touched.
Why now? The answer lies in the main thread of model progress.
Tasks Got Longer, Sandboxes Can’t Keep Up
The keyword in this generation’s model race is long-horizon tasks. The unit of work models handle is expanding from a single code change to an entire file, a whole codebase, and ultimately a complete project.
The longer the task, the harder it is to manually construct environments. Real projects are full of historical states, tool dependencies, organizational rules, and unexpected branches. In a codebase that’s been running for eight years, which modules no one dares touch, which processes can’t bypass approval—experts sitting in sandboxes can’t invent these details from thin air. Lab budgets have thus shifted from expert annotation to reinforcement learning environments, and now to real-world data procurement.
This market also has a structural mismatch. Model companies typically have many engineers but little data; enterprises are exactly the opposite—lots of data, few people who understand training. With misalignment on both ends, two types of businesses grow in the middle: one supplies data to labs, the other helps enterprises transform business processes into trainable environments, and sells training consulting on the side.
Good Data Has One Standard: It Sits at the Capability Boundary
There are two kinds of data. One is captured from real work, close to a recording of the work process—screen operations, collaboration records. The best versions even preserve the intent behind actions. GitHub is the exemplar of this type of data: commit messages, issues, and solutions strung together, the entire journey from problem emergence to resolution. The other kind is artificially constructed—find experts, design tasks, execute in sandboxes, collect results. Constructed data works for early capability ramp-up, but as models begin handling tasks with longer chains and higher economic value, real data becomes increasingly valuable.
There’s a counterintuitive criterion here. Many teams design tasks with the logic that creating a task the model can’t do creates opportunity. But making frontier models fail is not hard at all. What’s hard is making the task land right at the capability boundary. A task where the model doesn’t get it right on the first try, but when allowed to attempt independently thirty-two times achieves a 30% success rate—that shows it’s occasionally finding the path, just unstably. This kind of data has training value. Another task that fails every single time across 256 attempts is likely beyond scope or too obscure; feeding it in won’t help the model learn. Selecting data isn’t about picking the hardest—it’s about picking what’s just within reach.
The market is hot, and the pitfalls are systemic. Several types of distortion circulating in the industry are worth remembering: packaging constructed data as real data; deliberately increasing difficulty in eval sets to create the illusion that models still have much room for improvement; quality control relying on piling up human labor, where quality collapses once scale increases. My experience is that looking at a data company’s hiring structure reveals its DNA: frequent hiring of project managers means it’s selling human labor to take orders; hiring only engineers means it’s building a factory. As data value shifts toward the real end, the window for pure constructed-data companies may have only two years left.
Should Your Company Train Its Own Model?
As workflows become training data, enterprises naturally wonder: should we do it ourselves? There’s lots of discussion in Silicon Valley; those who get serious all have specific reasons.
Customer service companies were among the first to build in-house. General models are trained to be friendly and compliant, but enterprises don’t want models to grant every refund request—this kind of preference can’t be solved with prompts. Another reason is cost: if a coding tool company buys models at external API prices, every new user means paying a toll to competitors, who use that money to subsidize their own products. Training in-house to 80-90% of frontier model capability at 10-20% of the cost makes the competitive feel even out.
But in-house training is essentially trading one-time capital expenditure for long-term inference costs. Whether this math works out depends on four multipliers: data uniqueness, times evaluation clarity, times task frequency, times value of single improvement. If any multiplier approaches zero, the investment doesn’t pencil out.
| Multiplier | What It Determines | What Zero Looks Like |
|---|---|---|
| Data Uniqueness | Whether general models can learn it | Data is just messy, not unique |
| Evaluation Clarity | Whether training can optimize stably | Engineering with no acceptance criteria |
| Task Frequency | Whether costs can be amortized | Capability used twice a year |
| Single Improvement Value | The ceiling on returns | Savings amount to pocket change |
A positive case is menu entry for food delivery platforms. Over 100,000 merchants join annually, uploading menu images. The system needs to convert information in images into electronic menus according to internal platform rules—items, add-ons, specifications each have their own standards. General models recognize menus but don’t understand this internal ruleset, so the platform used internal data and collaborated with external vendors to train a small model that understands its own business standards. All four multipliers positive—the math works out.
In-house training has a hidden advantage. Companies occupying real workflow entry points naturally produce high-quality agent execution traces—this type of data can command over a thousand dollars per instance on external markets. And enterprise task scope is narrow, no need to worry about cross-domain generalization, making capital efficiency actually higher. General labs doing coding reinforcement learning must be careful about other skills regressing; vertical companies don’t have that baggage.
The Boundaries of This Business Should Be Clear Too
Three sober judgments.
First, the data market is a surfing business. Whenever a capability bottleneck breaks through, the most valuable data changes form, and players turn the page with it. If last round’s winners bet wrong on next round’s demand type, moats instantly become sunk costs. When valuing data companies, don’t value them as assets—value them by window period.
Second, enterprises are generally too optimistic about their own data uniqueness. Most so-called unique data has problems with messiness, not uniqueness: inconsistent calibers, unrecorded processes, unclear permissions. In the four multipliers, what often goes to zero first isn’t uniqueness but evaluation clarity—tasks where “done well” can’t even be defined leave training nowhere to start, and no amount of data is more than raw material piled up.
Third, this is an exchange with implicit conditions. Enterprises hand over workflow details in exchange for model capabilities or better service—data sovereignty and boundaries are worth spelling out in contracts. The more valuable your process data is, the more you should think through what you’re getting back.
For entrepreneurs, there’s only one criterion: data companies without a roadmap moving toward real data aren’t worth investing in or building. For enterprises, calculate the four multipliers first—if the math doesn’t work, don’t touch training. Treat data as a negotiable asset to exchange for better model services. The best position always belongs to whoever holds the workflow entry point—the mine is on your land, excavation methods can be negotiated slowly.
Key points: Long-horizon tasks make real workflow data the new scarce resource; two-sided mismatch in the data market spawns companies that supply data and those that build environments; good data sits at the model capability boundary—too hard and too easy both lack value; enterprise in-house training depends on four multipliers: data uniqueness, evaluation clarity, task frequency, single-instance value; the data business is valued by window period, uniqueness is often overestimated, evaluation clarity most commonly absent; occupying workflow entry points is the best structural position.