Kevin Scott, CTO @ Microsoft: An Evaluation of Deepseek and How We Underestimate the Chinese

Summary

• Where does value accrue? Scott’s core call: “models aren’t products” — models are “super valuable but they’re only valuable to the extent that you can connect them to things that users need via product,” and “in the limit I think product is the most important thing.” Infrastructure and compute monetize along the way, but “we don’t build infrastructure just for the sake of infrastructure.” Harry’s confusion is characteristic of the moment — every paradigm shift (internet, mobile) starts this way, and “very few of those ideas were actually the durable ones.”
• On scaling walls: “that’s a ridiculous statement” — “I can very clearly see what we’re doing now and what we’re doing next and I don’t see the limit to the scaling laws.” His hedge is economic, not technical: there will eventually be a point of diminishing returns where “it’s so expensive that we will decide it’s not worth spending that next dollar to make this thing one unit smarter” — but “it’s not in the viewfinder right now.”
• On DeepSeek R1: “we’ve had models more interesting than DeepSeek R1 that we chose not to even launch.” It was “good solid technical work,” but just “a point on a line of price performance improvement that maybe was invisible to everyone else” — inference price-performance improves much more through software than through the ~2x-per-generation hardware gain. And the surprise itself was the tell: “we should really really really respect the capability of Chinese entrepreneurs, scientists and engineers… that shouldn’t have been surprising.”
• The agent thesis: memory is the piece “conspicuously missing” that makes agents “awfully transactional” — he expects it to get much better over the next year, and he hopes for more asynchronous dispatch (“you sort of dispatching your agent to go do something… while you are not paying attention”). The trajectory: 5-second tasks → 5-minute tasks → delegation “the same way that you would to a coworker.” And it’s many agents, not one — because product managers will probably have to be domain experts, helping set up agents’ feedback loops.
• 95% of net new code will be AI-generated in five years, “very little… line by line human written” — but “the more important and interesting part of authorship is still going to be entirely human.” It’s the assembly-to-high-level-language transition again (“there were some old farts who would say you’re not a real programmer… nobody talks about that anymore), and he hopes small teams will be able to do big things more easily: “you can do a lot with 10 really great super motivated engineers.”
• Positioning tells from the quick-fire: the competitor he most respects is maybe Anthropic (“I think Dario’s doing a good job”); “frontier models are probably better health diagnosticians than your average GP” already; and his biggest learning from the program’s first year: “there’s a bigger gap than there was even two years ago between what the most capable frontier models can do and what they’re being used for.”

Deep dive

1. Models aren’t products — value pools where product meets user

• Harry’s opening confession — for the first time in a long time he doesn’t know where value lies — gets framed as characteristic: “that’s exactly the thing that happens at the beginning of every big technological paradigm shift.” In the early internet and early mobile, “everybody had these ideas about what was going to be valuable and very few of those ideas were actually the durable ones.”
• Scott’s line of the past two years: “models aren’t products.” Pressed on whether that makes them not valuable (Andrew, likely from Cerebras, had told Harry compute wins), he’s emphatic: “no no no, they’re super valuable but they’re only valuable to the extent that you can connect them to things that users need via product… in the limit I think product is the most important thing.” Good models, infrastructure, and efficient compute all monetize — “as people build those products they will need to consume your platform” — but “we don’t build infrastructure just for the sake of infrastructure.”
• The prescription for confusion: don’t sit on your hands. “This is the best time to be alive if you have an entrepreneurial spirit.” Launch, gather data, and “be super super brutal with your own self… you can’t love your idea so much that you overlook what it is you’re seeing.” A tell of every early cycle: “technical people who get swept up in the technical bits and forget that the only thing that really matters is making good product.”
• Startups vs. incumbents: past cycles show “a pretty good mix of where value gets created” across both. Microsoft’s approach is applying new capability to customers it already serves; but “it’s impossible for any entity like Microsoft… to have enough imagination and enough perspective to know what every interesting thing is” — and the tools have never been “as cheap and accessible… as they are right now.”

2. Scaling walls: “ridiculous” — the limit is economic, and not in view

• On claims we’re hitting scaling limits: “that’s a ridiculous statement. I can very clearly see what we’re doing now and what we’re doing next and I don’t see the limit to the scaling laws.” Note the shape of his hedge: he intuitively feels there “must be” a limit eventually — unlike some who think intelligence scales “into weird territory” past the human 20-watt skull envelope — but the asymptote he expects is a cost decision: “it’s so expensive that we will decide it’s not worth spending that next dollar to make this thing one unit smarter.” For now, “it’s not in the viewfinder.”
• On data: synthetic mix is rising, and high-quality data plus expert human feedback is much more useful in post-training than low-quality data — you can “amplify that into the right set of tokens” worth far more than “the undifferentiated tokens floating around on the web.”
• The open question he’d most like answered: there’s no science of data value. “It’s very hard to know quantitatively what the incremental value of data is to the quality of a model… most of the assertions that people make are just unfounded,” and measurements show “a pretty big disconnect” between what some people think their data is worth and what it contributes. The category error underneath: people treating models “like the world’s worst and most expensive databases” — the point is reasoning over information, not recalling it, and “you just need different tokens” to train for that.

3. DeepSeek was a point on a line Microsoft already saw

• Scott’s frame for R1: years of “mind-boggling progress in optimizing the performance of models” mean models got bigger while API calls got cheaper — hardware gives “maybe a 2x benefit price performance every generation if you’re lucky,” but software gives “a much bigger improvement.” R1 is “a point on a line of price performance improvement that maybe was invisible to everyone else” but not to people neck-deep in these systems — “and it’s not the last point.”
• The internal reaction, delivered flat: “I was surprised at how interesting people thought that it was. We’ve had models more interesting than DeepSeek R1 that we chose not to even launch.” He credits the work — “good solid technical work,” and releasing it open source was “super cool” — the surprise was the public’s, not his.
• What he took from it: “developers want lots and lots of choice… we have to give people more ‘how’ than we have been.” And on China, no hedging: “we should really really really respect the capability of Chinese entrepreneurs, scientists and engineers… how surprised everyone seemed to be — that shouldn’t have been surprising.”
• A confessed change of mind on open vs. closed: “when I was a graduate student I was a complete open-source zealot… I’ve become a lot more pragmatic.” His model for the future is search: open-source engines exist, search-as-a-service exists, Bing and Google exist — yet “all of the economics in search go to somebody who stood up a gigantic infrastructure” with its own feedback loop. He expects “lots of both” here too.

4. The 200-year interface breaks — and it’s many agents, not one

• His favorite framing: “we’ve had one paradigm for using computing devices for effectively 200 years, since likely Ada Lovelace wrote the first program” — you’re either a programmer or dependent on one who anticipated your need. AI changes that: it “can understand a thing that you want your computing device to go do for you and figure out a way to make that thing happen.” Timing hedged precisely: “I don’t think this is next year but it’s probably not going to be 10 years.”
• The need for anticipatory software shrinks — teams that guess granular user needs, write code, hang it on a UI, and grind on feedback. “You just aren’t going to need as much of that anymore.” Engineers remain builders of capability infrastructure; the interface surfacing those capabilities “will probably be agents.”
• The contrarian bit: “I don’t believe in this one-agent-for-everything sort of theory. I think you’ll have a lot of agents” — because product managers will probably have to become domain experts in medicine, drug discovery, “or early round venture investing,” setting up the feedback loops by which PMs and users together teach agents their craft.

5. Memory and async dispatch turn agents into coworkers

• Harry’s pushback — we overestimate near-term adoption; will the world’s largest companies really run agents in one to three years? Scott’s answer: “usage always follows utility.” Software-development agents already flipped developers from skeptics to “you will get this from my cold dying fingers.”
• On the no-lock-in worry: “there’s no lock-in in search… you can send your next query to a different search engine — and yet you don’t.” Retention is earned by grinding “every day to make the agent better” — do that well and “they will continue to choose you.”
• The missing piece: “they are conspicuously missing memory, which makes them awfully transactional.” Memory “is going to get a lot better over the next year or so,” enabling abstraction and compositionality — solve a problem once, record it, never re-derive from first principles. He also hopes for “more asynchronous things happening over the next 12 months”: dispatch the agent and let it work unattended. The plot to never lose: 5-second tasks → 5-minute tasks → “increasingly beefy work… the same way that you would to a coworker.” His concrete bar: a 5 a.m. agent that digests overnight email and drafts urgent replies “while I drink my coffee.”
• On agent skeptics: “I don’t know what prize you win for being skeptical about something if you’re not going to go do something about it” — citing a colleague’s book title, There’s No Prize for Pessimism. Harry’s addition, via likely one of the Collisons: “pessimists are right and optimists make money.” Scott: “they are not wrong.”

6. 95% AI-generated code — abstraction rises, authorship stays human

• The number: in five years, “95% is going to be AI generated… very little is going to be line-by-line human written code. Now that doesn’t mean that the AI is doing the software engineering job” — authorship “is still going to be entirely human.” The analogy, as told: he wrote his first program at 12 (he’s 52, “41 years already”), in the assembly-to-high-level transition “there were some old farts who would say you’re not a real programmer if you don’t know how to write in assembly language… nobody talks about that anymore.” Same with GUI builders emitting “a crap ton of boilerplate.”
• The elite still go deep: “the extraordinarily good programmers… understand all the way down” and will “spelunk down into the lower levels of abstraction” when AI-emitted code breaks. Is everyone a programmer in a Bolt/Lovable world? “I think so” — but “the world’s hardest computational problems” still need computer scientists, now insanely well-tooled.
• Team structure: “I’m hoping it will get easier for small teams to go do big things… you can do a lot with 10 really great super motivated engineers with really powerful tools.” His internal goal at Microsoft: “I don’t want there to be any space between an engineer’s ambition… and their ability to go try it.”
• His mortal enemy after 20 years leading engineering teams: tech debt — “just like financial debt, it carries interest.” AI can turn that “very zero-sum problem” non-zero-sum; an MSR lab started about a year ago whose whole mission is “eliminate tech debt at scale.” The year-one learning generalizes: “there’s a bigger gap than there was even two years ago between what the most capable frontier models can do and what they’re being used for.”

7. Quick-fire: Anthropic respect, the histogram rule, and “are we going fast enough?”

• Competitor he most respects — Google, Anthropic, or likely Meta: “maybe Anthropic… I think Dario’s doing a good job.” On physical constraints: since GPT-4 “we have been running a thousand miles an hour building infrastructure… literally going as fast as is possible” — the binding limits are “the rate at which concrete can be poured and power grids can be augmented.”
• Best advice he ever got: picture competence as a histogram from idiot to genius — with great effort you move something “one maybe two buckets to the right,” so “all of the time you spend trying to get to mediocre you are not spending doing the things that you’re a genius at.” His own conceded weak spot: bureaucracy — “I could probably be a very mediocre bureaucrat if I wanted to be.” From likely Satya Nadella, the two-part leadership principle: “create energy and produce clarity.”
• The science-fiction prediction he holds as fact: “frontier models are probably better health diagnosticians than your average GP” — already — citing his own family’s inadequate care in rural Central Virginia; the world needs to “wake up to the fact that they’re good so that we can go deploy this stuff.”
• The question he is not often asked: “are we going fast enough?” His answer — no. What faster looks like: invest “super heavily in education” so every child feels these tools are built for them, and point incentives at deploying AI wherever “we currently think there’s scarcity” — healthcare, climate, education. “Let us go invest in that.”