Is an AI Idea Worth Pursuing? Don't Trust the Numbers on the Pitch Deck

Deep thoughts on AI and aspirations —— ByteDance Deep Thinking Circle

When judging whether an AI startup idea is worth pursuing, the least useful information comes from outcome metrics—ARR, retention rates, growth velocity. Not that these numbers don’t matter, but at the idea stage you simply cannot access real numbers. Without real numbers, what you cobble together is prediction plus packaging. Making decisions based on predictions is negotiating with your own imagination.

Truly useful judgment can be made before the idea even launches. It answers not “how much money can this business make,” but three more fundamental questions.

First Question: Have You Chosen the Right Problem to Solve

Good ideas start with good problems. The criterion is not “this need sounds big,” but three words: real, painful, frequent.

Real means the problem actually exists, not manufactured. The measure of painful is: will customers suffer if they don’t solve it? Does it directly connect to revenue, costs, or critical productivity? Frequent means: does it happen regularly, or once a year? High-frequency pain points are worth pursuing; low-frequency ones can starve you.

There’s a common self-deception trap here: mistaking “I want to build this” for “customers need this.” The distinction is simple—look at how customers currently solve this problem, whether they’ve solved it, and what they’re paying to solve it. Problems that customers are already solving today using some method (even if clumsy) are real problems.

Second Question: Is Now the Right Time

The second threshold for good problems is timing. Has the technology reached the tipping point of “scalable delivery”?

Many projects with stunning demos die halfway not because the concept is flawed, but because technical capability hasn’t reached the level customers will accept. 90% accuracy in a demonstration might drop to 60% on real data, while customers only accept 95%.

Signals for judging timing: Is the cost for this capability still dropping rapidly? Has anyone achieved scale with similar capabilities? If costs remain high and no one has made it work, you might be too early; if someone has already reached scale, you might be too late. The signal to convince yourself about timing windows is: at this particular moment, is there a specific technical shift that enables you to do something previously impossible?

Third Question: Is There a Feedback Loop

This is the least considered yet most critical criterion. Good AI products generate “objective feedback” with every use—knowing whether this iteration worked well and where it went wrong.

Take coding tools: the compiler tells you whether the code is correct—that’s deterministic feedback. The more certain and immediate the feedback, the more the product can self-correct and improve with use, forming a flywheel.

Conversely, if product quality depends entirely on users’ subjective feelings without objective signals, it struggles to self-evolve. When judging an idea’s value, first check whether it has a “know if it’s right after using it” mechanism. Without this mechanism, even the smartest AI is just spinning its wheels.

Don’t Forget the Most Practical Question: Augmenting People or Replacing Them

You also need to clarify: are you selling “a tool to help people work” or “results that replace people’s work”?

If it’s a tool, you need enough people using it to make money; if it’s results, you need to prove the results truly meet standards and customers will pay for outcomes. These two businesses have completely different valuation logic, pricing methods, and growth paths. Choose wrong, and everything downstream goes wrong.

The Counter-Argument: Every Framework Will Mislead You

Finally, a dose of cold water: these judgment dimensions combined are only a filter, not the truth.

An idea that looks real, painful, frequent, well-timed, and has good feedback can still fail; one that looks unpromising everywhere might succeed by accident. The framework’s purpose is to raise the right questions, not provide answers. Real answers only emerge after validation with actual users.

So don’t treat this as a scorecard—treat it as a “validation sequence checklist.” Validate the most fatal assumption first, at lowest cost. The framework helps you think through what to do first; reality tells you whether it can succeed.

Key points: Outcome metrics are useless at the idea stage; validate real problems (real, painful, frequent), timing (capability-cost curve), feedback loops (knowing if it worked after use); clarify whether selling tools or results; frameworks are validation checklists not scorecards.