Four questions before you commit months to a build

The questions did not change. The discipline lapsed.

Good product people have always asked these four questions. When a build took a quarter, the cost of building made you ask them first: you questioned the idea hard, because getting it wrong was expensive. Now the build takes a weekend, so teams skip the questions and let the working thing stand in for the answer.

AI is the worst place to make that trade. An AI demo is the most convincing thing in software. It looks finished, it handles the cases you show it, and it tells you almost nothing about whether the product underneath holds up with a real customer. The questions below are the ones the demo can't answer for you.

Run them on whatever you're about to commit months to. Each one comes with a tell: what a real answer sounds like, and what the comforting answer that usually replaces it sounds like. Watch for the comforting answer, because it's the dangerous one. It's a real answer, just to an easier question than the one you asked, and in the room it sounds close enough to pass. A real answer is specific and a little uncomfortable. A comforting answer feels like relief.

1. Who actually pays for this, and why now?

This is the question founders least want to hear, which is usually a sign it's the one that matters. It has two halves, and both have to land. The who rules out the build that solves a real problem nobody has budget to fix. The why now rules out the build that's been possible for years and that nobody has bought, because whatever stopped them buying rarely went away on its own.

The AI version of getting this wrong is specific: "why now" gets answered with "because AI." But AI is a capability, not a reason anyone signs a contract. "We can finally build it" is not the same sentence as "they'll finally pay for it," and the gap between those two is where a lot of AI roadmaps sit.

Real answer

the person who signs, the budget line the money comes from, and what changed in their world this year to make now the moment.

Comforting answer

a list of everyone who said the demo was impressive.

2. What happens the first time it is wrong?

Plenty of products are wrong sometimes and the customer stays. They expect it, they work around it, they shrug and carry on. What matters is which kind of product you have: the kind where a wrong answer is a normal cost of doing business, or the kind where being wrong once costs you the customer's trust for good. In regulated, financial, or safety-critical work it's often the second kind. A wrong answer in front of the wrong person does real damage, and being right afterwards doesn't undo it.

AI makes this harder, because it gets things wrong in ways you can't fully predict. You can't list every case where it will fail the way you can with ordinary software, and "right most of the time" is a different product from "right." Where the cost of a wrong answer is high, that difference isn't a rough edge. It's the main risk, and you have to design for it from the start. Bolt it on later and the feature hits a wall fast.

Real answer

what the product does when it is wrong, who sees it, and what that failure costs you.

Comforting answer

"it will be accurate enough."

3. What does it look like outside the demo?

The demo runs on the clean case: tidy inputs, a happy path, data someone sorted out before the meeting. Production runs on what actually shows up - messy, missing, contradictory, the stuff nobody planned for. Plenty of products work beautifully in the room and never work again.

The AI version of this is the most common one I see, and the most hidden. The demo worked because a person had cleaned up the inputs. In production that person isn't there, and the value the demo promised was quietly resting on their judgment the whole time. The question isn't "is our data good" - it's "does the value survive the data we'll really get, with nobody in the loop to fix it first."

One of the AI product features I reviewed worked perfectly in every demo. The team had been preparing the inputs by hand before each one - cleaning them up, picking the examples that worked - without quite noticing they were doing it. The capability was real. But the hand-cleaning was part of the product too, and nobody had counted its cost. On a pilot with the customer's own data, untouched, the quality dropped, and the only way to bring it back was a person doing by hand what the demo had quietly assumed away. We didn't kill it. We repriced it: the honest version was the one with a person in the loop, at the margin that comes with paying for that person - a smaller business than the one on the slides.

Real answer

the results from running it on ugly, representative, uncurated input - the data you'll actually get.

Comforting answer

the demo, shown one more time.

4. What does this displace, and will anyone switch?

Question 1 asked whether anyone wants this enough to pay for it. This is the next question: even when they do, will they actually move. A build assumes people will adopt it. But a new product almost never competes with nothing. It competes with a spreadsheet, an existing tool, a habit, or doing nothing at all, and those are entrenched, and free. The cost of switching is real, and it almost never gets into the plan. Most products that fail didn't lose on quality. They lost because switching cost more than the product was worth, and nobody put a number on it.

AI features are especially prone to this, because the capability feels like magic and magic feels like it should sell itself. It doesn't. Magic doesn't pay the switching cost either, and "it's obviously better" is the assumption that quietly kills the most products.

Real answer

what the customer does today, what it costs them to stop doing it, and why the gain is bigger than that cost.

Comforting answer

that the product is clearly better.

None of these are technical

That's the point. The bet almost always dies on a product or commercial question, not an engineering one - which means the people best placed to catch it aren't the ones doing the build, and catching it doesn't take a single line of code.

A working demo is the most confident object in the building. It's built to look like the answer to all four questions at once, and it answers none of them. The discipline is asking anyway, out loud, while the idea is still cheap to be wrong about - and being willing to hear an answer you'd rather the demo had settled for you.