What a prototype can and cannot prove
Match each product assumption to evidence that can answer it. A room-booking example separates usability, demand, feasibility and running the service.
A prototype can provide evidence about the question it was designed to test. Completing a task in it may reveal usable interaction, but does not by itself establish demand, production reliability or whether the service is viable to operate.
A cardboard model can show whether a desk fits in your room. It cannot show whether the desk will carry a heavy computer. Both tests matter, but the first result cannot answer the second question.
A colleague chooses a room, moves a meeting and sees “Booking updated.” The prototype looks convincing. But its calendar is a fixed picture, it never contacts the booking system, and nobody has chosen to use it for a real meeting. Which part of the idea has the demonstration actually tested?
The room-booking service below is an original illustrative example. No interviews, user tests or adoption results are being reported. We will use it to separate a useful observation from a claim the observation cannot support.
One demo, four questions#
Marty Cagan’s four product risks distinguish value, usability, feasibility and business viability. Applied to this service, those become four different questions. A result about one should not silently become an answer to all four.
| Question | Evidence we would seek in this example |
|---|---|
| Value: will colleagues choose it? | Recent booking problems, their current workarounds and voluntary use in a clearly explained trial. |
| Usability: can they use it? | What participants do when asked to move a meeting, including mistakes and help needed. |
| Feasibility: can we build it? | A focused test of the real booking integration, including two requests competing for the same room. |
| Viability: can we run it? | Who owns the service, how it will be supported, and whether its costs and organisational constraints are acceptable. |
A participant completing the mock booking is evidence about that interaction. It does not tell us whether colleagues would abandon their shared calendar. A successful integration test answers a technical question, not whether the problem is worth solving. Keeping these distinctions visible prevents a polished demonstration from deciding questions it never examined.
Try it: what did the observation test?
Choose an illustrative observation from the room-booking service. Notice which question it informs and which questions still need their own evidence.
Question supported: Usability: can someone complete this task in the tested interface?
Still unproven: This does not establish demand, real-system reliability or operating costs.
Possible next test: Next, observe voluntary use in a real trial or test the booking integration.
Choose a question before a test#
Suppose we want to learn whether people can move a meeting without accidentally cancelling its existing room first. That uncertainty determines the prototype: participants need to see an existing booking, available alternatives and the result of changing it. A production notification service would add work without answering this particular question.
Strategyzer’s Test Card asks teams to state an assumption, a test, a measurement and a success criterion before collecting results. The following is our own test plan using that distinction, not a completed study.
A test plan, not a result
Assumption: people who arrange meetings can change a room booking without help and can tell whether the old booking remains active.
Task: “Your meeting is moving from two o’clock to three. Find a suitable room and arrange the change.” Do not tell the participant which control to press.
Observe: the actions taken, any incorrect cancellations, requests for help and the participant’s understanding of the final booking.
Decision rule for this round: an accidental loss of the existing booking is a reason to revise the flow before expanding the trial. Completing the task without that problem permits another test; it does not establish general demand or reliability.
Choose participants whose work makes the task meaningful. Explain the exercise and what will be recorded; use invented meeting details in the prototype rather than collecting unnecessary personal information. These are design choices for this proposed study, not a claim that a consent form alone settles every research obligation.
Write down what happened#
Keep observation and interpretation separate. In a hypothetical session, “The participant pressed Cancel, lost the original booking and asked how to undo it” is an observation. “The confirmation step is unclear” is one explanation to investigate. “People do not want this product” is a much larger claim that this event cannot establish.
Record assistance too. A participant who completes the flow after the moderator explains it has not completed it unaided. Record the version of the prototype and the starting task, so a later round can distinguish a changed design from changed testing conditions.
Positive results need the same restraint. Even if every participant in a small round completes the task, we have not shown that everyone can use the service, that they will return, or that it works with real booking data. The next question may now be more important than polishing this interaction again.
Change the condition#
Change just one condition: two people now try to book the final available room at the same time. Both prototype screens display success because neither consults a shared system. The earlier usability observation can remain valid while this new case exposes an unanswered engineering question.
A technical experiment should now exercise the real boundary that prevents conflicting bookings and inspect both clients’ outcomes. A test with every dependency replaced by a convenient fake cannot establish how that real integration behaves. The appropriate next step follows the risk, not a fixed rule that every project must start with screens.
The GOV.UK Service Manual explicitly separates prototype code from production standards and warns against simply copying it into a live service. Realistic appearance helps explore interaction; it is not evidence of security, performance or operational readiness.
Turn evidence into a decision#
End a round with a bounded decision: change the cancellation flow, investigate the booking integration, try voluntary use with a clearly defined group, or stop because the current workaround already serves the need. State what changed your view and what is still unknown. A request for another experiment is useful only when its result could change what you do.
Then turn accepted behaviour into requirements that are clear enough to implement. Keep the evidence and uncertainty beside them. “The ticket is ready” means the work is sufficiently specified; it does not retrospectively validate the product idea.
Use a design document when the proposed technical approach still needs debate. It can refer to the tests and their limits rather than presenting a successful prototype as a settled argument for the whole system.
An agent-built prototype is a reference, not a merge plan
In Software Factory Design Patterns, Dex Horthy describes exploring a large feature with agents, then separating the prototype into changes that can be reviewed incrementally. The useful distinction is between discovering the intended behaviour and deciding how to introduce it safely. His example is a personal workflow, not evidence for a universal pull-request size or a fixed number of agent review rounds.
Keep an exploratory branch separate while recording the accepted behaviour, fake dependencies and unresolved engineering questions. For the booking example, a possible sequence is to establish the conflict-handling contract and tests, add the compatible backend behaviour, then connect the room-change interface. Check each intermediate state: splitting by file or frontend/backend ownership is not enough if one change requires an unavailable version of another.
Ask the agent to propose those reviewable steps from the current code and the accepted behaviour, not merely to divide a large diff into equal pieces. Review and test the resulting changes on their actual revisions. An approval of the prototype does not approve a later reconstruction, and repeated model reviews do not replace the acceptance decision. The prototype can be discarded while the decisions and tests it helped discover remain useful.
IF YOU REMEMBER ONE THING
Before saying “the prototype worked,” finish the sentence: it gave us evidence about which question, under which conditions, and what decision does that justify?
Questions people also ask
4 QUESTIONSDoes a successful usability test prove people want the product?
No. It shows what the participants could do under the tested conditions. Choosing to adopt a service when their own work is at stake is a different observation. Keep the two claims separate.
How many people make a prototype validated?
There is no participant count that validates every kind of claim. Small formative sessions can uncover usability problems. Population estimates, business demand and reliability require appropriate evidence and a study design for those questions.
Can prototype code become production code?
Sometimes parts can, after engineering review. A prototype may deliberately omit security, concurrency, failure handling and operational behaviour. Looking real is not evidence that those properties exist.
Must every change start with another prototype?
No. Start with the uncertainty that could change the decision. For a familiar interface and an uncertain database constraint, an engineering experiment may be more useful than another screen mock-up.