When Is a Gen AI Proof of Concept Ready for Production? Five Gates to Check

The short answer#

A Gen AI proof of concept is ready for production when it has passed five gates, not when the demo impresses the room. The questions are agreed on a real evaluation set, the answers hold up against real data and permissions, the failure modes are known and contained, someone can run and pay for it, and a named owner will be accountable for it after launch.

Why most proofs of concept stall after the demo#

A proof of concept is built to answer one question: can this work at all? It usually answers it well. The trouble starts when the same code is asked a different question: can we rely on this every day?

The two questions need different evidence:

• The demo ran on a clean sample of documents. Production has years of duplicates, scanned PDFs and documents nobody is allowed to see.
• The demo was judged by the people who built it. Production is judged by users who ask questions nobody planned for.
• The demo had no budget line. Production has a monthly bill that grows with every user.
• The demo had no owner after the presentation. Production needs one on day one.

None of these are model problems. They are architecture and delivery problems, and they can be checked before the budget for a full rollout is committed.

The five gates#

1. Evaluated: "good" is a number, agreed with the business#

Before anyone argues about whether the assistant is good enough, write down what good means.

• Collect real questions from the people who will use it, including the awkward ones.
• Agree the expected answer, or the expected behaviour, for each one with a business owner.
• Run the system against that set every time the prompt, the model or the data changes.

If there is no evaluation set, every later conversation about readiness becomes a matter of opinion. With one, it becomes a score you can track and a threshold you can sign off.

2. Grounded: it answers from your data, with your permissions#

Most enterprise use cases depend on retrieval: finding the right passage in your own documents before the model writes an answer. In a proof of concept, retrieval is often the weakest part, and it is hidden by a good-looking demo.

Check that:

• Answers cite their source, and the source actually says what the answer claims.
• A user can only retrieve what they are already allowed to read in the source system.
• The system says "I don't know" when the documents don't contain the answer, instead of inventing one.
• Fresh documents appear within an agreed time, and withdrawn ones disappear.

3. Contained: failure modes are known and limited#

Every Gen AI system fails sometimes. The question is whether it fails safely.

• Prompt injection: text inside a document or an email can try to give the model new instructions. Test for it deliberately.
• Data leakage: personal or confidential data must not reach a model, a log or a user it shouldn't.
• Actions: if the system can call tools or change records, limit what it can do, and put a human approval step in front of anything that cannot be undone.
• Fallbacks: decide what the user sees when the model is slow, unavailable or unsure.

4. Operable: someone can run it and pay for it#

Operations teams will not take over a system they cannot see into.

• Trace every request: the question, what was retrieved, which model answered and what it cost.
• Set a cost per request and a monthly budget, and alert before either is exceeded.
• Agree response-time targets and test them under realistic load.
• Make the model a configuration choice, so a change of provider or version is a tested release, not a rewrite.

5. Owned: a named person is accountable after launch#

The last gate is organisational. Somebody must own the system's behaviour once the project team moves on: reviewing the evaluation scores, deciding when to retrain or re-index, handling user complaints and approving changes. Without an owner, quality drifts quietly until users stop trusting it.

How to use the gates#

Treat the five gates as the exit criteria of the proof of concept itself, not as a checklist for later.

1. Agree the gates at the start. Write the evaluation set and the containment rules into the scope of the proof of concept.
2. Build the thinnest slice that can pass all five. One use case, one data source, one user group, but production-grade in every gate.
3. Report against the gates, not the demo. The final review shows the evaluation score, the security tests, the cost per request and the named owner.
4. Decide. Scale it, fix one gate and re-test, or stop. Stopping early is a good outcome: it is the cheapest point to learn that a use case isn't worth funding.

A proof of concept that passes all five gates is no longer a prototype. It is the first release of a production system, and the next step is a rollout plan, not a rebuild.

Questions to ask before you approve the next phase#

• What is the evaluation score today, and what score did we agree would mean "ready"?
• Can a user retrieve anything they cannot already read in the source system?
• What happens when someone hides an instruction inside a document?
• What will this cost per month at the number of users we expect?
• Who owns the system's behaviour after launch?

If the answers are vague, the proof of concept has shown that the idea can work. It has not yet shown that it is ready.