From AI Pilot to Production: The 18-Point Readiness Checklist

The short answer

A pilot becomes production when six gates are closed: ownership, evaluation, data and permissions, failure handling, cost, and adoption. Eighteen concrete checks sit behind those gates. If more than two are open, you have a demo, not a system — and you should close them before scaling usage.

---

Gate 1 — Ownership

1. A named business owner who is measured on the outcome, not on the launch.
2. A named technical maintainer inside your organization who can change the prompts, retrieval and integrations without calling the vendor.
3. A documented decision log — why this model, this architecture, this scope. Whoever inherits it needs the reasoning, not just the code.

Gate 2 — Evaluation

4. A test set of real cases, including the awkward ones, with expected outcomes. Twenty well-chosen examples beat a vague sense that "it usually works."
5. A quality bar in numbers agreed before launch, matched to the stakes of the task.
6. A regression check you can re-run whenever a prompt, model version or data source changes — because all three will change.
7. A human review step wherever an error is expensive, with an explicit rule for what gets reviewed.

Gate 3 — Data and permissions

8. Per-user permissions respected. If the assistant can retrieve documents its user could not open, you have built a data leak with a friendly interface.
9. A legal basis for the data in use, confirmed with whoever owns privacy in your organization.
10. Retention and logging decided: what is stored, for how long, who can read it.
11. A source of truth for retrieval with a documented refresh mechanism, so answers do not silently go stale.

Gate 4 — Failure handling

12. Defined behaviour when the model is uncertain — escalate, ask, or refuse. Silence and confident guessing are both unacceptable.
13. Graceful degradation when an API is down or rate-limited, so the underlying workflow still functions.
14. An audit trail of what was requested, retrieved and returned — indispensable the first time someone disputes an output.

Gate 5 — Cost

15. Measured cost per task, and a projection at ten times current volume.
16. An alert or cap on runaway usage, ideally per user or per workflow.

Gate 6 — Adoption

17. Enablement for the people whose work changes, in their working language, using their real cases.
18. A usage metric reviewed weekly for the first two months, with a route for feedback that someone actually reads.

How to use the checklist

Run it as a go/no-go review with the business owner and the technical maintainer in the same room. Score each item open or closed — no partial credit. Then take one of three decisions:

• All closed: roll out to the next group of users.
• One or two open: ship to a limited group with a named date to close them.
• Three or more open: keep it in pilot. Scaling an unowned, unevaluated system is how AI programs earn their bad reputation internally.

The two items teams skip most

Permissions (item 8) and enablement (item 17). Both feel like someone else's job, and both are the ones that end programs — one through a security review, the other through quiet non-use.

Getting through the gates

Moving a pilot into production is what our build and scale phases exist for: closing these gates with your team, on your stack, so your people own the result. Talk it through with us at info@braightwave.com.