What changes when an AI prototype goes into production

An AI prototype is ready for production when its useful behavior survives ordinary failure, uncertain inputs, and routine changes under a named operating owner. A convincing demonstration establishes that something can work. Production readiness establishes the conditions under which users can depend on it and what happens outside those conditions.

Treat the transition as a review of evidence, not a final round of prompt polishing.

Replace hidden assumptions with a feature contract

List supported inputs, expected outputs, permission boundaries, and actions the feature may take. Identify which provider receives which data, how retention is configured, and who approved that data flow. Do not assume a development account’s settings match production.

Prototype shortcuts often hide in scripts: broad service credentials, local files, unlimited context, and manual cleanup. Each shortcut needs a production replacement or a documented reason to remove the affected capability.

Google’s SRE discussion of production readiness reviews treats operational readiness as work shared between development and the team operating the service. For an AI feature, include model and retrieval behavior in that review alongside the application infrastructure.

Freeze a baseline that can be compared

Version the prompt, model selection, retrieval configuration, and evaluation cases. Record results for supported languages and failure categories. A provider change or a larger context window should trigger evaluation rather than inherit approval from an earlier configuration.

Use release gates that reflect the consequences of error. A good average cannot excuse a permissions failure. Keep a holdout set separate from everyday tuning, and retain enough configuration history to reproduce an investigated result without storing unnecessary sensitive content.

A hypothetical document assistant under load

Imagine a prototype that summarizes one internal document at a time. In a hypothetical pilot, several users submit large files simultaneously. Requests time out, and automatic retries start more work while the original calls are still running.

The production design limits file size and concurrent work, reserves usage before dispatch, and gives accepted jobs a durable identity. It exposes a pending state rather than encouraging repeated submission. If the provider is unavailable, users can still access the original document and complete the task manually.

The test must interrupt the provider after acceptance and restart the worker. Success means the job reaches a known state without unauthorized content or duplicate side effects. It does not mean every request always gets an AI answer.

Put controls outside the prompt

Check these before expanding access:

• Server-side authorization limits retrieval and tool execution.
• Generated output is validated before entering trusted application fields.
• Timeouts, retries, context length, and tool iterations have explicit bounds.
• Operators can disable the AI path without disabling unrelated work.
• The interface distinguishes pending, failed, and unavailable outcomes.

A prompt asking the model to respect these rules is not their implementation. Test the enforcement component independently from the model’s wording.

Hand over an operable feature

Name an owner for incidents, usage limits, and configuration changes. Provide a rollback procedure, an exception queue, and a way to trace a user report to an operation. Rehearse the fallback with someone who did not build the prototype.

Before scheduling launch, run a short failure exercise covering denied access, provider outage, and a configuration rollback. Use the unresolved items as the remaining scope for custom software development. Record who accepts any remaining limitation and which users it affects before opening access.