Taking an AI POC to production: the industrialisation checklist
The demo impresses, production breaks. Here is the complete framework — the demo/production gap, the industrialisation checklist, MLOps/LLMOps, RACI, KPIs and hard-won lessons — to make an AI agent or automation stand up to the real world.
The reality
Why most AI POCs never reach production
A POC (proof of concept) exists to answer a single question: « is the value there? ». It runs on a developer’s laptop, on clean, hand-picked data, with no load, no security constraints, and it is allowed to fail one time in ten without anyone noticing. Production answers a radically different question: « does this system hold up on its own, 24/7, in the face of the real world, without waking us up at night? ». Between the two, it is not a change of scale — it is a change of nature.
That is why so many initiatives stall at the demo stage: the value has been proven, but the reliability was never built. Industrialising an AI project means rebuilding the POC so that it holds up, monitors itself and repairs itself — not rerunning it at a bigger scale. This guide gives you the complete checklist for getting over that wall, the roles to clarify, the metrics to track and the pitfalls that derail teams. It sits at the heart of our Industrialisation & automation offer.
MLOps / LLMOps, in one sentence
MLOps is the set of practices that make an AI system deployable, reproducible and operable on an ongoing basis. LLMOps is its counterpart for language models and agents: prompt versioning, guardrails, evaluations (evals), token cost tracking and trace observability. Same discipline, specific constraints.
How big is the real gap between a demo and production?
The gap is not theoretical: it is made of concrete situations that never show up in a demo and that all occur, without exception, the moment you go live. Knowing them means knowing where your POC will break before it does.
Which problems only appear in production?
- Volumes. Ten documents in the demo, ten thousand a day in production: queues, timeouts and API rate limits that simply did not exist before.
- Malformed data. In a demo, the inputs are clean. In production come skewed scanned PDFs, exotic encodings, empty fields and unexpected formats that crash the parsing.
- API errors. An LLM returns a 429, a 500 or truncated JSON. In a demo you retry by hand; in production, without error recovery, the entire flow grinds to a halt.
- Load spikes. Monday morning, month-end close, the marketing campaign: load is never constant and the system has to absorb the peaks.
- Secrets. An API key hard-coded in a notebook is fine for a demo. In production, it is a vulnerability — and one you cannot rotate.
- Drift. A prompt that used to work drifts when the model is updated, or when the input data evolves. Without evals, you never see it coming.
The method
How to take an AI POC to production, step by step
This six-step sequence is the backbone we follow on every production rollout. Each step is a prerequisite for the next: skipping security or observability does not remove the problem — it merely postpones it.
Secure secrets & access
Get every key out of the code, centralise them in a secrets manager, enforce least privilege and plan for rotation. Isolate the dev / staging / production environments.
Make the flow robust
Idempotency (replay a run without creating duplicates), error recovery, retries with backoff, queues and a dead-letter queue for whatever still fails.
Test & evaluate
Unit and integration tests on the code, plus a set of evals on the LLM outputs (reference datasets, scoring) to catch regressions before they reach your users.
Automate deployment
A reproducible CI/CD pipeline, with versioning of code, prompts and configurations. No more manual changes made directly in production.
Observe & control costs
Structured logs, traces, metrics, alerting. Token and cost-per-request tracking, budget guardrails, caching and the right model for each task.
Document the runbook
Operating procedures: what to do in an incident, who to alert, how to replay, how to roll back. Without a runbook, the knowledge lives in a single person’s head.
This checklist extends naturally into two related topics: the orchestration architecture you put underneath these flows, detailed in our guide n8n, MCP and LLM agents: the governed automation architecture, and the compliance that production demands, covered in AI governance: GDPR, the AI Act and the trust layer in production.
The numbers that sum up what’s at stake
Orders of magnitude observed in the field; they vary with the criticality and data maturity of the organisation.
Who does what: roles, RACI and environments
A POC is one person’s business. Production is a chain of responsibilities. If no one is explicitly accountable for availability, the LLM bill or output quality, those topics fall through the cracks — and that is exactly where incidents are born.
Which roles need to be clarified before going live?
Product / business (Accountable)
Owns the expected value and the acceptance criteria. Arbitrates the quality / cost / time trade-offs and signs off the acceptable eval thresholds.
AI engineering (Responsible)
Builds the robust flow, the versioned prompts, the tests and the evals. Responsible for error recovery and idempotency.
Platform / DevOps (Responsible)
CI/CD, isolated environments, secrets management, infrastructure (AWS, Docker), scalability and backups.
Run / SRE (Responsible)
Observability, alerting, on-call, runbook. Owns the production KPIs and drives incident resolution.
Security / compliance (Consulted)
Signs off access, traceability, GDPR and AI Act compliance. Consulted at the architecture stage — not afterwards.
Leadership / sponsor (Informed)
Kept informed of KPIs, costs and risks. Arbitrates budgets and the prioritisation of workstreams.
On environments, the rule is simple and non-negotiable: three isolated spaces — dev, staging, production — with separate secrets and no production data in dev. Staging should resemble production as closely as possible, because that is where evals and deployments are validated before anything is exposed to users.
Which KPIs should you track once in production?
A POC is judged on intuition; a production system is steered with metrics. Four families of indicators are enough to know, at any moment, whether the system is healthy — and to trigger action before your users start complaining.
- Availability (uptime). Is the system responding? An explicit target (e.g. 99.9%) and alerting whenever you drift away from it.
- Latency (p50 / p95 / p99). The median is not enough: it is the slow p95 requests that degrade the experience.
- Error rate. Technical errors (5xx, timeouts) and business errors (LLM outputs rejected by the guardrails).
- Cost per request. Tokens and euros per run, per workflow and per day — to catch any drift in the bill.
- Eval score. Output quality over time, to spot drift in a model or a prompt.
The classic pitfalls to avoid
These traps come up on almost every project we take over. None of them is fatal on its own, but combined they turn a production rollout into a never-ending project.
- « We’ll industrialise later. » Reliability debt is paid at a premium: rebuilding a fragile flow costs more than making it robust from the start.
- No evals. Deploying a prompt change without a reference dataset means gambling on quality at every model update.
- Observability bolted on afterwards. Without logs and traces from day one, your first production incident is a black hole.
- Uninstrumented LLM costs. The bill explodes in silence until the end of the month. Per-request tracking must exist before go-live.
- The POC = production. Pushing the prototype as-is, with no error recovery or idempotency, guarantees your first blocking incident.
- A single human who knows. Without a runbook and a RACI, the slightest absence becomes an operational risk.
Should you build on the POC or rebuild from scratch?
The answer depends on what has been validated. Build on the POC if the business logic and the prompts deliver, if the architecture is sound and only robustness and operability are missing: you keep the core of the value and add the production layer. Rebuild if the POC rests on assumptions that don’t hold at scale (a prototyping tool that can’t be deployed, hard-coded data, a drifting model), or if the technical debt exceeds the cost of a clean rewrite. In practice, the right answer is often hybrid: keep the prompts and the logic, rebuild the orchestration and the infrastructure. That is precisely the trade-off an AI assessment & scoping settles.
Frequently asked questions
Why do AI POCs fail so often?
Because they prove the value without building the reliability. A POC runs on clean data, with no load, no security constraints and no error recovery. The moment it hits production — volumes, malformed data, API errors, spikes, secrets — it breaks. Industrialising means rebuilding so that the system holds up, monitors itself and repairs itself — not rerunning the POC at a bigger scale.
How do you take an AI POC to production in practice?
By following a sequence: secure the secrets and isolate the environments, make the flow robust (idempotency, error recovery, queues), test and evaluate the outputs, automate deployment with CI/CD, put observability and cost tracking in place, then document an operational runbook.
What are MLOps and LLMOps in production?
MLOps brings together the practices that make an AI system deployable, reproducible and operable on an ongoing basis. LLMOps is its counterpart for language models and agents: prompt versioning, guardrails, evals, token cost tracking and trace observability. The same engineering discipline, with LLM-specific constraints.
How do you control the costs of an LLM agent in production?
By instrumenting every call: token and cost tracking per request and per workflow, budget guardrails with alerting, caching of recurring responses and choosing the right model for each task. The tracking must exist before go-live — otherwise the bill drifts in silence until month-end.
Which KPIs should you track for an AI system in production?
Four families: availability (uptime, with an explicit target), latency (p50/p95/p99, not just the median), error rate (technical and business), and cost per request. Add an eval score to track output quality over time and detect model or prompt drift.
Should you build on the existing POC or rebuild everything?
Build on the POC if the business logic, the prompts and the architecture are sound and only robustness and operability are missing. Rebuild if the prototype rests on assumptions that can’t be deployed or on debt that is too heavy. In practice the approach is often hybrid: keep the core of the value, rebuild the orchestration and the infrastructure.
Move from experimentation to AI in production
Start with a short, fixed-price diagnostic: maturity, high-ROI use cases, and a prioritised roadmap. No commitment.