AI governance: GDPR, the EU AI Act and the trust layer in production
AI compliance is not decreed in a committee: it is proven in production, with logs, evals and audit trails. This guide connects the EU AI Act, the GDPR and the technical trust layer into one operational method — no jargon, no hollow promises.
In brief. Governing your AI means joining up three planes: the EU AI Act (Regulation (EU) 2024/1689), which classifies systems by risk level; the GDPR, which governs personal data; and the technical trust layer — also known as AI observability, model governance and ML monitoring: logs, guardrails, continuous evals, cost and access control, which produce the evidence in production. According to the timeline published by the Commission, the regulation entered into force on 1 August 2024 and applies in stages: prohibitions since February 2025, GPAI obligations from 2 August 2025, the bulk of high-risk obligations from 2 August 2026, with some deadlines in 2027. The window to prepare is now.
The framework
What is AI governance, concretely?
AI governance is the set of rules, roles and technical mechanisms that let you demonstrate — at any moment — that an AI system does what it should, remains reliable and complies with the law. It is not a PowerPoint document signed off once and for all: it is a living layer that runs in production. Three planes overlap and must be joined up.
The regulatory plane dictates what you must prove: the EU AI Act (Regulation (EU) 2024/1689) classifies systems by risk level and sets graduated obligations, while the GDPR governs the personal data that feeds into or comes out of those systems. The organisational plane defines who decides and who is accountable: AI lead, DPO, business teams, security. The technical plane — the trust layer — produces the evidence: logs, traceability, guardrails, continuous evaluations, cost and access control.
The classic mistake is to treat these planes in silos: a lawyer drafts a policy, a data team deploys a model, and nobody checks that the policy is actually enforced by the live system. Useful governance is precisely the bridge between the three. The starting point — often missing — is a simple inventory: based on our audit engagements, in the majority of cases the organisation does not have, at the outset, a complete and up-to-date inventory of its AI systems, even though it is the very first item an auditor asks for. The sector-wide picture confirms this immaturity: according to the IAPP, AI Governance Profession Report, a majority of organisations have only structured their AI governance function very recently, or plan to do so in the short term.
EU AI Act: which obligations at which risk level?
The EU AI Act grades your obligations according to the AI system’s risk level, not the technology it uses. So the first question in an audit is never « which model are you using? » but « what is this system for, and who can it affect? ». Four categories structure the regulation.
How do you classify an AI system under the EU AI Act?
You classify a system by assigning it to one of the four risk levels, from the most constrained to the most permissive. The grid below sums up what each level covers and what it means in practice.
Unacceptable risk
Prohibited practices: social scoring, behavioural manipulation, real-time remote biometric identification in public spaces (save for strict exceptions), biometric categorisation to infer sensitive attributes, and mass scraping of facial images. No route to compliance exists — the use case itself is banned.
High risk
Annex III (HR, credit, education, justice, critical infrastructure…) and systems embedded in regulated products (Annex I). Heavy requirements: risk management, data quality, technical documentation, logging, human oversight, robustness.
Limited risk
Chatbots, content generation. The main obligation is transparency: users must know they are interacting with an AI, and generated content (text, images, deepfakes) must be flagged as such.
Minimal risk
The vast majority of internal uses (filters, benign recommendations). No specific obligation, though good governance practice remains recommended.
Case in point — the CV pre-screening assistant. A mid-sized industrial company deploys an LLM assistant to pre-screen applications and propose a shortlist to recruiters. The team sees it as « just a chatbot » (limited risk). Wrong: a system that takes part in recruitment selection falls under Annex III — high risk. It triggers technical documentation, logging, effective human oversight, and a DPIA on the GDPR side. Based on our engagements, this kind of reclassification is common: what the business labels « limited » often turns out to be « high risk ».
GPAI models and responsibilities: provider or deployer?
General-purpose AI (GPAI) models — the large LLMs such as Claude, Gemini, ChatGPT or Mistral — fall under their own documentation and transparency regime. Above a certain amount of training compute, a model is presumed to pose a systemic risk and inherits reinforced obligations: evaluations, risk mitigation, reporting of serious incidents, cybersecurity. That presumption rests on a training-compute threshold of around 1025 FLOP.
For an executive, the decisive point is the provider / deployer distinction. The provider designs the system or places it on the market; the deployer uses it under its own authority. When you integrate a third-party LLM into your product or your processes, you are most often the deployer — and you carry obligations of your own (informing individuals, human oversight, use in line with the documentation). If you substantially modify a GPAI model, you may flip to provider status: the dividing line hinges notably on the compute mobilised by the modification, assessed against the original model’s training compute (around one third, according to the guidelines). Marketing the system under your own brand has the same effect.
What is the EU AI Act’s application timeline?
According to the timeline published by the Commission, the regulation entered into force on 1 August 2024 and applies in stages. The prohibitions targeting unacceptable-risk practices have applied since February 2025. Obligations for GPAI models apply from 2 August 2025. The bulk of the obligations for high-risk systems applies from 2 August 2026, and some deadlines fall in 2027, notably for systems embedded in already-regulated products. Changes to the framework are under discussion at European level: to be confirmed legally as and when they are adopted and published in the OJEU.
Freshness note. The regulatory timeline and thresholds in this guide are stated on the basis of the published texts and the Commission’s timeline. Up to date as of 18 June 2026. Upcoming milestones remain subject to legal confirmation as official publications appear.
The operational translation: the deadline arrives faster than you think. Building an inventory of your systems, classifying them and standing up an auditable trust layer takes months, not weeks. Companies that wait for the final deadline will discover that a high-risk system cannot be brought into compliance after the fact without rebuilding everything.
How do the GDPR and the EU AI Act fit together?
The GDPR and the EU AI Act do not replace one another: they stack. The AI Act regulates the system; the GDPR regulates the personal data flowing through it. As soon as an AI system processes personal data — training, prompts containing customer information, re-identifiable outputs — both regimes apply simultaneously.
Concretely, a high-risk HR system will have to satisfy the AI Act (documentation, human oversight, logging) and the GDPR: legal basis for the processing, data minimisation, informing individuals, rights of access and objection, and the framework around automated decision-making (Article 22, see the CNIL). Many obligations usefully overlap: a Data Protection Impact Assessment (DPIA), required by Article 35 of the GDPR, feeds directly into the risk management demanded by the AI Act. The reverse trap also exists: a data transfer to an LLM hosted outside the EU can be compliant with the AI Act yet breach the GDPR. Governance is about closing those blind spots.
Practical rule: before any deployment, ask two questions in parallel. « What is this system’s EU AI Act risk level, and am I a provider or a deployer? » and « Which personal data flows through it, where, and under which legal basis? ». If you cannot answer either one clearly, the system is not ready for production.
What is an AI trust layer and what must it contain, at a minimum?
The trust layer is the technical layer that turns a compliance intention into verifiable evidence. It is our in-house term for what the ecosystem calls AI observability, model governance, ML monitoring and AI auditing. Without it, a governance policy remains a statement of intent. Here are the four minimum components of a production AI you can defend in front of an auditor, a customer or a regulator.
Logs & audit trails
Timestamped logging of inputs, outputs, model versions and decisions, retained immutably. This is the foundation of every traceability requirement in the EU AI Act.
Guardrails
Guardrails on inputs and outputs: filtering of sensitive data, blocking of out-of-scope answers, format validation, human oversight on critical decisions.
Continuous evals
Evaluation sets replayed automatically at every change to the model or the prompt, with quality thresholds and non-regression tests that block deployment on failure.
Costs & access
Tracking of inference cost per use case, budget alert thresholds, and granular access control (who can query which model with which data).
How do you instrument human oversight (human-in-the-loop)?
Human oversight takes shape through measurable mechanisms, not through a line in a policy. Concretely: confidence thresholds that automatically route any output below a given score to a human; review queues where an operator validates, corrects or rejects critical cases before they take effect; an override rate (the share of AI decisions modified by a human) tracked over time as a reliability indicator; and a kill switch to suspend the system. If the override rate sits near 0%, either the AI is excellent or the review is fictitious: that is exactly the nuance an auditor sets out to verify. It is also what separates an industrialisation firm from a consultancy: oversight is instrumented, not merely written down.
This layer is not a standalone project: it plugs into your orchestration architecture. It also extends the work of data mapping for AI, which identifies what flows where and under which legal basis. And it is exactly the scope of our AI Run, Ops & governance offer, which operates this trust layer over the long term.
LLM evaluation, drift and non-regression: how do you stay in control?
You stay in control by measuring quality continuously, never by trusting a single test at launch. An AI that is compliant on day one can become non-compliant in silence: providers update their models, your data evolves, usage drifts. Without continuous measurement, you will only find out at the moment of the incident — often flagged by an unhappy customer or a journalist.
What are LLM evals for?
An LLM eval is a reproducible test suite that automatically measures the quality of a model’s answers against representative cases, using objective criteria (accuracy, format, absence of hallucination, respect for the guardrails). You replay these evals at every change of model or prompt, and you block the release if the score falls below the defined threshold: it is a non-regression test applied to AI. Without evals, « it works better » is just an opinion; with them, it is a metric you can defend in front of an auditor.
How do you detect model drift in production?
You detect drift by instrumenting the distributions of inputs (are the questions changing?), outputs (are the answers drifting?) and quality (is the eval failure rate rising?). A significant shift triggers an alert and an investigation before users are affected. Drift detection is what separates a supervised AI from one that has merely been deployed.
Who does what: AI lead, DPO and business teams?
Governance almost always fails through lack of clear accountability, not through lack of tools. Three roles must be assigned by name, each with a written mandate.
AI lead
Keeps the inventory of systems, their EU AI Act classification, provider/deployer status, documentation and eval tracking. The single point of contact for any AI compliance question.
DPO
Owns everything personal data: legal bases, DPIAs, individuals’ rights, transfers. Works in tandem with the AI lead on high-risk systems.
Business & security
The business validates relevance and human oversight; security handles access, secrets and exposure. Governance is a team sport, not the affair of a lone lawyer.
What non-compliance really costs
Worth noting: not every breach is capped at €35M / 7%. That level targets prohibited practices; breaches of high-risk obligations and of GPAI model provider obligations fall under a distinct tier of €15M or 3% of worldwide turnover. Assuming the single top cap applies everywhere is a common mistake.
How do you audit an AI system? The compliance checklist
To audit an AI system, you work through an actionable checklist, point by point, before and during the move to production. If a box stays unticked, you have found your next priority — and exactly what an inspection will surface first.
- Inventory of AI systems, EU AI Act risk-level classification and provider/deployer status for each
- Mapping of the personal data processed, legal basis and DPIA (Article 35 GDPR) where required
- Up-to-date technical and transparency documentation (purpose, limitations, data, oversight)
- Immutable logs and audit trails covering inputs, outputs and model versions
- Input/output guardrails and instrumented human oversight (confidence thresholds, review queues, override rate) on critical decisions
- Eval suites with thresholds and non-regression tests replayed at every update
- Instrumented drift detection, with alerting and an investigation procedure
- Inference cost tracking and granular access control to the models
- Named roles: AI lead, DPO, business sign-off, security
- Users informed that an AI is in use, and generated content flagged as such
This checklist sits within a broader industrialisation effort: see Taking an AI POC to production: the industrialisation checklist. To take stock of where you stand, a short, fixed-price AI assessment & scoping delivers an objective baseline in a matter of days.
The pitfalls to avoid in AI governance
Here are the traps we run into most often in the field — the ones that turn box-ticking compliance into real risk.
- Governance-as-a-document. A beautiful policy that nothing in the live system enforces or measures. If you cannot pull a log, you are not compliant.
- The « it’s just a chatbot ». Underestimating classification: an assistant that pre-screens job applications lands in high risk (Annex III), not limited risk.
- Believing you are « only » a deployer. Integrating a third-party LLM does not exempt you: the deployer carries its own obligations, and a substantial modification of a GPAI model can flip you to provider status.
- Forgetting transfers outside the EU. Sending prompts containing customer data to an LLM hosted outside the EU: compliant with the AI Act, yet a GDPR breach if transfer safeguards are missing.
- The « set and forget ». Deploying without continuous evals or drift detection, then discovering the drift months later via a complaint.
- Diluted accountability. Nobody is named AI lead: come an inspection, everyone passes the buck and the documentation is nowhere to be found.
Governance must also reach down to the data feeding your systems: our guide on data mapping for AI details how to trace the flows before you even talk about models.
Frequently asked questions
Is my company within the scope of the EU AI Act?
As soon as you design, place on the market or operate an AI system touching the EU — including by plugging in a third-party LLM — you fall within the regulation’s scope. What varies is the intensity of the obligations: it depends on the use case’s risk level and on your role (provider or deployer). First concrete action: classify each use into one of the four levels (unacceptable, high, limited, minimal).
What is the EU AI Act’s application timeline?
According to the timeline published by the Commission, the regulation entered into force on 1 August 2024 and applies in stages: the prohibitions on unacceptable-risk practices since February 2025, GPAI obligations from 2 August 2025, the bulk of high-risk obligations from 2 August 2026, and some deadlines in 2027. Changes to the framework are under discussion at European level and remain to be confirmed after adoption and publication in the OJEU.
How do you join up the GDPR and the AI Act without doing everything twice?
There is no need to run two watertight parallel programmes: the regimes stack, but their requirements overlap substantially. A solid DPIA (Article 35 GDPR) feeds directly into the AI Act’s risk management. Best practice: an AI lead / DPO pairing that steers both from the design phase on any high-risk system.
What is a trust layer, and is it mandatory?
It is the technical tooling that materialises the evidence: immutable logging, audit trails, guardrails, continuous evals, control of costs and access. It also goes by AI observability, model governance or ML monitoring. The term « trust layer » does not appear in the law, but each of its components answers, point by point, the AI Act’s traceability, robustness and oversight requirements. In practice, without this layer you cannot demonstrate your compliance.
What penalties does the EU AI Act provide for?
The regulation sets several tiers: the highest, up to €35M or 7% of worldwide turnover, hits prohibited practices; an intermediate tier of €15M or 3% covers high-risk breaches and GPAI provider obligations; a lower tier penalises inaccurate information supplied to the authorities. On top of that, and independently, sits the GDPR cap of €20M or 4%.
How do you audit an AI system already in production?
Start with the inventory and the EU AI Act classification of each system, then walk up the chain of evidence: are the logs immutable and usable? Do the guardrails actually filter? Are there evals with thresholds, drift detection, instrumented human oversight (a tracked override rate) and named roles? The checklist in this article provides a ready-to-use audit framework.
Move from experimentation to AI in production
Start with a short, fixed-price diagnostic: maturity, high-ROI use cases, and a prioritised roadmap. No commitment.