Sovereign AI: what it means for an EU company

If you sell to a bank, a hospital, a government agency, or any company that handles personal data at scale, you have probably been asked in the last twelve months where your AI runs and where your data ends up.

That question is not a courtesy. It is a procurement filter. Three EU regulations now make the answer a binary check: either you can show the architecture, or you are excluded from the deal.

This post is the definition, the regulatory drivers, and the business case for one term that keeps coming up in those conversations: Sovereign AI.

Definition

Sovereign AI is AI whose infrastructure, training data lineage, and inference operations are governed by the legal and physical jurisdiction in which the deploying organisation operates. For a Polish or German company, that means the model weights, the serving stack, the logs, and the prompts stay inside hardware the company (or a contracted provider under contractual control) physically controls in the EU.

The term is not a marketing label. It is the operational answer to four engineering questions:

1. Where do the weights run? On hardware the operator owns or rents under EU jurisdiction, not on infrastructure owned by a US hyperscaler.
2. Where does inference data live? In a storage layer governed by GDPR and, where applicable, professional secrecy (advocate, physician, tax advisor).
3. Who can read the logs? The deploying organisation and, on lawful request, an EU regulator under EU procedural law. Not a third-country subpoena under CLOUD Act.
4. What is the audit trail? Every prompt, response, tool call, and human override is captured in a schema-stable format an auditor can request and replay.

A system that sends prompts to a US-hosted API and returns the response does not satisfy any of those four points. It is not Sovereign AI, regardless of what the vendor's sales deck says.

Why now: three regulatory drivers

EU law changed in 2024-2025, and the deadline for compliance is no longer abstract.

AI Act (Regulation 2024/1689) classifies most business-critical AI systems used in regulated industries as high-risk. High-risk systems must support Article 12 logging (automatic event recording over the system lifecycle), Article 13 transparency (instructions for use, information to deployers), and Article 14 human oversight (effective oversight by natural persons during use). Article 26 requires deployers to assign qualified personnel to operate the system. None of those obligations are satisfied by a vendor who gives you a chat API and a usage dashboard.

NIS2 (Directive 2022/2555) extended cybersecurity obligations to a much wider set of operators: financial services, health, digital infrastructure, public administration, and parts of the manufacturing and food sectors. Supply chain security is explicit. If your AI vendor experiences an incident, your regulator wants to know within 24 hours under Article 23. Article 21 requires that you actually enforce this on your supply chain, which means documented contractual and technical controls on AI providers, not a generic SOC2 attestation.

DORA (Regulation 2022/2554) is the financial-sector specific regulation, live since 17 January 2025. Article 28 requires that financial entities assess and document ICT third-party risk. A "the model is SOC2" check-the-box answer does not satisfy the regulation. DORA expects documented evidence of concentration risk, substitutability analysis, exit strategies, and contractual audit rights over the entire AI pipeline, including the inference layer.

For a regulated SME, the practical effect of these three together is this: the cloud LLM API category that worked for everything between 2020 and 2024 no longer works for credit scoring, claims automation, clinical documentation, contract analysis, customer onboarding, or any process touching regulated data.

What Sovereign AI looks like in practice

For a 50-person SaaS company serving banks or insurers, the reference architecture has four layers, in this order:

1. Open-weights model served on hardware you control. Qwen, Llama, or Mistral variants on premises or in an EU colo, served via vLLM or TensorRT-LLM. Quantized to 4-bit (AWQ or GPTQ) so a single enterprise GPU (NVIDIA RTX PRO 5000 Blackwell 48GB, AMD Radeon Pro W7900 48GB) handles a useful throughput. No egress, no retention by the model vendor.
2. Graph-augmented retrieval. A code-aware or document-aware RAG layer (ArcadeDB, Neo4j, or equivalent) that gives the model structure, not just text chunks. Cuts hallucinations on internal corpora and makes the retrieval layer auditable.
3. Compliance logging as a first-class system. Every prompt, response, tool call, retrieval hit, and human override logged against a schema-stable format. Not PDF screenshots, not chat-thread archives. Machine-readable, exportable, retention-bound to the regulatory minimum.
4. Human-in-the-loop gate on regulated decisions. Article 14 oversight is not a UI label. It is a routing rule: high-risk outputs do not become decisions until a qualified person reviews them, and the review is itself logged.

This stack costs more than curl https://api.openai.com/.... It also satisfies a procurement officer who has to defend the architecture to their own regulator, which is the position most bank and hospital procurement teams are now in.

The business case a CFO will sign off

Three numbers make the case for a regulated SME without putting hardware costs on the table as the headline.

Vendor risk consolidation. Two of the three largest Western European banks have publicly disclosed AI strategies that name vendor diversification as a strategic priority. If your product is the AI piece in their stack, and your architecture is single-vendor cloud, you are a risk they have to manage. Sovereign AI is substitutable by design (the weights are open, the serving stack is standard, the data stays on the operator side) and it shows up differently in their third-party risk register. For a SaaS company selling into FS, this is a deal-cycle issue, not a compliance tax.

Audit and certification efficiency. A documented Sovereign AI stack collapses the time spent answering the same questionnaire in every procurement cycle. One architecture fact sheet, signed by your compliance officer, clears the AI section of an RFI for most EU banks. The alternative is custom questionnaires per customer, costing engineering hours and elongating sales cycles.

Liability shape. Article 26 of the AI Act places deployer liability on the operator. If your AI vendor's terms include a cap on liability that is below your exposure under product liability law, you carry the delta. Sovereign AI inverts this: the inference surface is yours, the audit trail is yours, and the liability perimeter matches the contract perimeter.

What it is not

Three things to be clear about, because vendor decks are loose with the term.

• It is not "EU region" on a hyperscaler. EU region is a marketing claim. Sovereign AI is an engineering claim about hardware, control, and audit. The two are not the same.
• It is not a refusal to use external models. You can fine-tune an open-weights base model on a hyperscaler for training, then export and serve the weights yourself. Sovereignty is about inference and operations, not about every stage of the ML lifecycle.
• It is not a blocker on multi-cloud. The architecture is portable. The point is that you can point it at any EU-resident hardware you control, including a colo you exit from.

Where to start

For a regulated SME that has not made the architectural decision yet, the order matters.

Start with the use case classification. Is the AI system you are building or buying high-risk under AI Act Article 6 and Annex III? Most business-critical AI in FS, healthcare, and public sector is. Get this answer right first; it determines the rest.

Then map the data flow. Where does training data live, where do prompts go, where do outputs land, who can read each. This is the document your auditor will ask for.

Then decide the stack. On-premises, EU colo, multi-tenant private cloud, or hybrid. The answer depends on your hardware budget, your latency requirements, and whether your customer base accepts colocation in another EU member state.

This is what I do as a Fractional AI Architect through ArtCode Software: one to two days per week embedded into a regulated SME, producing the architectural decision record, the data flow map, and the compliance readiness checklist that the rest of the work depends on.

If you need a fixed-price external view on where you stand, the AI Readiness Audit is a two-day engagement that produces that map. If you want a one-hour working session first, the AI Blueprint Session is 299 PLN and produces a written next-step note you can hand to your CTO.