A safe AI environment with your own rules.

This is the practical companion to Practical AI governance and compliance. That note argues that compliance is mostly a procurement decision, and that governance, the risks of bad output, unintended actions and misuse, is yours to own. Standard tools take you some of the way. This guide is for when you need to go further: when you want to decide which models are used, what the AI can reach, and who can see what happened.

You do not build this from scratch. Everything below is assembled from proven, mostly open-source building blocks. The tools named are examples of each layer as of 2026, not recommendations or endorsements. The layers will stay the same longer than the names will.

The shape: one gateway, everything routed through it

The whole idea fits in one picture. Models on one side, your systems on the other, and one control point in the middle that every request passes through. The tools people actually work in sit behind the gateway, so their access is whatever the gateway allows.

What the gateway does

The gateway is where your AI policy stops being a document and becomes a setting. Five things matter most.

Control	What it does
Filter connector access	Allow-list tools per user or team. You can enforce read-only access to a system by exposing only its read tools.
Block sensitive data	Detect personal, financial or confidential information, and block the prompt or tool call before it reaches an external system.
Restrict which models	Allow-list the models each key or team may use. Keep sensitive work on approved or EU-hosted models only.
Cap spend and rate	Budgets per key, user and team, plus rate limits on requests and tokens. Runaway cost stops before it happens.
Log every request	One trail of who called what, which model, and what it cost. The evidence that your policy is being followed.

Map these back to the three governance risks. Read-only connectors and narrow tool lists stop unintended actions. Model allow-lists and data guardrails keep sensitive material where it belongs. Per-user access and the audit log are what stand between you and the employee who wants to take the data with them.

The building blocks, layer by layer

Models: the engine

Where the model runs decides how private your data is.

• Frontier providers. Anthropic, OpenAI and Google give you the most capable models via API. Mistral is EU-based, which helps where data residency matters.
• Open-weight models. Qwen, Gemma, Llama and others can be hosted in the EU only, for example through Scaleway.
• Local models. Run on your own infrastructure with a tool like Ollama. Nothing leaves the building. You give up some capability for complete privacy.

The gateway: the control point

LiteLLM is a control centre for model usage, connector (MCP) access and skills. You set access per person and per team, configure guardrails for personal and confidential information, and see who drives cost, on which models, and where the spend goes.

Front-ends: where people work

• LibreChat. A self-hosted ChatGPT. A private chat interface for your team, connected to any model, with users, roles and history you control.
• nao. Ask your data in plain English. It connects to your data warehouse and turns business questions into answers, a governed analytics assistant.
• Ohmo. A personal agent in the chat apps people already use, Slack or Teams, with memory, tools and permissions. Built on the OpenHarness framework.

Cowork tools: AI that carries out tasks

Eigent is an AI workforce on the desktop: a team of AI workers that carry out multi-step tasks, and an alternative to Claude’s cowork mode. It is available as open source and with paid support.

For developers

Coding tools such as Cursor and OpenCode can point at the same gateway, so the people building things are under the same keys, budgets and logs as everyone else.

Front-ends and cowork tools sit behind the gateway. That is the point: whichever tool someone opens, the rules are the same.

Where to start

You do not have to build the whole thing at once. Each step stands on its own.

1. Get the contracts right. Business plans with zero retention or no training, EU processing where it matters, and an AI policy that says which data goes where. This is compliance, and it comes first.
2. Put a gateway in front. Route API use through it. You get one place for keys, budgets, model allow-lists and a log, often in weeks, not months.
3. Add targeted tools. Pick the one or two use cases worth controlling closely, like analytics on your own data, and run them behind the gateway.
4. Move to your own setup. Chat, agents and connectors all behind the gateway, with the models you choose. The most control, and the most to own.

Every step still needs a named owner, a policy the setup enforces, and someone who actually reads the log.

Be realistic

You cannot fully stop people from going around this layer. A private account is one browser tab away. The infrastructure is there to help people comply and to prevent unintended mistakes, not to act as a hard wall. If the governed setup is slower or weaker than the private tab, people will use the private tab.

And every restriction you add removes use cases. Lock everything to read-only and nobody gets an agent that saves them an hour a day. The right balance matters more than the number of controls.

The risks are knowable. The building blocks already exist. The job is choosing how far along the spectrum you need to go.