
Local AI on your own servers — even without internet
The first question in a mid-sized company is almost never "what can AI do?". It is "where does our data end up?" There is a better answer to that than a clause in a contract: local AI means the language model runs on machines inside your network. Queries and documents never leave the building — and if you want, there is no connection to the outside at all.
- Models that run on your servers
- The hardware for it, properly sized
- Making your documents searchable
- Partly in house, partly in the cloud
- Operations, so it still runs in six months
- From the cloud back in house
Which data cannot go to the cloud?
Design data, costings, personnel files, customer contracts, formulations, source code. When an agent needs access to this to be useful at all, the setup decides whether the project can happen.
No accidental leakage
If the model runs on your hardware, no document and no query leaves your network. That is not in the contract — it is a property of how the system is built.
Independent of vendors
Prices change, models get retired, terms get revised. If you run the layer underneath yourself, you can swap the model without rebuilding everything.
Costs you know in advance
If you pay per request, you only learn the bill afterwards. Your own server costs once, and very little that is hard to predict after that.
You can show who saw what
In your own environment it can be logged who saw which information, when and by which route. That makes every conversation with data protection and audit easier.
GDPR and the EU AI Act
The AI Act has largely applied since August 2026. For most agents in a mid-sized company that means two things: your people must know what they are working with, and it must be recognisable when an answer comes from an AI. Both are settled in a paragraph at the start — or in a project later on. Anyone wanting to pre-sort job applications lands in a stricter category; we discuss that before building anything.
The industry term: on-premise
It always means the same thing: the models run on machines you own, inside your network. Renting a German data centre instead meets the same requirement without buying hardware. We discuss both.
Three levels: from your own server to fully offline
Local is not always equally local. How far the AI has to be separated from the internet depends on your data and your requirements. We build all three levels on the same foundation.
1 · Your own server
The model runs in your server room or in a rented German data centre. Your data stays with you; updates and new models come in over the normal internet connection.
2 · No cloud
No traffic to AI providers, no telemetry. The firewall only lets through what you have approved. For your staff it feels like an internal ChatGPT.
3 · Fully offline
No connection to the outside, not even for updates ("air-gapped" in the jargon). Models, software and updates are checked and brought in on physical media. For production networks, R&D departments and classified areas.
What it takes for AI to run in your own building
Models that run on your servers
Selecting, running and tuning open models on your hardware — matched to the task rather than "the biggest one that fits".
The hardware for it, properly sized
Sizing, procurement and setup of the servers. Dimensioned against real load, not against datasheet numbers.
Making your documents searchable
A source for every answer, and every employee only sees what they are already allowed to see. Without a single document leaving your network.
Partly in house, partly in the cloud
Sensitive material stays local, uncritical material may go out. You draw the line — we make sure it holds technically.
Operations, so it still runs in six months
Availability, updates, logging, cost control. So the infrastructure does not become IT’s problem.
From the cloud back in house
Bringing existing cloud solutions back in house step by step, without stopping day-to-day operations.
What we will say openly about this
Local is not automatically better. Your own hardware costs money and needs to be operated, and the largest models still run in the cloud. And it does not pay for itself on cost: an agent answering a few hundred requests a day is cheaper via the cloud for years. Your own server pays off when your data cannot leave the building — or when an agent works all day. Which path is right for you is what we settle before anybody orders hardware.
What companies ask us about local AI
Does AI work without internet?
Yes. Open language models run entirely on your own hardware and need no internet connection. The model is brought into the network once and then works offline: searching documents, answering questions, writing drafts. Only new models and updates have to come in — on physical media if necessary.
Can you use ChatGPT offline?
Not ChatGPT itself; it only runs in OpenAI’s cloud. But there are open models that run on your own servers, and interfaces that work like ChatGPT. The result is an internal ChatGPT where no input ever leaves your building.
How do updates get into a network without internet?
Through a defined route: new models and software are downloaded outside, checked, and brought into the isolated network on physical media or through a data gateway. Which versions are running is documented. We set it up so your IT can repeat it without us.
Does local AI run on a single PC or laptop?
Smaller models run on a good workstation, offline too — fine for trying things out or for individuals with highly confidential data. As soon as several people use it at once, or an agent accesses company data, the AI belongs on a central server.
What hardware do you need for local AI?
For most agents in a mid-sized company, a single server with powerful graphics cards, sitting in your existing server room, is enough. How big it needs to be depends on how many people use it at the same time and which models the task requires. We size the hardware against your real load, not datasheet figures — and before anyone orders anything.
Is local AI as good as ChatGPT?
For general world knowledge and very demanding tasks, the largest cloud models are still ahead. For what an agent does in a company — reading documents, giving answers with sources, writing drafts — open models running on your own hardware are good enough in many cases. Whether that holds for your use case is something we test on your data before you invest.
Is local AI automatically GDPR-compliant?
Not automatically, but it can be operated in line with data protection law, and a lot becomes simpler: if no data leaves your network, the question of what a cloud provider does with it disappears. Permissions, logging and retention periods still have to be set up properly. We build that in from the start.
Your own server or a German data centre?
Both can meet the requirement that your data stays under your control. Your own server pays off when data must not leave the building at all, or when an agent works all day. A rented data centre saves buying hardware. We work through the numbers for both with you.
What has to stay in house for you?
We look together at which of your data really cannot go out — and what that means for the setup.