Air-Gapped AI: Running LLMs in Fully Isolated Networks
Some data must never sit on a network that can reach the internet: unreleased product designs, defence-related engineering files, lab results, documents under government security clearance. If you want language models to work with that data, the usual answer is an air gap. Running a model offline is easy. Running a whole AI stack offline for years is not, because every dependency that normally downloads itself quietly now has to be solved by hand.
What is air-gapped AI and who needs it?
Air-gapped AI means the model, the inference software, any agents and all data run in a network with no connection to the outside world: no direct link, no proxy, no VPN, no remote support tunnel. Files enter only through a controlled transfer point. Organisations choose it when confidentiality rules or regulation exclude both cloud services and ordinary corporate networks.
Typical settings are R&D departments with classified designs, production and lab networks, public agencies, and suppliers bound by security clearance obligations. The tasks themselves look familiar: searching and summarising documents, drafting inspection reports, answering questions about manuals, preparing text.
It helps to separate three operating models that often get mixed up:
| Model | Outside connection | Typical risk |
|---|---|---|
| Self-hosted AI on the corporate network | Yes, for updates and downloads | Data paths are open by design |
| Segmented network, outbound blocked | Logical link exists, filtered | One rule change opens it |
| Air gap | None | Missing dependencies, slow updates |
Self-hosting on the normal network is the common starting point for sovereign AI. The air gap is the strictest variant, and it changes how you build, update and support everything.
How does an LLM work without internet access?
At runtime a language model does not need the internet. It consists of weight files, inference software and hardware with enough memory. The online dependencies almost always come from everything around it: first-run downloads, update checks, telemetry, licence validation and packages that install themselves on demand.
An air-gapped deployment has five building blocks:
- Model weights as files, pinned to a version and a checksum.
- Inference software such as Ollama, llama.cpp, vLLM or Hugging Face Transformers, delivered as packages or container images.
- Dependencies: Python packages, system libraries, GPU drivers, container base images.
- Application layer: chat interface, document search index, agents and their tools.
- Transfer point: the technical and organisational route by which all of the above enters the network.
The first four exist in any self-hosted setup. The fifth defines the air gap. It decides how fast you can patch, who may bring files in and how you prove what is actually running.
Offline switches you should set anyway
Many tools have an explicit offline mode. Set it even though the network is already isolated; otherwise software waits on timeouts or fails in ways that are hard to trace.
- Hugging Face Hub: According to the environment variable reference,
HF_HUB_OFFLINE=1stops all HTTP calls to the Hub and uses cached files only; a missing file raises an error.HF_HUB_DISABLE_TELEMETRY=1turns off telemetry across Hugging Face libraries, andHF_HUB_DISABLE_UPDATE_CHECK=1stops thehfCLI checking for new releases. - Transformers: You can also pass
local_files_only=Truewhen loading a model so only local files are used (Transformers installation guide, offline mode). - Ollama: The Ollama FAQ documents where models are stored and the
OLLAMA_MODELSvariable to move them. Cloud models and web search can be switched off withOLLAMA_NO_CLOUD=1ordisable_ollama_cloudin~/.ollama/server.json. The same FAQ notes that Ollama on macOS and Windows downloads updates automatically, so in an air gap you replace that with manual updates.
# Environment for an inference service inside the air gap
export HF_HUB_OFFLINE=1
export HF_HUB_DISABLE_TELEMETRY=1
export HF_HUB_DISABLE_UPDATE_CHECK=1
export HF_HOME=/srv/ai/huggingface
export OLLAMA_NO_CLOUD=1
export OLLAMA_MODELS=/srv/ai/ollama-models
Which deployment patterns exist?
There is no single air-gap pattern. The three common ones differ in how strict the separation is and how often data crosses the boundary. The right choice depends on how current your models and documents must be, and how heavy your approval process is.
Isolated workstation or single server. One GPU machine sits in a secured room without any network. Users work at it directly. This is the simplest option for a single team but does not scale once several people need access at the same time.
Isolated network segment with multiple users. An inference server, a document store and client machines form their own network. Users reach a chat interface through a browser; agents search internal file shares. This is the most common setup in engineering and administration.
AI in production and OT networks. The model runs close to machines and control rooms, for example on an industrial PC or a Jetson module, and works with alarm messages, shift logs or maintenance manuals. Operational technology adds its own demands: availability, fixed maintenance windows, zones and conduits between network areas. NIST SP 800-82 Rev. 3, published in September 2023, is a guide to securing operational technology and a useful reference for this architecture.
Dedicated articles on moving models across the gap, hidden online dependencies and AI in OT networks will be listed below this overview automatically once they are published.
What do you need before you start?
You need three things: hardware that will last for years without new purchases, a complete inventory of every software component, and a defined transfer process. Missing any one of them turns every change into a project, because inside an air gap nothing can be installed on the fly.
Hardware
Size GPU memory and disk generously. Swapping a model means the old and new versions sit side by side for a while, next to a local package mirror, saved container images and backups. Data centre GPU servers and Jetson-class edge computers are typical platforms. The general relationship between GPU memory and model size is covered in hardware sizing for local LLMs; the air gap mainly adds headroom for parallel versions and transfer data.
Software inventory
List every component with its version: operating system, GPU driver, container runtime, inference server, Python packages, models, user interface. This list doubles as your bill of materials for security reviews and is the starting point for every update.
Licences
Check for each model and each product whether offline operation without a licence server is permitted and technically possible. Some commercial software validates its licence online at intervals. Open-weight models come with their own licence terms, which you should read before anything goes in.
Process and ownership
Decide who may carry files across, who approves them and how that is recorded. Without such a rule a shadow route appears sooner or later: a USB stick with "just one quick package" on it.
How do you set it up, step by step?
The pattern is always the same: prepare outside, verify outside, carry across, test inside, then switch. The single most useful rule is to build a complete replica of the target environment outside the air gap and boot it with the network unplugged before anything crosses over.
1. Build an offline replica. On a machine with the same operating system and CPU architecture, install the whole stack. Then disconnect the network and restart everything. Whatever fails now would have failed inside the air gap too.
2. Download models pinned to a revision. Fetch weights at a fixed revision so you can always say which version went in. The hf CLI supports --revision and --local-dir for this (Hugging Face CLI guide).
# Download a model at a fixed revision into a transfer folder
hf download <org>/<model> --revision <commit-hash> --local-dir ./transfer/model
# Create checksums for the transfer
find ./transfer -type f -exec sha256sum {} + > ./transfer/SHA256SUMS
3. Check the weight format. Prefer Safetensors or GGUF. The Python documentation is explicit that the pickle module is not secure: malicious pickle data can execute arbitrary code during unpickling. Safetensors was designed as a safe alternative to pickle for storing tensors. A network you isolate precisely because the data is sensitive should not run files that execute code on load.
4. Bundle Python packages into a wheelhouse. pip download resolves dependencies like pip install but saves the files to a folder. With --platform and --only-binary=:all: you can collect wheels for a different target system; inside, you install with --no-index --find-links (pip documentation).
# Outside: collect packages and their dependencies
pip download -r requirements.txt -d ./transfer/wheelhouse
# Inside: install from the wheelhouse only
pip install --no-index --find-links=/srv/transfer/wheelhouse -r requirements.txt
5. Export container images. docker save writes images to a tar archive that docker load restores on the other side (Docker documentation). Pull images outside with a fixed version tag rather than latest, so your inventory stays unambiguous.
6. Carry the bundle across. Inside, verify every checksum against the list created outside. Record date, contents, origin and approval.
7. Install and test inside. In Ollama you can register a model from local files: a Modelfile whose FROM line points to a GGUF file or a Safetensors directory, followed by ollama create (Ollama: importing a model).
# Modelfile pointing to the transferred GGUF file
echo 'FROM /srv/transfer/model/model.gguf' > Modelfile
ollama create internal-model -f Modelfile
8. Connect the application and agents. Agents may only use tools that live inside the same network: file shares, databases, a local search index. Remove tools that need web search or external APIs from the configuration rather than letting them fail silently.
Limits and risks
An air gap stops data leaving over the network, but it makes operations slower and more expensive. Models age, security fixes arrive late, and the transfer point becomes an attack surface of its own. If you do not plan for these limits, you end up with a secure system that nobody uses.
- Freshness: Models and software are only as recent as the last delivery. New model versions and security patches need a fixed schedule.
- The transfer point as an entry route: Removable media and transfer workstations are the only way in, which makes them the obvious target. Checksums, format rules and malware scanning belong in every transfer.
- Hidden dependencies: Some software downloads tokenisers, fonts, dictionaries or licence data the first time a feature is used. This often shows up only in production.
- Fewer capabilities: Web search, cloud models and external tools are gone. Agents can only work with what is inside the network.
- Getting results out: Outputs sometimes have to leave the network too. That needs the same review process in the opposite direction.
- Not legal advice: Whether an air gap is legally required, for example under security clearance or export control rules, is a question for your legal and compliance team. This article covers the technical side.
When an air gap is the wrong choice
If your data is confidential but not classified, a self-hosted model in a segmented network with tightly filtered egress is often enough. It avoids the transfer process and keeps patching simple. Whether running models in-house makes sense at all is the subject of local AI for business, and hybrid AI describes the middle path of sending non-sensitive tasks to cloud models, which an air gap rules out by definition. An air gap pays off when any outside connection is explicitly ruled out.
Where do you start?
Start with one use case and a trial outside the isolated network: build the replica, unplug it and use it for a week. Only once that environment runs reliably without internet is it worth investing in the transfer process, approvals and hardware for the target network.
A sensible order for the first weeks:
- Pick a use case that only needs documents already inside the network, such as questions about internal manuals or pre-structuring inspection reports.
- Choose the model and inference software and confirm the licences allow offline use.
- Build the replica outside, disconnect it and log every failure.
- Write down the software inventory and the transfer process.
- Make the first delivery, test it inside, approve it.
- Agree an update rhythm, for example quarterly for models and as needed for security fixes.
Monitoring, ownership and update routines for self-hosted models in general are covered in operating self-hosted AI. All of it applies inside an air gap as well; each step simply passes through the transfer point.
In our own operations, keeping the offline replica permanently has proven worthwhile: every new delivery boots there without a network first, before it goes to the transfer point. It costs one machine, but it avoids debugging inside the air gap, where every fix means another delivery.
If agents are part of the plan, design the application layer from day one: which tools does the agent need, which data sources live inside the network, how are results reviewed? Background on agents and where they fit is in the AI agents section, and searching internal documents is covered under AI agents for knowledge management. For the organisational side of running models under your own control, see sovereign AI. More background articles are collected in the guides.
Documentation and settings referenced here were checked in November 2026.
Frequently asked questions
Is blocking outbound traffic on the firewall enough?
No. A firewall with outbound rules blocked is a segmented network, and one misconfiguration can open it again. An air gap has no logical link at all; data crosses only through a deliberate transfer process.
Which models work offline?
Any model whose weights you can download and whose licence allows use without a licence server. Prefer weights in Safetensors or GGUF format over pickle-based files.
How do I update a model inside the air gap?
Treat it like any other software release: download outside, record checksums and origin, carry it in, test it on a staging system and only then switch production.
Can AI agents use tools in an air-gapped network?
Yes, as long as every tool lives inside the same network, such as internal databases, file shares or a local search index. Tools that depend on web search or external APIs have to be removed.
Sources
- Hugging Face: huggingface_hub Environment variables (HF_HUB_OFFLINE, HF_HUB_DISABLE_TELEMETRY) huggingface.co
- Hugging Face Transformers: Installation, Offline mode huggingface.co
- Hugging Face: Command Line Interface (hf download) huggingface.co
- pip documentation: pip download pip.pypa.io
- Python documentation: pickle — Python object serialization docs.python.org
- Hugging Face: Safetensors huggingface.co
- Ollama FAQ docs.ollama.com
- Ollama: Importing a model docs.ollama.com
- Docker documentation: docker image save docs.docker.com
- NIST SP 800-82 Rev. 3: Guide to Operational Technology (OT) Security csrc.nist.gov