Skip to content

Installing Local AI: Running a Language Model Offline on a PC or Laptop

You do not need a server or a cloud account to try a language model privately. A recent Windows PC or Mac can run one locally: install a runtime, download a model once, and from then on it answers without an internet connection. This guide covers a single machine, how to match model size to hardware, and the point where one laptop stops being enough. Checked in November 2026.

What will you have at the end?

You will have a language model running on your own computer, usable from a terminal or a chat window, with prompts staying on the device. You also get a local API that scripts on the same machine can call, which is handy for small automations.

The setup consists of:

  • Ollama, the runtime that downloads and runs models. It is open source under the MIT licence.
  • A model that fits your hardware, for example from the Gemma 3 or Qwen3 families in the Ollama library.
  • LM Studio as an optional desktop interface for colleagues who would rather not use a terminal.

The focus is deliberately one workstation. If a team needs to share a model, a server is the better tool, and the options are compared in LLM Inference Servers: Ollama, vLLM, llama.cpp and More.

What does your computer need?

A current operating system, enough memory for the model you choose, and free disk space. Ollama requires Windows 10 22H2 or later, or macOS 14 Sonoma or later. A graphics card is optional, but it is the single biggest factor in how fast answers appear.

What the vendors state:

ItemOllamaLM Studio
WindowsWindows 10 22H2 or newer, Home or Prox64 with AVX2, or ARM such as Snapdragon X Elite
macOSmacOS 14 or newer; Apple silicon uses CPU and GPU, Intel is CPU onlymacOS 14 or newer, Apple silicon only
RAMno fixed figure16 GB recommended
GraphicsNVIDIA driver 551.61 or newer; AMD via ROCm or Vulkanon Windows, at least 4 GB dedicated VRAM recommended

Sources: Ollama on Windows, Ollama on macOS, LM Studio system requirements.

For disk space, the Windows documentation asks for at least 4 GB for the program itself, and notes that models can add tens to hundreds of gigabytes depending on what you keep. If the machine is managed by your IT department, ask before installing. Ollama does not need administrator rights on Windows, but that is a technical fact, not permission.

Step 1: Which model size fits which machine?

The model should fit completely into GPU memory to run quickly; on Apple silicon, unified memory plays that role. Use the download size listed in the Ollama library as a baseline, then leave headroom for the operating system, other apps and the conversation context.

Download sizes for two widely used model families (Ollama library, November 2026):

ModelDownloadContext window
gemma3:1b806 MB32K
gemma3:4b3.4 GB128K
gemma3:12b8.2 GB128K
qwen3:4b2.5 GB256K
qwen3:8b5.2 GB40K
qwen3:14b9.3 GB40K

Sources: gemma3, qwen3.

A rough starting point, to be confirmed on your own device:

  • 8 GB laptop, integrated graphics: 1B to 4B models (gemma3:1b, qwen3:4b). Suited to rewording, short summaries and simple questions.
  • 16 GB RAM or an 8 GB graphics card: 4B to 8B (gemma3:4b, qwen3:8b). Workable for drafts, emails and explanations.
  • 12 to 16 GB graphics card, or a 32 GB Mac: 12B to 14B (gemma3:12b, qwen3:14b). More reliable on longer texts; test with your own material, especially in languages other than English.

Larger models such as gemma3:27b (17 GB) belong on workstations with plenty of memory. Sizing dedicated hardware for bigger models and many users is a different exercise, covered in Hardware for Local LLMs: Sizing a GPU Server Properly. This guide is about the machine already on your desk.

Step 2: How do you install Ollama on Windows?

Download OllamaSetup.exe from the official site and run it. According to the Ollama documentation, the installer does not require administrator rights and installs into your home directory. Ollama then runs in the background, and the ollama command is available in any new terminal.

If drive C: is short on space, put the program and models elsewhere. The install location is a command-line flag; the model folder is set with the OLLAMA_MODELS environment variable:

# install to another drive (PowerShell or Command Prompt)
OllamaSetup.exe /DIR="D:\Apps\Ollama"

# set the model folder for your user account
setx OLLAMA_MODELS "D:\ollama-models"

Quit Ollama and start it again, or open a new terminal, so the change takes effect. Without it, models go to C:\Users\<name>\.ollama\models (FAQ). On NVIDIA hardware, check that the driver is version 551.61 or newer first.

Step 3: How do you install Ollama on a Mac?

Open the downloaded ollama.dmg and drag Ollama into Applications. On first launch the app asks whether it may link the ollama command into /usr/local/bin; allow it so the terminal can find the command.

Models and settings live in ~/.ollama (Ollama on macOS). Apple silicon Macs use both CPU and GPU; Intel Macs fall back to CPU only, which is generally slower. Confirm the install:

ollama --version

Step 4: How do you download and run a model?

Pull the model once while you are online, then run it. After that it sits on disk, and every later start loads from there with no download. The commands are the same on Windows and macOS.

# download once (needs internet)
ollama pull gemma3:4b

# start an interactive chat, leave with /bye
ollama run gemma3:4b

# list installed models
ollama list

Ollama also offers cloud-hosted models alongside local ones. To make sure this machine only ever computes locally, the FAQ describes two switches: set the environment variable OLLAMA_NO_CLOUD=1, or create ~/.ollama/server.json with:

{ "disable_ollama_cloud": true }

After a restart, the log shows "Ollama cloud disabled: true", and cloud models and web search are no longer available.

The default context window is 4,096 tokens. For longer documents, raise it with OLLAMA_CONTEXT_LENGTH, keeping in mind that a bigger window needs more memory.

Step 5: When is LM Studio the better front end?

LM Studio suits people who prefer a desktop app over a terminal. It combines model search, downloads, a chat window and chatting with your own documents in one interface. It does not support Intel Macs, and on x64 Windows machines it requires a CPU with AVX2.

Inside the app, search for a model and download it; the documentation suggests a 4-bit option or higher if the machine can handle it (downloading models). Per the offline documentation, chats and uploaded documents stay on the device, while model search, downloads and runtime or app updates need a connection.

On licensing, the terms of service allow personal and internal business use. When employees use it at work, the company is the customer, and some features may be charged for. You can install Ollama and LM Studio side by side, but each keeps its own copy of the models, so disk usage adds up.

Testing: is the model really local and on the GPU?

Two checks are enough. Disconnect from the network and ask a question; then run ollama ps to see where the model is loaded. If it answers offline and the PROCESSOR column reads "100% GPU", the machine is running it as fast as it can.

# in a second terminal while a model is loaded
ollama ps

According to the FAQ, PROCESSOR shows "100% GPU", "100% CPU" or a split such as "48%/52% CPU/GPU". A split means the model does not fully fit in graphics memory; pick a smaller model or a shorter context window.

Scripts on the same machine can reach the model through the local API (API reference):

curl http://localhost:11434/api/chat -d '{
  "model": "gemma3:4b",
  "messages": [{"role": "user", "content": "In two sentences, what is a delivery note?"}],
  "stream": false
}'

The reply is in the message.content field.

Running it day to day: cost and security

Apart from electricity there is no running cost: Ollama is open source, and models such as Gemma 3 or Qwen3 are free to download under their own licences. The security questions are network exposure, updates, and what data you paste into the chat.

Practical points:

  • Keep it local. By default Ollama listens on 127.0.0.1:11434, so only the same machine can reach it (FAQ). Setting OLLAMA_HOST to 0.0.0.0 exposes the API to the whole network, and controlling who can reach it becomes your job. At that point it is no longer a single-user setup.
  • Memory is released. A model stays loaded for five minutes after the last request by default; ollama stop <model> unloads it immediately.
  • Plan updates. Ollama ships frequently; the releases page lists version 0.40.15 as current in early November 2026. Updating needs internet, so schedule it rather than leaving it to chance.
  • Local is not exempt from data rules. Personal data typed into a local chat is still processing under the GDPR or your local equivalent, and should follow your internal policies. This is not legal advice.
  • Check the output. Small models write fluently but can get facts wrong. Verify figures, names and quotations.

When does a single workstation stop being enough?

One machine works well while one person uses it for their own, non-sensitive text. The limits show up when several people need the same model, when company data from shared drives or business systems should be connected, or when you need control over who can see what.

Typical signs:

  • Several users. Ten laptops each running their own model means ten installs, ten update states and usually models that are too small. A shared inference server with a capable GPU is easier to run.
  • Company data. Once the model should read document stores, tickets or an ERP system, you need connectors, search and logging. A laptop chat does not provide that.
  • Permissions and audit. Who may query which documents, and who asked what and when? A single workstation has no roles and no central log.
  • Networks without internet. In an isolated network, ollama pull cannot reach the registry. Models have to be verified and moved in through a controlled transfer process.

If the real question is whether a local model can replace ChatGPT across a company, see Using ChatGPT Offline? Local Alternatives for Businesses Compared. That comparison looks at options for whole teams; this guide gets exactly one computer working.

Next steps

Try two or three model sizes on real tasks from your work, such as replying to an email or summarising an inspection report, and note quality and response time. That tells you quickly whether one machine is enough or a shared server makes more sense.

Frequently asked questions

Do I still need internet after installation?

Only to download models and updates. Once a model is on disk, chatting works without a connection, which you can confirm by switching off Wi-Fi and asking a question.

Will a laptop without a dedicated graphics card work?

Yes for small models in the 1 to 4 billion parameter range, though replies are slower. Apple silicon Macs share memory between CPU and GPU, which gives them an advantage here.

Can I run local AI on an Intel Mac?

Ollama runs on Intel Macs but uses the CPU only. LM Studio does not support Intel Macs at all.

Is LM Studio allowed for business use?

Its terms permit personal and internal business use, and some features may be paid. Each model you download also comes with its own licence.

Sources

  1. Ollama Docs: Windows docs.ollama.com
  2. Ollama Docs: macOS docs.ollama.com
  3. Ollama Docs: FAQ docs.ollama.com
  4. Ollama Docs: API /api/chat docs.ollama.com
  5. Ollama Releases (GitHub) github.com
  6. Ollama Library: gemma3 ollama.com
  7. Ollama Library: qwen3 ollama.com
  8. LM Studio Docs: System Requirements lmstudio.ai
  9. LM Studio Docs: Offline Operation lmstudio.ai
  10. LM Studio Docs: Download an LLM lmstudio.ai
  11. LM Studio App Terms of Service lmstudio.ai