SV DE

Reliable IT support
for those with better things to do.

Networks, servers, NAS, security and support – remote or on-site in the Sundsvall region. Clear price before work begins.

Response within 24 hours First call free of charge RUT tax deduction available
Guides & prices Beginner 9 min

A local language model in your own office: hardware, costs and limits

A language model that sits in your office rather than in a data centre abroad: no text leaves the building, it works without internet, and there is no fee per user. For some tasks that is genuinely useful – for others it is not. Here is what you need, what it costs and where the limits are.

When local makes sense – and when it does not

A local language model (LLM) runs on a computer inside your own network. Prompts, documents and answers stay there. That is the main reason small businesses are interested. What an LLM actually is, and why it does not "think", is a topic in its own right: in short, it is a statistical model that predicts plausible text.

Local usually makes sense when

Local is usually the wrong choice when you need the best possible quality. The large cloud models are clearly stronger at difficult writing, long documents, programming and reasoning than anything that fits on an office computer.

Model size and memory: the key calculation

Models are named after their number of parameters: 8B means eight billion. Locally you almost always run a compressed ("quantised") version with roughly 4 bits per parameter. Rules of thumb for that version:

Model sizeMemory for the modelSensible hardware
3–4Bapprox. 2–3 GBalmost any current PC with 16 GB RAM
7–8Bapprox. 4–5 GB16 GB RAM or a graphics card with 8 GB
12–14Bapprox. 8–10 GBgraphics card with 12–16 GB
27–32Bapprox. 17–20 GBgraphics card with 24 GB or 32 GB+ unified memory
70Bapprox. 40–50 GB64 GB+ memory, very expensive

On top of that comes memory for the context – the text the model "sees" at once. If you want to process long documents in one go, you need several gigabytes more. If the model does not fit entirely into graphics memory, the processor handles the rest. That works, but it is noticeably slower.

Without a graphics card: possible, but slow

A model can run on the processor (CPU) alone. Speed depends mainly on how fast the memory is, not on the number of cores. On an ordinary office PC with 32 GB RAM, a 7–8B model typically produces a few words per second. That is enough for:

If several people are to work smoothly in a chat window during the day, a graphics card or a machine with fast unified memory is the better choice.

Three typical setups and what they cost

SetupSuitable forHardware (approx., October 2026)
Existing PC, 32 GB RAM, CPU onlytesting, background jobs, models up to 8BSEK 0, possibly more RAM
PC with a 16 GB graphics card (e.g. RTX 5060 Ti 16 GB)models up to approx. 14B, fast, several users taking turnscard approx. SEK 6,500–7,500, plus a suitable PC
Mini PC or workstation with lots of unified memory (e.g. with AMD Ryzen AI Max and 64–128 GB)models up to approx. 32B and beyond, quiet, frugaldepends heavily on memory – compare current prices

Hardware prices fluctuate a lot, especially for graphics cards and memory – check the current price before you buy. Then there is electricity: a PC with a graphics card draws a few hundred watts under load and much less when idle. Running around the clock, that is roughly a few hundred kronor a year.

By comparison, cloud licences are charged per person per month. With five people you quickly pass SEK 10,000 a year, every year. Whether a cloud assistant pays off is covered in Is Copilot for Microsoft 365 worth it?

Software: free, but not maintenance-free

A typical setup consists of a program that runs the models (for example Ollama or LM Studio) and a ChatGPT-style web interface (for example Open WebUI) that everyone in the office reaches through the browser. The models themselves can be downloaded freely, from families such as Llama, Mistral, Qwen and Gemma. Check the licence: many allow commercial use without further ado, some come with their own conditions.

What a local model is good for in the office

Clear, specific instructions matter even more with small local models than with the big cloud ones: give the context, the format you want and an example.

What it cannot do

Operation, updates and data protection

A local model is a server like any other and needs the same care:

"Local" does not automatically mean GDPR-compliant. Personal data is still personal data on your own server – the only thing that disappears is the cloud provider as a recipient. The EU AI Act (Article 4) also addresses staff understanding of the AI tools they use. In a small office, a short introduction and a written rule on what the model may be used for go a long way. This is not legal advice.

A note on licences: I generally recommend one-off purchased software over subscriptions, above all Office as a purchased version and Windows Server: no running licence costs, programs and data stay usable, no price rises, predictable and local. To be honest: purchased versions get security updates only for a fixed period, and email or video conferencing are services that have to run. Anyone who definitely wants Microsoft 365 will of course get it set up by me too. The trade-off is in Microsoft 365 or an alternative for small businesses.

Checklist before you buy

  1. Write down three to five specific tasks the model should handle.
  2. Test with real (anonymised) sample texts on an existing PC before buying hardware.
  3. Decide how many people will use it at the same time.
  4. Choose the model size first – then the hardware.
  5. Agree who installs updates and who is the contact person.

Frequently asked questions

Do I need a graphics card?
No. Small models up to about 8B run on the processor if there is enough memory (32 GB is comfortable). For several people to work smoothly, a 16 GB graphics card or a machine with lots of unified memory is clearly better.

Is a local model as good as ChatGPT?
No. For summaries, drafts and internal texts it is often enough, but on difficult tasks the large cloud models are clearly ahead. So test with your own texts first.

Is it automatically GDPR-compliant if everything is local?
No. Only the external provider disappears. You still need to manage access rights, how long chat histories are kept, and backups.

Does it work without internet?
Yes. Internet is only needed to download software and models and for updates. After that the system runs fully offline.

No affiliate links. Recommendations are based purely on hands-on experience.

Want to know whether a local language model can handle your texts? I test it with your sample documents, recommend hardware and set everything up – the "Local language model – trial setup" package for SEK 5,000 excl. VAT (hardware not included). In English, Swedish or German; reply within 24 hours.

IT consultant in Sundsvall Book a free call

External sources:

All articles Is Copilot for Microsoft 365 worth it? →