Where to run an open-source LLM, from a CPU-only server to eight GPUs

Where you can run a self-hosted large language model (LLM) depends on how much memory it needs, which follows from its size. By our calculation, its weights need about 2 GB per billion parameters at 16-bit precision and a little over 0.5 GB at 4-bit. So an 8-billion-parameter model runs on an ordinary server with no GPU, while the largest need a machine with eight GPUs, or even several machines.

Self-hosting an LLM means running an open model on hardware you control or rent, instead of calling someone else's API. An open model is one whose weights, the numbers it learned in training, are published for anyone to download, sometimes after accepting its licence.

If the weights fit in about 16 GB and one person can wait for each reply, a CPU-only server will do. Up to 80 GB, rent one GPU by the hour, and move to a monthly GPU server once it is busy most of the month. A monthly server with one card holds up to 141 GB; beyond that, you need several GPUs in one machine. An open-model API works at any size. Each route has a price box filled from our weekly price checks, so every price shows the day we last checked it.

Who this is for

• you want to run an open model such as Qwen, gpt-oss, Llama or DeepSeek on a server you rent
• you know which model you want, or at least its size, and need the cheapest machine it fits on
• you are weighing a rented server against paying per token through an API

Not for you if you only run n8n or an AI agent that uses a model through an API: see our AI agent hosting guide.

Key numbers

• 2 GB memory the weights need per billion parameters at 16-bit (about 0.56 GB at 4-bit) Our calculation, 25 Oct 2026
• 16 GB enough memory for gpt-oss-20b, according to OpenAI Official page, 25 Oct 2026
• 80 GB one GPU of this size holds gpt-oss-120b, according to OpenAI Official page, 25 Oct 2026
• 640 GB GPU memory of eight H100 cards in one machine, the most in our price records so far Our calculation, 25 Oct 2026
• €0.20 in / €0.75 out per million tokens (checked 25 October 2026) Infomaniak's API price for Mistral Small 4 (119B) Official page, 25 Oct 2026

1. A CPU-only server (small models in RAM): connects to One GPU by the hour (needs a GPU).
2. One GPU by the hour (24 to 80 GB of GPU memory): connects to A GPU server by the month (busy most of the month).
3. A GPU server by the month (24 to 141 GB, one fixed monthly fee): connects to Several GPUs in one machine (outgrows one GPU).
4. Several GPUs in one machine (2 to 8 GPUs, 640 GB in our records so far): connects to An open-model API (outgrows one machine).
5. An open-model API (no server; pay per million tokens)

Monthly costs for hourly prices assume the server runs all month (730 hours), or less where a provider caps the monthly bill.

Start with the model's size

Each weight is a parameter, and the number of parameters is the model's size, such as 8 billion. Running a model to answer requests is called inference. For inference, the whole model must sit in memory: the GPU's own memory (VRAM), the server's RAM or both.

Working out the memory the weights need is plain arithmetic. Multiply the number of parameters, in billions, by the number of bits stored for each one, then divide by 8 to get gigabytes. For each billion parameters, that gives:

• 2 GB at 16-bit precision, the form Qwen3.5-9B, Qwen3.8-27B and Llama 3.3 70B are published in
• 1 GB at 8-bit
• about 0.56 GB at 4-bit: 4-bit files also store small scale numbers, so we count 4.5 bits

Storing each weight in fewer bits is called quantisation (often spelled "quantization"). This makes the model smaller on disk and in memory, so it fits on smaller hardware. On top of the weights, every conversation needs working memory, the key-value (KV) cache. It grows with the context window, the amount of text the model keeps in view. Our GPU memory guide works out that part in full, model by model.

Models read and write text in tokens: short pieces of text, often a word or part of one. Some models, including gpt-oss, DeepSeek-V4.1-Flash and Kimi K3, are built as a mixture of experts: only part of the model works on each token. The memory rule still counts every parameter, because all of them must be loaded. OpenAI's gpt-oss-120b, for example, has 117 billion parameters, of which 5.1 billion are active for each token.

The 4-bit column is the lowest precision this guide sizes for, although llama.cpp can go down to 1.5-bit. Some models are published already compressed: Kimi K3's weights and the expert weights of gpt-oss are stored in a 4-bit format called MXFP4. For them, and for DeepSeek-V4.1-Flash, the table gives the size of the files as released. The KV cache comes on top. To try other models and context windows, use the GPU memory calculator: it does the sums for you.

Which route suits your model

Four questions settle most cases. Work out the memory with the formula above first. The largest single card in our price boxes, NVIDIA's H200, holds 141 GB.

A CPU-only server for small models

The cheapest place to run a model is an ordinary virtual private server (VPS) with no GPU at all. The model sits in the server's RAM and runs on its processor cores. Ollama, a model runner, reports "100% CPU" in ollama ps when a model sits entirely in RAM. The open-source engine llama.cpp supports the vector instructions of x86 processors (AVX to AMX), is tuned for Apple silicon and also runs on GPUs.

What fits depends on how much RAM is left once the operating system has taken its share. By our formula, Qwen3.5-9B needs about 5.4 GB at 4-bit. OpenAI states that gpt-oss-20b runs within 16 GB of memory. A server with 16 to 24 GB of RAM therefore covers most models up to about 20 billion parameters at 4-bit.

This route suits one person trying out models, running private experiments or background jobs where a slow answer does no harm.

Our guide to running Ollama on a server without a GPU shows the speeds public benchmarks report for small models on a processor alone.

One GPU by the hour

When a model needs a GPU, renting one by the hour is the most flexible way in. You start a cloud server with one GPU, run your job and delete the server when you have finished. The two providers in the price box below bill in different ways:

• DigitalOcean bills its GPU Droplets, as it calls these servers, by the second. It keeps billing a switched-off Droplet until you delete it.
• UpCloud bills its GPU servers by the hour, and any part of an hour counts as a full one. It caps each month's charge at 28 days of use.

The GPU's memory decides what fits, and NVIDIA's product pages give the sizes. On a 24 GB L4, Qwen3.8-27B fits at 4-bit, with about 15.6 GB of weights. A 48 GB L40S holds Llama 3.3 70B's 4-bit weights, about 40 GB, with room for short conversations only. At 32K tokens, it needs an 80 GB card. An 80 GB H100 runs gpt-oss-120b, which OpenAI says fits on a single GPU of that size.

Hourly rental suits experiments, batch jobs and work that runs a few hours a day. You pay for every hour the server runs, busy or idle, so a GPU left running overnight costs as much as a busy one. Here is what part-time use adds up to on one H100:

Runpod and Vast.ai also rent GPUs by the hour. They will join the box above once our weekly checks have confirmed a price and a location for each.

For more on this route, see our guide to the best GPU clouds for renting by the hour. Our guides to running Llama 3.3 70B on one rented GPU and to what gpt-oss-120b needs cover those two models.

A GPU server by the month

If the model answers requests all day, every day, a dedicated GPU server can cost less than renting by the hour. You pay one fixed fee a month, however much you use it. The whole server is yours for the term.

Read the terms before you compare. The GPU Mart prices we list are the monthly rate of a 24-month term, paid in advance. Liquid Web charges less for the first month. Its row in the box shows that first-month price; the note under it gives the full price from the second month. Unihost bills monthly with a one-month minimum, and some of its servers add a one-off set-up fee, given in the note under the price.

The chart below compares the two ways of paying for an 80 GB H100. Up to the point where the lines cross, paying by the hour costs less; beyond it, the monthly server is cheaper.

A month is counted as 730 hours (8,760 hours in a year divided by 12).

The numbers in this chart

Hours in a month	DigitalOcean GPU Droplet, 1 x NVIDIA H100	GPU Mart Enterprise Dedicated GPU Server - H100
0	$0.00	$2,099.00
200	$882.00	$2,099.00
400	$1,764.00	$2,099.00
475.9637 (same cost)	$2,099.00	$2,099.00
600	$2,646.00	$2,099.00
730	$3,219.30	$2,099.00

The monthly price is the rate of GPU Mart's 24-month term, paid in advance. The hourly line counts only the hours the server runs.

Our guide to dedicated GPU servers by the month works out the break-even point for each card. It also covers set-up fees and contract terms in more detail.

Several GPUs in one machine

Once a model's weights outgrow the largest single card, you need a server with two, four or eight GPUs that share the work. The documentation of vLLM, an open-source model server, calls this tensor parallelism. When a model is too big for one GPU but fits in one machine, vLLM splits it across that machine's GPUs. You set --tensor-parallel-size to the number of GPUs.

Our price records hold machines with 64 GB (two RTX 5090 cards) to 640 GB (eight 80 GB H100 cards) of GPU memory. That counts only the machines whose memory the provider states. DeepSeek-V4.1-Flash, whose files take 510.3 GB as released, needs one of the eight-card machines with 640 GB.

Beyond 640 GB, no machine in our price records is big enough yet. Kimi K3, with 2.8 trillion parameters, takes 1,560.9 GB as released. Larger single machines can be rented: UpCloud lists servers with eight B200 or eight B300 cards, and DigitalOcean lists GPU Droplets with eight H200 cards. By NVIDIA's figures, they hold about 1,400 GB, 2,100 GB and 1,128 GB, so only the B300 machine fits Kimi K3. We will add their prices once we have checked them.

Past one machine, you need a cluster: several machines that run one model together. The vLLM guide says this takes tensor parallelism plus pipeline parallelism, which spreads the model's layers over the machines. Clusters are beyond the plans in this guide. For most people, an API that serves such a model is the practical route. Our guide to running Kimi K3 or DeepSeek V4 yourself looks at the hardware in detail.

Or skip the server: an open-model API

An open-model API runs the model for you and charges by the token. You send a request over the internet and pay for the tokens you send and the tokens the model writes back, each priced per million. There is no server to size, patch or switch off, and the model can be any size the provider offers.

The trade-off is less choice and less control. You can only use the models and versions the provider runs, and your prompts leave your own systems. Infomaniak, a Swiss host, serves open models from its data centres in Switzerland. They include Mistral Small 4 (Mistral-Small-4-119B-2603, 119 billion parameters) and Apertus 70B (Apertus-v1.5-70B). Its AI Services page states that prompts are not recorded or used to train models (read 25 October 2026).

Novita AI gives a count of its regions, not their names, so ask it if your data must stay in the EU; Infomaniak runs in Switzerland. Our guide to using an open model without a server compares these services.

Cost per million tokens: the API price, and the sum for your own server

On the API route the price is already per million tokens, so the sums are short. Here is a month's cost for a small team, using our own example of 20 million tokens in and 5 million out:

On your own server the sum runs the other way, from the hourly price to the tokens the server can write in that hour. For a server that is busy all the time: cost per million tokens = hourly price ÷ (tokens per second × 3,600) × 1,000,000. Tokens per second is the speed at which a server writes its answer. It changes with the model, the software and the number of users.

Speed decides the cost per million tokens on your own server, and it depends on the model, the card and the software. Our guide to whether an API, a rented GPU or your own hardware is cheapest shows how to work out the break-even.

Which software to run the model with

Ollama, llama.cpp, vLLM and SGLang are four programs that serve open models on your own server. Each has its own minimum requirements, and the table lists what their official documentation asked for on 25 October 2026. Compute capability, in the last column, is NVIDIA's version number for the features of a GPU.

Our comparison of vLLM, Ollama, llama.cpp and SGLang covers how they differ. Both vLLM and llama.cpp come with a server that answers in the format of OpenAI's API. A chat front end or an agent built for that format can therefore work with either.

A private chat assistant for a team

A common reason to run your own model is a private chat assistant for a team. It has three parts: a chat front end opened in a browser, a model server and the GPU server that holds the model. Open WebUI, a self-hosted chat front end, describes itself as "built to run entirely offline". It connects to Ollama or to any OpenAI-compatible API, such as vLLM's.

1. Each team member (in a web browser): connects to Open WebUI (HTTPS).
2. Open WebUI (the chat front end): connects to vLLM or Ollama (OpenAI-style API).
3. vLLM or Ollama (serves the model through an API): connects to GPU server.
4. GPU server (holds the model in its memory)

The number of people matters too. By default, Ollama handles one request per model at a time, according to its FAQ, and the OLLAMA_NUM_PARALLEL setting raises that limit. Our guide to a private, ChatGPT-style assistant for your team will size the server for 5 to 20 people.

Is running your own model worth it?

It can be, if you need control or your traffic is steady and high. Control means you choose the exact model and version, decide where your data goes and keep that model for as long as you want. It also costs effort: you install and update the software, secure the server and pay for it whether it is busy or idle.

An API charges only for what you use, so it can stay cheaper until your traffic is steady and high. The fair test sets the API's price per million tokens against your server's cost per million tokens, from the formula above. That needs the speed your server reaches with your model. Our guide to whether an API, a rented GPU or your own hardware is cheapest will then run the numbers.

Meanwhile, the chart shows how much more a month costs once a GPU is involved, for plans priced in US dollars by the month.

$0 $1,000 $2,000 $3,000 Hostinger KVM 2 $8.79/month DigitalOcean Basic Droplet, 8 vCPUs, 16 GiB $96.00/month Kamatera Cloud server Type B (General Purpose), 4 cores, 16 GB $132.00/month GPU Mart Enterprise Dedicated GPU Server - RTX 4090 $409.00/month GPU Mart Enterprise Dedicated GPU Server - RTX 5090 $479.00/month GPU Mart Enterprise Multi-GPU Dedicated Server - 2xRTX 5090 $859.00/month GPU Mart Enterprise Dedicated GPU Server - A100(80GB) $1,559.00/month GPU Mart Enterprise Dedicated GPU Server - H100 $2,099.00/month GPU Mart Enterprise Multi-GPU Dedicated Server - 4xA100 $1,899.00/month

The numbers in this chart

Plan	Price	Checked on
Hostinger KVM 2	$8.79/month	25 Oct 2026
DigitalOcean Basic Droplet, 8 vCPUs, 16 GiB	$96.00/month	25 Oct 2026
Kamatera Cloud server Type B (General Purpose), 4 cores, 16 GB	$132.00/month	25 Oct 2026
GPU Mart Enterprise Dedicated GPU Server - RTX 4090	$409.00/month	25 Oct 2026
GPU Mart Enterprise Dedicated GPU Server - RTX 5090	$479.00/month	25 Oct 2026
GPU Mart Enterprise Multi-GPU Dedicated Server - 2xRTX 5090	$859.00/month	25 Oct 2026
GPU Mart Enterprise Dedicated GPU Server - A100(80GB)	$1,559.00/month	25 Oct 2026
GPU Mart Enterprise Dedicated GPU Server - H100	$2,099.00/month	25 Oct 2026
GPU Mart Enterprise Multi-GPU Dedicated Server - 4xA100	$1,899.00/month	25 Oct 2026

Hourly plans and plans priced in euros are left out, so that every dot is in the same unit and currency. Liquid Web is left out because its first month is discounted. Several of these prices need a 24-month term; the price boxes above give each term.

Mistakes to avoid

• Sizing a model by its parameter count alone

  Instead: Work out the memory at your precision, plus room for the KV cache.

• Counting only the active parameters of a mixture-of-experts model

  Instead: Count them all: gpt-oss-120b has 5.1 billion active parameters but 117 billion to load.

• Comparing a 24-month rate with an hourly price as if they were alike

  Instead: Read the term under each price and compare like with like.

• Leaving an hourly GPU server running after the job is done

  Instead: Delete it when you finish; see our guide to stopping a runaway AI bill.

• Opening the model server to the whole internet

  Instead: Keep Ollama on its default address, 127.0.0.1, which only the server itself can reach. Connect through a front end or a tunnel.

• Installing vLLM on a GPU it does not support

  Instead: Check the card's compute capability first: vLLM's GPU guide asks for 7.5 or newer.

Related guides and pages

For more detail, see these guides and pages:

• How much GPU memory an LLM needs, model by model
• GPU memory calculator for LLMs
• Running Ollama on a server without a GPU
• The best GPU clouds for renting by the hour
• Running Qwen3.8-27B on one 24 GB or 32 GB GPU
• Dedicated GPU servers by the month
• Can you run Kimi K3 or DeepSeek V4 yourself?
• Use an open model without a server
• How fast open models run on rented GPUs
• AI Hosting Price Index
• AI hosting in Europe: where your data sits

To find the plans for your project, start from the finder on our homepage. The comparison table on the same page lets you pick what you want to run, then filter by memory, billing and location. The providers in this guide have their own pages: Hostinger, DigitalOcean, Kamatera, Cherry Servers, Contabo and UpCloud. So do Runpod, Vast.ai, GPU Mart, Unihost, Liquid Web, Infomaniak and Novita AI.

How we price

Every price on this page comes from our price records; none is written into the text by hand. We read each price on the provider's own pricing page. We note its currency, whether it includes VAT, its billing unit and any term or promotion. We check it again every week, and the date beside each price is our latest check. Our page on how we check prices explains the rest.

We show a machine with several GPUs only when its provider states a price for the whole machine. This guide rests on the providers' published prices and the official documentation listed below.

Other questions

Can a model use the GPU and the processor at the same time?

Yes, when the model is too big for the GPU alone. Both llama.cpp and Ollama can split a model between GPU memory and RAM; Ollama's ollama ps command then shows a split such as "48%/52% CPU/GPU". The part held in RAM runs on the processor, so only the rest gets the GPU's speed.

Is a self-hosted copy as good as the online model?

Not always: a copy compressed to a low bit count is not identical to the original. Its context window, the text the model keeps in view, may also be short. Ollama's context-length page sets the default by GPU memory: 4k tokens below 24 GiB, 32k from 24 to 48 GiB and 256k from 48 GiB. Our guide to why a self-hosted model can seem worse will cover the fixes.

What are small models good for on a CPU-only server?

In a Lemmy Selfhosted thread from 16 June 2025, users named jobs for models of 3 billion parameters or fewer. They mentioned turning free text into structured data (JSON), handling voice commands in Home Assistant and writing short one-off scripts. Others there found anything below about 7 billion parameters too limited. Our guide to running Ollama on a server without a GPU shows the waiting times at typical processor speeds.

Sources and test notes

• OpenAI's gpt-oss-20b model card on Hugging Face, read 25 Oct 2026
• OpenAI's gpt-oss-120b model card on Hugging Face, read 25 Oct 2026
• Qwen3.5-9B model card on Hugging Face, read 25 Oct 2026
• Qwen3.8-27B model card on Hugging Face, read 25 Oct 2026
• Meta's Llama 3.3 70B Instruct model card on Hugging Face, read 25 Oct 2026
• DeepSeek-V4.1-Flash model card on Hugging Face, read 25 Oct 2026
• DeepSeek-V4.1-Flash-1017 model card on Hugging Face (the files we size), read 25 Oct 2026
• Kimi K3 model card on Hugging Face, read 25 Oct 2026
• Mistral Small 4 (119B) model card on Hugging Face, read 25 Oct 2026
• Apertus 70B model card on Hugging Face, read 25 Oct 2026
• vLLM documentation: installing on a GPU, read 25 Oct 2026
• vLLM documentation: online serving and its OpenAI-compatible server, read 25 Oct 2026
• vLLM documentation: parallelism and scaling, read 25 Oct 2026
• Ollama documentation: installing on Linux, read 25 Oct 2026
• Ollama documentation: hardware support, read 25 Oct 2026
• Ollama documentation: FAQ, read 25 Oct 2026
• Ollama documentation: context length, read 25 Oct 2026
• Ollama on GitHub (downloads for macOS, Windows and Linux), read 25 Oct 2026
• llama.cpp on GitHub (project page), read 25 Oct 2026
• SGLang documentation: installation, read 25 Oct 2026
• SGLang on GitHub (project page), read 25 Oct 2026
• Open WebUI documentation, read 25 Oct 2026
• NVIDIA: CUDA GPUs and compute capability, read 25 Oct 2026
• NVIDIA H100 product page, read 25 Oct 2026
• NVIDIA H200 product page, read 25 Oct 2026
• NVIDIA A100 product page, read 25 Oct 2026
• NVIDIA L40S product page, read 25 Oct 2026
• NVIDIA L4 product page, read 25 Oct 2026
• NVIDIA GeForce RTX 5090 product page, read 25 Oct 2026
• NVIDIA GeForce RTX 4090 product page, read 25 Oct 2026
• NVIDIA HGX page (total GPU memory of the B200 and B300 eight-GPU systems), read 25 Oct 2026
• Infomaniak AI Services page (data centres and prompts), read 25 Oct 2026
• Infomaniak AI Services prices, read 25 Oct 2026
• DigitalOcean Droplet pricing documentation (GPU Droplets: per-second billing, billing while switched off, eight-GPU machines), read 25 Oct 2026
• DigitalOcean Gradient AI Platform pricing (serverless inference), read 25 Oct 2026
• UpCloud pricing (GPU server prices, billed by the starting hour; eight-GPU B200 and B300 servers), read 25 Oct 2026
• UpCloud GPU Servers page (FAQ: hourly billing, monthly cost capped at 28 days), read 25 Oct 2026
• GPU Mart home page (prices shown on a 24-month basis), read 25 Oct 2026
• GPU Mart H100 server page (24-month term), read 25 Oct 2026
• GPU Mart multi-GPU servers, read 25 Oct 2026
• Database Mart billing FAQ (24-month term paid in advance), read 25 Oct 2026
• Liquid Web GPU server configurator (first-month discount), read 25 Oct 2026
• Unihost dedicated servers in the Netherlands (minimum term, set-up fees), read 25 Oct 2026
• Runpod pricing, read 25 Oct 2026
• Vast.ai pricing, read 25 Oct 2026
• Novita AI pricing, read 25 Oct 2026
• Lemmy Selfhosted thread on small self-hosted models (user comments, posted 16 June 2025), read 25 Oct 2026

No test was run for this guide. Prices come from our weekly price records, and each price box shows the date of its check. Memory figures come from our GPU memory guide. They are our calculation from the parameter counts and file sizes that Hugging Face lists for each model, read 25 October 2026.

Guides in this section

• 
  GPU memory by model

  How much GPU memory popular open models need at 16-bit, 8-bit and 4-bit, the extra that long conversations take, and which rentable GPUs fit each one.

• 
  Kimi K3 and DeepSeek V4 yourself

  Kimi K3 has 2.8 trillion parameters. We work out the GPUs it and DeepSeek V4 need, the hourly cost of the machines and when an API makes more sense.

• 
  Llama 3.3 70B on one GPU

  Which rented GPU holds Llama 3.3 70B at each precision, the speed each card can reach by NVIDIA's own figures, and the Ollama commands to run it.

• 
  gpt-oss-120b

  gpt-oss-120b needs one 80 GB GPU, gpt-oss-20b a 16 to 24 GB card. The memory each needs, the cards to rent and the Ollama and vLLM commands to run them.

• 
  Speed on rented GPUs Data page

  Tokens per second for open models on the H100, H200, A100, L40S, RTX 4090 and other GPUs, from public llama.cpp benchmarks, with sources and dates.

• 
  Qwen3.8-27B on one GPU

  Which Qwen3.8-27B files fit on one 24 GB or 32 GB GPU, how long a conversation each allows, the speed to expect and the commands to run it.

• 
  vLLM, Ollama, llama.cpp or SGLang

  Which serving engine to run on a rented GPU: published tests of vLLM, Ollama and SGLang, what each suits, and the set-up commands for all four.

• 
  Private chat for a team

  What server a private chat assistant for 5 to 20 people needs, Open WebUI or AnythingLLM, and step-by-step set-up with logins and HTTPS.

• 
  Dedicated GPU servers

  Monthly GPU servers from GPU Mart, Unihost and Liquid Web: cards, data centres, terms and fees, and when a monthly server beats paying by the hour.

• 
  Quantisation and quality

  How much quality 8-bit, 6-bit, 4-bit and 2-bit files lose, by llama.cpp's own tests, and the settings that make a self-hosted model answer worse.

• 
  GPU clouds by the hour

  Runpod, Novita, DigitalOcean, UpCloud and Vast.ai compared from their own pages: billing steps, stopped-machine charges, locations and live prices.

• GPU memory calculator Calculator

  Pick a model, precision and context length to see the GPU memory it needs and the GPU plans that fit, with prices from our weekly checks.

See every guide