Running Ollama on a server without a GPU: what speed to expect

On a processor alone, a small model writes about 35 tokens a second and an 8-billion-parameter one about 7.7. Those are the middle results in LocalScore, a public benchmark where people upload results from their own computers.

That speed suits background jobs such as sorting e-mails or tagging records. It is too slow for a chat that someone waits on. The server's memory decides which models fit; its processor decides how fast they run.

LocalScore's tests run on llamafile, which builds on the same llama.cpp project as Ollama. Treat its speeds as a guide, not an exact match.

The short answer

• speed, by LocalScore's middle results: about 35 tokens a second for a 1B model, 7.7 for an 8B model and 4.7 for a 14B model
• reading a long prompt is the slow part: about a minute for 2,048 tokens on an 8B model
• 8 GB of memory holds models up to about 4 billion parameters beside n8n
• a 9 billion parameter model fits in 8 GB at 4-bit only if little else runs
• 16 GB holds every small model on our list at either compression; 24 GB also holds gpt-oss-20b
• a GPU is the next step once people wait on the answers: see when to move to a GPU

What the public benchmark shows

LocalScore is a Mozilla Builders project. People run its benchmark on their own computers and can upload the results. It records the processor, the memory and the operating system with each one.

Each model runs nine jobs, from a short question to a long document. We took every result that used the processor alone and worked out the middle value for each model.

The processor makes a large difference. The table below shows single results for the 8B model, from server processors to a desktop chip and a virtual server. Each row is LocalScore's average for that machine.

What a rented server changes

A rented plan gives you a few cores of a large server processor, not all of them. LocalScore does not record how many cores a virtual machine had, so its results for them vary widely.

For the 1B model, results under the processor names that virtual machines usually report ranged from 2.9 to 38 tokens a second. The fastest two said "AMD EPYC-Rome"; a "QEMU Virtual CPU" was the slowest.

Most cheap plans share their cores with other customers' servers, so speed can change from hour to hour. Dedicated cores cost more.

Which jobs work at CPU speed, and which do not

Speed matters only when someone is waiting. This table turns the middle results into waiting times.

• good fits: sorting and tagging incoming messages, short summaries and pulling fields out of text into JSON
• also good: drafts that someone reads later, and other n8n or agent steps that run in the background
• poor fits: a chat window that people type into, long reports, and anything several people use at the same time

Set it up

Ollama's own install script sets it up as a service that starts with the server. On a fresh Ubuntu or Debian server, as root:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3.5:4b
ollama run qwen3.5:4b "Say hello in five words."
ollama ps   # the PROCESSOR column should say 100% CPU

Keep it private

By default Ollama only listens on the server itself (127.0.0.1, port 11434), and its documentation describes no login for its API. Keep it that way when n8n or your agent runs on the same server. To reach it from elsewhere through OLLAMA_HOST, put a reverse proxy with a password in front, or use a private network between your servers. Never open port 11434 to the internet.

Docker needs one extra care. Ollama's Docker command publishes the port with -p 11434:11434. Docker's own documentation says a port published like that is open on every address of the server, past a ufw firewall. Bind it to the server itself instead:

docker run -d -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run qwen3.5:4b

Settings worth changing

• OLLAMA_CONTEXT_LENGTH: the default context is 4,096 tokens, enough for short jobs; a longer one needs more memory and is slower
• OLLAMA_KEEP_ALIVE: a model leaves memory 5 minutes after its last answer, then loads again next time. A longer value avoids that if the server has the memory
• OLLAMA_NUM_PARALLEL: 1 by default, so a second request to the same model waits for the first

sudo systemctl edit ollama.service
# add, under [Service]:
#   Environment="OLLAMA_KEEP_ALIVE=30m"
#   Environment="OLLAMA_CONTEXT_LENGTH=8192"
sudo systemctl daemon-reload && sudo systemctl restart ollama

Memory: what fits in 8, 16 and 24 GB

On a server without a GPU, the model sits in the server's ordinary memory. Its file size plus room for the conversation must fit next to the operating system and your other software.

The last column is our reading, not a measurement. It leaves about 2 GB for n8n and the system, as in our guide to how much memory n8n needs.

Our GPU memory guide explains the same sums for larger models.

Connecting n8n or an agent

In n8n, the Ollama credential's Base URL is http://localhost:11434 by default. Three cases cover most errors, according to n8n's own documentation:

• ECONNREFUSED ::1:11434: your server prefers IPv6 and Ollama listens on IPv4. Use http://127.0.0.1:11434
• n8n in Docker, Ollama installed on the server: inside a container, localhost is the container itself. Start n8n with --add-host host.docker.internal:host-gateway, use http://host.docker.internal:11434, and let Ollama listen beyond 127.0.0.1 with OLLAMA_HOST, while the firewall keeps port 11434 closed to the internet
• both in Docker: put them on one user-defined network and use http://ollama:11434; the port needs no publishing at all

docker network create n8n-ollama
docker network connect n8n-ollama ollama
docker network connect n8n-ollama n8n
# Base URL in n8n: http://ollama:11434

Our guide to securing self-hosted n8n covers the rest of the server.

When to move to a GPU

Move when people wait on the answers or several use it at once. Move too when you need a model above about 10 billion parameters at a usable speed. A GPU by the hour is the cheapest way to try that. Our guide to where to run an open-source LLM compares every route with this week's prices.

On a GPU, an 8B model writes many times faster. In a public llama.cpp benchmark, an RTX 4090 wrote Llama 3 8B at 4-bit at about 128 tokens a second. Our speed figures by card list more cards.

For a chat assistant a whole team can log in to, our guide to a private chat assistant for a team sizes the server.

How we checked

We read LocalScore's results and its method on 27 October 2026 and worked out the medians from its own page data. Ollama's, n8n's and Docker's documentation and the model pages on ollama.com were read the same day.

Our own measurements will be added here, with their dates, when our test bench runs them.

Back to AI agent hosting