gpt-oss-120b: what it needs and where to run it

OpenAI's gpt-oss-120b runs on one rented 80 GB GPU, such as an H100, as its model card says. Its files take 65.2 GB, and even its longest conversation adds less than 5 GB.

It is also fast for its size: only 5.1 billion of its 117 billion parameters work on each token. Its smaller sibling, gpt-oss-20b, needs a 16 to 24 GB card.

The figures below come from OpenAI's model cards, vLLM's and Ollama's documentation and NVIDIA's own figures, read on 27 October 2026. The memory sums come from our GPU memory guide.

What gpt-oss-120b and gpt-oss-20b need

Both models come compressed: OpenAI trained their expert layers in MXFP4, a 4-bit format. Our GPU memory guide adds the cache each conversation needs and keeps 10% of the card free.

Ollama's library lists the two at 65 GB and 14 GB, each with a 128K context window. OpenAI's own figure for the smaller model, within 16 GB, leaves less room for long conversations than a 24 GB card does.

Why it runs faster than its size suggests

gpt-oss is a mixture-of-experts model. Each token passes through only a few of its many expert networks. The card therefore reads about 5 billion parameters per token instead of 117 billion.

That cuts the work per token, not the memory: the whole model must still sit on the card. The speed is closer to a small model's than to a dense 70B model's, such as Llama 3.3 70B.

NVIDIA's launch post of 5 August 2025 quotes up to 256 tokens a second on a GeForce RTX 5090, with Ollama or llama.cpp. That card has 32 GB, so the figure is for gpt-oss-20b.

A result posted to llama.cpp's public scoreboard points the same way: gpt-oss-20b wrote 241 tokens a second on a 32 GB RTX PRO 4500 Blackwell. Our speed figures by card compare it with dense models.

Which rented GPU to pick

For gpt-oss-120b, rent an 80 GB card or larger. The recipe from vLLM lists the H100, H200 and B200 among the cards it supports. It says support for older A100-class and L40S-class cards is still in progress.

For gpt-oss-20b, a 24 or 32 GB card is enough. On L4 or RTX cards, use Ollama, since vLLM's support for those cards is still in progress.

Step by step with Ollama

OpenAI's model card gives two commands for Ollama. After installing it on a GPU server:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull gpt-oss:120b      # 65 GB; use gpt-oss:20b (14 GB) on a 24 or 32 GB card
ollama run gpt-oss:120b
ollama ps                     # PROCESSOR should say 100% GPU

Keep Ollama's port 11434 on the server itself, as our guide to running Ollama explains.

Step by step with vLLM

vLLM serves the model to many users at once through an OpenAI-style API. Its recipe installs it in a Python environment, or runs it from its Docker image:

uv venv
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm serve openai/gpt-oss-120b

# or with Docker
docker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai --model openai/gpt-oss-120b

Both models were trained on OpenAI's harmony response format and only work correctly with it. The vLLM server handles the format, and its recipe recommends the Responses API that OpenAI suggests for this model.

Through an API instead

Several model API providers offer gpt-oss-120b and charge per token. For light or irregular use, that can cost less than renting an 80 GB card all month. Our guide to API, rented GPU or your own hardware shows where the costs cross.

The licence

Both models come under the Apache 2.0 licence, which allows commercial use and changes. OpenAI's model card describes it as free of copyleft restrictions and patent risk.

How we checked

We read OpenAI's model cards, vLLM's recipe, Ollama's library and NVIDIA's launch post on 27 October 2026. The memory sums come from our GPU memory guide, worked out on 25 October 2026.

Our own measurements will be added here, with their dates, when our test bench runs them.

Back to Where to run an open model