GPU memory calculator for LLMs

Pick a model, the precision you will run it at and how long your conversations are. The calculator shows the GPU memory the model needs and the GPU plans on this website that fit, with their checked prices.

It uses the same method as our GPU memory guide, which has the full model table and worked examples. Without JavaScript, the button sends the form and the page shows your result.

Monthly costs for hourly prices assume the server runs all month (730 hours), or less where a provider caps the monthly bill.

Results for three example models

"Fits" names the smallest GPU memory, from 24 GB up to eight 80 GB cards, that holds the total within 90%.

What the inputs mean

• Model: nine open models whose layers and attention we read from their own config files. For another model, enter its size in billions of parameters; only its weights are counted then.
• Precision: 16-bit is the full size most makers release; 8-bit and 4-bit are smaller, quantised copies. Our quantisation guide shows what each costs in quality.
• Tokens per conversation: the prompt, any documents and the answer together. A model reads at most its own longest input, which the calculator checks.
• Conversations at once: each one keeps its own conversation memory on the GPU.

How it calculates

Weights take two bytes per parameter at 16-bit, one at 8-bit and 0.5625 at 4-bit. The 4-bit figure is our rule: 4.5 bits a parameter, because 4-bit files also store scales.

Conversation memory, the key-value cache, keeps a key and a value for every token of every conversation. Each token takes layers × key-value heads × head size × four bytes. A layer with a sliding window counts at most its window of tokens.

A model fits when weights and conversation memory stay within 90% of the GPU memory, our rule, close to vLLM's default of 92%. The GPU memory guide uses the makers' own 4-bit and 8-bit files where they exist, so a few figures there differ slightly from this rule.

Limits of any estimate

• The serving software adds memory of its own, which the 10% margin is meant to cover.
• A server with two or more cards holds a model only when the serving software splits it across them. The vLLM software does this with its tensor-parallel setting.
• A mixture-of-experts model, such as gpt-oss-120b, needs room for all its weights, although only some of them work on each token.
• Storing the key-value cache at 8-bit, which vLLM offers, halves the conversation memory; the calculator counts it at 16-bit.
• Real 4-bit files vary a little: gpt-oss-120b's files hold about 4.47 bits a parameter.

Where to rent the GPU

Our guides compare GPU clouds by the hour and dedicated GPU servers by the month. The benchmark page shows how fast each card runs a model, from published results.

How we checked

The layers, attention heads and sizes of the nine models come from their own config files. We read them for our GPU memory guide on 25 October 2026; the GPU plans and prices come from our weekly price records.

Sources and test notes

• Qwen3.5-9B config file, read 25 Oct 2026
• Gemma 4 12B config file, read 25 Oct 2026
• gpt-oss-20b config file, read 25 Oct 2026
• Qwen3.8-27B config file, read 25 Oct 2026
• Gemma 4 31B config file, read 25 Oct 2026
• Qwen3.6-35B-A3B config file, read 25 Oct 2026
• Llama 3.3 70B model list (layers and heads), read 25 Oct 2026
• gpt-oss-120b config file, read 25 Oct 2026
• Mistral Medium 3.5 128B config file, read 25 Oct 2026
• vLLM documentation: engine arguments (memory use, key-value cache precision), read 25 Oct 2026

Model files read on 25 October 2026 for our GPU memory guide; the calculation is ours.

Back to Where to run an open model