Running Llama 3.3 70B on one rented GPU: what fits and how fast it runs

Llama 3.3 70B fits on one rented GPU only when compressed. At 4-bit its file is 40 to 43 GB. A 48 GB card then runs it with short conversations, an 80 GB card with long ones.

Speed depends mostly on the card's memory bandwidth. By our calculation one H100 can write at most about 84 tokens a second at 4-bit. Real runs reach roughly half of such a limit.

The figures below come from Ollama's model files, NVIDIA's specifications and NVIDIA's own benchmark of this model, read on 27 October 2026.

What a 70B model needs at each precision

Ollama offers Llama 3.3 70B from 2-bit to full 16-bit. Each conversation also needs memory for its cache. Our GPU memory guide works out about 0.33 MB per token at 16-bit, or 10.7 GB for 32K tokens.

Lower precision saves memory at some cost in quality. The 4-bit default is the usual compromise; 3-bit and 2-bit save more but lose more.

How fast it runs on one card

For one user, each new token means reading the whole model from the card's memory once. So the most tokens a second a card can write is its memory bandwidth divided by the size of the model file.

Where does the estimate come from? NVIDIA's own benchmark ran Llama 3.3 70B at 8-bit across two H100 cards. For one user and a 5,000-token prompt, it measured 51.67 tokens a second and 0.41 seconds to the first token.

Each card held about 35 GB of the model there, so its limit was about 95 tokens a second. The measured speed is 54% of that, and we apply the same share to the H100, H200 and A100.

A 2024 benchmark on cards rented from RunPod backs this up. With Llama 3 70B at 4-bit in llama.cpp, an A100 SXM wrote 24 tokens a second, about half its limit.

The L40S wrote 15, about three-quarters of its limit, so we use that share for it (our calculation). Our speed figures by card list these runs and more.

Many users at once change the picture. Serving software such as vLLM or NVIDIA's NIM handles requests together. In the same benchmark, 50 users at once got about 814 tokens a second in total. Our comparison of vLLM, Ollama, llama.cpp and SGLang helps you pick the software.

Step by step with Ollama

Ollama is the quickest way to try the model on a rented card. Its documentation asks for an NVIDIA card with compute capability 5.0 or higher and driver 550 or newer, which every card above meets.

nvidia-smi                          # the card, its memory and the driver
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.3:70b             # 43 GB, 4-bit
ollama run llama3.3:70b "Summarise the main risks of running AI on your own server."
ollama ps                            # PROCESSOR should say 100% GPU

On a 48 GB card, pull llama3.3:70b-instruct-q4_0 instead, which is 40 GB. If ollama ps shows part of the model on the CPU, the card is too small for that file and the speed drops sharply.

Ollama answers one request per model at a time unless you raise OLLAMA_NUM_PARALLEL. Keep its port 11434 off the internet, as our guide to running Ollama explains.

Which rented GPU to pick

For one person or a small team, an 80 GB card is the comfortable choice. The default 4-bit file and long conversations fit with room to spare. A 48 GB card works for short jobs at 4-bit.

To compare a card with a model API, work out its cost per million tokens. Divide the hourly price by (tokens a second x 3,600), then multiply by 1,000,000. Our guide to API, rented GPU or your own hardware does this in full.

When two GPUs make more sense

Two cards make sense for 8-bit or 16-bit quality, for many users at once, or for very long documents. NVIDIA benchmarks this model on two H100 cards at 8-bit, and our price records include servers with two and eight H100 cards.

The licence

Llama 3.3 comes under Meta's Llama 3.3 Community License. It is free to use, but a service with more than 700 million monthly users needs Meta's permission. Meta also asks you to show "Built with Llama" where you use it.

How we checked

We read Ollama's model page and GPU documentation, NVIDIA's product pages and benchmark, and Meta's licence on 27 October 2026. The same day we read a 2024 benchmark on rented cards. The memory and speed sums are ours, set out above.

Our own measurements will be added here, with their dates, when our test bench runs them.

Back to Where to run an open model