You can use an open model such as gpt-oss-120b or Llama 3.3 70B without running a server. A model API charges per token and costs nothing while you send no requests.
A serverless GPU runs your own copy of a model and bills by the second while it works. After a quiet spell, its first answer can be slow, while the model loads.
Both routes are compared below from the providers' own pages, read on 27 October 2026, with prices from our weekly records.
Who this is for
- you want answers from an open model without looking after a server
- you are weighing token prices against a rented GPU
- you need a model API that keeps data in Europe
Not for you if you want the model on a machine you control: see where to run an open-source LLM.
Key numbers
- 30 seconds for Novita's serverless GPUs to scale to zero, by its own GPU page Official page,
- 5 seconds a Runpod serverless worker waits after its last job by default, billed Official page,
- 2 token prices per model on a model API: one for input tokens and one for output tokens Official page,
Three ways to use an open model without a server
An API is the simplest start; a serverless GPU runs models that APIs do not offer, such as your own.
| How you pay | While idle | Suits | |
|---|---|---|---|
| Model API (Novita AI, Infomaniak) | Per million input and output tokens | Nothing | Light or uneven use of popular open models |
| Serverless GPU (Runpod, Novita AI) | By the second while a worker runs | Nothing once the workers scale to zero | Your own or a fine-tuned model, used in bursts |
| GPU by the hour | By the second or hour while the machine exists | Often still billed | Steady use or long jobs |
Our guide to GPU clouds by the hour covers the third route.
Model APIs: pay per token
A model API runs the model for you and bills each request by its tokens, with input and output priced separately. Both providers below accept requests in OpenAI's format, so existing code needs a new address and key.
| Plan | Price | New-customer offer | Link |
|---|---|---|---|
| Novita AI OpenAI GPT OSS 120B | $0.05 / $0.25 per 1M tokens, input / output · Pay per use; input and output tokens priced separately checked 27 Oct 2026 | No offer | Visit |
| Novita AI Llama 3.3 70B Instruct | $0.135 / $0.40 per 1M tokens, input / output · Pay per use; input and output tokens priced separately checked 27 Oct 2026 | No offer | Visit |
| Novita AI Qwen3.8 27B | $0.42 / $3 per 1M tokens, input / output · Pay per use; input and output tokens priced separately checked 27 Oct 2026 | No offer | Visit |
| Novita AI DeepSeek V4 Pro | $1.60 / $3.20 per 1M tokens, input / output · Pay per use; input and output tokens priced separately checked 27 Oct 2026 | No offer | Visit |
Input and output tokens are priced separately; both prices are shown.
| Plan | Price | New-customer offer | Link |
|---|---|---|---|
| Infomaniak AI Services: mistralai/Mistral-Small-4-119B-2603 | €0.20 / €0.75 per 1M tokens, input / output · pay as you go, no commitment; billed at the end of each month checked 25 Oct 2026 | No offer | Visit |
| Infomaniak AI Services: swiss-ai/Apertus-v1.5-70B | €0.70 / €2.50 per 1M tokens, input / output · pay as you go, no commitment; billed at the end of each month checked 25 Oct 2026 | 1M free credits to start with AI Services, then pay as you go checked 29 Oct 2026 | Visit |
Infomaniak says your queries are neither recorded nor used to train models.
Novita's terms say it does not train its own models on your content by default. Its pages give a count of regions, 14, without naming them, so ask before you rely on it for EU data.
Infomaniak runs its AI Services in its own data centres in Switzerland. Our EU AI hosting guide covers the data rules.
Serverless GPUs: your own model, billed by the second
A serverless GPU runs a model you choose, including a fine-tuned one, and starts workers only when requests arrive. When none come, the workers scale down and their billing stops.
What a cold start is
A cold start is the wait while a fresh worker loads the model into GPU memory before its first answer. Runpod's docs say larger models take longer to load, and that you pay for this time.
Runpod suggests FlashBoot, cached models or active workers to shorten cold starts, without giving a figure. Active workers run all the time and are billed all the time.
Runpod Serverless and Novita
- Runpod bills a worker from its start to its full stop, rounded up to the second. After its last job, a worker waits five seconds by default, and that wait is billed.
- Runpod's vLLM worker accepts requests in OpenAI's format, for a model you name from Hugging Face.
- Novita's GPU page says its serverless GPUs scale to zero in 30 seconds and bill only for the resources used.
Runpod lists its per-second serverless rates on its own pricing page. Our fine-tuning cost guide covers training a model to serve this way.
When a rented GPU starts to pay off
A model API costs more as your use grows, while a monthly server costs the same however much you use it. The chart shows where the two meet for Llama 3.3 70B.
| Million tokens in a month | Novita AI Llama 3.3 70B Instruct | GPU Mart Enterprise Dedicated GPU Server - A100(80GB) |
|---|---|---|
| 0 | $0.00 | $1,559.00 |
| 5,000 | $940.00 | $1,559.00 |
| 8,292.5532 (same cost) | $1,559.00 | $1,559.00 |
| 10,000 | $1,880.00 | $1,559.00 |
| 15,000 | $2,820.00 | $1,559.00 |
| 20,000 | $3,760.00 | $1,559.00 |
GPU Mart's price is the monthly rate of a 24-month term paid in advance. Neither price states VAT.
On 27 October 2026 the lines crossed at about 8 billion tokens a month. One A100 runs a 70B model only at 4-bit. Our benchmark page shows such a model writing 12 to 25 tokens a second for one user.
At 25 tokens a second without a break, that is about 66 million tokens a month, by our calculation. Serving several people at once raises the figure, but by our estimate billions of tokens a month would take several cards.
For a 70B model, then, the API is the cheaper choice unless you need your own copy for privacy or a model of your own. Our guide to an API, a rented GPU or your own hardware works through speed as well as cost.
Cost at three levels of use
| Example | Part | Cost |
|---|---|---|
| 10 million tokens a month | Input, 8 million tokens | $1.08 |
| 10 million tokens a month | Output, 2 million tokens | $0.80 |
| 10 million tokens a month | Total | $1.88 |
| 100 million tokens a month | Input, 80 million tokens | $10.80 |
| 100 million tokens a month | Output, 20 million tokens | $8.00 |
| 100 million tokens a month | Total | $18.80 |
| 1 billion tokens a month | Input, 800 million tokens | $108.00 |
| 1 billion tokens a month | Output, 200 million tokens | $80.00 |
| 1 billion tokens a month | Total | $188.00 |
Compare the largest bar with the monthly server prices in the chart above.
To place yourself on the chart, read your token count from the provider's usage page after your first week, then multiply it by four.
Why the same model can answer differently
The same open model can give different answers from different providers. One reason is quantisation: a model stored at fewer bits uses less memory, and can lose some quality.
Our quantisation guide shows how much 4-bit and 8-bit change a model, from llama.cpp's published tests. Ask which precision a provider serves before you compare it with your own server.
Keeping the bill under control
Token bills grow with use, and a looping agent can multiply them. Infomaniak lets a user with billing rights set a spending limit, and Runpod applies a default spend limit to its serverless use.
Our guide to stopping a runaway AI bill covers alerts and prepaid credit.
How we checked
We read Novita AI's pricing, GPU and API pages, Infomaniak's AI Services page and Runpod's serverless documentation on 27 October 2026. Prices come from our weekly price records; the token volumes are our examples.
Our own measurements will be added here, with their dates, when our test bench runs them.
Mistakes to avoid
Leaving active workers on
Instead: They are billed all the time; use workers that scale to zero.
Counting only input tokens
Instead: Output tokens cost more; count both.
Assuming the API runs in the EU
Instead: Novita gives a count of regions, not their names; Infomaniak runs in Switzerland.
Judging a model by one provider
Instead: Precision and settings differ; try the same prompts on two.
Other questions
What is a serverless GPU?
A GPU service that runs your chosen model only while requests arrive, billed by the second, and scales to zero when there are none.
Is a model API cheaper than renting a GPU?
For light or uneven use, yes, because an idle API costs nothing. With heavy, steady use a monthly server can cost less; the chart above shows the crossing for Llama 3.3 70B.
What is a cold start?
The wait while a new worker loads the model into GPU memory before its first answer. Larger models take longer, and on Runpod you pay for that time.
Sources and test notes
- Novita AI: model API pricing, read
- Novita AI documentation: LLM API (OpenAI client libraries), read
- Novita AI: GPU instances and serverless GPUs, read
- Novita AI: terms of service, read
- Infomaniak: AI Services, read
- Infomaniak: getting started with AI Services (spending limit), read
- Runpod documentation: Serverless overview, read
- Runpod documentation: Serverless pricing, read
- Runpod documentation: vLLM worker, OpenAI compatibility, read
- Runpod: pricing, read
Every page read on 27 October 2026. Prices come from our weekly price records; the token volumes are our examples.