Fine-tune a model or use a chatbot on your documents? How to choose

For an AI that answers from your own documents, use a document chatbot that searches them (retrieval-augmented generation, or RAG), not a fine-tuned model. Fine-tuning pays off mainly when the model itself must behave differently, for example to write in a fixed format or tone. A chatbot also costs far less than a tuned model on its own rented GPU, as the chart below shows. DocsBot AI's Standard plan is $149 a month (checked 25 October 2026), while DigitalOcean's NVIDIA L40S is $1.57 an hour (about $1,146 a month; checked 25 October 2026) and is billed for about 730 hours if you keep it all month.

Monthly costs for hourly prices assume the server runs all month (730 hours), or less where a provider caps the monthly bill.

Who this is for

• a business or team that wants an AI to answer questions from its own policies, manuals, product pages or past support tickets
• a developer asked to "train ChatGPT on our data" who has to explain the options and what each one costs
• anyone weighing a fine-tuning project against a monthly chatbot plan

Not for you if you already know you want a chatbot service and need to choose one: see our comparison of chatbots for websites.

Key numbers

• 100 or more examples Google's rule of thumb for the size of a tuning dataset Official page, 25 Oct 2026
• 6 Jan 2027 the day OpenAI's remaining active fine-tuning customers can no longer start new jobs Official page, 25 Oct 2026
• 730 hours in an average month: 8,760 in a year divided by 12 Our calculation, 25 Oct 2026
• 1.46 million GPU-hours the GPU time Meta used to train Llama 3.1 8B from scratch, on H100 80 GB cards Official page, 25 Oct 2026
• More than a third of the original model's training computation: the EU's indicative threshold for becoming a modified model's provider Official page, 25 Oct 2026

What RAG and fine-tuning each do, in plain words

Both options start from a large language model (LLM) that someone else has already trained, so neither means building a custom model from scratch. They differ in what they change.

RAG (retrieval-augmented generation) means the AI looks up passages in your documents and answers from them; nothing is trained. The term comes from a 2020 research paper by Patrick Lewis and colleagues. A document chatbot, also called a knowledge-base chatbot, is a ready-made RAG service: it splits your files into short passages and indexes them. For each question, it fetches the best-matching passages and hands them to the model together with the question. Update a document and, once the service has indexed it again, the next answer uses the new version.

1. Your files and web pages (added once, indexed again when they change): connects to Search index (split and stored).
2. Search index (your documents cut into short passages): connects to Best-matching passages (search).
3. Best-matching passages (found for each new question): connects to AI model (added to the prompt).
4. AI model (reads the question with the passages): connects to Answer (writes).
5. Answer (can point to the passages it used)

Fine-tuning trains an existing model further on examples you supply: questions paired with the answers you want, written the way you want them. Microsoft's fine-tuning guide describes it as changing the model's weights, the numbers it has learned, instead of adding examples to every prompt. The tuned model then writes in your format or tone without being told each time. It does not look anything up, so what it learned stays fixed until you train it again.

That difference matters most when your information changes. Microsoft's guide says fine-tuning does not take the place of retrieval for up-to-date facts. It also warns that a tuned model may need retraining as your data changes. A price list, an opening-hours page or a returns policy belongs in documents that a chatbot can search.

Which one fits you: four questions

Answer the questions in order. Each answer sends you to a later question or to the part of this page that covers your route.

The two options can also work together: Microsoft lists better use of retrieved passages among the reasons to fine-tune. We would still start with a chatbot, because it shows quickly whether your problem is missing knowledge or the way the model writes.

What each route costs per month

Keeping a tuned model online on its own rented GPU costs far more each month than a document chatbot plan. The reason is the GPU, which is billed by the hour whether or not anyone is using it. The chart sets two chatbot plans against one GPU rented for a whole month. The plans come from DocsBot and from CustomGPT.ai, which is made by CustomGPT Inc. in Massachusetts, not by OpenAI.

The numbers in this chart

Example	Part	Cost
Document chatbot: CustomGPT.ai Standard	Monthly plan	$99.00
Document chatbot: CustomGPT.ai Standard	Total	$99.00
Document chatbot: DocsBot Standard	Monthly plan	$149.00
Document chatbot: DocsBot Standard	Total	$149.00
Tuned model on one DigitalOcean L40S GPU	GPU rented all month	$1,146.10
Tuned model on one DigitalOcean L40S GPU	Total	$1,146.10

Not in the chart: the cost of the fine-tuning run itself, which depends on the model, the data and the GPU hours it takes.

Two costs are missing from these figures. The first is the training run itself: GPU time to train, plus the repeat runs that Microsoft tells its customers to budget for. Our guide to fine-tuning costs covers it in detail. The second is your own time, spent preparing examples and testing the result.

Document chatbots: what a monthly plan gives you

A document chatbot service does the indexing, the searching and the hosting for you. You add your files or point it at your website, then add the chat window to your site or connect it to other tools. DocsBot's home page describes a RAG pipeline that fetches passages, sorts them again by relevance and passes the best ones to the model. CustomGPT.ai says every answer links to the source it came from.

Compare the limits as well as the price. CustomGPT.ai's Standard plan includes 1,000 queries a month. DocsBot counts AI credits instead, and its pricing page says a bot stops answering once the month's credits are used up. Neither pricing page mentions VAT.

If your bot is one of OpenAI's custom GPTs today, our guide on where to move a custom GPT covers the choices.

Your data: training and where it is stored

CustomGPT.ai's pricing page promises that your data is not used for training. DocsBot's trust centre says it does not use your data to train any models. It adds that it does not opt in to the model providers' data sharing. DocsBot's plan table lists its data processing agreement from the Business plan up.

CustomGPT.ai says it runs in Amazon Web Services' US East region. DocsBot's agreement allows processing in the United States. Its pricing page mentions US and EU storage, but its plan table lists data residency options only on the Enterprise plan. Ask DocsBot before you rely on EU storage. If the location of your data matters, our guide to AI hosting in Europe compares the providers we cover.

Your own private assistant on a rented server

A middle route is to run an open model, with software that searches your documents, on a server you rent. Your files then stay on a machine you control, and the monthly bill is the rent for the server, however many questions people ask. In return, you take on the set-up, the updates and the security.

The server you need depends on the model. By our calculation, a model with 8 billion parameters needs about 16 GB of GPU memory at 16-bit precision, two bytes for each parameter. That covers the weights alone; conversations need extra memory on top. Three of our guides go further:

• How much GPU memory an LLM needs: the figures, model by model
• Where to run an open-source LLM: the kinds of server compared
• A private ChatGPT for your team: a server sized for 5 to 20 people

Fine-tuning: what it takes

Fine-tuning earns its cost when you can show the model what you want through many examples. Microsoft's guide lists several cases where it helps: answers in a consistent tone, format or structure; better use of tools; and shorter prompts. Fine-tuning can also turn a small model into a specialist for one job that would cost more on a larger model. Google's tuning guide for its Gemini models recommends finding the best prompt first and tuning only if that is not enough.

Before you start, you will need:

• a set of good examples: Google suggests thinking in terms of 100 or more
• a separate set of test questions the model never trains on, so you can compare it with the original (Microsoft's advice)
• GPU time for the training run, plus a budget for repeat runs, since Microsoft warns that results are not guaranteed
• a GPU, or a service, to run the tuned model whenever people use it
• time to train again when your examples change or the base model (the one you started from) is retired

Google describes a lighter way than retraining the whole model and calls it parameter-efficient or adapter tuning. One such method, LoRA (low-rank adaptation), keeps the original weights frozen and trains a small add-on next to them. Its 2021 paper reports 10,000 times fewer trainable parameters and a third of the GPU memory, compared with fully fine-tuning GPT-3 175B.

QLoRA also compresses the frozen model to 4-bit numbers. In their 2023 paper, its authors fine-tuned a 65-billion-parameter model on a single 48 GB GPU. A step-by-step guide will show how to fine-tune an LLM with Unsloth on a rented GPU, with both methods.

1. Your examples (100 or more questions with ideal answers): connects to Training run (uploaded).
2. Training run (on a rented GPU, repeated until it passes your tests): connects to LoRA add-on (produces).
3. LoRA add-on (a small file trained on top of the base model): connects to GPU server (loaded).
4. GPU server (runs the base model with the add-on): connects to Answers (replies).
5. Answers (in your format and tone)

The tuned model then has to run somewhere, every hour that people use it. By the same calculation, a 48 GB card such as the L40S leaves room to spare for an 8-billion-parameter model at 16-bit. Larger models need more memory or compression. The box below lists rented GPUs we cover, by the hour and by the month.

A training run costs far less than keeping the tuned model online. Our guide to fine-tuning costs works out the GPU-hours of published runs and prices them from the same weekly records.

OpenAI's fine-tuning platform is closing

If you hoped to fine-tune OpenAI's models on OpenAI's own platform, that route is closing. OpenAI's fine-tuning guide says the company is winding down the platform, which no longer accepts new users. OpenAI's deprecations page adds two dates. Since 2 July 2026, organisations that have not used a tuned model in the last 60 days cannot start new jobs. On 6 January 2027, the remaining active customers lose that ability too.

Models already tuned keep answering until their base models are retired. Microsoft Foundry, Microsoft's platform for AI models, still lists fine-tuning for several OpenAI models, such as GPT-4.1.

Training from scratch: why almost nobody should

Training from scratch means teaching a model language itself, from trillions of words, before it can do anything useful. Meta's model card for Llama 3.1 shows the scale. It lists about 15 trillion tokens (pieces of words) of training data. Training all three sizes took 39.3 million GPU-hours on H100 80 GB cards; one GPU-hour is one card working for one hour.

The practical lesson: start from an open model that someone else has trained, and adapt it with retrieval, fine-tuning or both.

EU rules: few fine-tuning projects make you a model provider

Under the EU's guidelines of 19 November 2025, fine-tuning a general-purpose AI model makes you a provider only if the change is significant. Since 2 August 2025, the EU's AI Act has placed duties on businesses that provide such models. This is information, not legal advice.

The guidelines give an indicative threshold: a modification that uses more than a third of the computation spent on training the original model. If that figure is unknown and cannot be estimated, a third of a fixed amount of computation is used instead. For most models, a business that crosses the threshold has duties for its modification only, and the guidelines expect few modifications to reach it today. Changing one of the most capable models, which the AI Act treats as carrying systemic risk, brings the fuller duties set for those models.

For a sense of scale, a third of Llama 3.1 8B's training time is about 487,000 GPU-hours, roughly 56 years of one card running non-stop. That is our calculation and only a rough guide, since the amount of computation also depends on how hard each card works.

Mistakes to avoid

• Fine-tuning a model to teach it facts

  Instead: Keep facts in documents a chatbot searches; a tuned model misses changes until you retrain it.

• Fine-tuning before trying better prompts

  Instead: Find the best prompt you can first, as Google's tuning guide recommends, and measure the result.

• Judging a tuned model on its own training examples

  Instead: Test it on questions it has never seen, side by side with the original model.

• Powering off a rented GPU and assuming the bill has stopped

  Instead: Read the provider's billing rules: DigitalOcean bills a powered-off GPU Droplet. Delete servers you no longer need, and set a spending limit.

• Choosing a chatbot plan on price alone

  Instead: Check the monthly cap on questions or credits; DocsBot bots stop answering when credits run out.

Other questions

Can I fine-tune ChatGPT on my own data?

Not on OpenAI's own platform as a new customer, because it no longer accepts new users. Active existing customers lose the ability to start new jobs there on 6 January 2027. Microsoft Foundry still lists fine-tuning for some OpenAI models, such as GPT-4.1. For an assistant that knows your documents, a document chatbot does the job without training anything.

Can I use RAG and fine-tuning together?

Yes. A tuned model can still read passages fetched from your documents, and Microsoft names making better use of them as one reason to fine-tune. Start with retrieval alone, and add tuning only if the answers hold the right facts in the wrong form.

How many examples do I need to fine-tune a model?

Google's tuning guide suggests thinking in terms of 100 examples or more. Quality counts as much as the number. Microsoft asks for accurate, consistent examples that cover the cases you care about, plus a separate set for testing.

Does a document chatbot train the AI on my files?

No. Your files are indexed and searched, and the model reads only the passages that match each question. Even so, check each service's data terms. CustomGPT.ai and DocsBot both say they do not train on your data.

How we priced

Every price on this page comes from its provider's own pages, shows the date we checked it, and refreshes from our weekly price records. Prices stay in the currency each provider uses and are never converted, so euro and dollar figures cannot be compared directly. How we check prices explains the weekly checks.

Monthly figures for hourly GPUs use 730 hours, which is 8,760 hours in a year divided by 12. That is an average month for a server you keep all the time. The exception is UpCloud, whose GPU page says it caps a month's charge at 28 days of use. Whether VAT is included depends on the provider. Each price in the boxes says what its page states, and one without a VAT note excludes VAT.

Our weekly price index will follow how GPU prices move, and the finder on our homepage suggests a set-up for other AI jobs.

Sources and test notes

• Google Cloud: Introduction to tuning (Gemini Enterprise Agent Platform), last updated 22 Oct 2026, read 25 Oct 2026
• Microsoft Learn: Fine-tuning in Microsoft Foundry, read 25 Oct 2026
• OpenAI: guide to model optimisation and fine-tuning, read 25 Oct 2026
• OpenAI: Deprecations, update to self-serve fine-tuning, read 25 Oct 2026
• Meta: Llama 3.1 model card, read 25 Oct 2026
• EU guidelines on the duties of general-purpose AI model providers, C(2025) 7719 of 19 Nov 2025, sections 3.2.1 and 3.2.2, read 25 Oct 2026
• EU questions and answers about the AI Act's rules for general-purpose models, read 25 Oct 2026
• Hu and others: LoRA, Low-Rank Adaptation of Large Language Models (2021), read 25 Oct 2026
• Dettmers and others: QLoRA, Efficient Finetuning of Quantized LLMs (2023), read 25 Oct 2026
• Lewis and others: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020), read 25 Oct 2026
• CustomGPT.ai: home page, read 25 Oct 2026
• CustomGPT.ai: about us, read 25 Oct 2026
• CustomGPT.ai: pricing, read 25 Oct 2026
• CustomGPT.ai: security and trust, read 25 Oct 2026
• DocsBot: home page, read 25 Oct 2026
• DocsBot: pricing, read 25 Oct 2026
• DocsBot: data processing agreement, read 25 Oct 2026
• DocsBot: terms of service, read 25 Oct 2026
• DigitalOcean: GPU Droplet pricing, read 25 Oct 2026
• DigitalOcean: Droplet pricing documentation, read 25 Oct 2026
• DigitalOcean: sign-up credit, read 25 Oct 2026
• UpCloud: GPU servers, read 25 Oct 2026
• UpCloud: pricing, read 25 Oct 2026
• UpCloud: free trial documentation, read 25 Oct 2026
• GPU Mart: RTX 4090 GPU hosting, read 25 Oct 2026
• Database Mart: billing questions, read 25 Oct 2026
• Database Mart: sales questions, read 25 Oct 2026
• Liquid Web: GPU server configurator, NVIDIA L4, read 25 Oct 2026
• DocsBot trust centre (training questions), read 27 Oct 2026

No test was run for this page. Every price comes from our price records, each checked on the provider's own page; the date of each check is shown with the price.

Guides in this section

• 
  Custom GPT retirement

  OpenAI is retiring custom GPTs on 11 December 2026. What moves to a ChatGPT plugin, what you lose, and two document chatbot services to use instead.

• 
  Fine-tune with Unsloth

  Fine-tune an open model with Unsloth on one rented GPU: the memory each size needs, the dataset format, a short training script and export to Ollama.

• 
  Fine-tuning costs

  Published LoRA and QLoRA runs took hours on one GPU; a tuned model online runs 730 hours a month. Run times, memory by method and live GPU prices.

• 
  Website chatbots

  CustomGPT.ai, DocsBot or AnythingLLM on your own server: limits, cited sources, refreshing, data storage and training terms, with live prices.

See every guide