A private ChatGPT-style assistant for your team: the server it needs for 5 to 20 people

A private chat assistant for a team has three parts: a chat app such as Open WebUI, an open model and a server. Your prompts and documents then stay on a machine you control.

A server without a GPU suits one or two people trying it out. For a team of 5 to 20, rent a GPU. In a published test, one RTX 4090 kept 64 requests at once writing about 29 tokens a second each.

The figures come from published tests and from Open WebUI's, AnythingLLM's and Docker's own documentation. We read them all on 27 October 2026.

What the assistant is made of

The chat app gives each person a login, keeps the chat history and lets people ask questions about uploaded documents. Open WebUI's documentation says it has no models of its own; it connects to an engine.

The serving engine runs the model on the card. Ollama is the simplest; vLLM handles many people at once far better, as our comparison of serving engines shows.

The model decides the quality of the answers and how big a card you need. Our guide to running an open model compares the routes, and our GPU memory guide sizes the card.

Open WebUI or AnythingLLM

Open WebUI's licence lets you change its name and logo only for up to 50 users in 30 days, or with permission. A team of 5 to 20 can rename it; a larger one keeps the branding.

AnythingLLM's README says it collects anonymous usage data unless you set DISABLE_TELEMETRY or turn it off under Privacy in the app.

Which server for how many people

The question is how many people ask at the same moment, not how many have a login. Size the server for the busiest moment, such as everyone trying it after a team meeting.

On a processor, a 300-token answer takes about 40 seconds at 7.7 tokens a second, by our calculation. With Ollama's default of one request at a time, the next person waits until it ends.

A 2019 review of reading studies put average silent reading at 238 words a minute, about four words a second. At 29 tokens a second, the answer arrives far faster than anyone reads it.

Our speed figures by card and our Qwen3.8-27B guide cover larger models.

Set it up step by step

These steps put Open WebUI and Ollama on one rented GPU server running Ubuntu, with Docker and NVIDIA's driver installed. They follow Open WebUI's quick start.

1. Start Open WebUI with Ollama inside

Open WebUI offers one container with Ollama built in. Bind it to the server itself, so it is reachable only through the HTTPS proxy in step 3:

SECRET=$(openssl rand -hex 32)
docker run -d -p 127.0.0.1:3000:8080 --gpus=all \
  -v ollama:/root/.ollama -v open-webui:/app/backend/data \
  -e WEBUI_SECRET_KEY=$SECRET \
  --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama
echo "$SECRET"   # keep it: updates need the same key

Docker's documentation warns that published ports are open to the world by default. Ubuntu's ufw firewall does not stop them either, so the 127.0.0.1: part matters.

2. Download a model

docker exec open-webui ollama pull qwen2.5:7b

For a stronger model on a 24 or 32 GB card, our Qwen3.8-27B guide explains the files and the memory they need.

3. Put HTTPS in front

A reverse proxy such as Caddy adds HTTPS with a free certificate. Point a domain at the server, install Caddy and give it this Caddyfile:

chat.example.com {
    reverse_proxy 127.0.0.1:3000
}

4. Create the accounts

1. Open the address and create the first account. Open WebUI makes it the administrator.
2. Sign-up then switches itself off. To let colleagues join, turn on New Sign Ups under Settings, Admin, Authentication.
3. New accounts wait as Pending until you approve them in the Admin Panel.

5. Keep it private and up to date

• Turn off the calls home Open WebUI lists in its quick start. Add -e ENABLE_VERSION_UPDATE_CHECK=false and -e RAG_EMBEDDING_MODEL_AUTO_UPDATE=false to the command in step 1.
• Delete the default OpenAI connection under Settings, Admin, Connections. Use ENABLE_OPENAI_API=false only if you will never add an OpenAI-style engine such as vLLM.
• Back up the open-webui volume before each update, then pull the new image and start it again with the same key.
• Read Open WebUI's hardening guide before you open it to people outside your team.

To serve many people at once, run vLLM next to it and add vLLM as an OpenAI-compatible connection under Settings, Admin, Connections. Our serving engine guide has the vLLM steps.

Chatting with your team's documents

Both apps let people upload files and ask questions about them. Open WebUI keeps them in a chat or in a shared knowledge base; AnythingLLM groups them in workspaces.

The documents stay on your server, but the model reads only the parts it finds relevant. Our guide to fine-tuning or a document chatbot explains when this beats training a model.

Keeping your team's data private

Running the assistant yourself keeps prompts off outside services, as long as no connection points to one. In Open WebUI, delete the default OpenAI connection unless you mean to use it.

Where the server stands still matters for personal data. Our guide to AI hosting in Europe lists which providers keep data in the EU.

How we checked

We read Open WebUI's and AnythingLLM's documentation, licences and GitHub releases, and Docker's documentation, on 27 October 2026. The load figures come from the tests named above.

Our own measurements will be added here, with their dates, when our test bench runs them.

Back to Where to run an open model