Scaling Ollama and Open WebUI: what changes when many people ask questions
Ollama and Open WebUI do not grow on their own. Out of the box, Ollama handles one request per model at a time; for many people, organisations usually use software such as vLLM, which batches requests. For several instances, Open WebUI needs PostgreSQL instead of SQLite, Redis, a separate vector database and shared file storage. A single installation thus becomes a distributed system of five or more services that someone has to run permanently.
Why does the test hit its limits?
A test with Ollama and Open WebUI often runs on a single computer: Ollama runs the model, and Open WebUI stores accounts, chats and documents in built-in SQLite databases. For ten people, this works well. As the number grows, two bottlenecks appear: the model, which processes requests one after another, and the interface, which is built as a single process with a local database.
What limits Ollama?
According to its documentation, Ollama handles one request per model at a time out of the box (OLLAMA_NUM_PARALLEL); further requests wait in a queue of up to 512. More parallelism can be configured. The memory needed for it grows with the number of simultaneous requests times the context length: four requests with 8,000 tokens of context each need as much memory as one with 32,000.
In August 2025, Red Hat compared Ollama and vLLM on the same GPU with 1 to 256 simultaneous people. vLLM reached a peak of 793 tokens per second, Ollama 41; even with 32 parallel requests, Ollama fell behind. Red Hat positions Ollama for individuals and vLLM for many simultaneous requests. The test covered a small model on one card; other measurements show smaller gaps.
What changes with vLLM?
vLLM is open-source software under the Apache 2.0 licence that continuously batches requests and manages the memory for conversations in blocks. In return, it is more demanding: it runs on Linux with GPUs, and the model, memory allocation, context length and format for tool calls must be configured to match. For several GPUs or servers, splitting the model and load balancing come on top. How many questions a server can handle at the same time depends on the model and the length of the conversations; only a load test with your own tasks gives a reliable figure.
What does Open WebUI need for many people?
For several processes or instances, the Open WebUI documentation describes:
| Component | In the test | For many people |
|---|---|---|
| Database | SQLite, built in | PostgreSQL, external; SQLite is not supported for several instances |
| Shared state | none | Redis, for settings, sessions and websockets |
| Vector database for document search | Chroma, built in | separate service, such as PGVector, Qdrant or Milvus; built-in Chroma is not safe for several processes |
| Files | local directory | shared storage or object storage |
| Text from documents | built-in extraction | separate service such as Tika or Docling |
| Secret key | per installation | identical on all instances |
For small teams of under 50 people, the documentation recommends PostgreSQL and Redis; from several processes, several instances or more than 1,000 people, they are required together with an external vector database.
What does this mean for operations?
Two programs become five to eight services: model server, interface, PostgreSQL, Redis, vector database, file storage, a service for extracting text from documents, and in front of them a load balancer with a certificate. Each needs updates, monitoring, backups and a plan for outages. On top of this come load tests whenever a new model or a new version arrives. The guide Security updates for a self-built AI stack describes what a process for security updates looks like; Run AI yourself or buy a platform calculates how much staff time this ties up.
How does this work with basebox?
basebox uses vLLM for the AI model and distributes requests across the GPUs automatically. Depending on the class, a server answers 100, 150 or more than 200 questions at the same moment, with the answer starting in under 2 seconds; further requests are distributed in turn. For more than 2,000 people, basebox runs on larger hardware that you procure yourself; apps, permissions and connections stay the same. In the cloud, we take care of operations and updates. If you already have hardware with GPUs, we check free of charge beforehand whether it is sufficient.
Note: Information on Ollama, vLLM and Open WebUI is based on their documentation and the test mentioned, as of 11 October 2026. The software changes quickly; check the current documentation for your version.
Frequently asked questions
Is it enough to raise OLLAMA_NUM_PARALLEL?
It helps, but only to a point. Memory requirements grow with the number of simultaneous requests times the context length. In the Red Hat test, Ollama gained less than vLLM even with 32 parallel requests. For a whole organisation, software built for many simultaneous requests is the norm.
From how many people does Open WebUI need external services?
The documentation recommends PostgreSQL and Redis even for small teams of under 50 people. From several processes, several instances or more than 1,000 people, it lists PostgreSQL, Redis and an external vector database as required.
How many people ask questions at the same time?
Far fewer than the number of accounts created. Someone who works with AI for an hour keeps the server busy for only a few minutes. Measure during the test how many requests run at the same time at peak hours, and plan with a reserve.
Sources
This article was created with the help of AI, based on the sources listed. Responsible for the content: basebox GmbH. Not legal advice, no liability. Research as of 11 October 2026.