TCO comparison: run AI yourself or buy a platform?
Over five years, the deciding cost of running AI yourself is not the hardware but the staff. Servers, power and software cost about the same either way. Building and maintaining permissions, connections, logs, security updates and proof yourself needs your own specialists; in the German public sector, one post costs roughly €90,000 a year including employer contributions. A platform you buy pays off as soon as running it yourself ties up more staff than the licence costs.
What belongs in the total cost of ownership (TCO) of AI?
TCO (total cost of ownership) means all costs over the period of use, not just the purchase price. For AI in your own organisation, five years is usual, because the server and service contract run that long. These items arise whether you build or buy; the difference lies in who carries them.
| Cost item | Running it yourself | Platform you buy |
|---|---|---|
| Hardware (server with GPUs) | you buy it | you buy it (or cloud without hardware) |
| Hardware service, power, rack space | you carry them | you carry them; rack space and power are included when it runs in the provider’s data centre |
| Software for the AI model | open source or a licence per GPU | in the licence |
| Sign-in, permissions, logs | build and maintain yourself | in the licence |
| Connection to specialist systems | build and maintain yourself | connectors from the library, new ones at a fixed price |
| Security updates, new models | check and install yourself | by update |
| Proof (DPA, TOMs, security concept, certificates) | produce yourself | from the provider, supplemented for your own obligations |
| Support for staff | your own IT | provider and your own IT |
What does the hardware cost?
As an order of magnitude, take the list prices of our hardware partner FAST LTA for ready-made servers: roughly €100,000 to €200,000 net, depending on specification (as of October 2026).
| Server | GPUs | Guide value |
|---|---|---|
| Class 1 | 3× NVIDIA RTX PRO 6000 (96 GB each) | about 200 to 500 people |
| Class 2 | 2× NVIDIA H200 (141 GB each) | about 500 to 1,000 people |
| Class 3 | 2× H200 + 1× RTX PRO 6000 | about 1,000 to 2,000 people |
On top of this comes a service contract for repair and replacement of defective parts; for these servers it is 9 % of the purchase price per year. Power costs around €1,500 to €2,500 a year with normal use, and at most around €7,000 under full load around the clock (€0.25 per kWh, including cooling). If you procure the hardware yourself, you end up in the same order of magnitude: the GPUs make up the largest part of the price.
What do the staff cost?
This is the item that decides the calculation. AI for a whole organisation is more than a model on a server. For staff to be allowed to use it with sensitive data, you need:
- Sign-in and permissions from your user directory, so that each person sees only what they are allowed to see
- Connection to document stores and specialist systems, with the permissions of the person asking
- Logs of who asked what and when, for data protection and audit
- Security updates for the operating system, inference software, interface and libraries
- New models to check, test and install
- Proof for data protection, information security and staff representatives
- Support and training for staff
On salary: an IT specialist in pay group 13 of the TVöD (the collective agreement for German municipal employees) earns €5,709.87 a month at level 3 since May 2026, which is €68,518 in table pay a year. With the annual bonus, social security and supplementary pension, employer costs are roughly €90,000 a year (estimate). In the private sector, specialists for AI infrastructure are often paid more.
How many posts running AI yourself ties up depends on its scope. One person looks after a test installation for one department alongside other work. AI for the whole organisation with permissions, several connected systems and proof needs staff permanently, because models, software and the security situation change all the time. The example calculation below assumes one and a half posts; check the figure for your organisation.
Example calculation over five years: 500 to 1,000 people
Same hardware (Class 2), server in your own data centre, 750 people. The staff figures are assumptions; insert your own. Net, converted over five years.
- The same in both cases: hardware spread over five years, the service contract (9 % of the purchase price per year) and power (around €1,600 a year).
- Running it yourself: plus staff, assumed at 1.5 posts at €90,000 each, so €135,000 a year.
- basebox on the server: plus the platform licence and staff for a quarter of a post, €22,500 a year.
Per person per year, that is around €246 when running it yourself and around €160 with basebox on the server. The staff make the difference.
The licence for Class 2 corresponds to about half a post. The rule of thumb that follows: if running it yourself ties up more than an additional half post, the platform you buy is cheaper. If you also want to hand over the running of the server, basebox charges €1,200 per month when the server is at your site. You can calculate your price in two minutes in the calculator.
Not included in this calculation: software licences when running it yourself (open source costs nothing; NVIDIA charges USD 4,500 per GPU per year for its Enterprise package, list price), costs of security reviews and certificates, and the time until launch, during which a self-built system brings no benefit yet.
What is often forgotten
- Lead time: a self-built system with permissions and connections is a software project lasting months. A finished platform runs in the cloud immediately, and on existing hardware within a few days.
- Dependence on individuals: if only one person knows the self-built system, holidays, illness and resignation become an operational risk.
- Follow-up effort: with a self-built system, every new feature, every new model and every piece of proof is commissioned or built separately.
- Hardware is not the difference: it is needed in both cases. If you already have GPUs, you save on them in both cases.
How does this work with basebox?
basebox runs in our cloud, on a ready-made server or on hardware of your choice. Features and security are the same in all operating models. The licence includes all features, apps, connectors from our library, support and updates. If you already have your own AI platform, we discuss the switch with you; you can keep using existing hardware.
Note: The range of hardware prices is based on list prices of our hardware partner FAST LTA. Staff costs and staffing needs are assumptions for illustration, not a commitment.
Frequently asked questions
Is open-source AI software not free of charge?
The licence is, running it is not. Software such as vLLM or Open WebUI costs nothing, but someone has to set it up, secure it, update it and connect it to the user directory and specialist systems. This effort is the largest item in the total cost.
Can we keep using hardware we already have?
Yes. If a server with GPUs is already in place, the largest one-off item disappears, whether you run AI yourself or buy a platform. We then license basebox according to your GPUs; we check beforehand, free of charge, whether the hardware is sufficient.
When does running AI yourself make sense?
When a team for AI infrastructure is there anyway, for example in research; when very specific requirements are not met by any finished product; or when the AI serves only a small group of specialists who do not need permissions and proof to the same extent as a whole organisation.
What happens to our platform if basebox no longer exists?
If basebox becomes insolvent, the code will be published as open source so that customers can keep working. With the server, the platform runs on hardware you own anyway.
Sources
This article was created with the help of AI, based on the sources listed. Responsible for the content: basebox GmbH. Not legal advice, no liability. Research as of 30 October 2026.