From AI prototype to production system: what changes between test and operation

Why do so many AI projects stall after the test?

In 2024, Gartner predicted that at least 30 % of generative AI projects would be abandoned after the proof of concept by the end of 2025. Gartner cites poor data quality, inadequate risk controls, escalating costs and unclear business value as the reasons. Two of these typically arise only at the step into operation: risk controls that nobody needed in the test, and costs that nobody saw in the test.

A prototype answers the question “Does it work?”. A production system has to answer the question “Does it work every day, for everyone, securely and verifiably?”. How to set up the test itself is described in the guide Proof of concept for AI.

What distinguishes a prototype from a production system?

	Prototype	Production system
Data	sample data	real personal, patient or citizen data
People	one team that knows the technology	everyone authorised, without prior knowledge
Sign-in	local accounts or none	the organisation’s identity management, second factor
Permissions	everyone sees everything	each person sees only what they are allowed to, including in connected systems
Logs	none	who did what and when, tamper-proof
Load	few requests	peak times with many simultaneous questions
Outage	annoying	fallback procedure needed
Documentation	none	record of processing activities, TOMs, depending on the use a DPIA, agreement with staff representatives
Responsibility	the person who built it	a designated unit with a deputy

What must be settled before the start?

• Real data: Which data goes in, on what legal basis? Record of processing activities (Art. 30 GDPR), security measures (Art. 32) and, for health or social data, usually a data protection impact assessment (Art. 35).
• Permissions: Take permissions from identity management instead of maintaining them twice. For connected systems: the AI may never do more than the person asking.
• Staff representatives: An AI that logs inputs is a technical facility within the meaning of § 87 para. 1 no. 6 BetrVG (German Works Constitution Act); in the public sector, the staff representation acts apply. More in the guide Model service agreement for AI.
• Security: A risk analysis for the specific use, as the BSI (German Federal Office for Information Security) recommends, with a view to manipulated instructions in documents and to answers that disclose data (OWASP Top 10 for language models).
• AI Act: Deployers must take measures to ensure the AI literacy of their staff (Art. 4). Additional deployer obligations apply to high-risk uses (Art. 26).

What must operations be able to take over?

• Load test: How many questions run simultaneously at peak times, and how long do people then wait for the answer to begin? You do not see this in the test.
• Backup and restore: What is backed up, how often, and has a restore been rehearsed at least once?
• Updates and model changes: Who installs security updates, and who tests a new model with the most important tasks before it goes live? See Testing AI after a model change.
• Outage: What do staff do when the AI does not answer? See the guide Planning emergency operation.
• Contact persons: Who helps with questions and faults, with a deputy? If only one person knows the system, that is a risk; see Bus factor.
• Training: Newcomers learn the limits and rules of the AI before they work with real data.

When is the prototype the wrong foundation?

A prototype is often built for speed: a model on one computer, an interface without sign-in, documents in a folder. If permissions, logs and operation are missing, the rebuild is sometimes bigger than a fresh start on a different foundation. Check honestly what remains from the test: usually the insights into tasks, instructions and test cases, not necessarily the technology. How a test setup with Ollama and Open WebUI grows is covered in Scaling Ollama and Open WebUI.

How does this work with basebox?

In the cloud, you start with 1 million tokens free of charge and without a time limit, with all features. If you book the licence, apps, knowledge bases and settings are retained. Your data is held in a separate environment for your organisation, in our data centre in Germany. Sign-in through your identity management, permissions, logs, updates and support are included in the licence; the DPA, TOMs and templates for the data protection impact assessment can be downloaded under Security. A later switch from the cloud to a server is possible at any time, at a flat rate of €4,200.

Not legal advice: This guide gives an overview, as of 11 October 2026. Which documents and approvals your organisation needs depends on the use; agree this with your data protection officer.