Criteria catalogue for AI platforms: how to weight and evaluate providers

How does a criteria catalogue differ from a checklist?

A checklist collects questions that you ask every provider in the first meeting; it is covered in the guide “Which questions should you ask an AI provider in the first meeting?”. A criteria catalogue turns these into a decision: it sets out what a bid must meet, how much each point counts and how points are awarded. Only then can bids be compared and the decision be justified to bidders, auditors and committees.

A catalogue has three levels:

1. Exclusion criteria: whatever is missing is excluded. No points.
2. Evaluation criteria: weighted, with a scoring scale.
3. Price: over the whole term, according to a formula announced in advance.

Which exclusion criteria suit an AI platform?

Exclusion criteria should be few, unambiguous and verifiable requirements:

• Processing of the data and of the AI model in the EU or on your own premises
• Data processing agreement under Art. 28 GDPR with a list of sub-processors
• No training with your data, contractually guaranteed
• Permissions per person from your identity management
• A log of who used the AI and when
• Return of your data at the end of the contract
• For health data in the cloud: proof under § 393 SGB V (German Social Code, Book V: a C5 attestation or equivalent proof)
• For professional or official secrets: agreement under § 203 StGB (German Criminal Code)

Whatever is listed here no longer earns points. Otherwise the same requirement counts twice.

How do you weight the evaluation criteria?

The weighting follows your goals. An example for a public administration or a hospital with 300 to 1,000 people:

Criterion	Weight	How it is measured
Price over five years	35 %	Total cost according to the price sheet, formula see below
Quality of answers in the test installation	25 %	Evaluation form per task, several evaluators
Source references and traceability	8 %	Test installation: does every statement lead to the passage in the text?
Permissions and sharing of applications	8 %	Test installation: does the AI see only what the person may see?
Connection of your systems	8 %	Concept: which systems today, which planned, at what price
Operation and data location	6 %	Concept: cloud, operation on your own premises, without an internet connection
Rollout and training	5 %	Concept: schedule, training in AI literacy
Support and updates	5 %	Bid: hours, channels, update cycle

The figures are a suggestion, not a requirement. What matters is that you set the criteria and weighting in advance and announce them (§ 127 para. 5 GWB, the German Act against Restraints of Competition, and § 58 VgV, the German Procurement Ordinance; in Austria § 91 BVergG 2018, in Switzerland Art. 29 para. 3 BöB).

How do you award points in a traceable way?

Every criterion needs a scale with described levels. An example for “source references”:

Points	Description
0	No sources
1	Source only as document name
3	Source with page or section
5	Every statement leads with one click to the quoted passage in the text

Have at least two people evaluate independently and record any differences in writing. The German federal government’s UfAB 2018 (guidance for tendering and evaluating IT services) describes evaluation methods and provides an evaluation matrix as a spreadsheet.

How do you evaluate the price?

Compare total costs over the same term, ideally five years. A price sheet should include licence, hardware and service contract, operation, rollout, training and connectors. Also ask about limits on usage and what it costs when they are reached.

A common formula: the cheapest bid receives full points, every other bid a proportional share (cheapest price divided by its own price, times the maximum number of points). Whether you use this formula or a method from the UfAB, you decide in advance. How to determine the costs yourself is shown in the guide Run it yourself or buy a platform.

What belongs in the test installation?

The test installation is the heart of the evaluation: all bidders solve the same tasks with the same sample documents in the same time. Good tasks come from your day-to-day work, for example a question about a service instruction, a draft of an administrative decision or a letter from bullet points, or a summary of a long document. Also check whether a person without permissions for a document gets an answer from it. Structure and procedure are described in the guide Proof of concept for AI.

Not legal advice: The catalogue is a suggestion; your procurement office checks criteria, weighting and formula. Research as of: 11 October 2026.

How does this work with basebox?

Answers to the exclusion criteria and many evaluation questions are arranged according to the BSI C5 catalogue on Answers for your security review; you can download the data processing agreement, TOMs, the list of sub-processors and the agreement under § 203 StGB on Security. For the price sheet, the calculator computes list prices for cloud and server over five years. The tender generator creates mandatory and optional criteria as a template, without naming basebox.