Stop rules for AI: set your exit criteria before go-live

Stop rules are the criteria for pausing, adjusting or stopping an AI application, set before the application goes live. They answer three questions: at what threshold do we step in, who decides, and how quickly can we technically switch the application off?

Most organisations have a decision to start an AI project. Almost none have a decision to stop one. The result is predictable: an application that performs poorly keeps running. Money has been invested, people have got used to it, and nobody ever agreed when enough is enough.

Stop rules are the third part of our AI governance framework, after value and ethical boundaries. Read the full framework in AI governance for boards and supervisory boards: value, ethics and stop rules.

Why before go-live?

Three reasons.

Afterwards, everyone is biased. Once an application is running, someone has attached their name to it. A threshold set at that point tends, without anyone intending it, to land just below where the application already is.

A threshold forces measurement. If you agree in advance that fewer than 90% correct answers is a reason to pause, you also have to arrange for correctness to be measured. Without a stop rule, nobody measures it.

In an incident, there is no time for discussion. If personal data ends up in the wrong place, the application must be switched off immediately. If you then still have to work out who is allowed to decide, you lose hours.

Two kinds of stop rules

Not every threshold calls for the same consequence. Distinguish between evaluative and absolute stop rules.

Evaluative stop rules: pause, fix or reconsider

These criteria are about performance. When a threshold is hit, a decision follows, not an automatic stop.

Absolute stop rules: off immediately, then investigate

These criteria are about harm. Here you don't wait for an evaluation.

The last criterion is often forgotten. An application approved for customer insight that is quietly used to assess employees changes risk category. Under the AI Act, it may then become high risk. See What is the EU AI Act?.

Evaluative stop rules lead to a decision, absolute stop rules switch the application off immediately.

How to set the thresholds

Stop rules are not arbitrary numbers. They follow from the business case.

1. Start with the baseline. How many hours does the work take now? How often is the current answer right? Without a baseline, you don't know what "worse" means.
2. Derive the threshold from the promise. If the business case promises 10 hours a week saved, 3 hours after two quarters is a signal. If it promises better answers than current reporting, a lower correctness rate is a reason to stop.
3. Choose a measurement period. One bad day is not a trend. Four weeks is reasonable for quality, three months for usage.
4. Decide who measures. A threshold without measurement is a wish. You can measure correctness, for example, through a feedback button on every answer and a sample check by the owner.
5. Have the executive board set the thresholds, per application, when deciding on the business case.

A threshold follows from the business case, not from a loose number.

The example thresholds in this article are a starting point, not a standard. An application that searches contracts needs higher correctness than one that generates ideas.

Who decides?

Stop rules only work if it's clear who may do what.

The key principle: under an absolute stop rule, the owner may stop immediately without asking permission. Explaining afterwards is better than waiting beforehand.

Can you actually stop?

This is the question that most often goes wrong in practice. A stop rule on paper is worthless if nobody knows how to switch the application off.

Ask these questions for each application:

• Where do you switch it off? Is it one switch, or do you have to go through five systems?
• What exactly stops? Only the chat interface, or also the agent retrieving data in the background?
• Which keys must you revoke? Does the application have its own API keys or connections with external parties?
• How long does it take? Minutes, hours or days?
• What does the user see? A clear message, or an error without explanation?

The more applications you have, the more important it is that they all run through the same layer. Then you can revoke access in one place, see what's happening in one place and stop in one place. If every agent has its own connections, stopping becomes a search. See Connect once, not per agent.

Test it. Deliberately switch off each application at least once a year, just like a disaster recovery test. Measure how long it takes and what went wrong.

Pitfalls

• Moving thresholds after the fact. If a threshold is hit and the response is a new threshold, you don't have a stop rule. Changing it is allowed, but only as a separate, recorded executive decision.
• Never hitting a threshold. It sounds good, but often means nobody is measuring. A healthy organisation occasionally hits an evaluative threshold and adjusts.
• Measuring only quality. An application that gives good answers but that nobody uses costs money without delivering value.
• Forgetting cost. With AI, costs grow with usage, and agents can become unexpectedly expensive. See An agent's bill is set before it runs.
• Seeing stopping as failure. Stopping an application because it doesn't deliver what it promised is good governance. It makes room for the next application on the list.

A template to get started

Record the following on one page per application:

Application:        [name]
Owner:              [person]
Purpose:            [what it is approved for, and what not]
Baseline:           [current situation]
Promise:            [benefit from the business case]

Evaluative stop rules
  Quality:          [threshold] over [period] → [consequence]
  Usage:            [threshold] after [period] → [consequence]
  Cost:             [threshold] → [consequence]
  Data quality:     [threshold] → [consequence]

Absolute stop rules
  Privacy/security: any incident → off immediately, escalate
  Bias/customer:    demonstrable harm → off immediately, investigate
  Outside purpose:  use beyond approval → off immediately

Stopping
  Where:            [place/switch]
  By whom:          [role]
  Recovery time:    [tested on date, duration]

Set by executive board on: [date]
Review:             monthly (owner), quarterly (executive and supervisory board)

Frequently asked questions

Are stop rules required under the AI Act? Not by that name. For high-risk AI, the AI Act does require risk management and human oversight, including the ability to interrupt the system.¹ For all other applications, stop rules are simply good governance.

Who sets the thresholds? The executive board, per application, when deciding on the business case. The owner makes a proposal.

What if a threshold is hit but the application is still valuable? Then the executive board makes a deliberate decision to continue, with a reason and a new review date. The difference from having no stop rule is that it's a decision, not a habit.

Do stop rules apply to a pilot? Especially to a pilot. A pilot without an end date and without stop criteria quietly becomes production.

Where do you start? With the application that is used most right now. Fill in the template and test whether you can switch it off. Request a foundation scan.