A supervisory board does not need to understand how a language model works to oversee AI well. What it needs are five fixed questions every quarter. The answers show whether AI delivers value, whether someone is accountable, and whether the organisation can stop when things go wrong.
Oversight of the annual accounts works the same way. You don't have to do the bookkeeping yourself to see whether the numbers add up. You need to know which questions to ask and what a good answer looks like.
These five questions come from our AI governance framework: value, ethical boundaries and stop rules, built on a foundation of cybersecurity and data sovereignty. Read the full framework in AI governance for executive and supervisory boards: value, ethics and stop rules.
Five fixed questions every quarter, from register to control.
1. Which AI applications are running, and who owns each one?
Why this question: you cannot oversee what you cannot see. In many organisations nobody knows exactly which AI is running. There's a chatbot here, a Copilot licence there, and a pilot that was never formally stopped.
A good answer: a one-page AI register. For each application it shows what it does, its status (pilot, in use, stopped) and who the business owner is. An owner is a person, not a department.
Probe further if:
- the answer is "IT" or "the data team". Then ownership is unclear;
- the register hasn't been updated since last quarter;
- employees use AI tools that aren't in the register.
2. What have they delivered compared with the business case?
Why this question: AI projects are often started on enthusiasm and continued out of habit. Without measurement, nobody knows whether an application is worth its cost.
A good answer: a business case for each application, written before the start, with a baseline and a target. Next to it is the current state. For example: hours per week spent checking figures, the percentage of AI answers rated correct, and the number of active users.
Probe further if:
- benefits are described rather than measured ("employees are enthusiastic");
- there is no baseline, because then every improvement is an assumption;
- running costs are unknown. With AI, they grow with usage. See also An agent's bill is set before it runs.
3. Has a stop criterion been hit, and what was decided?
Why this question: this is the question most often missing, and perhaps the most important. An application without stop rules keeps running simply because it has started.
A good answer: every application has stop criteria that were set before go-live. They cover thresholds for quality, usage, cost, data quality, privacy and bias. The answer also states whether a threshold was hit and what the executive board decided: continue, adjust or stop.
Probe further if:
- a criterion is never hit, because that may mean nobody is measuring;
- thresholds were moved after the fact;
- nobody can say how to stop an application technically, and how quickly.
4. Were there incidents, and were they reported within the deadlines?
Why this question: the Dutch Cyber Security Act (the national implementation of NIS2) and the GDPR set fixed reporting deadlines, and the clock starts at the moment of discovery.23 Executives carry final responsibility. Reporting late is a second incident on top of the first.
A good answer: an overview of incidents and near misses, showing for each when it was discovered, reported and resolved. Also an escalation ladder with fixed deadlines, tested every year.
Probe further if:
- there are never any near misses. A healthy organisation does see them;
- the supervisory board only hears about a significant incident afterwards;
- the last disaster recovery test was more than a year ago.
5. Does all data stay in the EU and under our control?
Why this question: every new connection with customers, partners or an AI model is a new route through which data can leak. Customers and regulators are also becoming more critical about where data is stored and who can access it.
A good answer: every data connection goes through the same checks. Data stays in the EU, a data processing agreement is in place before the first data flow, the purpose is recorded, access runs through one identity with minimal rights, and there is an exit strategy.
Probe further if:
- the answer is "we're in an EU region" and nothing more. With American vendors, that does not fully rule out access under US law;
- there are connections without a recorded purpose;
- nobody knows how long switching to another vendor would take.
Per question: what a good answer looks like and when to probe further.
How to organise this in practice
The five questions work best as a standing agenda item, with a one-page dashboard on which the executive board answers them. The dashboard contains:
- the AI register with owners and status;
- benefits against costs per application;
- stop criteria that were hit and the decisions taken;
- incidents and reporting deadlines;
- the status of cybersecurity and NIS2.
Once a year, the supervisory board reviews the framework itself: are these still the right questions? Schedule a short AI literacy session for the supervisory board at the same time. The AI Act requires AI literacy from those who work with AI, and oversight is part of that.1
Such a dashboard does ask something of your data. If every application has its own definitions and logging, filling it in takes days every quarter. If everything runs through one layer, it's an export. See Connect once, not per agent.
