Trust and Oversight Check
The points where a person must step in
- PhaseTest
- Group3 to 10 people
- Time35 minutes
- SettingRoom or remote
- ExecutivesYes
What it is
A trust and oversight check walks through each step the AI solution performs and asks what could go wrong, how a person would notice, how long a check would take and who is accountable. It finds oversight that exists on paper but would not work in practice.
When to use it
During or after a test, and before any decision to build, to find where a person must check, approve or stay in charge.
When to skip it
Before there is any described or prototyped behaviour to check.
How to run it
- 5 minWalk through the prototype step by step.
- 15 minFor each step done by the AI solution, say what could go wrong, how a person would notice, and how long a check takes.
- 10 minMark every step where a real check is not realistic.
- 5 minName the accountable person for each check.
About 35 minutes of method time. Add ten to fifteen minutes to open and close the session.
What to watch for
Oversight that is only a sign-off with no real check. Assuming a person will catch errors. Ignoring the time a check takes.
The AI lens
If a check takes as long as doing the work, the trust cost is higher than it looks. Change the design or keep that step with people.
What you leave with
The points where a person must step in, each with an accountable owner. Use them to design each check into the solution.
An example
A step where the AI drafts a decision letter needs a person to read the draft. If each check takes as long as writing the letter, the trust cost is higher than it looked and the design must change.
How it compares
A Trust and Oversight Check examines responsibility and checking. A Pre-mortem looks more broadly at what could make the project fail.
Related methods
Questions
What is Trust and Oversight Check?
A trust and oversight check walks through each step the AI solution performs and asks what could go wrong, how a person would notice, how long a check would take and who is accountable. It finds oversight that exists on paper but would not work in practice.
When should I use Trust and Oversight Check?
During or after a test, and before any decision to build, to find where a person must check, approve or stay in charge.
How long does Trust and Oversight Check take and how many people do I need?
Trust and Oversight Check takes about 35 minutes and suits 3 to 10 people, plus a facilitator. Add ten to fifteen minutes to open and close the session.
How is Trust and Oversight Check different from other methods?
A Trust and Oversight Check examines responsibility and checking. A Pre-mortem looks more broadly at what could make the project fail.
Can I run Trust and Oversight Check remotely or with executives?
Yes. Trust and Oversight Check works in the room or remotely. Remotely, put the canvas on a shared board in advance and write silently in the board, not in chat. Executives: Yes, works with executives present.