By eleven o’clock, she had approved 38 AI-drafted replies. By the afternoon, nobody could say what she had actually checked.
Not because she was careless. Nobody had told her what to look for. The drafts read well, most of them were fine, and the queue kept growing. Approving became the job.
This is how “a person stays in the loop” quietly turns into a signature on work nobody really reviewed.
Human review is one of the most common safeguards in an SME’s first AI workflow, and rightly so. But a reviewer is not a safeguard by default. Review is a step in the process, and like any other step it needs to be designed.
Route each output by what a mistake would cost.
A plain rule decides the lane. The reviewer checks against written criteria, with the source beside the output.
An AI-drafted output arrives, with its source attached.
Internal summaries, draft tasks, suggested categories.
Client replies about availability or next steps.
Money, contracts, sensitive data, binding commitments.
Uncertain cases wait in one visible queue, with an owner and a time limit.
Three ways review goes wrong
Most review problems fall into one of three patterns.
The rubber stamp
The output usually looks right, so the reviewer starts trusting it. Checks become skims, then clicks.
The team believes there is a control in place. In practice there is a delay.
The bottleneck
One experienced person reviews everything. The AI drafts in seconds, then the work waits two days for her.
Speed moves from the task to a queue, and the workflow ends up slower than before.
The review that teaches nothing
The reviewer corrects the same kind of mistake every week, in a private inbox, without recording why.
The workflow never improves, and the reviewer’s judgement leaves with her when she goes on holiday.
None of these are AI failures. They are process design failures that happen to sit next to an AI step.
Decide what the reviewer is actually checking
“Check the output” is not an instruction.
A useful review step names the few things that matter for this workflow. For a drafted client reply, that might be:
- ✓the facts match the source: the right price, date, address or reference
- ✓nothing is promised that the business has not agreed to
- ✓the tone fits the client and the situation
- ✓the request has been understood correctly
Spelling and phrasing are rarely the real risk. A wrong deposit amount written in perfect French is.
Write the checks down. Keep them short enough to apply to every case. If the reviewer cannot explain what a “pass” means, the AI cannot be held to a standard either.
Put the source next to the output
Review is only as good as what the reviewer can see.
If a person has to open three systems to confirm that an extracted invoice total is correct, they will eventually stop opening them. Show the original email, document or record beside the proposed output. Highlight the fields the AI filled in or changed.
This sounds like a detail. It is often the difference between a ten-second check and a skim.
Review by consequence, not by volume
Not every output deserves the same attention. Sort cases by what happens if a mistake gets through:
Light review
An internal summary, a draft task, a suggested category. Light review, or spot checks once the workflow has proved itself.
Full review
A client reply about availability or next steps. Review before sending, against the written checks.
Named approval
Money, contracts, sensitive personal data or a commitment the business must honour. A named person approves, and the approval is recorded.
Fixed automation can carry this routing. The AI can classify or flag uncertainty, but the rule that sends a contract-related case to a manager should be a plain, visible rule, not a judgement the model makes on its own.
This is also how you avoid the bottleneck. The most experienced person reviews the cases that need her judgement, not the whole queue.
Give uncertain cases one visible place to wait
Every AI workflow produces cases it should not handle alone: missing information, unusual requests, conflicting sources.
Those cases need one place to wait, with an owner and a time limit. Not a forwarded email. Not a message in someone’s chat. A queue or status the team can see, so that “waiting for review” never becomes “forgotten”.
If nobody can see how many cases are waiting, nobody can tell whether review is working.
Capture the reason, not just the correction
When a reviewer changes an output, the edit fixes one case. The reason fixes the next hundred.
Ask for a short reason with each meaningful correction: wrong source, outdated price, misunderstood request, tone, missing information. One line is enough.
Over a few weeks, those reasons show where the problem really sits. Often it is not the model at all. It is an outdated reference document, a missing field in the form or a rule nobody wrote down. Those are fixable, and fixing them reduces review effort for good.
Decide in advance when review can shrink
Review should not stay at full intensity forever, but it should not shrink on a feeling either.
Agree the conditions before launch. For example: after a set number of reviewed cases, if corrections in a category are rare and minor, that category moves from full review to spot checks. If corrections rise again, it moves back.
The decision belongs to the business owner of the workflow, based on the correction record, not to whoever is tired of reviewing.
Some categories should never lose their approval step. That is a business decision, not a technical limitation.
Make sure someone else can do it
If one person’s judgement is the whole review step, the workflow depends on one person.
The written checks, the visible queue and the recorded reasons make review transferable. A colleague covering a holiday week should be able to apply the same standard, not guess at it.
A short review design exercise
For one AI workflow you run or plan to run, write down:
the three to five things the reviewer must check
what the reviewer sees beside each output
which cases need full review, light review or named approval
where uncertain cases wait, and who owns them
how corrections and their reasons are recorded
the evidence that would let review shrink, and the categories where it never will
who reviews when the usual reviewer is away
If the answers are vague, the workflow has a reviewer but not yet a review step.
Human judgement is one of the most valuable parts of an AI workflow. It deserves a better design than a queue and an approve button.
Related reading
How should an SME choose its first AI workflow?
Six practical filters for choosing one bounded, measurable and controllable first workflow.
RELIABILITYHow should an SME keep an AI workflow reliable after launch?
Clear ownership, visible exceptions, tested changes and a safe fallback for live AI workflows.