5.5 Human review workflows and confidence calibration
Aggregate accuracy hides segment failure
97% overall accuracy can mask poor performance on a specific document type or field. Before reducing human review, analyse accuracy by document type and by field and verify performance is consistent across all segments. A single document type at 70% inside a 97% average will produce most of your downstream incidents.
Stratified random sampling
Sample high-confidence extractions on an ongoing basis to:
- measure the actual error rate inside the "safe" bucket, and
- detect novel error patterns that no existing check looks for.
Stratify so each document type and field segment is represented — otherwise the sample inherits the same blind spot as the aggregate.
Field-level confidence, calibrated
Have the model output field-level confidence scores, then calibrate the review thresholds using a labeled validation set. Uncalibrated raw scores are not a threshold, they are a number.
Routing rules
Route to human review when:
- model confidence for a field is low, or
- the source document is ambiguous or self-contradictory.
The goal is prioritising limited reviewer capacity, not reviewing less.
Quiz
- Question 1
Your pipeline reports 97% field-level accuracy across 50,000 processed documents. A proposal on the table is to stop human review of all high-confidence extractions and keep reviewers for the low-confidence tail. What should you establish before accepting it?
- Question 2
You have automated all high-confidence extractions after validating accuracy by document type and field. You now want ongoing measurement of the error rate inside that automated set, and early detection of error patterns nobody has seen yet, without reviewing everything. What do you implement?
- Question 3
Reviewer capacity covers about 8% of volume. Today a whole extraction is routed to review whenever any field is uncertain, which sends 22% of documents — and in most of those the only uncertain field is a low-stakes description string. What change fits the capacity?
- Question 4
A team proposes reducing human review of Claude-generated code on the strength of a 91% aggregate acceptance rate measured over six months and several thousand changes. What should they establish first?