Publication note20256 min read

Guided deferral with language models

A system that does more than say “ask a human”: it passes uncertain cases on with useful, targeted guidance.

A handover should carry something useful

Conventional deferral ends with a binary action: answer, or ask a human. In a clinical workflow, “I do not know” is an incomplete handover. The person receiving the case still has to reconstruct the model's reasoning, decide what to inspect, and judge whether the AI's uncertainty is meaningful.

Guided deferral asks a more collaborative question: when the model is uncertain, can it pass the case on with targeted information that helps the human make the final decision?

From report to prediction—or guided referral

Architecture of a guided deferral system that combines an LLM verbalised probability with a hidden-state classifier and supplies guidance on uncertain cases — The guided-deferral pipeline. A medical report is parsed by an instruction-tuned language model; uncertain cases are deferred with generated guidance, while sufficiently certain cases are handled by the model. From Figure 1 of the paper.

Text

Verbalised prediction

The model states a probability in its generated answer, together with reasons a disorder might or might not be present.

State

Hidden-state prediction

A small classifier reads the final-layer representation, extracting a second probability from what the model internally encodes.

Team

Combined prediction

The two signals are averaged for classification; uncertainty can be measured with the signal best suited to routing.

Confidence is measured by distance from the binary decision boundary. Cases closest to 0.5 are handed over first, so an operational threshold controls how much work remains autonomous and how much enters the guided pathway.

Calibration under imbalance

Clinical report labels are often overwhelmingly negative. Ordinary Expected Calibration Error weights each confidence bin by how many samples fall into it, so a dominant negative region can hide poor calibration elsewhere. The paper introduces Imbalanced ECE, blending empirical bin frequency with a uniform weight:

\operatorname{ECE}_{\mathrm{Imb}}=\sum_{m=1}^{M}\underbrace{\left((1-\gamma)\frac{|B_m|}{n}+\frac{\gamma}{M}\right)}_{\text{balanced bin weight }\Gamma_m}\,\underbrace{|\bar h_m-\bar y_m|}_{\text{calibration gap}}

When $\gamma=0$ , this is ordinary ECE. Moving $\gamma$ toward one gives sparse bins more influence. In this study, $\gamma=0.3$ exposed calibration error that standard ECE largely concealed in highly imbalanced data.

Did the guidance help people?

The pilot study involved 20 biomedical researchers who were not specialists in spinal disorders. Each reviewed 30 of the system's most uncertain report-level predictions. When a participant disagreed with the model, they received its guidance and could keep or revise their answer.

Box plots comparing participant accuracy without guidance and with AI guidance, against the language model's accuracy — Participant accuracy before and after guidance; the dashed line is the language model alone. Every participant improved with guidance, and the paired difference was significant (p< 0.01). From Figure 4 of the paper.

Importantly, guidance helped people respond to disagreement even when the model itself was wrong. That is the behaviour a collaborative system needs: the explanation should support judgement, not merely make the model more persuasive.

What this does—and does not—show

The pilot was small and involved biomedical researchers, not practising spinal specialists or a live clinical pathway.
The task used retrospective reports and a fixed set of uncertain cases; workload, time pressure, and downstream consequences were not modelled.
Generated guidance can still anchor users, omit important evidence, or sound more certain than warranted.
Clinical use would require prospective evaluation, drift monitoring, clear escalation rules, and training about the model's limits.

Conclusion

Deferral is an interaction, not an error code. This study shows the promise of pairing uncertainty-aware routing with useful context for the person taking over. The next step is to test whether that richer handover remains safe, calibrated, and genuinely helpful in real clinical work.