Bernardo Modenesi led a research team that wanted to know whether a large language model could score one of medicine’s quieter quality measures: shared decision-making, the moment a clinician and a family actually weigh a choice together rather than the doctor simply deciding.
Picture the setting the team studied. A parent of a child with multiple long-term conditions sits across from a surgeon, deciding whether an operation is worth the risk. The conversation is supposed to be shared — doctor and family weighing options together. But who actually decides whether that happened, across thousands of visits no one reads by hand?
Most labs would reach for the obvious tool: point a model at the recording and ask it, in plain language, to label the behavior. Modenesi’s group did exactly that — and then refused to stop there.
They analyzed 21 audio-recorded outpatient surgical decision encounters — 19 unique patients, 7,566 utterance segments, about 6.1 hours of talk — between families of children with multiple long-term conditions and their surgical providers arXiv paper. Trained human coders marked 12 shared-decision behaviors, agreeing with each other at a macro Cohen’s kappa of 0.695, a respectable bar for messy human judgment.
The prompt that fell short
First they tried a zero-shot local model, Qwen 2.5 32B, instructed to detect the behaviors on its own. It reached a macro kappa of just 0.139, barely above the floor of chance agreement DOI.
That number is the whole story in miniature. The model was not stupid. It was being asked, in a single instruction, to do a task that even trained humans find subtle — and it collapsed.
A small supervised classifier told a different story. Trained on frozen sentence embeddings, it reached kappa 0.227, a paired improvement of 0.088 over the zero-shot run DOI. Modenesi argued the point plainly: “Zero-shot prompting alone is not sufficient to measure SDM behavior as reliably as a small supervised model.”
Why the better number still hides a trap
Stacking the two approaches — a logistic blend of the prompt and the classifier — pushed kappa to 0.242, Modenesi and colleagues reported arXiv paper. Better, but the authors refused to let the result sit comfortably. They went hunting for leakage, the quiet ways a benchmark can lie about itself.
Their warning is specific, the team noted: “patient-level grouping alone does not prevent leakage when labeled prompt exemplars are precomputed outside the outer evaluation loop.” In plain terms, if examples from a patient held out for testing leak into the prompts used to fit later models, the score is contaminated arXiv paper. The team, which includes Jody Lin, Kimberly Kaphingst, and Angela Fagerlin, also catalogued pitfalls such as splitting sibling recordings into different groups so one family’s voice appears on both sides of the test author listing.
What this means for clinics betting on AI scoring
The study is small and narrow — 19 patients, one surgical setting, one codebook. The authors say external validation is needed before any of it generalizes beyond this population, model, prompt, and code arXiv paper. That caution is the point, not a disclaimer.
Measuring whether an AI truly helps in a clinical conversation turns out to require the same discipline as building the AI: a shared technical language for what “good” looks like, agreed thresholds, and honest data splits. It echoes a wider argument that AI rules need a shared technical language, not just laws AI rules need a shared technical language.
For any team tempted to drop a prompt in front of a recorder, the lesson is blunt. A clever instruction is not a measurement. Until someone proves otherwise on your patients, the small trained model — and a human in the loop — still beats the elegant one-liner.
