When LLMs Say "I'm Sure," Do They Actually Know? A New Study Measures the Gap

Four language models, 224,000 factual questions, and one uncomfortable finding: a model's stated confidence and its actual correctness are not the same thing, and the size of that gap depends heavily on which model you're using and what topic you're asking about.

That's the core result of a new research paper that borrows a decades-old framework from psychophysics, Signal Detection Theory (SDT), to pull apart two things that usually get lumped together when evaluating LLM outputs. The first is Type-1 accuracy: whether the model's answer is right or wrong. The second is Type-2 metacognitive sensitivity: whether the model's confidence score actually tracks its own correctness. A model can be accurate but a poor judge of its own reliability, or mediocre at answering questions yet unusually honest about when it's likely wrong. Standard accuracy benchmarks collapse these two dimensions into one number, obscuring which failure mode is actually at play.

The researchers tested four openly available models: Llama-3-8B-Instruct, Llama-3-8B-Base, Mistral-7B-Instruct-v0.3, and Gemma-2-9B-Instruct. Across the full dataset, each model's confidence signal followed an "unequal-variance" pattern familiar from human perceptual studies, but the shape of that pattern (measured via z-ROC slopes) varied from 0.81 to 1.18 depending on the model. In plain terms, different models distribute their confidence errors in structurally different ways, not just at different average rates.

Confidence Isn't One-Size-Fits-All

Perhaps the more practically relevant finding is that metacognitive quality is domain-dependent. All four models showed their strongest self-awareness when answering questions about Arts & Literature, suggesting that a model's ability to know what it doesn't know isn't a fixed trait but something that shifts with subject matter. Overall, the metacognitive signal varied by more than a factor of two between the best- and worst-performing models tested, a spread the authors argue is too large to ignore when deploying these systems in settings where miscalibrated confidence carries real risk, such as medical or legal question-answering tools.

The team also ran a temperature-sweep experiment that produced a notable dissociation: raising or lowering the sampling temperature shifted Type-1 accuracy up or down, but left the underlying metacognitive sensitivity largely untouched. That implies confidence calibration and raw correctness are governed by at least partially separate mechanisms inside these models, rather than one simply following from the other.

The paper, first posted to arXiv in March 2026 and revised in July 2026, swapped its original meta-d'/M-ratio statistical approach for a newer, model-free measure called meta-I_2r in the second version, a change the authors say produces more robust estimates without assuming a particular underlying distribution. The study was pre-registered, and both code and data have been released publicly, allowing other researchers to replicate or challenge the findings directly rather than take the confidence claims on faith.