When Medical AI Can't See Who It's Failing, CAPRA Fills the Blind Spot

Most fairness audits of medical imaging AI rely on a simple assumption: that datasets come with reliable demographic labels like age, sex, or race attached to every scan. In practice, that metadata is frequently missing, incomplete, or unreliable, leaving hospitals and researchers unable to check whether a diagnostic model quietly underperforms on certain patient groups. A new framework called CAPRA, detailed in a paper submitted to arXiv on July 10, 2026, tackles this gap directly.

CAPRA stands for a calibrated proxy-axis approach to subgroup analysis, built specifically for situations where the metadata needed to slice a dataset by demographic group simply isn't there. Rather than depending on labels that may not exist, the framework infers semantic axes directly from the images themselves. It then calibrates the posterior probabilities of those inferred axes using a small subset of the data that does have metadata labels, applying patient-level cross-fitting to keep the calibration statistically sound and avoid leakage between training and evaluation splits.

The research team, which includes submitting author Guanhua Ye, tested CAPRA across three distinct imaging modalities: fundus photography (used for retinal disease screening), dermoscopy (skin lesion imaging), and chest radiography. That spread matters because fairness tools that only work on one type of scan have limited practical value in a hospital setting where multiple imaging pipelines run simultaneously.

Catching What Metadata-Only Audits Miss

The paper's central claim is that CAPRA surfaces disparity patterns that conventional metadata-only slicing methods overlook entirely. In other words, even when demographic labels are available for part of a dataset, standard auditing techniques can still miss performance gaps that only become visible when the analysis incorporates image-derived signals alongside whatever sparse metadata exists. The authors also report that CAPRA stays informative even when the underlying data distribution shifts, a property that matters for real-world deployment where imaging equipment, patient populations, and clinical protocols change over time and location.

This kind of dataset shift is a persistent headache for deployed medical AI: a model validated on one hospital's imaging equipment can behave unpredictably when moved to another site with different scanners or patient demographics. A subgroup analysis tool that degrades under that shift is of limited use once a model leaves the lab.

The paper has been categorized under image and video processing, artificial intelligence, computer vision and pattern recognition, and multimedia on arXiv, reflecting its position at the intersection of clinical AI safety and core machine learning methodology. For teams building or auditing diagnostic imaging systems, CAPRA offers a concrete answer to a question that's increasingly hard to ignore: how do you check for bias when the very data needed to check for it isn't recorded in the first place?