Skip to main content
Privacy & Security

Medical AI's average privacy score can hide its most exposed patients

Researchers could identify some patients' inclusion in training data almost perfectly even when the attack looked weak overall.

6 Nov 2026

Disparate privacy risks from medical AI - Nature
Image: Nature

Knowing that someone helped train a model built only from cancer patients' records could itself reveal a diagnosis. An attacker would not need to recover the person's entire chart. Learning that they belonged to the training group might be enough.

That is the kind of clue examined in a Nature study published in June 2026. It found that some patients' participation could be detected almost perfectly even when the same attack looked little better than guessing overall.

One question can reveal something private

The technique is called membership inference: an outsider examines a model's predictions and tries to tell whether a particular person's data helped train it. The result concerns membership in that group, not the recovery of a whole medical record.

The researchers tested diagnostic models using seven clinical datasets, including images, heart traces and health records. Instead of stopping at an average score, they measured how well the attack worked against individual patients.

The average hid the people at greatest risk

Some people were much easier to identify as training participants than others. Underrepresented groups also faced disproportionately high attack success, including groups defined by disease, race, insurance, sex or imaging protocol. As model capacity increased, so did the number of patients facing high attack success in these tests.

The finding changes what a reassuring average can mean. It may describe many people reasonably well while concealing a small group for whom the attack works reliably.

The study tested this particular kind of privacy attack. Whether other attacks distribute risk in the same way remains an open question. Its warning is about how privacy is measured: a weak average attack is not necessarily weak against you.