Medical AI privacy attacks hit some patient groups harder

Some medical AI systems may expose whether particular patients were included in their training data, with risks falling unevenly across patient groups, according to a study published in Nature. The finding matters because the same systems are being developed to help diagnose disease from scans, records and other clinical data.

The study, by Moritz A. Knolle and colleagues, examined membership inference attacks, a type of privacy attack that tries to determine whether a specific person's information was used to train an AI model. The researchers found that average privacy scores can make a model look broadly safe while hiding severe exposure for individual patients.

How the attack works

Membership inference attacks do not necessarily require access to a model's underlying code, according to the study summary reported by Medical Xpress. An attacker may be able to submit a patient record, such as a chest X-ray, and use the model's output to infer whether that record was part of the training set.

The risk arises because AI systems can be slightly more confident when evaluating examples they encountered during training. In one example described by Medical Xpress, a model returning an 80% pneumonia probability for a chest X-ray could give an attacker information useful for testing whether that patient's image helped train the model.

Medical AI models are used or studied for tasks including pneumonia detection on X-rays and classifying skin lesions as benign or malignant. The Nature study said stronger diagnostic performance can come with a privacy cost: more capable disease-prediction models may create greater exposure to membership inference attacks.

Patient-level audit

To measure individual risk, the researchers used medical data spanning seven large real-world datasets, according to Medical Xpress. The data included chest X-rays, skin images, mammograms, eye scans, ECG readings and electronic health records.

Rather than testing a single model, the team trained 200 model versions on different randomly selected groups of patients. That design let the researchers compare models that had trained on a given patient's data with models that had not.

For each patient, the researchers assessed all records contributed by that person and used the highest-risk record as the patient's final risk score. The study's rationale was that identifying even one image or record could reveal the patient's membership in the dataset.

The results showed that privacy risk was not distributed evenly. Medical Xpress reported that people in underrepresented groups, including racial minorities, Medicaid recipients and patients with rare medical conditions, were more vulnerable than majority populations to privacy leaks.

The study also found that routine aggregate privacy metrics can conceal these differences. According to the researchers, that could undermine trust if some groups believe their data is less protected, and it could also affect future model quality if those groups become less willing to share data.

Limits of anonymization

The researchers said removing names or replacing direct identifiers is not enough protection against modern membership inference attacks. They pointed instead to patient-level differential privacy, which uses mathematically designed noise in data or model outputs to limit what can be learned about any one patient.

The Nature paper frames the issue as a fairness problem as well as a cybersecurity problem. As clinical AI systems become more accurate and more widely used, the study argues that privacy safeguards need to account for the patients at greatest individual risk, not only the average performance of an attack across a dataset.

This story draws on original reporting from Medical Xpress.