Researchers open medical AI build process for public audit

Researchers at EPFL have released a fully open framework for turning general large language models into medical AI systems, a move they say is aimed at making clinical models easier to inspect. The work matters because EPFL says many AI tools now used in health care keep their training data, model choices and evaluation methods out of public view.

The framework, called MeditronFO, was developed by EPFL’s Laboratory for Intelligent Global Health & Humanitarian Response Technologies in the School of Computer and Communication Sciences. The team describes the system in a preprint posted to arXiv.

EPFL says MeditronFO builds on Meditron, a medical language model project first released in 2023. Xavier Theimer-Lienhard, a Ph.D. student leading Meditron at LiGHT, said the new pipeline is meant to create medical versions of any open large language model while exposing the data, code, training steps and evaluation methods used along the way.

What the framework opens up

According to EPFL, the researchers used MeditronFO to medicalize several open base models, including OLMo, EuroLLM and Apertus. Apertus is Switzerland’s model developed by EPFL and ETH Zurich, according to the university.

The team argues that many systems described as open release the final trained model while withholding key parts of the development process. EPFL says that approach makes it harder for clinicians, hospitals and regulators to review how a system was built or why it produces a recommendation.

MeditronFO is designed to make the pipeline auditable. EPFL says the framework documents the datasets, data processing, training procedures and evaluation methods used to produce the medical models.

The research group combined public medical datasets with clinician-reviewed synthetic data drawn from medical examinations, clinical guidelines and realistic patient cases, according to EPFL. The team also used expert-curated clinical datasets based on more than 46,000 clinical practice guidelines.

Clinicians involved in development

EPFL says clinicians were involved throughout the project, including in data curation, output validation and safety review. The university said the work also uses MOOVE, short for Massive Open Online Validation and Evaluations, to bring clinicians into continuing assessment of model behavior.

Through MOOVE, clinicians can help review training material and validated model outputs, according to EPFL. Mary-Anne Hartley, a medical doctor and director of LiGHT, said the findings point to a path where health systems and communities can keep more control over medical AI development instead of depending only on proprietary vendors.

The arXiv preprint reports that every MeditronFO model performed better than its original base model. EPFL said the strongest result came from Apertus-70B-MeditronFO, which improved medical exam performance by 6.6 percentage points compared with the underlying model.

Testing in clinical settings

The MeditronFO release is part of a wider research program, according to EPFL. The team is preparing clinical trials at multiple sites, from Switzerland to Tanzania, to study how doctors use AI systems in real care settings.

EPFL says those studies will examine whether clinicians accept or reject AI-generated recommendations and how those choices affect patient care. The multi-year trial project, called MED.USE, also aims to assess whether AI can improve care quality while reducing unnecessary treatments and interventions.

Hartley said real-world feedback tied to patient outcomes is needed, according to EPFL. Theimer-Lienhard said the results show that fully open medical models can be built competitively, while Hartley framed the project as part of a broader push for transparency, scientific review and clinician and patient participation in medical AI.

This story draws on original reporting from Medical Xpress.