EU’s AI plan faces hard questions over high-risk rules
The bloc’s white paper calls for strict oversight of high-risk AI, but experts warned that the boundary between risky and routine uses may be hard to draw.
By Hana Yoshida · Markets Reporter
3 min read
The European Union released an artificial intelligence white paper that points toward strict controls for systems deemed high risk. Fortune’s Jeremy Kahn reported that the plan could affect companies well beyond Europe because EU technology rules often reach firms that serve European customers, process EU citizens’ data or employ workers in Europe.
Kahn compared the potential reach of the AI rules with the EU’s General Data Protection Regulation, saying other governments and some U.S. states could use the European framework as a model. The white paper is intended as a step toward future legislation and regulation for AI.
The European Commission’s paper says the 27-country bloc should impose legal requirements on high-risk uses of AI. According to the document, that category includes uses that could cause injury, death or significant material or nonmaterial harm, especially in fields such as health care, transportation, energy and government.
Margrethe Vestager, whose EU role covers competition enforcement and the digital economy, told The New York Times before the paper’s release that she was not focused on policing recommendation systems for music or movies. She said her concern was AI used for decisions such as loan approvals or medical diagnoses, according to the Times.
Risk categories may be hard to police
Kahn wrote that the EU’s approach sounds sensible in theory but could prove difficult for lawmakers to apply. AI systems often sit between clearly dangerous uses and consumer-facing tools that appear less consequential.
Geoff Hinton, a leading deep-learning researcher at Google and the University of Toronto, raised one concern in a post on X. He described a hypothetical choice between an AI surgeon with a higher cure rate but no clear explanation for its decisions and a human surgeon with a lower cure rate, asking whether the AI option should be barred.
Kahn noted that the EU paper does not require every high-risk AI system to be fully explainable. It does, however, call for clear information about a system’s strengths and limits, as well as human oversight.
Nick Cammarata, an OpenAI researcher who works on explainability, responded on X that performance numbers alone would not settle the question. He said he would choose the AI surgeon only if he knew the training data closely matched his own situation; otherwise, he would choose the human doctor.
Low-risk tools can still shape outcomes
Kahn also pointed to recommendation and advertising systems as a possible weakness in the EU framework. Vestager said she was less worried about recommendation engines, but Kahn argued that targeted advertising can affect access to financial products in ways that resemble direct discrimination.
Under the EU approach described by Kahn, an AI loan approval system would likely count as high risk. A system that denied credit based on surnames or neighborhoods, a practice often called digital redlining, would likely violate the rules.
Kahn said a company could produce a similar effect through advertising by withholding promotions for favorable loans from certain groups of people. Those consumers would be less likely to apply if they never learned the product existed, even though the ad system might avoid the high-risk label.
The debate shows the challenge facing EU policymakers as they try to regulate AI by use case. The white paper sets out a risk-based framework, but researchers and analysts cited by Fortune warned that safety, fairness and transparency questions may not fit neatly into the categories regulators create.
This story draws on original reporting from Fortune.