Penn nursing paper urges guardrails as AI enters patient care

Artificial intelligence is moving into nursing, and University of Pennsylvania researchers say hospitals need firmer safeguards before relying on it in patient care. A new paper in Nursing Outlook argues that AI may help nurses with documentation, monitoring and decision support, while also creating risks around safety, bias, privacy and professional accountability.

The paper, titled “Artificial Intelligence and Nursing Science: Opportunities, Challenges, Implications, and Guidelines,” was written by George Demiris and co-authors at the University of Pennsylvania School of Nursing. According to Penn Nursing, the work draws on a two-day interdisciplinary workshop and examines how AI is affecting clinical practice, nursing research and ethics.

Antonia Villarruel, dean of Penn Nursing and a co-author, said hospitals face a major obstacle in the absence of strong governance and evaluation systems for AI in nursing care. She said many health systems want to adopt AI tools quickly but may lack clear standards for validation, fairness checks, monitoring after rollout and responsibility when something goes wrong.

Villarruel also pointed to the practical problem of fitting AI into how nurses work. According to Penn Nursing, the paper warns that a system that looks technically strong can still fail if it adds tasks, disrupts care or does not match bedside practice.

Potential uses and limits

The authors say AI could support nursing by reducing administrative work, helping analyze large data sets, improving clinical decision-making and allowing more tailored patient education. Penn Nursing said the technology could also help identify patient needs and broader population health risks.

The paper also argues that nursing science has a distinct role in shaping AI because nurses focus on patient-centered care, clinical workflow and advocacy. The authors say nurses should be involved before AI tools reach the ward, rather than being asked only to use systems designed by others.

But the paper draws a clear boundary around what AI can do. According to the authors, AI cannot replace central nursing functions such as empathy, moral judgment and patient advocacy. The paper warns that treating AI systems as if they were clinical agents comparable to human professionals could blur responsibility and weaken professional standards.

Five recommendations

The authors set out five guidelines for bringing AI into nursing science and practice. They call first for AI literacy to become a regular part of nursing education, so nurses can assess how tools are built, where they may fail and how inaccurate or biased output could affect patients.

Second, the paper says nurses should help design and evaluate AI systems from the start. The authors recommend interdisciplinary teams that include nurses, engineers, data scientists, hospital leaders, ethicists, regulators and patients.

Third, the authors call for rigorous testing before broad deployment. According to the paper, evaluation should measure accuracy, reliability, bias, workflow effects and patient outcomes in real clinical settings.

Fourth, the paper says hospitals should study the full costs of AI, including effects on staffing, documentation, cognitive burden, morale and continuity of care. The authors caution that AI may create new work even when it appears efficient.

Fifth, the authors urge stronger ethical protections for privacy, informed consent, data use and bias prevention. They say patients should be told when AI influences clinical decisions and how their information may be used to train or operate those systems.

The paper concludes that AI can improve health care only if nurse scientists have a central role in developing, testing and deploying the technology, according to Penn Nursing.

This story draws on original reporting from Medical Xpress.