Executives warn AI agent oversight will need rules that scale

Corporate technology leaders said companies are running into a hard limit on the common advice to keep people supervising artificial intelligence at every step. At Fortune Brainstorm Tech in Aspen, executives from Salesforce, DraftKings, Indeed and Xero said agentic AI is moving into systems where the number of automated interactions can exceed what human reviewers can realistically monitor.

Zach Maybury, chief technology officer at DraftKings, said the online sports betting company already works with trillions of transactions and spread-out computing loads. He said adding AI agents that communicate with other AI agents increases the size and complexity of those systems beyond the reach of older oversight methods.

Maybury said DraftKings cannot put employees into every automated process. “We will never have enough humans to insert in all those loops,” he said during the panel, according to Fortune.

The discussion centered on how businesses should control AI in high-stakes settings as they move beyond small tests. Panelists said companies need clearer rules for what AI systems may do, how they are built and who is accountable when they are used.

LaShonda Anderson-Williams, Salesforce’s chief customer and commercial officer, said the stakes differ sharply by industry. In health care, she said, a bad decision is not comparable to a retailer sending a customer the wrong shirt size because a person’s life may be affected.

Anderson-Williams said companies should begin by defining the business problem and the outcome they want from AI. She also said many organizations adopted AI tools quickly without first setting rules for how those systems would be used.

Governance, the executives said, becomes more urgent as AI moves from pilot projects into broader operations. Maybury said companies may need to revisit older policies, revise them and expand them so they can handle AI systems operating at far greater speed and scale.

Anthony Moisant, Indeed’s chief information and security officer, described a similar oversight problem at a large job listings service. He said Indeed serves 645 million job seekers and 3.5 million employers, making full human review across all AI-influenced processes difficult at that level of activity.

Moisant said companies should test AI-related processes continuously and compare the results with the outcomes they intended to produce. That approach, he said, can help organizations spot whether automated systems are drifting from their goals.

Diya Jolly, chief product and technology officer at accounting software company Xero, said the type of task should shape how much freedom an AI agent receives. She said tasks with clear, measurable answers are easier to test and may allow more automation.

Jolly said decisions that require judgment are harder to remove from human control. In those cases, she said, keeping people involved remains more difficult to avoid because the correct answer may depend on context rather than a fixed result.

The executives did not offer a single model for AI oversight. Their shared message was that companies using agentic AI in critical operations will need governance systems that can keep pace with machines acting faster and in greater numbers than human teams can review one by one.

This story draws on original reporting from Fortune.