Salesforce executive says cheaper tokens will not solve enterprise AI costs

A Salesforce vice president is warning that lower AI token prices alone will not fix rising enterprise AI costs. In a Fortune commentary, the executive argued that many companies expanded pilots, agents and employee access before building systems that use tokens efficiently.

The argument lands as companies reassess the cost of generative AI programs after two years of broad experimentation. The Salesforce executive wrote that boardroom discussions have shifted from how to deploy more AI to how to produce measurable results while keeping spending under control.

The commentary cited several signs of that shift. Uber tightened internal AI spending after usage exceeded its plan, while Anthropic, OpenAI and Google cut input token prices and introduced prompt caching, according to the executive. Cursor’s Composer 2.5 now treats cost as a factor in model selection alongside performance, the commentary said.

Five areas where AI spending leaks

The Salesforce executive argued that model pricing is only part of the problem because many enterprise AI systems are built in ways that waste tokens during normal use. The commentary described the typical corporate AI pipeline as one that loses money at multiple stages unless companies redesign the architecture underneath their agents.

The executive pointed to five recurring problems: oversized context windows, weak data governance, overuse of high-end models, lack of persistent memory and repeated processing of requests that could be cached or reused.

On context, the commentary said companies often put too much information directly into prompts, raising costs and increasing the chance that models drift from the task. The executive argued that master data management can reduce that waste by creating unified records for customers, accounts, policies, transactions and recent activity before an AI system builds a prompt.

The commentary also said agents can waste tokens when they search through data warehouses without clear rules about which information is reliable. Data catalogs, lineage, permissions and quality signals can route agents to approved data earlier in the process, according to the executive.

Model choice and memory

The Salesforce executive also criticized the practice of sending too many tasks to the most capable frontier models. The commentary argued that companies should treat models as a portfolio, using smaller systems for lookups and classification, mid-tier models for domain-specific workflows and frontier models for more complex reasoning or sensitive judgments.

Another cost driver, according to the commentary, is stateless AI agents that begin each interaction without usable memory of prior work. The executive wrote that structured memory can let agents retrieve relevant facts, earlier decisions, user preferences and open tasks rather than rebuilding context each time.

The fifth problem involves repeated work. The commentary said many enterprise agents re-read the same context, run the same retrieval steps and generate answers that could have been cached, embedded or pre-computed. Pairing prompt caching, reusable embeddings and pre-computed outputs with enterprise data structures can make recurring requests cheaper and more consistent, the executive argued.

The Salesforce executive said a large company using AI agents in sales or service may handle thousands of interactions a day. Passing raw, uncurated data into prompts can consume five to 10 times more tokens than needed, the commentary said, and costs can add up quickly at $10 to $15 per million tokens.

The executive pointed to Informatica’s master data management integration with Salesforce Data 360 as one example of the architecture being advocated. The broader message, according to the Fortune commentary, is that the next phase of enterprise AI will reward companies that turn AI usage into governed and measurable business value rather than those that consume the most tokens.

This story draws on original reporting from Fortune.