EY says its 'invisible' AI router has helped cut token consumption by up to 60%

Bloomberg/Getty Images

The Big Four firm EY has built an "invisible" router into its AI tools to help manage its token spend.

The router redirects employees to the right AI tool for their task, said Dan Diasio, the firm's global consulting AI leader.

It's cut token consumption in some divisions by 60%, he said.

EY does not let every employee query go straight to the smartest AI model in the room.

The Big Four firm has introduced an "invisible" router to help manage its internal AI spending, Dan Diasio, EY's global consulting AI leader, told Business Insider.

Rolled out globally in April, the router sits behind some of EY's specialized AI tools and acts as an intermediary to direct employees towards the best AI model for the task at hand.

"To the users, it makes no apparent change," said Diasio. They open the frontier model, enter their prompt, and get a response. But behind the scenes, the router is deciding where their query should go.

"Many times, it's faster than the frontier models, because to use the heavier models means there's more thinking that gets put in place, and it takes longer to be able to get back to the answer," Diasio added.

Token costs have become a growing concern for companies as AI providers increasingly charge based on usage. From February to June, OpenAI, Anthropic, and GitHub introduced pricing models tied to tokens — the units that measure AI's input and output — rather than with flat-rate billing.

The shift has meant that using powerful models for simple tasks can quickly become costly.

EY US's AI Pulse survey, released Tuesday, found that 82% of senior leaders at companies investing in AI were concerned about token usage. The survey was based on responses from 534 senior decision-makers at US businesses between April and May.

"A lot of these big bills that companies are getting surprised by are by the top one or the top 2% of people inside the employee base that are just using the wrong tool for the job," said Diasio.

A router helps select the right model for the right job, limiting token waste and ensuring that more expensive AI requests are reserved for the functions where they can deliver the greatest value. It's a solution many EY clients are also building to help manage costs, said Diasio.

The router has not been deployed universally across all of EY's AI tools. Instead, it's been deployed on department-specific platforms, including tax and risk. It is not used for the general Microsoft Copilot chatbot available to all EY staff.

The solution has had a "meaningful impact" on how staff use AI tokens, Diasio said.

The router, together with training and governance strategies, has helped reduce token consumption by 60% since its implementation in April, EY said.

The next phase of AI monitoring

The first phase of the corporate AI boom was about getting employees to use the technology — cue the "tokenmaxxing" phenomenon that encouraged workers to use as much AI as possible.

Rising costs have since shifted the focus to controlling that use.

Corporate giants like Disney and JPMorgan have installed dashboards to track employees' use of AI, and AI-routing startups are on the rise, helping developers direct tasks to different AI models, monitor for overspending, and quickly resolve outages.

EY has set token budgets based on employees' roles and departments, Diasio said. Those who exceed their token allowance can request more through an approval process.

Some 64% of senior leaders surveyed for EY's AI Pulse report said their organizations now monitor AI token usage and have clear budgetary guardrails for how much they spend. Diasio expects that figure to rise quickly: "If we ask this question in six months from now, it would probably be like 90% of people."

Dan Diasio, EY's global consulting AI leader. EY

There are also other levers that can help companies get more value from their AI spending, Diasio said.

Businesses can overlook the importance of building strong internal knowledge layers and designing better prompts grounded in their own data. They should focus their investment on areas where AI can have the deepest impact on how work is done, rather than spreading resources across every function, he said.

Monitoring employee usage will continue to remain important, Diasio said, but corporate AI deployment is entering a new phase in which the focus is on value and outcomes.

Fundamentally, measuring AI's value requires looking beyond profit and loss, Diasio said.

Instead, he said, the broader questions businesses should be asking include: Is AI enabling employees to get through tasks faster? And how can you incentivize employees to explore new initiatives and responsibilities that aren't in their current job description?

Read the original article on Business Insider

Indexed and credited by AIPROPX. Originating outlet: Business Insider. Open at source →