The Hidden Cost of Scaling AI

Businesses have already decided AI matters. What’s far less settled is whether their internal networks can carry that ambition — and what happens as the demands placed on them increase. 

There is already evidence of a widening divide between companies building AI capabilities and those moving more cautiously. According to BCG, AI leaders commit around 1.7% of revenue to the technology, compared with 0.8% among laggards. Companies with deeper AI integration are also recording three times the cost reduction of peers, alongside stronger margins and returns on invested capital.

That does not mean simply spending more guarantees better results. But it raises the stakes for companies whose infrastructure prevents them from turning AI investment into scaled deployment.

The focus has largely been on AI models, computing power and applications, while the enterprise network infrastructure carrying data between them has often been treated as background infrastructure. As AI moves from pilot to full deployment, that assumption can become expensive. More data has to move between applications, devices, users and locations — creating the risk of additional IT costs, operational complexity and AI investments that fail to produce their intended return.

AI becomes a strategic and investment priority  

The level of investment shows how seriously businesses are taking the opportunity. Generative AI spending could reach $2.3 trillion by 2032, according to Bloomberg Intelligence.

That spending is also reshaping the infrastructure beneath AI. Bloomberg Intelligence forecasts AI servers could account for 79% of worldwide server revenue by 2028, up from 12% in 2022. 

Businesses across sectors see substantial potential. Industrial executives expect that AI could accelerate product development cycles by up to 40%, adding new premium-priced features. That could drive sales growth of 6% to 10%, Bloomberg reports.

In the consumer space, the same research reveals that 90% of companies have AI tools that are being used by more than one-fifth of their workforce. Beyond internal adoption, businesses are also exploring AI-powered personalization and engagement tools to create new ways to interact with customers. For others, AI offers new ways to extract value from the large volumes of data they already collect.

As these use cases multiply, the amount of data moving around an organization increases — along with the demands placed on its internal network infrastructure. 

The impact of more data in motion

AI applications depend on large and often continuous flows of information, particularly where connected devices, such as sensors and cameras are involved. 

Autonomous vehicles are a leading use case. For example, US-based Kodiak, which provides autonomous-driving technology for industrial and defense customers, has strengthened the network infrastructure supporting its operations to enable remote operators to access vehicles’ camera feeds and sensor data in near-real time: remote operators can help guide trucks through unexpected situations, even in remote locations. Kodiak plans to scale its operations across multiple markets.

The growing use of agentic AI — the use of tools and APIs to carry out end-to-end workstreams independently — is expected to intensify data demands. Systems capable of carrying out multi-step tasks independently require constant access to data and applications across different environments. That can increase traffic across an organization’s internal network as well as the complexity of managing connections between cloud, edge and enterprise systems. 

These and other data-intensive AI use cases increase demand on the enterprise network infrastructure that moves data around a business.

Scaling AI changes network requirements

As companies attempt to scale successful pilots, the challenge may be less about proving what AI can do and more about ensuring their enterprise infrastructure can support larger and more data-intensive deployments.  

There is already evidence of the productivity at stake. Customer-service research from Stanford Graduate School of Business found that access to a generative AI assistant increased productivity by 15% on average, measured by issues resolved per hour. The challenge for enterprises is reproducing gains like those consistently across larger operations. 

"When you run a pilot in one location and then try to roll it out across every site, response times can become terrible," says Sanjiv Gossain, Group Vice President and Head of EMEA at Verizon Business, which partners with organizations across 180 countries to deliver modern network infrastructure. 

"Often that’s because you're pushing far more data through a legacy network, in many cases designed 15 or 20 years ago, than it was ever built to carry,” he says. “You might have a single one-gigabit line into a building, and there's simply more data than that pipe can handle." 

That kind of bottleneck rarely shows up as a line item on its own, Gartner estimates that organizations failing to invest properly in AI could overspend on IT operations by 25% by 2030.

If an AI rollout requires additional layers of integration, manual intervention or workarounds, the productivity gains demonstrated in a controlled environment may prove harder to reproduce across an enterprise.

Reliability becomes even more important as organizations introduce systems designed to operate with less human intervention. Agentic operations working at machine speed depend on consistent access to data and applications, potentially making interruptions more consequential.

Cybersecurity adds another dimension: As AI expands the volume and velocity of activity across enterprise systems, the performance of the network infrastructure also becomes increasingly important to rapid threat detection and response.

“Neutralizing sophisticated threats requires robust access controls and zero-trust architecture running on high-performance network foundations,” says Gossain.

Preparing enterprise networks for what comes next

While the opportunities to harness AI are evident, the broader costs of falling behind on it are significant — in stalled rollouts and ceded ground to competitors that got their infrastructure right first.

Another danger of not keeping pace is the loss of market position. “Doing nothing really is giving way to the competition,” says Gossain.

Where necessary, updates to enterprise infrastructure may include the introduction of private wireless networks to achieve near-real-time connectivity. For example, mechanical construction firm Binsky & Snyder adopted this solution to boost productivity when its existing warehouse network could not support the instant data transfer required by engineers working across its 130,000-square-foot facility. 

This project demonstrates the kind of enterprise network capability that data-intensive and automated operations require and often depend upon. 

Gossain urges organizations to assess their enterprise network capabilities against current and future AI workloads before scaling — because the alternative is expensive.

Mapping where data, users and AI workloads sit, and how efficiently an organization’s network architecture connects them, can help determine whether an AI rollout keeps infrastructure costs under control.

Modernizing that architecture pays for itself in more than avoided overruns. Businesses that process AI workloads closer to where they’re generated can eliminate the bottlenecks that erode performance and slow ROI.

Some businesses are also exploring ways to create value from data generated by their connected environments. Retailers, for instance, can anonymize in-store foot traffic data and sell those insights to brands trying to understand how their products perform in different locations.

“Yes, compute and storage are important,” says Gossain, “but infrastructure really needs to be rethought and reconfigured for an AI-centric world.”