Meeting AI Demand Starts With The Right Infrastructure

​Juan Font is President and CEO of CoreSite.

gettyIn the words of Benjamin Franklin: “Great haste makes great waste.” It’s an apt perspective for viewing the rush to AI deployment in the enterprise today, where organizational leaders race into AI deployment decisions without always having the infrastructure in place to support them.

For many enterprises, the default setting is to contract directly with major frontier labs such as Anthropic or OpenAI and big cloud providers such as Google or Microsoft. Effectively, they’re racing to allow employees to use tokens for almost anything—often without much oversight.

This rush is strikingly reminiscent of when the cloud was introduced in the 2010s, and employees opened public cloud accounts, running up very large tabs—again with very little oversight over the projects being undertaken.

Today, companies are rushing to consume tokens with relatively little thought given to how that consumption is supported. In many cases, the problem isn’t the AI deployment itself. It’s the infrastructure supporting it. And solving that problem is critical to developing and realizing a sound AI strategy.

At the innermost layer, AI requires compute. But compute in the form of GPU chips is extremely scarce. To deal with that scarcity, a whole new segment of companies called neoclouds that offer GPU as a service has emerged.

Enterprises that want to develop or train their own medium-sized models are renting or leasing compute infrastructure from these neoclouds. As that market matures, some companies are beginning to buy the hardware themselves, particularly in financial services, where firms require massive amounts of compute as well as real-time access to financial data and exchanges.

Companies developing their own models and doing their own inferencing require space, power and cooling for GPU clusters, along with connectivity to the models they use for other workflows. Less sophisticated companies are simply using the cloud, which can become increasingly expensive as usage grows. Running GPUs is extremely costly, and performance can degrade when long-haul fiber is used to connect to compute resources. You end up with chips waiting for data—resources you’re paying for but not fully utilizing.

That’s where colocation data centers come into the picture, particularly those close to population centers and network hubs. Using colocation data centers as inferencing hubs to manage the handoff between private and public infrastructure can improve performance by enabling direct connections.

Adjacency to where compute is performed is of vital importance, but so is being in an environment that allows you to future-proof your infrastructure as the mix of workloads changes. Getting there takes a partner who can help you step back and build the right infrastructure foundation before you scale your AI.

We’re also seeing a preponderance of multi-cloud and multi-model environments. Cloud providers are trying to become the entry point into the intelligence realm, and they want to connect you to everybody. But to operate in a multi-platform environment, you need the interconnection infrastructure to support it. The ability for huge models to connect is what ultimately drives their value.

From a cost perspective, it’s impossible to overstate how expensive it would be to transfer terabits per second of data across regions. And now, companies are releasing agents. Meta recently released Muse, an agentic AI personal assistant, and we see OpenAI and Anthropic moving in the same direction. Each of these applications requires dedicated compute, cybersecurity controls and tremendous amounts of bandwidth. That rise in agentic inferencing workloads will require inexpensive, high-speed connectivity to these models. Having compute in metro centers, adjacent to the clouds, will be vital from both a cost and speed perspective.

The enterprises that have the greatest need to place their digital infrastructure in a colocation data center—and gain closer proximity to inferencing—are not necessarily traditional brick-and-mortar hospitals and banks. They’re the AI models themselves: Anthropic, Meta, OpenAI, CoreWeave, Microsoft and Google. These companies are deploying edge networking infrastructure to support data transfer because the focus until now has been on the compute layer primarily devoted to training those models. Only now is the interconnection layer requiring edge compute and network being built. Demand for this interconnection capacity is already outpacing supply. Existing data center inventory is extremely limited, and we’re racing to build incremental capacity to meet growing enterprise demand.

All that horizontal stratification between the different participants of the edge AI compute ecosystem is happening right now, and it requires multiple megawatts of capacity. That edge infrastructure must be built to support the exponential growth in demand that is coming. In essence, this is the “plumbing” that is going to enable the massive growth in agentic AI workloads by enterprises and individuals alike.

Ultimately, because the network was not designed for the projected explosion in bandwidth, leveraging colocation data centers becomes a matter of survival. You must colocate—ideally adjacent to the edge compute infrastructure hosted by hyperscalers, neoclouds and frontier labs—and cross connect directly to the infrastructure. That’s the edge compute and interconnection on-ramps we’re building right now, and we’re just scratching the surface. The companies that take the time to build that foundation, before they scale, will be better positioned to turn today’s rush into an AI strategy that is sustainable tomorrow.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?