How Modular Data Centers Will Scale AI Infrastructure

Steve Altizer, President & CEO of Compu Dynamics & Compu Dynamics Modular, pioneers high-density modular infrastructure for the AI era.

gettyAI is forcing us to rethink how we design, build and operate data centers. In this new era, data centers will look less like buildings and more like an industrial plant, made up of modules that plug into common power, cooling and network “utilities” the way you plug an electric car into a charger.

For decades, we treated data centers like landmark buildings: You select a site, pour concrete and build a facility for a wide variety of IT customers. That model still makes sense for cloud or enterprise IT workloads running at five to 30 kilowatts a rack.

However, AI requires an entirely different infrastructure stack. AI racks can draw significantly more power: 150 to 250 kilowatts or more. When AI systems are installed in a facility designed for the cloud, you end up with a football-field-sized white space containing tiny islands of racks. It is an expensive way to support a small amount of equipment.

What I see emerging instead is an industrial platform: a site that standardizes power, liquid cooling and networks, and then accepts factory-built IT modules the way charging stations accept cars. You roll a module up, plug into the electrical bus, the liquid cooling loop and the fiber, run your workload, then swap that module for another when the next generation of GPUs is available.

We refer to this future AI design as “bring your own white space,” but it is only half a joke. Imagine a site owner providing a robust utility yard, including medium-voltage power distribution, heat-rejection systems and backbone networking, while tenants bring their own white space modules the way they bring their own IT gear into a colocation facility today.

Colocation providers already standardize power distribution systems, cooling plants and meet-me rooms. We are taking that same mindset a bit further: Standardize site infrastructure, including power, cooling and network connectivity, while tailoring data hall modules to each AI customer’s requirements for rack layouts, power distribution, liquid cooling and cabling. Factory-built white-space modules ranging from 5 MW to 20 MW can accommodate both traditional colocation and high-density AI workloads, allowing operators to adapt capacity as hardware requirements evolve without rebuilding the entire facility.​

AI is not just another workload inside a traditional data center. It is a different species of facility. While a typical cloud rack operates at roughly 10 kW, an AI rack demands 150 kW, driving a 10 to 15 times increase in density that requires an entirely new approach to power distribution. This shift forces the primary cooling medium from air to liquid, dictating completely different plant designs and controls. Furthermore, because AI requires significantly lower floor space per megawatt, traditional space planning breaks down completely.

GPUs are advancing so quickly that we do not have 10 years to “prove” a design before deployment. As Peter Gross has said, “The data center industry loves innovation as long as it is 10 years old.” AI does not give us that luxury. We are essentially bolting the wings on the plane while we are in flight.

Modular fits because modules are designed around a specific rack density and cooling strategy, so you can optimize fluid temperatures, pump energy and distribution losses in ways far harder in a traditional data hall. A modular campus can adapt over time by swapping obsolete modules for new ones instead of stranding an entire building, as we see with 20-year-old sites in top tier markets that were never designed for current densities.

Sustainability is not just about power usage effectiveness (PUE); it is about how much you build, how long it lasts and how hard it is to change. Modular allows incremental scaling to add modules as demand grows, enabling compact layouts that reduce land disruption, and making higher-temperature liquid cooling more practical, which improves overall energy efficiency.

An emerging AI trend is the willingness to consider architectures without traditional UPS systems and diesel generators. For an existing colo, ripping out that infrastructure is unthinkable. In a modular world, you can create AI-specific modules that assume a different risk posture from day one. No traditional UPS battery strings or diesel generator sets, and a power path optimized for efficiency rather than ride-through. All while still plugging into standardized site utilities.

Trying to retrofit that into an existing colo is like trying to turn a passenger jet into a cargo ship without ever landing it. Building it into a modular design from the start is far more rational.

In conversations with AI customers, two words dominate: speed and scale. If you can deliver fast and compress a traditional 18-to-24-month build cycle down to six to nine months, you are very popular at the party. If you cannot, you may not get invited. Factories have a hard throughput limit, and the capacity now being requested for AI and cloud can easily outpace the industry’s manufacturing base.

That is why I often compare modular data center assembly lines to automotive lines. Once designs are standardized, you want modules rolling out like cars off an assembly line: same chassis and interfaces, different “trim packages” for each AI customer.

Leaders evaluating modular partners should look behind the marketing and ask whether the company truly manufactures at scale, if the standard module fits their AI use case, whether the provider can deliver a turnkey solution and not just drop off a box and whether the commercial terms reflect the capital intensity of modular manufacturing with appropriate milestone or staged payments.

In AI, the mission of a data center is straightforward: Produce tokens at the lowest possible cost, with the least energy and land, as close as practical to the users who need that compute. That increasingly points toward modular data centers that are truly and purely token factories rather than architectural trophies.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?