AI initiatives are moving from experimentation to deployment. As new workloads emerge across the business, AI infrastructure decisions depend on factors such as data gravity, security, cost, and governance.
In the fourth episode of DatacenterDynamics (DCD) AI Week 2026, Martin Olsen, Vice President of Segment Strategy & Deployment at Vertiv, joins Deloitte's Diana Kearns-Manolatos, Associate Director, and Matt Jacobs, Silicon2Service Leader. Hosted by Zoe Turner, Channel Manager at DCD, the conversation examines how enterprises are matching AI workloads to the infrastructure models that support them.
Zoe Turner, Channel Manager, DCD: What has changed about the conversations enterprises are having today?
Martin Olsen, Vice President, Segment Strategy & Deployment, Data Centers, Vertiv:
The past two to three years have been about foundational model training, gigawatt-scale data center deployment, and the vast amount of power required. That's proliferating into the practical usage of those large language models and other models within enterprises. You have to be thinking about what type of use cases and workloads you have because that determines your infrastructure setup and the considerations around supporting it.
Vertiv US, for example, is a classic enterprise. We've quickly discovered with our partners at Deloitte that this is not about moving everything from cloud to on-premises or everything in the cloud, but about different types of workloads. We have different workloads, from simulation and engineering with groups worldwide to traditional copilot. They all need different things based on the constraints you're dealing with.
For us, it's data gravity, security, volume of data, and the users we have for it.
Zoe: From Deloitte's perspective, what are you seeing in terms of workload growth across different hosting models, and what does that tell us about enterprise AI adoption?
Diana Kearns-Manolatos, Associate Director, Deloitte:
You have different types of workloads that are hosted in different places. In 2025, we ran a survey of 120 market operators to understand the supply and demand dynamics of workloads and what was driving demand across different modalities. We looked at the data center of gravity, where data was being stored and inferencing was happening.
We saw that workloads weren't just increasing on the cloud where most organizations start to experiment with frontier models. Workloads were increasing by 20% or more with AI adoption and scaling across every modality we asked about: mainframe, public cloud, private cloud, edge computing, neo clouds, and on-prem data centers. And of all those modalities, the two biggest beneficiaries of workload growth among almost 90% of respondents were the neo clouds and edge computing, and some of the on-prem options.
Matt Jacobs, Silicon2Service Leader, Deloitte:
We're in a maturation curve with AI, and it's accelerating faster than anything we've ever seen. We're moving beyond shiny object, beyond "this can do spot work for me," into a world where AI is starting to be bolted onto existing workflows and processes and reshape them.
As utilization increases, the bills increase. And then you compound that with agentic workflows and more autonomy, opening a tremendous cost overrun potential.
Organizations have a handful of levers they can pull. The first is private infrastructure: if I can take some of my high run rate needs and own that, I can significantly reduce my infrastructure costs. The second is model selection, allowing organizations to not just throw the biggest model at the smallest problem but to shape what models are suited for what problems. The third is model routing, to make that model selection and diversity actually work.
But we spend the last decade virtualizing, containerizing, and cloudifying. The enterprise user base is so far abstracted from the platforms that they run on these days that they've lost a lot of the musculature to bring that back inhouse. Power density in these systems is also high, bringing challenges to house it within their premises.
This is where data centers come into play, and Vertiv is working closely with that cohort to make sure that once enterprises do start to scale these AI initiatives, they have a place to put the infrastructure when they decide to go private with it.
Private infrastructure means hybrid. This is an augmentation of the existing capabilities that they have in the cloud with more private infrastructure.
Zoe: How does AI as a workload impact the infrastructure that it runs on?
Matt:
What we pushed into the cloud looks nothing like what's coming back out. The AI infrastructure has to be carefully architected so that they're balanced: massive compute systems, networking that can feed those GPUs, and a good data platform.
Most enterprise data centers are slated at around 15 to 30 kilowatts (kW) per rack, and we surpassed that nearly a decade ago. You have a data center asset that's a 24-month build and a 10-year amortization, with rapid GPU versioning where power densities increase almost by 2X at every version. Data centers are now co-designed to facilitate the content of that data center.
Martin:
The densities are going up dramatically; we're talking 145 kW quickly going to 200 kW for a single rack. If you're like most other enterprises, it's going to be anywhere from 30 to 50 kW in a single rack all the way to a couple of hundred kW. When we talk about scale and densities, we're talking about both ends, from 30 or 40 kW in a rack all the way through to 600 kW or a megawatt.
Diana:
Not every workload is created equally, and not every workload has the same density or cost needs. I'll throw in two other dimensions: privacy and security needs, as well as latency needs. All of these factors together need to create context-aware hybrid architecture. The workload, based on its density, whether you're using an open model or frontier model, whether it's language and text or images and 3D modeling, and where its data center of gravity is, needs to be part of the decision about where inferencing should happen. You then need to understand what type of load impact that's going to have on the hardware.
Zoe: What are the key emerging concerns that you're seeing for enterprises that are scaling AI initiatives into production?
Matt:
The three big concerns of our enterprise clients today are cost, return on investment, and governance.
The move to private and hybrid AI environments ultimately terminates in a distributed architecture, where factors such as latency, locality, point of origin of queries, modality, and model choice come together. As a result, we're moving from monolithic structures to a distributed computing mesh.
As AI is absorbed into more aspects of the enterprise, organizations need to think about resiliency and how to govern costs.
Governance means capturing data, tracking consumption, and creating accountability. Organizations are also correlating consumption with work output and connecting usage to business outcomes. As AI strategies mature, organizations define target outcomes, establish metrics, and tie those metrics to ROI.
Zoe: How do you see enterprises balancing that immediate AI demand versus longer-term infrastructure planning?
Martin:
We kind of think of it as "brownfield to Blackwell," if you understand the reference of Blackwell being the current GPU platform. That is typically the reality of many enterprises today, as well as for us, Vertiv. Some have to come on-prem. Could be colo or physically within our building. For most enterprises, the last data center manager probably retired about five or six years ago just in time for the AI explosion. So there's just not a lot of expertise within the enterprise today.
We've worked with several of our customers, helping them understand the right scenarios for bringing this on prem, particularly because most enterprises deal with a brownfield environment. We just completed one for a customer on the West Coast.
And it's probably something we have not seen before in the industry to this level, to where infrastructure, power and cooling guys like us, we've been very distant cousins of the compute guys or the server guys within an enterprise. But now we're just working very closely together, and that convergence is happening really quickly.
Zoe: Why do partnerships matter now more than ever?
Martin:
It usually takes a village to do any of this. It's a hybrid architecture. You start with the enterprise, determining outcomes, data, risk, and ownership. Then work with someone like Deloitte on strategy, economics, governance, and operating model. Then work with someone in the industry on the compute stack.
You have to think about the interfaces because that's usually where things break down. We see that from a business standpoint where handoffs and demarcations fall apart. We've built a partnership ecosystem with Deloitte and others to close those seams through productization, services, and pre-configured reference architectures. We're looking for repeatability.
Matt:
There's a comical irony in AI in that it's the most homogeneous workflow that we've seen, but it has this very complex web of partnerships that have to hit each of these different layers and make sure the connective tissue between them is functioning properly.
We think about the next wave of utility. The first is moving from agents for the sake of agents to digital workers with personalities that have built into them the associated governance to make them work. As we look down the road, AI will fall into the rearview mirror as a term. It will be such a pervasive piece of everything that we do that we may start to think about AI as just a utility that we consume.
Watch the full broadcast: From pilot to production - Matching enterprise AI workloads to infrastructure models
