I am not an AI researcher. I do not build models. I do not train LLMs. What I do is build infrastructure platforms — and I have noticed something interesting over the past eighteen months.
The teams deploying AI workloads at scale are rediscovering the same lessons that enterprise Kubernetes teams learned five years ago. Resource management. Multi-tenancy. Observability. Cost visibility. These are not new problems. They are platform engineering problems wearing different labels.
The same problems, different hardware
Five years ago, the question was: “How do we run containers in production without blowing everything up?” Today, the question is: “How do we run GPU workloads without blowing everything up?”
The hardware is different. The challenges are structurally identical.
Consider resource management. In Kubernetes, poorly configured resource requests and limits cause node pressure, pod evictions, and cluster instability. With GPUs, the stakes are higher because the resource is more expensive and less fungible. A GPU that is allocated but idle costs real money. A GPU that is oversubscribed causes training jobs to fail.
The solution is the same pattern that worked for Kubernetes: define resource boundaries, enforce them, and make consumption visible. The difference is that GPU scheduling is less mature than CPU scheduling, and the tooling is still catching up.
Multi-tenancy is harder with GPUs
Kubernetes multi-tenancy was a hard problem that took years to solve. Namespaces, RBAC, network policies, resource quotas — these patterns emerged through painful experience.
AI infrastructure faces an additional dimension of multi-tenancy: GPU isolation. Two training jobs running on the same GPU can interfere with each other in ways that CPU workloads do not. Memory bandwidth, cache contention, and kernel launch latency all affect training performance.
NVIDIA’s MIG (Multi-Instance GPU) and time-slicing help, but they are not perfect solutions. MIG partitions are static. Time-slicing does not provide memory isolation. The platform team’s job is to understand these trade-offs and provide guidance that application teams can act on.
This is exactly the role platform teams have been playing for Kubernetes: abstracting complexity, documenting trade-offs, and providing safe defaults.
Observability for AI is where Kubernetes observability was in 2019
When I started working with Kubernetes in production, observability was a mess. Teams had logs, metrics, and traces — but they were disconnected. Nobody could tell you why a pod crashed without correlating data from three different tools.
AI observability today is in a similar state. Teams can monitor GPU utilization and memory usage, but correlating infrastructure metrics with training performance is still hard. Why did training throughput drop at 3 AM? Was it a network issue, a GPU thermal throttle, or a problem with the data pipeline?
The platform team’s job is to build the observability layer that connects infrastructure to application outcomes. This is not fundamentally different from building a Kubernetes observability stack. The data sources are different. The patterns are the same.
Cost visibility is the same problem with bigger numbers
Enterprise Kubernetes adoptions inevitably reach the point where someone asks: “How much does this cost?” The answer is usually unsatisfying because infrastructure cost allocation is hard.
AI infrastructure makes this problem urgent because the numbers are larger. A single GPU node can cost more than an entire Kubernetes cluster of CPU nodes. Training runs can consume thousands of GPU-hours. Without cost visibility, teams cannot answer basic questions: Is this model training run worth the cost? Should we invest in more GPUs or optimize utilization?
Platform teams that built cost visibility for Kubernetes have a head start. The same patterns — resource tagging, usage attribution, cost allocation models — apply to GPU infrastructure. The difference is that the cost of getting it wrong is higher.
Where platform engineering fits
I have been part of enough infrastructure transitions to recognize the pattern. A new technology emerges. Early adopters struggle with operational complexity. Platform teams emerge to abstract that complexity. The technology becomes accessible to a broader audience.
AI infrastructure is in the early adopter phase. The teams running AI workloads are the ones who can afford dedicated infrastructure engineers. The platform teams are still forming. The tooling is still immature.
But the trajectory is clear. Within a few years, deploying AI workloads will look like deploying containerized applications today. Platform teams will provide golden paths for GPU scheduling, observability pipelines for training runs, and cost dashboards that connect infrastructure spend to business value.
I am not an AI expert. I am a platform engineer who recognizes the pattern. And the pattern tells me that AI infrastructure success will depend on platform engineering — not on any single technology.
If you are building AI infrastructure today, invest in the platform layer. Define resource boundaries. Build observability. Make costs visible. Automate everything. These are not new problems. They are the same problems, with higher stakes and bigger numbers.