You're trying to spin up a new cluster of H100s for a critical model training run. The request hangs. Then, an email lands in your inbox from your cloud provider: 'Due to unprecedented demand, capacity in us-east-1 is temporarily constrained.' This isn't a software bug or a networking issue. It's a hard physical limit. You've hit a wall made of copper, silicon, and a lack of megawatts. And it's going to happen a lot more.
The cloud was supposed to make infrastructure someone else's problem. For years, it worked. But the massive power and cooling demands of AI are tearing holes in that abstraction. The physical world is leaking through. This brings us to a recent, seemingly obscure piece of news that’s actually a massive red flag for anyone building AI products.
As TechCrunch reported, AI cloud company Crusoe has abandoned a $1.25 billion plan to power its data centers using stationary turbines from Boom Supersonic. This wasn't just a standard vendor swap. It was a high-stakes bet on an exotic power source to solve the fundamental constraint of modern AI: a shortage of electricity. The bet failed. And that failure is a preview of the new class of risks that CTOs and architects must now manage. Your software stack now includes a power plant, whether you like it or not.
The Leaky Abstraction of Infinite Compute
For two decades, we've designed software with the assumption that the underlying hardware was effectively infinite. Need more servers? Your auto-scaling group handles it. Need more storage? S3 will absorb it. The cloud's core promise was that you only had to worry about your code, not the physical machines running it. We traded control for convenience and scalability.
AI workloads, particularly large model training and inference, break this promise. An NVIDIA H100 GPU can pull 700 watts at peak load. A single rack of eight of them, like a p5.48xlarge instance on AWS, is a 5.6kW appliance before you even account for networking and cooling overhead. A modern AI data center consumes power on the scale of a small city, measured in hundreds of megawatts. This isn't a resource you can just provision on demand with an API call anymore. You are now competing for public utility contracts.
This is why Crusoe's deal with Boom was so telling. They weren't just buying more servers. They were trying to buy their own power grid because the public one isn't sufficient. They planned to use repurposed supersonic jet engines as power generators located right next to their compute. It was a desperate, innovative move to vertically integrate their power supply chain. And its cancellation shows just how fragile and experimental the physical layer of AI infrastructure really is. Your cloud provider is making wild bets on experimental hardware, and when those bets fail, it's your application's capacity that disappears.
How to De-Risk Your Physical Dependencies
If the physical layer is now a part of your architecture, you have to manage its risks just like you manage database failover or API versioning. Ignoring it is no longer an option. It's time to start asking your cloud vendors some very different questions. At AgileStack, when we help teams scope out major AI initiatives, we're now pushing them to look past the API documentation and into the provider's physical strategy.
Audit Your Provider's Energy Roadmap
Stop reading their developer blog and start reading their quarterly investor calls. What is their real strategy for power and data center expansion? Are they building new facilities or just leasing space in existing ones? Where are they building, and do those locations have stable power grids? A provider heavily concentrated in a region with a strained grid (like Northern Virginia) represents a real single point of failure. Are they making huge, speculative bets like Crusoe, or do they have a more conservative, diversified energy sourcing plan? A provider that can't give you a clear, five-year roadmap for power procurement is a provider that can't guarantee your capacity.
Treat 'Capacity' as a Feature, Not a Given
Your architecture needs to be resilient to capacity failures. This is more than just setting up auto-scaling. It means building systems that can degrade gracefully when the ideal hardware isn't available.
For example, if your primary inference endpoint relies on a large Llama 3 70B model running on H100s, what happens when you can't get them? Your system shouldn't just fall over. It should automatically fail over to a smaller, quantized Mixtral 8x7B model running on older, more available A10G GPUs. The user gets a slightly less capable answer instead of a 500 error. This requires building routing logic that understands not just latency and cost, but also hardware availability and model performance trade-offs.
Model Your 'Watts per Output'
Engineers track what matters. We obsess over milliseconds of latency and bytes of memory. It's time to add a new metric to your dashboards: watts per inference, or watts per training step. This isn't just for ESG reports. It's a critical efficiency metric that directly impacts availability.
Why? Because the most power-efficient models are the ones most likely to get scheduled on constrained hardware. A model that's 5% less accurate but uses 50% less power might be the only one that can run at scale during peak demand. You need to quantify this trade-off.
Here's some simple pseudocode showing how you might start to think about modeling this:
# A simplified cost & power model for inference routing
# This should live in a shared config accessible by your orchestration layer.
model_options:
- model_id: "llama3-70b-fp16"
provider: "aws"
instance: "p5.48xlarge"
cost_per_hour: 98.32
estimated_watts: 22400 # 8 GPUs * 700W * 4 PUE
quality_tier: 1
is_available: function_to_check_aws_capacity()
- model_id: "mixtral-8x7b-int8"
provider: "gcp"
instance: "a3-highgpu-8g"
cost_per_hour: 35.80
estimated_watts: 18000 # 8 GPUs * 500W * 4.5 PUE
quality_tier: 2
is_available: function_to_check_gcp_capacity()
# Your routing logic:
# 1. Check availability for tier 1 model.
# 2. If unavailable, check tier 2.
# 3. If all unavailable, return graceful error or queue request.
This forces you to make the trade-offs explicit. It's no longer just about picking the "best" model. It's about picking the best available model, where availability is a direct function of physical constraints like power and cooling.
Takeaways for Your Team
The Crusoe-Boom story is a canary in the coal mine. It signals a fundamental shift in the responsibilities of a software architect. Here's what it means for your team today:
- Vendor lock-in is now physical. Migrating your codebase from GCP to AWS is a pain. Migrating your entire operation because your provider's data centers in Ohio ran out of contracted power is a company-killing event. Your choice of cloud provider is now a bet on their ability to manage massive, physical infrastructure projects.
- Model efficiency is a tier-one requirement. For years, we could afford to be sloppy. We'd throw more hardware at inefficient code. That era is over for AI. The most valuable models in the near future will be the ones that deliver the best results per watt, because they are the only ones that will run reliably at scale.
- Multi-cloud is about physical redundancy. A multi-cloud strategy is no longer just a tool for price negotiation. It's a disaster recovery plan for grid-level failures. Diversifying across providers in different geographical regions is the only way to insulate yourself from a blackout or capacity crunch in one area.
- The platform team needs to understand thermodynamics. Your platform engineers, the ones managing your Kubernetes clusters and CI/CD pipelines, now need to have a working knowledge of power usage effectiveness (PUE), rack density, and data center cooling. The abstraction is broken, and they're the ones who will have to deal with the messy reality.
The next time you're in a planning meeting for your AI roadmap, ask a different kind of question. Don't just ask your cloud vendor about their latest API features or their serverless GPU offerings. Ask them about their 10-year power purchase agreements. Ask them what their PUE is in the regions you plan to use. Ask them how they're securing their supply chain for transformers and switchgear.
The quality of their answer will tell you more about the future stability of your AI application than any product demo ever could.
Building something in this space? AgileStack helps teams ship enterprise-grade software without the consulting-firm overhead. Book a 30-minute call and tell us what you're working on.