Building an enduring company is the wrong goal. It’s a lagging indicator. You get an enduring company by building an enduring product, and you get an enduring product by building an enduring architecture. As TechCrunch reported, the theme for their 2026 Disrupt conference is how to build an enduring company in the AI era. But the conversations on stage will likely miss the most critical part, the part that happens in your IDE and your terminal.
The real question isn't how to sprinkle AI features onto your roadmap. The real question is how to wire them into your stack without creating a time bomb of technical debt, vendor lock-in, and unpredictable costs. Get that wrong, and your 'enduring' company won't survive its first major LLM provider outage.
Imagine your product team wants to add AI-powered summaries to your app's dashboard. The fastest way to ship it is obvious. You find the right endpoint in the OpenAI or Anthropic docs, npm install their client, and write a function that makes a direct API call. You ship it. The PM is happy, the metrics look good, and you move on to the next ticket. Six months later, you've done this in a dozen different microservices for a dozen different features. Now you have a problem.
The 'AI Feature' Trap
That simple, direct API call is a seed for architectural rot. What started as a quick win becomes a distributed monolith, where your 'monolith' is a third-party API you don't control. Every team that makes a direct call to an LLM provider is making a hard-coded architectural decision.
Suddenly, you're facing questions you can't easily answer:
- Cost: How much are we spending on AI summaries versus AI chatbots? You can't tell, because the calls are coming from all over your system and are hitting one big API key. You just get a single, terrifying bill at the end of the month.
- Performance: A new model version, say GPT-5, drops your p99 latency from 200ms to 2 seconds. The degradation is system-wide, but since every service calls the API directly, you have to patch and redeploy twenty services to even test a fix.
- Resilience: Your chosen provider has a major outage. Your entire application is now throwing errors because a dozen features that depend on it are all failing independently.
- Vendor Lock-in: A competitor like Claude Opus 6 comes out and it's 50% cheaper and twice as fast for your specific use case. Migrating is a nightmare. It requires a coordinated, months-long effort to refactor every single service that baked in the OpenAI SDK and its specific prompt structures.
This is not an enduring architecture. It's a house of cards built on someone else's platform. The pursuit of feature velocity has created massive operational risk.
An AI Gateway Isn't Optional
The only way to build an enduring AI strategy is to treat all external AI providers as implementation details. Your internal services shouldn't know or care if they're talking to OpenAI, Cohere, or a self-hosted Llama 4 model running on your own hardware. The way you do this is by building a small, centralized service in front of them: an AI Gateway.
This isn't about building a massive, complex piece of infrastructure from day one. It can start as a single module in your monolith or a simple cloud function. The principle is what matters: centralize all AI interactions through one choke point that you control.
Centralize Model and Prompt Management
Your code shouldn't contain prompts. Prompts are configuration. They change, they need to be versioned, and they need to be tested against different models. Your gateway becomes the place to manage this.
When a service needs an AI generation, it doesn't send a raw prompt. It sends a structured request.
Here’s some pseudocode showing the difference. First, the 'trap' approach:
// Direct API call from a random service
import OpenAI from 'openai';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async function summarizeReport(reportText) {
const response = await openai.chat.completions.create({
model: 'gpt-4o',
messages: [
{ role: 'system', content: 'You are a helpful assistant that summarizes reports.' },
{ role: 'user', content: `Summarize this report: ${reportText}` }
],
});
return response.choices[0].message.content;
}
Now, the gateway approach:
// Internal service calling your gateway
async function summarizeReport(reportText) {
const response = await aiGatewayClient.generate({
promptId: 'report-summary-v3',
variables: { reportText: reportText },
userId: 'user-123' // For tracking and abuse monitoring
});
return response.result;
}
Inside the gateway, promptId: 'report-summary-v3' maps to a versioned prompt template stored in a database or config file. The gateway is responsible for fetching the template, injecting the variables, and deciding which underlying model (GPT-4o, Claude Opus 5, etc.) is best suited for the report-summary task. Now, if you want to switch to a new model provider, you change one record in your gateway's configuration, not 20 different services.
Take Control of Cost and Observability
With a gateway, every single AI call flows through one place. This is a superpower for an engineering leader.
You can now:
- Implement universal caching. If two users ask for a summary of the same public document, you only pay for the LLM call once.
- Enforce budgets and rate limits. You can kill a request if a specific user or feature is running up the bill. You can put a hard cap on monthly spend.
- Log everything. You can track tokens per call, cost per feature, and latency per model, all in one structured format. Your finance team will thank you.
This layer of control is the difference between using AI and being used by it.
The Real Tradeoff: Speed Now vs. Sanity Later
Building a gateway, even a simple one, is more work up front than making a direct API call. That's the tradeoff, and it's a real one. When you're a startup trying to find product-market fit, speed is everything.
But this is a false dichotomy. You can start small. The first version of your gateway might just be a single function in a shared library that wraps the SDK call and adds logging. That's it. It might take an extra day. But it establishes a pattern. It creates a seam in your architecture.
As your AI usage grows, you can evolve that simple function into a proper microservice without requiring a system-wide refactor. The cost of not creating that seam early on is immense. The technical debt accrued by scattering direct API calls across a codebase is some of the most expensive debt you can take on, because it ties you to an external dependency you can't control.
What This Means for Your Team
When your team comes back from Disrupt, inspired to build an 'enduring AI company', here’s how you translate that into engineering reality.
- Ban direct third-party AI SDK calls. All interaction with LLM providers must go through an internal abstraction, period. Start simple, but start now.
- Treat prompts as versioned configuration, not code. Store them outside your application service. Your
report-summary-v1prompt might be great for GPT-4 but terrible for Llama 3. You need to be able to manage them independently. - Instrument every call. From day one, log the prompt ID, the model used, the latency, the token count, and the user ID associated with every single AI generation. If you can't measure it, you can't manage it.
- Own your AI stack. An enduring company can't be a thin wrapper around someone else's technology. Your value is in the data, the user experience, and the unique architecture you build to connect them. The AI gateway is the heart of that architecture.
The most enduring companies are the ones that manage their dependencies best. In the AI era, your biggest dependency is your model provider. Don't let your architecture be dictated by their API docs. Build the stack that lets you choose the best tool for the job, today and tomorrow. That's how you build something that lasts.
Building something in this space? AgileStack helps teams ship enterprise-grade software without the consulting-firm overhead. Book a 30-minute call and tell us what you're working on.