The AI part is 10%. The plumbing is everything.

I've been building AI into products for a while now, and the thing that surprises people most when I explain how it works is this: the actual AI part is 10% of the system.

The other 90% is plumbing.

When people think about building an AI product, they picture choosing a model, writing a clever prompt, and shipping it. That's the part that gets the tweets and the demo videos.

It's also the easiest part of the whole stack.

The reality is data pipelines that break at 3am. Caching layers because model calls are expensive and slow. Error handling for hallucinations, API outages, token limits, and latency spikes. Evaluation pipelines so you actually know if things are getting better or worse.

Cost monitoring because one careless loop can burn through hundreds of dollars in API calls before anyone notices.

I see this constantly in the consulting work we do. A client comes in excited about their AI feature. The model works great in the notebook. Then we start talking about production and the real work begins.

How do you handle it when the model is down? What happens when the response is garbage? How do you keep costs predictable at scale? How do you even know if the thing is working well for your users?

These aren't AI problems. They're the same boring, fundamental, infrastructure-level problems that have always separated demos from products.

This is why I get uneasy when I see teams stacking their hiring entirely around prompt engineers and ML researchers. Those people matter. But you also need the person who knows how to build a reliable distributed system. Who thinks about error handling before the happy path.

Who can design something that doesn't fall over when real traffic hits it.

But the fundamentals of building reliable systems haven't changed. They just matter more now because the thing you're wrapping infrastructure around is nondeterministic and expensive.

The AI part is genuinely exciting. I'm not downplaying it.

But if you're spending most of your time on prompts and not enough on everything around them, you're building a demo, not a product.