There is a moment in almost every AI project when excitement starts to replace caution.
The model works.
The demo looks impressive.
The numbers are good.
Someone says, “We need to put this into production”
And sometimes, that is exactly when the real risk begins.
When AI leaves the laboratory
I was recently reminded of this by a very ordinary real-world example involving my father and a pair of prescription glasses.
He ordered new prescription glasses from a well-known and established online webshop. This wasn’t some new budget-friendly startup trying to disrupt the market. It was a company he already knew, and he had successfully ordered from them before.
This time, however, the ordering experience was slightly different.
Instead of the familiar select-and-choose approach he had used before – select the frame, choose the lenses, enter the prescription and confirm the options – parts of the process had been redesigned around a series of question-like steps.
Rather than simply choosing from clearly defined options, the system was asking questions and guiding the customer through the process.
It looked more conversational.
It also looked more intelligent.
But somewhere along the way, something went wrong.
The prescription/order information was mixed up, and the glasses that arrived were simply the wrong glasses.
Of course, the next step was obvious: contact customer support.
Except customer support had also been “improved” with AI.
Unfortunately, that didn’t help either.
The AI-powered support wasn’t able to understand or resolve the problem as expected. Instead of helping my father correct an error created somewhere in the process, it effectively became another obstacle between him and the solution.
Eventually, he did what customers have been doing for decades:
He picked up the phone and called them.
A human being finally solved what the AI-powered process couldn’t.
There is something almost amusing about that sequence:
AI changed the ordering process → something went wrong → AI support couldn’t fix it → a human fixed the problem.
But behind the story is a serious lesson about AI in production.
The problem wasn’t simply that an AI system made a mistake.
The bigger problem was that AI had been introduced into multiple parts of the customer journey without apparently providing an equally reliable fallback when those systems failed.
And there is another lesson here.
Sometimes, in our enthusiasm to make a process “smarter”, we make it less predictable.
A traditional select-and-choose interface may not feel particularly innovative, but it has one major advantage:
the customer can see what they are choosing.
A question-driven or conversational interface can be more natural and easier to use – when it works correctly. But it also introduces another layer of interpretation between the customer’s intention and the final transaction.
That distinction matters.
AI doesn’t have to fail spectacularly to cause a problem.
It only has to be wrong at the wrong point in the process.
Effort + wrong tool = no result
When a new technology becomes available, the natural temptation is to look for places where it can be applied.
We have AI. Where can we use it?
A better question is:
What problem are we actually trying to solve, and is AI the right tool for it?
AI isn’t automatically an improvement over an existing process.
Sometimes the old solution is deterministic, predictable and boring – and that’s precisely why it works.
If a process can be handled perfectly with a database lookup, a validation rule, a calculation or a simple workflow, introducing a probabilistic AI system may add risk rather than remove it.
The fact that AI can perform a task doesn’t mean that AI should perform that task.
And this is where organizations need to resist one of the strongest temptations surrounding AI today:
using AI simply because AI is available.
The AI upgrade trap
Imagine that you have a process that already works reasonably well.
It takes five steps.
Customers understand it.
Employees understand it.
The error rate is low.
It is relatively cheap to operate.
Then someone proposes an AI solution.
The new process will be more conversational. More intelligent. More personalized. More automated.
It sounds better.
But what if the new process introduces an entirely new category of errors?
The old process might have required customers to make explicit choices.
The new one interprets their answers.
The old process might have had deterministic validation.
The new one might rely on a model to determine what the customer meant.
The old customer support process might have routed a complicated problem directly to a human.
The new process might place an AI chatbot in front of that human.
None of these changes are necessarily bad.
But each introduces another place where things can go wrong.
And that is the point that is often missed:
Replacing a process with AI doesn’t remove complexity. It can simply move the complexity somewhere else.
The production problem nobody sees in the demo
There is something else that makes AI projects particularly dangerous.
I’m currently writing on a new “piece” of something… and part of it should be place here:
“There is a specific moment that goes wrong in AI delivery more reliably than any other. A model is built. It performs beautifully. It is demonstrated. Everyone is pleased. Commitments are made, and the project moves to production – where the performance collapses to something between mediocre and useless. Nobody lied. The number was real. And yet the demo was, in an important sense, a fiction.”
That’s the uncomfortable part.
The demo might not have been fake.
The model might genuinely have achieved 95% accuracy.
The problem is that the production environment is not the demo environment.
During a demo, we usually control the conditions.
We select the data.
We know what questions to ask.
We know what the model is supposed to demonstrate.
We can repeat the successful scenario.
Real customers don’t behave like that.
Real customers enter incomplete information.
They misunderstand questions.
They use unexpected terminology.
They make typos.
They change their minds halfway through the process.
They provide information that the development team never thought about.
And suddenly the beautiful demo encounters the messy reality of production.
AI introduces a different kind of uncertainty
Traditional software is generally expected to behave deterministically.
Give it the same valid input, and you expect the same output.
AI systems introduce a different relationship with uncertainty.
Even when the model itself is stable, its environment isn’t necessarily stable.
Data changes.
Users change.
Prompts change.
Context changes.
Business rules change.
External systems change.
And users inevitably find ways to interact with the system that nobody anticipated.
This creates a fundamental difference in how we should think about production readiness.
For traditional software, we often ask: Does it work?
For AI, we should also ask: How does it fail?
And perhaps even more importantly: What happens when it is wrong?
That last question should influence whether AI belongs in the process in the first place.
Not every process needs an AI upgrade
There is a tendency to treat AI adoption as a race.
Competitors are using it.
Customers expect it (?).
Management wants AI initiatives.
Employees are experimenting with AI.
So every existing process starts looking like an opportunity for transformation.
But there is nothing inherently wrong with leaving some processes alone.
A process that is already:
- fast enough,
- inexpensive enough,
- predictable,
- auditable,
- and sufficiently accurate
may not need AI.
Replacing it with an AI solution simply because AI is available can be a downgrade disguised as innovation.
This is especially important for processes where correctness matters more than convenience.
If the task is essentially: “Take this clearly defined input and produce this clearly defined output.” then a deterministic solution may be exactly what you need.
AI may be useful around the process of helping users understand their options, answering questions, detecting anomalies or assisting employees.
But that doesn’t necessarily mean AI should control the actual transaction.
Sometimes the best architecture is not AI instead of automation.
It is AI alongside automation, with each technology doing what it is best suited for.
The five questions before production
Before moving an AI solution from prototype to production, I believe organizations should ask five basic questions.
1. What happens when it is wrong?
Not if it is wrong. But, when.
Every AI system will eventually produce an incorrect output.
What happens next?
Does a human review it?
Is it automatically corrected?
Does the process stop?
Or does the incorrect result silently continue downstream?
If nobody has a good answer, you probably aren’t ready for production.
2. How expensive is the failure?
Not every AI error matters equally.
- A bad product recommendation is one thing.
- A wrong glasses prescription is another.
- A wrong financial decision can be more serious.
- A wrong medical or safety-related decision can be extremely serious.
The acceptable error rate depends on the consequences of failure.
There is no universal definition of “accurate enough.”
Accuracy must be evaluated in the context of the business risk.
3. Does AI actually improve the process?
Compare it with the existing solution.
- Is it faster?
- Cheaper?
- More accurate?
- More scalable?
- More useful?
Does it improve the customer experience?
And perhaps most importantly:
Does it solve a problem that actually exists?
If the answer is unclear, why introduce the additional complexity and risk?
4. Has it been tested against reality?
A successful demo proves that the model can work.
It does not prove that the production system will work.
Testing should include:
- real-world data,
- unusual cases,
- incomplete information,
- ambiguous requests,
- unexpected user behavior,
- changing inputs,
- and the kinds of mistakes users are actually likely to make.
The closer testing gets to reality, the less surprising production becomes.
5. Is there a way back?
This may be the most important question of all.
If the AI system starts producing bad results tomorrow, can you switch it off?
Can you revert to the previous process?
Can a human take over?
Can customers still reach someone?
If there is no practical fallback, the bar for production readiness should be much higher.
Human fallback isn’t a failure
There is sometimes an unfortunate assumption that introducing AI means reducing or eliminating human involvement.
But a human fallback isn’t necessarily evidence that the AI project failed.
It can be a risk-control mechanism.
In fact, some of the best AI-enabled processes may deliberately contain a point where a human takes over.
AI can handle the majority of routine cases.
Humans can handle exceptions.
AI can process information.
Humans can make the final decision when the consequences are significant.
AI can answer simple customer questions.
Humans can handle situations where the customer says:
“Something has gone wrong.”
That isn’t an admission that AI isn’t good enough.
It is an acknowledgment of reality.
AI systems will sometimes be wrong.
Designing for that reality is better than pretending it won’t happen.
Production isn’t the finish line
Perhaps the biggest mindset change we need with AI is this:
Production isn’t the end of the project. It’s the beginning of the real test.
A model that performs beautifully in a controlled environment has demonstrated potential.
It hasn’t yet demonstrated resilience.
Once AI becomes part of a real business process, it becomes part of a living system.
Data changes.
People change.
Requirements change.
The business changes.
And the AI needs to be monitored accordingly.
Accuracy shouldn’t be measured once during a project and then forgotten.
It should become part of ongoing operations.
You need monitoring.
You need feedback loops.
You need clear ownership.
You need thresholds that trigger human intervention.
And you need the ability to change, retrain, replace or disable the system when reality proves your assumptions wrong.
Don’t confuse innovation with replacement
Perhaps this is the most important distinction.
AI doesn’t have to replace a process to improve it.
It can augment it.
It can help employees make better decisions.
It can simplify complicated interactions.
It can identify patterns humans would miss.
It can reduce repetitive work.
It can help customers navigate complex information.
But sometimes the best use of AI is to sit around an existing reliable process rather than replace it.
For example, instead of replacing a structured ordering process with a conversational AI system, AI could help a customer understand the available choices – while the actual order still relies on explicit, validated selections.
Instead of replacing human customer support entirely, AI could handle routine questions and route exceptions to humans.
Instead of allowing an AI model to make an irreversible decision, let it make a recommendation that a human can approve.
These approaches may not look as futuristic.
But they may be considerably safer.
The uncomfortable conclusion
The lesson from a pair of wrong glasses isn’t that AI is bad.
Quite the opposite.
AI can be extraordinarily useful.
But usefulness doesn’t eliminate risk.
And introducing AI into a process simply because the technology is available is not digital transformation.
It’s technology looking for a problem.
The best AI implementations aren’t necessarily the ones with the most AI.
They are the ones where AI is used where it creates meaningful value, where its limitations are understood, and where its failures can be contained.
The webshop in this story wasn’t an inexperienced company. It wasn’t a questionable startup. It was an established business with a customer who had already successfully used its services.
And that’s precisely why the example is interesting.
AI risk isn’t something that only happens to companies that don’t know what they’re doing.
It can happen when experienced organizations take a working process, introduce AI into it, and underestimate what happens when the AI doesn’t behave as expected.
So before asking: “Where can we put AI into production?” perhaps we should ask a different question: “Where would AI actually make this process better — and what happens when it doesn’t?”
Because sometimes the most mature AI decision isn’t to deploy a model.
It’s to recognize that the process doesn’t need one.
Effort + wrong tool = no result.
Leave a Reply