Adding AI to a business platform is rarely the thing that fails. What fails is the assumption underneath it: that an AI feature is one more service call, and the rest of the system can stay as it is.
It can't. A conventional backend is built around deterministic rules — the same input produces the same output, every call either succeeds or throws, and a workflow is a fixed sequence someone drew once. AI breaks all three assumptions at the same time. And the parts of your system that break are not the parts anyone expects.
Three assumptions that stop holding
Determinism. The same request will not always produce the same answer, and that isn't a bug you can configure away. Temperature is only part of it: context ordering, tool results that changed since yesterday, and even parallel execution on the GPU can shift the output. Any code path that compares two runs for equality, caches on the assumption that identical inputs give identical results, or writes a test asserting an exact string will start failing intermittently.
Fixed sequences. Traditional workflows follow the same path for every case. The value of AI in a process is largely that it doesn't have to: it can read the incoming data, decide which step is actually appropriate, and take a different route for an unusual customer request than for a routine one. That only works if your orchestration layer can express "it depends" — and most can't, because they were written as state machines with every transition enumerated in advance.
Binary outcomes. A deterministic call succeeds or fails. An AI call can also succeed badly — return a confident, well-formed, plausible answer that happens to be wrong. There is no exception to catch. If nothing downstream validates the result, that answer becomes a business decision.
What the backend has to grow
Orchestration, not just invocation. Calling a model is one line. Deciding which model, with what context, after which prior step, and what to do when the answer is unusable — that's a layer, and it needs to exist somewhere explicit. Model routing, retries with different prompts or different models, and fallbacks belong in that layer, not scattered through business code.
Context assembly as a first-class concern. The model's answer is only as good as what reached it. That means retrieval pipelines over both structured business data and unstructured documents, and a deliberate decision about what makes it into the window and what doesn't. "Pass in everything we have" is not a strategy — it's expensive, it's slower, and past a point it makes answers worse, because the relevant fact is buried in noise.
Asynchrony and variable workloads. Model calls take seconds, not milliseconds, and the variance is wide. A synchronous request-response design that worked when every call was a database query will start timing out at the edges. Queues, job status, partial results and progress reporting stop being nice-to-haves.
Observability built for this, not adapted to it. Standard APM tells you latency and error rate. For AI workloads you also need to see token usage per call, which model actually served the request, what context was assembled, what the model decided, and which downstream action it triggered. Without that, three things become impossible at once: controlling cost, debugging a wrong answer, and explaining to a stakeholder why the system did what it did.
Agents make the dependency problem explicit
An agent that performs a single action is a function call with extra steps. Agents earn their complexity when they coordinate several actions — and that surfaces a problem most workflow engines already solved for deterministic steps but not for probabilistic ones.
Order matters, and some of it is discoverable only at runtime. Validating customer information has to happen before creating the order. Two independent lookups can run in parallel. But with an agent, which steps are needed at all may depend on what earlier steps returned — so the dependency graph isn't fully known when the workflow starts.
State has to survive. Completed actions, intermediate results, failures, pending work: if this isn't tracked outside the model's context, a long workflow either loses its place or repeats steps it already did. Repeating a read is wasteful. Repeating a write is an incident.
Failure needs a policy, not a retry. When one step fails, "try again" is only one of four reasonable answers. The others are: take a different route, pause and preserve state, or hand the case to a person. Which one applies depends on whether the action was reversible and how expensive it is to be wrong — and that's a business decision that has to be encoded, not inferred.
Knowing when to stop is a feature
The most useful thing we've seen in production AI workflows isn't higher accuracy. It's the system knowing when not to proceed.
Automation doesn't have to mean removing people. It means routine cases run unattended while the unusual ones reach someone who can judge them — and the system distinguishing between the two on its own. Practically, that comes from two sources working together.
The first is the model's own signal: ambiguous input, incomplete data, a case that falls outside the conditions the workflow was designed for. The second is business rules that don't care what the model thinks — high transaction values, compliance-sensitive operations, anything irreversible. These escalate on the rule, every time, regardless of confidence.
There's a second-order benefit that takes a few months to show up. Once escalations are logged with their reasons, the pattern of what gets escalated becomes the most honest specification you'll ever get of where automation is weak. Categories that escalate constantly are either badly defined or missing data. Categories that never escalate are candidates for wider autonomy. Neither is visible if escalation is an unlogged fallback.
What this means in practice
If you're planning AI in an existing platform, the questions worth answering before writing code are less about the model than about everything around it:
- Where does context get assembled, and who decides what's in it?
- What validates the model's output before it becomes an action?
- Which operations are irreversible, and what stops the system from performing them unattended?
- Where is the escalation path, and does anyone actually watch it?
- Can you reconstruct, three weeks later, why the system made a particular decision?
None of these are AI questions. They're architecture questions that AI makes unavoidable — and answering them badly is what turns a promising pilot into a system nobody trusts enough to leave running.
Planning AI in an existing platform? We build AI into business systems and the architecture around them — orchestration, context pipelines, observability, and the controls that keep it accountable. Get in touch.