Context engineering isn't enough

Context engineering has become the respectable answer to why AI systems underperform. The framing is appealing because it is mostly true and entirely tractable. If the agent gave a bad answer, the reasoning goes, it lacked something: the customer's plan tier, the state of their account, the policy that applies in their market, the last three things they tried. Give it those, structure them well, keep the window clean, and quality follows.

I've watched teams get very good at this. Retrieval gets tuned. Tools get built so the model can look things up rather than guess. Prompts get rewritten by people who genuinely understand what they're doing. And quality does improve, substantially, until it stops.

The place it stops is instructive. What's left after the context problem is solved is a residue of questions the system keeps getting wrong in ways that feel almost arbitrary. You trace one back expecting a retrieval miss and find the retrieval was fine. The model had everything. It still produced an answer that the business considers wrong.

So you go and ask what the right answer was. And this is where it gets uncomfortable, because frequently there isn't one. There are three, held by different parts of the organisation, none of them written down, all of them defensible, and the difference between them has been absorbed for years by human judgment nobody ever had to make explicit.

The model didn't hallucinate the ambiguity. It inherited it.

Judgment was doing more work than anyone admitted

Every service organisation runs on a quiet layer of discretion. An experienced agent reads a situation, weighs the customer's history against the letter of the policy, and makes a call. Multiply that by thousands of interactions and you have an operating model that is enormously effective and almost entirely undocumented.

That layer works because it is distributed and forgiving. Two agents can resolve near-identical cases differently and nothing breaks. Nobody compares them. The variance is absorbed by the fact that each customer only ever sees one outcome.

An AI system removes that forgiveness. It applies whatever rule it has consistently, at volume, in public, in writing, at three in the morning. Inconsistency that was previously invisible becomes a pattern that someone will screenshot. What used to be individual judgment now has to be a decision the company has actually made.

This is the part I think the industry keeps under-weighting. Deploying AI into service doesn't just automate work. It forces an organisation to render its own policies as executable rules, and most organisations have never done that. They have documents that describe intent, and people who interpret it.

Organisational engineering

The work that unblocks the next tier of quality isn't technical. It looks like this:

• Naming an owner for each decision the agent has to make. Not the AI team. The function that actually holds the risk — pricing, billing, compliance, trust and safety. If no name goes next to a decision, the agent's behaviour is whoever wrote the prompt last.
• Converting intent into rules and accepting the loss. Something is genuinely lost when discretion is codified. The answer isn't to refuse; it's to codify well, keep a real escalation path for the cases the rule handles badly, and revisit the rule when those cases pile up.
• Deciding what the agent is allowed to be uncertain about. There's an enormous difference between a system that guesses and one that is explicitly permitted to say it doesn't know and route onward. That's a policy choice, not a model capability.
• Closing the loop between what the agent does and what the business learns. Every escalation is evidence about a policy gap. Most organisations throw that evidence away.

None of this shows up in an architecture diagram. All of it determines whether the architecture produces good outcomes.

Why teams avoid it

Because context engineering is legible and organisational engineering isn't. You can point at a retrieval improvement in a sprint review. You cannot easily point at four weeks spent getting three functions to agree on what happens when a customer in one market asks for something the policy in another market forbids.

The second piece of work is worth more. It is also slower, more political, and much harder to attribute. So it gets deferred, and teams keep optimising the tractable thing, and quality plateaus somewhere short of where it should be while everyone concludes the models aren't ready yet.

The models are usually ready. The clarity isn't.

What this changes about how you staff the work

If you accept the argument, the composition of an AI service team looks different. You still need people who can build. You also need people who can walk into a room with legal, operations and finance, surface a disagreement those functions have been comfortably avoiding for years, and get it resolved in writing.

That's an unusual skill set and it is rarely what gets recruited for. Job specs ask for retrieval pipelines and evaluation frameworks. They should also be asking who can run the decision-forcing conversation, because that is the constraint on the second half of the quality curve.

My rough test now: when an AI system gives a bad answer, before touching the pipeline, ask whether a well-briefed new starter with access to everything the model had would have got it right. If the honest answer is that they'd have had to go and ask someone, the problem isn't context. It's that the company hasn't decided yet.

Fix the deciding. The engineering gets much easier afterwards.