Deciding when to reduce or replace AI

We compare each system against its intended outcome, not its launch metrics. A document-processing pipeline may have every output re-checked by the team it was intended to relieve. In that case, the organisation pays for inference, integration maintenance, and the original manual process. A rationalisation review should measure this review effort explicitly because infrastructure dashboards do not usually show it.

The review covers operating cost, failure modes, review effort, latency, data exposure, and the value the system creates in practice. It also identifies downstream dependencies. A low-value tool used by another process requires a different change plan from one with no active users or dependants.

The review also asks whether a traditional software component, a simpler workflow, or a human decision would provide a more dependable result. Classification, extraction, and routing should be compared with rules engines and smaller supervised models where those approaches can meet the requirement with lower cost, latency, or operational uncertainty.

Inference spend is the visible line, and often not the largest one. The full cost of an AI system includes human review, prompt and model maintenance, evaluation infrastructure, and added workflow latency. It also includes the compliance exposure of the data processed by the system. A system whose API bill is modest can still be expensive once the people around it are counted.

The review must also account for risk. A system that makes consequential decisions without adequate evaluation creates potential liability. A system that sends sensitive data to an external provider without a clear legal basis creates compliance risk. These risks belong in the cost and decision record even when they do not appear as current expenditure.

Rationalisation can produce five outcomes:

• Keep and invest when the system performs well but needs stronger evaluation or monitoring.
• Narrow when it performs reliably on only part of its current scope.
• Migrate when the design remains sound but the model or vendor no longer meets the requirements.
• Replace when a deterministic component can meet the requirement more reliably or efficiently.
• Retire when the measured benefit does not justify the cost and risk.

Narrowing is appropriate when evaluation identifies a subset with acceptable performance and risk. Reducing scope can turn an unreliable general system into a predictable component. Migration reviews address a different condition: model capability, pricing, vendor terms, and support change, so an earlier build or procurement decision may no longer be suitable.

Each decision needs an owner, a sequence of changes, and a way to confirm it is working. Retirement requires engineering work. Dependent processes need rerouting, and the interim workflow needs enough capacity. Data flows must close in accordance with retention obligations, and the system needs a confirmed end date. Systems that remain partially active after retirement can become unowned risks.

The review should update a lightweight portfolio record of what runs, what each system is for, what it costs, and when it is next reviewed. This makes rationalisation a recurring management process rather than an incident-driven exercise.

Reducing or removing AI where it adds cost, compliance risk, or unreliability is a valid engineering decision. A controlled retirement process also provides evidence that the organisation can manage the complete system lifecycle.