Serving, observability, cost control and reliability for AI systems that are already in production and now have to stay correct, affordable and defensible.
An AI feature reaching production is the start of its cost curve, not the end. Content drifts away from what the system was tuned on. Providers change model behaviour and pricing underneath you. Traffic patterns shift. Someone discovers a prompt that makes it misbehave. None of this shows up in a launch plan, and all of it lands on whoever is on call.
Cost optimisation is the most common reason we get called about an existing system.
One API key serves eight features, so the invoice is a single number with no breakdown. You cannot tell which feature is expensive, which customer is unprofitable, or which change last month tripled the bill. Attribution has to be instrumented at the call site before any optimisation is meaningful.
The frontier model gets chosen during the prototype and never revisited. A large share of real traffic is routine and would be handled identically by a far cheaper model. Routing by difficulty, with the expensive model reserved for the cases that need it, is usually the single largest saving available.
Stuffing twenty retrieved passages into every prompt costs money on every call and frequently makes answers worse, because the relevant passage gets buried. Tighter retrieval is both cheaper and more accurate, which is a rare combination.
Repeated system prompts, repeated document context and genuinely repeated questions all get paid for again every time. Prompt caching and a semantic cache on common questions are unglamorous and immediately effective.
A failing call that silently retries three times costs four times as much and appears in no dashboard. Retry budgets, timeouts and circuit breakers are basic reliability engineering that AI stacks routinely skip.
Cheap models handle the routine majority, expensive models handle the hard minority, with the routing decision measured against the evaluation set rather than guessed. Quality is held constant while cost falls.
Routing, tiered inferenceCost tagged per feature, per tenant and per workflow at the call site, with ceilings and alerts, so a runaway loop is caught in hours rather than at the end of the billing month.
FinOps for LLM workloadsA system can be fully available and completely wrong. Sampled outputs scored continuously against the evaluation set, with alerts on grounding and abstention rates rather than only on error rates and latency.
Drift detection, online evaluationNo single provider becomes an unremovable dependency. Failover across providers, and the ability to move a workload when pricing, behaviour or terms change, which they do.
Multi provider, graceful degradationFull traces covering retrieval, tool calls, prompt version and model version, so an incident can actually be investigated rather than guessed at from a transcript.
Observability, tracing, replayModel inventory, versioned prompts, change history, evaluation results over time and a documented human oversight path. What an internal audit, a security review or a regulator will ask for, prepared before they ask.
AI governance, audit readinessAI systems sit on ordinary infrastructure, and most of the reliability problems we are asked to solve are ordinary infrastructure problems with an AI system on top.
We take on other people's AI systems. A thirty minute call to understand what you are running, what it costs and where it hurts.