Building agents for on-call and incidents?
Why does AI flounder with production when it is so great with code? The answer is simple to state but hard to build for: Production is a distinct problem. Not harder; an entirely different problem.
Running software is where engineering time actually goes. About 70% of it, spread across maintaining platforms, scaling infrastructure, and monitoring deployments, as well as debugging incidents, shipping hotfixes, responding to alerts, updating runbooks, patching vulnerabilities, and restoring service.
None of that work belongs to a single team, either. One engineer can deploy a fleet of agents to build a new product in a day, but a single production incident routinely spans app, infra, database, and platform ownership before anyone can find the cause. A typical Sev1 pulls in eight senior engineers over 100 minutes: L1 support, infra, two app teams, database, an eng director, a comms manager, or an incident commander. No one has the full picture.
So why have we “solved” code with AI, while leaving production alone? Because code cooperates. It's self-documenting: names, types, and tests say what everything is supposed to do. It's modular, bounded, structured units a model can hold in context and reason about as a whole. It's one domain, the same material in every repo. And the feedback loop answers in seconds: write, run, pass, or fail.
Production is different on all four counts:
None of the four shows up in an afternoon prototype where it’s one engineer, one session, a narrow alert set, and nothing at stake.
A DIY agent can solve your trivial production problems. Point a frontier model at prod, give it more context, more tokens, more reasoning on every problem, and it demos beautifully, because a frontier model can do almost anything passably once.
However, production is where “passably” and “once” aren’t good enough. The answer has to be right, not plausible. There are no shades of grey, and it has to be right again tomorrow, on a different service and after a handoff. And if you err on the side of a plausible answer, production is unforgiving and where costs and mistakes compound.
Getting from passably once to reliably every time takes a purpose-built agentic harness, the layer of systems around the multiple models that a demo never needs. DIY builds skip it, and the six resulting gaps sit below the waterline.
Tribal knowledge lives in individual team prompts, skills, code repos, knowledge sources, and with individuals, and the scattering follows the org chart. Each team owns a set of services, so each team captures the operational knowledge for those services in its own runbooks, context docs, and, these days, markdown files written for agents. Every team running its own AI setup makes it worse: in 2026, that's a CLAUDE.md, a prompt folder, and a skills directory per team. Your agent can only see the pile that your own team gave it.
Fixing it means two jobs:
The gathering isn’t a one-and-done type of task, either. Services get renamed, deploys rewire the dependency graph, and the runbook you indexed in March is wrong by April. A knowledge base that has drifted is worse than none, because the agent trusts it.
Production needs a graph of services, dependencies, deploys, and team knowledge that stays current, and most DIY builds don't survive that requirement.
Frontier models are designed to produce the most coherent answer to any prompt. In general tasks, that's exactly what you want, because there are many right answers and coherence is the quality bar. Production investigations are the opposite case: there's only one right answer, and what proves you found it is a causal chain of citable evidence leading from symptom to root cause, not a fluent explanation.
Without an architecture built to force that chain, anchoring bias takes over. The model over-indexes on the first plausible answer, stops looking, and presents the confident wrong answer. In some cases, the on-call engineer applies the wrong fix, which can become catastrophic. But in most cases, the on-caller becomes the tester of the agent's first guess: the human is the investigation loop. Verifying a confident hypothesis against production can take longer than investigating unaided. In both cases, trust quickly erodes, and teams begin to quietly abandon the tool.
A multi-agent system that actually tests alternatives leaves a different artifact:
A first-guess system produces none of that.
General-purpose models aren't post-trained for production troubleshooting. Their weights come from documentation and tutorial-grade fixes, so when a symptom matches a common cause, they anchor on it. For the tasks at the core of an investigation, no frontier model cuts it. Those need models purpose-built for production reasoning, where no two production environments are the same.
Building custom models is a research project in its own right. An open-ended investigation has no answer key to score against, so you need custom verifier models to judge them. The verifier itself is a trained system built on extensive investigation data most teams don't have.
Then there's selecting between models. Every choice trades off quality, latency, and cost, and production investigation is many tasks, not one: reasoning over a dashboard image is different work from reasoning over logs, and a routine triage doesn't justify the model a novel Sev1 needs. Getting all three right means breaking the problem into tasks and matching each to the model that wins its tradeoff. And the picks don't stay right, either. Models change every month, so the matching is continuous.
What the domain needs is all three running as one system: models post-trained on production investigations, verifiers trained to judge them, and an orchestrator that updates as models change. A DIY build takes each on as a permanent job.
With no way to measure whether a change improved or degraded the agent, teams change blindly. You can't tell whether you're at 50%, 70%, or 90% accuracy without measuring, and every model upgrade becomes a blind migration.
The fix is measurement infrastructure, not a prompt change. A production eval platform needs to confirm five things:
It can do that through positive and negative evals, alongside calibration. Positive evals let you know the agent correctly identifies the root cause; negative evals and calibration catch different failures. Negative evals capture cases where the system should have said it doesn't know. Calibration catches the case where it says 90% confident and is right 60% of the time.
Evals also become rot. Models get stronger and saturate the cases, and your environment drifts away from the conditions they encode. Eval generation has to run continuously, regenerating cases to track what production looks like now, not a golden set you build once.
Agents spend tokens at every step of an investigation.
The bill also arrives twice. Model usage grows because the logs are the tokens, and observability costs grow because the agent is hammering the same metered query APIs your engineers use.
At demo volume, ten investigations a week, nobody notices. At production volume, a frontier model on every task does not survive a budget review. The pattern has a name, tokenmaxxing: more context, more tokens, more reasoning on every problem, because that's what made the demo work.
The build that survives that review targets a flat cost per investigation, whether you run ten a week or thousands. That means retrieving only what the current hypothesis needs, rather than stuffing the context window. Telemetry is infinite, but your budget isn't.
Agents with production access are new principals in your security model, and most DIY builds treat them as scripts running under a human's credentials. Each agent needs its own identity, with permissions scoped at two levels:
An engineer with read access to one namespace shouldn't inherit cluster-wide reach because the agents' service accounts have it.
Integrations are where that scoping gets hard. The fastest DIY path is to wire up general-purpose connectors, and for a demo, it works. But each integration needs to run with least-privilege credentials scoped by your team, revocable at any time, with reads and writes on separate rails. Every integration that bundles those together or defaults to broad OAuth scopes creates a surface area that credential scoping alone doesn't close in a system that already has production access.
Write operations are where the architecture either holds or doesn't. The end state of any production agent worth building is closed-loop remediation, where the agents don't just surface a fix but execute it. Getting there safely requires a hard separation between the models that reason and the engine that acts, plus a policy layer that sorts every action into one of three categories:
A DIY build that collapses those into a single approval gate, or skips the policy layer entirely, can never safely expand its autonomy. The categories have to be designed from the start, because the system's ability to earn trust over time depends on the precision of that boundary.
Secrets, tokens, and customer data leak into logs despite best efforts, so telemetry needs a redaction layer before any of it reaches the models. The layer sits at the boundary, stripping sensitive fields before data transmits anywhere, with customer-configurable patterns on top of the defaults.
Audit is where DIY builds fail first. Most harnesses log tool calls, and auditors ask why an action was taken, not just what it was. The system that passes can produce, on demand, every action the agents took in production, the reasoning behind each one, and the human who approved it.
Underneath all of it sits infrastructure governance. A DIY system can send production telemetry to model providers on every investigation, so data residency, retention, and key ownership questions don't go away; they move to your LLM vendor relationships. None of this is a milestone you pass once. Credential rotation, access reviews, and evidence collection are standing work, which is why a compliant AI layer is a build measured in quarters, not sprints.
A team that has read this far might still build anyway. Gather the knowledge, add the evals, and then watch the token bill. Each gap looks like a line item you can work through. The problem is that the gaps don't fail one at a time. They combine during incidents and form the same three shapes every time.
The failures below aren't hypotheticals. They're the pattern we see when teams run DIY agents against real production, and each one traces back to several gaps at once.
The system surfaces a hypothesis. Well-formatted, cites real log evidence, sounds authoritative. The junior engineer on call acts on it. The actual root cause was three services upstream. Cost: 40 minutes of a live incident.
Nothing here is hallucinated. Every citation checks out. So spot-checking the evidence, the standard verification habit, passes. A senior engineer has seen enough incidents to distrust a tidy answer. A junior engineer checks the citations, verifies they are real, and acts. The failure selects for the person least equipped to catch it.
The root cause sat three services upstream because the agent reasoned one hop at a time. Each piece of evidence was correct for its service, but the conclusion was wrong for the system. Models optimize for coherent output, and in production, coherence and accuracy diverge under pressure and incomplete information.
The tool works for Engineer A on their services in their session. The investigation lives on one laptop. Context resets on every handoff. Even when it works, it only works in a very limited form.
Production is a multiplayer problem, and most AI tooling is single-player by design. Local tools scope state to a session on purpose, for privacy and simplicity. Making an investigation durable and shared changes the data model, the permissions model, and the product. That's a rebuild, not a feature request.
A shared investigation has to persist on two timescales:
Either way, the investigation is a team artifact: an evidence trail, conclusions, and a causal chain that outlive the session and stay accessible to the next on-call engineer.
An agent with tool access and a planner executed an action based on an incomplete understanding of the system. It missed a nuance in a critical batch job that another team had mentioned on Slack months earlier. The "reasonable" action caused a system failure because the deciding context was institutional and never in the logs.
The action was defensible on every piece of data the agent could see. The failure was a lack of knowledge the agent had no way to know existed, and a smarter model doesn't fix that.
Getting the right context in is a knowledge architecture problem: raw APIs are messy and not designed for machine consumption. Keeping bad actions out takes approval gates, which are harder to build than the AI itself and which most teams treat as afterthoughts.
DIY can be done in a day for a few hundred dollars in tokens. An actual AI for production agent takes months and millions of dollars to build.
Let’s price it for a mid-size company: 200 engineers, a regulated industry, and standard alert volume. Year one lands around $5M, and steady state runs about $4M a year after that. Where the money goes matters more than the total:
| Line item | Ballpark | What it covers |
|---|---|---|
| People | ~$1.8M year one | A 5-person build team (3 AI engineers, platform/security, UI/UX) for a three-month v1, then 2.25 FTE of maintenance |
| Evals | ~$1.8M year one | ~$300K to build the platform, then a 2.25 FTE program (eval platform eng, quality/ML eng, SRE annotation time) plus compute and tooling |
| Infrastructure | ~$1.2M/yr | Cluster compute times a 2-2.5x multiplier for storage, transfer, and managed services |
| Ops tooling | ~$350K/yr | Observability, security, and compliance tooling, each around 15% of infra spend |
| LLM tokens | <$100K/yr | ~$60 per alert and ~$150 per incident investigation at production cache rates |
Tokens, the line everyone budgets for, is the smallest number on the list. A hundred investigations a month run about $6K. That assumes the context discipline from the token explosion section, though. Without it, this line stops being small.
The real spend is on people. Steady state is roughly 4.5 FTE of specialized engineers before anyone touches a feature. The maintenance half is required regardless of how well v1 ships:
The eval half is where hiring gets hard. Building agentic evals is research-grade ML systems work: trace-replay infrastructure, LLM-as-judge calibration, statistical significance testing, and meta-eval to ensure eval scores track production outcomes. The people who've done it work at frontier labs and on senior FAANG infra teams, at $600K+ fully loaded. Hiring juniors at startup salaries gets you flaky eval runners nobody trusts.
And all of this buys a functional system, one that investigates alerts, calls tools, and produces reasonable triage. Reaching state of the art adds what the line items don't price:
Teams that get there typically spend 18-36 months past "functional" before the system is trusted to act on its own. The demo is the easy 80%. Investigating production is the 20% that's 100x the work.
That harness is what Resolve AI is. Post-trained models, a knowledge graph that stays current, causal reasoning with the evidence attached, governed actions behind approval gates, a learning loop, and the eval platform underneath, running as one system instead of six projects.
It's working in demanding production environments. Zscaler runs 150K+ monthly alerts through it. Coinbase cut MTTR by 72% across 100+ engineers. DoorDash took investigations from 40 minutes to 5.
The operating model is the one this whole article points at: agents should run and fix your software, and engineers should run the agents. That only works when the agents have earned it.
Wire a model to your logs, and you'll have a demo by Friday. That part was always true. Whether it becomes something an engineer trusts at 3 am depends on the harness around the model, and now you know what that system costs. If you'd rather not spend 18 months building it, we should talk.
Join our engineering leads for "Behind the Build", a webinar series deep-dive into how we built agents that run software.
Varun Krovvidi
Product Marketing Manager
Varun is a product marketer at Resolve AI. As an engineer turned marketer, he is passionate about making complex technology accessible by blending his technical fluency and storytelling. Most recently, he was at Google, bringing the story of multi-agent systems and products like Agent2Agent protocol to market
The question isn't whether AI belongs in production anymore. Here's what engineers at AWS Summit NYC 2026 told us about how agents run your software, what guardrails they want, and how the pricing should work.
A frontier model can produce a thousand coherent answers. Most enterprise work needs exactly one correct one, and closing that gap is not a bigger model. It is the agent architecture around it. Here are the six layers that turn open-ended capability into a defined outcome, and why production incidents are the hardest test of whether they work.
Watch how Resolve AI investigates a service timeout from application logs through Kubernetes pods down to failing memory modules in a UCS blade - building a complete causation chain in 3 minutes. See the stark contrast between traditional multi-team incident response (4 teams, multiple tools, hours of coordination) and AI-native investigation that maps dependencies from app code to storage infrastructure without organizational handoffs. Learn why engineering silos slow incident response and how AI agents can reason across the entire production stack as one connected system.
