How do you evaluate an AI agent that investigates production systems?
Why do AI agents need evals?
Because without them, every release becomes an argument that nobody in the room can settle with evidence. A new model comes out, or someone rewrites a prompt, and the only available test is to ship it and see what customers say. That produces a debate about whether the improvement is real rather than an answer. An eval set exists to answer one question before a release rather than after it, which is whether the change actually works.
What happens when a new frontier model comes out?
If the evals are built correctly, nothing about the process changes. A well-designed eval tests the functionality of the system rather than its internals, so it should not care whether the answer came from a new model, the previous one, or the company's best engineer producing the answer by hand. That independence is the property that turns a model migration into a measurement exercise instead of a research project. The model ships, the evals run, and the decision to adopt it or skip it takes a day or two.
What has an eval caught that would otherwise have shipped?
A cost reduction that quietly traded away latency. The team had been working on lowering the cost of an investigation without giving up accuracy, and the early results looked strong on accuracy. The eval set showed that the same change made investigations take two to two and a half times longer to reach an answer. Scored on accuracy alone it would have cleared. Without a representative eval set, it ships, and the signal arrives later as customers asking why responses have slowed down.
Why is evaluating an incident investigation harder than evaluating a coding agent?
Coding has a scoring mechanism built into the workflow. You write tests, apply the change, run the tests, and a pass tells you the change was good. It is clean and it can be overly simplistic, but the ground truth is already sitting there.
Troubleshooting has no equivalent. A coding task usually has one prompt and many acceptable answers. An incident runs in the opposite direction: one underlying problem produces a hundred different symptoms, and the agent has to arrive back at that same problem regardless of which symptom it was handed. Scoring that requires knowing what the problem actually was, and it requires domain judgment about what a good answer looks like, how much evidence belongs in it, and where it becomes too verbose to be useful.
Where does ground truth come from when the postmortem doesn't have the answer?
This is the hardest part of the problem. In real incidents people frequently never establish what caused the alert. The postmortem gets written, the incident gets closed, and some portion of the explanation is hand-waved. An agent cannot be scored against a conclusion that was never reached.
Getting usable ground truth at volume means having people who understand the system well enough to reconstruct what happened, which is a different and rarer skill than labeling data. The scarcity is the constraint, not the labeling effort.
How do you know the scoring itself is right?
By calibrating the scorer against expert judgment and measuring the agreement. Resolve AI recently rebuilt how investigations are scored and validated the new method against what experienced engineers independently judged to be good and bad work. The new scoring aligns considerably more closely with expert opinion than the method it replaced.
The interesting signal in a rollout like that is what happens when someone disagrees with a score. They read the reasoning behind it, and often end up agreeing with the score rather than overriding it.
Underneath the calibration, the methodology itself gets rebuilt on a cycle. Every three or four months the team re-examines how it decides whether a change is a genuine improvement, which in practice is a long argument about statistics.
How does customer feedback become an eval case?
Through a path built specifically to make that conversion cheap, and a set of checks that stop it from being automatic. When a customer reports that an investigation went wrong, that report gets converted into a representative evaluation case, and the case is checked for quality before it enters the set. The checks carry more weight than the conversion. An eval set that accumulates poorly specified cases stops being a measurement instrument.
Alongside the tooling, a large share of what the team learns still comes from reading trajectories directly. Staring at the data is not a technique anyone would call cutting edge, and it remains one of the higher-yield things the team does.
Why can't an enterprise build this in-house?
The blocking constraint is ground truth rather than tooling. Assembling enough labeled investigations requires people who know the system deeply enough to say what really happened, and inside most organizations those people are a scarce resource that already has a job.
The volume requirement is what makes it a standing commitment rather than a one-time project. Once an agent sits on the critical path for production incidents, every change to it has to clear four bars: it cannot regress quality, it cannot cost materially more, it cannot get slower, and it cannot leave a better available model unused. Answering those four questions for every change is the ongoing work. The eval set is what answers them before a change reaches production instead of after.
