An investigation is not one task. Triaging an alert means deciding in seconds whether this needs a human, across every alert that fires. Finding the root cause behind a remediation decision means following evidence across systems until a chain holds, on the small fraction of alerts that earn it. Same method, budgets that differ by two orders of magnitude.
Inside either one, the work keeps changing shape. Compressing a million log lines into five patterns is a throughput problem. Reading a latency histogram is image recognition. Writing a scoped PromQL query is code generation in a language that barely appears in training data. Weighing four surviving hypotheses against the evidence is the hardest reasoning in the loop, and it runs maybe twice per investigation.
No single model is best at all of that, and which model leads on each keeps moving. What decides how well automated RCA works is not which model is underneath. It is whether the system knows which model to use for which step, and can tell when that answer changes.
Key facts
- An investigation is many tasks with different cost, latency, and quality budgets. Triage and root cause are the same method at different depths.
- Subtasks demand different capabilities: multimodal chart reading, query generation, high-throughput summarization, adversarial verification.
- Model rankings per task shift with every frontier release, so routing is a standing question rather than a one-time configuration.
- Evals decide the routing. Without them, orchestration is guessing.
- Post-training earns its place where no frontier model sits at the right point on cost, quality, and latency.
What models mean for automated RCA
The models are the LLMs running the investigation. Two things about them matter for automated RCA:
- Post-training on production troubleshooting. Additional training on the investigation domain itself, so the model learns how incidents are actually handled, not just code and general text.
- Multi-model orchestration. Each model gets the work it's good at. Cheap, fast models compress logs and metrics. The strongest models do the hard reasoning over the evidence.
This is different from taking the newest frontier model and giving it a good prompt. The prompt changes what the model is asked to do. It doesn't change which model is doing it, and on most steps of an investigation that is the decision that matters.
Models sit at the base of that architecture. Context gives them something to know, and reasoning structures how they work.
What do investigations demand from a model?
Investigating an alert well takes six distinct abilities, and a model has to get each one from somewhere.
- Hypothesis discipline. Holding several candidate causes at once, choosing the next query that best separates them, updating on the result, and knowing when the evidence is enough versus when to keep digging.
- Tool fluency. Writing correct, scoped queries in each tool's own language, PromQL, SPL, LogQL, Datadog's syntax, and interpreting what comes back. That includes empty results, which can mean the events don't exist or that the query was wrong, and those are very different findings.
- Reading telemetry. Making sense of histograms, percentiles, log formats, and charts, which in practice is multimodal work.
- Temporal reasoning. Lining up deploys, config changes, and symptom onset across sources with different clocks, because "the errors started five minutes after the deploy" is often the most important point.
- Failure semantics. Knowing how retry storms, connection pool exhaustion, cache stampedes, and hot shards actually present in telemetry, not just what the terms mean.
- Calibrated uncertainty. Reporting how confident it is and meaning it, because incident costs are asymmetric. An incorrect hypothesis stated confidently can redirect the whole response.
This list is a description of how a strong senior engineer works. Watch one work a hard investigation. They start with what changed, because they know most outages follow a change. They hold two or three theories at once and run the one query that would separate them, in a query language they know cold. They glance at a latency chart and dismiss it, because that spike happens every night at batch time. They notice the errors began four minutes after a deploy and treat that as the strongest evidence they have. And when they hand off, they say how sure they are and what would change their mind, so the team knows how much weight to put on the answer.
Tool fluency, reading telemetry, and temporal reasoning come with a general model largely out of the box. The other three are where models differ, and none of them is generic intelligence. Some of it is capability that varies by model, and some of it is behavior that no general model was trained to have. Those two problems have different answers. Capability differences are handled by routing each task to the best model for it. Missing behavior gets built in the harness or trained into the model.
Why general models struggle with production investigation
There are four reasons general models fall short here.
1. They were trained for a different task
Chat training optimizes for producing a good final answer in one turn. Investigation means deciding what to look at next, over and over, and updating on what comes back. That's why an unaided model responds to an incident with an immediate answer. Its training pushes it toward answering rather than investigating.
In practice: the agent stops at the first correlated anomaly and reports it as the cause.
2. The training data is biased, not just missing
Postmortems are written after the incident by the people who solved it. They keep the conclusion and discard the search; they make messy investigations look linear, and they select for interesting failures. Nobody writes one about a full disk or an expired certificate, so a model trained on writeups arrives with the wrong base rates over causes and mis-ranks hypotheses from the first step.
In practice: the agent blames the component that's usually guilty, often the database, whether or not the evidence points there. It also writes queries that parse and run but use the wrong metric name or aggregation window, then reasons over the plausible-looking result with no error signal anywhere.
3. They collapse to the familiar answer
Dozens of distinct causes produce "latency is up," so the job is holding many candidates and gathering evidence to tell them apart. Next-token prediction jumps to the most familiar candidate instead. Assistant tuning makes it worse twice over. Raters prefer decisive answers, so the model overstates certainty, and raters prefer agreement, so it confirms whatever theory the engineer floated in the channel.
In practice: the agent agrees with whatever theory the engineer floats in the incident channel, and it hedges everything equally, so its confidence carries no information.
4. No one model is best at everything
Rankings shift with each release, a release that improves one task can regress another, and betting the whole investigation on one model locks in that model's weak subtasks permanently.
In practice: a model swap changes the agent's behavior mid-quarter, and nobody can say why quality moved.
Why one model can't cover the whole investigation
An investigation is made of very different tasks:
- Compressing raw logs and metrics into patterns.
- Generating scoped queries in each tool's language.
- Reading charts and dashboards.
- Synthesizing hypotheses across the accumulated evidence.
- Verifying the reasoning chain before it ships.
Orchestration means assigning a model to each task. Evals decide the assignments, so which model summarizes best and which reasons best gets measured, not guessed.
Why the small models matter
Most of the work in an investigation is reading data, not producing text. The bulk of the tokens goes to summarizing, clustering logs, deduplicating errors, and pulling anomalies from metric series. None of that needs frontier reasoning, and running a frontier model on raw telemetry costs a lot, while a small model does just as well.
Take one alert investigation. It pulls thousands of log lines, a dozen metric series, and traces from five services, and nearly all of that has to be compressed into a handful of findings before any reasoning happens. Run that compression through a frontier model and each investigation costs enough that you ration them. Run it through small models and the cost is low enough to investigate every alert.
Cost decides how much you can investigate. If each investigation is cheap, you can afford to investigate every alert in the background. If it's expensive, you only investigate the incidents someone declares.
Cheap models also make parallel work affordable. The system can investigate many hypotheses across many data sources at once, whereas a single expensive model that does everything works through one thread at a time.
Handling new model releases
The best model overall and the best model for a specific task are different questions, and the answers to both change roughly every three months. New versions also behave a little differently, in how they call tools, how much they write, and how quickly they act. Across a 40-step investigation, small differences add up.
For example, Resolve AI benchmarked Claude Sonnet 4.6's adaptive thinking in production incident investigations and found that, at a medium level of reasoning effort, it came close to Opus 4.6 at a fraction of the cost. A result like that means the cheaper model can take over tasks where the difference doesn't matter, and the eval suite is what confirms the switch is safe.
You can't detect this from production metrics, because incidents are too rare and MTTR too noisy to compare one model against another. What works is adopting new models task by task, with evals confirming each change, and keeping specialized models on the tasks frontier models handle poorly.
What this looks like in practice
On a live incident, the models divide the work:
- Small, fast models cluster and compress the raw logs and metrics into a handful of patterns worth reasoning about, and their cost makes it affordable to fan out across many data sources in parallel.
- The post-trained model writes scoped queries for each tool, reads the results, and identifies how the failure manifests.
- The strongest model weighs the possible causes against the assembled evidence and ranks them, with the evidence attached.
- A second model checks the reasoning chain before the engineer sees it, catching the anchoring that a single model can't see on its own.
The outcomes follow. Fewer confident wrong answers, upgrades that land without surprises, and a cost per investigation that stays low and predictable enough to investigate every alert, not just the declared incidents.
Why is post-training essential?
Orchestration decides which model runs each task, but it can't create abilities that no model has.
For the steps that run on every alert, under latency pressure, on domain-specific input, no frontier model sits at the right point on cost, quality, and latency. That is the gap post-training fills. A model trained on the task can hold quality at a fraction of the cost, which is what makes running the loop on everything affordable rather than only on declared incidents.
The alternative is post-training: taking a capable general model and training it further on the investigation domain. What that changes:
- The model learns how failures present in telemetry, so a symptom maps to the production failure modes that actually cause it, rather than to the most familiar story.
- Its sense of which causes are likely comes from real production incidents rather than write-ups, which fixes the base-rate problem.
- It becomes fluent in the query languages and tools of production, so its queries ask the right question, not just a well-formed one.
- Its confidence starts to mean something, because it has been checked against investigations with known outcomes.
- It learns to investigate rather than answer, working step by step and updating on evidence instead of committing to the first plausible cause.
Resolve AI post-trains frontier models on production-specific data, the procedural knowledge experienced engineers use to debug production, which never appears in general training sets. It generates synthetic incidents in simulated environments, allowing investigations to be practiced and graded at scale. And because production rarely offers a clean, right answer, engineers disagree, and postmortems are revised months later, we built our own verifier models to assess the quality of open-ended investigations. That grading machinery is the same as what a real eval platform needs, which is why the models layer and the evals layer are built together.
The training happens on the Resolve AI side, not on your data. The general skill lives in the model's weights, and your specific systems live in the context layer, so the product doesn't need to train on your incidents to investigate your stack.
How to evaluate the models behind an automated RCA tool
- How is the work split across models, and what decides which model handles which step?
- When a new model ships, what has to happen before it reaches an investigation?
- Can the vendor explain what their models were trained on, or is it a general model with a prompt?
- Is it one model or several, and how is the work divided across them?
- What happens when a new model comes out, and how do upgrades land without breaking what works?
- How do they measure whether their models investigate well?
- Does the stated confidence mean anything? Ask whether it's calibrated against real outcomes.
- Do they need your data in their weights? Skill should live in the model, and your systems should live in the context layer, so per-customer adaptation shouldn't mean training on your incidents.
Almost nobody in the category answers these questions in public. The stock answers are "proprietary models" or the name of a frontier partner, and neither tells you what the system actually knows how to investigate. Until that changes, these questions belong in the sales call, with follow-ups.
How Resolve AI approaches models
Resolve AI's agents use the best model for the job, with each investigation step routed to the model suited for it. Resolve AI also post-trains its own models on production troubleshooting.
New releases go through the eval framework before they're adopted, task by task, so the system absorbs frontier progress without regressions.
The results hold up in production. DoorDash Ads reduced time to root cause by up to 87% with Resolve AI, and Salesforce reduced it by roughly 60%. Book a demo to see the agents investigate on your own stack.
FAQ
What models do automated RCA tools use?
DIY setups typically call a general frontier model through an API. Production-grade systems combine post-trained models for troubleshooting with frontier models, routed per task. What separates tools is the training and the orchestration, not which model logo is underneath.
Why can't a frontier model do RCA out of the box?
Its training optimizes for producing a confident answer in one turn, and its data records conclusions rather than investigations. So it explains failures fluently, anchors on familiar-looking symptoms, and commits early instead of running the multi-step search an incident actually needs.
What is post-training for production troubleshooting?
Additional training on the investigation domain. Continued pretraining on tools and query languages, supervised learning on real investigation trajectories, fault-injected incidents that provide validated ground truth, and reinforcement learning that scores the investigation path, not just the final answer. The model learns the procedure and how failures present in telemetry, not just definitions.
What is model orchestration in automated RCA?
Splitting the investigation into tasks and routing each to the model suited for it. Small, fast models compress telemetry, post-trained models run the investigation steps, and frontier models handle synthesis. It keeps costs predictable and lets new releases slot in where they win.
Does a new model release automatically improve automated RCA?
No single model is best for automated RCA. The best model overall and the best model for each investigation task are different questions, and the answers change with every release. Orchestrated systems test each release against investigation evals and adopt it task by task, where it measurably wins.
