Agent memory: the system that makes the agent better every time it runs
Noah Schlager, Member of Technical Staff at Resolve AI, on the manual work memory automates, how the agent knows an environment before the first alert fires, and why generating knowledge is easy but useful knowledge is hard.
What is agent memory for?
Agent memory gives the agent additional context it can use during an investigation. An investigation is only as good as what the investigator knows about the environment, and memory is the standing pool in which that knowledge lives. When the agent picks up an alert, it brings everything it has already learned about the services involved, in addition to whatever the alert itself says.
It's also the mechanism that makes the agent better every time it runs. Each investigation leaves something behind, and the next one starts knowing more than the last. Without that accumulation, every run would start from zero and relearn the same environment.
What manual work does memory replace?
Memory automates the work of a dedicated person. Before this was automated, generating the knowledge required a full-time engineer to go into a customer account, conduct many customer interviews, write documents, and go back and forth with the team until the picture was complete. The work was slow because the knowledge existed only in people's heads, and it had to be repeated with every new account. The big lift memory provides is making all of that automated.
Why does the agent need so much context?
The environments Resolve AI operates in are extremely large, and the context required to work in them is extensive even for a seasoned engineer. Picture an engineer showing up to figure out why a particular alert is firing. They'd want the runbooks, so they know the documented procedures. They'd want a service map, so they know what sits upstream and downstream of the thing that's paging. And they'd want the pile of knowledge nobody ever wrote down, like which alerts tend to be noise and what has broken before.
That's the bar memory is held to. The knowledge Resolve AI holds about an environment should be roughly equal to what that seasoned engineer would need. Anything less and the agent is investigating with less context than the humans it's meant to help.
How does the agent avoid starting cold?
The agent explores the environment before it's ever asked to do anything. Resolve AI proactively maps the customer's environment and identifies dependencies among services, logs, and alerts. So when the first alert fires, the agent already knows which services sit around the one that's paging and which logs are worth reading. The durable set of memory is there before the first investigation needs it.
The same exploration changes what onboarding costs. The weeks of back-and-forth used to exist because knowledge had to be transferred from one person to another. When the agent can get most of it from the environment itself, onboarding comes down to a matter of hours.
How does memory learn from real use?
Through the signals engineers already give while using it. A thumbs-up or thumbs-down on an answer feeds back into the system. So does an engineer directly asking the agent to remember something, and so does a correction made partway through a conversation, the moment where someone says that's the wrong service, check the one upstream.
Failures are part of the loop too. When an investigation goes wrong, that gets incorporated into core memory, so the wrong path one investigation took becomes something the next one knows to avoid. The same mistake doesn't happen twice.
What's the hard part of building agent memory?
Making the knowledge useful. LLMs are very good at generating knowledge in volume. The team could generate endless random knowledge about an environment, and none of it would improve an investigation. Keeping only what helps takes three things.
- It takes a deep understanding of the different types of integrations the knowledge comes from, because a memory is only as reliable as the reading of the source behind it.
- It takes being very opinionated about what's worth learning, which is as much about deciding what not to keep as what to keep.
- It requires a framework for evaluating the knowledge being generated, so there's proof that a given memory actually helps, rather than an assumption that it does.
Without those three, the team is just shooting in the dark.