Memory Consolidation Strategies for Long-Running AI Agents

1. Introduction

An agent that operates for weeks or months produces a torrent of interaction logs: tool outputs, user messages, intermediate plans, error traces. Storing every token is feasible; retrieving the right thing later is not. We frame this as a memory consolidation problem: which subset of past tokens should be retained as a privileged, fast-access memory atom, and which should be archived behind cold-storage retrieval?

This paper compares four consolidation policies along two axes: information retention and compute cost.

2. Problem Setting

Let an agent observe a sequence of episodes $E_1, E_2, \ldots$. After each round, a consolidator $C$ produces a memory atom set $M_t \subseteq \mathcal{A}$ subject to a budget $|M_t| \le B$. At test time, a retrieval query $q$ returns the top-$k$ atoms; we measure fact recall: the fraction of ground-truth facts that the agent can recover via $M_t$.

3. Consolidation Policies

3.1 Recency-only

Keep the $B$ most recent atoms. Simple but pathological for long horizons.

3.2 Frequency-only

Keep atoms with the highest historical retrieval count. Strong on stable facts, weak on rare-but-important events.

3.3 Surprise-weighted

Define surprise as $\sigma(e) = -\log p_\theta(e \mid \text{context})$ under the agent's own model. Atoms with high surprise are likely to be informative novel facts. Score:

$$s(e) = \alpha \sigma(e) + \beta , \text{recency}(e) + \gamma , \text{usage}(e)$$

with $\alpha = 0.55, \beta = 0.25, \gamma = 0.20$ chosen on a 7-day pilot.

3.4 Sleep-replay

During idle periods, the agent replays sampled episodes through a summarizer LLM and merges semantically overlapping atoms. Loosely inspired by hippocampal replay [Wilson and McNaughton 1994] and recent neural-replay work in agent literature [Park et al. 2024].

def sleep_replay(atoms, summarizer, k=8):
    clusters = cluster_by_embedding(atoms, k)
    return [summarizer.merge(c) for c in clusters]

4. Experimental Setup

We simulated 27 agents over a 90-day horizon. Each agent received a daily mix of (a) recurring routine queries, (b) one-shot novel queries, and (c) distractor traffic. Ground truth was a curated set of 1,144 facts the agent should still know on day 90. Memory budget was $B = 512$ atoms at $\le 256$ tokens each.

5. Results

5.1 Retention vs. compression

Differences between recency-only and surprise-weighted were significant at $p < 0.001$ (paired Wilcoxon, $n = 27$).

5.2 Sensitivity to budget

At a tighter budget of $B = 128$, surprise-weighted dropped to 84.1% and sleep-replay to 86.7%, while recency-only collapsed to 49.0%. The gap widens as the budget tightens — exactly when a policy is most needed.

6. Discussion

Surprise-weighted consolidation is the practical sweet spot. Sleep-replay's marginal +1.8% recall is rarely worth a 4x compute hit unless agent traffic is light enough to amortize idle replay cycles.

A notable failure mode of surprise-weighting: in adversarial settings, an attacker can inject high-surprise garbage to evict legitimate atoms. We recommend coupling surprise-weighting with a per-source rate limit.

7. Limitations

Our 90-day horizon is short by deployed-agent standards; effects at year scale may diverge. Our fact set is also drawn from a narrow domain (a synthetic personal-assistant scenario).

8. Conclusion

For long-running agents, the choice of memory-consolidation policy can swing fact recall by nearly 20 percentage points at fixed budget. We recommend surprise-weighted consolidation as a default and sleep-replay only when idle compute is genuinely available.

References

1. Wilson, M. A. and McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories.
2. Park, S. et al. (2024). Generative Agents and Memory Streams.
3. Borgeaud, S. et al. (2022). Improving Language Models by Retrieving from Trillions of Tokens.
4. Khattab, O. et al. (2023). DSPy.