Long-running AI agents forget their own safety rules fast

Ask an AI agent to compact its memory five times, and it may remember almost nothing of what it was told never to do. A new study finds that Claude Code's built-in /compact feature, running on Sonnet 4.6, retains just 10% of safety rules across 20 production agent configurations after five rounds of summarization. After a single round, retention is already down to 53%.

The paper, "The Compaction Cliff in Long-Running AI Agent Memory," was submitted to arXiv on August 24, 2026, and has been accepted to CIKM 2026, the ACM's information and knowledge management conference. Its central diagnosis is simple: when an agent's context window fills up, standard compaction tools summarize everything at the same rate, treating a compliance rule the same way they treat a log of routine actions.

That's a problem because rules need exact wording to stay enforceable, while episodic logs can tolerate lossy compression without breaking anything. Compressing both uniformly means the instructions an agent absolutely must follow degrade at the same pace as information that barely matters.

A triage-based fix

The researchers propose "Knowledge Triage," a framework built around three operators: TypeCompact, TypeDecompose, and TypeRetrieve. Each targets a different failure mode instead of applying one blanket summarization pass to an agent's entire memory.

TypeCompact is the headline result: it preserves 2 to 4 times more safety rules than the best single-shot LLM compactor at every compression ratio tested, holding 96% recall even after five compaction rounds. Against medical-compliance rules specifically, it beat the production Sonnet compactor with strong statistical significance (paired McNemar test, p < 10^-8, N = 200).

TypeDecompose addresses a related issue: rules getting scattered or entangled when memory is partitioned. Under naive, uniform partitioning, 93% of test cases showed locality violations, meaning related rule components ended up split apart. TypeDecompose brought that down to zero.

TypeRetrieve, meanwhile, tackles retrieval accuracy once memory has been triaged and stored. It reached 100% recall@50 on the benchmark, compared with 73% for the strongest single-shot LLM retriever tested.

A dataset to test against

Alongside the framework, the team released AgentArtifactCorpus, a dataset of 396,934 agent configurations pulled from 54,628 public GitHub repositories. It gives other researchers a large, real-world basis for testing how agent memory systems behave outside curated lab benchmarks.

For teams deploying agents that run for extended sessions, especially in regulated domains like healthcare or finance, the findings point to a concrete blind spot in current tooling. Generic summarization, however capable the underlying model, is not the same as preserving the specific rules an agent is required to obey.