Forgotten duties: auditing how coding agents maintain their own context in a queue-driven autonomous loop

Abstract

Long-running agentic work keeps the agent's context in files: rules, a statement of intent, a work queue, notes and logs. In a case study of one user's deployment, we audit, from preserved transcripts, how Claude Code agents keep these files up in cleanvibe, a scaffold for open-ended projects with a scheduled autonomous loop. Across 17 sessions on two scaffold generations (391 loop ticks), the duties agents kept were those whose cue arrives in their context (a launch prompt, a user message): 14 of 14 session-level cases. Duties the agent has to notice for itself, from its own state or its own actions, held in 20 of 35, and the intent file went hours stale while commits continued, before and after a fix that named it in every tick prompt. Cue source was coded blind from the instruction text. The one duty moved from "notice it yourself" to "the launch prompt says it" went from 0 of 8 sessions to 6 of 6, while a same-week control without the change stayed at 0. This is not "explicit prompts beat implicit ones": every duty was stated explicitly and stayed in context, and naming the intent file in every tick prompt did not keep it current, because the prompt arrived but the change that made an update due still had to be noticed. The sample is small and the claim is a hypothesis with a stated prediction, not a general result about language models. The accompanying skill reruns the audit on any cleanvibe user's projects.

1. Motivation

An autonomous loop wakes the agent every half hour to "continue working on the queue"; each wake-up is a tick. Between ticks, what the agent knows about the project lives in files it is asked to maintain: INTENT.md (what the project is for), queue.md (the work), README.md, and a devlog. If those files drift, a later session starts from a wrong picture. We ask which of these upkeep duties agents actually carry out, and what makes the difference. The obvious answer, that agents follow what they are told explicitly and drop what is implicit, does not fit: every duty here is written out in CLAUDE.md, which the agent has in context throughout.

2. Design

The scaffold. cleanvibe creates a git repository with a CLAUDE.md of standing rules, which stays in the agent's context; an INTENT.md the agent is told to keep current; skills for practices such as queue-driven work; and a hook that commits every session's full transcript. Two kinds of message reach the agent during a session: the launch prompt at the start, and one cron prompt every half hour ("commit and push, then continue working on the queue"). Everything else the agent should do is written in CLAUDE.md or a skill and waits there until the agent applies it.

What we measure. A script reads every cleanvibe project under a folder. Per session it records human messages, cron firings, commits, and how long INTENT.md had gone without a commit when the session's last commit was made. We call that interval staleness. We do not judge whether an update was due at a given moment, which would need a judgment about when the agent's understanding changed; staleness only says the file went uncommitted while the agent kept committing other work. Per tick it records which files the tick read or edited, whether it ran date, and whether it committed. A second script rebuilds the committed queue.md at each tick, so a tick that found the queue empty can be classed as having refilled it or not. Rules that cannot be counted, such as quoting the user before recording their view, are coded by a fresh agent that sees only the codebook and the raw transcripts, not our hypotheses or counts; a human spot-checks each violation it reports. Its agreement with our own coding of the first four sessions was κ = 0.69, which is moderate; only the rule outcomes in the Round 2 table rest on this coding.

Two rounds. Round 1 audited 9 sessions on cleanvibe 2.0.0–2.0.2 (312 ticks) and led to eight fixes, released as 2.0.3. Round 2 audited the first 8 sessions on 2.0.3 (79 ticks), with 2 sessions from the same week still on 2.0.2 as a control.

3. Results

Round 1. Of 141 applicable rule-by-session cases, 109 were followed. The failures were upkeep duties that no message named: the weekly update check (missed in 9 of 9 sessions), the intent file (hours behind in 6), the README, and refilling an empty queue (one loop idled for 100 ticks). Only 4 of 312 ticks read INTENT.md.

Round 2: the fixes.

Every idle tick said why it did not refill: one would not pad a question it had only guessed, one was waiting on phone calls only the user could make, one had only a pending decision. These ticks saw the empty queue and declined, which is a different failure from not noticing.

The finding: where the cue comes from. What separates the duties that held from those that lapsed is where the cue that makes a duty due comes from. In some duties the cue arrives: the launch prompt says "run the update check", or the user states a constraint. In others the agent must notice it: its understanding of the project has changed, or it is about to write a time, some prose, or a "done" entry. The instruction is in context throughout in both cases. Two fresh agents, working independently from the instruction text alone and without outcomes, coded each duty's cue and agreed on all of them.

Quoting the user is the exception: it must be noticed, yet held in 7 of 8. Both coders also called recording constraints borderline, since the agent has to recognise a remark made in passing as a constraint. And the tick prompt's "re-read INTENT.md" arrives every tick, yet ticks seldom read the file; the harness keeps a file in context once read, so a re-read may be unnecessary, and we count it neither way.

The update check changed cue, and adherence followed. It is the one duty whose cue moved between rounds. In 2.0.2 the agent had to notice that the weekly check was due, and 0 of 8 sessions ran it (we exclude our own project, which reads the updates page as data). In 2.0.3 the launch prompt names it, and 6 of 6 first sessions ran it. The same-week control on 2.0.2 did not.

4. Discussion

The split matches a distinction from human prospective memory, the study of remembering to act later. In McDaniel and Einstein's multiprocess framework (Applied Cognitive Psychology 14, 2000), an intention whose cue is part of the task at hand is retrieved spontaneously, while one whose cue is not needs costly monitoring, and that monitoring is what suffers when the task is demanding. A launch prompt or a user message is the first kind of cue; a file quietly going stale is the second. Work on long-context recall in language models points the same way: recall drops when nothing in the current text matches the stored instruction (Modarressi et al., NoLiMa, ICML 2025, arXiv:2502.05167). We use the framework to predict which duties lapse, not as a claim about how models work inside.

It also explains why naming the intent file in every tick prompt did not help, the result that separates this account from "make the instruction more explicit". The prompt arrives, and it is as explicit as an instruction can be, but "update it if your understanding has changed" still asks the agent to notice the change. More explicit wording moves the instruction; it does not move the cue. For people who build agent scaffolds, the practical rule is to deliver the trigger condition, not to restate the duty. The fix this suggests is to make the staleness itself arrive. Prediction for the next round: a tick prompt or session hook that states "INTENT.md last changed H hours and N commits ago" will raise intent edits, in ticks where it is over two hours stale, from 6% to most of them; and a hook that flags a shell command writing a Markdown file will cut shell-written prose (now 6 of 8 sessions). We will report the result whichever way it goes.

5. Limitations

One user, one scaffold, one agent model, 17 sessions: enough to describe this deployment and to state a testable prediction, not to estimate how often agents in general keep such duties. The author built the scaffold and is its only user, which invites bias; for that reason the headline numbers come from logs and git, and the cue codes and rule outcomes come from fresh agents that never saw our hypotheses. The cue distinction was drawn after seeing round 2, so round 3 is its first real test. The update-check comparison has one control session, is not randomised, and the 2.0.3 sessions also carried the other fixes, although none of those touches whether the check runs. The coding agents share a model family with the agents they code; the headline numbers (ticks touching a file, staleness, whether the check ran) are counts from logs and git, and only the rule outcomes and the cue codes rest on a model's judgment. Tick reads are detected heuristically from shell commands, but the intent result comes from git history. No session reached context compaction, so the risk the literature stresses most is untested. The transcripts are personal and are not released, so the numbers here cannot be checked against them directly. The skill reruns the same scripts on a reader's own corpus, and the per-session and per-tick counts, with project names replaced by session codes and no conversation text, are prepared for release.

References

• McDaniel, M. A., and Einstein, G. O. (2000). Strategic and automatic processes in prospective memory retrieval: a multiprocess framework. Applied Cognitive Psychology 14(7), S127–S144.
• Modarressi, A., Deilamsalehy, H., Dernoncourt, F., Bui, T., Rossi, R. A., Yoon, S., and Schütze, H. (2025). NoLiMa: Long-context evaluation beyond literal matching. ICML 2025. arXiv:2502.05167.