Automatic memory for Claude Code: puts the notes of the vault that bear on a prompt in the agent's context

⚠️ Breaking change. The pre-vault TOML memory-record storage layer (
~/.local/share/engram/memory/) was removed. Engram now writes only to an agent-memory Obsidian vault. Migration from the old layout is not automated. An LLM should be able to migrate easily.
Engram gives Claude Code and Pi agents persistent memory via a zettelkasten-style vault. All six skills — recall, learn, please, route, curate, and write-memory — ship as SKILL.md installed by engram update. Each skill may additionally carry a companion skill-claude-<name> vault runbook note (key claude:<name>) offered by engram update/engram register-skills and curated like any other pending offer. Registration also offers runbook notes for every other skill, command and Pi prompt template the harnesses load by default: user, synced, plugin, Pi-configured, and project-local sources, namespaced by source (e.g. skill-superpowers-brainstorming, skill-project-github-com-toejough-engram-cmd-opsx-apply). please orchestrates end-to-end work by sequencing recall, learn, and other skills around a user's <ask> (matched by the literal triggers /please, take this end-to-end, please), route encodes the delegate-everything doctrine please draws on to staff its subagents, curate judges engram serve's pending-offer notes against the host vault (matched by the literal triggers curate, /curate, pending offers, pending offer), and write-memory executes the vault-write commands recall/learn invoke natively at their write sites (parents judge, the worker writes). The please and route skills are pure meta-orchestration.
engram update installs the skills into Claude Code and Pi, and the vault is harness-agnostic — so recall and learn work the same on each. Automatic sweeping of raw session transcripts into the chunk index reads Claude Code JSONL and Pi session files.
After a few months of use, the vault's wikilink graph looks like this in Obsidian — each dot is a note, each line a [[wikilink]]; dense clusters are groups of related notes, and the connective tissue reflects thematic proximity:

Screenshot pre-dates the 2026-07-10 vocab→tags migration (#678): the ~25 vocab term-note hubs visible here no longer exist — vocab membership now rides tags: [vocab/<term>], not wikilinks, so a fresh graph view no longer shows these hubs.
Requires Go 1.25+, git and git-lfs on PATH: engram update builds the binary from the engram checkout it runs in, or from a fresh clone when there is none, and refuses a clone whose embedded model is a git-lfs pointer.
go install github.com/toejough/engram/cmd/engram@latest
Make sure $GOBIN (or the first $GOPATH entry's bin, default ~/go/bin; go env GOBIN GOPATH shows them, go env -w settings included, and engram update installs there too) is on your PATH. This proxy-built binary is only a bootstrap: its embedded model is a git-lfs pointer (#645), and step 2 replaces it with a build from a clone.
engram update # install / refresh
engram update --with-guidance # also sync guidance docs (recall.md, delegate.md, learn.md, shim.md) to engram-owned roots; --with-guidance is a one-time opt-in: once any guidance file is imported, plain `engram update` keeps all of them current
engram update --dry-run # show what would change, without writing
engram update --allow-downgrade # override the local-mode refusal to install an older revision than what's installed
engram update syncs engram artifacts to engram-owned roots per harness (~/.claude/engram/, ~/.pi/agent/engram/) and materializes them as symlinks in each harness's discovery paths (~/.claude/skills/, ~/.pi/agent/skills/, etc.). Removals from the source propagate on every update. First update performs a dark migration of pre-existing copies to symlinks. Run it again any time to upgrade — it also reinstalls the binary via go install, then re-execs the fresh binary so the sync itself runs with the new logic (792a11e The rest of the update runs in the new binary). --with-guidance additionally syncs guidance docs to the root's guidance/ subtree (canonical paths) and materializes symlinks; compat symlinks at flat paths (~/.claude/engram/*.md) keep existing @import lines in CLAUDE.md and AGENTS.md resolving (Claude Code + Pi; opt-in). It's a one-time opt-in per file — once your CLAUDE.md (or AGENTS.md) imports a guidance file, plain engram update keeps it current. Until then, plain engram update prints a one-line hint. See install-and-update for the deployment-as-sync behavior.
On Claude Code, engram update also turns on automatic memory, the engram-memory mod: from the next session on, each prompt arrives with the vault notes that bear on it, with nothing else to run. After each turn the lessons it holds become notes and the conversation is indexed for engram query (a headless claude -p run is judged and indexed only with ENGRAM_AUTO_CAPTURE_SDK=1). At a prompt Claude Code shows you at most four lines in the session: the notes delivered, the notes written, the lessons left out, and what could not be done (a delivery, write or sweep, with the reason); the agent reads none of them (a headless run receives them as ui_log events of its JSON stream, and prints none as plain text). A conversation too long to index within the time of a sweep is left alone, and you are told once to run engram ingest --auto. engram update --dry-run lists it without installing it, and Pi is not given it.
| Skill | What it does |
|---|---|
recall | Surfaces relevant notes and raw chunks via a single engram query call: a clustered relevance channel (recency-biased per-phrase cosine over notes+chunks → bounded matched set → one unified chunk+note clustering that builds candidate_l2s from within-cluster top-5 only) plus an explore channel (notes sampled from vocab-term centroids by proximity to the query, budgeted to the matched-note count, delivered as top-level items tagged provenance: explore) plus an un-clustered recency channel (the newest chunks, tagged recent). For each cluster (and the explore items) it judges coverage inline (covered/near/absent) and crystallizes via engram amend (update an existing note) or a handoff to the write-memory skill (create one) in its default deep mode (a glance pass is read-only), activates only the notes it actually used, then reports whether the surfaced memory changed the agent's plan. |
learn | Captures the session's explicit lessons — corrections, explicit save-requests, self-discovered reversals, and confirmed approaches (positive reinforcement, user-praised or self-validated) — as permanent vault notes via write-memory handoffs. Along the way it mechanically sweeps every conversation and doc into the searchable chunk index (engram ingest --auto) and checks vocab liveness (engram vocab stats, auto-refitting when due), so raw event memory stays current even when no explicit lesson exists. |
See agent-instructions/skills/{recall,learn,please,route,curate,write-memory}/SKILL.md for the full skill definitions.
All six ship with a SKILL.md (see skill-runbook-registration for the mechanism). please drives an ask end-to-end through a fixed seven-step workflow (capture, orient, plan, execute (TDD), document, complete, capture), tracked on the task list, with four adversarial review gates dispatching fresh per-angle reviewer subagents over the plan, each refactor, touched docs, and outward prose, plus a step-7 lessons audit; it is one skill and runbook note (its former three sub-runbooks merged into the single note/skill body), triggered by /please <ask> and the literal phrases take this end-to-end and please. route guides subagent selection (agent type, model, effort): easy work goes to a cheap model, complex work is decomposed before dispatch, and every dispatched subagent recalls first; please consults it when staffing gate reviewers. curate judges pending-offer notes (a child's served learn, a note pulled down from the parent) against the host vault using the same covered/near/absent reasoning recall documents, authors the runbook fields of pending skill notes instead of judging them, and composes and executes engram amend directly (an offer's content already exists as a note file, so it does not hand off to write-memory); it runs host-local only and is triggered by the literal words curate, /curate, pending offers, and pending offer. write-memory is invoked natively as a skill by recall/learn at their write sites, not reached via basename/wikilink.
Engram reads and writes a zettelkasten vault. Resolution order:
--vault <path> flagENGRAM_VAULT_PATH environment variable$XDG_DATA_HOME/engram/vault (fallback: ~/.local/share/engram/vault)Every vault-resolving command (not only engram learn) creates a missing vault on first use: the directory is bootstrapped with a minimal .obsidian/ config so Obsidian recognizes it, a .gitignore, a .engram/ state directory and a vault ID, and one stderr line (engram: created new vault at <path>) announces it so a mistyped --vault path is visible. An existing vault is left untouched by read-only use. See 792a2f Existing vaults are left alone.
Vault layout (flat since the 2026-06-12 flat-vault migration — notes live at the vault root; the Permanent/ and MOCs/ subdirectories are retired and ignored by the scanner):
<vault>/
<luhmann-id>.<YYYY-MM-DD>.<slug>.md atomic notes at the root
<luhmann-id>.<YYYY-MM-DD>.<slug>.vec.json sibling embedding sidecar
engram learn feedback --slug ... --source ... --situation ... --behavior ... --impact ... --action ... [--tag <family>[/<value>] ...] [--project <slug>] [--issue <id>]
engram learn fact --slug ... --source ... --situation ... --subject ... --predicate ... --object ... [--tag <family>[/<value>] ...] [--project <slug>] [--issue <id>]
engram learn qa --slug ... --source ... [--question <text>] [--answer <text>|--answer-file <path>] [--contributors <basename>...] [--certainty high|medium|low] Write a QA pair (Q+A notes) to the vault. --slug and --source required; --answer and --answer-file are mutually exclusive; --contributors repeatable, validated against the vault; --certainty defaults to medium.
engram learn runbook --slug ... --source ... --situation ... --done-when ... [--body <numbered steps>] [--red-flag <text> ...] [--trigger <text> ...] [--tag <family>[/<value>] ...] [--project <slug>] [--issue <id>] Write a runbook note. `--trigger` (repeatable, each ≥ 3 chars) declares literal cues — a slash form or multi-word phrase, never a lone common word — that surface the runbook first when they appear in `engram query --text`; `engram amend --trigger` replaces the list.
engram embed apply [--all|--missing|--stale|--force|--dry-run] (Re-)embed notes per selection (default: missing)
engram embed status Report counts per state (total / with-embeddings / without / stale / incompatible / broken)
engram query --phrase <p> [--phrase <p>...] [--text <user message>] [--limit N] [--project <slug>] [--chunks-dir <dir>] [--content-budget N] [--recent-fill N] [--lazy-chunks] Semantic search over vault notes + chunk index; YAML output. `--limit` (default 20) caps the returned `items[]` count — a real, enforced cap, not report-only metadata — except runbook trigger hits, which do not count against it. (A runbook can only be a trigger hit if it has a compatible embedding sidecar.) `--text` takes the user's message verbatim (never embedded; at least one of `--phrase`/`--text` is required): a runbook whose `triggers:` list has an entry appearing in it (case-insensitive whole-word match, whitespace collapsed: `curate` hits "curate!" and "/curate" but not "accurate") is a trigger hit and leads `items[]` with provenance `trigger`, regardless of similarity. With `ENGRAM_PARENT` set, merges local + parent results into one payload — see [Parent sync](#parent-sync-engram_parent). Recency-weights chunks AND notes. Builds a bounded matched set (per-phrase top-30, union/dedup, relevance floor 0.25, cap ~300), clusters it in one pass (AutoK k-means), emits `candidate_l2s: [{path, cosine, content}]` per cluster — within-cluster top-5 notes only — plus superseded-note ride-alongs at the next rank. Separately samples an **explore** half from vocab-term centroids by proximity to the query (softmax allocation, budgeted to the matched-note count, δ=0.05 match-evidence bonus for terms with an exploit-half member, core-first within-term selection, dedupe+backfill), delivered as top-level `items[]` entries (`provenance: explore`, `source_term`; budget reported in `explore_allocated`, a term → delivered-count map, always present — `{}` on missing/unreadable centroid data). Appends the newest chunks un-clustered (tagged `recent`; default 25, controlled by `--recent-fill`). `--content-budget` caps how many chunk items render with full content (default 15; later chunks get a snippet). `--lazy-chunks` renders matched chunk items path/score only — the agent fetches evidence on demand via `engram show-chunk`. Activation is agent-driven — the binary emits no `activated` flag. --project restricts items to notes whose frontmatter `project:` matches.
engram query-chunks --phrase <p> [--phrase <p>...] [--limit N] [--chunks-dir <dir>] Semantic search over the chunk index only (YAML output). Scores chunks by max cosine across phrases; clusters results with AutoK k-means. No vault notes, no recency channel — chunk-space search only.
engram resituate --note <ref> --situation <text> Rewrite a note's situation field in sync: frontmatter, body opener, and sidecar situation_vector (D4/INV-S2). Both flags required; no --dry-run.
engram check Run vault-invariant checks (G0 and G3 links, M5 situation, S1 sidecars); a PASS/WARN/FAIL line each, exit non-zero only on FAIL
engram ingest [--auto] Merge-append session transcripts + markdown into the per-source chunk index — re-chunks/re-embeds only changed content; within one source this is append-only (a re-chunk never drops a prior record). Across sources, byte-identical content is deduplicated by content hash: only one canonical copy per hash group is indexed, and a duplicate's index file is removed only once its retained twin's index file is verified to cover every one of its own records — never on hash-match alone. `--auto` sweeps all known sources, skips session-log directories whose slugified project path starts with a non-persistent-workspace prefix (slugified forms of `/private/tmp`, `/tmp`, and macOS `$TMPDIR`), and additionally drops any resolved sweep root whose own path sits under `/tmp`, `/private/tmp`, `/var/folders`, or `/private/var/folders` — preventing eval/test runs from bloating the main index. An ancestor `.claude` dir's sweep also now excludes its `jobs/` subdirectory (agent-harness scratch, including whole snapshot copies of the vault) — matching the exclude the `.pi` ancestor sweep already had; `.claude/jobs` was previously swept and indexed. Configurable via `.engram/sweep.json` (`non_persistent_prefixes` / `non_persistent_path_prefixes` keys); bypassed by explicit `--sweep`/`--transcript`/`--markdown`/`--pi-sessions` or an isolated index via `ENGRAM_CHUNKS_DIR`. Used by /learn and /recall.
engram prune [--empty [--dry-run]] [--duplicates [--dry-run]] Detach chunk index entries whose source file no longer exists (GC). Operator-run; reads the manifest and drops the stale per-source manifest entry, keeping the embedded chunks on disk (still searchable). `--empty` instead removes existing 0-byte `.jsonl` chunk-index files left by zero-record sources (ranking-neutral — empties hold zero records); `--duplicates` retroactively collapses every exact-content-hash group down to one canonical member, removing the rest's index files + manifest entries — safe by construction (a duplicate is removed only once its retained twin's index file is verified to cover its records; otherwise refused, not removed) and convergent (a second run removes nothing); `--dry-run` previews the count without deleting or removing. Not part of the recall/learn flows.
engram count [--group-by <attr> [--filter <attr=value>...]] [--backlinks-of <basename>] Read-only vault aggregation, off the query/similarity path. `--group-by` counts distinct note membership per frontmatter attribute value (list attrs count one per distinct element; a value listed twice in one note still counts once), optionally restricted by repeatable AND-ed `--filter attr=value` predicates (scalar equality or list-contains); output is `value<TAB>count` lines sorted count-desc then value-asc, an `(attr absent): N` bucket when any in-set note lacks the attribute, then `total: N` (empty result prints nothing). `--backlinks-of <basename>` prints a vault-graph node's wikilink in-degree plus its sorted linkers. The two modes are mutually exclusive and each independently Obsidian-verifiable (group-by against a property/tag filter, backlinks-of against the backlinks panel) — they are NOT equal to each other: backlinks-of counts every linker (e.g. an index/MOC page) while group-by counts only frontmatter members, so the two diverge by the count of non-member linkers.
engram show <ref> [--parent] Print a note (frontmatter + body) and its outbound wikilink targets, read-only. One required positional; no --ref flag. (candidate_l2s carry content inline, so /recall no longer shows candidates.) `--parent` resolves the ref against `ENGRAM_PARENT` instead of the local vault — errors if unset. On a local miss, with `--parent` not passed, `engram show` falls back to `ENGRAM_PARENT` automatically if configured, prefixing output with `# from_parent: true`; a local hit never contacts the parent, and with no `ENGRAM_PARENT` configured the same not-found error surfaces as before.
engram show-chunk <source#anchor> [--chunks-dir <dir>] Print a chunk's text by its source#anchor id (read-only). Used by /recall with `--lazy-chunks` to fetch a specific chunk's evidence on demand. Local only — chunks never cross vaults, so there is no `--parent` flag and no parent fallback.
engram amend --target <ref> [--activate] [--clear-pending] [--supersedes "<basename>|<type>|<claim>"] [--chunk-source <source#anchor>...] [--situation/--subject/--predicate/--object | --behavior/--impact/--action | --done-when/--body ...] [--red-flag <text>...] [--trigger <text>...] [--discard [--into <existing>]] [--expect-hash <exchange hash>] Amend a note in place: merge chunk-source provenance (idempotent), overwrite only supplied content fields, re-embed only on a content change; `--activate` bumps `LastUsed`; `--clear-pending` clears the pending-offer marker a served write or a pull-down set, accepting it as a normal live note; `--supersedes` writes typed supersession frontmatter + inverse + body line. `--discard` deletes the target note and its sidecar; with `--into <existing>` it folds the target's basename, aliases and parent links into `<existing>` first (curation's covered/near fold), instead of a bare delete. `--expect-hash` is required for `--clear-pending`, `--discard` and `--discard --into` on a note carrying `offer.origin`: the command fails without changing anything unless the note's current exchange hash equals the expected one (any hash that is not equal to the current one — including one from another hash version, which compares as unknown — fails), so a curator's judgment can never be silently overtaken by an in-place offer update. The /recall update path: covered link-enriches, near re-synthesizes content.
engram activate --note <ref> [--note <ref>...] [--parent] Mark note(s) as recently used — bumps `LastUsed` in the sidecar so usefulness keeps useful notes fresh (called by /recall on only the notes the agent actually used). `--note` paths are vault-relative (resolved against the vault root / `ENGRAM_VAULT_PATH`); absolute paths are used as-is. On a local miss, a basename or `<basename>.md` ref is a parent candidate when `ENGRAM_PARENT` is set: it is fetched and written as a new local pending offer instead of failing (a bare Luhmann id never goes to the parent — ids are minted per vault). `--parent` forces parent resolution even when a local file of that name exists, and errors if `ENGRAM_PARENT` is unset; a path ref with `--parent` is refused as "not sent to the parent" rather than looked up locally. Either way, activating a parent note also bumps its use on the parent, best-effort (a failure there is not fatal). With `ENGRAM_PARENT` set, a local hit on a note linked to a parent note (a previously pulled or offered copy) also re-checks that parent note: when its exchange hash has changed since the link was recorded, and that version was not declined, the new version is pulled down as another local pending offer; an unchanged note, a declined version or an unreachable parent writes nothing and never fails the local activate. The re-check does not bump the parent note's use. A served `POST /activate` resolves refs only against the vault's note names and refuses an absolute path, a path separator or `..` with a 400.
engram vocab bootstrap --seed <yaml> [--floor <f>] Seed vocab definition fact notes (bare `vocab` tag plus their own `vocab/<term>` self-tag per term — one family note stays bare-only) from the validated term set (--seed, required); embed them; tag all existing notes with `vocab/<term>` entries in the shared `tags:` list (no separate index file — the index is emergent via `engram count`). --floor sets the minimum cosine similarity for vocab assignment (default 0.35). Idempotent.
engram vocab tag-definitions [--vault <dir>] Idempotent backfill: adds the missing `vocab/<term>` self-tag to every existing per-term definition note (skips the family note). Tags are not content-hash inputs so no re-embed occurs. `--vault` overrides the vault root.
engram vocab propose --term <t> --description <d> LLM-gated: create a new vocab definition note if no existing term covers it and projected attachment ≤ 20% of vault (~$0.05/proposal). Both flags required.
engram vocab stats Per-term member counts, vault untagged-rate, hub terms (> 25% of vault), orphan terms (< 2 members), version staleness.
engram vocab refit [--dry-run] [--names <file>] Derivational: derives the vocabulary from the vault's note embeddings; matches clusters to existing terms, retires unmatched derived terms (proposed terms are never auto-retired), and emits naming requests for new clusters — answer via --names <file>. --dry-run previews the matched/new/retired diff. Rewrites member `tags:` entries in the `vocab/<term>` namespace; major version bump on the family definition note (no index to regenerate — the index is emergent). Triggered growth-only (≥40 new notes AND ≥14 days).
engram auto deliver | judge | write | sweep The commands the `engram-memory` mod runs for automatic memory, not meant to be run by hand: each reads one JSON object on standard input, creates no vault, and acts only for the prompts the user sends (a headless `claude -p` prompt only with `ENGRAM_AUTO_CAPTURE_SDK=1`). What became of each lesson is recorded under `$XDG_DATA_HOME/engram/auto`; `engram auto <command> --helphooks/register.js 654 lines1// Automatic memory: the notes and passages of the vault that apply to a prompt arrive beside it (793a2b), and after the
2// turn the lessons it holds become notes (793a2c) and its conversation is indexed (793a2f). The logic is in the engram
3// binary (`engram auto moment`, `phrases`, `recall`, `deliver`, `judge`, `place`, `write` and `sweep`); this module
4// routes to it, asks the model for the requests the steps print, puts the block the delivery prints beside the prompt,
5// and shows the user the lines it prints for the user, through the host's log (793d2a).
6
7// Each wait of the flow of a prompt, a step or a model call, has 2 minutes of its own (793a2d): a wait that is only
8// slow is waited for, and one that has not answered by then has stalled, and the flow steps aside.
9const STEP_BOUND_MS = 120000
10
11// The steps of the flow of a prompt that print a request, in order; the delivery follows them (793d2a).
12const CHAIN = ['moment', 'phrases', 'recall']
13
14// Every request a step of the flow of a prompt prints begins with this; anything else is not a request.
15const STEP = '{"request":'
16
17// Every block the command prints opens with this (793d4c); anything else is not a block.
18const MARKER = '[engram memory]'
19
20// The bounds of the stages of the capture of a turn (793d2b): the call for the session's ID, the steps `engram auto
21// moment`, `judge` and `place`, `engram auto write` and the run of it that reports a failure. The three model calls
22// hold the limits of their requests, 10, 30 and 15 seconds; together they are 115 seconds, under the 2 minutes of the
23// product (793a2c).
24const ID_BOUND_MS = 5000
25const MOMENT_BOUND_MS = 5000
26const JUDGE_BOUND_MS = 10000
27const PLACE_BOUND_MS = 5000
28const WRITE_BOUND_MS = 20000
29const REPORT_BOUND_MS = 15000
30
31// The write that follows a deep delivery has a bound of its own (793d2a): it holds neither the prompt nor the agent.
32const DEEP_WRITE_BOUND_MS = 60000
33
34// The sweep of the conversation has a bound of its own (793d2c): the call for the ID, the sweep and the report of its
35// failure are 110 seconds, under the 2 minutes of the product (793a2f), and the capture's bounds are not extended.
36const SWEEP_BOUND_MS = 90000
37
38// A sweep that is ended is run once more, as ended, so that the command gives up for the session (793d2c): the call
39// for the ID (5), the sweep (90), this run (5) and the report of its failure (15) are 115 seconds, under the 2 minutes.
40const GIVE_UP_BOUND_MS = 5000
41
42// The prompts the session raised that have started no turn, oldest first, each with its text and its origin; and the
43// text and the origin of each turn the main loop has started and not completed, by the turn's ID (793d2b): the prompt
44// names its origin, the start of the turn names its text, the end of the turn names only the turn.
45const raised = []
46const turns = new Map()
47
48// The IDs of the notes and passages the module delivered in each session, by the session's ID (855d2c): the file name
49// of a note and the chunk ID of a passage, each given to the delivery as held, so that a note is delivered once.
50const delivered = new Map()
51
52// The last delivery of each session, as a promise that settles when it is over: the deliveries of one session run one
53// after another, so that each reads the set after the one before added to it, and a note is marked as used once
54// (793d2a).
55const deliveries = new Map()
56
57// The writes after the deliveries of a session, each chained after the one before it, for the same reason.
58const writes = new Map()
59
60// The text and the origin of the turn the main loop is in, which the events of its tools carry none of (855d2c): the
61// last turn started.
62let current = { text: '', origin: '' }
63
64// The IDs of the calls whose failure was judged and whose batch has not come (855d2c): a batch of calls all judged as
65// failures is not judged again.
66const failed = new Set()
67
68// The keys a tool.call event carries beside the arguments of the tool (855d2c).
69const RESERVED = ['tool', 'tool_use_id', 'agentId', 'requestMeta', 'consent']
70
71// The Bash commands that are the agent's own use of the memory, which the binary holds the text and result of as well
72// (855d2c1o), and the skills that are: a call that is one is not looked up for.
73const MEMORY_COMMANDS = ['engram query', 'engram show', 'engram show-chunk', 'engram recall', 'engram learn',
74 'engram amend']
75const MEMORY_SKILLS = ['recall', 'learn', 'curate', 'write-memory']
76
77// A Bash command is one of the memory commands only at the position of a command: at the start, after `;`,
78// `&`, `|` or `(`, or after `-c` and a quote, with a path before `engram` allowed, so that a commit message or a grep
79// that names a command is not one.
80const MEMORY_COMMAND = new RegExp(String.raw`(?:^|[;&|(]|-c\s+['"])\s*(?:\S*/)?engram\s+(?:${
81 MEMORY_COMMANDS.map((memory) => memory.replace('engram ', '')).join('|')})\b`)
82
83// So many prompts are kept at most: one that is dropped, or delivered into a running turn, starts no turn.
84const RAISED_MAX = 20
85
86export function register(on) {
87 on('prompt.submit', async ($, e, next) => {
88 raised.push({ text: e.text, origin: originOf(e) })
89
90 if (raised.length > RAISED_MAX) {
91 raised.shift()
92 }
93
94 const delivery = await deliveryFor($, 'a prompt', (state) => promptChain($, e, state))
95
96 // the lines go to the user and the model reads none of them: they are logged, not added to the context (793d2a)
97 for (const line of delivery.shown) {
98 $.ui.log(line)
99 }
100
101 return next(delivery.block ? { ...e, context: [...(e.context ?? []), delivery.block] } : e)
102 })
103
104 on('turn.start', async (_$, e, next) => {
105 if (e.agentId === undefined) {
106 current = { text: e.text, origin: originOfStart(e.text) }
107 turns.set(e.turnId, current)
108 }
109
110 return next(e)
111 })
112
113 // the three cues of a turn (855d2c): the chain of the prompt runs for the event, and the block rides the answer
114 on('tool.call', async ($, e, next) => {
115 if (e.agentId !== undefined || isMemoryCall(e.tool, e)) {
116 return next(e)
117 }
118
119 const input = Object.fromEntries(Object.entries(e).filter(([key]) => !RESERVED.includes(key)))
120 const event = { tool: e.tool, input }
121 // the call is let run while its chain runs; the block rides the result once both are over
122 const [delivery, result] = await Promise.all([
123 cueDelivery($, 'before-tool', () => bounded($.session.id(), ID_BOUND_MS, "the call for the session's ID"), event),
124 next(e),
125 ])
126
127 // a call that was denied has no context to add to; the notes of the block are held no longer
128 if (result.deny !== undefined) {
129 release(delivery)
130
131 return result
132 }
133
134 return withBlock(result, 'context', delivery.block)
135 })
136
137 on('classic.PostToolUseFailure', async ($, e, next) => {
138 if (e.agent_id !== undefined || e.is_interrupt) {
139 return next(e)
140 }
141
142 failed.add(e.tool_use_id)
143
144 if (isMemoryCall(e.tool_name, e.tool_input)) {
145 return next(e)
146 }
147
148 const event = { tool: e.tool_name, input: e.tool_input, error: e.error }
149 const delivery = await cueDelivery($, 'failure', async () => e.session_id, event)
150
151 return withBlock(await next(e), 'additionalContext', delivery.block)
152 })
153
154 on('classic.PostToolBatch', async ($, e, next) => {
155 const calls = e.tool_calls ?? []
156 // a batch is left alone when each call was judged as a failure or is the agent's own use of the memory
157 const skipped = calls.every((call) => failed.has(call.tool_use_id) || isMemoryCall(call.tool_name, call.tool_input))
158
159 for (const call of calls) {
160 failed.delete(call.tool_use_id)
161 }
162
163 if (e.agent_id !== undefined || skipped) {
164 return next(e)
165 }
166
167 const event = {
168 calls: calls.map((call) => ({ tool: call.tool_name, input: call.tool_input, response: call.tool_response })),
169 }
170 const delivery = await cueDelivery($, 'batch-done', async () => e.session_id, event)
171
172 return withBlock(await next(e), 'additionalContext', delivery.block)
173 })
174
175 on('turn.complete', async ($, e, next) => {
176 const result = await next(e)
177 const turn = turns.get(e.turnId) ?? { text: '', origin: '' }
178
179 turns.delete(e.turnId)
180
181 // the lessons are captured and the conversation is swept (793d2c) apart from the turn, and apart from each other,
182 // from a timer: the host holds the next event of the mod, a prompt among them, until the calls an event left in
183 // flight settle, and a timer's calls are no event's; the turn of a subagent is left out
184 if (e.agentId === undefined) {
185 $.clock.after(0, () => {
186 void afterTurn($, e, (session) => captureLessons($, e, turn, session))
187 void afterTurn($, e, (session) => sweep($, session, turn.origin))
188 })
189 }
190
191 return result
192 })
193}
194
195// isMemoryCall reports whether a call is the agent's own use of the memory (855d2c1o): a Bash command that runs one of
196// the memory commands, or the Skill tool with one of the memory skills. The input holds the command or the skill.
197function isMemoryCall(tool, input) {
198 if (tool === 'Bash') {
199 return typeof input?.command === 'string' && MEMORY_COMMAND.test(input.command)
200 }
201
202 return tool === 'Skill' && MEMORY_SKILLS.includes(input?.skill)
203}
204
205// ask is the text of the model's answer to a request, and fails when the model gives none (793d2b). The call ends at
206// the time limit that the request holds.
207async function ask($, request) {
208 let reply
209
210 try {
211 reply = await $.model.complete(request)
212 } catch (error) {
213 throw new Error(`the model call failed: ${describe(error)}`)
214 }
215
216 if (!reply.isAnswered) {
217 throw new Error(`the model call was not answered: ${reply.reason}`)
218 }
219
220 return reply.text
221}
222
223// askModel is the text of the model's answer to a request of the flow of a prompt, and fails when the model gives none.
224// The request is asked exactly as the step printed it; the call is aborted through its signal when it has not answered
225// within 2 minutes (793d2a).
226async function askModel($, request) {
227 const stop = new AbortController()
228 const timer = setTimeout(() => stop.abort(), STEP_BOUND_MS)
229 let reply
230
231 try {
232 reply = await $.model.complete(request, { signal: stop.signal })
233 } catch (error) {
234 // a call that is aborted may reject, and it was the bound that ended it
235 throw new Error(stop.signal.aborted ? `the model call gave nothing within ${limitText(STEP_BOUND_MS)}`
236 : `the model call failed: ${describe(error)}`)
237 } finally {
238 clearTimeout(timer)
239 }
240
241 if (!reply.isAnswered) {
242 const why = stop.signal.aborted
243 ? `gave nothing within ${limitText(STEP_BOUND_MS)}`
244 : `was not answered: ${reply.reason}`
245
246 throw new Error(`the model call ${why}`)
247 }
248
249 return reply.text
250}
251
252// bounded is what a promise gives, or a failure that names it when it gives nothing within the bound: the host gives
253// some calls no time limit of their own (793d2b).
254async function bounded(promise, bound, what) {
255 let timer
256 const late = new Promise((_resolve, reject) => {
257 timer = setTimeout(() => reject(new Error(`${what} gave nothing within ${bound} ms`)), bound)
258 })
259
260 try {
261 return await Promise.race([promise, late])
262 } finally {
263 clearTimeout(timer)
264 }
265}
266
267// afterTurn does a job of the capture of a turn that ended, with the session's ID, and has the writing record the
268// failure of a step that fails: the turn went on as if automatic memory were not installed. A session whose ID does not
269// come is left alone, since the writing cannot record anything without it (793d2b, 793d2c).
270async function afterTurn($, e, job) {
271 let session = ''
272
273 try {
274 session = await bounded($.session.id(), ID_BOUND_MS, "the call for the session's ID")
275
276 await job(session)
277 } catch (error) {
278 await report($, session, e.turnId, error)
279 }
280}
281
282// captureLessons judges a turn that ended and has the lessons written (793d2b): the moment, the judgement and the
283// placement each print the request for the model, which is asked exactly as printed, and the writing is given the
284// replies and the carry of the placement. It ends quietly when the moment or the judgement prints nothing, and writes
285// the lessons at the top when the placement prints nothing. Each stage has its bound.
286async function captureLessons($, e, turn, session) {
287 const fields = { session, turn: e.turnId, text: turn.text, answer: e.answer, origin: turn.origin }
288 const moment = await asked($, 'moment', { ...fields, cue: 'turn-end' }, MOMENT_BOUND_MS)
289
290 if (moment === null) {
291 return
292 }
293
294 const judged = await asked($, 'judge', { ...fields, reply: moment.reply, carry: moment.carry }, JUDGE_BOUND_MS)
295
296 if (judged === null) {
297 return
298 }
299
300 const written = { session, turn: e.turnId, reply: judged.reply }
301
302 try {
303 const placed = await asked($, 'place', { session, turn: e.turnId, reply: judged.reply, carry: judged.carry },
304 PLACE_BOUND_MS)
305
306 if (placed !== null) {
307 written.carry = placed.carry
308 written.placement = placed.reply
309 }
310 } catch (error) {
311 // a placement that fails does not fail the capture: the lessons are written at the top and the writing records why
312 written.placementFailure = describe(error)
313 }
314
315 await run($, 'write', written, WRITE_BOUND_MS)
316}
317
318// asked runs a step of the capture and asks the model for the request it prints (793d2b): the reply and the carry, or
319// nothing when the step prints nothing. A request that is not an object is a failure of the step.
320async function asked($, step, input, bound) {
321 const { stdout } = await run($, step, input, bound)
322
323 if (stdout === '') {
324 return null
325 }
326
327 const printed = stepOf(stdout, step)
328
329 try {
330 return { reply: await ask($, printed.request), carry: printed.carry }
331 } catch (error) {
332 throw new Error(`${step}: ${describe(error)}`)
333 }
334}
335
336// sweep indexes the conversation of the session (793d2c).
337// A sweep that is ended at its bound is run once more, as ended:
338// the command gives up for the session and fails once, naming what to run (793d4n), and that failure is reported.
339async function sweep($, session, origin) {
340 try {
341 await run($, 'sweep', { session, origin }, SWEEP_BOUND_MS)
342 } catch (error) {
343 if (!error.ended) {
344 throw error
345 }
346
347 await run($, 'sweep', { session, origin, ended: true }, GIVE_UP_BOUND_MS)
348 }
349}
350
351// limitText is a time limit written for a person: whole minutes, else milliseconds.
352function limitText(ms) {
353 return ms >= 60000 && ms % 60000 === 0 ? `${ms / 60000} minutes` : `${ms} ms`
354}
355
356// describe is the text of a failure.
357function describe(error) {
358 return error instanceof Error ? error.message : String(error)
359}
360
361// deliveryFor is the block and the lines for the user for a prompt or a tool event, or nothing when the flow does not
362// finish or fails: the work goes on as if automatic memory were not installed, and the failure is reported (793d2a,
363// 855d2c). The chain is given the state, in which the session's ID is left as soon as it is known, for the report.
364async function deliveryFor($, what, chainOf) {
365 const state = { session: '', write: null }
366
367 try {
368 const delivery = await chainOf(state)
369
370 writeAfter($, state)
371
372 return delivery
373 } catch (error) {
374 reportDelivery($, state.session, what, error)
375
376 return { block: '', shown: [] }
377 }
378}
379
380// cueDelivery is deliveryFor for the event of a tool: the chain of a prompt with the event and the text the turn began
381// with in place of the prompt's text (855d2c). The lines for the user are logged here.
382async function cueDelivery($, cue, sessionOf, event) {
383 const delivery = await deliveryFor($, 'a tool call', async (state) => {
384 state.session = await sessionOf()
385
386 return chain($, state, { origin: current.origin, cue }, { text: current.text, event })
387 })
388
389 for (const line of delivery.shown) {
390 $.ui.log(line)
391 }
392
393 return delivery
394}
395
396// withBlock is the answer of the hooks beneath, with the block added as one more entry of the list the key names, or
397// the answer as it is when there is no block (855d2c).
398function withBlock(result, key, block) {
399 return block ? { ...result, [key]: [...(result[key] ?? []), block] } : result
400}
401
402// promptChain runs the chain of a prompt (793d2a).
403async function promptChain($, e, state) {
404 state.session = await bounded($.session.id(), ID_BOUND_MS, "the call for the session's ID")
405
406 return chain($, state, { origin: originOf(e), cue: 'prompt' }, { text: e.text })
407}
408
409// chain runs the steps of the flow in order (793d2a): the moment, the phrases and the recall, each given the reply to
410// the request before it and its carry and asking the model for the request it prints, and then the delivery with the
411// last reply and its carry. A step that prints nothing ends the chain there, and the delivery is run with the fields
412// of the moment alone: the prompt's text, or for a tool event the event and the text the turn began with.
413async function chain($, state, who, fields) {
414 const base = { session: state.session, ...who, held: [...(delivered.get(state.session) ?? [])] }
415 let answer = {}
416
417 for (const step of CHAIN) {
418 const { stdout } = await run($, step, { ...base, ...(step === 'moment' ? fields : answer) }, STEP_BOUND_MS)
419
420 if (stdout === '') {
421 answer = {}
422
423 break
424 }
425
426 const printed = stepOf(stdout, step)
427
428 answer = { reply: await askModel($, printed.request), carry: printed.carry }
429
430 // a deep recall says that a write follows the delivery (793d2a)
431 state.write = printed.write === true ? { cue: base.cue, ...answer } : null
432 }
433
434 return inOrder(deliveries, state.session, async () => {
435 const held = [...(delivered.get(state.session) ?? [])]
436 const { stdout, stderr } = await run($, 'deliver', { ...base, held, ...fields, ...answer }, STEP_BOUND_MS)
437 const lines = stderr.split('\n').map((text) => text.trim()).filter((text) => text !== '')
438
439 // the lines go to the user and the model reads none of them
440 return stdout.startsWith(MARKER) ? keepNew(state.session, stdout, lines) : { block: '', shown: lines }
441 })
442}
443
444// writeAfter has the lessons and the notes of a deep delivery written from a timer once the block is given, so that the
445// writing holds neither the prompt nor the agent and no call is in flight when the hook settles; a write that fails is
446// reported (793d2a). The writes of a session are run one after the other, so that two deep cues in a turn do
447// not crystallize the same lesson twice.
448function writeAfter($, state) {
449 if (state.write === null) {
450 return
451 }
452
453 const { session, write } = state
454
455 $.clock.after(0, () => {
456 void (async () => {
457 await inOrder(writes, session, async () => {
458 try {
459 await run($, 'write', { session, ...write }, DEEP_WRITE_BOUND_MS)
460 } catch (error) {
461 await report($, session, '', new Error(`the write after a deep delivery failed: ${describe(error)}`))
462 }
463 })
464 })()
465 })
466}
467
468// inOrder runs a job of a session after the one before it in the chains is over, whatever became of that one (793d2a).
469async function inOrder(chains, session, job) {
470 const before = chains.get(session) ?? Promise.resolve()
471 let over
472 const mine = new Promise((resolve) => { over = resolve })
473
474 chains.set(session, before.then(() => mine))
475 await before
476
477 try {
478 return await job()
479 } finally {
480 over()
481 }
482}
483
484// reportDelivery has the writing record that the delivery for a prompt or a tool call could not be done, from a timer,
485// so that the prompt is held no longer and no call is left in flight when the hook settles (793d2a). A delivery whose
486// session's ID did not come is left alone, since the writing cannot record anything without it.
487function reportDelivery($, session, what, error) {
488 if (session === '') {
489 return
490 }
491
492 $.clock.after(0, () => {
493 void report($, session, '', new Error(`the delivery for ${what} could not be done: ${describe(error)}`))
494 })
495}
496
497// stepOf is what a step printed: the request for the model and the carry, or a failure that names the step when what it
498// printed is not one (793d2a, 793d2b).
499function stepOf(printed, step) {
500 if (!printed.startsWith(STEP)) {
501 throw new Error(`engram auto ${step} printed what is not a request`)
502 }
503
504 try {
505 return JSON.parse(printed)
506 } catch (error) {
507 throw new Error(`engram auto ${step} printed a request that is not JSON: ${describe(error)}`)
508 }
509}
510
511// keepNew is the block less each line whose ID the session's set holds, and the lines for the user, less the notes and
512// passages that line of theirs names which the block left out, with the IDs of the lines left added to the set in the
513// same step, so that a note or a passage is delivered once however many chains find it; no block when no line is left
514// (855d2c, 793d2a).
515function keepNew(session, block, shown) {
516 const ids = delivered.get(session) ?? new Set()
517 const lines = block.split('\n')
518 const left = []
519 const kept = lines.filter((line) => {
520 if (!line.startsWith('- ')) {
521 return true
522 }
523
524 const entry = entryOf(line)
525
526 if (ids.has(entry.id)) {
527 return false
528 }
529
530 left.push(entry)
531
532 return true
533 })
534
535 for (const entry of left) {
536 ids.add(entry.id)
537 }
538
539 delivered.set(session, ids)
540
541 const all = lines.filter((line) => line.startsWith('- ')).length
542
543 return {
544 block: left.length > 0 ? kept.join('\n') : '',
545 shown: left.length === all ? shown : shown.flatMap((line) => deliveredLineOf(line, left)),
546 session,
547 added: left.map((entry) => entry.id),
548 }
549}
550
551// release takes the IDs of a block that was not given off the session's set, so that its notes are delivered later
552// (855d2c).
553function release(delivery) {
554 for (const id of delivery.added ?? []) {
555 delivered.get(delivery.session)?.delete(id)
556 }
557}
558
559// entryOf is the ID of a line of a block, whole, and whether the line is a passage's (855d2c): the line of a note is a
560// dash, the file name, which holds no space, and the situation in parentheses; that of a passage is a dash, the chunk
561// ID, which may hold spaces, and `(passage)`.
562function entryOf(line) {
563 const rest = line.slice(2)
564 const note = /^(\S+\.md) \(/.exec(rest)
565
566 if (note) {
567 return { id: note[1], passage: rest.startsWith(`${note[1]} (passage): `) }
568 }
569
570 const at = rest.indexOf(' (passage): ')
571
572 if (at >= 0) {
573 return { id: rest.slice(0, at), passage: true }
574 }
575
576 const open = rest.indexOf(' (')
577
578 return { id: open >= 0 ? rest.slice(0, open) : rest, passage: false }
579}
580
581// deliveredLineOf is the line for the user that names what was delivered, written again for the entries that are left,
582// or nothing when none is left; any other line goes through (793d2a).
583function deliveredLineOf(line, left) {
584 if (!line.startsWith('delivered ')) {
585 return [line]
586 }
587
588 const parts = []
589
590 for (const [passage, word] of [[false, 'note'], [true, 'passage']]) {
591 const ids = left.filter((entry) => entry.passage === passage).map((entry) => entry.id)
592
593 if (ids.length > 0) {
594 parts.push(`the ${word}${ids.length === 1 ? '' : 's'} ${ids.join(', ')}`)
595 }
596 }
597
598 return parts.length > 0 ? [`delivered ${parts.join(' and ')}`] : []
599}
600
601// originOf is the kind of origin that Claude Code stamped on a prompt, or nothing when it stamped none (793d2a).
602function originOf(e) {
603 return e.origin?.kind ?? ''
604}
605
606// originOfStart is the origin of the prompt that a turn which starts with a text began with, and takes that prompt off
607// the ones kept, with the prompts before it, which began no turn of their own. The prompt is the oldest kept with the
608// text; when none has it, the newest one; a turn that starts with no text has no origin (793d2b).
609function originOfStart(text) {
610 const at = raised.findIndex((prompt) => prompt.text === text)
611 const taken = at >= 0 ? raised.splice(0, at + 1) : text === '' ? [] : raised.splice(0)
612
613 return taken.length > 0 ? taken[taken.length - 1].origin : ''
614}
615
616// report has the writing record the failure of a step of the capture, or of a delivery, when the session is known:
617// the failure goes whole, for the writing redacts it and cuts it (793d4k). A report that fails is the end of it.
618async function report($, session, turn, error) {
619 if (session === '') {
620 return
621 }
622
623 try {
624 await run($, 'write', { session, turn, failure: describe(error) }, REPORT_BOUND_MS)
625 } catch {
626 // nothing more can be done
627 }
628}
629
630// run is what a step of `engram auto` printed for an input, on standard output and on standard error, and fails, naming
631// the command, when it exits with other than 0 or cannot be run. The command is ended after timeoutMs, and after the
632// default of the process call when there is none.
633async function run($, step, input, timeoutMs) {
634 const command = `engram auto ${step}`
635 let result
636
637 try {
638 result = await $.process.run(['engram', 'auto', step], { stdin: JSON.stringify(input), timeoutMs })
639 } catch (error) {
640 throw Object.assign(
641 new Error(
642 `${command} was ended at its limit of ${limitText(timeoutMs ?? 30000)} or could not run: ${describe(error)}`,
643 ),
644 { ended: true },
645 )
646 }
647
648 if (result.exitCode !== 0) {
649 throw new Error(`${command} exited with ${result.exitCode}: ${String(result.stderr ?? '').trim().split('\n')[0]}`)
650 }
651
652 return { stdout: result.stdout.trim(), stderr: String(result.stderr ?? '') }
653}
654