SLOPSHOPPER

debrief-probe

Throwaway P0 probe: records session metadata only (no text) to verify mod API assumptions.

newguardcommandstatusprompttimer
v0.0.1MITupdated 2026-10-16matrix-freed/debrief/probe
A shopper browsing a rack in a slop shop
Preview · a replayed session in a sandbox
claude · ~/work/app · debrief-probe
› fix the failing auth test and add an audit log call ⏺ Read(src/auth.ts) ⎿ Read 6 lines ⏺ Update(src/auth.ts) ⎿ Added 2 lines, removed 1 line ⏺ Bash(bun test) ⎿ 3 pass, 1 fail ● Done. refresh now rejects expired claims and logs an audit event. ✻ Worked for 42s · done 4:20 PM › /probe-status ⎿ debrief-probe: debrief probe run 1760000000020-d9ef: 0 steps, 1 turns, 0 loops, 0 agents, reloads 0. probe 0k tok, main 0% ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── › ? for shortcuts
README

debrief

A Claude Code mod that records session metadata only (models, tokens, the engine cost ledger, time, subagents, friction counts) and turns it into a project after-action review (AAR): plan, actual, gap, improvement, with charts and copy-paste improvement prompts.

한국어는 아래에 있습니다.


What it records, and what it never records

Recorded, per process run, as one JSON file on your own machine:

  • tokens per loop (main or each subagent) and per model: input, output, cache read, cache write, requests
  • effort level counts, request timestamps (a bounded ring), turn timing
  • the engine cost ledger snapshot (session cumulative, API-equivalent USD)
  • subagents: declared type, model, first seen, duration, spawn attempts
  • friction counters: aborted requests, tool errors, rejected tools, compactions
  • tool and skill names with counts; session message counts by origin kind
  • instruction file fingerprints as hashes only (path hash, content hash, size)
  • phase tags you type yourself with /debrief tag <label> (the only text kept)

Never recorded: prompts, responses, tool inputs (including file paths and commands), command arguments, skill bodies, subagent descriptions or prompts, session message bodies. Reports show sessions as S1, S2, agents as S1-A1, external runs as X1; session IDs, paths and project folder names never reach the HTML. Every report value passes an export allowlist (key paths) and a value check (home paths, UUIDs, request IDs) before it is written.

Structure

PartWhereRole
Core recorder (mod)debrief/hooks/Writes <root>/runs/<project>/<session>/<run>.json. No vendor, machine, team or person names in code
Report programdebrief/tools/debrief_report/Python 3 standard library only. Backfill, AAR, summary, compaction, ledger
Adapters (optional)same programExternal agent logs (Codex rollouts) and /insights data; silently skipped when absent
Profile (optional)<root>/profile.jsonLanguage, lanes you declare, thresholds, external agent types. Built by /debrief setup. Never committed. Without one, reports and the setup dialogue follow the terminal locale (LC_ALL, then LANG: ko_* is Korean, anything else English)

<root> is DEBRIEF_DIR, else <CLAUDE_CONFIG_DIR or ~/.claude>/debrief.

Requirements

  • Claude Code 2.1.288 or later (mods are early access)
  • macOS, Windows or Linux
  • Python 3 for the report commands (no packages to install). Without it the recorder still works and the report commands print an install hint.

Install

/plugin marketplace add matrix-freed/debrief
/plugin install debrief@debrief

On Claude Code 2.1.275 or later this also works in one step:

/plugin install debrief --marketplace matrix-freed/debrief

Let Claude install it

Paste this into a Claude Code session:

Install the debrief plugin for me: add the marketplace matrix-freed/debrief, install
debrief@debrief, then tell me to restart the session. After the restart, run /debrief setup
and walk me through the questions. Do not change any other settings.

Commands

CommandWhat it does
/debriefThis session in numbers (tokens by model and loop, cost, time, agents)
/debrief tag <label>Marks a phase; the AAR timeline draws a band from here to the next tag
/debrief setupBuilds your profile (language, lanes, thresholds)
/debrief aar [--from ISO] [--to ISO] [--cards PATH]Project AAR HTML for the current project, written under <root>/aar/
/debrief backfill [--days N]Imports closed past sessions from transcripts (metadata only)
/debrief compact [--dry-run]Folds run files older than 30 days into monthly rollups and archives the raw files
`/debrief ledger [list [--status S] \set ID STATUS]`Improvement ledger: proposed, applied, verified, dropped

The plan, actual, gap and improvement cards are written by people (or a session you ask) in a small cards.json; debrief does not generate judgement text. Improvements get ledger IDs and a self-contained apply prompt with a copy button.

Running the report program directly

python3 debrief/tools/debrief_report backfill --config ~/.claude --root <root> --days 30
python3 debrief/tools/debrief_report aar --project <project folder name> --cards cards.json \
  --root <root> --tz +09:00 --out aar.html
python3 debrief/tools/debrief_report summary --session <session id> --root <root> --out summary.html
python3 debrief/tools/debrief_report compact --root <root> --dry-run
python3 debrief/tools/debrief_report ledger --root <root> list

On Windows use py -3 instead of python3. Every command prints a final RESULT {...} line.

Cost label

Cost is shown as "engine ledger (API equivalent), session cumulative": the engine's own running total, summed as per-run differences. On a subscription plan it is an API-equivalent figure, not a bill. Sessions without a ledger count as zero and are flagged as estimated. Processed tokens are input + cache write + output; cache reads are shown separately.

Privacy and what this is not

  • Records stay on your machine. Nothing is uploaded; there is no server.
  • There is no per-person ranking or comparison. The AAR is about a project's way of working (where time and tokens went, what to change next), and its hypotheses are phrased as questions, not verdicts.
  • Sharing is opt-in: the profile has shareConsent: false by default, and reports only contain what the export allowlist lets through.

Status

P0 (API probe), P1 (recorder) and P2 (report program, backfill, project AAR) are done; see P0-RESULTS.md, P1-RESULTS.md, P2-RESULTS.md. Known gaps from the first real AAR:

  • sessions recorded before the mod was enabled have no cost ledger (counted as zero, flagged)
  • automated one-shot SDK sessions are folded into one row by a simple rule (entrypoint and length)
  • child sessions started for experiments carry no parent marker unless the parent sets DEBRIEF_PARENT_SESSION
  • the --from/--to window selects sessions but does not clip inside a session
  • open-session detection for backfill uses file idle time

License

MIT. See LICENSE.


한국어

debrief

Claude Code mod다. 세션 메타데이터만(모델, 토큰, 엔진 비용 장부, 시간, 서브에이전트, 마찰 횟수) 기록하고, 이를 프로젝트 AAR(사후 검토: 계획, 실제, 차이, 개선)로 만든다. AAR에는 도식과 복사해 쓰는 개선 프롬프트가 들어간다.

기록하는 것과 절대 기록하지 않는 것

프로세스 실행 단위로, 본인 PC에 JSON 파일 하나로 남긴다.

  • 루프(메인 또는 서브에이전트별)·모델별 토큰: 입력, 출력, 캐시 읽기, 캐시 쓰기, 요청 수
  • effort 단계별 횟수, 요청 시각(크기 제한 링 버퍼), 턴 시간
  • 엔진 비용 장부 스냅샷(세션 누적, API 환산 USD)
  • 서브에이전트: 선언 유형, 모델, 처음 본 시각, 지속 시간, spawn 시도 수
  • 마찰 카운터: 중단된 요청, 도구 오류, 거부된 도구, 압축
  • 도구·스킬 이름과 횟수, 세션 간 메시지의 출처 종류별 횟수
  • 지침 파일 지문(해시만: 경로 해시, 내용 해시, 크기)
  • /debrief tag <라벨>로 직접 입력한 페이즈 라벨(유일하게 남기는 텍스트)

절대 기록하지 않는 것: 프롬프트, 응답, 도구 입력(파일 경로와 명령 포함), 명령 인자, 스킬 본문, 서브에이전트 설명·프롬프트, 세션 간 메시지 본문. 리포트에서 세션은 S1, S2, 에이전트는 S1-A1, 외부 실행은 X1로만 나온다. 세션 ID·경로·프로젝트 폴더 이름은 HTML에 들어가지 않는다. 리포트의 모든 값은 쓰기 전에 내보내기 허용 목록(키 경로)과 값 검사(홈 경로, UUID, 요청 ID)를 통과해야 한다.

구조

부분위치역할
코어 기록기(mod)debrief/hooks/<root>/runs/<프로젝트>/<세션>/<run>.json을 쓴다. 코드에 벤더·머신·팀·사람 이름 없음
리포트 프로그램debrief/tools/debrief_report/Python 3 표준 라이브러리만. 백필, AAR, 서머리, 압축, 장부
어댑터(선택)같은 프로그램외부 에이전트 로그(Codex rollout)와 /insights 데이터. 없으면 조용히 건너뜀
프로필(선택)<root>/profile.json언어, 직접 선언한 레인, 임계값, 외부 에이전트 유형. /debrief setup이 만든다. 저장소에 넣지 않는다. 프로필이 없으면 보고서와 setup 대화는 터미널 로케일(LC_ALL, 없으면 LANG: ko_*면 한국어, 그 외 영어)을 따른다

<root>는 DEBRIEF_DIR, 없으면 <CLAUDE_CONFIG_DIR 또는 ~/.claude>/debrief.

요구사항

  • Claude Code 2.1.288 이상(mods는 얼리 액세스)
  • macOS, Windows, Linux
  • 리포트 명령에는 Python 3(설치할 패키지 없음). 없어도 기록기는 동작하고, 리포트 명령은 설치 안내만 출력한다.

설치

/plugin marketplace add matrix-freed/debrief
/plugin install debrief@debrief

Claude Code 2.1.275 이상에서는 한 줄로도 된다.

/plugin install debrief --marketplace matrix-freed/debrief

Claude에게 설치 맡기기

Claude Code 세션에 그대로 붙여 넣는다.

debrief 플러그인을 설치해 줘. 마켓플레이스 matrix-freed/debrief 를 추가하고
debrief@debrief 를 설치한 뒤, 세션을 다시 시작하라고 알려 줘. 재시작 후 /debrief setup 을
실행해서 질문에 답하도록 안내해 줘. 다른 설정은 바꾸지 마.

명령

명령하는 일
/debrief이 세션의 숫자(모델·루프별 토큰, 비용, 시간, 에이전트)
/debrief tag <라벨>페이즈 표시. AAR 타임라인에 여기서 다음 태그까지 띠로 그린다
/debrief setup프로필 만들기(언어, 레인, 임계값)
/debrief aar [--from ISO] [--to ISO] [--cards 경로]현재 프로젝트의 AAR HTML. <root>/aar/ 아래에 쓴다
/debrief backfill [--days N]지난 종료 세션을 transcript에서 가져온다(메타데이터만)
/debrief compact [--dry-run]30일 지난 run 파일을 월별 rollup으로 접고 원본은 아카이브로 옮긴다
`/debrief ledger [list [--status S] \set ID 상태]`개선 장부: proposed, applied, verified, dropped

계획·실제·차이·개선 카드는 사람(또는 요청받은 세션)이 작은 cards.json에 쓴다. debrief는 판정 문장을 만들지 않는다. 개선 항목에는 장부 ID와 자기완결 적용 프롬프트, 복사 버튼이 붙는다.

리포트 프로그램 직접 실행

python3 debrief/tools/debrief_report backfill --config ~/.claude --root <root> --days 30
python3 debrief/tools/debrief_report aar --project <프로젝트 폴더 이름> --cards cards.json \
  --root <root> --tz +09:00 --out aar.html
python3 debrief/tools/debrief_report summary --session <세션 ID> --root <root> --out summary.html
python3 debrief/tools/debrief_report compact --root <root> --dry-run
python3 debrief/tools/debrief_report ledger --root <root> list

Windows에서는 python3 대신 py -3. 모든 명령은 마지막에 RESULT {...} 줄을 출력한다.

비용 표기

비용은 "엔진 장부(API 환산), 세션 누적"으로 표시한다. 엔진이 직접 쌓는 누적값을 run별 차로 더한 것이다. 구독 요금제에서는 청구액이 아니라 API 환산값이다. 장부가 없는 세션은 0으로 세고 추정으로 표시한다. 처리 토큰은 입력 + 캐시 쓰기 + 출력이고, 캐시 읽기는 따로 보여 준다.

프라이버시, 그리고 이것이 아닌 것

  • 기록은 본인 PC에만 남는다. 업로드도, 서버도 없다.
  • 사람별 순위나 비교는 없다. AAR은 프로젝트의 일하는 방식(시간과 토큰이 어디에 갔고 다음에 무엇을 바꿀지)을 다루며, 가설은 판정이 아니라 질문으로 쓴다.
  • 공유는 선택이다. 프로필 기본값은 shareConsent: false이고, 리포트에는 내보내기 허용 목록을 통과한 값만 들어간다.

상태

P0(API 실측), P1(기록기), P2(리포트 프로그램, 백필, 프로젝트 AAR) 완료. P0-RESULTS.md, P1-RESULTS.md, P2-RESULTS.md 참고. 첫 실데이터 AAR에서 확인한 알려진 갭:

  • mod를 켜기 전에 기록된 세션은 비용 장부가 없다(0으로 세고 추정 표시)
  • 자동 단발 SDK 세션은 간단한 규칙(진입점과 길이)으로 한 줄에 접는다
  • 실험용으로 띄운 자식 세션은 부모가 DEBRIEF_PARENT_SESSION을 넘기지 않으면 부모 표식이 없다
  • --from/--to 창은 세션을 고를 뿐 세션 안을 자르지 않는다
  • 백필의 열린 세션 판정은 파일 유휴 시간 기준이다

라이선스

MIT. LICENSE 참고.

Source 3 files
hooks/register.ts 559 lines
1import { atom, read, update } from 'claude-code'
2import type { EngineInterface, Register } from 'claude-code'
3
4import type { ProbeRun, ProbeUsage } from '../types'
5import { bashCommandOf, isCodexCli, isRemoteDirective } from './classify'
6
7// Throwaway P0 probe. Records metadata only: event names, ids, counts, token
8// numbers, models. Never prompt/answer text, tool input, command args or paths.
9
10const RUN_REF = { plugin: 'debrief-probe', key: 'run' } as const
11const RUN = atom(RUN_REF, null as ProbeRun | null)
12
13const STEP_CAP = 2000
14const TURN_CAP = 1000
15
16// Module-local counters for places where `$` cannot be called (a .catch on
17// re-entry). Merged into state at each turn end; lost on reload by design.
18const pendingToolCalls = { main: 0, agent: 0, reentry: {} as Record<string, number>, byLoop: {} as Record<string, number> }
19const writes = { ok: 0, failed: 0, lastError: undefined as string | undefined }
20let writeChain: Promise<void> = Promise.resolve()
21let dirty = false
22
23type $ = EngineInterface
24
25const bump = (counts: Record<string, number>, key: string, by = 1) => {
26  counts[key] = (counts[key] ?? 0) + by
27}
28
29const sha = async (text: string): Promise<string> => {
30  const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(text))
31  return [...new Uint8Array(digest)].slice(0, 8).map(b => b.toString(16).padStart(2, '0')).join('')
32}
33
34const hex4 = () => {
35  const bytes = new Uint8Array(2)
36  crypto.getRandomValues(bytes)
37  return [...bytes].map(b => b.toString(16).padStart(2, '0')).join('')
38}
39
40const dataDir = async ($: $): Promise<{ dir: string; fromEnv: boolean }> => {
41  const fromEnv = await $.env.get('DEBRIEF_PROBE_DIR')
42  if (fromEnv) return { dir: fromEnv.replace(/[\\/]$/, ''), fromEnv: true }
43  const home = (await $.env.get('HOME')) ?? (await $.env.get('USERPROFILE')) ?? '.'
44  return { dir: `${home}/.debrief/probe`, fromEnv: false }
45}
46
47const freshRun = async ($: $, surface: string | null, isInteractive: boolean, rotatedBy?: string): Promise<ProbeRun> => {
48  const now = await $.clock.now()
49  const version = await $.session.version()
50  const { fromEnv } = await dataDir($)
51  return {
52    schema: 'probe-1',
53    engine: version.version,
54    sessionId: await $.session.id(),
55    runId: `${now}-${hex4()}`,
56    startedAt: now,
57    ...(rotatedBy ? { rotatedBy } : {}),
58    surface,
59    isInteractive,
60    reloads: 0,
61    lastCheckpointAt: now,
62    env: {
63      project: (await $.env.get('DEBRIEF_PROJECT')) ?? undefined,
64      parentSession: (await $.env.get('DEBRIEF_PARENT_SESSION')) ?? undefined,
65      dirFromEnv: fromEnv,
66    },
67    events: {},
68    classicSessionStart: [],
69    usage: {},
70    steps: [],
71    turns: [],
72    agents: {},
73    spawnDenied: 0,
74    toolCall: { main: 0, agent: 0, reentry: {}, byLoop: {} },
75    requestedTools: {},
76    skills: {},
77    commands: {},
78    commandArgsChars: 0,
79    messaging: { sent: {}, received: {} },
80    classify: { remoteDirective: 0, codexCli: 0, bashSteps: 0 },
81    friction: {
82      abortedTurns: 0,
83      toolErrors: 0,
84      toolInterrupts: 0,
85      permissionDenied: 0,
86      compactions: 0,
87      compactEvents: 0,
88      stopFailures: 0,
89    },
90    instructions: [],
91    cost: { provenance: 'unknown' },
92    context: { measures: 0 },
93    status: { updates: 0 },
94    setup: { asked: 0, answered: 0 },
95    insights: { sessionMetaDir: false, facetsDir: false, sessionMetaFiles: 0 },
96    writes: { ok: 0, failed: 0 },
97  }
98}
99
100// Applies fn to the current run (if any) and marks it dirty.
101// Events that arrive before session.start made a run (classic SessionStart,
102// InstructionsLoaded at startup) wait here and are applied once a run exists.
103let early: ((run: ProbeRun) => void)[] = []
104
105const mutate = async ($: $, fn: (run: ProbeRun) => void) => {
106  await update($, RUN, run => {
107    if (run === null) {
108      early.push(fn)
109      return run
110    }
111    const copy = structuredClone(run) as ProbeRun
112    fn(copy)
113    return copy
114  })
115  dirty = true
116}
117
118const applyEarly = (run: ProbeRun) => {
119  for (const fn of early) {
120    fn(run)
121    bump(run.events, 'early-buffered')
122  }
123  early = []
124}
125
126const flush = ($: $): Promise<void> => {
127  writeChain = writeChain.then(async () => {
128    const run = await read($, RUN)
129    if (run === null) return
130    const { dir } = await dataDir($)
131    const now = await $.clock.now()
132    const out: ProbeRun = {
133      ...run,
134      lastCheckpointAt: now,
135      writes: { ok: writes.ok + 1, failed: writes.failed, ...(writes.lastError ? { lastError: writes.lastError } : {}) },
136    }
137    try {
138      await $.fs.write(`${dir}/${run.sessionId}/${run.runId}.json`, JSON.stringify(out, null, 1))
139      writes.ok += 1
140      dirty = false
141    } catch (error) {
142      writes.failed += 1
143      writes.lastError = error instanceof Error ? error.name : 'unknown'
144    }
145  }).catch(() => undefined)
146  return writeChain
147}
148
149const mergePendingToolCalls = (run: ProbeRun) => {
150  run.toolCall.main += pendingToolCalls.main
151  run.toolCall.agent += pendingToolCalls.agent
152  for (const [k, n] of Object.entries(pendingToolCalls.reentry)) bump(run.toolCall.reentry, k, n)
153  for (const [k, n] of Object.entries(pendingToolCalls.byLoop)) bump(run.toolCall.byLoop, k, n)
154  pendingToolCalls.main = 0
155  pendingToolCalls.agent = 0
156  pendingToolCalls.reentry = {}
157  pendingToolCalls.byLoop = {}
158}
159
160// Opens a new run when none is live for the current session id (after /clear).
161const ensureCurrent = async ($: $, why: string) => {
162  const run = await read($, RUN)
163  const sid = await $.session.id()
164  if (run !== null && run.sessionId === sid && run.endedAt === undefined) return
165  const surface = await $.session.surface()
166  const next = await freshRun($, surface, run?.isInteractive ?? surface !== null, why)
167  applyEarly(next)
168  await $.state.set(RUN_REF, next)
169  await flush($)
170}
171
172const statusText = (run: ProbeRun): string => {
173  let total = 0
174  let main = 0
175  for (const [loop, byModel] of Object.entries(run.usage)) {
176    for (const u of Object.values(byModel)) {
177      const t = u.in + u.out + u.cacheRead + u.cacheWrite
178      total += t
179      if (loop === 'main') main += t
180    }
181  }
182  const share = total === 0 ? 0 : Math.round((main / total) * 100)
183  return `probe ${Math.round(total / 1000)}k tok, main ${share}%`
184}
185
186const insightsCheck = async ($: $, run: ProbeRun) => {
187  // Fail-open: the /insights output is another tool's; read listings only.
188  try {
189    const home = (await $.env.get('HOME')) ?? (await $.env.get('USERPROFILE')) ?? '.'
190    const base = `${home}/.claude/usage-data`
191    run.insights.sessionMetaDir = await $.fs.exists(`${base}/session-meta`)
192    run.insights.facetsDir = await $.fs.exists(`${base}/facets`)
193    if (run.insights.sessionMetaDir) {
194      const list = await $.fs.list(`${base}/session-meta`)
195      run.insights.sessionMetaFiles = list.filter(f => f.name.endsWith('.json')).length
196    }
197  } catch (error) {
198    run.insights.error = error instanceof Error ? error.name : 'unknown'
199  }
200}
201
202// S16a fallback: fingerprint the instruction files the engine would load,
203// without waiting for InstructionsLoaded. Hashes only; no path or text kept.
204const scanInstructions = async ($: $, run: ProbeRun) => {
205  try {
206    const found = await $.fs.ancestors({ names: ['CLAUDE.md', 'CLAUDE.local.md'] })
207    const configDir = (await $.env.get('CLAUDE_CONFIG_DIR')) ?? `${(await $.env.get('HOME')) ?? '.'}/.claude`
208    const files = found.map(f => ({ path: `${f.dir}/${f.name}`, text: f.content, memoryType: 'Project' }))
209    const userPath = `${configDir}/CLAUDE.md`
210    if (await $.fs.exists(userPath)) files.push({ path: userPath, text: await $.fs.read(userPath), memoryType: 'User' })
211    for (const f of files) {
212      const stat = await $.fs.stat(f.path).catch(() => undefined)
213      run.instructions.push({
214        memoryType: f.memoryType,
215        loadReason: 'probe-scan',
216        pathHash: await sha(f.path),
217        contentHash: await sha(f.text),
218        bytes: new TextEncoder().encode(f.text).length,
219        mtimeMs: stat?.mtimeMs ?? 0,
220      })
221    }
222  } catch (error) {
223    bump(run.events, `instructions-scan-failed:${error instanceof Error ? error.name : 'unknown'}`)
224  }
225}
226
227export const register: Register = on => {
228  on('session.start', async ($, e, next) => {
229    const sid = await $.session.id()
230    const current = await read($, RUN)
231    if (current !== null && current.sessionId === sid && current.endedAt === undefined) {
232      const run = structuredClone(current) as ProbeRun
233      run.reloads += 1
234      bump(run.events, 'session.start')
235      if (run.instructions.length === 0) await scanInstructions($, run)
236      await $.state.set(RUN_REF, run)
237    } else {
238      const run = await freshRun($, e.surface, e.isInteractive)
239      bump(run.events, 'session.start')
240      applyEarly(run)
241      await scanInstructions($, run)
242      await insightsCheck($, run)
243      await $.state.set(RUN_REF, run)
244    }
245    await $.command.register({ name: 'probe-status', description: 'debrief probe: show this run (metadata only)' })
246    await $.command.register({ name: 'probe-setup', description: 'debrief probe: ask one structured question and store the answer' })
247    $.clock.every(15000, () => {
248      if (dirty) void flush($)
249    })
250    await flush($)
251    return next(e)
252  })
253
254  on('classic.SessionStart', async ($, e, next) => {
255    if (e.source === 'clear') await ensureCurrent($, 'classic-clear')
256    await mutate($, run => {
257      bump(run.events, 'classic.SessionStart')
258      run.classicSessionStart.push(e.source)
259    })
260    return next(e)
261  })
262
263  on('classic.InstructionsLoaded', async ($, e, next) => {
264    try {
265      const [text, stat] = await Promise.all([$.fs.read(e.file_path), $.fs.stat(e.file_path)])
266      const entry = {
267        memoryType: e.memory_type,
268        loadReason: e.load_reason,
269        pathHash: await sha(e.file_path),
270        contentHash: await sha(text),
271        bytes: new TextEncoder().encode(text).length,
272        mtimeMs: stat.mtimeMs,
273      }
274      await mutate($, run => {
275        bump(run.events, 'classic.InstructionsLoaded')
276        run.instructions.push(entry)
277      })
278    } catch {
279      await mutate($, run => bump(run.events, 'classic.InstructionsLoaded.unreadable'))
280    }
281    return next(e)
282  })
283
284  on('turn.start', async ($, e, next) => {
285    if (e.agentId === undefined) await ensureCurrent($, 'turn-start-mismatch')
286    await mutate($, run => {
287      bump(run.events, 'turn.start')
288      const agent = e.agentId === undefined ? undefined : run.agents[e.agentId]
289      if (agent) agent.seen.start += 1
290    })
291    return next(e)
292  })
293
294  on('turn.step', async function* ($, e, next) {
295    const result = yield* next(e)
296    const loop = e.agentId ?? 'main'
297    const now = await $.clock.now()
298    let remote = 0
299    let codex = 0
300    let bash = 0
301    for (const use of result.toolUses) {
302      if (use.name !== 'Bash') continue
303      const command = bashCommandOf(use.input)
304      if (command === undefined) continue
305      bash += 1
306      if (isRemoteDirective(command)) remote += 1
307      if (isCodexCli(command)) codex += 1
308    }
309    let status: string | undefined
310    await mutate($, run => {
311      bump(run.events, 'turn.step')
312      const model = result.usage?.model ?? e.model
313      const byModel = (run.usage[loop] ??= {})
314      const u: ProbeUsage = (byModel[model] ??= { in: 0, out: 0, cacheRead: 0, cacheWrite: 0, steps: 0, nullUsageSteps: 0 })
315      u.steps += 1
316      if (result.usage === null) u.nullUsageSteps += 1
317      else {
318        u.in += result.usage.input_tokens
319        u.out += result.usage.output_tokens
320        u.cacheRead += result.usage.cache_read_input_tokens
321        u.cacheWrite += result.usage.cache_creation_input_tokens
322      }
323      run.steps.push({
324        loop,
325        turnId: e.turnId,
326        index: e.index,
327        model,
328        ...(e.effort === undefined ? {} : { effort: String(e.effort) }),
329        stopReason: result.stopReason,
330        usage:
331          result.usage === null
332            ? null
333            : {
334                in: result.usage.input_tokens,
335                out: result.usage.output_tokens,
336                cacheRead: result.usage.cache_read_input_tokens,
337                cacheWrite: result.usage.cache_creation_input_tokens,
338              },
339        tools: result.toolUses.map(t => t.name),
340        at: now,
341      })
342      if (run.steps.length > STEP_CAP) run.steps.splice(0, run.steps.length - STEP_CAP)
343      const tools = (run.requestedTools[loop] ??= {})
344      for (const t of result.toolUses) bump(tools, t.name)
345      run.classify.bashSteps += bash
346      run.classify.remoteDirective += remote
347      run.classify.codexCli += codex
348      const agent = e.agentId === undefined ? undefined : run.agents[e.agentId]
349      if (agent) agent.seen.steps += 1
350      status = statusText(run)
351      run.status.updates += 1
352      run.status.last = status
353    })
354    if (status !== undefined) $.ui.status(status)
355    return result
356  })
357
358  on('turn.complete', async ($, e, next) => {
359    await mutate($, run => {
360      bump(run.events, 'turn.complete')
361      mergePendingToolCalls(run)
362      const loop = e.agentId ?? 'main'
363      run.turns.push({
364        loop,
365        turnId: e.turnId,
366        hadStart: run.steps.some(s => s.turnId === e.turnId),
367        durationMs: e.durationMs,
368        reason: e.reason,
369        isAborted: e.isAborted,
370        ...(e.usage ? { usageModel: e.usage.model } : {}),
371      })
372      if (run.turns.length > TURN_CAP) run.turns.splice(0, run.turns.length - TURN_CAP)
373      if (e.isAborted) run.friction.abortedTurns += 1
374      const agent = e.agentId === undefined ? undefined : run.agents[e.agentId]
375      if (agent) agent.seen.complete += 1
376    })
377    await flush($)
378    return next(e)
379  })
380
381  on('agent.spawn', async ($, e, next) => {
382    // P0 test switch: a `.force-fg` file in the data dir asks for a foreground run.
383    const { dir } = await dataDir($)
384    const forceForeground = e.background && e.workflow === undefined && (await $.fs.exists(`${dir}/.force-fg`))
385    const result = await next(forceForeground ? { ...e, background: false } : e)
386    if (forceForeground) await mutate($, run => bump(run.events, 'agent.spawn.forcedForeground'))
387    const now = await $.clock.now()
388    await mutate($, run => {
389      bump(run.events, 'agent.spawn')
390      if (result.deny !== undefined || result.agentId === undefined) {
391        run.spawnDenied += 1
392        return
393      }
394      const prior = run.agents[result.agentId]
395      run.agents[result.agentId] = {
396        subagentType: e.subagentType,
397        model: result.model,
398        parentModel: e.parentModel,
399        ...(e.parentAgentId ? { parentAgentId: e.parentAgentId } : {}),
400        background: e.background,
401        fork: e.fork,
402        teammate: e.isTeammate === true,
403        ...(e.workflow ? { workflowRunId: e.workflow.runId, workflowIndex: e.workflow.agentIndex } : {}),
404        resumed: e.resume !== undefined,
405        spawnedAt: now,
406        seen: prior?.seen ?? { start: 0, steps: 0, complete: 0 },
407      }
408    })
409    await flush($)
410    return result
411  })
412
413  on('tool.call', async ($, e, next) => {
414    if (e.agentId === undefined) pendingToolCalls.main += 1
415    else pendingToolCalls.agent += 1
416    bump(pendingToolCalls.byLoop, e.agentId === undefined ? 'main' : 'agent')
417    const ran = await next(e)
418    if (ran.deny !== undefined) bump(pendingToolCalls.byLoop, 'result:deny')
419    else if (ran.isError === true) bump(pendingToolCalls.byLoop, 'result:error')
420    return ran
421  }).catch(($, e, next) => {
422    bump(pendingToolCalls.reentry, next.error.kind)
423    return next(e)
424  })
425
426  on('skill.prompt', async ($, e, next) => {
427    await mutate($, run => bump(run.skills, e.skill))
428    return next(e)
429  })
430
431  on('command.run', async ($, e, next) => {
432    await mutate($, run => {
433      bump(run.commands, e.command)
434      run.commandArgsChars += e.args.length
435    })
436    return next(e)
437  })
438
439  on('command.run', { command: 'probe-status' }, async $ => {
440    const run = await read($, RUN)
441    if (run === null) return { text: 'debrief probe: no run.' }
442    const loops = Object.keys(run.usage).length
443    return {
444      text: `debrief probe run ${run.runId}: ${run.steps.length} steps, ${run.turns.length} turns, ${loops} loops, ${Object.keys(run.agents).length} agents, reloads ${run.reloads}. ${statusText(run)}`,
445    }
446  })
447
448  on('command.run', { command: 'probe-setup' }, async $ => {
449    await mutate($, run => {
450      run.setup.asked += 1
451    })
452    try {
453      const label = await $.ui.ask('debrief probe: which test label should be stored?', {
454        options: ['alpha', 'beta'],
455        header: 'probe',
456      })
457      const stored = label === 'alpha' || label === 'beta' ? label : 'other'
458      await $.store.set('setupAnswer', stored)
459      await mutate($, run => {
460        run.setup.answered += 1
461        run.setup.answerLabel = stored
462      })
463      await flush($)
464      return { text: `debrief probe: stored ${stored}.` }
465    } catch (error) {
466      await mutate($, run => {
467        run.setup.error = error instanceof Error ? error.name : 'rejected'
468      })
469      await flush($)
470      return { text: 'debrief probe: no answer.' }
471    }
472  })
473
474  on('session.send', async ($, e, next) => {
475    const result = await next(e)
476    await mutate($, run => {
477      bump(run.events, 'session.send')
478      const slot = (run.messaging.sent[e.origin.kind] ??= { queued: 0, refused: 0 })
479      if (result.isDelivered) slot.queued += 1
480      else slot.refused += 1
481    })
482    return result
483  })
484
485  on('session.receive', async ($, e, next) => {
486    await mutate($, run => {
487      bump(run.events, 'session.receive')
488      bump(run.messaging.received, e.origin.kind)
489    })
490    return next(e)
491  })
492
493  on('session.measure', async ($, e, next) => {
494    const now = await $.clock.now()
495    await mutate($, run => {
496      bump(run.events, 'session.measure')
497      run.context.measures += 1
498      if (e.context.percent !== undefined) run.context.lastPercent = e.context.percent
499      if (e.cost !== undefined) run.cost = { usd: e.cost.usd, at: now, provenance: 'engine-ledger' }
500    })
501    return next(e)
502  })
503
504  on('session.compact', async ($, e, next) => {
505    await mutate($, run => {
506      run.friction.compactEvents += 1
507    })
508    return next(e)
509  })
510
511  on('classic.PreCompact', async ($, e, next) => {
512    await mutate($, run => {
513      run.friction.compactions += 1
514    })
515    return next(e)
516  })
517
518  on('classic.PostToolUseFailure', async ($, e, next) => {
519    await mutate($, run => {
520      run.friction.toolErrors += 1
521      if (e.is_interrupt === true) run.friction.toolInterrupts += 1
522    })
523    return next(e)
524  })
525
526  on('classic.PermissionDenied', async ($, e, next) => {
527    await mutate($, run => {
528      run.friction.permissionDenied += 1
529    })
530    return next(e)
531  })
532
533  on('classic.StopFailure', async ($, e, next) => {
534    await mutate($, run => {
535      run.friction.stopFailures += 1
536    })
537    return next(e)
538  })
539
540  on('session.end', async ($, e, next) => {
541    const now = await $.clock.now()
542    let cost: number | undefined
543    try {
544      cost = (await $.session.usage()).cost?.usd
545    } catch {
546      cost = undefined
547    }
548    await mutate($, run => {
549      bump(run.events, 'session.end')
550      mergePendingToolCalls(run)
551      run.endedAt = now
552      run.endReason = e.reason
553      if (cost !== undefined) run.cost = { usd: cost, at: now, provenance: 'engine-ledger' }
554    })
555    await flush($)
556    return next(e)
557  })
558}
559
hooks/classify.ts 16 lines
1// Pure classifiers over a Bash command string. The caller keeps only the
2// boolean; the command text is dropped right after the test.
3
4const REMOTE_DIRECTIVE = /\bssh\s+(?:-\S+\s+)*remote-host\b[\s\S]*\btmux\s+send-keys\b/
5const CODEX_CLI = /(?:^|[\s;&|(])codex(?:-companion)?\s+(?:exec|e|review)\b/
6
7export const isRemoteDirective = (command: string): boolean => REMOTE_DIRECTIVE.test(command)
8
9export const isCodexCli = (command: string): boolean => CODEX_CLI.test(command)
10
11export const bashCommandOf = (input: unknown): string | undefined => {
12  if (typeof input !== 'object' || input === null) return undefined
13  const command = (input as { command?: unknown }).command
14  return typeof command === 'string' ? command : undefined
15}
16
types/index.d.ts 112 lines
1// debrief-probe state contract. Metadata only: no prompt, answer, tool input,
2// command argument or path text is ever held here.
3
4export type ProbeCounts = { [name: string]: number }
5
6export type ProbeUsage = {
7  in: number
8  out: number
9  cacheRead: number
10  cacheWrite: number
11  steps: number
12  nullUsageSteps: number
13}
14
15export type ProbeStep = {
16  loop: string
17  turnId: string
18  index: number
19  model: string
20  effort?: string
21  stopReason: string | null
22  usage: { in: number; out: number; cacheRead: number; cacheWrite: number } | null
23  tools: string[]
24  at: number
25}
26
27export type ProbeTurn = {
28  loop: string
29  turnId: string
30  hadStart: boolean
31  durationMs: number
32  reason: string
33  isAborted: boolean
34  usageModel?: string
35}
36
37export type ProbeAgent = {
38  subagentType: string
39  model: string
40  parentModel: string
41  parentAgentId?: string
42  background: boolean
43  fork: boolean
44  teammate: boolean
45  workflowRunId?: string
46  workflowIndex?: number
47  resumed: boolean
48  spawnedAt: number
49  seen: { start: number; steps: number; complete: number }
50}
51
52export type ProbeInstruction = {
53  memoryType: string
54  loadReason: string
55  pathHash: string
56  contentHash: string
57  bytes: number
58  mtimeMs: number
59}
60
61export type ProbeRun = {
62  schema: 'probe-1'
63  engine: string
64  sessionId: string
65  runId: string
66  startedAt: number
67  endedAt?: number
68  endReason?: string
69  rotatedBy?: string
70  surface: string | null
71  isInteractive: boolean
72  reloads: number
73  lastCheckpointAt: number
74  env: { project?: string; parentSession?: string; dirFromEnv: boolean }
75  events: ProbeCounts
76  classicSessionStart: string[]
77  usage: { [loop: string]: { [model: string]: ProbeUsage } }
78  steps: ProbeStep[]
79  turns: ProbeTurn[]
80  agents: { [agentId: string]: ProbeAgent }
81  spawnDenied: number
82  toolCall: { main: number; agent: number; reentry: ProbeCounts; byLoop: ProbeCounts }
83  requestedTools: { [loop: string]: ProbeCounts }
84  skills: ProbeCounts
85  commands: ProbeCounts
86  commandArgsChars: number
87  messaging: { sent: { [kind: string]: { queued: number; refused: number } }; received: ProbeCounts }
88  classify: { remoteDirective: number; codexCli: number; bashSteps: number }
89  friction: {
90    abortedTurns: number
91    toolErrors: number
92    toolInterrupts: number
93    permissionDenied: number
94    compactions: number
95    compactEvents: number
96    stopFailures: number
97  }
98  instructions: ProbeInstruction[]
99  cost: { usd?: number; at?: number; provenance: 'engine-ledger' | 'unknown' }
100  context: { measures: number; lastPercent?: number }
101  status: { updates: number; last?: string }
102  setup: { asked: number; answered: number; answerLabel?: string; error?: string }
103  insights: { sessionMetaDir: boolean; facetsDir: boolean; sessionMetaFiles: number; error?: string }
104  writes: { ok: number; failed: number; lastError?: string }
105}
106
107declare module 'claude-code' {
108  interface PluginState {
109    'debrief-probe': { run: ProbeRun | null }
110  }
111}
112