SLOPSHOPPER

stall-watch

Auto-retries transient API failures (capped), pins a usage-limit countdown, and can resume after a limit reset (opt-in via /stall-watch).

newcommandtoaststatuspromptprocess
v0.1.0MITupdated 2026-10-12Joseph-Ayoub/claude-mods/stall-watch
A shopper browsing a rack in a slop shop
Preview · a replayed session in a sandbox
claude · ~/work/app · stall-watch
› fix the failing auth test and add an audit log call ⏺ Read(src/auth.ts) ⎿ Read 6 lines ⏺ Update(src/auth.ts) ⎿ Added 2 lines, removed 1 line ⏺ Bash(bun test) ⎿ 3 pass, 1 fail ● Done. refresh now rejects expired claims and logs an audit event. ✻ Worked for 42s · done 4:20 PM › /stall-watch ⎿ stall-watch: stall-watch: resume after reset is OFF. ⎿ stall-watch: Automated continues used: 0/3. ⎿ stall-watch: No retry pending. ⎿ stall-watch: No resume armed. ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── › ? for shortcuts
README

stall-watch

A Claude Code mod (function-hooks plugin) that keeps a session from stalling:

  1. Auto-retry. When the main conversation's turn dies on a transient API error (rate_limit, overloaded, server_error, unknown), it waits 30 s, then 2 min, then 8 min and sends a short continuation: "The previous turn ended on a transient API error; continue from where you left off without restarting the work." It is framed as the plugin's message (not as your words), so the model knows where it came from. A toast says stall-watch: auto-retry N/3 in Xs.
  2. Task-notification hint. After a turn started by a background task's notification ends normally, it puts resume in the prompt box as a dim Tab-to-accept suggestion. It never sends it for you.
  3. Usage-limit countdown. When a rate-limit window reaches 85 %, the status line under the prompt shows e.g. weekly 92% · resets 11:00pm (in 2h 13m), refreshed every minute while shown and cleared when usage drops.
  4. Resume after reset (opt-in, off by default). /stall-watch on makes the session continue by itself 30 s after a usage limit resets, if the last turn failed on rate_limit against a window at 95 % or more and nothing has run since (no prompt of yours, no other turn). Only you can turn it on: at the terminal, over Remote Control, or in the desktop app (which hosts sessions over the SDK). A /stall-watch on from another plugin, a peer session, a task or a schedule is refused; off and status answer anyone.
  5. Resume note. At 90 % it writes a short note (branch, working directory, the first 80 characters of your last prompt with credentials redacted, time, the limit) to ~/.claude/stall-watch/<sessionId>.md, once per window. Notes older than 7 days are deleted at the next session start.

Hooks it registers

EventWhat it doesOn failure
session.startregisters /stall-watch, reads the local UTC offset (date +%z), restores this session's resume switch and arm from the store (and refreshes its age), re-arms a pending retry, an armed resume and the countdown ticker from $.state (it fires again on a hot reload or worker respawn), prunes store records older than 14 days and resume notes older than 7 daysreplays next(e)
session.endon a switch to another conversation (clear, resume, logout, or any reason other than the process exiting): cancels anything armed, resets the cap, turns resume off; /clear also forgets the old session's stored switch, the others keep it but drop its arm; /stall-watch is registered again at the next hookreplays next(e)
prompt.submit (gating site, observer)notes which origin started the turn; on your own prompt (composer, bridge or sdk) cancels pending retries and resumes, resets the caps, notes the running turn it was typed over, keeps a redacted 80-character excerpt for the note, and keeps an enabled switch fresh in the store; another origin's prompt re-arms nothing (its own turn.start comes next).catch replays next(e): never blocks a prompt
classic.StopFailure (gating site, observer)records the error kind of a main-loop failure and pairs it with the error turn.catch replays next(e): never blocks
turn.start (main loop only)any turn starting means the session moved past the failure: drops a pending retry, an armed resume, and a failure kind or error turn still waiting for its other half, before re-arming anythingreplays next(e)
turn.complete (main loop only)on error, pairs with the recorded kind and decides; on any other ending, drops a pending retry and an armed resume (should turn.start have missed); on answer after a task notification, offers the resume hintreplays next(e); the answer is never changed
session.measureupdates the countdown and writes the resume notereplays next(e)
command.run { command: 'stall-watch' }answers `/stall-watch on \off \status`answers a fixed error line

claude plugin validate reports prompt.submit and classic.StopFailure as "gating hook with .catch"; both are observers that fail open.

Never retried

billing_error, authentication_failed, invalid_request, max_output_tokens, model_not_found (and every other non-transient kind), a context overflow (detected from the API's own wording in error_details), a subagent's failure, an interrupted turn, and an empty answer (a thinking-only turn is not a failure).

Runaway guards

  • At most 3 automated turns (auto-retries and resumes together) per prompt of yours; the counter resets only on a composer, bridge or sdk prompt (the desktop app hosts its sessions over the SDK), never on the plugin's own, a peer's, a task's, a schedule's or an unclassified one. Each timer checks the cap again when it fires.
  • Each submit is counted where it is made (the plugin's own prompt.submit hook never sees it). A submit another hook drops is not counted and not re-armed, so a refusing guard cannot become a loop.
  • Typing anything cancels a pending retry and an armed resume.
  • Any other main-loop turn (a background task's notification, a peer's prompt, a scheduled one) cancels them too, as soon as it starts: a continuation waits on the failed turn, never on a session that has moved on. A failure kind left unpaired by a turn that ended without an error goes with it, so it can never decide the next turn (a billing_error is never retried under an earlier turn's overloaded).
  • A failed turn you typed over is not retried: your queued prompt runs next.
  • A rate_limit while a window is at 95 % or more with its reset still ahead is not retried on the 30 s / 2 min / 8 min schedule (it cannot clear before the reset); it waits for that reset if resume is on, and otherwise says so in a toast; no retry attempt is spent on it. A window at 100 % is waited on before a nearly-full one (so a 96 % weekly window days from its reset never outranks the full 5-hour one). Any other rate_limit (no window at 95 %, an unknown reset, or a stale reading whose reset has passed) gets the ordinary backoff retry. (A deliberate, lead-approved departure from the spec, which backs off on every rate_limit.)
  • Everything armed belongs to one conversation. If the session id changes without a session.end reaching the plugin, the next event or timer notices and drops what was armed instead of firing it into the new conversation.
  • A resume is armed only for a reset still in the future; one that passed while no session was running is dropped at session.start.

Options

No userConfig fields. Resume is switched per session with the command:

/stall-watch on       # resume 30 s after a usage-limit reset
/stall-watch off
/stall-watch status   # shows the switch, retries used, what is armed

The switch is kept in the plugin's store under session:<sessionId> as { sessionId, enabled, resetsAt?, armedAt?, updatedAt }, so it survives a hot reload and a restart of the same session. A record not seen for 14 days is pruned; a session start and, while resume is on, each prompt of yours refresh it.

Load it

claude --plugin-dir /absolute/path/to/mods/stall-watch

or set CLAUDE_CODE_PLUGIN_DIRS=/absolute/path/to/mods/stall-watch in the environment (or in the env block of ~/.claude/settings.json). Check and test it with:

claude plugin validate mods/stall-watch
claude plugin test mods/stall-watch

Known limits

  • The engine retries API errors itself before a turn reports error; these retries come on top, deliberately slow. Each one re-sends the conversation.
  • The engine setting autoContinueAtUsageLimit already covers resuming at a usage limit; that is why resume here is opt-in. Turn on one of the two, not both.
  • Countdown and resume need subscription rate-limit readings; on an API key they stay silent.
  • Timers do not survive quitting. A hot reload re-arms from $.state at the session.start the reload fires (and at the next event, should that fail); a restart re-arms the resume from the store at session.start.
  • /clear and an in-process /resume move to another session id: anything armed is cancelled and resume is off again for the new session. Whether a resumed conversation's own stored switch is restored depends on the engine firing session.start for it; a restart with claude --resume <id> does.
  • The local time comes from one date +%z at session start; a daylight-saving change mid-session shifts the shown time by an hour until the next session. If date cannot run, the line shows only the relative time.
  • The resume note holds only the first 80 characters of your last prompt, redacted before it is cut, in three layers: vendor token shapes (OpenAI/Anthropic/Stripe-style sk-, whsec_, GitHub, Shopify shpat_, Google AIza, npm, GitLab, Slack tokens and webhook URLs, AWS key ids, JWTs, PEM keys), auth headers and cookies, URL passwords (postgres://user:REDACTED@host, redis://:[redacted]@host), mysql -p; any value after a name holding a secret word (password, pass, secret, token, api key, access key, private key, _KEY, credentials) in env-var (DB_PASSWORD=, SHOPIFY_API_SECRET=, ENCRYPTION_KEY=), camelCase, JSON ("password": "..."), flag (--password x) and prose (password is ..., password hunter2) form; and any bare run of 20+ token characters that mixes letters and digits (or 32+ mixing upper and lower case), which is how an AWS secret access key pasted on its own is caught. What it cannot know: a letters-only secret after a plain word (password hunter, use correcthorse to log in) passes. It over-redacts on purpose: a commit hash, a UUID or a path holding digits reads [redacted] in the note. The file is written with default permissions and deleted after 7 days. Prompt text is never put in a toast, a log line or the store, and only the redacted excerpt is held in memory.
  • A failure is paired with its turn by order, as the events carry no shared id. Should a turn's StopFailure arrive only after the next turn started, and that next turn end in an error before its own StopFailure, the next turn is decided by the earlier kind. Both orders are undocumented; the common orders (either half first, within one turn) are handled.
  • A resume more than a day away is waited out in steps of a day (a timer may not hold a wait of weeks); each step re-arms only while that resume still stands.
  • Note pruning runs /bin/rm by argv (the hooks API has no delete); on a system without /bin/rm, old notes stay until removed by hand.
  • Plugin prompts do not expand @file mentions or images, so the continuation is a fixed sentence rather than a replay of your prompt.
Source 3 files
hooks/register.tsx 726 lines
1// stall-watch: retries a turn that died on a transient API error (capped,
2// backed off), pins a usage-limit countdown, writes a resume note near the
3// limit, and, opt-in per session, resumes once a usage limit resets.
4//
5// Why it is built this way:
6// - turn.complete says only that a turn ended on an 'error'; classic.StopFailure
7//   carries the error kind. The two are paired (either may arrive first) and
8//   only a transient kind is retried, so billing, auth, invalid-request and
9//   context-overflow failures are never re-run.
10// - Every automated submit (retry or resume, one shared cap) is counted at its
11//   call site (the plugin's own prompt.submit hook never sees its own submits)
12//   and the counter resets only on the person's own prompt (composer, bridge
13//   or sdk: isPerson), so a persistent error cannot loop. A submit another
14//   hook drops is not counted.
15// - Everything armed belongs to one conversation (`owner`). A /clear or an
16//   in-process resume moves the process to another session id; whatever was
17//   armed for the old one is forgotten, both at session.end and, should that
18//   be missed, by the next event or timer that sees a different session id.
19// - Everything armed waits on one failed turn. Once the session moves past it
20//   (any main-loop turn starts, or one ends without an error), it is dropped;
21//   the plugin's own turns start with nothing armed, as each fire clears its
22//   plan before submitting. A failure kind or error turn still unpaired goes
23//   with it, so it never decides a later turn. A prompt the person typed over
24//   the failed turn runs next, so that failure arms nothing.
25// - Timers die on hot reload and quit, so what they are for lives in $.state
26//   (and the resume arm in $.store); session.start (which a reload fires
27//   again) and each hook re-arm them.
28
29import { atom, read, update } from 'claude-code'
30import type { EngineInterface, Register, Timer } from 'claude-code'
31
32import type { StallWatchErrorTurn, StallWatchPending, StallWatchResumeArm, StallWatchWindow } from '../types'
33import {
34  MAX_ATTEMPTS,
35  NOTE_THRESHOLD,
36  PAIR_WINDOW_MS,
37  backoffSeconds,
38  cleanWindows,
39  clockText,
40  exhaustedWindow,
41  isContextOverflow,
42  isPerson,
43  isRetryable,
44  isSafeId,
45  parseOffset,
46  parseTime,
47  promptExcerpt,
48  statusText,
49  waitMs,
50  windowText,
51} from './format'
52
53type $ = EngineInterface
54
55const CONTINUE_TEXT =
56  'The previous turn ended on a transient API error; continue from where you left off without restarting the work.'
57const RESUME_TEXT =
58  'The usage limit has reset; continue from where you left off without restarting the work.'
59
60/** Store records older than this belong to sessions long gone. */
61const STORE_TTL_MS = 14 * 24 * 3_600_000
62/**
63 * Resume notes hold prompt text, so they are kept for less time than the
64 * store's switches: a week covers a weekly limit's reset.
65 */
66const NOTE_TTL_MS = 7 * 24 * 3_600_000
67/** A reset is followed by this much slack before resuming. */
68const RESUME_SLACK_MS = 30_000
69/** The suggestion waits for the turn to have fully ended (it is refused mid-turn). */
70const SUGGEST_DELAY_MS = 1_500
71
72const FAILURE = { plugin: 'stall-watch', key: 'failure' } as const
73const ERROR_TURN = { plugin: 'stall-watch', key: 'errorTurn' } as const
74const ATTEMPTS = { plugin: 'stall-watch', key: 'attempts' } as const
75const PENDING = { plugin: 'stall-watch', key: 'pending' } as const
76const LAST_COMPOSER_AT = { plugin: 'stall-watch', key: 'lastComposerAt' } as const
77const LAST_ORIGIN = { plugin: 'stall-watch', key: 'lastOrigin' } as const
78const LAST_PROMPT = { plugin: 'stall-watch', key: 'lastPrompt' } as const
79const WINDOWS = { plugin: 'stall-watch', key: 'windows' } as const
80const IS_STATUS_SHOWN = { plugin: 'stall-watch', key: 'isStatusShown' } as const
81const TZ_OFFSET = { plugin: 'stall-watch', key: 'tzOffsetMin' } as const
82const RESUME_ENABLED = { plugin: 'stall-watch', key: 'resumeEnabled' } as const
83const RESUME_ARM = { plugin: 'stall-watch', key: 'resumeArm' } as const
84const NOTE_KEY = { plugin: 'stall-watch', key: 'noteKey' } as const
85const OWNER = { plugin: 'stall-watch', key: 'owner' } as const
86const COMPOSER_TURN = { plugin: 'stall-watch', key: 'composerTurnId' } as const
87
88const failure = atom(FAILURE, null)
89const errorTurn = atom(ERROR_TURN, null)
90const attempts = atom(ATTEMPTS, 0)
91const pending = atom(PENDING, null)
92const lastComposerAt = atom(LAST_COMPOSER_AT, 0)
93const lastOrigin = atom(LAST_ORIGIN, null)
94const lastPrompt = atom(LAST_PROMPT, null)
95const windows = atom(WINDOWS, [])
96const isStatusShown = atom(IS_STATUS_SHOWN, false)
97const tzOffset = atom(TZ_OFFSET, null)
98const resumeEnabled = atom(RESUME_ENABLED, false)
99const resumeArm = atom(RESUME_ARM, null)
100const noteKey = atom(NOTE_KEY, null)
101const owner = atom(OWNER, null)
102const composerTurn = atom(COMPOSER_TURN, null)
103
104/** What $.store keeps per session: never prompt text, only the resume switch. */
105type StoreRecord = {
106  sessionId: string
107  enabled: boolean
108  resetsAt?: string
109  armedAt?: number
110  updatedAt: number
111}
112
113const storeKey = (sessionId: string) => `session:${sessionId}`
114
115function asRecord(value: unknown): StoreRecord | undefined {
116  if (typeof value !== 'object' || value === null) return undefined
117  const r = value as Record<string, unknown>
118  if (typeof r.sessionId !== 'string' || typeof r.enabled !== 'boolean') return undefined
119  return {
120    sessionId: r.sessionId,
121    enabled: r.enabled,
122    ...(typeof r.resetsAt === 'string' ? { resetsAt: r.resetsAt } : {}),
123    ...(typeof r.armedAt === 'number' ? { armedAt: r.armedAt } : {}),
124    updatedAt: typeof r.updatedAt === 'number' ? r.updatedAt : 0,
125  }
126}
127
128// Timer handles, and only those, are module-scoped: a hot reload cancels the
129// timers together with this environment, so the handles die with them. What a
130// timer was for lives in $.state, from which ensureTimers re-arms it.
131const timers: { retry?: Timer; resume?: Timer; tick?: Timer; suggest?: Timer } = {}
132/** Whether this environment has registered /stall-watch (a reload starts a new one). */
133const registered = { isCommand: false }
134
135function cancel(name: keyof typeof timers) {
136  timers[name]?.cancel()
137  timers[name] = undefined
138}
139
140async function persist(
141  $: $,
142  patch: Partial<Omit<StoreRecord, 'sessionId' | 'updatedAt'>>,
143  drop: ('resetsAt' | 'armedAt')[] = [],
144  forSession?: string,
145) {
146  const sessionId = forSession ?? (await $.session.id())
147  const now = await $.clock.now()
148  const prev = asRecord(await $.store.get(storeKey(sessionId)))
149  const next: StoreRecord = { ...(prev ?? { enabled: false }), ...patch, sessionId, updatedAt: now }
150  for (const field of drop) delete next[field]
151  await $.store.set(storeKey(sessionId), next)
152}
153
154// ---- conversation ownership ---------------------------------------------
155
156/**
157 * Drops everything armed or counted for the conversation that just ended, so
158 * nothing meant for it fires into the one the process moved on to. Usage
159 * windows and the countdown are the account's, not the conversation's, and
160 * stay.
161 */
162async function forgetConversation($: $) {
163  cancel('retry')
164  cancel('resume')
165  cancel('suggest')
166  await $.state.set(PENDING, null)
167  await $.state.set(RESUME_ARM, null)
168  await $.state.set(RESUME_ENABLED, false)
169  await $.state.set(FAILURE, null)
170  await $.state.set(ERROR_TURN, null)
171  await $.state.set(ATTEMPTS, 0)
172  await $.state.set(LAST_ORIGIN, null)
173  await $.state.set(LAST_PROMPT, null)
174  await $.state.set(NOTE_KEY, null)
175  await $.state.set(COMPOSER_TURN, null)
176}
177
178/**
179 * Whether the conversation the state belongs to is still the current one. A
180 * different session id means a switch whose session.end was missed: the old
181 * conversation's state is forgotten and the current one adopted.
182 */
183async function ensureOwner($: $): Promise<boolean> {
184  const current = await $.session.id()
185  const held = await read($, owner)
186  if (held === current) return true
187  if (held !== null) await forgetConversation($)
188  await $.state.set(OWNER, current)
189  return held === null
190}
191
192// ---- auto-retry ---------------------------------------------------------
193
194function armRetry($: $, plan: StallWatchPending, now: number) {
195  cancel('retry')
196  timers.retry = $.clock.after(Math.max(0, plan.dueAt - now), () => {
197    // Fails closed: nothing is sent, and a plan the failure left in $.state is
198    // re-armed by the next session.start or hook, against the same cap.
199    void fireRetry($, plan.armedAt).catch(() => undefined)
200  })
201}
202
203async function fireRetry($: $, armedAt: number) {
204  timers.retry = undefined
205  // Armed for a conversation the process has since left: never fire into another.
206  if (!(await ensureOwner($))) return
207  let isMine = false
208  await update($, pending, p => {
209    isMine = p !== null && p.armedAt === armedAt
210    return isMine ? null : p
211  })
212  if (!isMine) return
213  // The person typed since this was armed: their prompt is the next turn.
214  if ((await read($, lastComposerAt)) > armedAt) return
215  const used = await read($, attempts)
216  if (used >= MAX_ATTEMPTS) return
217  await $.state.set(ATTEMPTS, used + 1)
218  await $.state.set(LAST_ORIGIN, 'plugin')
219  const sent = await $.prompt.submit({ text: CONTINUE_TEXT }).catch(() => undefined)
220  if (sent === undefined || sent.drop !== undefined) {
221    // Refused below us (another plugin or a settings hook): not an attempt,
222    // and not re-armed, so a refusing guard cannot become a loop.
223    await $.state.set(ATTEMPTS, used)
224    $.ui.toast('stall-watch: auto-retry was not sent (refused)')
225  }
226}
227
228/**
229 * Acts on one paired failure of the main-loop turn `turnId`: retry, wait for a
230 * reset, or leave it alone.
231 */
232async function decide($: $, kind: string, contextOverflow: boolean, now: number, turnId: string | undefined) {
233  if (!isRetryable(kind, contextOverflow)) return
234  // The person typed over the turn that failed: their prompt is queued to run
235  // next (or went into that turn), and an automated continue would race it.
236  if (turnId !== undefined && (await read($, composerTurn)) === turnId) return
237  if (kind === 'rate_limit') {
238    // An exhausted usage window (95 %+) cannot clear before its reset: backing
239    // off for seconds would only spend the attempts, so none is spent. With no
240    // exhausted window whose reset is ahead (none, unknown reset, stale
241    // reading) it is retried as usual.
242    const exhausted = exhaustedWindow(await read($, windows), now)
243    if (exhausted !== undefined) {
244      await armResumeFor($, exhausted, now)
245      return
246    }
247  }
248  const used = await read($, attempts)
249  if (used >= MAX_ATTEMPTS) {
250    $.ui.toast(`stall-watch: auto-retry stopped after ${MAX_ATTEMPTS}/${MAX_ATTEMPTS}`)
251    return
252  }
253  const attempt = used + 1
254  const seconds = backoffSeconds(attempt)
255  const plan: StallWatchPending = { dueAt: now + seconds * 1000, attempt, armedAt: now }
256  let isArmed = false
257  await update($, pending, p => {
258    isArmed = p === null
259    return p ?? plan
260  })
261  if (!isArmed) return
262  armRetry($, plan, now)
263  $.ui.toast(`stall-watch: auto-retry ${attempt}/${MAX_ATTEMPTS} in ${seconds}s`)
264}
265
266// ---- resume after a usage-limit reset (opt-in) --------------------------
267
268function armResume($: $, arm: StallWatchResumeArm, now: number) {
269  cancel('resume')
270  timers.resume = $.clock.after(waitMs(arm.dueAt, now), () => {
271    timers.resume = undefined
272    void (async () => {
273      const at = await $.clock.now()
274      if (at < arm.dueAt) {
275        // One step of a long wait: wait on, if this arm still stands.
276        if ((await read($, resumeArm))?.armedAt === arm.armedAt) armResume($, arm, at)
277        return
278      }
279      await fireResume($, arm.armedAt)
280    })().catch(() => undefined)
281  })
282}
283
284/** `window` is exhaustedWindow's answer: full, with a reset still ahead. */
285async function armResumeFor($: $, window: StallWatchWindow, now: number) {
286  if (!(await read($, resumeEnabled))) {
287    $.ui.toast('stall-watch: usage limit reached; /stall-watch on resumes after the reset')
288    return
289  }
290  const resetMs = parseTime(window.resetsAt)
291  if (resetMs === null || window.resetsAt === undefined || resetMs <= now) return
292  if ((await read($, attempts)) >= MAX_ATTEMPTS) {
293    $.ui.toast(`stall-watch: resume stopped after ${MAX_ATTEMPTS}/${MAX_ATTEMPTS}`)
294    return
295  }
296  const arm: StallWatchResumeArm = { resetsAt: window.resetsAt, dueAt: resetMs + RESUME_SLACK_MS, armedAt: now }
297  await $.state.set(RESUME_ARM, arm)
298  armResume($, arm, now)
299  await persist($, { enabled: true, resetsAt: arm.resetsAt, armedAt: arm.armedAt })
300  const offset = await read($, tzOffset)
301  const when = offset === null ? 'after the reset' : `at ${clockText(arm.dueAt, now, offset)}`
302  $.ui.toast(`stall-watch: will resume ${when}`)
303}
304
305async function fireResume($: $, armedAt: number) {
306  if (!(await ensureOwner($))) return
307  let isMine = false
308  await update($, resumeArm, a => {
309    isMine = a !== null && a.armedAt === armedAt
310    return isMine ? null : a
311  })
312  if (!isMine) return
313  await persist($, {}, ['resetsAt', 'armedAt'])
314  if (!(await read($, resumeEnabled))) return
315  if ((await read($, lastComposerAt)) > armedAt) return
316  // One cap for retries and resumes: at most MAX_ATTEMPTS automated turns per prompt of the person's.
317  const used = await read($, attempts)
318  if (used >= MAX_ATTEMPTS) return
319  await $.state.set(ATTEMPTS, used + 1)
320  await $.state.set(LAST_ORIGIN, 'plugin')
321  const sent = await $.prompt.submit({ text: RESUME_TEXT }).catch(() => undefined)
322  if (sent === undefined || sent.drop !== undefined) {
323    await $.state.set(ATTEMPTS, used)
324    $.ui.toast('stall-watch: resume was not sent (refused)')
325  }
326}
327
328async function disarmResume($: $) {
329  cancel('resume')
330  await $.state.set(RESUME_ARM, null)
331  await persist($, {}, ['resetsAt', 'armedAt'])
332}
333
334/**
335 * The session moved past the failed turn: a main-loop turn started (a task
336 * notification, a peer's prompt, a scheduled one, the engine's own
337 * continuation) or a turn ended without an error. A retry or resume still
338 * waiting for that failure would now fire into a session that went on without
339 * it, so both are dropped. The plugin's own turns reach here with nothing
340 * armed (fireRetry and fireResume clear theirs before submitting), so its own
341 * submits need no telling apart. Should this turn fail too, its own failure
342 * arms afresh, against the same cap.
343 */
344async function supersede($: $) {
345  cancel('retry')
346  if ((await read($, pending)) !== null) await $.state.set(PENDING, null)
347  if ((await read($, resumeArm)) !== null) await disarmResume($)
348  // A failure kind or an error turn still waiting for its other half belongs
349  // to the turn the session moved past. Paired with the next turn's error, it
350  // would decide that turn by the wrong kind (a billing_error retried as
351  // 'overloaded'), or arm a retry while another turn runs.
352  if ((await read($, failure)) !== null) await $.state.set(FAILURE, null)
353  if ((await read($, errorTurn)) !== null) await $.state.set(ERROR_TURN, null)
354}
355
356// ---- status line --------------------------------------------------------
357
358/** Draws (or clears) the countdown from `list`; the minute ticker runs only while shown. */
359async function drawStatus($: $, list: readonly StallWatchWindow[], wasShown: boolean) {
360  const now = await $.clock.now()
361  const text = statusText(list, now, await read($, tzOffset))
362  if (text !== undefined) {
363    $.ui.status(text)
364    if (!wasShown) await $.state.set(IS_STATUS_SHOWN, true)
365    if (timers.tick === undefined) startTick($)
366    return
367  }
368  cancel('tick')
369  if (wasShown) {
370    $.ui.status(undefined)
371    await $.state.set(IS_STATUS_SHOWN, false)
372  }
373}
374
375function startTick($: $) {
376  timers.tick = $.clock.every(60_000, () => {
377    void (async () => {
378      await drawStatus($, await read($, windows), await read($, isStatusShown))
379    })().catch(() => cancel('tick'))
380  })
381}
382
383// ---- resume note --------------------------------------------------------
384
385async function maybeWriteNote($: $, list: readonly StallWatchWindow[], now: number) {
386  const top = list
387    .filter(w => w.percentUsed >= NOTE_THRESHOLD)
388    .sort((a, b) => b.percentUsed - a.percentUsed)[0]
389  if (top === undefined) return
390  const key = `${top.kind}|${top.resetsAt ?? ''}`
391  if ((await read($, noteKey)) === key) return
392  // Claimed before the write, so a burst of measurements writes once.
393  await $.state.set(NOTE_KEY, key)
394  const home = await $.env.get('HOME')
395  const sessionId = await $.session.id()
396  if (home === undefined || home === '' || !isSafeId(sessionId)) return
397  const cwd = await $.session.cwd()
398  const branch = await $.process
399    .run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'], { cwd, timeoutMs: 5_000 })
400    .then(r => (r.exitCode === 0 ? r.stdout.trim() || 'unknown' : 'unknown'))
401    .catch(() => 'unknown')
402  const offset = await read($, tzOffset)
403  const prompt = await read($, lastPrompt)
404  const written =
405    offset === null
406      ? `${new Date(now).toISOString().slice(0, 16).replace('T', ' ')} UTC`
407      : `${new Date(now + offset * 60_000).toISOString().slice(0, 10)} ${clockText(now, now, offset)} (local)`
408  const lines = [
409    '# stall-watch resume note',
410    '',
411    `- session: ${sessionId}`,
412    `- written: ${written}`,
413    `- branch: ${branch}`,
414    `- cwd: ${cwd}`,
415    `- limit: ${windowText(top, now, offset)}`,
416    '',
417    '## Last prompt',
418    '',
419    // Held as an excerpt already; cut again in case $.state predates that (a reload keeps it).
420    prompt === null ? '(none captured this session)' : promptExcerpt(prompt),
421    '',
422    `Resume with: claude --resume ${sessionId}`,
423    '',
424  ]
425  await $.fs.write(`${home}/.claude/stall-watch/${sessionId}.md`, lines.join('\n')).catch(() => {
426    $.ui.toast('stall-watch: could not write the resume note')
427  })
428}
429
430/**
431 * Data minimisation: deletes this mod's resume notes older than NOTE_TTL_MS.
432 * Only regular `<session id>.md` files in its own folder are touched; there is
433 * no $.fs delete, so it runs /bin/rm by argv (no shell).
434 */
435async function pruneNotes($: $, now: number) {
436  const home = await $.env.get('HOME')
437  if (home === undefined || home === '') return
438  const dir = `${home}/.claude/stall-watch`
439  const entries = await $.fs.list(dir).catch(() => [])
440  const stale = entries.filter(
441    entry =>
442      entry.kind === 'file' &&
443      !entry.isLink &&
444      entry.name.endsWith('.md') &&
445      isSafeId(entry.name.slice(0, -3)) &&
446      entry.mtimeMs > 0 &&
447      now - entry.mtimeMs > NOTE_TTL_MS,
448  )
449  // A handful per start keeps session.start inside its hook budget; the rest go next time.
450  for (const entry of stale.slice(0, 20)) {
451    await $.process.run(['/bin/rm', '-f', '--', `${dir}/${entry.name}`], { timeoutMs: 3_000 }).catch(() => undefined)
452  }
453}
454
455// ---- lazy re-arm after a hot reload -------------------------------------
456
457/**
458 * Registers /stall-watch once per environment: at session.start (which a
459 * reload or worker respawn fires again), and lazily at any other hook, should
460 * that session.start have failed.
461 */
462async function ensureCommand($: $) {
463  if (registered.isCommand) return
464  registered.isCommand = true
465  await $.command
466    .register({
467      name: 'stall-watch',
468      description: 'Resume this session after a usage-limit reset: on, off or status.',
469      argumentHint: 'on|off|status',
470      immediate: true,
471    })
472    .catch(() => {
473      registered.isCommand = false
474    })
475}
476
477async function ensureTimers($: $) {
478  await ensureOwner($)
479  await ensureCommand($)
480  const now = await $.clock.now()
481  const plan = await read($, pending)
482  if (plan !== null && timers.retry === undefined) armRetry($, plan, now)
483  const arm = await read($, resumeArm)
484  if (arm !== null && timers.resume === undefined && (await read($, resumeEnabled))) armResume($, arm, now)
485  if (timers.tick === undefined && (await read($, isStatusShown))) startTick($)
486}
487
488export const register: Register = on => {
489  // ---- hooks --------------------------------------------------------------
490
491  on('session.start', async ($, e, next) => {
492    await ensureOwner($)
493    await ensureCommand($)
494
495    const zone = await $.process.run(['date', '+%z'], { timeoutMs: 3_000 }).catch(() => undefined)
496    await $.state.set(TZ_OFFSET, zone !== undefined && zone.exitCode === 0 ? parseOffset(zone.stdout) : null)
497
498    const now = await $.clock.now()
499    const sessionId = await $.session.id()
500    const record = asRecord(await $.store.get(storeKey(sessionId)))
501    // Seen again: refreshed, so another session's pruning below never takes
502    // the switch of a session that is still in use.
503    if (record !== undefined) await persist($, {}).catch(() => undefined)
504    if (record?.enabled === true) {
505      await $.state.set(RESUME_ENABLED, true)
506      const resetMs = parseTime(record.resetsAt)
507      if (record.resetsAt !== undefined && resetMs !== null && resetMs + RESUME_SLACK_MS > now) {
508        const arm: StallWatchResumeArm = {
509          resetsAt: record.resetsAt,
510          dueAt: resetMs + RESUME_SLACK_MS,
511          armedAt: record.armedAt ?? now,
512        }
513        await $.state.set(RESUME_ARM, arm)
514        armResume($, arm, now)
515      } else if (record.resetsAt !== undefined) {
516        // The reset passed while no session was running: the person is here
517        // now, so it is theirs to continue, not ours.
518        await persist($, {}, ['resetsAt', 'armedAt'])
519      }
520    }
521
522    // A reload or worker respawn fires session.start again and its timers are
523    // gone: re-arm a pending retry, an armed resume and the minute ticker from
524    // $.state now rather than at the next event. Before the pruning below,
525    // which is the slow part.
526    await ensureTimers($)
527
528    // Data minimisation: forget switches of sessions long gone.
529    const keys = await $.store.keys().catch(() => [] as string[])
530    for (const key of keys.filter(k => k.startsWith('session:')).slice(0, 200)) {
531      const r = asRecord(await $.store.get(key))
532      if (r === undefined || now - r.updatedAt > STORE_TTL_MS) await $.store.delete(key)
533    }
534    await pruneNotes($, now)
535
536    return next(e)
537  }).catch(($, e, next) => next(e))
538
539  on('session.end', async ($, e, next) => {
540    // The process ends with the session ('prompt_input_exit', 'other'): its
541    // timers die with it, and the stored arm is kept for a restart of the same
542    // session. Any other reason ('clear', 'resume', 'logout', or one this
543    // build does not know) moves the process on to another conversation, into
544    // which nothing armed for this one may fire.
545    if (e.reason !== 'prompt_input_exit' && e.reason !== 'other') {
546      // A command is registered for the session it was registered in: the
547      // next hook registers it again for the one the process moved on to
548      // (registering a name again replaces it).
549      registered.isCommand = false
550      await forgetConversation($)
551      await $.state.set(OWNER, null)
552      if (e.reason === 'clear') {
553        await $.store.delete(storeKey(e.sessionId)).catch(() => undefined)
554      } else if (asRecord(await $.store.get(storeKey(e.sessionId)).catch(() => undefined)) !== undefined) {
555        // Left, not wiped: keep the switch for a later --resume, drop the arm.
556        await persist($, {}, ['resetsAt', 'armedAt'], e.sessionId).catch(() => undefined)
557      }
558    }
559    return next(e)
560  }).catch(($, e, next) => next(e))
561
562  // Observer: never refuses a prompt, so a failure replays next(e) (fail open).
563  on('prompt.submit', async ($, e, next) => {
564    await ensureOwner($)
565    const kind = e.origin?.kind ?? 'unclassified'
566    const isOwn = isPerson(kind)
567    if (isOwn) {
568      // The person is back: their prompt supersedes anything armed, and the
569      // runaway cap starts over. Done before ensureTimers, which after a hot
570      // reload would otherwise re-arm an overdue retry to fire beside it.
571      cancel('retry')
572      await $.state.set(LAST_COMPOSER_AT, await $.clock.now())
573      await $.state.set(PENDING, null)
574      await $.state.set(ATTEMPTS, 0)
575      // Typed over a running turn: should that turn fail, this prompt runs
576      // next, so decide() leaves that failure alone.
577      if (e.turnId !== undefined) await $.state.set(COMPOSER_TURN, e.turnId)
578      // Only a redacted excerpt is ever held, never the raw prompt.
579      await $.state.set(LAST_PROMPT, promptExcerpt(e.text))
580      if ((await read($, resumeArm)) !== null) await disarmResume($)
581    }
582    if (isOwn || e.turnId !== undefined) {
583      await ensureTimers($)
584    } else {
585      // Another origin's prompt starts a turn of its own, whose turn.start
586      // drops whatever is armed. Re-arming an overdue plan here would let it
587      // fire before that turn starts, so the re-arm waits for that turn.start.
588      await ensureCommand($)
589    }
590    // A prompt typed or delivered into a running turn starts no turn of its own.
591    if (e.turnId === undefined) await $.state.set(LAST_ORIGIN, kind)
592    // While resume is on, the person's prompt keeps the stored switch fresh
593    // against another session's pruning (and restores it, should that have run).
594    if (isOwn && (await read($, resumeEnabled))) await persist($, { enabled: true }).catch(() => undefined)
595    return next(e)
596  }).catch(($, e, next) => next(e))
597
598  on('classic.StopFailure', async ($, e, next) => {
599    if (e.agent_id === undefined) {
600      await ensureTimers($)
601      const now = await $.clock.now()
602      const kind = String(e.error)
603      const overflow = isContextOverflow(e.error_details)
604      // turn.complete may have arrived first: pair with it here.
605      const found: { turn: StallWatchErrorTurn | null } = { turn: null }
606      await update($, errorTurn, t => {
607        found.turn = t !== null && now - t.at <= PAIR_WINDOW_MS ? t : null
608        return found.turn === null ? t : null
609      })
610      await $.state.set(FAILURE, { at: now, kind, isContextOverflow: overflow, isHandled: found.turn !== null })
611      if (found.turn !== null) await decide($, kind, overflow, now, found.turn.turnId)
612    }
613    return next(e)
614  }).catch(($, e, next) => next(e))
615
616  // Observer, main loop only (a subagent's run raises no turn.start).
617  on('turn.start', async ($, e, next) => {
618    // Ownership first, so a missed session switch forgets the old state rather
619    // than disarming into the new session's store record. Then dropped before
620    // re-arming: re-arming first would let an overdue plan fire beside this turn.
621    await ensureOwner($)
622    await supersede($)
623    await ensureTimers($)
624    return next(e)
625  }).catch(($, e, next) => next(e))
626
627  on('turn.complete', async ($, e, next) => {
628    if (e.agentId !== undefined) return next(e)
629    await ensureTimers($)
630    const now = await $.clock.now()
631    if (e.reason === 'error') {
632      let paired: { kind: string; isContextOverflow: boolean } | null = null
633      await update($, failure, f => {
634        paired = f !== null && !f.isHandled && now - f.at <= PAIR_WINDOW_MS ? f : null
635        return paired === null ? f : { ...f!, isHandled: true }
636      })
637      if (paired === null) {
638        // StopFailure may still be on its way: leave the turn for it to pair.
639        await $.state.set(ERROR_TURN, { at: now, turnId: e.turnId })
640      } else {
641        const { kind, isContextOverflow: overflow } = paired as { kind: string; isContextOverflow: boolean }
642        await decide($, kind, overflow, now, e.turnId)
643      }
644    } else {
645      // Answered or interrupted: the session is past the failed turn. Normally
646      // turn.start already dropped what was armed; this covers a missed one.
647      await supersede($)
648    }
649    if (e.reason === 'answer') {
650      if ((await read($, lastOrigin)) === 'task-notification') {
651        await $.state.set(LAST_ORIGIN, null)
652        // A hint only (Tab to accept); never submitted on the person's behalf.
653        cancel('suggest')
654        timers.suggest = $.clock.after(SUGGEST_DELAY_MS, () => {
655          timers.suggest = undefined
656          void $.prompt.suggest({ text: 'resume' }).catch(() => undefined)
657        })
658      }
659    }
660    return next(e)
661  }).catch(($, e, next) => next(e))
662
663  on('session.measure', async ($, e, next) => {
664    await ensureTimers($)
665    const list = cleanWindows(e.rateLimits)
666    // An empty reading (off a subscription, before the first response) is no news.
667    if (list.length > 0) {
668      await $.state.set(WINDOWS, list)
669      await drawStatus($, list, await read($, isStatusShown))
670      await maybeWriteNote($, list, await $.clock.now())
671    }
672    return next(e)
673  }).catch(($, e, next) => next(e))
674
675  on('command.run', { command: 'stall-watch' }, async ($, e) => {
676    await ensureTimers($)
677    const arg = e.args.trim().toLowerCase()
678    const now = await $.clock.now()
679    if (arg === 'on' && !isPerson(e.origin?.kind)) {
680      // Turning on is consent to automated submits: the person's alone (composer,
681      // bridge or sdk), never another plugin's or a peer's. Off and status are safe from anyone.
682      return { text: 'stall-watch: only you can turn resume on (type /stall-watch on).' }
683    }
684    if (arg === 'on') {
685      await $.state.set(RESUME_ENABLED, true)
686      await persist($, { enabled: true })
687      // Turned on after the limit already hit: arm for the pending reset now.
688      const last = await read($, failure)
689      const exhausted = exhaustedWindow(await read($, windows), now)
690      if (last?.kind === 'rate_limit' && exhausted !== undefined && exhausted.resetsAt !== undefined) {
691        const resetMs = parseTime(exhausted.resetsAt)
692        if (resetMs !== null && resetMs > now && (await read($, attempts)) < MAX_ATTEMPTS) {
693          const arm: StallWatchResumeArm = { resetsAt: exhausted.resetsAt, dueAt: resetMs + RESUME_SLACK_MS, armedAt: now }
694          await $.state.set(RESUME_ARM, arm)
695          armResume($, arm, now)
696          await persist($, { enabled: true, resetsAt: arm.resetsAt, armedAt: arm.armedAt })
697          return { text: 'stall-watch: resume after reset is ON; armed for the current limit.' }
698        }
699      }
700      return { text: 'stall-watch: resume after reset is ON for this session.' }
701    }
702    if (arg === 'off') {
703      await $.state.set(RESUME_ENABLED, false)
704      cancel('resume')
705      await $.state.set(RESUME_ARM, null)
706      await persist($, { enabled: false }, ['resetsAt', 'armedAt'])
707      return { text: 'stall-watch: resume after reset is OFF for this session.' }
708    }
709    if (arg === '' || arg === 'status') {
710      const enabled = await read($, resumeEnabled)
711      const arm = await read($, resumeArm)
712      const plan = await read($, pending)
713      const offset = await read($, tzOffset)
714      const at = (ms: number) => (offset === null ? new Date(ms).toISOString() : clockText(ms, now, offset))
715      const lines = [
716        `stall-watch: resume after reset is ${enabled ? 'ON' : 'OFF'}.`,
717        `Automated continues used: ${await read($, attempts)}/${MAX_ATTEMPTS}.`,
718        plan === null ? 'No retry pending.' : `Retry ${plan.attempt}/${MAX_ATTEMPTS} at ${at(plan.dueAt)}.`,
719        arm === null ? 'No resume armed.' : `Resume armed for ${at(arm.dueAt)}.`,
720      ]
721      return { text: lines.join('\n') }
722    }
723    return { text: 'Usage: /stall-watch on | off | status' }
724  }).catch(() => ({ text: 'stall-watch: the command failed; run /stall-watch status to see what stands.' }))
725}
726
hooks/format.ts 335 lines
1// Pure helpers for stall-watch: classification and text formatting. Kept free
2// of `$` so the decisions that gate an auto-submit can be tested directly.
3
4import type { StallWatchWindow } from '../types'
5
6/** Error kinds that may clear on their own (ModelApiError's own doc says so). */
7export const TRANSIENT_KINDS: readonly string[] = ['rate_limit', 'overloaded', 'server_error', 'unknown']
8
9/** Seconds to wait before auto-retry 1, 2 and 3. */
10export const BACKOFF_SECONDS: readonly number[] = [30, 120, 480]
11
12export const MAX_ATTEMPTS = 3
13
14/**
15 * The prompt origins that are the person's own words: their Enter at the
16 * terminal (`composer`), the Remote Control bridge, and the SDK, through which
17 * the desktop app hosts its sessions. Only these reset the runaway cap,
18 * supersede what is armed, mark a running turn as typed over, are excerpted
19 * for the note, or turn resume on. Every other kind (a peer, a task
20 * notification, a schedule, another plugin, unclassified) is not the person.
21 */
22export const PERSON_ORIGINS: readonly string[] = ['composer', 'bridge', 'sdk']
23
24export function isPerson(kind: string | undefined): boolean {
25  return kind !== undefined && PERSON_ORIGINS.includes(kind)
26}
27
28/**
29 * The longest single wait handed to $.clock.after. A spend-limit reset can be
30 * weeks away, past what a 32-bit timer holds (about 24.8 days), and a wait the
31 * host cannot hold may fire at once; a longer one is waited out in steps.
32 */
33export const MAX_WAIT_MS = 24 * 3_600_000
34
35/** The next timer step toward `dueAt`: never negative, never past MAX_WAIT_MS. */
36export function waitMs(dueAt: number, now: number): number {
37  return Math.min(Math.max(0, dueAt - now), MAX_WAIT_MS)
38}
39
40/** A StopFailure older than this is not paired with a turn.complete. */
41export const PAIR_WINDOW_MS = 30_000
42
43/** Status line shows windows at or above this percentage. */
44export const STATUS_THRESHOLD = 85
45
46/** The resume note is written once a window reaches this percentage. */
47export const NOTE_THRESHOLD = 90
48
49/**
50 * A window at least this full counts as exhausted for a rate_limit failure: a
51 * retry before its reset cannot succeed, so rate_limit there means "wait for
52 * the reset", not "back off". Readings lag and round to one decimal, so a
53 * window that refuses requests often reads 95-99.9 %, not 100.
54 */
55export const EXHAUSTED_THRESHOLD = 95
56
57/** A window this full is certainly refusing requests. */
58export const FULL_THRESHOLD = 100
59
60/**
61 * The engine reports a context overflow as a plain API error; its details
62 * carry the API's own wording. Matched only to refuse a retry, never shown.
63 */
64const CONTEXT_OVERFLOW = /prompt is too long|context (window|length|limit)|too many tokens|maximum context/i
65
66export function isContextOverflow(details: string | undefined): boolean {
67  return typeof details === 'string' && CONTEXT_OVERFLOW.test(details)
68}
69
70/** Whether a recorded failure is one an auto-retry may answer. */
71export function isRetryable(kind: string, contextOverflow: boolean): boolean {
72  return !contextOverflow && TRANSIENT_KINDS.includes(kind)
73}
74
75export function backoffSeconds(attempt: number): number {
76  const index = Math.min(Math.max(attempt, 1), BACKOFF_SECONDS.length) - 1
77  return BACKOFF_SECONDS[index] ?? 480
78}
79
80/** Parses `date +%z` output (`-0700`, `+0530`) into minutes east of UTC. */
81export function parseOffset(text: string): number | null {
82  const match = /^([+-])(\d{2})(\d{2})$/.exec(text.trim())
83  if (!match) return null
84  const minutes = Number(match[2]) * 60 + Number(match[3])
85  if (minutes > 14 * 60) return null
86  return match[1] === '-' ? -minutes : minutes
87}
88
89/** Epoch ms of an ISO timestamp, or null for anything that does not parse. */
90export function parseTime(iso: string | undefined): number | null {
91  if (typeof iso !== 'string' || iso === '') return null
92  const ms = Date.parse(iso)
93  return Number.isFinite(ms) ? ms : null
94}
95
96export function kindLabel(kind: string): string {
97  if (kind === 'five_hour') return '5h'
98  if (kind === 'seven_day') return 'weekly'
99  if (kind === 'spend_limit') return 'spend'
100  // Unknown windows are shown by their own name rather than hidden.
101  return kind.replace(/[^A-Za-z0-9_-]/g, '').slice(0, 24) || 'limit'
102}
103
104const DAYS = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat']
105
106/**
107 * Wall-clock time in the person's zone, from a fixed UTC offset (the hooks
108 * environment has no reliable local zone of its own): `11:00pm`, or
109 * `Tue 11:00pm` when the moment is more than 20 hours from now.
110 */
111export function clockText(ms: number, nowMs: number, offsetMin: number): string {
112  const local = new Date(ms + offsetMin * 60_000)
113  const hours24 = local.getUTCHours()
114  const minutes = String(local.getUTCMinutes()).padStart(2, '0')
115  const hours12 = hours24 % 12 === 0 ? 12 : hours24 % 12
116  const time = `${hours12}:${minutes}${hours24 < 12 ? 'am' : 'pm'}`
117  return Math.abs(ms - nowMs) > 20 * 3_600_000 ? `${DAYS[local.getUTCDay()]} ${time}` : time
118}
119
120/** `in 2h 13m`, `in 45m`, `in <1m`. */
121export function relativeText(ms: number, nowMs: number): string {
122  const totalMin = Math.floor((ms - nowMs) / 60_000)
123  if (totalMin < 1) return 'in <1m'
124  const days = Math.floor(totalMin / 1440)
125  const hours = Math.floor((totalMin % 1440) / 60)
126  const minutes = totalMin % 60
127  if (days > 0) return `in ${days}d ${hours}h`
128  return hours > 0 ? `in ${hours}h ${minutes}m` : `in ${minutes}m`
129}
130
131/** Keeps only well-formed readings, so nothing downstream ever sees NaN. */
132export function cleanWindows(raw: readonly unknown[] | undefined): StallWatchWindow[] {
133  if (!Array.isArray(raw)) return []
134  const out: StallWatchWindow[] = []
135  for (const one of raw) {
136    if (typeof one !== 'object' || one === null) continue
137    const { kind, percentUsed, resetsAt } = one as Record<string, unknown>
138    if (typeof kind !== 'string' || typeof percentUsed !== 'number' || !Number.isFinite(percentUsed)) continue
139    out.push(
140      typeof resetsAt === 'string' && parseTime(resetsAt) !== null
141        ? { kind, percentUsed, resetsAt }
142        : { kind, percentUsed },
143    )
144  }
145  return out
146}
147
148/** The one-window text: `weekly 92% · resets 11:00pm (in 2h 13m)`. */
149export function windowText(w: StallWatchWindow, nowMs: number, offsetMin: number | null): string {
150  const head = `${kindLabel(w.kind)} ${Math.floor(w.percentUsed)}%`
151  const resetMs = parseTime(w.resetsAt)
152  if (resetMs === null) return head
153  const relative = relativeText(resetMs, nowMs)
154  return offsetMin === null
155    ? `${head} · resets ${relative}`
156    : `${head} · resets ${clockText(resetMs, nowMs, offsetMin)} (${relative})`
157}
158
159/**
160 * The status line for every window at or above the threshold, fullest first,
161 * or undefined when none is (or a window's reset has already passed, which
162 * makes its reading stale).
163 */
164export function statusText(
165  windows: readonly StallWatchWindow[],
166  nowMs: number,
167  offsetMin: number | null,
168  threshold: number = STATUS_THRESHOLD,
169): string | undefined {
170  const shown = windows
171    .filter(w => w.percentUsed >= threshold)
172    .filter(w => {
173      const resetMs = parseTime(w.resetsAt)
174      return resetMs === null || resetMs > nowMs
175    })
176    .sort((a, b) => b.percentUsed - a.percentUsed)
177  if (shown.length === 0) return undefined
178  return shown.map(w => windowText(w, nowMs, offsetMin)).join('  |  ')
179}
180
181/**
182 * The window a rate_limit failure is waiting on, among the exhausted windows
183 * whose reset is still ahead. A full window wins, the one that resets last
184 * among several (work cannot continue until every full window has reset), so
185 * a nearly-full window with a days-away reset never outranks the full one the
186 * 429 is about. With none full, the fullest (on a tie, the sooner reset).
187 * Undefined when no exhausted window has a reset ahead (none is that full, the
188 * reset is unknown, or the reading is stale), which leaves the failure to the
189 * ordinary backoff retry.
190 */
191export function exhaustedWindow(windows: readonly StallWatchWindow[], nowMs: number): StallWatchWindow | undefined {
192  const ahead = windows
193    .filter(w => w.percentUsed >= EXHAUSTED_THRESHOLD)
194    .map(w => ({ w, ms: parseTime(w.resetsAt) }))
195    .filter((x): x is { w: StallWatchWindow; ms: number } => x.ms !== null && x.ms > nowMs)
196  const full = ahead.filter(x => x.w.percentUsed >= FULL_THRESHOLD).sort((a, b) => b.ms - a.ms)[0]
197  if (full !== undefined) return full.w
198  return ahead.sort((a, b) => b.w.percentUsed - a.w.percentUsed || a.ms - b.ms)[0]?.w
199}
200
201/**
202 * Words that mark what follows a name as a secret, wherever they sit inside
203 * it. `pass` and `key` count only as a whole part of a name (DB_PASS, SMTP_PASS,
204 * "pass", ENCRYPTION_KEY, JWT_SIGNING_KEY), so `bypass`, `passenger` and
205 * `monkey` are not names of secrets.
206 */
207const SECRET_WORDS = String.raw`password|passwd|passphrase|pwd|secret|token|api[-_ ]?key|access[-_ ]?key|private[-_ ]?key|credentials?|(?<![a-z])pass(?![a-z])|(?<=[_.-])key(?![a-z0-9])`
208
209/** A name holding a secret word; it starts at a token boundary (`\b` sits after `_`, so it is not used). */
210const SECRET_NAME = String.raw`(?<![A-Za-z0-9_.-])([A-Za-z0-9_.-]{0,64}?(?:${SECRET_WORDS})[A-Za-z0-9_.-]{0,64})`
211
212/** A secret value: a quoted string (to its closing quote) or the next run of non-space. */
213const SECRET_VALUE = String.raw`"(?:[^"\\\n]|\\.)*"|'(?:[^'\\\n]|\\.)*'|\S+`
214
215/**
216 * `name=value`, `name: value`, `"name": "value"`, `name := value`, `name => value`
217 * whose name holds a secret word anywhere: DB_PASSWORD, SHOPIFY_API_SECRET,
218 * KLAVIYO_TOKEN, DB_PASS, ENCRYPTION_KEY, dbPassword, aws_secret_access_key,
219 * X-Api-Key. Every part is bounded, so a long pasted blob costs linear time.
220 */
221const SECRET_ASSIGNMENT = new RegExp(String.raw`${SECRET_NAME}(["']?)(\s*(?::=|=>|[:=])\s*)(${SECRET_VALUE})`, 'gi')
222
223/** A command-line flag naming a secret, its value after a space: `--password hunter2`, `--db-pass x`. */
224const SECRET_FLAG = new RegExp(
225  String.raw`(?<![A-Za-z0-9_-])(--?[A-Za-z0-9-]{0,32}?(?:password|passwd|passphrase|secret|token|api-?key|(?<![a-z])pass(?![a-z]))[A-Za-z0-9-]{0,32})(\s+)(?!-)(${SECRET_VALUE})`,
226  'gi',
227)
228
229/** Prose: `my password is hunter2`, `the token is: abc`, `DB_PASSWORD is x`. */
230const SECRET_PROSE = new RegExp(String.raw`${SECRET_NAME}(\s+is\b:?\s*)(${SECRET_VALUE})`, 'gi')
231
232/**
233 * Prose with no `is`: `login with password hunter2`. Only a value holding a
234 * digit, a symbol or quotes, so `the password reset page` reads as written.
235 */
236const SECRET_NEXT_WORD = new RegExp(
237  String.raw`(?<![A-Za-z0-9_.-])([A-Za-z0-9_.-]{0,64}?(?:password|passwd|passphrase)[A-Za-z0-9_.-]{0,64})(\s+)(?!is\b)(?![=:[])(?=["']|\S*[^A-Za-z\s])(${SECRET_VALUE})`,
238  'gi',
239)
240
241/** mysql's attached password flag: `mysql -uroot -phunter2`. */
242const MYSQL_PASSWORD = /(\b(?:mysql|mysqldump|mysqladmin|mariadb)\b[^\n]{0,200}?\s-p)(?=\S)(\S+)/gi
243
244/**
245 * URL userinfo: `postgres://admin:REDACTED@db`, `redis://:pw@cache` keep the user
246 * and drop the password, up to the last `@` before a space (a password may
247 * hold `/` or `@`).
248 */
249const URL_PASSWORD = /(?<![A-Za-z0-9+.-])([A-Za-z][A-Za-z0-9+.-]{0,30}:\/\/)([^\s:@/]{0,128}):(\S{1,256})@/g
250
251/**
252 * A credential pasted with no name in front of it: a run of 20 or more token
253 * characters (base64, hex, URL-safe) that mixes letters and digits, or 32 or
254 * more that mixes upper and lower case. That is an AWS secret access key (40
255 * base64 characters), a hex API key, a session token. Words are letters only
256 * and a path rarely mixes in digits, so prose and most file names pass; a
257 * commit hash or a UUID does not, which a resume note can live with.
258 */
259const BARE_SECRET = /(?<![A-Za-z0-9+/=_-])[A-Za-z0-9+/=_-]{20,}(?![A-Za-z0-9+/=_-])/g
260
261function isSecretLike(run: string): boolean {
262  const hasLetter = /[A-Za-z]/.test(run)
263  return (hasLetter && /\d/.test(run)) || (run.length >= 32 && /[a-z]/.test(run) && /[A-Z]/.test(run))
264}
265
266/** A quoted value keeps its quotes, so JSON stays readable: `"password": "[redacted]"`. */
267function masked(value: string): string {
268  const quote = value[0]
269  return (quote === '"' || quote === "'") && value.length >= 2 && value.endsWith(quote)
270    ? `${quote}[redacted]${quote}`
271    : '[redacted]'
272}
273
274/**
275 * Scrub of credential-shaped substrings before any prompt text is held or
276 * written to the resume note on disk. Three layers: the common vendor token
277 * shapes (and auth headers, cookies, webhook URLs, PEM keys, URL passwords);
278 * any value after a name holding a secret word (`name=value`, `name: value`,
279 * `"name": "value"`, `--name value`, `name is value`, `password value`),
280 * env-var and camelCase names included; and last, any bare run that looks
281 * random (BARE_SECRET), which catches a secret pasted with no name at all,
282 * an AWS secret access key on its own included. It runs last, so the names
283 * the earlier layers keep are not swallowed with their values.
284 */
285export function redact(text: string): string {
286  return text
287    .replace(/-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----/g, '[redacted]')
288    .replace(/\b(sk|pk|rk)[-_][A-Za-z0-9_-]{16,}/g, '[redacted]')
289    .replace(/\b(ghp|gho|ghu|ghs|ghr|github_pat)_[A-Za-z0-9_]{16,}/g, '[redacted]')
290    .replace(/\bshp(at|ca|pa|ss)_[A-Za-z0-9]{16,}/g, '[redacted]')
291    .replace(/\bAIza[0-9A-Za-z_-]{20,}/g, '[redacted]')
292    .replace(/\bnpm_[A-Za-z0-9]{20,}/g, '[redacted]')
293    .replace(/\bglpat-[A-Za-z0-9_-]{16,}/g, '[redacted]')
294    .replace(/\bxox[abposr]-[A-Za-z0-9-]{10,}/g, '[redacted]')
295    .replace(/\bwhsec_[A-Za-z0-9+/=]{16,}/g, '[redacted]')
296    .replace(/\b(AKIA|ASIA)[0-9A-Z]{16}\b/g, '[redacted]')
297    .replace(/\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}/g, '[redacted]')
298    .replace(/\b(hooks\.slack\.com\/(?:services|workflows|triggers)\/|discord(?:app)?\.com\/api\/webhooks\/)\S+/gi, '$1[redacted]')
299    .replace(URL_PASSWORD, '$1$2:[redacted]@')
300    .replace(/\b(Bearer|Basic|Token)\s+[A-Za-z0-9._~+/=-]{8,}/g, '$1 [redacted]')
301    .replace(/\b((?:Set-)?Cookie)(\s*:\s*)[^\n]+/gi, '$1$2[redacted]')
302    .replace(MYSQL_PASSWORD, '$1[redacted]')
303    .replace(SECRET_ASSIGNMENT, (_m, name: string, quote: string, sep: string, value: string) => `${name}${quote}${sep}${masked(value)}`)
304    .replace(SECRET_FLAG, (_m, flag: string, gap: string, value: string) => `${flag}${gap}${masked(value)}`)
305    .replace(SECRET_PROSE, (_m, name: string, sep: string, value: string) => `${name}${sep}${masked(value)}`)
306    .replace(SECRET_NEXT_WORD, (_m, name: string, gap: string, value: string) => `${name}${gap}${masked(value)}`)
307    .replace(BARE_SECRET, run => (isSecretLike(run) ? '[redacted]' : run))
308}
309
310/** How much of the person's last prompt the resume note keeps. */
311export const NOTE_PROMPT_CHARS = 80
312
313/**
314 * Redaction reads this much of a prompt. A secret that starts past it could
315 * only reach the excerpt if the excerpt were this long, which it never is.
316 */
317const REDACT_SCAN_CHARS = 16_000
318
319/**
320 * The only form in which prompt text is kept or written (data minimisation):
321 * redacted first, on the whole head of the prompt so a secret is never cut in
322 * half before it is recognised, then cut to NOTE_PROMPT_CHARS on one line.
323 */
324export function promptExcerpt(text: string): string {
325  const clean = redact(text.slice(0, REDACT_SCAN_CHARS)).replace(/\s+/g, ' ').trim()
326  return clean.length > NOTE_PROMPT_CHARS || text.length > REDACT_SCAN_CHARS
327    ? `${clean.slice(0, NOTE_PROMPT_CHARS).trimEnd()}...`
328    : clean
329}
330
331/** A session id safe to use as a file name (no separators, no `..`). */
332export function isSafeId(id: string): boolean {
333  return /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/.test(id) && !id.includes('..')
334}
335
types/index.d.ts 64 lines
1// The $.state contract of the stall-watch mod. Every key the hooks module
2// reads or writes through $.state is declared here, by literal plugin/key.
3
4/** One rate-limit window as the last session.measure reported it. */
5export type StallWatchWindow = { kind: string; percentUsed: number; resetsAt?: string }
6
7/**
8 * The last main-loop StopFailure: when, the engine's error kind, and whether a
9 * turn.complete has already been paired with it (so it is acted on once).
10 */
11export type StallWatchFailure = { at: number; kind: string; isContextOverflow: boolean; isHandled: boolean }
12
13/** A main-loop turn.complete that ended with reason 'error'. */
14export type StallWatchErrorTurn = { at: number; turnId: string }
15
16/** An auto-retry that is armed but has not fired yet. */
17export type StallWatchPending = { dueAt: number; attempt: number; armedAt: number }
18
19/** A resume-after-reset that is armed but has not fired yet. */
20export type StallWatchResumeArm = { resetsAt: string; dueAt: number; armedAt: number }
21
22declare module 'claude-code' {
23  interface PluginState {
24    'stall-watch': {
25      failure: StallWatchFailure | null
26      errorTurn: StallWatchErrorTurn | null
27      /**
28       * Automated submits (auto-retries and resumes, one shared cap) made since
29       * the person's last prompt (composer, bridge or sdk origin).
30       */
31      attempts: number
32      pending: StallWatchPending | null
33      /** Epoch ms of the person's last prompt, composer, bridge or sdk (0 = none yet). */
34      lastComposerAt: number
35      /** Origin kind of the prompt that started the current/last main turn. */
36      lastOrigin: string | null
37      /**
38       * Redacted excerpt (promptExcerpt) of the person's last prompt,
39       * held in memory only, for the resume note; never the raw text.
40       */
41      lastPrompt: string | null
42      windows: StallWatchWindow[]
43      isStatusShown: boolean
44      /** Local UTC offset in minutes from `date +%z`; null when it could not be read. */
45      tzOffsetMin: number | null
46      resumeEnabled: boolean
47      resumeArm: StallWatchResumeArm | null
48      /** kind|resetsAt of the window the resume note was last written for. */
49      noteKey: string | null
50      /**
51       * Session id of the conversation everything above belongs to; null until
52       * adopted, and after a session.end that moved the process on.
53       */
54      owner: string | null
55      /**
56       * turnId of the running turn the person's last prompt (composer, bridge, sdk) was
57       * typed over (prompt.submit's `turnId`); a failure of that turn is not
58       * retried, as the prompt runs next.
59       */
60      composerTurnId: string | null
61    }
62  }
63}
64