SLOPSHOPPER

goalpost-guard

Stops Claude from passing a check by lowering the bar: weakened or deleted tests, new suppressions, relaxed config, softened CI, lowered coverage, skipped git…

newpanebandguardcommandprocess
A shopper browsing a rack in a slop shop
Preview · a replayed session in a sandbox
claude · ~/work/app · goalpost-guard
│ ┃ Goalpost Guard ✕ › fix the failing auth test and add an audit log call │ ┃ [ Ledger 0 ] [ Exceptions 0 ] [ Rules ] [ Pa │ ┃ ⏺ Read(src/auth.ts) │ ┃ mode ask (inherited) 0 held, 0 let through ⎿ Read 6 lines │ ┃ ⏺ Update(src/auth.ts) │ ┃ Nothing held yet. Changes that lower the bar ⎿ Added 2 lines, removed 1 line │ ┃ show up here. ⏺ Bash(bun test) │ ⎿ 3 pass, 1 fail │ │ ● Done. refresh now rejects expired claims and logs an audit event. │ │ ✻ Worked for 42s · done 4:20 PM │ │ › /goalposts │ ⎿ goalpost-guard: Goalpost Guard ledger opened. │ │ ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── › ? for shortcuts

Draws

Pane · Goalpost Guard
[ Ledger 0 ] [ Exceptions 0 ] [ Rules ] [ Pause ] mode ask (inherited) 0 held, 0 let through Nothing held yet. Changes that lower the bar show up here.
README

goalpost-guard

Stops Claude from making a check pass by lowering the bar. Tested with Claude Code 2.1.287, terminal and desktop.

When a test fails, the shortest way to green is often not to fix the code but to move the goalposts: delete the assertion, skip the test, add @ts-ignore, turn strict off, put continue-on-error on the CI step, drop the coverage threshold, or commit with --no-verify. goalpost-guard reads every edit and shell command before it runs, and when one does that, it stops and asks you. With emcc-core's standard profile and someone at the screen, it lets the change through instead, tells Claude to report it and flags it for witness; strict denies it.

<img src="../../docs/media/goalpost-guard/demo.gif" width="880" alt="Claude Code with Haiku 4.5, told src/ is frozen, rewrites a test's expected value; goalpost-guard holds the edit; the answer goes back to Claude, which fixes src/cart.js instead">

<img src="../../docs/media/goalpost-guard/dialog.png" alt="Goalpost Guard: this change test/cart.test.js changes the values in 1 assertion. Allow it? Block it, Allow once, Allow in this file, Allow in this project, Type something">

Anything you type instead goes to Claude, word for word, as the reason the change was blocked:

<img src="../../docs/media/goalpost-guard/answer.png" alt="The fifth option filled in: Not like that. src isn't really frozen: fix the rounding in withTax and run npm test.">

Install

/plugin marketplace add Xabilimon1/emcc-claude-code-mods
/plugin install goalpost-guard@emcc

Works on its own. With emcc-core installed it follows the EMCC profile, posts its count to the EMCC band and flags each lowered bar for witness.

What it catches

FamilyCaught when a change…
Testsremoves or comments out assertions; rewrites the value an assertion expects (24.19 to 24); points the test at a stand-in module while the real one is still there (../src/cart.js to ../lib/cart.js; an import that follows a moved file is fine); swaps an imported function for a copy written inside the test; removes test cases or deletes a test file (also with rm or git rm); adds .skip, xit, @pytest.mark.skip, xfail, t.Skip(), #[ignore], @Disabled; adds .only; trades exact checks for loose ones (toBe(3) for toBeDefined()); rewrites a snapshot by hand or runs the tests with -u; runs them with --testPathIgnorePatterns, -k "not …", --deselect, --passWithNoTests
Suppressionsadds @ts-ignore, @ts-expect-error, @ts-nocheck, eslint-disable, biome-ignore, # noqa, # type: ignore, pyright: ignore, pylint: disable, //nolint, #[allow(…)], @SuppressWarnings, rubocop:disable, NOLINT, istanbul/c8 ignore, pragma: no cover and more
Configturns off a strict flag in a tsconfig (one strict still covers is fine); lowers or removes an ESLint, Biome or Stylelint rule; turns mypy's strict off or ignore_errors on; grows a ruff, flake8 or pylint ignore list
CIadds continue-on-error: true, allow_failure: true, `\\true, if: false or set +e`; removes a step that runs tests, lint, types or a build; removes a pre-commit hook. GitHub Actions, GitLab, CircleCI, Azure, Bitbucket, Buildkite, Jenkins, Travis, pre-commit, lefthook, husky
Coveragelowers or removes a threshold: jest coverageThreshold, vitest thresholds, coverage.py fail_under, --cov-fail-under, codecov target, nyc and c8, SimpleCov, JaCoCo
Git hooksgit commit or push --no-verify, git commit -n, HUSKY=0, SKIP=…, LEFTHOOK=0, git config core.hooksPath

It also holds sed -i, perl -pi and > redirects into a test, config or CI file: it cannot see that diff, so it asks Claude to use Edit instead.

Adding tests, assertions, checks and thresholds is never a finding, and neither is writing about @ts-ignore in a Markdown file.

What happens next

ModeWhat it does
inherit (default)Follows the EMCC profile: standard lets it through and reports it (asks when there is no screen, so headless applies), strict denies, off is off. Without emcc-core, asks
askThe dialog above
denyRefuses the change and tells Claude to fix the code instead
warnLets it through, records it, and tells Claude to report it in its summary
offNothing

With emcc-core, every change that goes through lowering the bar also posts a flag that witness shows on the step; held changes do not.

In claude -p and other runs with no screen, nobody can answer the dialog, so ask refuses (set headless to allow to let it through and report it instead). If goalpost-guard cannot read the file it is checking, it holds the change rather than let it through.

Exceptions last across sessions and are scoped to the project: "Allow in this file" covers that rule in that file, "Allow in this project" covers that rule everywhere in the project. Revoke them in the ledger.

The ledger: /goalposts

/goalposts opens a pane with three tabs (keys 1, 2, 3; p pauses, Esc closes):

<table> <tr> <td width="34%" valign="top"><img src="../../docs/media/goalpost-guard/ledger.png" alt="Ledger tab: an allowed change with the old and new assertion, and a held one with a button to allow it in the file"></td> <td width="33%" valign="top"><img src="../../docs/media/goalpost-guard/exceptions.png" alt="Exceptions tab: one standing exception with a Revoke button"></td> <td width="33%" valign="top"><img src="../../docs/media/goalpost-guard/rules.png" alt="Rules tab: six families, each on, with a Mute button"></td> </tr> </table>

  • Ledger: every decision this session, newest first: held, allowed, standing exception or noted, the file, what the change did, and the lines it removed (-) or added (+). A held change has an Allow in this file button.
  • Exceptions: the standing ones for this project, each with Revoke.
  • Rules: each family on or off. Mute turns one off for this session; /config turns it off for good (a family off there shows /config instead of the button).

Pause turns goalpost-guard off for the session, like /goalposts pause (/goalposts resume brings it back). Without emcc-core, a line above the prompt counts what was held and let through, with a Ledger button.

Commit trailers

Whatever was let through ends up in the next commit Claude makes, as a git trailer, so a reviewer sees it in git log:

<img src="../../docs/media/goalpost-guard/commit.png" alt="git log -1: Round tax to whole euros, Goalpost-Change: test/cart.test.js: changes the values in 1 assertion (allowed by the user: from now on in this file)">

Each change is reported once; a git commit --amend of that commit carries its trailers again, since an amend rewrites the message. A commit that fails keeps them for the next. Needs git 2.32 or later; turn it off with commitTrailer.

Audit what is already there: /goalposts audit [ref]

Checks every change since ref (default HEAD), untracked files included, with the same rules:

<img src="../../docs/media/goalpost-guard/audit.png" alt="/goalposts audit main: 3 changes lower the bar since main: continue-on-error in ci.yml, an eslint-disable in src/legacy.js, a skipped test in test/cart.test.js">

Options

Set them in /config, or at install with --config key=value.

OptionDefaultWhat it does
modeinheritinherit, ask, deny, warn or off, as above
tests, suppressions, config, ci, coverage, bypassonEach family of checks
headlessdenyWith no one to ask: deny, or allow and report
commitTraileronThe Goalpost-Change trailers
testPatternnoneA regular expression for test files with unusual names, on top of *.test.ts, test_*.py, *_test.go, *Test.java, *_spec.rb, tests/…
ignorePathsnoneGlobs to leave alone, such as vendor/**

What it hooks and calls

  • Hooks: session.start, command.run (goalposts), tool.call (Edit, Write, Bash, before they run), ui.render (AbovePrompt, the goalposts pane)
  • Calls: $.fs.read and $.fs.exists (the file a change touches), $.process.run (git diff, git show and git ls-files, only for /goalposts audit), $.ui.ask, $.ui.open, $.ui.close, $.store (exceptions), $.state, $.session.cwd, $.session.surfaces, $.clock.now, $.command.register, $.emcc.profile, $.emcc.badge and $.emcc.flag (when emcc-core is installed)
  • It rewrites one thing: it adds --trailer flags to a git commit command. It never changes an edit.
  • No network.

Limits

  • It reads the change, not your intent: the checks are patterns, tuned to keep false alarms rare. A rename that drops one expect and adds another is fine; one that drops two and adds one is asked about.
  • NotebookEdit is not checked yet.
  • A file changed by a program it cannot read (a Python one-liner, a code generator) is not seen until /goalposts audit.

Tests

claude plugin test mods/goalpost-guard

81 tests: the detectors (37), the gate, modes, options, exceptions, retargeted imports, trailers and audit (30), and the band and the pane on both terminal and desktop, plus the EMCC profile, badge and flags (14). Each one was seen failing before it passed, by breaking the code it checks.

Source 3 files
hooks/register.tsx 797 lines
1import { atom, read, update } from 'claude-code'
2import type { EngineInterface, Register } from 'claude-code'
3
4import type {
5  GoalpostCategory,
6  GoalpostDecision,
7  GoalpostEntry,
8  GoalpostException,
9  GoalpostFinding,
10} from '../types'
11import { applyEdit, categoryOf, importTarget, inspectChange, inspectCommand } from './detect'
12import type { DetectOptions } from './detect'
13
14const PANE = 'goalposts'
15const KEEP = 100
16const STORE_KEY = 'exceptions'
17const TRAILERS = 10
18
19// Anthropic's palette: clay for what was held, olive for what was let through,
20// sky for standing exceptions, stone for everything secondary.
21const CLAY = '#d97757'
22const OLIVE = '#788c5d'
23const SKY = '#6a9bcc'
24const STONE = '#b0aea5'
25
26const ledger = atom({ plugin: 'goalpost-guard', key: 'ledger' } as const, [])
27const exceptions = atom({ plugin: 'goalpost-guard', key: 'exceptions' } as const, [])
28const tab = atom({ plugin: 'goalpost-guard', key: 'tab' } as const, 'ledger')
29const muted = atom({ plugin: 'goalpost-guard', key: 'muted' } as const, [])
30const isPaused = atom({ plugin: 'goalpost-guard', key: 'isPaused' } as const, false)
31
32type Mode = 'ask' | 'deny' | 'warn' | 'off'
33type Configured = Mode | 'inherit'
34
35const MODES: readonly Configured[] = ['inherit', 'ask', 'deny', 'warn', 'off']
36const CATEGORIES: readonly GoalpostCategory[] = ['tests', 'suppressions', 'config', 'ci', 'coverage', 'bypass']
37
38const CATEGORY_LABEL: Record<GoalpostCategory, string> = {
39  tests: 'Tests: assertions, skips, .only, deleted tests, snapshots',
40  suppressions: 'Suppressions: @ts-ignore, eslint-disable, noqa…',
41  config: 'Config: tsconfig strictness, lint rules, mypy, ruff',
42  ci: 'CI: continue-on-error, || true, removed checks and hooks',
43  coverage: 'Coverage thresholds',
44  bypass: 'Skipped git hooks: --no-verify, HUSKY=0, SKIP=',
45}
46
47const DECISION_LOOK: Record<GoalpostDecision, { label: string; color: string }> = {
48  blocked: { label: 'HELD', color: CLAY },
49  allowed: { label: 'ALLOWED', color: OLIVE },
50  excepted: { label: 'EXCEPTION', color: SKY },
51  warned: { label: 'NOTED', color: STONE },
52}
53
54const ANSWER = {
55  block: 'Block it',
56  once: 'Allow once',
57  file: 'Allow in this file',
58  project: 'Allow in this project',
59} as const
60
61type Settings = {
62  mode: Configured
63  categories: ReadonlySet<GoalpostCategory>
64  testPattern?: RegExp
65  ignore: readonly RegExp[]
66  headless: 'deny' | 'allow'
67  isTrailerOn: boolean
68}
69
70type Verdict = { kind: 'pass' } | { kind: 'warn'; note: string } | { kind: 'deny'; reason: string }
71
72// ---------------------------------------------------------------- settings
73
74function settingsOf(options: Record<string, unknown>): Settings {
75  const mode = MODES.find(one => one === options.mode) ?? 'inherit'
76  const categories = new Set(CATEGORIES.filter(category => options[category] !== false))
77  let testPattern: RegExp | undefined
78  if (typeof options.testPattern === 'string' && options.testPattern !== '') {
79    try {
80      testPattern = new RegExp(options.testPattern)
81    } catch {
82      testPattern = undefined
83    }
84  }
85  const globs = Array.isArray(options.ignorePaths) ? options.ignorePaths : []
86
87  return {
88    mode,
89    categories,
90    testPattern,
91    ignore: globs.filter((glob): glob is string => typeof glob === 'string' && glob !== '').map(globToRegExp),
92    headless: options.headless === 'allow' ? 'allow' : 'deny',
93    isTrailerOn: options.commitTrailer !== false,
94  }
95}
96
97// One pass, longest token first, so `**/` never reads as `*` twice and no placeholder is needed.
98const GLOB_TOKENS: Record<string, string> = { '**/': '(?:.*/)?', '**': '.*', '*': '[^/]*', '?': '[^/]' }
99
100function globToRegExp(glob: string): RegExp {
101  const body = glob
102    .replace(/^\.\//, '')
103    .replace(/\/+$/, '')
104    .replace(/\*\*\/|\*\*|\*|\?|[.+^${}()|[\]\\]/g, token => GLOB_TOKENS[token] ?? `\\${token}`)
105
106  return new RegExp(`(?:^|/)${body}(?:/.*)?$`)
107}
108
109function relative(path: string, cwd: string): string {
110  return path.startsWith(`${cwd}/`) ? path.slice(cwd.length + 1) : path
111}
112
113async function detectOptions($: EngineInterface, settings: Settings): Promise<DetectOptions> {
114  const off = new Set(await read($, muted))
115
116  return {
117    categories: new Set([...settings.categories].filter(category => !off.has(category))),
118    testPattern: settings.testPattern,
119  }
120}
121
122/**
123 * The EMCC profile, or null when emcc-core is not installed. `profile()` is
124 * `off` while EMCC is halted, so the kill switch reaches this mod too.
125 */
126async function emccProfile($: EngineInterface): Promise<'standard' | 'strict' | 'off' | null> {
127  try {
128    return await $.emcc.profile()
129  } catch {
130    return null
131  }
132}
133
134async function modeOf($: EngineInterface, configured: Configured): Promise<Mode> {
135  if (await read($, isPaused)) {
136    return 'off'
137  }
138  const profile = await emccProfile($)
139  if (profile === 'off') {
140    return 'off'
141  }
142  if (configured !== 'inherit') {
143    return configured
144  }
145
146  if (profile === 'strict') {
147    return 'deny'
148  }
149
150  // standard reports instead of asking when someone can read the report: witness shows the flag on the step
151  return profile === 'standard' && (await $.session.surfaces()).length > 0 ? 'warn' : 'ask'
152}
153
154// ---------------------------------------------------------------- ledger, badge, exceptions
155
156function summary(findings: readonly GoalpostFinding[], cwd: string): string {
157  return findings.map(finding => `${finding.file ? `${relative(finding.file, cwd)} ` : ''}${finding.title}`).join('; ')
158}
159
160async function record(
161  $: EngineInterface,
162  tool: string,
163  id: string,
164  findings: readonly GoalpostFinding[],
165  decision: GoalpostDecision,
166  note?: string,
167): Promise<void> {
168  if (findings.length === 0) {
169    return
170  }
171  const at = await $.clock.now()
172  const entries: GoalpostEntry[] = findings.map((finding, index) => ({
173    id: `${id}:${index}`,
174    at,
175    tool,
176    finding,
177    decision,
178    ...(note === undefined ? {} : { note }),
179  }))
180  await update($, ledger, list => [...list, ...entries].slice(-KEEP))
181  await refreshBadge($)
182  // Only a bar that was lowered is flagged: a held change lowered nothing, and the ledger already shows it.
183  if (decision === 'blocked') {
184    return
185  }
186  for (const finding of findings) {
187    try {
188      await $.emcc.flag({
189        source: 'goalpost-guard',
190        rule: finding.rule,
191        ...(finding.file === undefined ? {} : { file: finding.file }),
192        text: `${finding.title} (${TRAILER_WORDS[decision]})`,
193      })
194    } catch {
195      // emcc-core is not installed, or is older than 0.2.0: nothing to tell.
196    }
197  }
198}
199
200async function refreshBadge($: EngineInterface): Promise<void> {
201  const list = await read($, ledger)
202  const held = list.filter(entry => entry.decision === 'blocked').length
203  const through = list.filter(entry => entry.decision !== 'blocked').length
204  const parts = [held > 0 ? `${held} held` : '', through > 0 ? `${through} let through` : ''].filter(Boolean)
205  try {
206    await $.emcc.badge({
207      id: 'goalpost-guard',
208      text: parts.length > 0 ? `goalposts: ${parts.join(', ')}` : undefined,
209      tone: through > 0 ? 'warn' : 'good',
210    })
211  } catch {
212    // emcc-core is not installed: this mod draws its own line instead.
213  }
214}
215
216function isCovered(exception: GoalpostException, finding: GoalpostFinding, project: string): boolean {
217  return (
218    exception.project === project &&
219    exception.rule === finding.rule &&
220    (exception.file === undefined || exception.file === finding.file)
221  )
222}
223
224async function storedExceptions($: EngineInterface): Promise<GoalpostException[]> {
225  const value = await $.store.get(STORE_KEY)
226
227  return Array.isArray(value) ? (value as GoalpostException[]) : []
228}
229
230async function addExceptions(
231  $: EngineInterface,
232  findings: readonly GoalpostFinding[],
233  scope: 'file' | 'project',
234  note: string,
235): Promise<void> {
236  const project = await $.session.cwd()
237  const at = await $.clock.now()
238  const added: GoalpostException[] = findings.map(finding => ({
239    project,
240    rule: finding.rule,
241    ...(scope === 'file' && finding.file !== undefined ? { file: finding.file } : {}),
242    note,
243    at,
244  }))
245  const kept = (await storedExceptions($)).filter(old => !added.some(one => sameException(one, old)))
246  await $.store.set(STORE_KEY, [...kept, ...added])
247  await update($, exceptions, () => [...kept, ...added].filter(one => one.project === project))
248}
249
250function sameException(a: GoalpostException, b: GoalpostException): boolean {
251  return a.project === b.project && a.rule === b.rule && a.file === b.file
252}
253
254async function revokeException($: EngineInterface, gone: GoalpostException): Promise<void> {
255  const kept = (await storedExceptions($)).filter(one => !sameException(one, gone))
256  await $.store.set(STORE_KEY, kept)
257  await update($, exceptions, list => list.filter(one => !sameException(one, gone)))
258}
259
260// ---------------------------------------------------------------- the gate
261
262async function decide(
263  $: EngineInterface,
264  tool: string,
265  id: string,
266  found: readonly GoalpostFinding[],
267  settings: Settings,
268  mode: Mode,
269): Promise<Verdict> {
270  const cwd = await $.session.cwd()
271  const watched = found.filter(
272    finding => finding.file === undefined || !settings.ignore.some(glob => glob.test(relative(finding.file ?? '', cwd))),
273  )
274  const standing = await storedExceptions($)
275  const excepted = watched.filter(finding => standing.some(one => isCovered(one, finding, cwd)))
276  const open = watched.filter(finding => !excepted.includes(finding))
277  await record($, tool, id, excepted, 'excepted')
278  if (open.length === 0) {
279    return { kind: 'pass' }
280  }
281  const what = summary(open, cwd)
282
283  if (mode === 'warn') {
284    await record($, tool, id, open, 'warned')
285
286    return { kind: 'warn', note: `goalpost-guard noted that this change ${what}. Say so to the user in your summary.` }
287  }
288  if (mode === 'deny') {
289    await record($, tool, id, open, 'blocked')
290
291    return {
292      kind: 'deny',
293      reason:
294        `goalpost-guard refused this change: it ${what}. ` +
295        'This session does not let a check pass by lowering the bar. Fix the code instead, or ask the user to allow it.',
296    }
297  }
298  if ((await $.session.surfaces()).length === 0) {
299    if (settings.headless === 'allow') {
300      await record($, tool, id, open, 'warned', 'no one could be asked')
301
302      return { kind: 'warn', note: `goalpost-guard let this through unasked (no screen): it ${what}. Report it.` }
303    }
304    await record($, tool, id, open, 'blocked', 'no one could be asked')
305
306    return {
307      kind: 'deny',
308      reason: `goalpost-guard held this change: it ${what}, and no one is here to allow it. Make the check pass without lowering the bar.`,
309    }
310  }
311
312  const answer = await $.ui
313    .ask(`Goalpost Guard: this ${tool === 'Bash' ? 'command' : 'change'} ${what}. Allow it?`, {
314      header: 'Goalpost',
315      options: [ANSWER.block, ANSWER.once, ANSWER.file, ANSWER.project],
316    })
317    .catch(() => ANSWER.block)
318
319  if (answer === ANSWER.once) {
320    await record($, tool, id, open, 'allowed')
321
322    return { kind: 'pass' }
323  }
324  if (answer === ANSWER.file || answer === ANSWER.project) {
325    await addExceptions($, open, answer === ANSWER.file ? 'file' : 'project', `allowed from the dialog: ${what}`)
326    await record($, tool, id, open, 'allowed', answer === ANSWER.file ? 'from now on in this file' : 'from now on in this project')
327
328    return { kind: 'pass' }
329  }
330  const said = answer === ANSWER.block ? undefined : answer.trim()
331  await record($, tool, id, open, 'blocked', said)
332
333  return {
334    kind: 'deny',
335    reason:
336      `goalpost-guard held this change: it ${what}. The user blocked it. ` +
337      (said ? `The user says: "${said}"` : 'Make the check pass without lowering the bar, or explain why the bar has to move.'),
338  }
339}
340
341function normalized(path: string): string {
342  const parts: string[] = []
343  for (const part of path.split('/')) {
344    if (part === '..') {
345      parts.pop()
346    } else if (part !== '.' && part !== '') {
347      parts.push(part)
348    }
349  }
350
351  return `/${parts.join('/')}`
352}
353
354/** Where a module an import names could be on disk, relative to the file importing it. */
355function candidatesFor(file: string, target: string, cwd: string): string[] | null {
356  const dir = file.slice(0, file.lastIndexOf('/')) || cwd
357  if (target.startsWith('.') && target.includes('/')) {
358    const base = normalized(`${dir.startsWith('/') ? dir : `${cwd}/${dir}`}/${target}`)
359
360    return ['', '.js', '.ts', '.tsx', '.jsx', '.mjs', '.cjs', '/index.js', '/index.ts'].map(ext => `${base}${ext}`)
361  }
362  if (/^\.+\w/.test(target) || /^\w+(?:\.\w+)+$/.test(target)) {
363    const ups = target.match(/^\.*/)?.[0].length ?? 0
364    const root = ups === 0 ? cwd : normalized(`${dir.startsWith('/') ? dir : `${cwd}/${dir}`}${'/..'.repeat(Math.max(0, ups - 1))}`)
365    const base = `${root}/${target.slice(ups).replace(/\./g, '/')}`
366
367    return [`${base}.py`, `${base}/__init__.py`]
368  }
369
370  return null
371}
372
373/**
374 * Keeps a retargeted import only when the module the test used to load is
375 * still on disk: then the test was pointed away from real code. When it is
376 * gone, the code moved and the import followed it.
377 */
378async function withRealRetargets($: EngineInterface, found: GoalpostFinding[]): Promise<GoalpostFinding[]> {
379  const cwd = await $.session.cwd()
380  const kept: GoalpostFinding[] = []
381  for (const finding of found) {
382    const old = finding.rule === 'tests.retargeted' ? importTarget(finding.before?.[0] ?? '')?.target : undefined
383    const places = old === undefined || finding.file === undefined ? null : candidatesFor(finding.file, old, cwd)
384    if (places === null) {
385      kept.push(finding)
386      continue
387    }
388    let isThere = false
389    for (const place of places) {
390      if (await $.fs.exists(place)) {
391        isThere = true
392        break
393      }
394    }
395    if (isThere) {
396      kept.push(finding)
397    }
398  }
399
400  return kept
401}
402
403async function fileFindings(
404  $: EngineInterface,
405  path: string,
406  toAfter: (before: string | null) => string | null | undefined,
407  options: DetectOptions,
408): Promise<GoalpostFinding[]> {
409  const before = (await $.fs.exists(path)) ? await $.fs.read(path) : null
410  const after = toAfter(before)
411  if (after === undefined) {
412    return []
413  }
414
415  return withRealRetargets($, inspectChange({ path, before, after }, options))
416}
417
418function heldForError(error: unknown): Verdict {
419  const why = error instanceof Error ? error.message : String(error)
420
421  return {
422    kind: 'deny',
423    reason: `goalpost-guard could not check this change (${why.slice(0, 200)}), so it held it. The user can run /goalposts pause to let changes through.`,
424  }
425}
426
427// ---------------------------------------------------------------- commit trailers
428
429const COMMIT = /\bgit\b(?:\s+-[cC]\s+\S+)*\s+commit\b/g
430const AMEND = /\bgit\b(?:\s+-[cC]\s+\S+)*\s+commit\b[^;&|]*\s--amend\b/
431
432function quoted(text: string): string {
433  return `'${text.replace(/'/g, `'\\''`)}'`
434}
435
436const TRAILER_WORDS: Record<GoalpostDecision, string> = {
437  blocked: 'held',
438  allowed: 'allowed by the user',
439  excepted: 'standing exception',
440  warned: 'let through and reported',
441}
442
443function trailerOf(entry: GoalpostEntry, cwd: string): string {
444  const where = entry.finding.file ? `${relative(entry.finding.file, cwd)}: ` : ''
445  const why = entry.note ? `: ${entry.note}` : ''
446
447  return `Goalpost-Change: ${where}${entry.finding.title} (${TRAILER_WORDS[entry.decision]}${why})`.replace(/\s+/g, ' ').slice(0, 240)
448}
449
450/** The command with `--trailer` lines after its first `git commit` outside quotes, or null without one. */
451function withTrailers(command: string, entries: readonly GoalpostEntry[], cwd: string): string | null {
452  const quotedSpans = [...command.matchAll(/'[^']*'|"(?:\\.|[^"\\])*"/g)].map(match => [match.index, match.index + match[0].length])
453  for (const match of command.matchAll(COMMIT)) {
454    const at = match.index
455    if (quotedSpans.some(([from = 0, to = 0]) => at > from && at < to)) {
456      continue
457    }
458    const end = at + match[0].length
459    const flags = entries.map(entry => ` --trailer ${quoted(trailerOf(entry, cwd))}`).join('')
460
461    return command.slice(0, end) + flags + command.slice(end)
462  }
463
464  return null
465}
466
467// ---------------------------------------------------------------- audit
468
469async function audit($: EngineInterface, base: string, options: DetectOptions): Promise<string> {
470  const listed = await $.process.run(['git', 'diff', '--name-status', '-M', base])
471  if (listed.exitCode !== 0) {
472    return `git diff ${base} failed: ${listed.stderr.trim().slice(0, 200)}`
473  }
474  const untracked = await $.process.run(['git', 'ls-files', '--others', '--exclude-standard'])
475  const rows = listed.stdout
476    .split('\n')
477    .filter(Boolean)
478    .map(line => line.split('\t'))
479  for (const path of untracked.stdout.split('\n').filter(Boolean)) {
480    rows.push(['A', path])
481  }
482  const cwd = await $.session.cwd()
483  const found: GoalpostFinding[] = []
484  for (const [status = '', first = '', second] of rows.slice(0, 300)) {
485    const path = `${cwd}/${second ?? first}`
486    const shown = await $.process.run(['git', 'show', `${base}:${first}`])
487    const before = status.startsWith('A') || shown.exitCode !== 0 ? null : shown.stdout
488    const after = status.startsWith('D') || !(await $.fs.exists(path)) ? null : await $.fs.read(path)
489    found.push(...(await withRealRetargets($, inspectChange({ path, before, after }, options))))
490  }
491  if (found.length === 0) {
492    return `Nothing in the changes since ${base} lowers the bar (${rows.length} files checked).`
493  }
494
495  return [
496    `${found.length} ${found.length === 1 ? 'change lowers' : 'changes lower'} the bar since ${base}:`,
497    ...found.map(finding => `- ${relative(finding.file ?? '', cwd)} ${finding.title}: ${finding.detail}`),
498  ].join('\n')
499}
500
501// ---------------------------------------------------------------- register
502
503export const register: Register = (on, options) => {
504  const settings = settingsOf(options as Record<string, unknown>)
505
506  on('session.start', async ($, e, next) => {
507    await $.command.register({
508      name: 'goalposts',
509      description: 'Goalpost Guard: open the ledger, or "audit [ref]", "pause", "resume"',
510    })
511    const project = await $.session.cwd()
512    const standing = (await storedExceptions($)).filter(one => one.project === project)
513    await update($, exceptions, () => standing)
514
515    return next(e)
516  })
517
518  on('command.run', { command: 'goalposts' }, async ($, e) => {
519    const [verb = '', ...rest] = e.args.trim().split(/\s+/)
520    if (verb === 'pause' || verb === 'resume') {
521      await update($, isPaused, () => verb === 'pause')
522
523      return { text: verb === 'pause' ? 'Goalpost Guard paused for this session.' : 'Goalpost Guard is on again.' }
524    }
525    if (verb === 'audit') {
526      return { text: await audit($, rest[0] ?? 'HEAD', await detectOptions($, settings)) }
527    }
528    const opened = await $.ui.open({ id: PANE, title: 'Goalpost Guard', focus: true, closeOnEscape: true })
529    if (!opened.isPlaced) {
530      const list = await read($, ledger)
531
532      return {
533        text:
534          list.length === 0
535            ? 'Goalpost Guard has not held anything this session.'
536            : list.map(entry => `${DECISION_LOOK[entry.decision].label} ${entry.finding.title} ${entry.finding.file ?? ''}`).join('\n'),
537      }
538    }
539
540    return { text: 'Goalpost Guard ledger opened.' }
541  })
542
543  on('tool.call', { tool: 'Edit' }, async ($, e, next) => {
544    const mode = await modeOf($, settings.mode)
545    if (mode === 'off') {
546      return next(e)
547    }
548    let verdict: Verdict
549    try {
550      const found = await fileFindings(
551        $,
552        e.file_path,
553        before => (before === null ? undefined : (applyEdit(before, e.old_string, e.new_string, e.replace_all === true) ?? undefined)),
554        await detectOptions($, settings),
555      )
556      verdict = await decide($, 'Edit', e.tool_use_id, found, settings, mode)
557    } catch (error) {
558      verdict = heldForError(error)
559    }
560    if (verdict.kind === 'deny') {
561      return { deny: verdict.reason }
562    }
563    const ran = await next(e)
564    if (verdict.kind === 'warn' && ran.deny === undefined) {
565      return { ...ran, context: [...(ran.context ?? []), verdict.note] }
566    }
567
568    return ran
569  })
570
571  on('tool.call', { tool: 'Write' }, async ($, e, next) => {
572    const mode = await modeOf($, settings.mode)
573    if (mode === 'off') {
574      return next(e)
575    }
576    let verdict: Verdict
577    try {
578      const found = await fileFindings($, e.file_path, () => e.content, await detectOptions($, settings))
579      verdict = await decide($, 'Write', e.tool_use_id, found, settings, mode)
580    } catch (error) {
581      verdict = heldForError(error)
582    }
583    if (verdict.kind === 'deny') {
584      return { deny: verdict.reason }
585    }
586    const ran = await next(e)
587    if (verdict.kind === 'warn' && ran.deny === undefined) {
588      return { ...ran, context: [...(ran.context ?? []), verdict.note] }
589    }
590
591    return ran
592  })
593
594  on('tool.call', { tool: 'Bash' }, async ($, e, next) => {
595    const mode = await modeOf($, settings.mode)
596    if (mode === 'off') {
597      return next(e)
598    }
599    let verdict: Verdict
600    try {
601      const found = inspectCommand(e.command, await detectOptions($, settings))
602      verdict = await decide($, 'Bash', e.tool_use_id, found, settings, mode)
603    } catch (error) {
604      verdict = heldForError(error)
605    }
606    if (verdict.kind === 'deny') {
607      return { deny: verdict.reason }
608    }
609
610    const cwd = await $.session.cwd()
611    const listed = settings.isTrailerOn ? await read($, ledger) : []
612    const pending = listed.filter(entry => entry.decision !== 'blocked' && entry.reportedAt === undefined)
613    // An amend rewrites the last commit's message: carry that commit's trailers over to it.
614    const lastAt = Math.max(-1, ...listed.map(entry => entry.reportedAt ?? -1))
615    const again = AMEND.test(e.command) && lastAt >= 0 ? listed.filter(entry => entry.reportedAt === lastAt) : []
616    const reported = [...again, ...pending].slice(-TRAILERS)
617    const rewritten = reported.length > 0 ? withTrailers(e.command, reported, cwd) : null
618    const ran = await next(rewritten === null ? e : { ...e, command: rewritten })
619    if (rewritten !== null && ran.deny === undefined && ran.isError !== true) {
620      const done = new Set(reported.map(entry => entry.id))
621      const at = await $.clock.now()
622      await update($, ledger, list => list.map(entry => (done.has(entry.id) ? { ...entry, reportedAt: at } : entry)))
623    }
624    if (verdict.kind === 'warn' && ran.deny === undefined) {
625      return { ...ran, context: [...(ran.context ?? []), verdict.note] }
626    }
627
628    return ran
629  })
630
631  on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => {
632    if (e.props.hasSurvey || (await emccProfile($)) !== null) {
633      return next(e)
634    }
635    const list = await read($, ledger)
636    const paused = await read($, isPaused)
637    if (list.length === 0 && !paused) {
638      return next(e)
639    }
640    const held = list.filter(entry => entry.decision === 'blocked').length
641    const through = list.length - held
642    const { Box, Button, Text } = $.ui.resolve(e)
643
644    return (
645      <Box>
646        <Text color={CLAY} bold>
647          Goalpost Guard{' '}
648        </Text>
649        {paused && <Text color={STONE}>paused </Text>}
650        {held > 0 && <Text color={CLAY}>{held} held </Text>}
651        {through > 0 && <Text color={OLIVE}>{through} let through </Text>}
652        <Button key="ledger" label="Ledger" onPress={() => $.ui.open({ id: PANE, title: 'Goalpost Guard', focus: true, closeOnEscape: true })} />
653      </Box>
654    )
655  })
656
657  on('ui.render', { component: 'Pane', requestId: PANE }, async ($, e) => {
658    const { Box, Button, Text } = $.ui.resolve(e)
659    const shown = await read($, tab)
660    const list = await read($, ledger)
661    const standing = await read($, exceptions)
662    const off = await read($, muted)
663    const paused = await read($, isPaused)
664    const mode = await modeOf($, settings.mode)
665    const cwd = await $.session.cwd()
666    const held = list.filter(entry => entry.decision === 'blocked').length
667    const room = Math.max(3, Math.floor(((e.viewport?.rows ?? 24) - 6) / 4))
668
669    const tabs = (
670      <Box gap={1}>
671        <Button key="tab-ledger" hotkey="1" label={`Ledger ${list.length}`} variant={shown === 'ledger' ? 'primary' : undefined} onPress={() => update($, tab, () => 'ledger')} />
672        <Button key="tab-exceptions" hotkey="2" label={`Exceptions ${standing.length}`} variant={shown === 'exceptions' ? 'primary' : undefined} onPress={() => update($, tab, () => 'exceptions')} />
673        <Button key="tab-rules" hotkey="3" label="Rules" variant={shown === 'rules' ? 'primary' : undefined} onPress={() => update($, tab, () => 'rules')} />
674        <Button key="pause" hotkey="p" label={paused ? 'Resume' : 'Pause'} onPress={() => update($, isPaused, value => !value)} />
675      </Box>
676    )
677    const status = (
678      <Text>
679        <Text color={STONE}>mode </Text>
680        <Text color={mode === 'off' ? STONE : CLAY} bold>
681          {mode}
682        </Text>
683        <Text color={STONE}>
684          {settings.mode === 'inherit' && mode !== 'off' ? ' (inherited)' : ''}
685          {'   '}
686          {held} held, {list.length - held} let through
687        </Text>
688      </Text>
689    )
690
691    if (shown === 'exceptions') {
692      return (
693        <Box flexDirection="column" gap={1}>
694          {tabs}
695          {status}
696          {standing.length === 0 && <Text color={STONE}>No standing exceptions in this project.</Text>}
697          {standing.map(one => (
698            <Box key={`exception-${one.rule}-${one.file ?? '*'}`} flexDirection="column">
699              <Text>
700                <Text color={SKY} bold>
701                  {one.rule}
702                </Text>
703                <Text> {one.file ? relative(one.file, cwd) : 'anywhere in this project'}</Text>
704              </Text>
705              <Text color={STONE}>{one.note}</Text>
706              <Button key={`revoke-${one.rule}-${one.file ?? '*'}`} label="Revoke" onPress={() => revokeException($, one)} />
707            </Box>
708          ))}
709        </Box>
710      )
711    }
712
713    if (shown === 'rules') {
714      return (
715        <Box flexDirection="column" gap={1}>
716          {tabs}
717          {status}
718          {CATEGORIES.map(category => {
719            const isConfigured = settings.categories.has(category)
720            const isOn = isConfigured && !off.includes(category)
721
722            return (
723              <Box key={`rule-${category}`} gap={1}>
724                <Box width={10} flexShrink={0}>
725                  {isConfigured ? (
726                    <Button
727                      key={`toggle-${category}`}
728                      label={isOn ? 'Mute' : 'Unmute'}
729                      onPress={() => update($, muted, now => (now.includes(category) ? now.filter(one => one !== category) : [...now, category]))}
730                    />
731                  ) : (
732                    <Text color={STONE}>/config</Text>
733                  )}
734                </Box>
735                <Box width={3} flexShrink={0}>
736                  <Text color={isOn ? OLIVE : STONE} bold>
737                    {isOn ? 'on' : 'off'}
738                  </Text>
739                </Box>
740                <Box flexGrow={1} flexShrink={1}>
741                  <Text color={isOn ? undefined : STONE}>{CATEGORY_LABEL[category]}</Text>
742                </Box>
743              </Box>
744            )
745          })}
746          <Text color={STONE}>Mute lasts this session; /config changes it for good.</Text>
747        </Box>
748      )
749    }
750
751    return (
752      <Box flexDirection="column" gap={1}>
753        {tabs}
754        {status}
755        {list.length === 0 && <Text color={STONE}>Nothing held yet. Changes that lower the bar show up here.</Text>}
756        {[...list]
757          .reverse()
758          .slice(0, room)
759          .map(entry => {
760            const look = DECISION_LOOK[entry.decision]
761
762            return (
763              <Box key={`entry-${entry.id}`} flexDirection="column">
764                <Text>
765                  <Text color={look.color} bold inverse>
766                    {` ${look.label} `}
767                  </Text>
768                  <Text color={STONE}> {categoryOf(entry.finding)} </Text>
769                  <Text>{entry.finding.file ? relative(entry.finding.file, cwd) : entry.tool}</Text>
770                </Text>
771                <Text bold>{entry.finding.title}</Text>
772                {(entry.finding.before ?? []).map(line => (
773                  <Text color={CLAY} wrap="truncate-end">
774                    {`  - ${line}`}
775                  </Text>
776                ))}
777                {(entry.finding.after ?? []).map(line => (
778                  <Text color={SKY} wrap="truncate-end">
779                    {`  + ${line}`}
780                  </Text>
781                ))}
782                {entry.note !== undefined && <Text color={STONE} italic>{`  ${entry.note}`}</Text>}
783                {entry.decision === 'blocked' && entry.finding.file !== undefined && (
784                  <Button
785                    key={`allow-${entry.id}`}
786                    label="Allow in this file"
787                    onPress={() => addExceptions($, [entry.finding], 'file', 'allowed from the ledger')}
788                  />
789                )}
790              </Box>
791            )
792          })}
793      </Box>
794    )
795  })
796}
797
hooks/detect.ts 935 lines
1import type { GoalpostCategory, GoalpostFinding, GoalpostRule } from '../types'
2
3/**
4 * What the detectors look at, as the hooks module configured them.
5 */
6export type DetectOptions = {
7  /** The categories that are on; a finding of any other is dropped. */
8  categories: ReadonlySet<GoalpostCategory>
9  /** An extra pattern for test files, on top of the usual names. */
10  testPattern?: RegExp
11}
12
13/** One file as it was and as the change leaves it; `null` when it does not exist. */
14export type FileChange = { path: string; before: string | null; after: string | null }
15
16const CATEGORY_OF: Record<GoalpostRule, GoalpostCategory> = {
17  'tests.assertions': 'tests',
18  'tests.removed': 'tests',
19  'tests.skipped': 'tests',
20  'tests.focused': 'tests',
21  'tests.loosened': 'tests',
22  'tests.expected': 'tests',
23  'tests.retargeted': 'tests',
24  'tests.snapshots': 'tests',
25  'tests.narrowed': 'tests',
26  suppressions: 'suppressions',
27  config: 'config',
28  ci: 'ci',
29  coverage: 'coverage',
30  bypass: 'bypass',
31  'shell-edit': 'tests',
32}
33
34const SAMPLE = 3
35
36// ---------------------------------------------------------------- text helpers
37
38function count(text: string, pattern: RegExp): number {
39  return text.match(new RegExp(pattern.source, pattern.flags.includes('g') ? pattern.flags : `${pattern.flags}g`))?.length ?? 0
40}
41
42/** The text without whole-line comments, so commenting a line out counts as removing it. */
43function code(text: string): string {
44  return text
45    .split('\n')
46    .filter(line => !/^\s*(?:\/\/|#(?![!\[])|\/\*|\*|--|<!--)/.test(line))
47    .join('\n')
48}
49
50/** The lines of `after` that match `pattern` and were not in `before`, trimmed, at most a few. */
51function addedLines(before: string, after: string, global: RegExp): string[] {
52  const pattern = new RegExp(global.source, global.flags.replace('g', ''))
53  const held = new Map<string, number>()
54  for (const line of before.split('\n')) {
55    held.set(line.trim(), (held.get(line.trim()) ?? 0) + 1)
56  }
57  const added: string[] = []
58  for (const line of after.split('\n')) {
59    const key = line.trim()
60    const left = held.get(key) ?? 0
61    if (left > 0) {
62      held.set(key, left - 1)
63    } else if (pattern.test(line)) {
64      added.push(key)
65    }
66  }
67
68  return added.slice(0, SAMPLE)
69}
70
71/**
72 * Assertion lines rewritten in place: a removed line and an added one that
73 * read the same once every number and string is blanked out, as when
74 * `toBe(24.19)` becomes `toBe(24)`.
75 */
76function rewrittenValues(before: string, after: string): Array<readonly [string, string]> {
77  const literal = /-?\d+(?:\.\d+)?(?:e-?\d+)?|'(?:\\.|[^'\\])*'|"(?:\\.|[^"\\])*"|`(?:\\.|[^`\\])*`/g
78  const shape = (line: string): string => line.replace(literal, '_')
79  const single = new RegExp(ASSERTION.source)
80  const gone = removedLinesAll(before, after).filter(line => single.test(line))
81  const added = removedLinesAll(after, before).filter(line => single.test(line))
82  const pairs: Array<readonly [string, string]> = []
83  for (const line of gone) {
84    const index = added.findIndex(other => shape(other) === shape(line))
85    if (index >= 0) {
86      pairs.push([line, added[index] ?? ''] as const)
87      added.splice(index, 1)
88    }
89  }
90
91  return pairs
92}
93
94const IMPORT_FORMS: readonly RegExp[] = [
95  /^(\s*(?:import|export)\b[^'"`]*?\bfrom\s*)(['"`])([^'"`]+)\2/,
96  /^(\s*import\s*)(['"`])([^'"`]+)\2/,
97  /^(.*?\brequire\s*\(\s*)(['"`])([^'"`]+)\2/,
98  /^(\s*from\s+)()([\w.]+)(?=\s+import\b)/,
99  /^(\s*import\s+)()([\w.]+)\s*$/,
100]
101
102/** The module an import line loads, and the line with that module blanked out. */
103export function importTarget(line: string): { target: string; shape: string } | null {
104  for (const form of IMPORT_FORMS) {
105    const match = form.exec(line)
106    if (match) {
107      return { target: match[3] ?? '', shape: line.replace(form, (_all, lead: string, quote: string) => `${lead}${quote}_${quote}`) }
108    }
109  }
110
111  return null
112}
113
114/**
115 * Imports that now load the same names from a different module: a test
116 * pointed at a stand-in instead of the code it was written for. Whether the
117 * old module still exists (a stand-in) or was moved (a fix) is the hooks'
118 * call, since only they can look at the disk.
119 */
120function retargetedImports(before: string, after: string): Array<readonly [string, string]> {
121  const gone = removedLinesAll(before, after).filter(line => importTarget(line) !== null)
122  const added = removedLinesAll(after, before).filter(line => importTarget(line) !== null)
123  const pairs: Array<readonly [string, string]> = []
124  for (const line of gone) {
125    const shape = importTarget(line)?.shape
126    const index = added.findIndex(other => importTarget(other)?.shape === shape)
127    if (index >= 0) {
128      pairs.push([line, added[index] ?? ''] as const)
129      added.splice(index, 1)
130    }
131  }
132
133  return pairs
134}
135
136/** The names an import line brings in: `{ a, b as c }`, a default, `* as x`, or Python's `import a, b`. */
137function importedNames(line: string): string[] {
138  const js = /^\s*import\s+(.+?)\s+from\b/.exec(line) ?? /^\s*(?:const|let|var)\s+(.+?)\s*=\s*require\b/.exec(line)
139  const py = /^\s*from\s+[\w.]+\s+import\s+\(?([^)#]+)/.exec(line)
140  const list = js?.[1] ?? py?.[1] ?? ''
141
142  return list
143    .replace(/[{}()]/g, ',')
144    .split(',')
145    .map(part => part.trim().split(/\s+as\s+|\s*:\s*/).pop()?.replace(/^\*\s*as\s+/, '').trim() ?? '')
146    .filter(name => /^[A-Za-z_$][\w$]*$/.test(name))
147}
148
149/**
150 * Names a test stopped importing and now defines itself: the code under test
151 * swapped for a copy that lives inside the test.
152 */
153function inlinedStandIns(before: string, after: string): string[] {
154  const names = (text: string): Set<string> =>
155    new Set(text.split('\n').filter(line => importTarget(line) !== null).flatMap(importedNames))
156  // Still imported counts too: in Python a later `def` shadows the import.
157  return [...names(before)].filter(name => {
158    const defined = new RegExp(`(?:^|\\n)\\s*(?:export\\s+)?(?:(?:const|let|var)\\s+${name}\\s*=|(?:async\\s+)?function\\s*\\*?\\s*${name}\\s*\\(|class\\s+${name}\\b|def\\s+${name}\\s*\\()`)
159
160    return defined.test(after) && !defined.test(before)
161  })
162}
163
164/** Every line of `before`, trimmed, that is not in `after` (counting repeats). */
165function removedLinesAll(before: string, after: string): string[] {
166  const held = new Map<string, number>()
167  for (const line of after.split('\n')) {
168    held.set(line.trim(), (held.get(line.trim()) ?? 0) + 1)
169  }
170  const removed: string[] = []
171  for (const line of before.split('\n')) {
172    const key = line.trim()
173    const left = held.get(key) ?? 0
174    if (left > 0) {
175      held.set(key, left - 1)
176    } else if (key !== '') {
177      removed.push(key)
178    }
179  }
180
181  return removed
182}
183
184/** The lines of `before` that match `pattern` and are gone from `after`. */
185function removedLines(before: string, after: string, pattern: RegExp): string[] {
186  return addedLines(after, before, pattern)
187}
188
189function basename(path: string): string {
190  return path.split('/').pop() ?? path
191}
192
193function plural(n: number, one: string, many = `${one}s`): string {
194  return `${n} ${n === 1 ? one : many}`
195}
196
197// ---------------------------------------------------------------- tests
198
199const TEST_PATH = [
200  /(?:^|\/)(?:tests?|__tests__|specs?|e2e)\//,
201  /\.(?:test|spec|e2e)\.[cm]?[jt]sx?$/,
202  /(?:^|\/)test_[^/]*\.py$/,
203  /_test\.(?:py|go|exs?)$/,
204  /(?:Test|Tests|Spec)\.(?:java|kt|cs|swift|scala)$/,
205  /_spec\.rb$/,
206  /\.(?:test|spec)\.(?:php|dart)$/,
207]
208
209const BACKUP = /\.(?:bak|orig|rej|swp|tmp)$|~$/
210
211export function isTestFile(path: string, extra?: RegExp): boolean {
212  if (BACKUP.test(path)) {
213    return false
214  }
215
216  return TEST_PATH.some(pattern => pattern.test(path)) || (extra?.test(path) ?? false)
217}
218
219const ASSERTION =
220  /\bexpect\s*\(|\bassert(?:\.\w+|_\w+|[A-Z]\w*)?\s*!?\s*\(|\bassert\s+(?=[^\s(=:])|\.should\b|\bt\.(?:Errorf?|Fatalf?|Fail(?:Now)?)\s*\(|\brequire\.[A-Z]\w*\s*\(|\bXCTAssert\w*\s*\(|\bpytest\.raises\s*\(/g
221
222const TEST_CASE =
223  /\b[xf]?(?:it|test|specify)(?:\.(?:skip|only|todo|concurrent|failing|each\s*\(.*?\)))*\s*\(\s*['"`]|\bdef\s+test\w*\s*\(|\bfunc\s+Test\w*\s*\(|#\[(?:tokio::)?test\]|@Test\b|\bfn\s+test_\w*\s*\(/g
224
225const SKIP =
226  /\b(?:it|test|describe|context|suite|specify)\.(?:skip|todo)\b|\bx(?:it|describe|test|context|specify)\s*\(|@pytest\.mark\.(?:skip|skipif|xfail)\b|@unittest\.(?:skip|skipIf|skipUnless|expectedFailure)\b|\bpytest\.(?:skip|xfail)\s*\(|\bt\.Skip(?:f|Now)?\s*\(|#\[ignore\b|@(?:Disabled|Ignore)\b|\bpending\s*\(|\bself\.skipTest\s*\(|\bmarkTestSkipped\s*\(/g
227
228const FOCUS = /\b(?:it|test|describe|context|suite|specify)\.only\b|\bf(?:it|describe|context)\s*\(/g
229
230const STRONG =
231  /\.(?:toBe|toEqual|toStrictEqual|toHaveBeenCalledWith|toHaveBeenCalledTimes|toThrow|toThrowError|toMatchObject|toContain|toHaveLength|toHaveProperty|toMatchSnapshot|toMatchInlineSnapshot)\s*\(|\bassert(?:Equals?|_eq|_ne|NotEqual|Raises|Is|In|Count(?:Equal)?|DictEqual|ListEqual|AlmostEqual|Same|Throws)\s*!?\s*\(|\bassert\b[^\n]*(?:==|!=|\bin\b|\bis\b)/g
232
233const WEAK =
234  /\.(?:toBeTruthy|toBeDefined|toBeFalsy|toBeInstanceOf|toBeCloseTo|toHaveBeenCalled|toBeGreaterThanOrEqual\s*\(\s*0\s*\)|toBeGreaterThan\s*\(\s*-?1\s*\))\s*\(|\.not\.(?:toBeNull|toBeUndefined|toThrow)\s*\(|\bassert(?:True|IsNotNone|IsInstance|Truthy)\s*\(|\bassert\s+\w+(?:\.\w+)*\s*$|\.toMatch\s*\(\s*\/\.\*\/|expect\.anything\s*\(/gm
235
236const SNAPSHOT_FILE = /(?:__snapshots__\/|\.snap$|\.ambr$|__image_snapshots__\/)/
237
238function testFindings(change: FileChange, testPattern?: RegExp): GoalpostFinding[] {
239  const { path, before, after } = change
240  const isTest = isTestFile(path, testPattern) || /#\[cfg\(test\)\]/.test(before ?? '')
241  const found: GoalpostFinding[] = []
242
243  if (SNAPSHOT_FILE.test(path) && before !== null && before !== after) {
244    found.push({
245      rule: 'tests.snapshots',
246      file: path,
247      title: after === null ? 'deletes a snapshot' : 'rewrites a snapshot',
248      detail: `${basename(path)} is a recorded expectation; changing it by hand makes the test agree with the code`,
249    })
250  }
251  if (!isTest || before === null) {
252    if (isTest && after !== null) {
253      const skips = count(code(after), SKIP)
254      const focus = count(code(after), FOCUS)
255      if (skips > 0) {
256        found.push({
257          rule: 'tests.skipped',
258          file: path,
259          title: `adds a test file with ${plural(skips, 'skipped test')}`,
260          detail: `${addedLines('', after, SKIP).join(' | ')}`,
261          after: addedLines('', after, SKIP),
262        })
263      }
264      if (focus > 0) {
265        found.push({
266          rule: 'tests.focused',
267          file: path,
268          title: 'adds a focused test (.only)',
269          detail: `only the focused tests run, the rest are silently skipped`,
270          after: addedLines('', after, FOCUS),
271        })
272      }
273    }
274
275    return found
276  }
277
278  const was = code(before)
279  const now = code(after ?? '')
280
281  if (after === null) {
282    const cases = count(was, TEST_CASE)
283    found.push({
284      rule: 'tests.removed',
285      file: path,
286      title: `deletes a test file${cases > 0 ? ` (${plural(cases, 'test')})` : ''}`,
287      detail: `the file and its ${plural(count(was, ASSERTION), 'assertion')} go away`,
288    })
289
290    return found
291  }
292
293  const casesBefore = count(was, TEST_CASE)
294  const casesAfter = count(now, TEST_CASE)
295  if (casesAfter < casesBefore) {
296    found.push({
297      rule: 'tests.removed',
298      file: path,
299      title: `removes ${plural(casesBefore - casesAfter, 'test')}`,
300      detail: `${casesBefore} → ${casesAfter} tests`,
301      before: removedLines(before, after, TEST_CASE),
302    })
303  }
304
305  const assertsBefore = count(was, ASSERTION)
306  const assertsAfter = count(now, ASSERTION)
307  if (assertsAfter < assertsBefore) {
308    found.push({
309      rule: 'tests.assertions',
310      file: path,
311      title: `removes ${plural(assertsBefore - assertsAfter, 'assertion')}`,
312      detail: `${assertsBefore} → ${assertsAfter} assertions`,
313      before: removedLines(before, after, ASSERTION),
314    })
315  }
316
317  if (count(now, SKIP) > count(was, SKIP)) {
318    found.push({
319      rule: 'tests.skipped',
320      file: path,
321      title: `skips ${plural(count(now, SKIP) - count(was, SKIP), 'test')}`,
322      detail: `a skipped test still reads as passing`,
323      after: addedLines(before, after, SKIP),
324    })
325  }
326
327  if (count(now, FOCUS) > count(was, FOCUS)) {
328    found.push({
329      rule: 'tests.focused',
330      file: path,
331      title: 'focuses a test (.only)',
332      detail: `only the focused tests run, the rest are silently skipped`,
333      after: addedLines(before, after, FOCUS),
334    })
335  }
336
337  const rewritten = rewrittenValues(code(before), code(after))
338  if (rewritten.length > 0) {
339    found.push({
340      rule: 'tests.expected',
341      file: path,
342      title: `changes the values in ${plural(rewritten.length, 'assertion')}`,
343      detail: 'an expected value moved; if it moved to what the code returns now, the test checks nothing',
344      before: rewritten.slice(0, SAMPLE).map(([was]) => was),
345      after: rewritten.slice(0, SAMPLE).map(([, now]) => now),
346    })
347  }
348
349  const retargeted = retargetedImports(code(before), code(after))
350  if (retargeted.length > 0) {
351    const [[was = '', is = ''] = []] = retargeted
352    found.push({
353      rule: 'tests.retargeted',
354      file: path,
355      title: `points the test at other code (${importTarget(was)?.target ?? '?'} → ${importTarget(is)?.target ?? '?'})`,
356      detail: 'the test now checks a different module than before; a stand-in can pass while the real code stays broken',
357      before: retargeted.slice(0, SAMPLE).map(([line]) => line),
358      after: retargeted.slice(0, SAMPLE).map(([, line]) => line),
359    })
360  }
361
362  const inlined = inlinedStandIns(code(before), code(after))
363  if (inlined.length > 0) {
364    found.push({
365      rule: 'tests.retargeted',
366      file: path,
367      title: `replaces the imported ${inlined.slice(0, 2).join(', ')} with a copy inside the test`,
368      detail: 'the test now checks its own copy of the code; the copy can pass while the real code stays broken',
369      after: addedLines(before, after, new RegExp(`\\b(?:${inlined.join('|')})\\b`)),
370    })
371  }
372
373  const weakUp = count(now, WEAK) - count(was, WEAK)
374  const strongDown = count(was, STRONG) - count(now, STRONG)
375  if (weakUp > 0 && strongDown > 0) {
376    found.push({
377      rule: 'tests.loosened',
378      file: path,
379      title: `loosens ${plural(Math.min(weakUp, strongDown), 'assertion')}`,
380      detail: `exact checks traded for ones that pass on almost anything`,
381      before: removedLines(before, after, STRONG),
382      after: addedLines(before, after, WEAK),
383    })
384  }
385
386  return found
387}
388
389// ---------------------------------------------------------------- suppressions
390
391const SUPPRESSIONS: ReadonlyArray<readonly [string, RegExp]> = [
392  ['@ts-ignore', /@ts-ignore\b/],
393  ['@ts-expect-error', /@ts-expect-error\b/],
394  ['@ts-nocheck', /@ts-nocheck\b/],
395  ['eslint-disable', /eslint-disable(?:-next-line|-line)?\b/],
396  ['biome-ignore', /biome-ignore\b/],
397  ['oxlint-disable', /oxlint-disable\b/],
398  ['# noqa', /#\s*noqa\b/],
399  ['# type: ignore', /#\s*type:\s*ignore\b/],
400  ['pyright: ignore', /pyright:\s*ignore\b/],
401  ['pylint: disable', /pylint:\s*disable\b/],
402  ['mypy: ignore-errors', /mypy:\s*ignore-errors\b/],
403  ['//nolint', /\/\/\s*nolint\b/],
404  ['#[allow(...)]', /#!?\[allow\(/],
405  ['@SuppressWarnings', /@SuppressWarnings\b/],
406  ['rubocop:disable', /rubocop:disable\b/],
407  ['swiftlint:disable', /swiftlint:disable\b/],
408  ['NOLINT', /\bNOLINT(?:NEXTLINE|BEGIN)?\b/],
409  ['@phpstan-ignore', /@phpstan-ignore/],
410  ['@psalm-suppress', /@psalm-suppress\b/],
411  ['#pragma warning disable', /#pragma\s+warning\s+disable\b/],
412  ['coverage ignore', /(?:istanbul|c8|v8)\s+ignore\b|pragma:\s*no\s*cover\b/],
413]
414
415const PROSE = /\.(?:md|mdx|markdown|txt|rst|adoc)$/
416
417function suppressionFindings({ path, before, after }: FileChange): GoalpostFinding[] {
418  if (after === null || PROSE.test(path)) {
419    return []
420  }
421  const added: string[] = []
422  const lines: string[] = []
423  for (const [name, pattern] of SUPPRESSIONS) {
424    const more = count(after, pattern) - count(before ?? '', pattern)
425    if (more > 0) {
426      added.push(more > 1 ? `${more}× ${name}` : name)
427      lines.push(...addedLines(before ?? '', after, pattern))
428    }
429  }
430  if (added.length === 0) {
431    return []
432  }
433
434  return [
435    {
436      rule: 'suppressions',
437      file: path,
438      title: `adds ${added.join(', ')}`,
439      detail: `the check still runs, it just stops reporting here`,
440      after: lines.slice(0, SAMPLE),
441    },
442  ]
443}
444
445// ---------------------------------------------------------------- config
446
447const TS_CONFIG = /(?:^|\/)[jt]sconfig(?:\.[\w-]+)?\.json$/
448const TS_STRICT = [
449  'strict',
450  'noImplicitAny',
451  'strictNullChecks',
452  'strictFunctionTypes',
453  'strictBindCallApply',
454  'strictPropertyInitialization',
455  'noImplicitThis',
456  'useUnknownInCatchVariables',
457  'alwaysStrict',
458  'noUncheckedIndexedAccess',
459  'exactOptionalPropertyTypes',
460  'noImplicitReturns',
461  'noImplicitOverride',
462  'noFallthroughCasesInSwitch',
463  'noUnusedLocals',
464  'noUnusedParameters',
465  'checkJs',
466]
467const COVERED_BY_STRICT = new Set([
468  'noImplicitAny',
469  'strictNullChecks',
470  'strictFunctionTypes',
471  'strictBindCallApply',
472  'strictPropertyInitialization',
473  'noImplicitThis',
474  'useUnknownInCatchVariables',
475  'alwaysStrict',
476])
477
478function flag(text: string, key: string): boolean | undefined {
479  const match = new RegExp(`["']?\\b${key}\\b["']?\\s*[:=]\\s*(true|false|True|False)\\b`).exec(code(text))
480
481  return match ? match[1]?.toLowerCase() === 'true' : undefined
482}
483
484const LINT_CONFIG =
485  /(?:^|\/)(?:\.eslintrc(?:\.\w+)?|eslint\.config\.[cm]?[jt]s|\.oxlintrc\.json|biome\.jsonc?|\.stylelintrc(?:\.\w+)?|stylelint\.config\.[cm]?js|deno\.jsonc?)$/
486const SEVERITY_RANK: Record<string, number> = { error: 2, '2': 2, warn: 1, '1': 1, info: 1, off: 0, '0': 0 }
487
488function severities(text: string): Map<string, number> {
489  const found = new Map<string, number>()
490  const pattern = /['"]([@\w/.-]+)['"]\s*:\s*\[?\s*['"]?(error|warn|info|off|[012])\b/g
491  for (const match of code(text).matchAll(pattern)) {
492    const [, rule = '', level = ''] = match
493    found.set(rule, SEVERITY_RANK[level] ?? 0)
494  }
495
496  return found
497}
498
499const PY_CONFIG = /(?:^|\/)(?:pyproject\.toml|setup\.cfg|tox\.ini|ruff\.toml|\.ruff\.toml|\.flake8|mypy\.ini|\.mypy\.ini|\.pylintrc|pylintrc)$/
500const PY_STRICT = [
501  'strict',
502  'disallow_untyped_defs',
503  'disallow_incomplete_defs',
504  'disallow_any_generics',
505  'check_untyped_defs',
506  'warn_return_any',
507  'no_implicit_optional',
508  'strict_optional',
509  'warn_unused_ignores',
510]
511const PY_LOOSE = ['ignore_errors', 'ignore_missing_imports', 'allow_untyped_defs', 'allow_redefinition']
512
513function ignoredCodes(text: string): number {
514  let total = 0
515  const pattern = /\b(?:extend-ignore|ignore|disable|extend-per-file-ignores|per-file-ignores|ignore-names)\s*=\s*(\[[^\]]*\]|[^\n]*)/g
516  for (const match of code(text).matchAll(pattern)) {
517    total += count(match[1] ?? '', /\b[A-Z]{1,4}\d{1,4}\b|\b[a-z]+(?:-[a-z]+)+\b/g)
518  }
519
520  return total
521}
522
523function configFindings({ path, before, after }: FileChange): GoalpostFinding[] {
524  if (before === null) {
525    return []
526  }
527  const now = after ?? ''
528  const weaker: string[] = []
529
530  if (TS_CONFIG.test(path)) {
531    const strictAfter = flag(now, 'strict') === true
532    for (const key of TS_STRICT) {
533      if (flag(before, key) === true && flag(now, key) !== true && !(strictAfter && COVERED_BY_STRICT.has(key))) {
534        weaker.push(`${key} turned off`)
535      }
536    }
537  }
538
539  if (LINT_CONFIG.test(path)) {
540    const was = severities(before)
541    const is = severities(now)
542    for (const [rule, rank] of was) {
543      const level = is.get(rule)
544      if (rank > 0 && level === undefined) {
545        weaker.push(`${rule} removed`)
546      } else if (level !== undefined && level < rank) {
547        weaker.push(`${rule} ${rank === 2 ? 'error' : 'warn'} → ${level === 1 ? 'warn' : 'off'}`)
548      }
549    }
550  }
551
552  if (PY_CONFIG.test(path)) {
553    for (const key of PY_STRICT) {
554      if (flag(before, key) === true && flag(now, key) !== true) {
555        weaker.push(`${key} turned off`)
556      }
557    }
558    for (const key of PY_LOOSE) {
559      if (flag(before, key) !== true && flag(now, key) === true) {
560        weaker.push(`${key} turned on`)
561      }
562    }
563    const more = ignoredCodes(now) - ignoredCodes(before)
564    if (more > 0) {
565      weaker.push(`${plural(more, 'more ignored rule')}`)
566    }
567  }
568
569  if (after === null && (TS_CONFIG.test(path) || LINT_CONFIG.test(path))) {
570    weaker.push('the whole file deleted')
571  }
572
573  if (weaker.length === 0) {
574    return []
575  }
576
577  return [
578    {
579      rule: 'config',
580      file: path,
581      title: `relaxes ${basename(path)}`,
582      detail: `${weaker.slice(0, 4).join(', ')}${weaker.length > 4 ? `, +${weaker.length - 4} more` : ''}`,
583      after: weaker.slice(0, SAMPLE),
584    },
585  ]
586}
587
588// ---------------------------------------------------------------- CI
589
590const CI_FILE =
591  /(?:^|\/)(?:\.github\/workflows\/[^/]+\.ya?ml|\.gitlab-ci\.ya?ml|\.circleci\/config\.ya?ml|azure-pipelines\.ya?ml|bitbucket-pipelines\.ya?ml|\.buildkite\/[^/]+\.ya?ml|Jenkinsfile|\.travis\.ya?ml|\.pre-commit-config\.ya?ml|lefthook\.ya?ml|\.husky\/[^/]+)$/
592
593const CHECK_COMMAND =
594  /\b(?:(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:test|lint|typecheck|check|build)|vitest|jest|pytest|go\s+(?:test|vet)|cargo\s+(?:test|clippy|check|nextest)|ctest|mocha|playwright\s+test|rspec|phpunit|dotnet\s+test|tsc|mypy|pyright|eslint|ruff|flake8|pylint|biome|golangci-lint|stylelint|prettier\s+--check|pre-commit\s+run|claude\s+plugin\s+(?:test|validate))\b/g
595
596const CI_SOFTENERS: ReadonlyArray<readonly [string, RegExp]> = [
597  ['continue-on-error: true', /continue-on-error:\s*true\b/g],
598  ['allow_failure: true', /allow_failure:\s*true\b/g],
599  ['|| true', /\|\|\s*(?:true|:)\s*(?:$|[;&)])|\|\|\s*exit\s+0\b/gm],
600  ['if: false', /\bif:\s*(?:false|\$\{\{\s*false\s*\}\})/g],
601  ['set +e', /\bset\s+\+e\b/g],
602]
603
604function ciFindings({ path, before, after }: FileChange): GoalpostFinding[] {
605  if (before === null || !CI_FILE.test(path)) {
606    return []
607  }
608  const was = code(before)
609  const now = code(after ?? '')
610  const weaker: string[] = []
611  const lines: string[] = []
612
613  for (const [name, pattern] of CI_SOFTENERS) {
614    const more = count(now, pattern) - count(was, pattern)
615    if (more > 0) {
616      weaker.push(more > 1 ? `${more}× ${name}` : name)
617      lines.push(...addedLines(before, after ?? '', pattern))
618    }
619  }
620  const checksGone = count(was, CHECK_COMMAND) - count(now, CHECK_COMMAND)
621  if (checksGone > 0) {
622    weaker.push(`${plural(checksGone, 'check')} removed`)
623    lines.push(...removedLines(before, after ?? '', CHECK_COMMAND).map(line => `- ${line}`))
624  }
625  const hooksGone = count(was, /^\s*-\s+id:/gm) - count(now, /^\s*-\s+id:/gm)
626  if (hooksGone > 0) {
627    weaker.push(`${plural(hooksGone, 'hook')} removed`)
628  }
629  if (after === null) {
630    weaker.push('the whole file deleted')
631  }
632  if (weaker.length === 0) {
633    return []
634  }
635
636  return [
637    {
638      rule: 'ci',
639      file: path,
640      title: `weakens ${basename(path)}`,
641      detail: `${weaker.join(', ')}`,
642      after: lines.slice(0, SAMPLE),
643    },
644  ]
645}
646
647// ---------------------------------------------------------------- coverage
648
649const COVERAGE_FILE =
650  /(?:^|\/)(?:package\.json|jest\.config\.[\w.]+|vitest\.config\.[\w.]+|vite\.config\.[\w.]+|vitest\.workspace\.[\w.]+|pyproject\.toml|setup\.cfg|tox\.ini|\.coveragerc|pytest\.ini|codecov\.ya?ml|\.codecov\.ya?ml|\.nycrc(?:\.\w+)?|\.c8rc(?:\.\w+)?|karma\.conf\.[cm]?js|\.simplecov|Makefile|build\.gradle(?:\.kts)?|pom\.xml|\.github\/workflows\/[^/]+\.ya?ml|\.gitlab-ci\.ya?ml)$/
651
652const THRESHOLDS: ReadonlyArray<readonly [string, RegExp]> = [
653  ['fail_under', /\bfail_under\s*[=:]\s*(\d+(?:\.\d+)?)/g],
654  ['--cov-fail-under', /--cov-fail-under[= ](\d+(?:\.\d+)?)/g],
655  ['--fail-under', /(?<!cov)--fail-under[= ](\d+(?:\.\d+)?)/g],
656  ['branches', /["']?\bbranches["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
657  ['functions', /["']?\bfunctions["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
658  ['lines', /["']?\blines["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
659  ['statements', /["']?\bstatements["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
660  ['target', /\btarget:\s*(\d+(?:\.\d+)?)%?/g],
661  ['minimum_coverage', /\bminimum_coverage\s*\(?\s*(\d+(?:\.\d+)?)/g],
662  ['minimum', /<minimum>(\d*\.?\d+)<\/minimum>|\bminimum\s*=\s*(?:BigDecimal\s*\(\s*)?["']?(\d*\.?\d+)/g],
663  ['--check-coverage', /--(?:lines|branches|functions|statements)[= ](\d+(?:\.\d+)?)/g],
664]
665
666function thresholds(text: string): Map<string, number[]> {
667  const found = new Map<string, number[]>()
668  for (const [name, pattern] of THRESHOLDS) {
669    const values = [...code(text).matchAll(pattern)].map(match => Number(match[1] ?? match[2]))
670    if (values.length > 0) {
671      found.set(name, values)
672    }
673  }
674
675  return found
676}
677
678function coverageFindings({ path, before, after }: FileChange): GoalpostFinding[] {
679  if (before === null || after === null) {
680    return []
681  }
682  if (!COVERAGE_FILE.test(path) && !/coverage|cov-fail|fail_under|codecov/i.test(before)) {
683    return []
684  }
685  const relevant = /coverage|cov-fail|fail_under|codecov|threshold|minimum|jacoco|simplecov/i
686  if (!relevant.test(before)) {
687    return []
688  }
689  const was = thresholds(before)
690  const is = thresholds(after)
691  const lowered: string[] = []
692  for (const [name, values] of was) {
693    const now = is.get(name) ?? []
694    values.forEach((value, index) => {
695      const later = now[index]
696      if (later === undefined && now.length < values.length) {
697        lowered.push(`${name} ${value} removed`)
698      } else if (later !== undefined && later < value) {
699        lowered.push(`${name} ${value} → ${later}`)
700      }
701    })
702  }
703  if (lowered.length === 0) {
704    return []
705  }
706
707  return [
708    {
709      rule: 'coverage',
710      file: path,
711      title: 'lowers a coverage threshold',
712      detail: `${lowered.slice(0, 4).join(', ')}`,
713      after: lowered.slice(0, SAMPLE),
714    },
715  ]
716}
717
718// ---------------------------------------------------------------- files
719
720/**
721 * Everything a change to one file does to lower the bar, in the categories
722 * that are on.
723 */
724export function inspectChange(change: FileChange, options: DetectOptions): GoalpostFinding[] {
725  if (change.before === change.after) {
726    return []
727  }
728  const all = [
729    ...testFindings(change, options.testPattern),
730    ...suppressionFindings(change),
731    ...configFindings(change),
732    ...ciFindings(change),
733    ...coverageFindings(change),
734  ]
735
736  return all.filter(finding => options.categories.has(CATEGORY_OF[finding.rule]))
737}
738
739/**
740 * The file an Edit leaves, or null when `old` is not in it (the tool will
741 * refuse the edit itself).
742 */
743export function applyEdit(before: string, old: string, replacement: string, isAll = false): string | null {
744  if (old === '' || !before.includes(old)) {
745    return null
746  }
747
748  return isAll ? before.split(old).join(replacement) : before.replace(old, () => replacement)
749}
750
751// ---------------------------------------------------------------- shell
752
753/** The command with every quoted string emptied, so flags inside a message don't count. */
754function withoutQuoted(command: string): string {
755  return command.replace(/'[^']*'|"(?:\\.|[^"\\])*"|\$'[^']*'/g, '""').replace(/<<-?\s*['"]?(\w+)['"]?[\s\S]*?\n\1\b/g, '')
756}
757
758type Segment = { words: string[]; redirects: string[] }
759
760/**
761 * The simple commands of a command line, split at `&&`, `||`, `;`, `|` and
762 * newlines, each as its words (quotes taken off, text kept) and the files it
763 * redirects output to. Heredoc bodies are text, not commands, so they go.
764 */
765function segmentsOf(command: string): Segment[] {
766  const text = command.replace(/<<-?\s*(['"]?)(\w+)\1[^\n]*\n[\s\S]*?\n\s*\2(?=\s|$)/g, '')
767  const tokens = text.match(/'[^']*'|"(?:\\.|[^"\\])*"|&&|\|\||[;|\n]|\d?>>?|[^\s;&|()<>]+/g) ?? []
768  const segments: Segment[] = [{ words: [], redirects: [] }]
769  let isRedirect = false
770  for (const token of tokens) {
771    const current = segments[segments.length - 1] ?? { words: [], redirects: [] }
772    if (['&&', '||', ';', '|', '\n'].includes(token)) {
773      segments.push({ words: [], redirects: [] })
774      isRedirect = false
775    } else if (/^\d?>>?$/.test(token)) {
776      isRedirect = !/^2/.test(token)
777    } else {
778      const word = token.replace(/^['"]|['"]$/g, '')
779      if (isRedirect) {
780        current.redirects.push(word)
781        isRedirect = false
782      } else {
783        current.words.push(word)
784      }
785    }
786  }
787
788  return segments.filter(segment => segment.words.length > 0 || segment.redirects.length > 0)
789}
790
791/** The words of a simple command without leading `VAR=value`, `sudo`, `command`, `env` or `xargs`. */
792function withoutPrefix(words: string[]): string[] {
793  let index = 0
794  while (index < words.length && (/^\w+=/.test(words[index] ?? '') || ['sudo', 'command', 'env', 'xargs', 'nohup', 'time'].includes(words[index] ?? ''))) {
795    index += 1
796  }
797
798  return words.slice(index)
799}
800
801const GIT_VERB = (verb: string): RegExp => new RegExp(`\\bgit\\b(?:\\s+-[cC]\\s+\\S+)*\\s+${verb}\\b[^;&|]*`, 'g')
802
803const SKIP_HOOK_ENV = /(?:^|[\s;&|(])(?:export\s+)?(HUSKY=0|HUSKY_SKIP_HOOKS=1|SKIP=\S+|LEFTHOOK=0|LEFTHOOK_EXCLUDE=\S+|PRE_COMMIT_ALLOW_NO_CONFIG=1|OVERCOMMIT_DISABLE=1)/
804
805const RUNNER = /\b(?:(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test|vitest|jest|pytest|py\.test|go\s+test|cargo\s+(?:test|nextest)|mocha|playwright\s+test|rspec|phpunit|dotnet\s+test)\b/
806
807const NARROWING: ReadonlyArray<readonly [string, RegExp]> = [
808  ['--testPathIgnorePatterns', /--testPathIgnorePatterns\b/],
809  ['--passWithNoTests', /--passWithNoTests\b/],
810  ['--deselect', /--deselect\b/],
811  ['-k "not …"', /\s-k\s+["']?not\b/],
812  ['--ignore', /\s--ignore(?:-glob)?[= ]/],
813  ['--exclude', /\s--exclude[= ]/],
814  ['--skip', /\s--skip\b/],
815  ['-p no:', /\s-p\s+no:/],
816]
817
818const SNAPSHOT_UPDATE = /\b(?:jest|vitest|playwright\s+test|(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test)\b[^;&|]*\s(?:-u|--updateSnapshot|--update-snapshots|--update)\b|--snapshot-update\b|--snapshot-update\b|\binsta\s+accept\b|\bINSTA_UPDATE=always\b|UPDATE_SNAPSHOTS?=(?:1|true)/
819
820const GUARDED_FOR_SHELL = (path: string, testPattern?: RegExp): GoalpostCategory | null => {
821  if (isTestFile(path, testPattern) || SNAPSHOT_FILE.test(path)) {
822    return 'tests'
823  }
824  if (CI_FILE.test(path)) {
825    return 'ci'
826  }
827  if (TS_CONFIG.test(path) || LINT_CONFIG.test(path) || PY_CONFIG.test(path)) {
828    return 'config'
829  }
830
831  return null
832}
833
834/**
835 * Everything a shell command does to lower the bar: skipping git hooks,
836 * deleting tests, narrowing or rewriting what a test run checks, and editing
837 * a guarded file where no diff can be seen.
838 */
839export function inspectCommand(command: string, options: DetectOptions): GoalpostFinding[] {
840  const plain = withoutQuoted(command)
841  const found: GoalpostFinding[] = []
842
843  const bypass: string[] = []
844  for (const verb of ['commit', 'push', 'merge', 'am', 'rebase', 'cherry-pick', 'revert']) {
845    for (const [segment] of plain.matchAll(GIT_VERB(verb))) {
846      if (/\s--no-verify\b/.test(segment)) {
847        bypass.push(`git ${verb} --no-verify`)
848      } else if (verb === 'commit' && /\s-[a-zA-Z]*n[a-zA-Z]*(?=\s|$)/.test(segment)) {
849        bypass.push('git commit -n')
850      }
851    }
852  }
853  const env = SKIP_HOOK_ENV.exec(plain)
854  if (env) {
855    bypass.push(env[1] ?? 'hook switch')
856  }
857  if (/\bgit\b[^;&|]*\bconfig\b[^;&|]*\bcore\.hooksPath\b/.test(plain)) {
858    bypass.push('git config core.hooksPath')
859  }
860  if (bypass.length > 0) {
861    found.push({
862      rule: 'bypass',
863      title: `skips the git hooks (${bypass[0]})`,
864      detail: 'the checks a commit or push runs are switched off for this one',
865      after: [command.slice(0, 200)],
866    })
867  }
868
869  if (RUNNER.test(plain)) {
870    const narrowed = NARROWING.filter(([, pattern]) => pattern.test(command)).map(([name]) => name)
871    if (narrowed.length > 0) {
872      found.push({
873        rule: 'tests.narrowed',
874        title: `runs the tests with ${narrowed.join(', ')}`,
875        detail: 'a narrowed run can pass while the tests it leaves out fail',
876        after: [command.slice(0, 200)],
877      })
878    }
879  }
880  if (SNAPSHOT_UPDATE.test(plain)) {
881    found.push({
882      rule: 'tests.snapshots',
883      title: 'rewrites the snapshots',
884      detail: 'updating snapshots makes every snapshot test agree with the code as it is now',
885      after: [command.slice(0, 200)],
886    })
887  }
888
889  const deleted: string[] = []
890  const edited: Array<readonly [string, GoalpostCategory]> = []
891  for (const segment of segmentsOf(command)) {
892    const [verb = '', ...rest] = withoutPrefix(segment.words)
893    const args = rest.filter(word => !word.startsWith('-'))
894    const isDelete = ['rm', 'unlink', 'trash'].includes(verb) || (verb === 'git' && rest[0] === 'rm')
895    const isInPlace =
896      (['sed', 'gsed'].includes(verb) && rest.some(word => /^-[a-zA-Z]*i/.test(word) || word === '--in-place')) ||
897      (verb === 'perl' && rest.some(word => /^-[a-zA-Z]*i/.test(word)))
898    if (isDelete) {
899      deleted.push(...(verb === 'git' ? args.slice(1) : args).filter(word => isTestFile(word, options.testPattern) || SNAPSHOT_FILE.test(word)))
900    }
901    const targets = [...(isInPlace ? args : []), ...segment.redirects]
902    for (const target of targets) {
903      const category = GUARDED_FOR_SHELL(target, options.testPattern)
904      if (category !== null) {
905        edited.push([target, category] as const)
906      }
907    }
908  }
909  if (deleted.length > 0) {
910    found.push({
911      rule: 'tests.removed',
912      file: deleted[0],
913      title: `deletes ${deleted.length === 1 ? 'a test file' : plural(deleted.length, 'test file')}`,
914      detail: deleted.slice(0, SAMPLE).join(', '),
915      before: deleted.slice(0, SAMPLE),
916    })
917  } else if (edited.length > 0) {
918    const [first] = edited
919    found.push({
920      rule: 'shell-edit',
921      file: first?.[0],
922      title: `edits ${basename(first?.[0] ?? '')} from the shell`,
923      detail: 'a guarded file changed where goalpost-guard cannot see the diff; use Edit so it can',
924      after: [command.slice(0, 200)],
925      category: first?.[1],
926    })
927  }
928
929  return found.filter(finding => options.categories.has(finding.category ?? CATEGORY_OF[finding.rule]))
930}
931
932export function categoryOf(finding: GoalpostFinding): GoalpostCategory {
933  return finding.category ?? CATEGORY_OF[finding.rule]
934}
935
types/index.d.ts 93 lines
1/**
2 * goalpost-guard's contract: what it finds, what it decided, and the values it
3 * keeps in `$.state` for its band slot and its pane.
4 */
5
6/**
7 * The families of change a person can turn on or off, one `userConfig`
8 * switch each.
9 */
10export type GoalpostCategory = 'tests' | 'suppressions' | 'config' | 'ci' | 'coverage' | 'bypass'
11
12/**
13 * Each way of lowering the bar the detectors know.
14 */
15export type GoalpostRule =
16  | 'tests.assertions'
17  | 'tests.removed'
18  | 'tests.skipped'
19  | 'tests.focused'
20  | 'tests.loosened'
21  | 'tests.expected'
22  | 'tests.retargeted'
23  | 'tests.snapshots'
24  | 'tests.narrowed'
25  | 'suppressions'
26  | 'config'
27  | 'ci'
28  | 'coverage'
29  | 'bypass'
30  | 'shell-edit'
31
32/**
33 * One thing a change does to lower the bar. `before` and `after` hold a few
34 * sample lines (removed and added) for the pane; `category` is set only when
35 * it differs from the rule's own (a shell edit takes its file's).
36 */
37export type GoalpostFinding = {
38  rule: GoalpostRule
39  title: string
40  detail: string
41  file?: string
42  before?: string[]
43  after?: string[]
44  category?: GoalpostCategory
45}
46
47/**
48 * What happened to a finding: held and refused, let through by the person
49 * (once, or from now on for that file), let through by an exception made
50 * earlier, or only reported (`warn` mode).
51 */
52export type GoalpostDecision = 'blocked' | 'allowed' | 'excepted' | 'warned'
53
54/**
55 * One finding and its decision, as the pane lists it and a commit trailer
56 * reports it.
57 */
58export type GoalpostEntry = {
59  id: string
60  at: number
61  tool: string
62  finding: GoalpostFinding
63  decision: GoalpostDecision
64  /** Why the person allowed it, or what they told Claude to do instead. */
65  note?: string
66  /** When a `git commit` carried it in a trailer (session clock); an amend of that commit carries it again. */
67  reportedAt?: number
68}
69
70/**
71 * A standing permission: this rule, in this file (or anywhere in the project
72 * when `file` is absent), for this project. Kept in `$.store`, across sessions.
73 */
74export type GoalpostException = {
75  project: string
76  rule: GoalpostRule
77  file?: string
78  note: string
79  at: number
80}
81
82declare module 'claude-code' {
83  interface PluginState {
84    'goalpost-guard': {
85      ledger: GoalpostEntry[]
86      exceptions: GoalpostException[]
87      tab: 'ledger' | 'exceptions' | 'rules'
88      muted: GoalpostCategory[]
89      isPaused: boolean
90    }
91  }
92}
93