Stops Claude from passing a check by lowering the bar: weakened or deleted tests, new suppressions, relaxed config, softened CI, lowered coverage, skipped git…

Stops Claude from making a check pass by lowering the bar. Tested with Claude Code 2.1.287, terminal and desktop.
When a test fails, the shortest way to green is often not to fix the code but to move the goalposts: delete the assertion, skip the test, add @ts-ignore, turn strict off, put continue-on-error on the CI step, drop the coverage threshold, or commit with --no-verify. goalpost-guard reads every edit and shell command before it runs, and when one does that, it stops and asks you. With emcc-core's standard profile and someone at the screen, it lets the change through instead, tells Claude to report it and flags it for witness; strict denies it.
<img src="../../docs/media/goalpost-guard/demo.gif" width="880" alt="Claude Code with Haiku 4.5, told src/ is frozen, rewrites a test's expected value; goalpost-guard holds the edit; the answer goes back to Claude, which fixes src/cart.js instead">
<img src="../../docs/media/goalpost-guard/dialog.png" alt="Goalpost Guard: this change test/cart.test.js changes the values in 1 assertion. Allow it? Block it, Allow once, Allow in this file, Allow in this project, Type something">
Anything you type instead goes to Claude, word for word, as the reason the change was blocked:
<img src="../../docs/media/goalpost-guard/answer.png" alt="The fifth option filled in: Not like that. src isn't really frozen: fix the rounding in withTax and run npm test.">
/plugin marketplace add Xabilimon1/emcc-claude-code-mods
/plugin install goalpost-guard@emcc
Works on its own. With emcc-core installed it follows the EMCC profile, posts its count to the EMCC band and flags each lowered bar for witness.
| Family | Caught when a change… | ||
|---|---|---|---|
| Tests | removes or comments out assertions; rewrites the value an assertion expects (24.19 to 24); points the test at a stand-in module while the real one is still there (../src/cart.js to ../lib/cart.js; an import that follows a moved file is fine); swaps an imported function for a copy written inside the test; removes test cases or deletes a test file (also with rm or git rm); adds .skip, xit, @pytest.mark.skip, xfail, t.Skip(), #[ignore], @Disabled; adds .only; trades exact checks for loose ones (toBe(3) for toBeDefined()); rewrites a snapshot by hand or runs the tests with -u; runs them with --testPathIgnorePatterns, -k "not …", --deselect, --passWithNoTests | ||
| Suppressions | adds @ts-ignore, @ts-expect-error, @ts-nocheck, eslint-disable, biome-ignore, # noqa, # type: ignore, pyright: ignore, pylint: disable, //nolint, #[allow(…)], @SuppressWarnings, rubocop:disable, NOLINT, istanbul/c8 ignore, pragma: no cover and more | ||
| Config | turns off a strict flag in a tsconfig (one strict still covers is fine); lowers or removes an ESLint, Biome or Stylelint rule; turns mypy's strict off or ignore_errors on; grows a ruff, flake8 or pylint ignore list | ||
| CI | adds continue-on-error: true, allow_failure: true, `\ | \ | true, if: false or set +e`; removes a step that runs tests, lint, types or a build; removes a pre-commit hook. GitHub Actions, GitLab, CircleCI, Azure, Bitbucket, Buildkite, Jenkins, Travis, pre-commit, lefthook, husky |
| Coverage | lowers or removes a threshold: jest coverageThreshold, vitest thresholds, coverage.py fail_under, --cov-fail-under, codecov target, nyc and c8, SimpleCov, JaCoCo | ||
| Git hooks | git commit or push --no-verify, git commit -n, HUSKY=0, SKIP=…, LEFTHOOK=0, git config core.hooksPath |
It also holds sed -i, perl -pi and > redirects into a test, config or CI file: it cannot see that diff, so it asks Claude to use Edit instead.
Adding tests, assertions, checks and thresholds is never a finding, and neither is writing about @ts-ignore in a Markdown file.
| Mode | What it does |
|---|---|
inherit (default) | Follows the EMCC profile: standard lets it through and reports it (asks when there is no screen, so headless applies), strict denies, off is off. Without emcc-core, asks |
ask | The dialog above |
deny | Refuses the change and tells Claude to fix the code instead |
warn | Lets it through, records it, and tells Claude to report it in its summary |
off | Nothing |
With emcc-core, every change that goes through lowering the bar also posts a flag that witness shows on the step; held changes do not.
In claude -p and other runs with no screen, nobody can answer the dialog, so ask refuses (set headless to allow to let it through and report it instead). If goalpost-guard cannot read the file it is checking, it holds the change rather than let it through.
Exceptions last across sessions and are scoped to the project: "Allow in this file" covers that rule in that file, "Allow in this project" covers that rule everywhere in the project. Revoke them in the ledger.
/goalposts/goalposts opens a pane with three tabs (keys 1, 2, 3; p pauses, Esc closes):
<table> <tr> <td width="34%" valign="top"><img src="../../docs/media/goalpost-guard/ledger.png" alt="Ledger tab: an allowed change with the old and new assertion, and a held one with a button to allow it in the file"></td> <td width="33%" valign="top"><img src="../../docs/media/goalpost-guard/exceptions.png" alt="Exceptions tab: one standing exception with a Revoke button"></td> <td width="33%" valign="top"><img src="../../docs/media/goalpost-guard/rules.png" alt="Rules tab: six families, each on, with a Mute button"></td> </tr> </table>
-) or added (+). A held change has an Allow in this file button./config turns it off for good (a family off there shows /config instead of the button).Pause turns goalpost-guard off for the session, like /goalposts pause (/goalposts resume brings it back). Without emcc-core, a line above the prompt counts what was held and let through, with a Ledger button.
Whatever was let through ends up in the next commit Claude makes, as a git trailer, so a reviewer sees it in git log:
<img src="../../docs/media/goalpost-guard/commit.png" alt="git log -1: Round tax to whole euros, Goalpost-Change: test/cart.test.js: changes the values in 1 assertion (allowed by the user: from now on in this file)">
Each change is reported once; a git commit --amend of that commit carries its trailers again, since an amend rewrites the message. A commit that fails keeps them for the next. Needs git 2.32 or later; turn it off with commitTrailer.
/goalposts audit [ref]Checks every change since ref (default HEAD), untracked files included, with the same rules:
<img src="../../docs/media/goalpost-guard/audit.png" alt="/goalposts audit main: 3 changes lower the bar since main: continue-on-error in ci.yml, an eslint-disable in src/legacy.js, a skipped test in test/cart.test.js">
Set them in /config, or at install with --config key=value.
| Option | Default | What it does |
|---|---|---|
mode | inherit | inherit, ask, deny, warn or off, as above |
tests, suppressions, config, ci, coverage, bypass | on | Each family of checks |
headless | deny | With no one to ask: deny, or allow and report |
commitTrailer | on | The Goalpost-Change trailers |
testPattern | none | A regular expression for test files with unusual names, on top of *.test.ts, test_*.py, *_test.go, *Test.java, *_spec.rb, tests/… |
ignorePaths | none | Globs to leave alone, such as vendor/** |
session.start, command.run (goalposts), tool.call (Edit, Write, Bash, before they run), ui.render (AbovePrompt, the goalposts pane)$.fs.read and $.fs.exists (the file a change touches), $.process.run (git diff, git show and git ls-files, only for /goalposts audit), $.ui.ask, $.ui.open, $.ui.close, $.store (exceptions), $.state, $.session.cwd, $.session.surfaces, $.clock.now, $.command.register, $.emcc.profile, $.emcc.badge and $.emcc.flag (when emcc-core is installed)--trailer flags to a git commit command. It never changes an edit.expect and adds another is fine; one that drops two and adds one is asked about.NotebookEdit is not checked yet./goalposts audit.claude plugin test mods/goalpost-guard
81 tests: the detectors (37), the gate, modes, options, exceptions, retargeted imports, trailers and audit (30), and the band and the pane on both terminal and desktop, plus the EMCC profile, badge and flags (14). Each one was seen failing before it passed, by breaking the code it checks.
hooks/register.tsx 797 lines1import { atom, read, update } from 'claude-code'
2import type { EngineInterface, Register } from 'claude-code'
3
4import type {
5 GoalpostCategory,
6 GoalpostDecision,
7 GoalpostEntry,
8 GoalpostException,
9 GoalpostFinding,
10} from '../types'
11import { applyEdit, categoryOf, importTarget, inspectChange, inspectCommand } from './detect'
12import type { DetectOptions } from './detect'
13
14const PANE = 'goalposts'
15const KEEP = 100
16const STORE_KEY = 'exceptions'
17const TRAILERS = 10
18
19// Anthropic's palette: clay for what was held, olive for what was let through,
20// sky for standing exceptions, stone for everything secondary.
21const CLAY = '#d97757'
22const OLIVE = '#788c5d'
23const SKY = '#6a9bcc'
24const STONE = '#b0aea5'
25
26const ledger = atom({ plugin: 'goalpost-guard', key: 'ledger' } as const, [])
27const exceptions = atom({ plugin: 'goalpost-guard', key: 'exceptions' } as const, [])
28const tab = atom({ plugin: 'goalpost-guard', key: 'tab' } as const, 'ledger')
29const muted = atom({ plugin: 'goalpost-guard', key: 'muted' } as const, [])
30const isPaused = atom({ plugin: 'goalpost-guard', key: 'isPaused' } as const, false)
31
32type Mode = 'ask' | 'deny' | 'warn' | 'off'
33type Configured = Mode | 'inherit'
34
35const MODES: readonly Configured[] = ['inherit', 'ask', 'deny', 'warn', 'off']
36const CATEGORIES: readonly GoalpostCategory[] = ['tests', 'suppressions', 'config', 'ci', 'coverage', 'bypass']
37
38const CATEGORY_LABEL: Record<GoalpostCategory, string> = {
39 tests: 'Tests: assertions, skips, .only, deleted tests, snapshots',
40 suppressions: 'Suppressions: @ts-ignore, eslint-disable, noqa…',
41 config: 'Config: tsconfig strictness, lint rules, mypy, ruff',
42 ci: 'CI: continue-on-error, || true, removed checks and hooks',
43 coverage: 'Coverage thresholds',
44 bypass: 'Skipped git hooks: --no-verify, HUSKY=0, SKIP=',
45}
46
47const DECISION_LOOK: Record<GoalpostDecision, { label: string; color: string }> = {
48 blocked: { label: 'HELD', color: CLAY },
49 allowed: { label: 'ALLOWED', color: OLIVE },
50 excepted: { label: 'EXCEPTION', color: SKY },
51 warned: { label: 'NOTED', color: STONE },
52}
53
54const ANSWER = {
55 block: 'Block it',
56 once: 'Allow once',
57 file: 'Allow in this file',
58 project: 'Allow in this project',
59} as const
60
61type Settings = {
62 mode: Configured
63 categories: ReadonlySet<GoalpostCategory>
64 testPattern?: RegExp
65 ignore: readonly RegExp[]
66 headless: 'deny' | 'allow'
67 isTrailerOn: boolean
68}
69
70type Verdict = { kind: 'pass' } | { kind: 'warn'; note: string } | { kind: 'deny'; reason: string }
71
72// ---------------------------------------------------------------- settings
73
74function settingsOf(options: Record<string, unknown>): Settings {
75 const mode = MODES.find(one => one === options.mode) ?? 'inherit'
76 const categories = new Set(CATEGORIES.filter(category => options[category] !== false))
77 let testPattern: RegExp | undefined
78 if (typeof options.testPattern === 'string' && options.testPattern !== '') {
79 try {
80 testPattern = new RegExp(options.testPattern)
81 } catch {
82 testPattern = undefined
83 }
84 }
85 const globs = Array.isArray(options.ignorePaths) ? options.ignorePaths : []
86
87 return {
88 mode,
89 categories,
90 testPattern,
91 ignore: globs.filter((glob): glob is string => typeof glob === 'string' && glob !== '').map(globToRegExp),
92 headless: options.headless === 'allow' ? 'allow' : 'deny',
93 isTrailerOn: options.commitTrailer !== false,
94 }
95}
96
97// One pass, longest token first, so `**/` never reads as `*` twice and no placeholder is needed.
98const GLOB_TOKENS: Record<string, string> = { '**/': '(?:.*/)?', '**': '.*', '*': '[^/]*', '?': '[^/]' }
99
100function globToRegExp(glob: string): RegExp {
101 const body = glob
102 .replace(/^\.\//, '')
103 .replace(/\/+$/, '')
104 .replace(/\*\*\/|\*\*|\*|\?|[.+^${}()|[\]\\]/g, token => GLOB_TOKENS[token] ?? `\\${token}`)
105
106 return new RegExp(`(?:^|/)${body}(?:/.*)?$`)
107}
108
109function relative(path: string, cwd: string): string {
110 return path.startsWith(`${cwd}/`) ? path.slice(cwd.length + 1) : path
111}
112
113async function detectOptions($: EngineInterface, settings: Settings): Promise<DetectOptions> {
114 const off = new Set(await read($, muted))
115
116 return {
117 categories: new Set([...settings.categories].filter(category => !off.has(category))),
118 testPattern: settings.testPattern,
119 }
120}
121
122/**
123 * The EMCC profile, or null when emcc-core is not installed. `profile()` is
124 * `off` while EMCC is halted, so the kill switch reaches this mod too.
125 */
126async function emccProfile($: EngineInterface): Promise<'standard' | 'strict' | 'off' | null> {
127 try {
128 return await $.emcc.profile()
129 } catch {
130 return null
131 }
132}
133
134async function modeOf($: EngineInterface, configured: Configured): Promise<Mode> {
135 if (await read($, isPaused)) {
136 return 'off'
137 }
138 const profile = await emccProfile($)
139 if (profile === 'off') {
140 return 'off'
141 }
142 if (configured !== 'inherit') {
143 return configured
144 }
145
146 if (profile === 'strict') {
147 return 'deny'
148 }
149
150 // standard reports instead of asking when someone can read the report: witness shows the flag on the step
151 return profile === 'standard' && (await $.session.surfaces()).length > 0 ? 'warn' : 'ask'
152}
153
154// ---------------------------------------------------------------- ledger, badge, exceptions
155
156function summary(findings: readonly GoalpostFinding[], cwd: string): string {
157 return findings.map(finding => `${finding.file ? `${relative(finding.file, cwd)} ` : ''}${finding.title}`).join('; ')
158}
159
160async function record(
161 $: EngineInterface,
162 tool: string,
163 id: string,
164 findings: readonly GoalpostFinding[],
165 decision: GoalpostDecision,
166 note?: string,
167): Promise<void> {
168 if (findings.length === 0) {
169 return
170 }
171 const at = await $.clock.now()
172 const entries: GoalpostEntry[] = findings.map((finding, index) => ({
173 id: `${id}:${index}`,
174 at,
175 tool,
176 finding,
177 decision,
178 ...(note === undefined ? {} : { note }),
179 }))
180 await update($, ledger, list => [...list, ...entries].slice(-KEEP))
181 await refreshBadge($)
182 // Only a bar that was lowered is flagged: a held change lowered nothing, and the ledger already shows it.
183 if (decision === 'blocked') {
184 return
185 }
186 for (const finding of findings) {
187 try {
188 await $.emcc.flag({
189 source: 'goalpost-guard',
190 rule: finding.rule,
191 ...(finding.file === undefined ? {} : { file: finding.file }),
192 text: `${finding.title} (${TRAILER_WORDS[decision]})`,
193 })
194 } catch {
195 // emcc-core is not installed, or is older than 0.2.0: nothing to tell.
196 }
197 }
198}
199
200async function refreshBadge($: EngineInterface): Promise<void> {
201 const list = await read($, ledger)
202 const held = list.filter(entry => entry.decision === 'blocked').length
203 const through = list.filter(entry => entry.decision !== 'blocked').length
204 const parts = [held > 0 ? `${held} held` : '', through > 0 ? `${through} let through` : ''].filter(Boolean)
205 try {
206 await $.emcc.badge({
207 id: 'goalpost-guard',
208 text: parts.length > 0 ? `goalposts: ${parts.join(', ')}` : undefined,
209 tone: through > 0 ? 'warn' : 'good',
210 })
211 } catch {
212 // emcc-core is not installed: this mod draws its own line instead.
213 }
214}
215
216function isCovered(exception: GoalpostException, finding: GoalpostFinding, project: string): boolean {
217 return (
218 exception.project === project &&
219 exception.rule === finding.rule &&
220 (exception.file === undefined || exception.file === finding.file)
221 )
222}
223
224async function storedExceptions($: EngineInterface): Promise<GoalpostException[]> {
225 const value = await $.store.get(STORE_KEY)
226
227 return Array.isArray(value) ? (value as GoalpostException[]) : []
228}
229
230async function addExceptions(
231 $: EngineInterface,
232 findings: readonly GoalpostFinding[],
233 scope: 'file' | 'project',
234 note: string,
235): Promise<void> {
236 const project = await $.session.cwd()
237 const at = await $.clock.now()
238 const added: GoalpostException[] = findings.map(finding => ({
239 project,
240 rule: finding.rule,
241 ...(scope === 'file' && finding.file !== undefined ? { file: finding.file } : {}),
242 note,
243 at,
244 }))
245 const kept = (await storedExceptions($)).filter(old => !added.some(one => sameException(one, old)))
246 await $.store.set(STORE_KEY, [...kept, ...added])
247 await update($, exceptions, () => [...kept, ...added].filter(one => one.project === project))
248}
249
250function sameException(a: GoalpostException, b: GoalpostException): boolean {
251 return a.project === b.project && a.rule === b.rule && a.file === b.file
252}
253
254async function revokeException($: EngineInterface, gone: GoalpostException): Promise<void> {
255 const kept = (await storedExceptions($)).filter(one => !sameException(one, gone))
256 await $.store.set(STORE_KEY, kept)
257 await update($, exceptions, list => list.filter(one => !sameException(one, gone)))
258}
259
260// ---------------------------------------------------------------- the gate
261
262async function decide(
263 $: EngineInterface,
264 tool: string,
265 id: string,
266 found: readonly GoalpostFinding[],
267 settings: Settings,
268 mode: Mode,
269): Promise<Verdict> {
270 const cwd = await $.session.cwd()
271 const watched = found.filter(
272 finding => finding.file === undefined || !settings.ignore.some(glob => glob.test(relative(finding.file ?? '', cwd))),
273 )
274 const standing = await storedExceptions($)
275 const excepted = watched.filter(finding => standing.some(one => isCovered(one, finding, cwd)))
276 const open = watched.filter(finding => !excepted.includes(finding))
277 await record($, tool, id, excepted, 'excepted')
278 if (open.length === 0) {
279 return { kind: 'pass' }
280 }
281 const what = summary(open, cwd)
282
283 if (mode === 'warn') {
284 await record($, tool, id, open, 'warned')
285
286 return { kind: 'warn', note: `goalpost-guard noted that this change ${what}. Say so to the user in your summary.` }
287 }
288 if (mode === 'deny') {
289 await record($, tool, id, open, 'blocked')
290
291 return {
292 kind: 'deny',
293 reason:
294 `goalpost-guard refused this change: it ${what}. ` +
295 'This session does not let a check pass by lowering the bar. Fix the code instead, or ask the user to allow it.',
296 }
297 }
298 if ((await $.session.surfaces()).length === 0) {
299 if (settings.headless === 'allow') {
300 await record($, tool, id, open, 'warned', 'no one could be asked')
301
302 return { kind: 'warn', note: `goalpost-guard let this through unasked (no screen): it ${what}. Report it.` }
303 }
304 await record($, tool, id, open, 'blocked', 'no one could be asked')
305
306 return {
307 kind: 'deny',
308 reason: `goalpost-guard held this change: it ${what}, and no one is here to allow it. Make the check pass without lowering the bar.`,
309 }
310 }
311
312 const answer = await $.ui
313 .ask(`Goalpost Guard: this ${tool === 'Bash' ? 'command' : 'change'} ${what}. Allow it?`, {
314 header: 'Goalpost',
315 options: [ANSWER.block, ANSWER.once, ANSWER.file, ANSWER.project],
316 })
317 .catch(() => ANSWER.block)
318
319 if (answer === ANSWER.once) {
320 await record($, tool, id, open, 'allowed')
321
322 return { kind: 'pass' }
323 }
324 if (answer === ANSWER.file || answer === ANSWER.project) {
325 await addExceptions($, open, answer === ANSWER.file ? 'file' : 'project', `allowed from the dialog: ${what}`)
326 await record($, tool, id, open, 'allowed', answer === ANSWER.file ? 'from now on in this file' : 'from now on in this project')
327
328 return { kind: 'pass' }
329 }
330 const said = answer === ANSWER.block ? undefined : answer.trim()
331 await record($, tool, id, open, 'blocked', said)
332
333 return {
334 kind: 'deny',
335 reason:
336 `goalpost-guard held this change: it ${what}. The user blocked it. ` +
337 (said ? `The user says: "${said}"` : 'Make the check pass without lowering the bar, or explain why the bar has to move.'),
338 }
339}
340
341function normalized(path: string): string {
342 const parts: string[] = []
343 for (const part of path.split('/')) {
344 if (part === '..') {
345 parts.pop()
346 } else if (part !== '.' && part !== '') {
347 parts.push(part)
348 }
349 }
350
351 return `/${parts.join('/')}`
352}
353
354/** Where a module an import names could be on disk, relative to the file importing it. */
355function candidatesFor(file: string, target: string, cwd: string): string[] | null {
356 const dir = file.slice(0, file.lastIndexOf('/')) || cwd
357 if (target.startsWith('.') && target.includes('/')) {
358 const base = normalized(`${dir.startsWith('/') ? dir : `${cwd}/${dir}`}/${target}`)
359
360 return ['', '.js', '.ts', '.tsx', '.jsx', '.mjs', '.cjs', '/index.js', '/index.ts'].map(ext => `${base}${ext}`)
361 }
362 if (/^\.+\w/.test(target) || /^\w+(?:\.\w+)+$/.test(target)) {
363 const ups = target.match(/^\.*/)?.[0].length ?? 0
364 const root = ups === 0 ? cwd : normalized(`${dir.startsWith('/') ? dir : `${cwd}/${dir}`}${'/..'.repeat(Math.max(0, ups - 1))}`)
365 const base = `${root}/${target.slice(ups).replace(/\./g, '/')}`
366
367 return [`${base}.py`, `${base}/__init__.py`]
368 }
369
370 return null
371}
372
373/**
374 * Keeps a retargeted import only when the module the test used to load is
375 * still on disk: then the test was pointed away from real code. When it is
376 * gone, the code moved and the import followed it.
377 */
378async function withRealRetargets($: EngineInterface, found: GoalpostFinding[]): Promise<GoalpostFinding[]> {
379 const cwd = await $.session.cwd()
380 const kept: GoalpostFinding[] = []
381 for (const finding of found) {
382 const old = finding.rule === 'tests.retargeted' ? importTarget(finding.before?.[0] ?? '')?.target : undefined
383 const places = old === undefined || finding.file === undefined ? null : candidatesFor(finding.file, old, cwd)
384 if (places === null) {
385 kept.push(finding)
386 continue
387 }
388 let isThere = false
389 for (const place of places) {
390 if (await $.fs.exists(place)) {
391 isThere = true
392 break
393 }
394 }
395 if (isThere) {
396 kept.push(finding)
397 }
398 }
399
400 return kept
401}
402
403async function fileFindings(
404 $: EngineInterface,
405 path: string,
406 toAfter: (before: string | null) => string | null | undefined,
407 options: DetectOptions,
408): Promise<GoalpostFinding[]> {
409 const before = (await $.fs.exists(path)) ? await $.fs.read(path) : null
410 const after = toAfter(before)
411 if (after === undefined) {
412 return []
413 }
414
415 return withRealRetargets($, inspectChange({ path, before, after }, options))
416}
417
418function heldForError(error: unknown): Verdict {
419 const why = error instanceof Error ? error.message : String(error)
420
421 return {
422 kind: 'deny',
423 reason: `goalpost-guard could not check this change (${why.slice(0, 200)}), so it held it. The user can run /goalposts pause to let changes through.`,
424 }
425}
426
427// ---------------------------------------------------------------- commit trailers
428
429const COMMIT = /\bgit\b(?:\s+-[cC]\s+\S+)*\s+commit\b/g
430const AMEND = /\bgit\b(?:\s+-[cC]\s+\S+)*\s+commit\b[^;&|]*\s--amend\b/
431
432function quoted(text: string): string {
433 return `'${text.replace(/'/g, `'\\''`)}'`
434}
435
436const TRAILER_WORDS: Record<GoalpostDecision, string> = {
437 blocked: 'held',
438 allowed: 'allowed by the user',
439 excepted: 'standing exception',
440 warned: 'let through and reported',
441}
442
443function trailerOf(entry: GoalpostEntry, cwd: string): string {
444 const where = entry.finding.file ? `${relative(entry.finding.file, cwd)}: ` : ''
445 const why = entry.note ? `: ${entry.note}` : ''
446
447 return `Goalpost-Change: ${where}${entry.finding.title} (${TRAILER_WORDS[entry.decision]}${why})`.replace(/\s+/g, ' ').slice(0, 240)
448}
449
450/** The command with `--trailer` lines after its first `git commit` outside quotes, or null without one. */
451function withTrailers(command: string, entries: readonly GoalpostEntry[], cwd: string): string | null {
452 const quotedSpans = [...command.matchAll(/'[^']*'|"(?:\\.|[^"\\])*"/g)].map(match => [match.index, match.index + match[0].length])
453 for (const match of command.matchAll(COMMIT)) {
454 const at = match.index
455 if (quotedSpans.some(([from = 0, to = 0]) => at > from && at < to)) {
456 continue
457 }
458 const end = at + match[0].length
459 const flags = entries.map(entry => ` --trailer ${quoted(trailerOf(entry, cwd))}`).join('')
460
461 return command.slice(0, end) + flags + command.slice(end)
462 }
463
464 return null
465}
466
467// ---------------------------------------------------------------- audit
468
469async function audit($: EngineInterface, base: string, options: DetectOptions): Promise<string> {
470 const listed = await $.process.run(['git', 'diff', '--name-status', '-M', base])
471 if (listed.exitCode !== 0) {
472 return `git diff ${base} failed: ${listed.stderr.trim().slice(0, 200)}`
473 }
474 const untracked = await $.process.run(['git', 'ls-files', '--others', '--exclude-standard'])
475 const rows = listed.stdout
476 .split('\n')
477 .filter(Boolean)
478 .map(line => line.split('\t'))
479 for (const path of untracked.stdout.split('\n').filter(Boolean)) {
480 rows.push(['A', path])
481 }
482 const cwd = await $.session.cwd()
483 const found: GoalpostFinding[] = []
484 for (const [status = '', first = '', second] of rows.slice(0, 300)) {
485 const path = `${cwd}/${second ?? first}`
486 const shown = await $.process.run(['git', 'show', `${base}:${first}`])
487 const before = status.startsWith('A') || shown.exitCode !== 0 ? null : shown.stdout
488 const after = status.startsWith('D') || !(await $.fs.exists(path)) ? null : await $.fs.read(path)
489 found.push(...(await withRealRetargets($, inspectChange({ path, before, after }, options))))
490 }
491 if (found.length === 0) {
492 return `Nothing in the changes since ${base} lowers the bar (${rows.length} files checked).`
493 }
494
495 return [
496 `${found.length} ${found.length === 1 ? 'change lowers' : 'changes lower'} the bar since ${base}:`,
497 ...found.map(finding => `- ${relative(finding.file ?? '', cwd)} ${finding.title}: ${finding.detail}`),
498 ].join('\n')
499}
500
501// ---------------------------------------------------------------- register
502
503export const register: Register = (on, options) => {
504 const settings = settingsOf(options as Record<string, unknown>)
505
506 on('session.start', async ($, e, next) => {
507 await $.command.register({
508 name: 'goalposts',
509 description: 'Goalpost Guard: open the ledger, or "audit [ref]", "pause", "resume"',
510 })
511 const project = await $.session.cwd()
512 const standing = (await storedExceptions($)).filter(one => one.project === project)
513 await update($, exceptions, () => standing)
514
515 return next(e)
516 })
517
518 on('command.run', { command: 'goalposts' }, async ($, e) => {
519 const [verb = '', ...rest] = e.args.trim().split(/\s+/)
520 if (verb === 'pause' || verb === 'resume') {
521 await update($, isPaused, () => verb === 'pause')
522
523 return { text: verb === 'pause' ? 'Goalpost Guard paused for this session.' : 'Goalpost Guard is on again.' }
524 }
525 if (verb === 'audit') {
526 return { text: await audit($, rest[0] ?? 'HEAD', await detectOptions($, settings)) }
527 }
528 const opened = await $.ui.open({ id: PANE, title: 'Goalpost Guard', focus: true, closeOnEscape: true })
529 if (!opened.isPlaced) {
530 const list = await read($, ledger)
531
532 return {
533 text:
534 list.length === 0
535 ? 'Goalpost Guard has not held anything this session.'
536 : list.map(entry => `${DECISION_LOOK[entry.decision].label} ${entry.finding.title} ${entry.finding.file ?? ''}`).join('\n'),
537 }
538 }
539
540 return { text: 'Goalpost Guard ledger opened.' }
541 })
542
543 on('tool.call', { tool: 'Edit' }, async ($, e, next) => {
544 const mode = await modeOf($, settings.mode)
545 if (mode === 'off') {
546 return next(e)
547 }
548 let verdict: Verdict
549 try {
550 const found = await fileFindings(
551 $,
552 e.file_path,
553 before => (before === null ? undefined : (applyEdit(before, e.old_string, e.new_string, e.replace_all === true) ?? undefined)),
554 await detectOptions($, settings),
555 )
556 verdict = await decide($, 'Edit', e.tool_use_id, found, settings, mode)
557 } catch (error) {
558 verdict = heldForError(error)
559 }
560 if (verdict.kind === 'deny') {
561 return { deny: verdict.reason }
562 }
563 const ran = await next(e)
564 if (verdict.kind === 'warn' && ran.deny === undefined) {
565 return { ...ran, context: [...(ran.context ?? []), verdict.note] }
566 }
567
568 return ran
569 })
570
571 on('tool.call', { tool: 'Write' }, async ($, e, next) => {
572 const mode = await modeOf($, settings.mode)
573 if (mode === 'off') {
574 return next(e)
575 }
576 let verdict: Verdict
577 try {
578 const found = await fileFindings($, e.file_path, () => e.content, await detectOptions($, settings))
579 verdict = await decide($, 'Write', e.tool_use_id, found, settings, mode)
580 } catch (error) {
581 verdict = heldForError(error)
582 }
583 if (verdict.kind === 'deny') {
584 return { deny: verdict.reason }
585 }
586 const ran = await next(e)
587 if (verdict.kind === 'warn' && ran.deny === undefined) {
588 return { ...ran, context: [...(ran.context ?? []), verdict.note] }
589 }
590
591 return ran
592 })
593
594 on('tool.call', { tool: 'Bash' }, async ($, e, next) => {
595 const mode = await modeOf($, settings.mode)
596 if (mode === 'off') {
597 return next(e)
598 }
599 let verdict: Verdict
600 try {
601 const found = inspectCommand(e.command, await detectOptions($, settings))
602 verdict = await decide($, 'Bash', e.tool_use_id, found, settings, mode)
603 } catch (error) {
604 verdict = heldForError(error)
605 }
606 if (verdict.kind === 'deny') {
607 return { deny: verdict.reason }
608 }
609
610 const cwd = await $.session.cwd()
611 const listed = settings.isTrailerOn ? await read($, ledger) : []
612 const pending = listed.filter(entry => entry.decision !== 'blocked' && entry.reportedAt === undefined)
613 // An amend rewrites the last commit's message: carry that commit's trailers over to it.
614 const lastAt = Math.max(-1, ...listed.map(entry => entry.reportedAt ?? -1))
615 const again = AMEND.test(e.command) && lastAt >= 0 ? listed.filter(entry => entry.reportedAt === lastAt) : []
616 const reported = [...again, ...pending].slice(-TRAILERS)
617 const rewritten = reported.length > 0 ? withTrailers(e.command, reported, cwd) : null
618 const ran = await next(rewritten === null ? e : { ...e, command: rewritten })
619 if (rewritten !== null && ran.deny === undefined && ran.isError !== true) {
620 const done = new Set(reported.map(entry => entry.id))
621 const at = await $.clock.now()
622 await update($, ledger, list => list.map(entry => (done.has(entry.id) ? { ...entry, reportedAt: at } : entry)))
623 }
624 if (verdict.kind === 'warn' && ran.deny === undefined) {
625 return { ...ran, context: [...(ran.context ?? []), verdict.note] }
626 }
627
628 return ran
629 })
630
631 on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => {
632 if (e.props.hasSurvey || (await emccProfile($)) !== null) {
633 return next(e)
634 }
635 const list = await read($, ledger)
636 const paused = await read($, isPaused)
637 if (list.length === 0 && !paused) {
638 return next(e)
639 }
640 const held = list.filter(entry => entry.decision === 'blocked').length
641 const through = list.length - held
642 const { Box, Button, Text } = $.ui.resolve(e)
643
644 return (
645 <Box>
646 <Text color={CLAY} bold>
647 Goalpost Guard{' '}
648 </Text>
649 {paused && <Text color={STONE}>paused </Text>}
650 {held > 0 && <Text color={CLAY}>{held} held </Text>}
651 {through > 0 && <Text color={OLIVE}>{through} let through </Text>}
652 <Button key="ledger" label="Ledger" onPress={() => $.ui.open({ id: PANE, title: 'Goalpost Guard', focus: true, closeOnEscape: true })} />
653 </Box>
654 )
655 })
656
657 on('ui.render', { component: 'Pane', requestId: PANE }, async ($, e) => {
658 const { Box, Button, Text } = $.ui.resolve(e)
659 const shown = await read($, tab)
660 const list = await read($, ledger)
661 const standing = await read($, exceptions)
662 const off = await read($, muted)
663 const paused = await read($, isPaused)
664 const mode = await modeOf($, settings.mode)
665 const cwd = await $.session.cwd()
666 const held = list.filter(entry => entry.decision === 'blocked').length
667 const room = Math.max(3, Math.floor(((e.viewport?.rows ?? 24) - 6) / 4))
668
669 const tabs = (
670 <Box gap={1}>
671 <Button key="tab-ledger" hotkey="1" label={`Ledger ${list.length}`} variant={shown === 'ledger' ? 'primary' : undefined} onPress={() => update($, tab, () => 'ledger')} />
672 <Button key="tab-exceptions" hotkey="2" label={`Exceptions ${standing.length}`} variant={shown === 'exceptions' ? 'primary' : undefined} onPress={() => update($, tab, () => 'exceptions')} />
673 <Button key="tab-rules" hotkey="3" label="Rules" variant={shown === 'rules' ? 'primary' : undefined} onPress={() => update($, tab, () => 'rules')} />
674 <Button key="pause" hotkey="p" label={paused ? 'Resume' : 'Pause'} onPress={() => update($, isPaused, value => !value)} />
675 </Box>
676 )
677 const status = (
678 <Text>
679 <Text color={STONE}>mode </Text>
680 <Text color={mode === 'off' ? STONE : CLAY} bold>
681 {mode}
682 </Text>
683 <Text color={STONE}>
684 {settings.mode === 'inherit' && mode !== 'off' ? ' (inherited)' : ''}
685 {' '}
686 {held} held, {list.length - held} let through
687 </Text>
688 </Text>
689 )
690
691 if (shown === 'exceptions') {
692 return (
693 <Box flexDirection="column" gap={1}>
694 {tabs}
695 {status}
696 {standing.length === 0 && <Text color={STONE}>No standing exceptions in this project.</Text>}
697 {standing.map(one => (
698 <Box key={`exception-${one.rule}-${one.file ?? '*'}`} flexDirection="column">
699 <Text>
700 <Text color={SKY} bold>
701 {one.rule}
702 </Text>
703 <Text> {one.file ? relative(one.file, cwd) : 'anywhere in this project'}</Text>
704 </Text>
705 <Text color={STONE}>{one.note}</Text>
706 <Button key={`revoke-${one.rule}-${one.file ?? '*'}`} label="Revoke" onPress={() => revokeException($, one)} />
707 </Box>
708 ))}
709 </Box>
710 )
711 }
712
713 if (shown === 'rules') {
714 return (
715 <Box flexDirection="column" gap={1}>
716 {tabs}
717 {status}
718 {CATEGORIES.map(category => {
719 const isConfigured = settings.categories.has(category)
720 const isOn = isConfigured && !off.includes(category)
721
722 return (
723 <Box key={`rule-${category}`} gap={1}>
724 <Box width={10} flexShrink={0}>
725 {isConfigured ? (
726 <Button
727 key={`toggle-${category}`}
728 label={isOn ? 'Mute' : 'Unmute'}
729 onPress={() => update($, muted, now => (now.includes(category) ? now.filter(one => one !== category) : [...now, category]))}
730 />
731 ) : (
732 <Text color={STONE}>/config</Text>
733 )}
734 </Box>
735 <Box width={3} flexShrink={0}>
736 <Text color={isOn ? OLIVE : STONE} bold>
737 {isOn ? 'on' : 'off'}
738 </Text>
739 </Box>
740 <Box flexGrow={1} flexShrink={1}>
741 <Text color={isOn ? undefined : STONE}>{CATEGORY_LABEL[category]}</Text>
742 </Box>
743 </Box>
744 )
745 })}
746 <Text color={STONE}>Mute lasts this session; /config changes it for good.</Text>
747 </Box>
748 )
749 }
750
751 return (
752 <Box flexDirection="column" gap={1}>
753 {tabs}
754 {status}
755 {list.length === 0 && <Text color={STONE}>Nothing held yet. Changes that lower the bar show up here.</Text>}
756 {[...list]
757 .reverse()
758 .slice(0, room)
759 .map(entry => {
760 const look = DECISION_LOOK[entry.decision]
761
762 return (
763 <Box key={`entry-${entry.id}`} flexDirection="column">
764 <Text>
765 <Text color={look.color} bold inverse>
766 {` ${look.label} `}
767 </Text>
768 <Text color={STONE}> {categoryOf(entry.finding)} </Text>
769 <Text>{entry.finding.file ? relative(entry.finding.file, cwd) : entry.tool}</Text>
770 </Text>
771 <Text bold>{entry.finding.title}</Text>
772 {(entry.finding.before ?? []).map(line => (
773 <Text color={CLAY} wrap="truncate-end">
774 {` - ${line}`}
775 </Text>
776 ))}
777 {(entry.finding.after ?? []).map(line => (
778 <Text color={SKY} wrap="truncate-end">
779 {` + ${line}`}
780 </Text>
781 ))}
782 {entry.note !== undefined && <Text color={STONE} italic>{` ${entry.note}`}</Text>}
783 {entry.decision === 'blocked' && entry.finding.file !== undefined && (
784 <Button
785 key={`allow-${entry.id}`}
786 label="Allow in this file"
787 onPress={() => addExceptions($, [entry.finding], 'file', 'allowed from the ledger')}
788 />
789 )}
790 </Box>
791 )
792 })}
793 </Box>
794 )
795 })
796}
797hooks/detect.ts 935 lines1import type { GoalpostCategory, GoalpostFinding, GoalpostRule } from '../types'
2
3/**
4 * What the detectors look at, as the hooks module configured them.
5 */
6export type DetectOptions = {
7 /** The categories that are on; a finding of any other is dropped. */
8 categories: ReadonlySet<GoalpostCategory>
9 /** An extra pattern for test files, on top of the usual names. */
10 testPattern?: RegExp
11}
12
13/** One file as it was and as the change leaves it; `null` when it does not exist. */
14export type FileChange = { path: string; before: string | null; after: string | null }
15
16const CATEGORY_OF: Record<GoalpostRule, GoalpostCategory> = {
17 'tests.assertions': 'tests',
18 'tests.removed': 'tests',
19 'tests.skipped': 'tests',
20 'tests.focused': 'tests',
21 'tests.loosened': 'tests',
22 'tests.expected': 'tests',
23 'tests.retargeted': 'tests',
24 'tests.snapshots': 'tests',
25 'tests.narrowed': 'tests',
26 suppressions: 'suppressions',
27 config: 'config',
28 ci: 'ci',
29 coverage: 'coverage',
30 bypass: 'bypass',
31 'shell-edit': 'tests',
32}
33
34const SAMPLE = 3
35
36// ---------------------------------------------------------------- text helpers
37
38function count(text: string, pattern: RegExp): number {
39 return text.match(new RegExp(pattern.source, pattern.flags.includes('g') ? pattern.flags : `${pattern.flags}g`))?.length ?? 0
40}
41
42/** The text without whole-line comments, so commenting a line out counts as removing it. */
43function code(text: string): string {
44 return text
45 .split('\n')
46 .filter(line => !/^\s*(?:\/\/|#(?![!\[])|\/\*|\*|--|<!--)/.test(line))
47 .join('\n')
48}
49
50/** The lines of `after` that match `pattern` and were not in `before`, trimmed, at most a few. */
51function addedLines(before: string, after: string, global: RegExp): string[] {
52 const pattern = new RegExp(global.source, global.flags.replace('g', ''))
53 const held = new Map<string, number>()
54 for (const line of before.split('\n')) {
55 held.set(line.trim(), (held.get(line.trim()) ?? 0) + 1)
56 }
57 const added: string[] = []
58 for (const line of after.split('\n')) {
59 const key = line.trim()
60 const left = held.get(key) ?? 0
61 if (left > 0) {
62 held.set(key, left - 1)
63 } else if (pattern.test(line)) {
64 added.push(key)
65 }
66 }
67
68 return added.slice(0, SAMPLE)
69}
70
71/**
72 * Assertion lines rewritten in place: a removed line and an added one that
73 * read the same once every number and string is blanked out, as when
74 * `toBe(24.19)` becomes `toBe(24)`.
75 */
76function rewrittenValues(before: string, after: string): Array<readonly [string, string]> {
77 const literal = /-?\d+(?:\.\d+)?(?:e-?\d+)?|'(?:\\.|[^'\\])*'|"(?:\\.|[^"\\])*"|`(?:\\.|[^`\\])*`/g
78 const shape = (line: string): string => line.replace(literal, '_')
79 const single = new RegExp(ASSERTION.source)
80 const gone = removedLinesAll(before, after).filter(line => single.test(line))
81 const added = removedLinesAll(after, before).filter(line => single.test(line))
82 const pairs: Array<readonly [string, string]> = []
83 for (const line of gone) {
84 const index = added.findIndex(other => shape(other) === shape(line))
85 if (index >= 0) {
86 pairs.push([line, added[index] ?? ''] as const)
87 added.splice(index, 1)
88 }
89 }
90
91 return pairs
92}
93
94const IMPORT_FORMS: readonly RegExp[] = [
95 /^(\s*(?:import|export)\b[^'"`]*?\bfrom\s*)(['"`])([^'"`]+)\2/,
96 /^(\s*import\s*)(['"`])([^'"`]+)\2/,
97 /^(.*?\brequire\s*\(\s*)(['"`])([^'"`]+)\2/,
98 /^(\s*from\s+)()([\w.]+)(?=\s+import\b)/,
99 /^(\s*import\s+)()([\w.]+)\s*$/,
100]
101
102/** The module an import line loads, and the line with that module blanked out. */
103export function importTarget(line: string): { target: string; shape: string } | null {
104 for (const form of IMPORT_FORMS) {
105 const match = form.exec(line)
106 if (match) {
107 return { target: match[3] ?? '', shape: line.replace(form, (_all, lead: string, quote: string) => `${lead}${quote}_${quote}`) }
108 }
109 }
110
111 return null
112}
113
114/**
115 * Imports that now load the same names from a different module: a test
116 * pointed at a stand-in instead of the code it was written for. Whether the
117 * old module still exists (a stand-in) or was moved (a fix) is the hooks'
118 * call, since only they can look at the disk.
119 */
120function retargetedImports(before: string, after: string): Array<readonly [string, string]> {
121 const gone = removedLinesAll(before, after).filter(line => importTarget(line) !== null)
122 const added = removedLinesAll(after, before).filter(line => importTarget(line) !== null)
123 const pairs: Array<readonly [string, string]> = []
124 for (const line of gone) {
125 const shape = importTarget(line)?.shape
126 const index = added.findIndex(other => importTarget(other)?.shape === shape)
127 if (index >= 0) {
128 pairs.push([line, added[index] ?? ''] as const)
129 added.splice(index, 1)
130 }
131 }
132
133 return pairs
134}
135
136/** The names an import line brings in: `{ a, b as c }`, a default, `* as x`, or Python's `import a, b`. */
137function importedNames(line: string): string[] {
138 const js = /^\s*import\s+(.+?)\s+from\b/.exec(line) ?? /^\s*(?:const|let|var)\s+(.+?)\s*=\s*require\b/.exec(line)
139 const py = /^\s*from\s+[\w.]+\s+import\s+\(?([^)#]+)/.exec(line)
140 const list = js?.[1] ?? py?.[1] ?? ''
141
142 return list
143 .replace(/[{}()]/g, ',')
144 .split(',')
145 .map(part => part.trim().split(/\s+as\s+|\s*:\s*/).pop()?.replace(/^\*\s*as\s+/, '').trim() ?? '')
146 .filter(name => /^[A-Za-z_$][\w$]*$/.test(name))
147}
148
149/**
150 * Names a test stopped importing and now defines itself: the code under test
151 * swapped for a copy that lives inside the test.
152 */
153function inlinedStandIns(before: string, after: string): string[] {
154 const names = (text: string): Set<string> =>
155 new Set(text.split('\n').filter(line => importTarget(line) !== null).flatMap(importedNames))
156 // Still imported counts too: in Python a later `def` shadows the import.
157 return [...names(before)].filter(name => {
158 const defined = new RegExp(`(?:^|\\n)\\s*(?:export\\s+)?(?:(?:const|let|var)\\s+${name}\\s*=|(?:async\\s+)?function\\s*\\*?\\s*${name}\\s*\\(|class\\s+${name}\\b|def\\s+${name}\\s*\\()`)
159
160 return defined.test(after) && !defined.test(before)
161 })
162}
163
164/** Every line of `before`, trimmed, that is not in `after` (counting repeats). */
165function removedLinesAll(before: string, after: string): string[] {
166 const held = new Map<string, number>()
167 for (const line of after.split('\n')) {
168 held.set(line.trim(), (held.get(line.trim()) ?? 0) + 1)
169 }
170 const removed: string[] = []
171 for (const line of before.split('\n')) {
172 const key = line.trim()
173 const left = held.get(key) ?? 0
174 if (left > 0) {
175 held.set(key, left - 1)
176 } else if (key !== '') {
177 removed.push(key)
178 }
179 }
180
181 return removed
182}
183
184/** The lines of `before` that match `pattern` and are gone from `after`. */
185function removedLines(before: string, after: string, pattern: RegExp): string[] {
186 return addedLines(after, before, pattern)
187}
188
189function basename(path: string): string {
190 return path.split('/').pop() ?? path
191}
192
193function plural(n: number, one: string, many = `${one}s`): string {
194 return `${n} ${n === 1 ? one : many}`
195}
196
197// ---------------------------------------------------------------- tests
198
199const TEST_PATH = [
200 /(?:^|\/)(?:tests?|__tests__|specs?|e2e)\//,
201 /\.(?:test|spec|e2e)\.[cm]?[jt]sx?$/,
202 /(?:^|\/)test_[^/]*\.py$/,
203 /_test\.(?:py|go|exs?)$/,
204 /(?:Test|Tests|Spec)\.(?:java|kt|cs|swift|scala)$/,
205 /_spec\.rb$/,
206 /\.(?:test|spec)\.(?:php|dart)$/,
207]
208
209const BACKUP = /\.(?:bak|orig|rej|swp|tmp)$|~$/
210
211export function isTestFile(path: string, extra?: RegExp): boolean {
212 if (BACKUP.test(path)) {
213 return false
214 }
215
216 return TEST_PATH.some(pattern => pattern.test(path)) || (extra?.test(path) ?? false)
217}
218
219const ASSERTION =
220 /\bexpect\s*\(|\bassert(?:\.\w+|_\w+|[A-Z]\w*)?\s*!?\s*\(|\bassert\s+(?=[^\s(=:])|\.should\b|\bt\.(?:Errorf?|Fatalf?|Fail(?:Now)?)\s*\(|\brequire\.[A-Z]\w*\s*\(|\bXCTAssert\w*\s*\(|\bpytest\.raises\s*\(/g
221
222const TEST_CASE =
223 /\b[xf]?(?:it|test|specify)(?:\.(?:skip|only|todo|concurrent|failing|each\s*\(.*?\)))*\s*\(\s*['"`]|\bdef\s+test\w*\s*\(|\bfunc\s+Test\w*\s*\(|#\[(?:tokio::)?test\]|@Test\b|\bfn\s+test_\w*\s*\(/g
224
225const SKIP =
226 /\b(?:it|test|describe|context|suite|specify)\.(?:skip|todo)\b|\bx(?:it|describe|test|context|specify)\s*\(|@pytest\.mark\.(?:skip|skipif|xfail)\b|@unittest\.(?:skip|skipIf|skipUnless|expectedFailure)\b|\bpytest\.(?:skip|xfail)\s*\(|\bt\.Skip(?:f|Now)?\s*\(|#\[ignore\b|@(?:Disabled|Ignore)\b|\bpending\s*\(|\bself\.skipTest\s*\(|\bmarkTestSkipped\s*\(/g
227
228const FOCUS = /\b(?:it|test|describe|context|suite|specify)\.only\b|\bf(?:it|describe|context)\s*\(/g
229
230const STRONG =
231 /\.(?:toBe|toEqual|toStrictEqual|toHaveBeenCalledWith|toHaveBeenCalledTimes|toThrow|toThrowError|toMatchObject|toContain|toHaveLength|toHaveProperty|toMatchSnapshot|toMatchInlineSnapshot)\s*\(|\bassert(?:Equals?|_eq|_ne|NotEqual|Raises|Is|In|Count(?:Equal)?|DictEqual|ListEqual|AlmostEqual|Same|Throws)\s*!?\s*\(|\bassert\b[^\n]*(?:==|!=|\bin\b|\bis\b)/g
232
233const WEAK =
234 /\.(?:toBeTruthy|toBeDefined|toBeFalsy|toBeInstanceOf|toBeCloseTo|toHaveBeenCalled|toBeGreaterThanOrEqual\s*\(\s*0\s*\)|toBeGreaterThan\s*\(\s*-?1\s*\))\s*\(|\.not\.(?:toBeNull|toBeUndefined|toThrow)\s*\(|\bassert(?:True|IsNotNone|IsInstance|Truthy)\s*\(|\bassert\s+\w+(?:\.\w+)*\s*$|\.toMatch\s*\(\s*\/\.\*\/|expect\.anything\s*\(/gm
235
236const SNAPSHOT_FILE = /(?:__snapshots__\/|\.snap$|\.ambr$|__image_snapshots__\/)/
237
238function testFindings(change: FileChange, testPattern?: RegExp): GoalpostFinding[] {
239 const { path, before, after } = change
240 const isTest = isTestFile(path, testPattern) || /#\[cfg\(test\)\]/.test(before ?? '')
241 const found: GoalpostFinding[] = []
242
243 if (SNAPSHOT_FILE.test(path) && before !== null && before !== after) {
244 found.push({
245 rule: 'tests.snapshots',
246 file: path,
247 title: after === null ? 'deletes a snapshot' : 'rewrites a snapshot',
248 detail: `${basename(path)} is a recorded expectation; changing it by hand makes the test agree with the code`,
249 })
250 }
251 if (!isTest || before === null) {
252 if (isTest && after !== null) {
253 const skips = count(code(after), SKIP)
254 const focus = count(code(after), FOCUS)
255 if (skips > 0) {
256 found.push({
257 rule: 'tests.skipped',
258 file: path,
259 title: `adds a test file with ${plural(skips, 'skipped test')}`,
260 detail: `${addedLines('', after, SKIP).join(' | ')}`,
261 after: addedLines('', after, SKIP),
262 })
263 }
264 if (focus > 0) {
265 found.push({
266 rule: 'tests.focused',
267 file: path,
268 title: 'adds a focused test (.only)',
269 detail: `only the focused tests run, the rest are silently skipped`,
270 after: addedLines('', after, FOCUS),
271 })
272 }
273 }
274
275 return found
276 }
277
278 const was = code(before)
279 const now = code(after ?? '')
280
281 if (after === null) {
282 const cases = count(was, TEST_CASE)
283 found.push({
284 rule: 'tests.removed',
285 file: path,
286 title: `deletes a test file${cases > 0 ? ` (${plural(cases, 'test')})` : ''}`,
287 detail: `the file and its ${plural(count(was, ASSERTION), 'assertion')} go away`,
288 })
289
290 return found
291 }
292
293 const casesBefore = count(was, TEST_CASE)
294 const casesAfter = count(now, TEST_CASE)
295 if (casesAfter < casesBefore) {
296 found.push({
297 rule: 'tests.removed',
298 file: path,
299 title: `removes ${plural(casesBefore - casesAfter, 'test')}`,
300 detail: `${casesBefore} → ${casesAfter} tests`,
301 before: removedLines(before, after, TEST_CASE),
302 })
303 }
304
305 const assertsBefore = count(was, ASSERTION)
306 const assertsAfter = count(now, ASSERTION)
307 if (assertsAfter < assertsBefore) {
308 found.push({
309 rule: 'tests.assertions',
310 file: path,
311 title: `removes ${plural(assertsBefore - assertsAfter, 'assertion')}`,
312 detail: `${assertsBefore} → ${assertsAfter} assertions`,
313 before: removedLines(before, after, ASSERTION),
314 })
315 }
316
317 if (count(now, SKIP) > count(was, SKIP)) {
318 found.push({
319 rule: 'tests.skipped',
320 file: path,
321 title: `skips ${plural(count(now, SKIP) - count(was, SKIP), 'test')}`,
322 detail: `a skipped test still reads as passing`,
323 after: addedLines(before, after, SKIP),
324 })
325 }
326
327 if (count(now, FOCUS) > count(was, FOCUS)) {
328 found.push({
329 rule: 'tests.focused',
330 file: path,
331 title: 'focuses a test (.only)',
332 detail: `only the focused tests run, the rest are silently skipped`,
333 after: addedLines(before, after, FOCUS),
334 })
335 }
336
337 const rewritten = rewrittenValues(code(before), code(after))
338 if (rewritten.length > 0) {
339 found.push({
340 rule: 'tests.expected',
341 file: path,
342 title: `changes the values in ${plural(rewritten.length, 'assertion')}`,
343 detail: 'an expected value moved; if it moved to what the code returns now, the test checks nothing',
344 before: rewritten.slice(0, SAMPLE).map(([was]) => was),
345 after: rewritten.slice(0, SAMPLE).map(([, now]) => now),
346 })
347 }
348
349 const retargeted = retargetedImports(code(before), code(after))
350 if (retargeted.length > 0) {
351 const [[was = '', is = ''] = []] = retargeted
352 found.push({
353 rule: 'tests.retargeted',
354 file: path,
355 title: `points the test at other code (${importTarget(was)?.target ?? '?'} → ${importTarget(is)?.target ?? '?'})`,
356 detail: 'the test now checks a different module than before; a stand-in can pass while the real code stays broken',
357 before: retargeted.slice(0, SAMPLE).map(([line]) => line),
358 after: retargeted.slice(0, SAMPLE).map(([, line]) => line),
359 })
360 }
361
362 const inlined = inlinedStandIns(code(before), code(after))
363 if (inlined.length > 0) {
364 found.push({
365 rule: 'tests.retargeted',
366 file: path,
367 title: `replaces the imported ${inlined.slice(0, 2).join(', ')} with a copy inside the test`,
368 detail: 'the test now checks its own copy of the code; the copy can pass while the real code stays broken',
369 after: addedLines(before, after, new RegExp(`\\b(?:${inlined.join('|')})\\b`)),
370 })
371 }
372
373 const weakUp = count(now, WEAK) - count(was, WEAK)
374 const strongDown = count(was, STRONG) - count(now, STRONG)
375 if (weakUp > 0 && strongDown > 0) {
376 found.push({
377 rule: 'tests.loosened',
378 file: path,
379 title: `loosens ${plural(Math.min(weakUp, strongDown), 'assertion')}`,
380 detail: `exact checks traded for ones that pass on almost anything`,
381 before: removedLines(before, after, STRONG),
382 after: addedLines(before, after, WEAK),
383 })
384 }
385
386 return found
387}
388
389// ---------------------------------------------------------------- suppressions
390
391const SUPPRESSIONS: ReadonlyArray<readonly [string, RegExp]> = [
392 ['@ts-ignore', /@ts-ignore\b/],
393 ['@ts-expect-error', /@ts-expect-error\b/],
394 ['@ts-nocheck', /@ts-nocheck\b/],
395 ['eslint-disable', /eslint-disable(?:-next-line|-line)?\b/],
396 ['biome-ignore', /biome-ignore\b/],
397 ['oxlint-disable', /oxlint-disable\b/],
398 ['# noqa', /#\s*noqa\b/],
399 ['# type: ignore', /#\s*type:\s*ignore\b/],
400 ['pyright: ignore', /pyright:\s*ignore\b/],
401 ['pylint: disable', /pylint:\s*disable\b/],
402 ['mypy: ignore-errors', /mypy:\s*ignore-errors\b/],
403 ['//nolint', /\/\/\s*nolint\b/],
404 ['#[allow(...)]', /#!?\[allow\(/],
405 ['@SuppressWarnings', /@SuppressWarnings\b/],
406 ['rubocop:disable', /rubocop:disable\b/],
407 ['swiftlint:disable', /swiftlint:disable\b/],
408 ['NOLINT', /\bNOLINT(?:NEXTLINE|BEGIN)?\b/],
409 ['@phpstan-ignore', /@phpstan-ignore/],
410 ['@psalm-suppress', /@psalm-suppress\b/],
411 ['#pragma warning disable', /#pragma\s+warning\s+disable\b/],
412 ['coverage ignore', /(?:istanbul|c8|v8)\s+ignore\b|pragma:\s*no\s*cover\b/],
413]
414
415const PROSE = /\.(?:md|mdx|markdown|txt|rst|adoc)$/
416
417function suppressionFindings({ path, before, after }: FileChange): GoalpostFinding[] {
418 if (after === null || PROSE.test(path)) {
419 return []
420 }
421 const added: string[] = []
422 const lines: string[] = []
423 for (const [name, pattern] of SUPPRESSIONS) {
424 const more = count(after, pattern) - count(before ?? '', pattern)
425 if (more > 0) {
426 added.push(more > 1 ? `${more}× ${name}` : name)
427 lines.push(...addedLines(before ?? '', after, pattern))
428 }
429 }
430 if (added.length === 0) {
431 return []
432 }
433
434 return [
435 {
436 rule: 'suppressions',
437 file: path,
438 title: `adds ${added.join(', ')}`,
439 detail: `the check still runs, it just stops reporting here`,
440 after: lines.slice(0, SAMPLE),
441 },
442 ]
443}
444
445// ---------------------------------------------------------------- config
446
447const TS_CONFIG = /(?:^|\/)[jt]sconfig(?:\.[\w-]+)?\.json$/
448const TS_STRICT = [
449 'strict',
450 'noImplicitAny',
451 'strictNullChecks',
452 'strictFunctionTypes',
453 'strictBindCallApply',
454 'strictPropertyInitialization',
455 'noImplicitThis',
456 'useUnknownInCatchVariables',
457 'alwaysStrict',
458 'noUncheckedIndexedAccess',
459 'exactOptionalPropertyTypes',
460 'noImplicitReturns',
461 'noImplicitOverride',
462 'noFallthroughCasesInSwitch',
463 'noUnusedLocals',
464 'noUnusedParameters',
465 'checkJs',
466]
467const COVERED_BY_STRICT = new Set([
468 'noImplicitAny',
469 'strictNullChecks',
470 'strictFunctionTypes',
471 'strictBindCallApply',
472 'strictPropertyInitialization',
473 'noImplicitThis',
474 'useUnknownInCatchVariables',
475 'alwaysStrict',
476])
477
478function flag(text: string, key: string): boolean | undefined {
479 const match = new RegExp(`["']?\\b${key}\\b["']?\\s*[:=]\\s*(true|false|True|False)\\b`).exec(code(text))
480
481 return match ? match[1]?.toLowerCase() === 'true' : undefined
482}
483
484const LINT_CONFIG =
485 /(?:^|\/)(?:\.eslintrc(?:\.\w+)?|eslint\.config\.[cm]?[jt]s|\.oxlintrc\.json|biome\.jsonc?|\.stylelintrc(?:\.\w+)?|stylelint\.config\.[cm]?js|deno\.jsonc?)$/
486const SEVERITY_RANK: Record<string, number> = { error: 2, '2': 2, warn: 1, '1': 1, info: 1, off: 0, '0': 0 }
487
488function severities(text: string): Map<string, number> {
489 const found = new Map<string, number>()
490 const pattern = /['"]([@\w/.-]+)['"]\s*:\s*\[?\s*['"]?(error|warn|info|off|[012])\b/g
491 for (const match of code(text).matchAll(pattern)) {
492 const [, rule = '', level = ''] = match
493 found.set(rule, SEVERITY_RANK[level] ?? 0)
494 }
495
496 return found
497}
498
499const PY_CONFIG = /(?:^|\/)(?:pyproject\.toml|setup\.cfg|tox\.ini|ruff\.toml|\.ruff\.toml|\.flake8|mypy\.ini|\.mypy\.ini|\.pylintrc|pylintrc)$/
500const PY_STRICT = [
501 'strict',
502 'disallow_untyped_defs',
503 'disallow_incomplete_defs',
504 'disallow_any_generics',
505 'check_untyped_defs',
506 'warn_return_any',
507 'no_implicit_optional',
508 'strict_optional',
509 'warn_unused_ignores',
510]
511const PY_LOOSE = ['ignore_errors', 'ignore_missing_imports', 'allow_untyped_defs', 'allow_redefinition']
512
513function ignoredCodes(text: string): number {
514 let total = 0
515 const pattern = /\b(?:extend-ignore|ignore|disable|extend-per-file-ignores|per-file-ignores|ignore-names)\s*=\s*(\[[^\]]*\]|[^\n]*)/g
516 for (const match of code(text).matchAll(pattern)) {
517 total += count(match[1] ?? '', /\b[A-Z]{1,4}\d{1,4}\b|\b[a-z]+(?:-[a-z]+)+\b/g)
518 }
519
520 return total
521}
522
523function configFindings({ path, before, after }: FileChange): GoalpostFinding[] {
524 if (before === null) {
525 return []
526 }
527 const now = after ?? ''
528 const weaker: string[] = []
529
530 if (TS_CONFIG.test(path)) {
531 const strictAfter = flag(now, 'strict') === true
532 for (const key of TS_STRICT) {
533 if (flag(before, key) === true && flag(now, key) !== true && !(strictAfter && COVERED_BY_STRICT.has(key))) {
534 weaker.push(`${key} turned off`)
535 }
536 }
537 }
538
539 if (LINT_CONFIG.test(path)) {
540 const was = severities(before)
541 const is = severities(now)
542 for (const [rule, rank] of was) {
543 const level = is.get(rule)
544 if (rank > 0 && level === undefined) {
545 weaker.push(`${rule} removed`)
546 } else if (level !== undefined && level < rank) {
547 weaker.push(`${rule} ${rank === 2 ? 'error' : 'warn'} → ${level === 1 ? 'warn' : 'off'}`)
548 }
549 }
550 }
551
552 if (PY_CONFIG.test(path)) {
553 for (const key of PY_STRICT) {
554 if (flag(before, key) === true && flag(now, key) !== true) {
555 weaker.push(`${key} turned off`)
556 }
557 }
558 for (const key of PY_LOOSE) {
559 if (flag(before, key) !== true && flag(now, key) === true) {
560 weaker.push(`${key} turned on`)
561 }
562 }
563 const more = ignoredCodes(now) - ignoredCodes(before)
564 if (more > 0) {
565 weaker.push(`${plural(more, 'more ignored rule')}`)
566 }
567 }
568
569 if (after === null && (TS_CONFIG.test(path) || LINT_CONFIG.test(path))) {
570 weaker.push('the whole file deleted')
571 }
572
573 if (weaker.length === 0) {
574 return []
575 }
576
577 return [
578 {
579 rule: 'config',
580 file: path,
581 title: `relaxes ${basename(path)}`,
582 detail: `${weaker.slice(0, 4).join(', ')}${weaker.length > 4 ? `, +${weaker.length - 4} more` : ''}`,
583 after: weaker.slice(0, SAMPLE),
584 },
585 ]
586}
587
588// ---------------------------------------------------------------- CI
589
590const CI_FILE =
591 /(?:^|\/)(?:\.github\/workflows\/[^/]+\.ya?ml|\.gitlab-ci\.ya?ml|\.circleci\/config\.ya?ml|azure-pipelines\.ya?ml|bitbucket-pipelines\.ya?ml|\.buildkite\/[^/]+\.ya?ml|Jenkinsfile|\.travis\.ya?ml|\.pre-commit-config\.ya?ml|lefthook\.ya?ml|\.husky\/[^/]+)$/
592
593const CHECK_COMMAND =
594 /\b(?:(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:test|lint|typecheck|check|build)|vitest|jest|pytest|go\s+(?:test|vet)|cargo\s+(?:test|clippy|check|nextest)|ctest|mocha|playwright\s+test|rspec|phpunit|dotnet\s+test|tsc|mypy|pyright|eslint|ruff|flake8|pylint|biome|golangci-lint|stylelint|prettier\s+--check|pre-commit\s+run|claude\s+plugin\s+(?:test|validate))\b/g
595
596const CI_SOFTENERS: ReadonlyArray<readonly [string, RegExp]> = [
597 ['continue-on-error: true', /continue-on-error:\s*true\b/g],
598 ['allow_failure: true', /allow_failure:\s*true\b/g],
599 ['|| true', /\|\|\s*(?:true|:)\s*(?:$|[;&)])|\|\|\s*exit\s+0\b/gm],
600 ['if: false', /\bif:\s*(?:false|\$\{\{\s*false\s*\}\})/g],
601 ['set +e', /\bset\s+\+e\b/g],
602]
603
604function ciFindings({ path, before, after }: FileChange): GoalpostFinding[] {
605 if (before === null || !CI_FILE.test(path)) {
606 return []
607 }
608 const was = code(before)
609 const now = code(after ?? '')
610 const weaker: string[] = []
611 const lines: string[] = []
612
613 for (const [name, pattern] of CI_SOFTENERS) {
614 const more = count(now, pattern) - count(was, pattern)
615 if (more > 0) {
616 weaker.push(more > 1 ? `${more}× ${name}` : name)
617 lines.push(...addedLines(before, after ?? '', pattern))
618 }
619 }
620 const checksGone = count(was, CHECK_COMMAND) - count(now, CHECK_COMMAND)
621 if (checksGone > 0) {
622 weaker.push(`${plural(checksGone, 'check')} removed`)
623 lines.push(...removedLines(before, after ?? '', CHECK_COMMAND).map(line => `- ${line}`))
624 }
625 const hooksGone = count(was, /^\s*-\s+id:/gm) - count(now, /^\s*-\s+id:/gm)
626 if (hooksGone > 0) {
627 weaker.push(`${plural(hooksGone, 'hook')} removed`)
628 }
629 if (after === null) {
630 weaker.push('the whole file deleted')
631 }
632 if (weaker.length === 0) {
633 return []
634 }
635
636 return [
637 {
638 rule: 'ci',
639 file: path,
640 title: `weakens ${basename(path)}`,
641 detail: `${weaker.join(', ')}`,
642 after: lines.slice(0, SAMPLE),
643 },
644 ]
645}
646
647// ---------------------------------------------------------------- coverage
648
649const COVERAGE_FILE =
650 /(?:^|\/)(?:package\.json|jest\.config\.[\w.]+|vitest\.config\.[\w.]+|vite\.config\.[\w.]+|vitest\.workspace\.[\w.]+|pyproject\.toml|setup\.cfg|tox\.ini|\.coveragerc|pytest\.ini|codecov\.ya?ml|\.codecov\.ya?ml|\.nycrc(?:\.\w+)?|\.c8rc(?:\.\w+)?|karma\.conf\.[cm]?js|\.simplecov|Makefile|build\.gradle(?:\.kts)?|pom\.xml|\.github\/workflows\/[^/]+\.ya?ml|\.gitlab-ci\.ya?ml)$/
651
652const THRESHOLDS: ReadonlyArray<readonly [string, RegExp]> = [
653 ['fail_under', /\bfail_under\s*[=:]\s*(\d+(?:\.\d+)?)/g],
654 ['--cov-fail-under', /--cov-fail-under[= ](\d+(?:\.\d+)?)/g],
655 ['--fail-under', /(?<!cov)--fail-under[= ](\d+(?:\.\d+)?)/g],
656 ['branches', /["']?\bbranches["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
657 ['functions', /["']?\bfunctions["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
658 ['lines', /["']?\blines["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
659 ['statements', /["']?\bstatements["']?\s*:\s*(-?\d+(?:\.\d+)?)/g],
660 ['target', /\btarget:\s*(\d+(?:\.\d+)?)%?/g],
661 ['minimum_coverage', /\bminimum_coverage\s*\(?\s*(\d+(?:\.\d+)?)/g],
662 ['minimum', /<minimum>(\d*\.?\d+)<\/minimum>|\bminimum\s*=\s*(?:BigDecimal\s*\(\s*)?["']?(\d*\.?\d+)/g],
663 ['--check-coverage', /--(?:lines|branches|functions|statements)[= ](\d+(?:\.\d+)?)/g],
664]
665
666function thresholds(text: string): Map<string, number[]> {
667 const found = new Map<string, number[]>()
668 for (const [name, pattern] of THRESHOLDS) {
669 const values = [...code(text).matchAll(pattern)].map(match => Number(match[1] ?? match[2]))
670 if (values.length > 0) {
671 found.set(name, values)
672 }
673 }
674
675 return found
676}
677
678function coverageFindings({ path, before, after }: FileChange): GoalpostFinding[] {
679 if (before === null || after === null) {
680 return []
681 }
682 if (!COVERAGE_FILE.test(path) && !/coverage|cov-fail|fail_under|codecov/i.test(before)) {
683 return []
684 }
685 const relevant = /coverage|cov-fail|fail_under|codecov|threshold|minimum|jacoco|simplecov/i
686 if (!relevant.test(before)) {
687 return []
688 }
689 const was = thresholds(before)
690 const is = thresholds(after)
691 const lowered: string[] = []
692 for (const [name, values] of was) {
693 const now = is.get(name) ?? []
694 values.forEach((value, index) => {
695 const later = now[index]
696 if (later === undefined && now.length < values.length) {
697 lowered.push(`${name} ${value} removed`)
698 } else if (later !== undefined && later < value) {
699 lowered.push(`${name} ${value} → ${later}`)
700 }
701 })
702 }
703 if (lowered.length === 0) {
704 return []
705 }
706
707 return [
708 {
709 rule: 'coverage',
710 file: path,
711 title: 'lowers a coverage threshold',
712 detail: `${lowered.slice(0, 4).join(', ')}`,
713 after: lowered.slice(0, SAMPLE),
714 },
715 ]
716}
717
718// ---------------------------------------------------------------- files
719
720/**
721 * Everything a change to one file does to lower the bar, in the categories
722 * that are on.
723 */
724export function inspectChange(change: FileChange, options: DetectOptions): GoalpostFinding[] {
725 if (change.before === change.after) {
726 return []
727 }
728 const all = [
729 ...testFindings(change, options.testPattern),
730 ...suppressionFindings(change),
731 ...configFindings(change),
732 ...ciFindings(change),
733 ...coverageFindings(change),
734 ]
735
736 return all.filter(finding => options.categories.has(CATEGORY_OF[finding.rule]))
737}
738
739/**
740 * The file an Edit leaves, or null when `old` is not in it (the tool will
741 * refuse the edit itself).
742 */
743export function applyEdit(before: string, old: string, replacement: string, isAll = false): string | null {
744 if (old === '' || !before.includes(old)) {
745 return null
746 }
747
748 return isAll ? before.split(old).join(replacement) : before.replace(old, () => replacement)
749}
750
751// ---------------------------------------------------------------- shell
752
753/** The command with every quoted string emptied, so flags inside a message don't count. */
754function withoutQuoted(command: string): string {
755 return command.replace(/'[^']*'|"(?:\\.|[^"\\])*"|\$'[^']*'/g, '""').replace(/<<-?\s*['"]?(\w+)['"]?[\s\S]*?\n\1\b/g, '')
756}
757
758type Segment = { words: string[]; redirects: string[] }
759
760/**
761 * The simple commands of a command line, split at `&&`, `||`, `;`, `|` and
762 * newlines, each as its words (quotes taken off, text kept) and the files it
763 * redirects output to. Heredoc bodies are text, not commands, so they go.
764 */
765function segmentsOf(command: string): Segment[] {
766 const text = command.replace(/<<-?\s*(['"]?)(\w+)\1[^\n]*\n[\s\S]*?\n\s*\2(?=\s|$)/g, '')
767 const tokens = text.match(/'[^']*'|"(?:\\.|[^"\\])*"|&&|\|\||[;|\n]|\d?>>?|[^\s;&|()<>]+/g) ?? []
768 const segments: Segment[] = [{ words: [], redirects: [] }]
769 let isRedirect = false
770 for (const token of tokens) {
771 const current = segments[segments.length - 1] ?? { words: [], redirects: [] }
772 if (['&&', '||', ';', '|', '\n'].includes(token)) {
773 segments.push({ words: [], redirects: [] })
774 isRedirect = false
775 } else if (/^\d?>>?$/.test(token)) {
776 isRedirect = !/^2/.test(token)
777 } else {
778 const word = token.replace(/^['"]|['"]$/g, '')
779 if (isRedirect) {
780 current.redirects.push(word)
781 isRedirect = false
782 } else {
783 current.words.push(word)
784 }
785 }
786 }
787
788 return segments.filter(segment => segment.words.length > 0 || segment.redirects.length > 0)
789}
790
791/** The words of a simple command without leading `VAR=value`, `sudo`, `command`, `env` or `xargs`. */
792function withoutPrefix(words: string[]): string[] {
793 let index = 0
794 while (index < words.length && (/^\w+=/.test(words[index] ?? '') || ['sudo', 'command', 'env', 'xargs', 'nohup', 'time'].includes(words[index] ?? ''))) {
795 index += 1
796 }
797
798 return words.slice(index)
799}
800
801const GIT_VERB = (verb: string): RegExp => new RegExp(`\\bgit\\b(?:\\s+-[cC]\\s+\\S+)*\\s+${verb}\\b[^;&|]*`, 'g')
802
803const SKIP_HOOK_ENV = /(?:^|[\s;&|(])(?:export\s+)?(HUSKY=0|HUSKY_SKIP_HOOKS=1|SKIP=\S+|LEFTHOOK=0|LEFTHOOK_EXCLUDE=\S+|PRE_COMMIT_ALLOW_NO_CONFIG=1|OVERCOMMIT_DISABLE=1)/
804
805const RUNNER = /\b(?:(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test|vitest|jest|pytest|py\.test|go\s+test|cargo\s+(?:test|nextest)|mocha|playwright\s+test|rspec|phpunit|dotnet\s+test)\b/
806
807const NARROWING: ReadonlyArray<readonly [string, RegExp]> = [
808 ['--testPathIgnorePatterns', /--testPathIgnorePatterns\b/],
809 ['--passWithNoTests', /--passWithNoTests\b/],
810 ['--deselect', /--deselect\b/],
811 ['-k "not …"', /\s-k\s+["']?not\b/],
812 ['--ignore', /\s--ignore(?:-glob)?[= ]/],
813 ['--exclude', /\s--exclude[= ]/],
814 ['--skip', /\s--skip\b/],
815 ['-p no:', /\s-p\s+no:/],
816]
817
818const SNAPSHOT_UPDATE = /\b(?:jest|vitest|playwright\s+test|(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test)\b[^;&|]*\s(?:-u|--updateSnapshot|--update-snapshots|--update)\b|--snapshot-update\b|--snapshot-update\b|\binsta\s+accept\b|\bINSTA_UPDATE=always\b|UPDATE_SNAPSHOTS?=(?:1|true)/
819
820const GUARDED_FOR_SHELL = (path: string, testPattern?: RegExp): GoalpostCategory | null => {
821 if (isTestFile(path, testPattern) || SNAPSHOT_FILE.test(path)) {
822 return 'tests'
823 }
824 if (CI_FILE.test(path)) {
825 return 'ci'
826 }
827 if (TS_CONFIG.test(path) || LINT_CONFIG.test(path) || PY_CONFIG.test(path)) {
828 return 'config'
829 }
830
831 return null
832}
833
834/**
835 * Everything a shell command does to lower the bar: skipping git hooks,
836 * deleting tests, narrowing or rewriting what a test run checks, and editing
837 * a guarded file where no diff can be seen.
838 */
839export function inspectCommand(command: string, options: DetectOptions): GoalpostFinding[] {
840 const plain = withoutQuoted(command)
841 const found: GoalpostFinding[] = []
842
843 const bypass: string[] = []
844 for (const verb of ['commit', 'push', 'merge', 'am', 'rebase', 'cherry-pick', 'revert']) {
845 for (const [segment] of plain.matchAll(GIT_VERB(verb))) {
846 if (/\s--no-verify\b/.test(segment)) {
847 bypass.push(`git ${verb} --no-verify`)
848 } else if (verb === 'commit' && /\s-[a-zA-Z]*n[a-zA-Z]*(?=\s|$)/.test(segment)) {
849 bypass.push('git commit -n')
850 }
851 }
852 }
853 const env = SKIP_HOOK_ENV.exec(plain)
854 if (env) {
855 bypass.push(env[1] ?? 'hook switch')
856 }
857 if (/\bgit\b[^;&|]*\bconfig\b[^;&|]*\bcore\.hooksPath\b/.test(plain)) {
858 bypass.push('git config core.hooksPath')
859 }
860 if (bypass.length > 0) {
861 found.push({
862 rule: 'bypass',
863 title: `skips the git hooks (${bypass[0]})`,
864 detail: 'the checks a commit or push runs are switched off for this one',
865 after: [command.slice(0, 200)],
866 })
867 }
868
869 if (RUNNER.test(plain)) {
870 const narrowed = NARROWING.filter(([, pattern]) => pattern.test(command)).map(([name]) => name)
871 if (narrowed.length > 0) {
872 found.push({
873 rule: 'tests.narrowed',
874 title: `runs the tests with ${narrowed.join(', ')}`,
875 detail: 'a narrowed run can pass while the tests it leaves out fail',
876 after: [command.slice(0, 200)],
877 })
878 }
879 }
880 if (SNAPSHOT_UPDATE.test(plain)) {
881 found.push({
882 rule: 'tests.snapshots',
883 title: 'rewrites the snapshots',
884 detail: 'updating snapshots makes every snapshot test agree with the code as it is now',
885 after: [command.slice(0, 200)],
886 })
887 }
888
889 const deleted: string[] = []
890 const edited: Array<readonly [string, GoalpostCategory]> = []
891 for (const segment of segmentsOf(command)) {
892 const [verb = '', ...rest] = withoutPrefix(segment.words)
893 const args = rest.filter(word => !word.startsWith('-'))
894 const isDelete = ['rm', 'unlink', 'trash'].includes(verb) || (verb === 'git' && rest[0] === 'rm')
895 const isInPlace =
896 (['sed', 'gsed'].includes(verb) && rest.some(word => /^-[a-zA-Z]*i/.test(word) || word === '--in-place')) ||
897 (verb === 'perl' && rest.some(word => /^-[a-zA-Z]*i/.test(word)))
898 if (isDelete) {
899 deleted.push(...(verb === 'git' ? args.slice(1) : args).filter(word => isTestFile(word, options.testPattern) || SNAPSHOT_FILE.test(word)))
900 }
901 const targets = [...(isInPlace ? args : []), ...segment.redirects]
902 for (const target of targets) {
903 const category = GUARDED_FOR_SHELL(target, options.testPattern)
904 if (category !== null) {
905 edited.push([target, category] as const)
906 }
907 }
908 }
909 if (deleted.length > 0) {
910 found.push({
911 rule: 'tests.removed',
912 file: deleted[0],
913 title: `deletes ${deleted.length === 1 ? 'a test file' : plural(deleted.length, 'test file')}`,
914 detail: deleted.slice(0, SAMPLE).join(', '),
915 before: deleted.slice(0, SAMPLE),
916 })
917 } else if (edited.length > 0) {
918 const [first] = edited
919 found.push({
920 rule: 'shell-edit',
921 file: first?.[0],
922 title: `edits ${basename(first?.[0] ?? '')} from the shell`,
923 detail: 'a guarded file changed where goalpost-guard cannot see the diff; use Edit so it can',
924 after: [command.slice(0, 200)],
925 category: first?.[1],
926 })
927 }
928
929 return found.filter(finding => options.categories.has(finding.category ?? CATEGORY_OF[finding.rule]))
930}
931
932export function categoryOf(finding: GoalpostFinding): GoalpostCategory {
933 return finding.category ?? CATEGORY_OF[finding.rule]
934}
935types/index.d.ts 93 lines1/**
2 * goalpost-guard's contract: what it finds, what it decided, and the values it
3 * keeps in `$.state` for its band slot and its pane.
4 */
5
6/**
7 * The families of change a person can turn on or off, one `userConfig`
8 * switch each.
9 */
10export type GoalpostCategory = 'tests' | 'suppressions' | 'config' | 'ci' | 'coverage' | 'bypass'
11
12/**
13 * Each way of lowering the bar the detectors know.
14 */
15export type GoalpostRule =
16 | 'tests.assertions'
17 | 'tests.removed'
18 | 'tests.skipped'
19 | 'tests.focused'
20 | 'tests.loosened'
21 | 'tests.expected'
22 | 'tests.retargeted'
23 | 'tests.snapshots'
24 | 'tests.narrowed'
25 | 'suppressions'
26 | 'config'
27 | 'ci'
28 | 'coverage'
29 | 'bypass'
30 | 'shell-edit'
31
32/**
33 * One thing a change does to lower the bar. `before` and `after` hold a few
34 * sample lines (removed and added) for the pane; `category` is set only when
35 * it differs from the rule's own (a shell edit takes its file's).
36 */
37export type GoalpostFinding = {
38 rule: GoalpostRule
39 title: string
40 detail: string
41 file?: string
42 before?: string[]
43 after?: string[]
44 category?: GoalpostCategory
45}
46
47/**
48 * What happened to a finding: held and refused, let through by the person
49 * (once, or from now on for that file), let through by an exception made
50 * earlier, or only reported (`warn` mode).
51 */
52export type GoalpostDecision = 'blocked' | 'allowed' | 'excepted' | 'warned'
53
54/**
55 * One finding and its decision, as the pane lists it and a commit trailer
56 * reports it.
57 */
58export type GoalpostEntry = {
59 id: string
60 at: number
61 tool: string
62 finding: GoalpostFinding
63 decision: GoalpostDecision
64 /** Why the person allowed it, or what they told Claude to do instead. */
65 note?: string
66 /** When a `git commit` carried it in a trailer (session clock); an amend of that commit carries it again. */
67 reportedAt?: number
68}
69
70/**
71 * A standing permission: this rule, in this file (or anywhere in the project
72 * when `file` is absent), for this project. Kept in `$.store`, across sessions.
73 */
74export type GoalpostException = {
75 project: string
76 rule: GoalpostRule
77 file?: string
78 note: string
79 at: number
80}
81
82declare module 'claude-code' {
83 interface PluginState {
84 'goalpost-guard': {
85 ledger: GoalpostEntry[]
86 exceptions: GoalpostException[]
87 tab: 'ledger' | 'exceptions' | 'rules'
88 muted: GoalpostCategory[]
89 isPaused: boolean
90 }
91 }
92}
93