effort-probe
Measures whether a turn.step effort rewrite preserves the prompt cache: switches one middle step and logs each step's usage
new
Measures whether a turn.step effort rewrite preserves the prompt cache: switches one middle step and logs each step's usage
Measures whether a turn.step effort rewrite preserves the prompt cache. One scripted claude -p conversation of four main-loop requests (three chained Reads, then the answer) runs with this mod loaded; in switch mode the hook rewrites request 1's effort to low, in control mode it rewrites nothing. Each request's usage lands in a JSONL ledger.
mods/effort-probe/run.sh control # baseline
mods/effort-probe/run.sh switch # request 1 at low effort
mods/effort-probe/run.sh switch sonnet # another modelClaude Code 2.1.295, opus[1m] answered by claude-opus-5-5, --effort high.
| request | effort sent (control / switch) | cache_read control | cache_read switch | cache_creation control | cache_creation switch |
|---|---|---|---|---|---|
| 0 | high / high | 21,977 | 23,760 | 37,656 | 35,878 |
| 1 | high / low | 59,633 | 59,638 | 3,644 | 3,644 |
| 2 | high / high | 63,277 | 63,282 | 148 | 148 |
| 3 | high / high | 63,425 | 63,430 | 143 | 143 |
The switched request and the switch-back request read the whole prefix from cache and wrote only the new turn, the same as the control, so a turn.step effort rewrite is cache-safe on Opus 5.5 on this build. Request 0 differs only because the two first requests ran concurrently and each wrote its own entry. The debug log records request ids, not bodies, so the wire shape (per-message effort system message or top-level field) is not observed directly.
Re-run after a Claude Code upgrade before trusting effort-phase with per-step switching; its max_switches_per_turn option is the fallback if the numbers change.
claude plugin validate mods/effort-probe
claude plugin test mods/effort-probe
mods/typecheck.sh mods/effort-probehooks/register.ts 47 lines1import type { Register } from 'claude-code'
2
3// One row per main-loop model request. `effortIn` is what the engine was about to send; `effortOut` is what this
4// hook sent. The usage counts are what the API reported for that request, so a prefix rewrite shows up as
5// cache_read collapsing and cache_creation jumping on the switched step relative to its neighbours.
6type Row = {
7 index: number
8 effortIn: unknown
9 effortOut: unknown
10 model: string
11 answeredBy: string | null
12 usage: unknown
13 toolUses: string[]
14}
15
16const rows: Row[] = []
17let mode = 'switch'
18let outPath = ''
19
20export const register: Register = (on) => {
21 on('session.start', async ($, e, next) => {
22 mode = (await $.env.get('EFFORT_PROBE_MODE')) ?? 'switch'
23 const out = await $.env.get('EFFORT_PROBE_OUT')
24 const id = await $.session.id()
25 outPath = out ?? `${await $.session.cwd()}/effort-probe-${mode}-${id}.jsonl`
26 return next(e)
27 })
28
29 on('turn.step', async function* ($, e, next) {
30 if (e.agentId !== undefined) return yield* next(e)
31 const switched = mode === 'switch' && e.index === 1
32 const effortOut = switched ? 'low' : e.effort
33 const result = yield* next(switched ? { ...e, effort: 'low' } : e)
34 rows.push({
35 index: e.index,
36 effortIn: e.effort,
37 effortOut,
38 model: e.model,
39 answeredBy: result.usage?.model ?? null,
40 usage: result.usage,
41 toolUses: result.toolUses.map((t) => t.name),
42 })
43 await $.fs.write(outPath, rows.map((r) => JSON.stringify(r)).join('\n') + '\n')
44 return result
45 })
46}
47