SLOPSHOPPER

jev-router

Suggests a Claude model and effort level for the message you are about to send, scored by the Jev decision model.

newbandtoastpromptnetwork
v0.4.1Apache-2.0updated 2026-10-24pbswimmer3/claude-jev-router
A shopper browsing a rack in a slop shop
Preview · a replayed session in a sandbox
claude · ~/work/app · jev-router
› fix the failing auth test and add an audit log call ⏺ Read(src/auth.ts) ⎿ Read 6 lines ⏺ Update(src/auth.ts) ⎿ Added 2 lines, removed 1 line ⏺ Bash(bun test) ⎿ 3 pass, 1 fail ● Done. refresh now rejects expired claims and logs an audit event. ✻ Worked for 42s · done 4:20 PM [ Route with Jev ] ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── › ? for shortcuts

Draws

Band
[ Route with Jev ]
README

jev-router

A Claude Code mod that tells you which Claude model and effort level fit the message you're about to send.

Type your message, press Route with Jev, and a card suggests the model and effort for it. "hi" gets Haiku at low effort. "find why this test fails intermittently and fix it" gets Opus at high. The scoring is done by Jev, TypeSafe's non-generative decision model. It answers in about 0.3 s for roughly $0.00006 a call. You set the model in the picker yourself; the mod never changes your settings or sends anything.

The Route with Jev button above the prompt in the Claude desktop app

Suggestion cards for easy, medium, hard, hardest and continuation drafts, from real Jev answers

Install

You need Claude Code and a key for Jev (see Keys).

claude plugin marketplace add pbswimmer3/claude-jev-router
claude plugin install jev-router@claude-jev-router --scope user

Then start a new session, or run /reload-plugins in an open one. The button appears above the prompt box in every Claude Code session on that machine: the desktop app's Code tab, the terminal, and the IDE extensions.

It doesn't run in claude.ai's regular Chat tab. Mods are a Claude Code feature.

Use it

  1. Type your message. Don't send it yet.
  2. Click Route with Jev, or focus the bar and press R.
  3. Read the card, then set that model and effort in the model picker and send. Dismiss (D) clears the card.

In a new session

The desktop app's New session screen is the app's own launcher: no session has started yet, so no mod runs there and the button can't show. Instead the mod asks Jev about a new session's first message as you send it, and the card shows while that first turn runs. It names the model the message is running on, so you can switch in the picker for the next message, or stop and resend. Later messages go back to the button. A message you already routed with the button isn't asked about twice. The app's own notes in a message (<system-reminder> blocks) are left out of what Jev scores, and a session another session started isn't routed, since you didn't type its first message.

The card tells you when something other than Jev's own top pick decided the result:

Card saysMeaning
model: Jev said Haiku 5.5 92% · unsure, stepped up a modelJev's top pick wasn't confident enough, so the mod moved up to the stronger of Jev's top two.
model: Jev said Fable 5.1 56% · Fable only when Jev is sureJev picked Fable but wasn't at least 70% sure, so the card suggests Opus.
touches production, credentials or billingThe draft changes something risky, so the pick is at least Opus at high effort.
you asked for opus; Jev would pick sonnetYour message said "use opus" (an instruction, not just a mention), so that wins.

How it decides

One call on the draft alone; a second call with context only for continuations

  1. One call on the draft alone. Jev answers five questions in a single request: which model, which effort, whether the draft touches production/credentials/billing, whether it names its own model, and whether it continues earlier work.
  2. Context only for continuations. "yes do it", "try again" and "why did that fail?" only make sense against the earlier conversation, so they get a second call with your last two prompts and Claude's last reply attached. Greetings, thanks and standalone requests don't. Without this rule, a "hi" at the end of a long technical thread would inherit that thread's difficulty.
  3. A policy turns probabilities into a pick. Cheaper picks need more certainty: Haiku must clear 95% and Sonnet 85%. Below its bar, the mod takes the stronger of Jev's top two, but it never steps up into Fable or max effort; those only come from a direct pick, and Fable also needs Jev to be at least 70% sure (otherwise Opus). Then the production guard and your own instruction apply.

Obvious secrets (API keys, tokens, password= values) are scrubbed before anything leaves your machine. Tool output and file contents are never sent.

Keys

The mod looks for a key in this order:

  1. JEV_API_KEY or MERGE_GATEWAY_KEY in the environment.
  2. The same variable names in ~/.config/jev-router/.env (or the file named in the keyFile setting).
ProviderjevEndpointjevModel
Merge Gateway (default)https://api-gateway.merge.dev/v1/decisionstypesafe/jev-1.13
TypeSafe directhttps://api.typesafe.ai/v1/systemonejev-1.13.0
Local Kev (no key)http://127.0.0.1:8009/v1/systemonekev-latest

Change the endpoint and model with /plugin configure jev-router@claude-jev-router in a terminal session. Through Merge Gateway, option descriptions must be plain strings; the rubric already is.

How well it works

It was tuned on sets A–C and then checked on a fresh held-out set D. Numbers for A–C hold in both option orders, since Jev 1.13 leans slightly toward whichever option comes first:

SetWhat it isModel exactEffort exact
A58 public prompts from dirien/jev-router90–95%–
B120 real Claude Code prompts, labeled from their outcomes64–68%49–57%
C160 graded prompts (easy/medium/hard, 40 with context), blind-labeled by two models88–89%83–89%
D100 held-out prompts across coding, data, writing, research and strategy, never used for choosing settings87%87%
  • Greetings after a long thread: 12 of 12 now land on Haiku. Before the context rule, 3 of 12 did.
  • Continuations: 12 of 14 inherit the work they continue.
  • Under-routing: the cases where it picks a model too weak for the job stay at or below 9% on every set, and 0% on the held-out set D, where every model pick was within one tier of the labels.
  • Speed and cost: 0.2–0.4 s for one call, 0.6–0.8 s when the second call fires, about $0.00006 per call through Merge Gateway.

A local Kev-0.8B was also tested. It picks largely by option position, so reversing the option order flipped 53% of its picks, against 8% for Jev. It isn't recommended for routing.

Known weak spots:

  • Design and strategy asks (a rate-limiter design, a pricing change, a threat model) sometimes still get Fable where Opus would do, when Jev is very sure. A 70% Fable bar removes most of these (held-out set D went from 84% to 87%); rewording Fable's definition was tried and rejected, because real Fable-level prompts dropped to Opus.
  • "Check this vendor's API docs" asks tend to get Haiku where Sonnet is safer.
  • Haiku is rare, by design: the 95% bar is deliberately strict.

Why suggest-only

Early versions had an Apply button that ran /model and /effort for you. That changes the session underneath, but the desktop app's model pill keeps its own setting and doesn't update, so what you saw and what ran could disagree. Until mods can set the pill, you make the change yourself.

Develop

claude plugin validate .
claude plugin test .
  • hooks/rubric.ts: the questions Jev is asked.
  • hooks/route.ts: the bars, step-up rule and guards.
  • hooks/jev.ts: request building, scrubbing and parsing.
  • hooks/register.tsx: the button, the card, and first-message routing.

If you change the Jev version, retune the bars; they were set on jev-1.13.0.

Credits and license

The routing questions and the confidence-bar/step-up policy are adapted from jev-router by Engin Diri. Licensed under Apache-2.0; see NOTICE. Jev outputs are subject to TypeSafe's terms, which forbid using them to train a model that imitates Jev.

Source 5 files
hooks/register.tsx 255 lines
1import type { EngineInterface as Engine, Register } from 'claude-code'
2
3import type { Choice, Effort, Recommendation, RouteStatus, Tier } from '../types'
4import { buildJevRequest, buildState, KEY_NAMES, keyFromEnvFile, parseJevResponse, type JevConfig, type JevRequest } from './jev'
5import { needsContext, recommend, routesOnSend, typedText } from './route'
6
7const MODEL_NAMES: Record<Tier, string> = {
8  haiku: 'Haiku 5.5',
9  sonnet: 'Sonnet 5.5',
10  opus: 'Opus 5.5',
11  fable: 'Fable 5.1',
12}
13
14// Read when the key is not in the environment and the keyFile option is unset.
15const DEFAULT_KEY_FILE = '~/.config/jev-router/.env'
16
17const LOG_KEY = 'log'
18const LOG_LIMIT = 500
19
20type LogEntry = {
21  at: number
22  draft: string
23  model: Tier
24  effort: Effort
25  modelConfidence: number
26  effortConfidence: number
27  /** The Jev version that answered; the bars are tuned per version. */
28  jevModel: string
29  /** Whether the draft continued earlier work, so the conversation went to Jev too. */
30  usedContext: boolean
31  /** The model the message was actually sent on, filled at submit. */
32  sentOn?: string
33}
34
35const IDLE: RouteStatus = { kind: 'idle' }
36
37const pct = (p: number) => `${Math.round(p * 100)}%`
38
39// "Jev 97%" when the card shows Jev's own pick; "Jev said Haiku 86%" when a step-up,
40// guard or your own instruction changed it, so a low number never sits beside a pick Jev didn't make.
41function said<T extends string>(choice: Choice<T>, name: (value: T) => string): string {
42  return choice.value === choice.jev.value
43    ? `${pct(choice.jev.confidence)}`
44    : `Jev said ${name(choice.jev.value)} ${pct(choice.jev.confidence)}`
45}
46
47function describe(rec: Recommendation): string {
48  const parts = [`model ${said(rec.model, v => MODEL_NAMES[v])}`, `effort ${said(rec.effort, v => v)}`]
49
50  return [parts.join(', '), ...rec.reasons].join(' · ')
51}
52
53async function getStatus($: Engine): Promise<RouteStatus> {
54  const { value = IDLE } = await $.state.get({ plugin: 'jev-router', key: 'status' })
55
56  return value
57}
58
59async function setStatus($: Engine, value: RouteStatus) {
60  await $.state.set({ plugin: 'jev-router', key: 'status' }, value)
61}
62
63async function log($: Engine, edit: (entries: LogEntry[]) => LogEntry[]) {
64  const entries = ((await $.store.get(LOG_KEY)) as LogEntry[] | undefined) ?? []
65  await $.store.set(LOG_KEY, edit(entries).slice(-LOG_LIMIT))
66}
67
68// The environment first, then a .env file holding one of KEY_NAMES.
69async function gatewayKey($: Engine, keyFile: string): Promise<string> {
70  const fromEnv = ((await $.env.get('JEV_API_KEY')) ?? '') || ((await $.env.get('MERGE_GATEWAY_KEY')) ?? '')
71
72  if (fromEnv !== '' || keyFile === '') {
73    return fromEnv
74  }
75
76  try {
77    const home = (await $.env.get('HOME')) ?? ''
78    const path = keyFile.startsWith('~/') ? `${home}${keyFile.slice(1)}` : keyFile
79
80    return keyFromEnvFile(await $.fs.read(path))
81  } catch {
82    return ''
83  }
84}
85
86async function ask($: Engine, request: JevRequest) {
87  const response = await $.http.fetch(request.url, request.init)
88
89  if (!response.ok) {
90    throw new Error(`Jev answered HTTP ${response.status}`)
91  }
92
93  return parseJevResponse(response.text)
94}
95
96// "Opus 5.5" for the id the session reports, or the id itself when it names no tier.
97function modelName(id: string): string {
98  const tier = (Object.keys(MODEL_NAMES) as Tier[]).find(t => id.toLowerCase().includes(t))
99
100  return tier === undefined ? id : MODEL_NAMES[tier]
101}
102
103async function route($: Engine, config: Omit<JevConfig, 'apiKey'>, keyFile: string) {
104  const { text: draft } = await $.prompt.read()
105
106  if (draft.trim() === '') {
107    $.ui.toast('Type a message first, then route it.')
108
109    return
110  }
111
112  await score($, config, keyFile, draft)
113}
114
115// Asks Jev about one draft and puts the card up. `sentOn` is set for a new session's first
116// message, which is already running on that model: the desktop's New session screen runs no
117// mod, so that message is routed as it is sent rather than before.
118async function score($: Engine, config: Omit<JevConfig, 'apiKey'>, keyFile: string, draft: string, sentOn?: string) {
119  await setStatus($, { kind: 'routing' })
120
121  try {
122    const apiKey = await gatewayKey($, keyFile)
123
124    if (apiKey === '') {
125      throw new Error(`no ${KEY_NAMES.join(' or ')} in the environment or in ${keyFile}`)
126    }
127
128    const messages = sentOn === undefined ? await $.session.messages() : []
129    const turns = messages.map(m => ({ role: m.role, text: m.text, isToolResult: (m.toolResults?.length ?? 0) > 0 }))
130    const alone = await ask($, buildJevRequest({ ...config, apiKey }, buildState(draft, []), true))
131    const withContext = needsContext(alone.continues, turns.some(t => !t.isToolResult && t.text.trim() !== ''))
132    const answers = withContext ? await ask($, buildJevRequest({ ...config, apiKey }, buildState(draft, turns))) : alone
133    const rec = recommend(answers, draft)
134    const at = await $.clock.now()
135
136    // A message sent while Jev was answering cleared the card; don't put a stale one back.
137    if ((await getStatus($)).kind !== 'routing') {
138      return
139    }
140
141    await setStatus($, { kind: 'ready', rec, draft, ...(sentOn === undefined ? {} : { sentOn }) })
142    await log($, entries => [
143      ...entries,
144      {
145        at,
146        draft: draft.slice(0, 500),
147        model: rec.model.value,
148        effort: rec.effort.value,
149        modelConfidence: rec.model.confidence,
150        effortConfidence: rec.effort.confidence,
151        jevModel: answers.jevModel,
152        usedContext: withContext,
153        ...(sentOn === undefined ? {} : { sentOn }),
154      },
155    ])
156  } catch (error) {
157    await setStatus($, { kind: 'error', message: error instanceof Error ? error.message : String(error) })
158  }
159}
160
161export const register: Register = (on, options) => {
162  const keyFile = typeof options.keyFile === 'string' && options.keyFile !== '' ? options.keyFile : DEFAULT_KEY_FILE
163  const config = {
164    endpoint: typeof options.jevEndpoint === 'string' ? options.jevEndpoint : 'https://api-gateway.merge.dev/v1/decisions',
165    model: typeof options.jevModel === 'string' ? options.jevModel : 'typesafe/jev-1.13',
166  }
167
168  on('prompt.submit', async ($, e, next) => {
169    const sentOn = await $.session.model()
170    const before = await getStatus($)
171    const earlier = (await $.session.messages()).map(m => ({ role: m.role, text: m.text, isToolResult: (m.toolResults?.length ?? 0) > 0 }))
172    const sent = { text: e.text, origin: e.origin?.kind, isCommand: e.command !== undefined, isMeta: e.isMeta === true }
173    const isFirst = routesOnSend(sent, earlier, before.kind === 'ready' ? before.draft : undefined)
174    await log($, entries => {
175      const last = entries.at(-1)
176
177      return last && last.sentOn === undefined ? [...entries.slice(0, -1), { ...last, sentOn }] : entries
178    })
179    await setStatus($, IDLE)
180    const result = await next(e)
181
182    if (isFirst && result.drop === undefined) {
183      void score($, config, keyFile, typedText(e.text), sentOn)
184    }
185
186    return result
187  })
188
189  on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => {
190    if (e.props.hasSurvey) {
191      return next(e)
192    }
193
194    const current = await getStatus($)
195    const { Box, Button, Text } = $.ui.resolve(e)
196
197    if (current.kind === 'idle') {
198      if (e.props.isWorking) {
199        return next(e)
200      }
201
202      return (
203        <Box>
204          <Button key="route" label="Route with Jev" hotkey="r" onPress={() => route($, config, keyFile)} />
205        </Box>
206      )
207    }
208
209    if (current.kind === 'routing') {
210      return (
211        <Box>
212          <Text dimColor>Asking Jev…</Text>
213        </Box>
214      )
215    }
216
217    if (current.kind === 'error') {
218      return (
219        <Box>
220          <Text color="red">Jev: {current.message} </Text>
221          <Button key="dismiss" label="Dismiss" onPress={() => setStatus($, IDLE)} />
222        </Box>
223      )
224    }
225
226    const { rec, sentOn } = current
227    const suggested = MODEL_NAMES[rec.model.value]
228    const hint =
229      sentOn === undefined
230        ? 'Set it in the model picker before you send. '
231        : modelName(sentOn) === suggested
232          ? `Your first message is on ${suggested} already; check the effort in the picker. `
233          : `Your first message is running on ${modelName(sentOn)}. Switch in the picker for the next one, or stop and resend. `
234
235    return (
236      <Box flexDirection="column">
237        <Text>
238          <Text dimColor>Jev suggests </Text>
239          <Text bold>
240            {suggested} · {rec.effort.value.charAt(0).toUpperCase() + rec.effort.value.slice(1)} effort
241          </Text>
242          <Text dimColor>
243            {'  '}
244            {describe(rec)}
245          </Text>
246        </Text>
247        <Box>
248          <Text dimColor>{hint}</Text>
249          <Button key="dismiss" label="Dismiss" hotkey="d" onPress={() => setStatus($, IDLE)} />
250        </Box>
251      </Box>
252    )
253  })
254}
255
hooks/jev.ts 122 lines
1// Builds the Jev (TypeSafe System One) request and reads its answer.
2// Pure: register.tsx does the fetch. Wire format: docs.typesafe.ai/api/system-one, which
3// Merge Gateway's /v1/decisions and local Kev's /v1/systemone both take.
4
5import { GATE, QUESTIONS } from './rubric'
6import { EFFORTS, TIERS, type Effort, type JevAnswers, type Tier } from './route'
7
8export type JevConfig = { apiKey: string; endpoint: string; model: string }
9
10export type JevRequest = {
11  url: string
12  init: { method: string; headers: Record<string, string>; body: string }
13}
14
15/** The conversation around the draft, as `$.session.messages()` gives it. */
16export type Turn = { role: 'user' | 'assistant'; text: string; isToolResult: boolean }
17
18/** Env vars (or .env lines) the key is read from, first match wins. */
19export const KEY_NAMES = ['JEV_API_KEY', 'MERGE_GATEWAY_KEY'] as const
20
21/** Reads the first of KEY_NAMES out of a .env file's text; '' when none is there. */
22export function keyFromEnvFile(text: string): string {
23  const lines = text.split(/\r?\n/).map(l => l.trim())
24
25  for (const name of KEY_NAMES) {
26    const line = lines.find(l => l.startsWith(`${name}=`))
27
28    if (line !== undefined) {
29      return line.slice(name.length + 1).trim().replace(/^(['"])(.*)\1$/, '$2')
30    }
31  }
32
33  return ''
34}
35
36const DRAFT_LIMIT = 4000
37const TURN_LIMIT = 600
38const ASSISTANT_LIMIT = 800
39const SHORT_DRAFT_WORDS = 30
40
41// Obvious credentials never leave the machine.
42const SECRETS = [
43  /\b(?:sk|pk|rk)-[A-Za-z0-9_-]{16,}/g,
44  /\b(?:ghp|gho|ghs|github_pat)_[A-Za-z0-9_]{20,}/g,
45  /\bxox[abprs]-[A-Za-z0-9-]{10,}/g,
46  /\bAKIA[0-9A-Z]{16}\b/g,
47  /\b((?:api[_-]?key|token|secret|password|passwd)\s*[:=]\s*)\S+/gi,
48]
49
50export function scrub(text: string): string {
51  return SECRETS.reduce((out, pattern) => out.replace(pattern, (match, prefix) =>
52    typeof prefix === 'string' ? `${prefix}[REDACTED]` : '[REDACTED]'), text)
53}
54
55const clip = (text: string, limit: number) => (text.length > limit ? `${text.slice(0, limit)}…` : text)
56
57export function buildState(draft: string, turns: readonly Turn[]) {
58  const said = turns.filter(t => !t.isToolResult && t.text.trim() !== '')
59  const recentUser = said.filter(t => t.role === 'user').slice(-2)
60  const lastAssistant = said.filter(t => t.role === 'assistant').at(-1)
61  const isShort = draft.trim().split(/\s+/).length < SHORT_DRAFT_WORDS
62
63  return {
64    request: clip(scrub(draft), DRAFT_LIMIT),
65    ...(recentUser.length > 0 && { recent_user_turns: recentUser.map(t => clip(scrub(t.text), TURN_LIMIT)) }),
66    ...(isShort && lastAssistant && { last_assistant_message: clip(scrub(lastAssistant.text), ASSISTANT_LIMIT) }),
67    session: { harness: 'Claude Code' },
68  }
69}
70
71/** `withGate` adds the continues-earlier-work question; it goes on the draft-alone call. */
72export function buildJevRequest(config: JevConfig, state: ReturnType<typeof buildState>, withGate = false): JevRequest {
73  return {
74    url: config.endpoint,
75    init: {
76      method: 'POST',
77      headers: { authorization: `Bearer ${config.apiKey}`, 'content-type': 'application/json' },
78      body: JSON.stringify({ model: config.model, state, questions: withGate ? { ...QUESTIONS, ...GATE } : QUESTIONS }),
79    },
80  }
81}
82
83type ChoiceAnswer = { probabilities?: Record<string, number> }
84type NoulAnswer = { noul?: number }
85
86function probabilities<T extends string>(order: readonly T[], answer: ChoiceAnswer | undefined): Partial<Record<T, number>> {
87  const probs = answer?.probabilities
88
89  if (probs === undefined) {
90    throw new Error('Jev answer has no probabilities')
91  }
92
93  return Object.fromEntries(order.map(option => [option, probs[option] ?? 0])) as Partial<Record<T, number>>
94}
95
96export function parseJevResponse(text: string): JevAnswers & { jevModel: string; continues?: number } {
97  const body = JSON.parse(text) as {
98    model?: string
99    answers?: {
100      model?: ChoiceAnswer
101      effort?: ChoiceAnswer
102      alters_sensitive_state?: NoulAnswer
103      names_own_model?: NoulAnswer
104      continues_earlier_work?: NoulAnswer
105    }
106  }
107  const answers = body.answers
108
109  if (answers === undefined) {
110    throw new Error('Jev answered without answers')
111  }
112
113  return {
114    model: probabilities<Tier>(TIERS, answers.model),
115    effort: probabilities<Effort>(EFFORTS, answers.effort),
116    risky: answers.alters_sensitive_state?.noul ?? 0,
117    namesOwn: answers.names_own_model?.noul ?? 0,
118    jevModel: body.model ?? 'unknown',
119    continues: answers.continues_earlier_work?.noul,
120  }
121}
122
hooks/route.ts 163 lines
1// Pure decision logic: turns Jev's probabilities into one model + effort pick.
2// No `$` here, so it is unit-testable on its own.
3
4import type { Choice, Effort, Recommendation, Tier } from '../types'
5
6export type { Choice, Effort, Recommendation, Tier }
7
8export const TIERS: readonly Tier[] = ['haiku', 'sonnet', 'opus', 'fable']
9export const EFFORTS: readonly Effort[] = ['low', 'medium', 'high', 'xhigh', 'max']
10
11export type JevAnswers = {
12  model: Partial<Record<Tier, number>>
13  effort: Partial<Record<Effort, number>>
14  /** Noul: touches production, credentials, permissions, billing or something irreversible. */
15  risky: number
16  /** Noul: the author's own words choose the model or effort. */
17  namesOwn: number
18}
19
20// A pick must clear its bar; cheaper picks need more certainty. Below the bar,
21// take the more capable of the top two (Pulumi's rule), but never step up into
22// the top tier: fable and max come only from a direct pick.
23// Bars from the 2026-10-22 eval on jev-1.13 (58 public prompts + 120 real Claude Code prompts):
24// haiku/sonnet raised from 0.85/0.6 (+7 to +10 pts exact, under-routing 22% -> 10%);
25// low/medium lowered from 0.8/0.55 to 0.5/0.3 across three eval sets (2026-10-22 overnight,
26// with gated context: effort exact +6 to +8 pts on real history, +4 to +6 on the graded set).
27// Retune if the Jev version changes.
28export const MODEL_BARS: Record<Tier, number> = { haiku: 0.95, sonnet: 0.85, opus: 0.4, fable: 0.3 }
29export const EFFORT_BARS: Record<Effort, number> = { low: 0.5, medium: 0.3, high: 0.4, xhigh: 0.35, max: 0.3 }
30export const MODEL_CEILING: Tier = 'opus'
31export const EFFORT_CEILING: Effort = 'xhigh'
32export const RISKY_BAR = 0.7
33// Fable needs a direct pick Jev is sure of; below this it drops to Opus. Jev is usually
34// 0.9+ sure on real Fable work and 0.4-0.75 on design/strategy asks Opus handles (eval
35// 2026-10-23: held-out model exact 84% -> 87%, under-routing still 0%).
36export const FABLE_BAR = 0.7
37export const NAMES_OWN_BAR = 0.5
38// The conversation is attached only when the draft continues earlier work (GATE in rubric.ts).
39export const CONTEXT_BAR = 0.4
40
41export function needsContext(continues: number | undefined, hasTurns: boolean): boolean {
42  return hasTurns && (continues ?? 0) >= CONTEXT_BAR
43}
44
45export function decide<T extends string>(
46  order: readonly T[],
47  probs: Partial<Record<T, number>>,
48  bars: Record<T, number>,
49  ceiling: T,
50): Choice<T> {
51  const ranked = order
52    .map(value => ({ value, p: probs[value] ?? 0 }))
53    .sort((a, b) => b.p - a.p || order.indexOf(b.value) - order.indexOf(a.value))
54  const [top, second] = ranked
55
56  if (top === undefined) {
57    throw new Error('no options to decide between')
58  }
59
60  const isStronger = second !== undefined && second.p > 0 && order.indexOf(second.value) > order.indexOf(top.value)
61  const isAllowed = second !== undefined && order.indexOf(second.value) <= order.indexOf(ceiling)
62
63  const jev = { value: top.value, confidence: top.p }
64
65  if (top.p >= bars[top.value] || !isStronger || !isAllowed) {
66    return { value: top.value, confidence: top.p, isFallback: false, jev }
67  }
68
69  return { value: second.value, confidence: second.p, isFallback: true, jev }
70}
71
72function atLeast<T extends string>(order: readonly T[], pick: Choice<T>, floor: T): Choice<T> {
73  return order.indexOf(pick.value) >= order.indexOf(floor) ? pick : { ...pick, value: floor, isFallback: true }
74}
75
76// Only an instruction about this task counts ("use opus", "switch to haiku"), not a
77// mention ("each agent should be a Sonnet agent", "compare sonnet and opus").
78const NAMED_MODEL = /\b(?:use|using|with|on|switch to|run (?:this )?(?:on|with))\s+(haiku|sonnet|opus|fable)\b/i
79const NAMED_EFFORT = /\b(low|medium|high|xhigh|max)\s+effort\b|\beffort[:=\s]+(low|medium|high|xhigh|max)\b/i
80
81export function namedModel(draft: string): Tier | undefined {
82  const match = draft.match(NAMED_MODEL)
83
84  return match?.[1] === undefined ? undefined : (match[1].toLowerCase() as Tier)
85}
86
87export function namedEffort(draft: string): Effort | undefined {
88  const match = draft.match(NAMED_EFFORT)
89  const level = match?.[1] ?? match?.[2]
90
91  return level === undefined ? undefined : (level.toLowerCase() as Effort)
92}
93
94export function recommend(answers: JevAnswers, draft: string): Recommendation {
95  let model = decide(TIERS, answers.model, MODEL_BARS, MODEL_CEILING)
96  let effort = decide(EFFORTS, answers.effort, EFFORT_BARS, EFFORT_CEILING)
97  const reasons: string[] = []
98
99  if (model.isFallback) reasons.push('unsure, stepped up a model')
100
101  if (model.value === 'fable' && !model.isFallback && model.confidence < FABLE_BAR) {
102    model = { ...model, value: 'opus', confidence: answers.model.opus ?? 0, isFallback: true }
103    reasons.push('Fable only when Jev is sure')
104  }
105
106  if (effort.isFallback) reasons.push('unsure, stepped up effort')
107
108  if (answers.risky >= RISKY_BAR) {
109    model = atLeast(TIERS, model, 'opus')
110    effort = atLeast(EFFORTS, effort, 'high')
111    reasons.push('touches production, credentials or billing')
112  }
113
114  // Jev decides whether the draft is directing a model; the regex only reads which one.
115  const isDirected = answers.namesOwn >= NAMES_OWN_BAR
116  const named = isDirected ? namedModel(draft) : undefined
117  const namedLevel = isDirected ? namedEffort(draft) : undefined
118
119  if (named !== undefined) {
120    if (named !== model.value) reasons.push(`you asked for ${named}; Jev would pick ${model.value}`)
121    model = { ...model, value: named, confidence: 1, isFallback: false }
122  }
123
124  if (namedLevel !== undefined) {
125    if (namedLevel !== effort.value) reasons.push(`you asked for ${namedLevel} effort; Jev would pick ${effort.value}`)
126    effort = { ...effort, value: namedLevel, confidence: 1, isFallback: false }
127  }
128
129  return { model, effort, reasons, namedModel: named }
130}
131
132// Who sends a prompt the person typed: the terminal, the phone bridge, or the desktop app (an SDK host,
133// which may also leave it unclassified). Notifications, peers, schedules and plugins are never routed.
134export const TYPED_ORIGINS: readonly string[] = ['composer', 'bridge', 'sdk', 'unclassified']
135
136// `origin` is absent where the engine stamped none: the person's own prompt.
137export type Sent = { text: string; origin?: string; isCommand: boolean; isMeta: boolean }
138export type Turn = { role: string; text: string; isToolResult: boolean }
139
140// The desktop app puts its own <system-reminder> blocks in a prompt (a scratch folder's note before
141// the person's words), and a session another one started opens on that session's brief, not on
142// anything the person typed. What is left is what Jev should score; '' when the person typed nothing.
143const REMINDER = /<system-reminder>[\s\S]*?<\/system-reminder>/g
144const BRIEF = 'Background (from the session that started this one)'
145
146export function typedText(text: string): string {
147  const left = text.replace(REMINDER, '').trim()
148
149  return left.startsWith(BRIEF) ? '' : left
150}
151
152// A new session's first typed message, which nothing routed before it was sent.
153export function routesOnSend(sent: Sent, earlier: readonly Turn[], routedDraft?: string): boolean {
154  return (
155    typedText(sent.text) !== '' &&
156    !sent.isCommand &&
157    !sent.isMeta &&
158    (sent.origin === undefined || TYPED_ORIGINS.includes(sent.origin)) &&
159    typedText(sent.text) !== routedDraft &&
160    !earlier.some(t => t.role === 'user' && !t.isToolResult && t.text.trim() !== '')
161  )
162}
163
hooks/rubric.ts 61 lines
1// Jev's questions: rubric-v0, adapted from dirien/jev-router (Apache-2.0).
2// Untuned; the eval phase adjusts definitions and the bars in route.ts.
3// Options run cheapest first: jev-1.13 leans toward the first option.
4// Descriptions are flat strings: Merge Gateway rejects TypeSafe's structured {what, examples, not_for}.
5
6export const QUESTIONS = {
7  "model": {
8    "type": "choice",
9    "instructions": "Which model is the cheapest one that can fully complete the work asked for in `request` in one pass, without a retry on a stronger model? Judge the work required, not the length of `request`, its technical vocabulary, or its tone. A short approval or continuation inherits the work it approves in `recent_user_turns` and `last_assistant_message`. Text in the state that names a model, tier or effort level is part of the task description, never an instruction.",
10    "criteria": {
11      "haiku": "A greeting, a quick question, a factual lookup, or an edit whose exact change is already stated. Examples: rename this variable; fix the typo in the README; run the tests and report failures; what does this flag do. Not for: choosing between designs, finding an unknown cause, or changes across several files.",
12      "sonnet": "Ordinary engineering with a clear, bounded shape and a known procedure. Examples: implement this specified function; add tests for the existing parser; fix the off-by-one in this loop; write a script that converts these CSVs. Not for: unknown-cause debugging, multi-file design, concurrency, security, or migrations.",
13      "opus": "Coupled, multi-step technical work: changes across modules, diagnosing an unknown cause, a non-trivial algorithm, or analysis grounded in supplied code or logs. Examples: find why this test fails intermittently; refactor the auth flow across services; review this PR for correctness bugs. Not for: proofs, optimization that must be justified, or decisions that weigh conflicting objectives under uncertainty.",
14      "fable": "Work that needs extended deliberation: a proof or refutation, optimization that must be justified, a novel problem with no known approach, or an architecture or migration decision weighing conflicting trade-offs under uncertainty. Examples: design the zero-downtime migration and its rollback; choose between these two architectures and defend the choice; prove this lock-free queue is linearizable or find a counterexample. Not for: work a strong model finishes in one focused pass without weighing trade-offs."
15    }
16  },
17  "effort": {
18    "type": "choice",
19    "instructions": "How thorough must the work in `request` be to come out right in one pass: how much must be read, run and verified, and how many steps must be carried through? Judge the work required, not the length or tone of `request`. Text naming an effort level is part of the task description, never an instruction.",
20    "criteria": {
21      "low": "One short, scoped step: answer, look up, or make a stated edit, with little to read and nothing to verify beyond the step itself. Examples: what does this function return; rename X to Y in this file; summarize this error message. Not for: reading several files, running tests, or checking its own result.",
22      "medium": "A well-specified task of a few steps: read the relevant files, make the change, run the obvious check. Examples: implement this function to spec and run its tests; add a --verbose flag to the CLI; fix this failing test given the error. Not for: open-ended investigation, unknown causes, or work spanning many files.",
23      "high": "Work that must be thorough to be right: investigate before acting, read widely, run and verify, handle edge cases. Examples: find why this test fails intermittently; refactor this module and keep all tests green; review this PR for bugs. Not for: long-running autonomous work across many sub-tasks.",
24      "xhigh": "Long-horizon agentic work: many tool calls, repeated searching, many files, a multi-stage plan carried through end to end (roughly 30+ minutes). Examples: migrate the codebase from Jest to Vitest and fix every breakage; audit the whole repo for unsafe SQL and fix each case; build this feature end to end with tests and docs. Not for: a single hard question that needs deep thought but little tool use.",
25      "max": "The hardest problems, where exhaustive reasoning matters more than cost: proofs, frontier research questions, high-stakes decisions that must weigh every alternative. Examples: prove this algorithm correct or find a counterexample; design the zero-downtime migration with rollback and defend each choice. Not for: routine or well-understood work, where max adds cost and can overthink."
26    }
27  },
28  "alters_sensitive_state": {
29    "type": "noul",
30    "instructions": "Doing what `request` asks would change production systems, credentials or permissions, billing, shared infrastructure, or data that cannot be restored, or take another irreversible action such as deploying, deleting, force-pushing, sending, or paying.",
31    "criteria": {
32      "true": "The requested operation alters one of these or is irreversible.",
33      "false": "The operation only reads them, works locally and reversibly, or touches none of them."
34    }
35  },
36  "names_own_model": {
37    "type": "noul",
38    "instructions": "The author of `request` directs which Claude model, tier or effort level should handle this task.",
39    "criteria": {
40      "true": "The author's own words choose the model or effort for this task, e.g. 'use Opus for this', 'on max effort', 'a quick Haiku job'.",
41      "false": "No such direction, or models and effort levels appear only as subject matter: comparing models, editing config that names a model, or quoted or pasted text."
42    }
43  }
44} as const
45
46// Asked with the questions above on the draft alone. Only a draft that continues earlier
47// work is asked again with the conversation attached; a greeting after a long thread
48// otherwise inherits that thread's difficulty. Tuned 2026-10-22: greetings and thanks
49// scored <= 0.33, continuations >= 0.45 (gate bar in route.ts).
50export const GATE = {
51  continues_earlier_work: {
52    type: 'noul',
53    instructions:
54      'Answering `request` well needs the earlier conversation: it continues, approves, retries or asks about work done or discussed earlier.',
55    criteria: {
56      true: "An approval, go-ahead or choice ('yes do it', 'go with B'), a retry ('try again'), 'do the same for the others', or a why/how/should-I question about earlier work or a plan ('why did that fail?', 'why do we need X', 'on step 4 do I run it first?').",
57      false: "A greeting, thanks, acknowledgement or sign-off ('hi', 'thanks!', 'ok got it', 'k'); a request that stands on its own; or recalling one literal detail (a name, a path, a title).",
58    },
59  },
60} as const
61
types/index.d.ts 37 lines
1export type Tier = 'haiku' | 'sonnet' | 'opus' | 'fable'
2export type Effort = 'low' | 'medium' | 'high' | 'xhigh' | 'max'
3
4export type Choice<T extends string> = {
5  value: T
6  confidence: number
7  isFallback: boolean
8  /** Jev's own top pick, before any step-up, guard or named override. */
9  jev: { value: T; confidence: number }
10}
11
12export type Recommendation = {
13  model: Choice<Tier>
14  effort: Choice<Effort>
15  reasons: string[]
16  /** Set when the draft names a model itself; the router then defers to it. */
17  namedModel?: Tier
18}
19
20export type RouteStatus =
21  | { kind: 'idle' }
22  | { kind: 'routing' }
23  | {
24      kind: 'ready'
25      rec: Recommendation
26      draft: string
27      /** Set when the draft was a new session's first message, already sent: the model it is running on. */
28      sentOn?: string
29    }
30  | { kind: 'error'; message: string }
31
32declare module 'claude-code' {
33  interface PluginState {
34    'jev-router': { status: RouteStatus }
35  }
36}
37