SLOPSHOPPER

qsm-web

Repeatable web research: one loop (searxng search, keyword rounds, Context7 for a library, Tavily when the keywords will not come, one page in a browser, the…

newprocess
A shopper browsing a rack in a slop shop
README

qsm-web

Repeatable web research for the agent: one loop, one skill per tool, and the qsm-web CLI beneath them.

Why it exists

An agent that needs something from the internet guesses its keywords once, trusts the first page, answers for the wrong version, and stops at the first wall or hands it to whoever is nearest. The harness's own WebSearch and WebFetch are enough to look; qsm-web replaces that default with a loop that walks the same steps for the same question, with sharper tools beside it and the person reached directly when a wall needs a human. It takes the stance that the web is a source to be read with its sources named, that every skipped or blocked step is said out loud, and that a tool someone else wrote well is installed, not rewritten. It refuses to be a crawler, an account automation tool, or a second copy of an upstream skill. When the harness's own search walks this loop and clears its own walls, it has no reason to exist.

What you get

KindNameWhat it does
Skillweb-ladderRuns the loop: searxng search, up to three keyword rounds, Context7 for a library, Tavily when the keywords will not come, one page in a browser, the person on a wall.
SkillsearxngSearches through your own SearXNG instance and narrows by engine, category, time range or language.
SkillbrowserboxOpens one page in a real browser, scripts a sequence, and hands a captcha to the person.
SkillopensrcPuts a library's source on disk at the version the project pins, to read.
Agentweb-researchRuns the loop away from the main context, handles a wall itself, and returns an answer with sources.
CLIqsm-webopensrc, searxng, browserbox, doctor. See USAGE.md.

Context7 and Tavily are upstream: install puts their skills in place as they are.

Install

claude plugin install qsm-web@qsm-marketplace
bun plugins/qsm-web/scripts/install.ts
npx ctx7 setup --claude              # interactive: Context7 sign-in and skill
qsm-secrets set tavily-api-key       # the Tavily key, never in the conversation
qsm-web doctor

install.ts compiles qsm-web into $QSM_BIN_DIR (default ~/.local/bin), installs opensrc, shot-scraper, the Tavily CLI (tvly) and Tavily's skills when missing, and prints the two interactive steps for you. It does not set up search or the browser image. See INSTALL.md.

Use

Ask the agent: "look up how React hooks handle cleanup", or hand a question to web-research.

qsm-web searxng search "io uring vs epoll" --time-range year
qsm-web opensrc get react --project .
name=$(qsm-web browserbox start --json | jq -r .data.name)
qsm-web browserbox fetch https://example.test --in "$name" --html
qsm-web browserbox takeover "$name" 0 --open --reason "captcha"
qsm-web browserbox stop "$name"

--json prints one outcome envelope. Every verb and flag is in USAGE.md.

Caveats

  • Search needs an instance: qsm-web searxng setup or QSM_SEARXNG_INSTANCE.
  • opensrc get of a package spec needs opensrc on PATH.
  • Only the browser step needs docker or podman. The first browserbox start builds an image (minutes, about 1 GB).
  • The live view has no password. Its port is loopback only: never publish it elsewhere.
  • A login is filled only by the main agent with you present, from Bitwarden (rbw or bw) through fill-secret, never from the conversation. The web-research agent stops at a login.
  • The captcha solve step (mouse) is not in the browser driver yet, so a captcha goes straight to the takeover until it lands.
  • It reads pages. It is not a crawler or an account automation tool.
Source 3 files
hooks/register.tsx 11 lines
1// qsm-web's function-hooks module.
2import type { On } from "claude-code";
3import { registerNoun } from "./noun.generated.ts";
4
5export function register(on: On): void {
6  // Adds $.web: one method per qsm-web verb that has a JSON form, generated by
7  // lib/generate-noun.ts. Never hand-edit noun.generated.ts or types/index.d.ts; run the generator
8  // after changing qsm.json.
9  registerNoun(on);
10}
11
hooks/noun.generated.ts 133 lines
1// Generated by lib/generate-noun.ts from qsm.json. Do not hand-edit; run
2// `bun lib/generate-noun.ts plugins/<name>` again after changing qsm.json.
3import type { Caught, EngineInterface, On } from "claude-code";
4import type { Web } from "../types/index.d.ts";
5
6/** How one option becomes argv: a positional value (`flag` null), a bare
7 * boolean flag, or a flag and its value; a list repeats it per element. */
8export interface VerbArg {
9  readonly name: string;
10  readonly flag: string | null;
11  readonly boolean?: true;
12}
13
14export interface VerbRow {
15  readonly words: readonly string[];
16  readonly args: readonly VerbArg[];
17  readonly passthrough?: true;
18}
19
20export const verbs: Readonly<Record<string, VerbRow>> = {
21  opensrcGet: { words: ["qsm-web","opensrc","get"], args: [{ name: "target", flag: null }, { name: "project", flag: "--project" }] },
22  doctor: { words: ["qsm-web","doctor"], args: [] },
23  searxngSearch: { words: ["qsm-web","searxng","search"], args: [{ name: "query", flag: null }, { name: "numResults", flag: "--num-results" }, { name: "engines", flag: "--engines" }, { name: "category", flag: "--category" }, { name: "timeRange", flag: "--time-range" }, { name: "language", flag: "--language" }, { name: "safesearch", flag: "--safesearch" }, { name: "instance", flag: "--instance" }] },
24  browserboxStart: { words: ["qsm-web","browserbox","start"], args: [{ name: "out", flag: "--out" }, { name: "dryRun", flag: "--dry-run", boolean: true }] },
25  browserboxLs: { words: ["qsm-web","browserbox","ls"], args: [{ name: "dryRun", flag: "--dry-run", boolean: true }] },
26  browserboxStop: { words: ["qsm-web","browserbox","stop"], args: [{ name: "name", flag: null }, { name: "all", flag: "--all", boolean: true }, { name: "dryRun", flag: "--dry-run", boolean: true }] },
27  browserboxFetch: { words: ["qsm-web","browserbox","fetch"], args: [{ name: "url", flag: null }, { name: "in", flag: "--in" }, { name: "html", flag: "--html", boolean: true }, { name: "screenshot", flag: "--screenshot", boolean: true }, { name: "cookies", flag: "--cookies", boolean: true }, { name: "settle", flag: "--settle" }, { name: "timeout", flag: "--timeout" }, { name: "keep", flag: "--keep", boolean: true }, { name: "dryRun", flag: "--dry-run", boolean: true }] },
28  browserboxScript: { words: ["qsm-web","browserbox","script"], args: [{ name: "name", flag: null }, { name: "file", flag: "--file" }, { name: "steps", flag: "--steps" }, { name: "dryRun", flag: "--dry-run", boolean: true }] },
29  browserboxTakeover: { words: ["qsm-web","browserbox","takeover"], args: [{ name: "name", flag: null }, { name: "tab", flag: null }, { name: "reason", flag: "--reason" }, { name: "timeout", flag: "--timeout" }, { name: "open", flag: "--open", boolean: true }, { name: "dryRun", flag: "--dry-run", boolean: true }] },
30};
31
32/** One verb's argv: its words, each option in declaration order, the
33 * passthrough rest after `--`, and `--json` last. */
34export function argvOf(row: VerbRow, options: Readonly<Record<string, unknown>> = {}): string[] {
35  const argv = [...row.words];
36  for (const arg of row.args) {
37    const value = options[arg.name];
38    const values = Array.isArray(value) ? value : [value];
39    for (const one of values) {
40      if (one === undefined || one === false) {
41        continue;
42      }
43      argv.push(...(arg.flag === null ? [] : [arg.flag]), ...(arg.boolean ? [] : [String(one)]));
44    }
45  }
46  const rest = options.rest;
47  if (row.passthrough && Array.isArray(rest)) {
48    argv.push("--", ...rest.map(String));
49  }
50  argv.push("--json");
51  return argv;
52}
53
54/** The binary prints one outcome envelope on stdout under --json:
55 * { code, exit, reason, message, warnings, data }. Data on ok; otherwise the
56 * code and message become the error. A stale or misbehaving binary that
57 * prints something else (not JSON, or JSON with no numeric `exit`) throws
58 * loud here instead of a raw JSON.parse SyntaxError or a silent `undefined`. */
59export function unwrap(
60  argv: readonly string[],
61  result: { readonly exitCode: number; readonly stdout: string; readonly stderr: string },
62  label = argv.slice(0, 2).join(" "),
63): unknown {
64  const text = result.stdout.trim();
65  const stderr = result.stderr.trim().slice(0, 200);
66  const cause = `(exit ${result.exitCode}): ${stderr.length > 0 ? stderr : "no stderr"}`;
67  if (text.length === 0) {
68    throw new Error(`${argv.join(" ")}: printed no envelope ${cause}`);
69  }
70  let parsed: unknown;
71  try {
72    parsed = JSON.parse(text);
73  } catch {
74    parsed = undefined;
75  }
76  if (typeof parsed !== "object" || parsed === null || typeof (parsed as { exit?: unknown }).exit !== "number") {
77    throw new Error(`${argv.join(" ")}: stdout is not an outcome envelope ${cause}: ${text.slice(0, 200)}`);
78  }
79  const envelope = parsed as { code: string; exit: number; message: string; data: unknown };
80  if (envelope.exit !== 0) {
81    throw new Error(`${label}: ${envelope.code}: ${envelope.message}`);
82  }
83  return envelope.data;
84}
85
86/** What every `.catch` does first: a toast and a log for the person. The toast says `fixedReason`
87 * when the cause may carry something the toast must not show, else the cause; the cause goes to
88 * the log and comes back for the caller's own policy. */
89export function failure(
90  $: EngineInterface,
91  event: string,
92  error: Caught["error"],
93  fixedReason?: string,
94): string {
95  const said = error.message ?? error.kind;
96  $.ui.toast(`qsm-web ERROR: ${event}: ${(fixedReason ?? said).slice(0, 120)}`);
97  $.ui.log(`${event} failed (${error.kind}): ${said}`);
98  return said;
99}
100
101/** Adds $.web to the engine. Call once from the plugin's own
102 * `register(on, options)` — never call the engine.create hook's own
103 * `next(e)` result from anywhere but here. */
104export function registerNoun(on: On): void {
105  on("engine.create", async (_$, e, next) => {
106    const beneath = await next(e);
107    const run = async (row: VerbRow, options?: Readonly<Record<string, unknown>>): Promise<unknown> => {
108      const argv = argvOf(row, options);
109      let result: Awaited<ReturnType<typeof beneath.process.run>>;
110      try {
111        result = await beneath.process.run(argv);
112      } catch (error) {
113        // A spawn in a deleted directory fails as ENOENT naming the binary; name the directory. A
114        // check that cannot run leaves the spawn's own error as the cause.
115        const gone = await beneath.session
116          .cwd()
117          .then(async (cwd) => ((await beneath.fs.exists(cwd)) ? null : cwd))
118          .catch(() => null);
119        if (gone !== null) {
120          throw new Error(`${row.words.join(" ")}: the session's working directory ${gone} no longer exists; cd to one that does`);
121        }
122        throw error;
123      }
124      return unwrap(argv, result, row.words.join(" "));
125    };
126    const methods = Object.entries(verbs).map(([method, row]) => [
127      method,
128      (options?: Readonly<Record<string, unknown>>) => run(row, options),
129    ]);
130    return { ...beneath, web: Object.fromEntries(methods) as Web };
131  });
132}
133
types/index.d.ts 64 lines
1// Generated by lib/generate-noun.ts from qsm.json. Do not hand-edit; run
2// `bun lib/generate-noun.ts plugins/<name>` again after changing qsm.json.
3
4export type Web = {
5  opensrcGet(options: {
6    target: string;
7    project?: string;
8  }): Promise<unknown>;
9  doctor(): Promise<unknown>;
10  searxngSearch(options: {
11    query: (string)[];
12    numResults?: number;
13    engines?: string;
14    category?: string;
15    timeRange?: "day" | "week" | "month" | "year";
16    language?: string;
17    safesearch?: "0" | "1" | "2";
18    instance?: string;
19  }): Promise<unknown>;
20  browserboxStart(options: {
21    out?: string;
22    dryRun?: boolean;
23  }): Promise<unknown>;
24  browserboxLs(options: {
25    dryRun?: boolean;
26  }): Promise<unknown>;
27  browserboxStop(options: {
28    name?: string;
29    all?: boolean;
30    dryRun?: boolean;
31  }): Promise<unknown>;
32  browserboxFetch(options: {
33    url: string;
34    in?: string;
35    html?: boolean;
36    screenshot?: boolean;
37    cookies?: boolean;
38    settle?: number;
39    timeout?: number;
40    keep?: boolean;
41    dryRun?: boolean;
42  }): Promise<unknown>;
43  browserboxScript(options: {
44    name: string;
45    file?: string;
46    steps?: string;
47    dryRun?: boolean;
48  }): Promise<unknown>;
49  browserboxTakeover(options: {
50    name: string;
51    tab: number;
52    reason?: string;
53    timeout?: number;
54    open?: boolean;
55    dryRun?: boolean;
56  }): Promise<unknown>;
57};
58
59declare module 'claude-code' {
60  interface EngineInterface {
61    web: Web;
62  }
63}
64