SLOPSHOPPER

ruflo-ruvector

Self-learning vector database via npx ruvector@0.2.25 — HNSW, adaptive LoRA embeddings, code-graph clustering, hooks routing, brain/SONA, 103 MCP tools. As a…

newguardcommand
★ 74,184v0.3.4MITupdated 2026-10-07ruvnet/ruflo/plugins/ruflo-ruvector
A shopper browsing a rack in a slop shop
Preview · a replayed session in a sandbox
claude · ~/work/app · ruflo-ruvector
› fix the failing auth test and add an audit log call ⏺ Read(src/auth.ts) ⎿ Read 6 lines ⏺ Update(src/auth.ts) ⎿ Added 2 lines, removed 1 line ⏺ Bash(bun test) ⎿ 3 pass, 1 fail ● Done. refresh now rejects expired claims and logs an audit event. ✻ Worked for 42s · done 4:20 PM › /ruvector-mod ⎿ ruflo-ruvector: /ruvector-mod status ⎿ ruflo-ruvector: /ruvector-mod scan <text> ⎿ ruflo-ruvector: /ruvector-mod tools ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── › ? for shortcuts
README

ruflo-ruvector

Self-learning vector database powered by ruvector@0.2.25 — HNSW, Adaptive LoRA embeddings, hooks-based intelligence, SONA self-optimizing patterns, brain (collective knowledge), and 91 MCP tools (verified via ruvector mcp tools).

Pinned version: this plugin targets ruvector@0.2.25. Earlier 0.1.x versions are missing several commands (brain, route, sona); some legacy docs referenced 2.x features that do not exist on npm. Always invoke with the pin.

Overview

Wraps the ruvector npm package as a Ruflo plugin, providing vector embedding, semantic search, code-graph clustering, hyperbolic projection, self-learning hooks, and SONA / Brain diagnostics. ruvector's Rust backend delivers sub-millisecond queries and 52,000+ inserts/sec.

Prerequisites

# Required
npm install ruvector@0.2.25

# Optional add-ons (install as needed)
npm install ruvector-onnx-embeddings-wasm   # required for `embed text` to work
npm install @ruvector/pi-brain              # required for `brain` subcommands
npm install @ruvector/ruvllm                # required for `sona` subcommands (JS fallback)

Run a health check:

npx -y ruvector@0.2.25 doctor

Installation

claude --plugin-dir plugins/ruflo-ruvector

MCP Integration (91 Tools)

Register with the pinned version:

claude mcp add ruvector -- npx -y ruvector@0.2.25 mcp start

Key tool categories: hooks routing, AST analysis, diff classification, coverage routing, graph clustering, security scanning, RAG context, brain knowledge, SONA learning.

Agents

AgentModelRole
vector-engineersonnetEmbedding, HNSW indexing, code-graph clustering, hyperbolic projection, hooks routing, brain/SONA

Skills

SkillUsageDescription
vector-setup/vector-setup [--full]First-run installer: pins ruvector@0.2.25, adds ONNX/Brain/SONA/router add-ons, registers MCP, runs doctor
vector-embed/vector-embed <text>ONNX embeddings (384-dim) via embed text
vector-cluster/vector-cluster <files...>Spectral/Louvain code-graph clustering via hooks graph-cluster
vector-hyperbolic/vector-hyperbolic <text>Standard ONNX embed + Poincare projection in user code

Commands (/vector slash command)

The full surface is documented in commands/vector.md (80+ subcommands). Quick reference:

# Embedding
/vector embed <text>                 # ruvector embed text "<text>"
/vector embed-adaptive <text>        # ruvector embed text "<text>" --adaptive --domain code
/vector embed-file <path>            # read file, pass content as text
/vector embed-benchmark              # ruvector embed benchmark

# Database lifecycle
/vector db create <path>             # ruvector create <path> -d 384 -m cosine
/vector db stats <path>              # ruvector stats <path>
/vector insert <db> <json>           # ruvector insert <db> <json>
/vector search <db> <vector-json>    # ruvector search <db> -v <json> -k N
/vector export <db>                  # ruvector export <db> -o backup.json
/vector import <file>                # ruvector import <file> -d <database>

# RVF cognitive containers (45 example stores)
/vector rvf create|ingest|query|status|segments|derive|compact|export|examples|download

# GNN + attention (real native bindings)
/vector gnn info|layer|search|compress
/vector attention list|compute|benchmark|hyperbolic|info

# Code intelligence (hooks)
/vector ast <file>                   # ruvector hooks ast-analyze <file>
/vector hooks ast-complexity <files...>
/vector hooks coverage-route <file> | coverage-suggest <files...>
/vector hooks rag-context <query>
/vector hooks route|route-enhanced|suggest-context
/vector cluster <files...>           # ruvector hooks graph-cluster <files>
/vector hooks security-scan <files...>
/vector hooks diff-analyze|diff-classify|diff-similar [commit]
/vector hooks remember|recall <query>
/vector hooks coedit-record|coedit-suggest|error-record|error-suggest
/vector hooks trajectory-begin|trajectory-step|trajectory-end

# Native + workers
/vector native list|run <type>|benchmark|compare
/vector workers triggers|presets|phases|dispatch|status|...

# Collective intelligence
/vector brain status|search|share|list|drift|partition|transfer|sync|page  (needs @ruvector/pi-brain)
/vector sona status|info|stats|patterns|train|export                       (needs @ruvector/ruvllm)
/vector llm models|embed|benchmark|info                                    (needs @ruvector/ruvllm)

# Identity + edge compute (pi network)
/vector identity generate|show|export|import
/vector edge status|balance|tasks|join|dashboard

# Server / decompile / demo / system
/vector server [-p 8080] [-g 50051]
/vector decompile <npm-pkg-or-file-or-url>
/vector demo --basic | --gnn | --graph
/vector doctor | info | benchmark | install | setup

End-to-End Example: store + search project files

# 0. One-time setup
/vector-setup

# 1. Create a database
npx -y ruvector@0.2.25 create project.db -d 384 -m cosine

# 2. Embed every TypeScript source file (loop — no built-in --batch)
mkdir -p .vec
for f in $(find src -name '*.ts'); do
  npx -y ruvector@0.2.25 embed text "$(cat "$f")" -o ".vec/${f//\//_}.json"
done

# 3. Insert all embeddings (assumes a JSON array of {id, vector, metadata})
jq -s '[.[] | {id: input_filename, vector: .vector}]' .vec/*.json > corpus.json
npx -y ruvector@0.2.25 insert project.db corpus.json

# 4. Search by query embedding
QV=$(npx -y ruvector@0.2.25 embed text "JWT refresh-token rotation" --output -)
npx -y ruvector@0.2.25 search project.db -v "$QV" -k 5

# 5. Inspect index health
npx -y ruvector@0.2.25 stats project.db

For an alternative store format with lineage tracking, replace steps 1–3 with:

npx -y ruvector@0.2.25 rvf create project.rvf
npx -y ruvector@0.2.25 rvf ingest project.rvf < corpus.json
npx -y ruvector@0.2.25 rvf query project.rvf

Capabilities (ruvector@0.2.25, verified)

FeatureCLINotes
HNSW searchsearch <db> -v ... -k N~0.045ms latency
Adaptive LoRA embeddingsembed text "..." --adaptive --domain codeLoRA-tuned
Distance metrics`create <path> -m cosine\euclidean\dot`set at create time
RVF cognitive containers`rvf createingestqueryderivecompact`45 example stores via rvf examples
Attention mechanismsattention listDotProduct, MultiHead, Flash, Hyperbolic, Linear, MoE, GraphRoPe, EdgeFeatured, DualSpace, LocalGlobal
GNN ops`gnn layersearchcompress`multi-head attention layers, differentiable search, tensor compression
Code-graph clusteringhooks graph-cluster <files>spectral / Louvain
Diff embeddings`hooks diff-analyzediff-classifydiff-similar`git-aware
Coverage-aware routing`hooks coverage-routecoverage-suggest`test-gap-aware
RAG contexthooks rag-context "query"works in CLI and MCP
AST analysis`hooks ast-analyzeast-complexity`symbols, complexity, parse time
Self-learning loop`hooks rememberrecallcoedit-*error-*trajectory-*`persistent intelligence
Native workers`native listrun <securityanalysislearning>`no external deps
Background workers`workers dispatchstatuspresetsphases`first run installs agentic-flow
Decompile npm/JSdecompile <target>inspect upstream packages
Serverserver -p 8080HTTP/gRPC mode
Demo`demo --basic--gnn--graph`interactive tutorial
Identity (pi key)`identity generateshowexportimport`for brain + edge
Edge compute`edge statusbalancetasksjoin`distributed, rUv currency

Known limitations / bugs in 0.2.25

IssueDetailWorkaround
ONNX runtime missingembed text → ONNX WASM files not bundlednpm i ruvector-onnx-embeddings-wasm (see /vector-setup)
optimizeSelf-reports "not yet shipped in this release"none — track upstream issue 401
hooks force-learnTypeError intel.tick is not a functionrun a real trajectory via trajectory-begin/step/end
hooks graph-mincutCannot read properties of undefined (reading 'length')use hooks graph-cluster
hooks git-churnFails outside a git reporun from inside the repo
benchmarkSome installs fail with Missing field 'dimensions'use attention benchmark or gnn search benchmarking
cluster (top-level)Status: Coming Soonuse hooks graph-cluster
compare, top-level index, midstream, embed --file/--batch/--glob/--model poincareDon't existsee commands/vector.md for replacements

Self-Learning Hooks

# Full pretrain pipeline + agent generation
npx -y ruvector@0.2.25 hooks init --pretrain --build-agents quality

# Smart agent routing (positional task!)
npx -y ruvector@0.2.25 hooks route "implement OAuth flow"
npx -y ruvector@0.2.25 hooks route-enhanced "fix CVE-2025-1234"

# Code analysis (positional file!)
npx -y ruvector@0.2.25 hooks ast-analyze src/module.ts
npx -y ruvector@0.2.25 hooks diff-analyze HEAD
npx -y ruvector@0.2.25 hooks coverage-route src/module.ts
npx -y ruvector@0.2.25 hooks security-scan src/

Brain (Collective Knowledge)

npm install @ruvector/pi-brain   # required dependency

npx -y ruvector@0.2.25 brain status
npx -y ruvector@0.2.25 brain search "authentication patterns"
npx -y ruvector@0.2.25 brain list
npx -y ruvector@0.2.25 brain drift code   # knowledge drift for a domain

SONA (Self-Optimizing Neural Architecture)

npx -y ruvector@0.2.25 sona status
npx -y ruvector@0.2.25 sona patterns "auth refactor"
npx -y ruvector@0.2.25 sona stats

Performance

OperationLatencyNotes
HNSW search~0.045ms8,800x vs ONNX inference
Memory cache~0.01ms40,000x vs ONNX inference
Insert52,000+/secRust backend (@ruvector/core)
Memory per vector~50 bytesEfficient storage

Known Caveats

  • ONNX runtime not bundled by default — embed text will report ONNX WASM files not bundled until you install ruvector-onnx-embeddings-wasm.
  • No --file, --batch, --glob, --namespace, --k, --task, --model poincare flags — these were in older docs but never shipped in 0.2.25. See agents/vector-engineer.md for the replacement table.
  • brain requires @ruvector/pi-brain — install separately.
  • sona requires @ruvector/ruvllm — install separately (the native binding is not always present in the npm tarball).
  • Top-level cluster is "Coming Soon" — for actual clustering use hooks graph-cluster <files>.
  • compare, midstream, top-level index subcommands do not exist.

Architecture Decisions

Smoke test

bash plugins/ruflo-ruvector/scripts/smoke.sh
# Expected: "11 passed, 0 failed"

Related Plugins

  • ruflo-agentdb — HNSW storage backend in AgentDB
  • ruflo-intelligence — SONA pattern learning integration
  • ruflo-knowledge-graph — Graph RAG for multi-hop retrieval
  • ruflo-rag-memory — Simple semantic search via ruvector

License

MIT

As a mod

Function-hook mod (ADR-445, pattern of ruflo-agentdb). It loads from hooks/hooks.json → hooks/register.ts.

A write guard for the vector store and shared brain: hooks_remember, hooks_compress_store, hooks_learn, rvf_ingest, brain_share, brain_transfer and memory_store refuse secrets (a shared brain is the worst place for one). Calls to any tool of the ruvector MCP server are counted.

  • Command: /ruvector-mod answers locally with no model call. Verbs: status, scan <text>, tools (which ruvector tools are connected).
  • Status file: .claude-flow/ruvector-mod/status.json ({version, updatedMs, ...counters}), written at session start and as counters change.
  • Safety: no network, no process spawning; it only uses tools already connected.
  • Option guard (on by default, off to disable): a tighten-only tool.call guard. A call to one of the tools above whose input holds a key, token, private key or password is denied. The reason never repeats the secret.
  • Test: claude plugin validate plugins/ruflo-ruvector, claude plugin test plugins/ruflo-ruvector, bash plugins/ruflo-ruvector/scripts/smoke.sh.
Source 7 files
hooks/register.ts 79 lines
1import type { Hook, Register } from 'claude-code'
2
3import { answer } from './command'
4import { verdict } from './guard'
5import { readOptions, type ModOptions } from './options'
6import { newStats, STATUS_PATH, statusText, type Stats } from './status'
7import { isOwned, isWriter, shortName } from './tools'
8
9const FLUSH_EVERY_MS = 5000
10
11type Dollar = Parameters<Hook<'session.start'>>[0]
12
13/** Everything one session of the mod keeps: its settings, counters, the project root and when the status file was last written. */
14type Session = { readonly opts: ModOptions; readonly stats: Stats; root?: string; flushedMs: number }
15
16async function flush($: Dollar, s: Session): Promise<void> {
17  if (s.root === undefined) return
18  try {
19    const now = await $.clock.now()
20    s.flushedMs = now
21    await $.fs.write(`${s.root}/${STATUS_PATH}`, statusText(s.stats, s.opts, now))
22  } catch {
23    /* the status file is a courtesy */
24  }
25}
26
27/** Counts a call to one of this plugin's tools; the file is rewritten at most every few seconds. */
28async function note($: Dollar, s: Session): Promise<void> {
29  s.stats.seen++
30  try {
31    if ((await $.clock.now()) - s.flushedMs >= FLUSH_EVERY_MS) await flush($, s)
32  } catch {
33    /* counting never fails a tool call */
34  }
35}
36
37/**
38 * ruflo-ruvector as a mod (ADR-445): a tighten-only guard that keeps secrets out of this plugin's write tools, `/ruvector-mod`, and a status
39 * file the console reads. No network, no process: only tools already connected.
40 */
41export const register: Register = (on, options) => {
42  const s: Session = { opts: readOptions(options), stats: newStats(), flushedMs: 0 }
43
44  on('session.start', async ($, e, next) => {
45    const result = await next(e)
46    try {
47      s.root = (await $.session.root()) as string | undefined
48    } catch {
49      /* no project root: the mod still answers, it just writes no status file */
50    }
51    try {
52      await $.command.register({ name: 'ruvector-mod', description: 'ruflo-ruvector mod: status, scan <text>, tools' })
53    } catch {
54      /* a name taken by another plugin must not stop the mod */
55    }
56    await flush($, s)
57    return result
58  })
59
60  on('tool.call', async ($, e, next) => {
61    if (!isOwned(e.tool) && !isWriter(e.tool)) return next(e)
62    const reason = s.opts.guard ? verdict(e.tool, e) : undefined
63    if (reason === undefined) {
64      await note($, s)
65      return next(e)
66    }
67    s.stats.blocked++
68    s.stats.lastBlocked = shortName(e.tool)
69    await flush($, s)
70    return { deny: reason }
71  })
72
73  /** `/ruvector-mod` (a name the plugin's own commands and skills do not use). */
74  on('command.run', { command: 'ruvector-mod' }, async ($, e) => {
75    const text = await answer(typeof e.args === 'string' ? e.args : '', { opts: s.opts, stats: s.stats, tools: () => $.tool.list() })
76    return { text }
77  })
78}
79
hooks/command.ts 37 lines
1import type { ToolInfo } from 'claude-code'
2
3import type { ModOptions } from './options'
4import { secretsIn } from './screen'
5import type { Stats } from './status'
6import { isOwned, shortName } from './tools'
7
8/** `/ruvector-mod` is answered locally and takes no model turn. */
9export type CommandDeps = { readonly opts: ModOptions; readonly stats: Stats; readonly tools: () => Promise<readonly ToolInfo[]> }
10
11const HELP = ['/ruvector-mod status', '/ruvector-mod scan <text>', '/ruvector-mod tools'].join('\n')
12
13export async function answer(args: string, deps: CommandDeps): Promise<string> {
14  const [verb = '', ...rest] = args.trim().split(/\s+/)
15  const arg = rest.join(' ')
16  const { opts, stats } = deps
17
18  if (verb === '' || verb === 'help') return HELP
19
20  if (verb === 'status') {
21    return `guard ${opts.guard ? 'on' : 'off'} · calls seen ${stats.seen} · blocked ${stats.blocked}${stats.lastBlocked ? ` · last blocked ${stats.lastBlocked}` : ''}`
22  }
23
24  if (verb === 'scan') {
25    if (arg === '') return 'usage: /ruvector-mod scan <text>'
26    const found = secretsIn(arg)
27    return found.length ? `The guard would refuse a call carrying that (${found.join(', ')}).` : 'Nothing found: the guard would let that through.'
28  }
29
30  if (verb === 'tools') {
31    const names = (await deps.tools()).filter(t => t.mcp && isOwned(t.name)).map(t => shortName(t.name))
32    return names.length ? `${names.length} connected: ${[...new Set(names)].sort().join(', ')}` : 'None of this plugin\'s tools is connected. Connect the MCP server and try again.'
33  }
34
35  return `Unknown: ${verb}\n${HELP}`
36}
37
hooks/guard.ts 31 lines
1import { hasSecret } from './screen'
2import { isWriter } from './tools'
3import { textsOf } from './screen'
4export { textsOf }
5
6/** This plugin's namespaces. A `memory_store` outside them is another plugin's write: still screened, but its refusal must not claim it. */
7const OWN_NS = /^(?:vector|hyperbolic|ruvector)/i
8const namespaceOf = (input: unknown): string | undefined => {
9  const top = typeof input === 'object' && input !== null ? (input as Record<string, unknown>) : {}
10  const inner = typeof top.input === 'object' && top.input !== null ? (top.input as Record<string, unknown>) : {}
11  return [top.namespace, inner.namespace].find((n): n is string => typeof n === 'string')
12}
13const foreignStore = (tool: string, input: unknown): boolean => /(?:^|__)memory_store$/.test(tool) && !OWN_NS.test(namespaceOf(input) ?? '')
14/** The refusal for a secret in another plugin's `memory_store`: what was found and which namespace, never an owner and never content. */
15function foreignRefusal(input: unknown): string {
16  const ns = namespaceOf(input)
17  const label = (ns ?? '').replace(/[^\w.:-]/g, '').slice(0, 40)
18  const where = !ns ? 'no namespace' : label && !hasSecret(ns) && !hasSecret(label) ? `namespace "${label}"` : 'a namespace not shown here'
19  return `ruflo-ruvector: a secret-shaped value (a key, token or password) was found in a memory write; the call targeted ${where}. Store a reference to where it lives, not the value.`
20}
21
22/** The reason a call is refused, or undefined when it may go. Never names or echoes the secret. */
23export function verdict(tool: string, input: unknown): string | undefined {
24  if (!isWriter(tool)) return undefined
25  return textsOf(input).some(hasSecret)
26    ? foreignStore(tool, input)
27      ? foreignRefusal(input)
28      : 'ruflo-ruvector: this call holds what looks like a secret (a key, token or password). Keep secrets out of the vector store and the shared brain; store a reference instead.'
29    : undefined
30}
31
hooks/options.ts 12 lines
1import type { PluginOptions } from 'claude-code'
2
3/** The plugin's `userConfig`, validated: a bad value is the default (guard on). */
4export type ModOptions = { readonly guard: boolean }
5
6// BEGIN SHARED FLAG (generated by scripts/sync-mod-screen.mjs; edit plugins/ruflo-agentdb/hooks/options.ts)
7export const flag = (value: unknown, fallback: boolean) =>
8  value === true || value === 'true' || value === 'on' ? true : value === false || value === 'false' || value === 'off' ? false : fallback
9// END SHARED FLAG
10
11export const readOptions = (options: PluginOptions | undefined): ModOptions => ({ guard: flag((options ?? {}).guard, true) })
12
hooks/status.ts 13 lines
1import type { ModOptions } from './options'
2
3/** Counters the mod keeps for the session and writes to `.claude-flow/ruvector-mod/status.json`. */
4export type Stats = { seen: number; blocked: number; lastBlocked?: string }
5
6export const newStats = (): Stats => ({ seen: 0, blocked: 0 })
7
8export const STATUS_PATH = '.claude-flow/ruvector-mod/status.json'
9
10/** The file's text; `version` lets a reader refuse a shape it does not know. */
11export const statusText = (stats: Stats, opts: ModOptions, nowMs: number): string =>
12  `${JSON.stringify({ version: 1, updatedMs: nowMs, guard: opts.guard, ...stats }, null, 2)}\n`
13
hooks/tools.ts 33 lines
1/** `mcp__<server>__<tool>` into its two halves; tool names never hold a double underscore. */
2export function splitName(name: string): { server: string; tool: string } | undefined {
3  if (!name.startsWith('mcp__')) return undefined
4  const at = name.lastIndexOf('__')
5  return at > 5 ? { server: name.slice(5, at), tool: name.slice(at + 2) } : undefined
6}
7
8const bare = (name: string) => splitName(name)?.tool ?? name
9
10/** The name without its `mcp__<server>__` prefix. */
11export const shortName = bare
12
13/** This plugin's tools: counted in the status file. */
14const OWNED = new Set([
15  'memory_search',
16  'memory_store',
17  'memory_list',
18])
19
20/** The tools that put text somewhere durable or shared. A call through any of them is screened by the guard. */
21const WRITERS = new Set([
22  'hooks_remember',
23  'hooks_compress_store',
24  'hooks_learn',
25  'rvf_ingest',
26  'brain_share',
27  'brain_transfer',
28  'memory_store',
29])
30
31export const isOwned = (name: string) => OWNED.has(bare(name)) || splitName(name)?.server === 'ruvector'
32export const isWriter = (name: string) => WRITERS.has(bare(name))
33
hooks/screen.ts 262 lines
1/**
2 * Pure secret screening for the ruflo-ruvector mod (ADR-445 screen.ts, secrets only). Findings are NAMES only: the matched text is never
3 * returned, logged or counted by value.
4 */
5
6// BEGIN SHARED SCREEN (generated from plugins/ruflo-agentdb/hooks/screen.ts by scripts/sync-mod-screen.mjs; do not edit in a copy)
7export type Rules = readonly (readonly [string, RegExp])[]
8
9/** The secret shapes every mod screens for. A plugin adds its own after these, outside the markers. */
10export const COMMON_SECRETS: Rules = [
11  ['private key', /-----BEGIN [A-Z ]*PRIVATE KEY-----/],
12  ['aws access key', /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/],
13  ['github token', /\b(?:gh[pousr]_[A-Za-z0-9]{30,}|github_pat_[A-Za-z0-9_]{40,})\b/],
14  ['slack token', /\bxox[abprs]-[A-Za-z0-9-]{10,}/],
15  ['slack webhook', /\bhooks\.slack\.com\/services\/T[A-Z0-9]{6,}\/B[A-Z0-9]{6,}\/[A-Za-z0-9]{16,}/],
16  ['google api key', /\bAIza[0-9A-Za-z_-]{35}\b/],
17  ['anthropic or openai key', /\bsk-(?:(?:ant|proj|svcacct|admin)-[A-Za-z0-9_-]{20,}|(?=[A-Za-z]{0,40}\d)[A-Za-z0-9]{32,})/],
18  ['stripe key', /\b[rs]k_live_[A-Za-z0-9]{16,}/],
19  ['npm token', /\bnpm_[A-Za-z0-9]{36}\b/],
20  ['huggingface token', /\bhf_[A-Za-z0-9]{30,}\b/],
21  ['sendgrid key', /\bSG\.[A-Za-z0-9_-]{16,}\.[A-Za-z0-9_-]{16,}/],
22  ['twilio key', /\bSK[0-9a-f]{32}\b/],
23  ['jwt', /\beyJ[A-Za-z0-9_-]{8,}\.eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/],
24  ['bearer token', /\bBearer\s+([A-Za-z0-9._~+/=-]{24,})/],
25  ['database url with credentials', /\b[a-z][a-z0-9+.-]{1,20}:\/\/[^\s:@/]+:[^\s@/]{3,}@[^\s/]+/i],
26]
27
28export const INJECTION: Rules = [
29  ['override instructions', /\b(?:ignore|disregard|forget|override)\b[^.\n]{0,40}\b(?:previous|prior|above|earlier|all|any|system)\b[^.\n]{0,30}\b(?:instructions?|rules?|prompts?|guidelines?)\b/i],
30  ['role reassignment', /\byou are (?:now|no longer)\b|\bact as (?:an? )?(?:unrestricted|jailbroken)\b/i],
31  ['new instructions', /\b(?:new|updated|real) (?:system )?instructions?\s*:/i],
32  ['fake role tags', /<\/?\s*(?:system|assistant|developer|instructions?)\s*>|^\s*(?:system|assistant)\s*:/im],
33  ['concealment', /\bdo not (?:tell|inform|mention|reveal)[^.\n]{0,30}\b(?:user|human|operator)\b/i],
34  ['exfiltration', /\b(?:exfiltrate|send|post|upload)\b[^.\n]{0,50}\b(?:secrets?|credentials?|tokens?|api keys?|\.env)\b/i],
35  ['shell pipe', /\b(?:curl|wget)\b[^|\n]{0,200}\|\s*(?:sudo\s+)?(?:ba|z)?sh\b/i],
36]
37
38// C0/C1 controls (keeping tab and newline), DEL, soft hyphen, combining grapheme joiner, Arabic letter mark, Hangul and Mongolian fillers/separators,
39// zero-width, bidi (overrides and isolates) and invisible-format characters, variation selectors; built with escapes, never raw.
40const INVISIBLE = new RegExp(
41  '[\\u0000-\\u0008\\u000b-\\u001f\\u007f-\\u009f\\u00ad\\u034f\\u061c\\u115f\\u1160\\u17b4\\u17b5\\u180b-\\u180e\\u200b-\\u200f\\u2028-\\u202e\\u2060-\\u206f\\u3164\\ufe00-\\ufe0f\\ufeff\\uffa0\\ufff9-\\ufffb]',
42  'g',
43)
44
45/** Longest input scanned in one pass; a longer one keeps its head and tail halves. One regex pass per rule, so cost stays linear. */
46const MAX_SCAN = 200_000
47
48/** Input bounded to MAX_SCAN characters with invisible characters removed, so none can hide a secret or a phrase. */
49export const bare = (text: string) =>
50  (text.length > MAX_SCAN ? text.slice(0, MAX_SCAN / 2) + '\n' + text.slice(-MAX_SCAN / 2) : text).replace(INVISIBLE, '')
51
52// A value is a secret CANDIDATE only when it is not a reference (env var, call, identifier path, placeholder, secret-manager path) and its
53// shape is random enough: at least two character classes, one of them a digit or symbol, and Shannon entropy of at least 2.5 bits per character.
54const PLACEHOLDER = /placeholder|your[-_ ]|example|changeme|change[-_]?me|redacted|dummy|replace[-_]?me|insert[-_]|\*{3,}|x{5,}|\.{3}|^(?:none|null|undefined|true|false)$/i
55const REFERENCE =
56  /^(?:\$(?:\{[^}]*\}|\(|[A-Za-z_]\w*$)|%[^%]*%$|<[^>]*>$|\{\{|process\.env|os\.environ|env[.[]|import\.meta|System\.getenv|secrets?\.|vault:|op:\/\/|ref\+|arn:|projects\/[^/]+\/secrets\/|gcp:|kms:|aws:|file:)/i
57const CALL = /^[A-Za-z_$][\w$]*(?:\.[A-Za-z_$][\w$]*)*\(|^[A-Za-z_$][\w$]*\[/
58const IDENT_PATH = /^[A-Za-z_$][\w$]*(?:\.[A-Za-z_$][\w$]*)+$/
59const UUID = /^[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12}$/i
60const NAME_LIKE = /^[a-z][a-z0-9]*(?:[-_./][a-z0-9]+){2,}$/
61
62function entropy(v: string): number {
63  const counts = new Map<string, number>()
64  for (const ch of v) counts.set(ch, (counts.get(ch) ?? 0) + 1)
65  let h = 0
66  for (const n of counts.values()) h -= (n / v.length) * Math.log2(n / v.length)
67  return h
68}
69
70/** True when `v` is a name, call, path or placeholder rather than a literal credential. */
71function isReference(v: string): boolean {
72  if (PLACEHOLDER.test(v) || REFERENCE.test(v) || CALL.test(v) || IDENT_PATH.test(v) || URL_NO_CREDS.test(v)) return true
73  return NAME_LIKE.test(v) && v.replace(/\D/g, '').length / v.length < 0.15
74}
75
76export function plausibleSecret(v: string): boolean {
77  if (v.length < 8 || v.length > 256 || /\s/.test(v) || UUID.test(v) || isReference(v)) return false
78  const symbol = /[^A-Za-z0-9]/.test(v)
79  const digit = /\d/.test(v)
80  const classes = [/[a-z]/.test(v), /[A-Z]/.test(v), digit, symbol].filter(Boolean).length
81  return classes >= 2 && (digit || symbol) && entropy(v) >= 2.5
82}
83
84/** Under a secret-named key a literal this long is a secret even with one character class or a UUID shape, unless it is a clear reference. */
85const KEYED_MIN = 20
86const PLACEHOLDER_WORD = /(?:^|[^a-z])(?:your|placeholder|changeme|change[-_]?me|example|redacted|dummy|replace[-_]?me|insert)(?:[^a-z]|$)|\*{3,}|x{5,}|\.{3}|^(?:none|null|undefined|true|false)$/i
87const URL_NO_CREDS = /^[a-z][a-z0-9+.-]{1,20}:\/\/[^\s@]*$/i
88
89/** Three or more lowercase hyphen-separated words (no hex or digit-only run of 8+, few digits), such as my-k8s-secret-name-for-database. */
90function hyphenName(v: string): boolean {
91  const parts = v.split('-')
92  return parts.length >= 3 && parts.every(p => /^[a-z0-9]{2,}$/.test(p) && !/^[0-9a-f]{8,}$/.test(p)) && v.replace(/\D/g, '').length / v.length < 0.15
93}
94
95function keyedSecret(v: string): boolean {
96  if (v.length < KEYED_MIN || v.length > 256 || /\s/.test(v)) return false
97  return !(PLACEHOLDER_WORD.test(v) || REFERENCE.test(v) || CALL.test(v) || IDENT_PATH.test(v) || URL_NO_CREDS.test(v) || (!UUID.test(v) && hyphenName(v)))
98}
99
100const KEY_NAME = /(?:api[_-]?key|secret|token|passw(?:or)?d|passwd|pwd|credential|private[_-]?key|auth(?!or))s?[A-Za-z0-9_-]{0,40}["']?\s*[:=]\s*/gi
101const QUOTED = /(["'\x60])((?:(?!\1)[^\n]){1,256})\1/y
102const BARE_VALUE = /[^\s"'\x60,;]{1,256}/y
103const QUERY_VALUE = /[^\s"'\x60,;&]{1,256}/y
104
105/** A secret-named key assigned a literal value: env style, JSON, YAML, code. Values that are calls, references or placeholders do not count. */
106function assignmentSecret(text: string): boolean {
107  let valueEnd = 0
108  for (const m of text.matchAll(KEY_NAME)) {
109    if (m.index < valueEnd) continue // a key-looking word inside the previous value, such as secretsmanager in an ARN
110    const at = m.index + m[0].length
111    let back = m.index
112    while (back > 0 && m.index - back < 64 && /[A-Za-z0-9_.-]/.test(text.charAt(back - 1))) back--
113    const re = /["'\x60]/.test(text.charAt(at)) ? QUOTED : /[?&]/.test(text.charAt(back - 1)) ? QUERY_VALUE : BARE_VALUE
114    re.lastIndex = at
115    const hit = re.exec(text)
116    const v = hit && (hit[2] ?? hit[0])
117    valueEnd = hit ? at + hit[0].length : at
118    if (v && (plausibleSecret(v) || keyedSecret(v))) return true
119  }
120  return false
121}
122
123/** A password in a URL's userinfo that is not a placeholder such as user:password or ${DB_PASSWORD}. */
124function urlCredential(url: string): boolean {
125  const pass = /^[^:]+:\/\/[^\s:@/]+:([^\s@/]+)@/.exec(url)?.[1]
126  if (!pass || /\$\{|\{\{|%\(|%s/.test(pass)) return false
127  return !/^(?:password|passwd|pass|pwd|secret|changeme|dbpassword|db_password|\$\w*|<.*>|\{.*\}|\*+|x+)$/i.test(pass) && !PLACEHOLDER.test(pass)
128}
129
130const CHECKS: Readonly<Record<string, (m: RegExpMatchArray) => boolean>> = {
131  'bearer token': m => !isReference(m[1] ?? ''),
132  'database url with credentials': m => urlCredential(m[0]),
133  'database url with password': m => urlCredential(m[0]),
134}
135const globals = new WeakMap<RegExp, RegExp>()
136
137function matches(name: string, re: RegExp, text: string): boolean {
138  if (name === 'key assignment') return assignmentSecret(text)
139  const check = CHECKS[name]
140  if (!check) return re.test(text)
141  let g = globals.get(re)
142  if (!g) globals.set(re, (g = new RegExp(re.source, re.flags.includes('g') ? re.flags : re.flags + 'g')))
143  for (const m of text.matchAll(g)) if (check(m)) return true
144  return false
145}
146
147/**
148 * The text textsOf appends when it had to drop input (a node, character or per-string budget ran out). It is never matched against a rule:
149 * `names` reports it as a finding of its own, so every guard that asks "is there a secret in these texts" refuses what it could not read in full.
150 */
151export const TRUNCATED = 'ruflo-screen: input exceeded the screening budget'
152export const TRUNCATED_NAME = 'input too large to screen'
153
154/** Names of the rules that match `text` (already bare'd). A rule named 'key assignment' is judged by assignmentSecret, whatever its regex. */
155export const names = (rules: Rules, text: string) => text === TRUNCATED ? [TRUNCATED_NAME] : rules.filter(([name, re]) => matches(name, re, text)).map(([name]) => name)
156
157export type Findings = { readonly secrets: readonly string[]; readonly injection: readonly string[] }
158
159/** Names of every secret shape in `secrets` and every injection phrase found in `text`. Cost is linear in the capped input. */
160export function screenWith(secrets: Rules, text: string): Findings {
161  const bounded = bare(text)
162  return { secrets: names(secrets, bounded), injection: names(INJECTION, bounded) }
163}
164
165export const hasSecretIn = (secrets: Rules, text: string) => names(secrets, bare(text)).length > 0
166
167/** Makes stored text safe to show: no control or bidi characters, whitespace collapsed, at most `max` characters. */
168export function tidy(text: string, max: number): string {
169  const flat = text.replace(INVISIBLE, '').replace(/\s+/g, ' ').trim()
170  return flat.length > max ? `${flat.slice(0, Math.max(0, max - 1))}…` : flat
171}
172/** Bounds for textsOf: nodes visited, characters returned, the longest string read in full, the size of one returned chunk, chunk overlap. */
173export type TextLimits = { readonly nodes?: number; readonly chars?: number; readonly perString?: number }
174const NODES = 20_000
175const CHARS = 2_000_000
176const PER_STRING = 1_500_000
177const OVERLAP = 2_048
178const BARE_KEY_MIN = 8
179
180/** A string as texts the screen can read whole: one text up to MAX_SCAN, else overlapping MAX_SCAN windows so a secret anywhere is inside one. */
181function windows(text: string, out: string[]): void {
182  if (text.length <= MAX_SCAN) {
183    out.push(text)
184    return
185  }
186  for (let at = 0; ; at += MAX_SCAN - OVERLAP) {
187    out.push(text.slice(at, at + MAX_SCAN))
188    if (at + MAX_SCAN >= text.length) return
189  }
190}
191
192/**
193 * Every string in a tool input, for the screen to read: iterative (no recursion, so nesting 5000 deep cannot overflow the stack) and
194 * breadth-first (siblings before depth, so a long list cannot hide a nested value). A string under an object key comes back as `key=value`,
195 * so a secret-named key is judged with its value; a key whose value is not a string is returned bare. Strings longer than the screen window
196 * come back as overlapping windows; one over `perString` keeps its head and tail. Work is bounded by `nodes` slots and `chars` characters.
197 * Anything dropped (slots or characters ran out, or a string lost its middle) is reported by a final TRUNCATED text, which `names` and
198 * `hasSecretIn` count as a finding, so the screen fails closed instead of passing what it did not read.
199 */
200export function textsOf(input: unknown, limits: TextLimits = {}): string[] {
201  const out: string[] = []
202  let slots = limits.nodes ?? NODES
203  let chars = limits.chars ?? CHARS
204  const perString = limits.perString ?? PER_STRING
205  let truncated = false
206  const take = (text: string): void => {
207    if (chars <= 0) {
208      truncated = true
209      return
210    }
211    if (text.length <= Math.min(perString, chars)) {
212      chars -= text.length
213      windows(text, out)
214      return
215    }
216    truncated = true
217    const half = Math.floor(Math.min(perString, chars) / 2)
218    chars -= 2 * half
219    windows(text.slice(0, half), out)
220    windows(text.slice(-half), out)
221  }
222  const queue: unknown[] = [input]
223  let head = 0
224  for (; head < queue.length && chars > 0; head++) {
225    const node = queue[head]
226    if (typeof node === 'string') take(node)
227    else if (Array.isArray(node)) {
228      let i = 0
229      for (; i < node.length && slots > 0; i++, slots--) if (i in node) queue.push(node[i])
230      if (i < node.length) truncated = true
231    } else if (typeof node === 'object' && node !== null) {
232      for (const k in node) {
233        if (slots-- <= 0) {
234          truncated = true
235          break
236        }
237        if (!Object.prototype.hasOwnProperty.call(node, k)) continue
238        const v = (node as Record<string, unknown>)[k]
239        if (typeof v === 'string') queue.push(k + '=' + v)
240        else {
241          if (k.length >= BARE_KEY_MIN) take(k)
242          queue.push(v)
243        }
244      }
245    }
246  }
247  if (queue.length > head) truncated = true
248  if (truncated) out.push(TRUNCATED)
249  return out
250}
251// END SHARED SCREEN
252
253const SECRETS: Rules = [
254  ...COMMON_SECRETS,
255  ['key assignment', /\b(?:api[_-]?key|secret|token|passw(?:or)?d|credential)s?["']?\s*[:=]\s*["']?[A-Za-z0-9/+=_.-]{16,}/i],
256]
257
258/** Names of every secret shape found in `text`. Cost is linear in the capped input. */
259export const secretsIn = (text: string): string[] => names(SECRETS, bare(text))
260
261export const hasSecret = (text: string) => hasSecretIn(SECRETS, text)
262