Landing blockers, runs in flight and other worktrees, live from probe.py landing

A k3s homelab managed as infrastructure-as-code with Ansible, with a small residual Docker footprint on the Pi. Services are fronted by Traefik with Authelia SSO, secrets encrypted with SOPS/age, storage on Longhorn, and reverse-proxied behind Cloudflare.
<!-- The exact count is whatever containers_list says in inventory/host_vars/*.yml — don't restate a precise number here or in CLAUDE.md; two hand-maintained copies drift apart (they read 44 and 49 while the real total was 52). -->
grep -c '^ - name:' ansible/inventory/host_vars/*.yml # the actual per-host tally
Day-to-day conventions and the agent contract live in
CLAUDE.md. Most directories and many roles have their ownCLAUDE.mdwith role-specific notes.
The migration from Docker Compose to k3s completed on 2026-08-14, when Docker was uninstalled from daniel-server. The slice-by-slice record is in docs/archive/k3s-migration/ — those documents are historical and describe work already executed.
| Host | Role | Notes |
|---|---|---|
daniel-box | k3s server (control plane) | Runs almost every workload. Traefik edge, Authelia SSO + OIDC issuer, Pi-hole DNS, Longhorn storage, CrowdSec. Internet-exposed via Cloudflare, and the public WireGuard endpoint (wg-easy, 51820/udp — the router forward moved here). Ansible runs on this host. |
daniel-server | k3s agent node | Intel XE iGPU (Jellyfin/Tdarr transcode), LVM storage, UPS hardware + the NUT shutdown chain. Docker uninstalled 2026-08-14 — it hosts no Compose services. |
daniel-pi | Raspberry Pi — Docker | LAN-only, never internet-exposed or on the tunnel. A second, LAN-only wg-easy (51822/udp) + a small utility stack (Glances, autoheal, docker-proxy). The only remaining Docker host. |
ansible/ # Playbooks, roles, inventory, templates ← EDIT HERE
deploy.yml # Deploy playbook — a Docker play (dependency-ordered) + a k8s play
k3s-bringup.yml # Cluster foundation (k3s, Longhorn, Traefik CRDs, …)
initial_setup.yml # Host hardening / bootstrap
roles/k8s/ # One role per k8s workload (manifests rendered from templates)
roles/containers/ # One role per Docker service (+ a shared `common` role)
archive/ # Roles retired by the k3s migration, kept for reference
filter_plugins/ # Custom dependency-resolution filters (toposort.py)
inventory/ # hosts.ini, group_vars/all.yml, host_vars/<host>.yml
vars/secrets.yml # SOPS-encrypted secrets
scripts/ # Helper scripts (template validation, …)
docs/ # Runbooks, design specs, security notes
archive/ # Superseded planning docs (incl. the Docker → k3s migration)
containers/ is not in this repo. It is a runtime directory Ansible renders on the target host (/home/<user>/server/containers/<svc>/docker-compose.yml) and is untracked — git ls-files containers/ returns nothing. Post-migration it exists only on daniel-pi; neither cluster node has one, since neither runs Docker. Edits there are overwritten on the next deploy: always change ansible/roles/containers/<svc>/templates/ instead. The cluster's equivalent is the manifests rendered from ansible/roles/k8s/.
Traefik is the single ingress on daniel-box, routing to workloads via Traefik IngressRoute CRDs; Authelia gates protected routes via forward-auth middleware and issues OIDC tokens for apps that speak it. Cloudflare proxies the public hostnames; *.local.<domain> names are resolved by Pi-hole on the LAN and are not exposed.
flowchart TD
net[Internet] --> cf[Cloudflare DNS + proxy]
cf --> traefik[Traefik ingress — daniel-box]
traefik -. forward-auth .-> authelia[Authelia SSO + 2FA + OIDC]
subgraph cluster["k3s cluster (daniel-box server, daniel-server agent)"]
traefik
authelia
crowdsec[CrowdSec engine + node agents]
apps["apps · pihole · n8n · code-server · karakeep · freshrss · livesync · wg-easy"]
media["media · jellyfin · arr-stack · qbittorrent · tdarr · bazarr"]
monitoring["monitoring · prometheus · grafana · uptime-kuma · scrutiny · loki"]
longhorn[(Longhorn PVs)]
end
traefik --> apps
traefik --> media
traefik --> monitoring
apps --- longhorn
media --- longhorn
monitoring --- longhorn
subgraph pi["daniel-pi — Docker, LAN-only"]
wg[wg-easy] --- alloy[Alloy · autoheal]
end
Network segmentation is not blanket-enforced: only a few roles define NetworkPolicies (n8n, prowlarr, registry) — everything else is reachable pod-to-pod within the namespace. Where policies do exist, ingress rules are enforced; egress rules are not enforced by this cluster's CNI, so never read an egress policy as a control.
deploy.yml runs two plays over the host's containers_list, split by each entry's platform: key:
platform: docker, only daniel-pi) — deploys roles in containers_list order. The Pi's one ordering constraint, docker-proxy before autoheal, is that list's order, and ansible/tests/deploy/test_platform_filter_real_inventory.py pins it. A tagged run (--tags autoheal) deploys only the tagged role and does not pull up its dependency.platform: k8s) — toposorts roles/k8s/<name> with build_k8s_dep_map / toposort_containers (ansible/filter_plugins/toposort.py, unit-tested in ansible/tests/deploy/test_toposort.py). A role rendering a Traefik CRD gets an edge onto traefik, which installs the CRDs every later IngressRoute depends on; use_authelia: true gets one onto authelia, which creates the middleware other routes reference. Both are derived from the role's own templates/entry, not hand-listed; an edge no template carries (crowdsec before traefik, for the LAPI machine credential) is declared as depends_on: on the entry instead. List order in host_vars/daniel-box.yml is only the sort's tiebreak.Both plays run against the host named by -e target= — but daniel-server and daniel-box are ansible_connection=local in hosts.ini, so -e target= only selects whose variables to use while tasks still execute locally. A pre-task refuses that case outright. Only daniel-pi is genuinely remote, so -e target=daniel-pi works as it reads.
uv run ansible-playbook ansible/deploy.yml --tags "<service>" # deploy one service (+ unmet deps)
uv run ansible-playbook ansible/deploy.yml --tags "<service>" --check # dry run
uv run ansible-playbook ansible/deploy.yml # deploy everything
uv run ansible-playbook ansible/deploy.yml --tags "<service>" -e target=daniel-pi
uv run ansible-playbook ansible/k3s-bringup.yml # cluster foundation
uv run ansible-playbook ansible/initial_setup.yml # host bootstrap/hardening
First-host bring-up (uv → SOPS onboarding → initial_setup.yml) is ordered in ansible/README.md; ansible/bring-up.sh drives those steps (--scaffold for inventory, no flag for uv + SOPS, --continue for the playbooks). The manual post-deploy setup Ansible can't do is verified by uv run python scripts/diagnostics/postflight.py.
ansible/vars/secrets.yml, encrypted with SOPS + age (.sops.yaml auto-encrypts anything under vars//secrets/). Decrypted at runtime via the community.sops lookup. Edit with sops ansible/vars/secrets.yml. Never commit plaintext secrets — gitleaks runs pre-commit. The age private key is backed up out-of-band (single point of recovery).ansible/roles/k8s/observability/, alongside roles/k8s/loki-homelab (Loki + an Alloy DaemonSet) for homelab logs. Prometheus scrapes node-exporter / cAdvisor / Traefik / CrowdSec. Grafana dashboards stay provisioned as code from roles/k8s/observability/files/dashboards/ — that tree is the single source of truth and is mounted into the cluster Grafana, which is why the role survives the migration despite not being in any containers_list. Uptime Kuma takes monitors from AutoKuma labels and a static-monitors Secret; monitor-bridge turns Prometheus/Longhorn signals into Uptime Kuma push alerts (backup freshness, disk, cert, memory, restarts/OOM, CPU throttling, scrape-target down, Traefik 5xx).docs/longhorn-backup-tiering.md. Recovery procedure: docs/longhorn-disaster-recovery.md. The Pi's own data is covered by pi-peer-backup. Kopia is retired (2026-08-13; repo deleted 2026-08-14) — docs/archive/kopia-disaster-recovery.md is kept only as history.prek.toml hook revisions (see renovate.json); it requires installing the Renovate GitHub App on the repo once. Watchtower was retired with the migration — image updates are PR-driven, not automatic.docs/security-tools.md.The repo uses prek (prek.toml): YAML/JSON lint, ansible-lint, gitleaks, rendered-template validation (compose + config + Home Assistant + Grafana), secret-rotation-registry sync, ruff (lint + format), and the pytest suite.
prek run --all-files
Almost every new service is a k8s workload: add a role under ansible/roles/k8s/<name>/ rendering its manifests (Deployment/Service/IngressRoute/PVC), then an entry in host_vars/daniel-box.yml containers_list with platform: k8s. Position in the list doesn't matter — the k8s play toposorts on derived Traefik-CRD and authelia edges, plus any depends_on: the entry declares — so add it wherever reads best. Deploy tags derive from the name.
For the Pi's Docker services, ansible/roles/containers/ + the new-container workflow scaffold a role: a tasks/main.yml, a templates/docker-compose.yml.j2 (Traefik + AutoKuma labels, healthcheck, resource limits), an entry in host_vars/daniel-pi.yml containers_list, and any secrets. See CLAUDE.md → "Adding a New Container Service".
hooks/register.tsx 267 lines1// The deck mod: CLAUDE.md's *When to wait* state, live (#3676).
2//
3// Every fact comes from `probe.py landing --json`, run once a minute. That command computes
4// the blocker list itself, so the band, the system-prompt section and the pane cannot disagree
5// with it or with each other. The mod reads and gates nothing: when the probe fails, the pane
6// says so and the band and section keep the last good snapshot.
7import { atom, read, update } from 'claude-code'
8import type { EngineInterface, Register } from 'claude-code'
9
10import type { DeckSnapshot } from '../types'
11
12const PANE = 'deck'
13const REFRESH_MS = 60_000
14const PROBE_TIMEOUT_MS = 45_000
15const PROBE = ['uv', 'run', 'python', 'scripts/diagnostics/probe.py', 'landing', '--json']
16
17const snapshot = atom({ plugin: 'deck', key: 'snapshot' } as const, null)
18const readError = atom({ plugin: 'deck', key: 'readError' } as const, null)
19const isBandHidden = atom({ plugin: 'deck', key: 'isBandHidden' } as const, false)
20const isContextOff = atom({ plugin: 'deck', key: 'isContextOff' } as const, false)
21
22// `$.store` keys: the toggles survive the session, the snapshot does not.
23const STORE_PANE = 'isPaneOpen'
24const STORE_BAND = 'isBandHidden'
25const STORE_CONTEXT = 'isContextOff'
26
27const USAGE =
28 'Usage: /deck (open or close the pane), /deck band on|off, /deck context on|off, /deck refresh'
29
30/** Formats the blocker list as the system-prompt section the model reads. */
31export function blockedSection(blockers: readonly string[]): string {
32 return [
33 '# Landing is blocked',
34 '',
35 "CLAUDE.md's *When to wait* applies to landing or deploying a change of your own: name the",
36 'blocker below that applies, then stop. Each blocker names its own way out, and taking it',
37 '(applying an owed plane, fixing a red master) is not blocked.',
38 'The deck mod read these from `probe.py landing`. Run it for the current state and the',
39 'apply commands.',
40 '',
41 ...blockers.map(b => `- ${b}`),
42 ].join('\n')
43}
44
45// Module variables: a hot reload starts them over, which costs one extra probe run.
46let inFlight: Promise<void> | undefined
47let root = ''
48
49/**
50 * Finds the checkout the session runs in and whether it carries the probe. A session outside
51 * the server repo answers false, and the mod then starts no timer there.
52 */
53async function findProbe($: EngineInterface): Promise<boolean> {
54 const top = await $.process.run(['git', 'rev-parse', '--show-toplevel'])
55 if (top.exitCode !== 0) {
56 return false
57 }
58 root = top.stdout.trim()
59 const probe = await $.process.run(['test', '-f', `${root}/scripts/diagnostics/probe.py`])
60 return probe.exitCode === 0
61}
62
63/** Runs the probe once and stores its snapshot; a call made while one runs joins it. */
64function refresh($: EngineInterface): Promise<void> {
65 inFlight ??= readProbe($).finally(() => {
66 inFlight = undefined
67 })
68 return inFlight
69}
70
71async function readProbe($: EngineInterface): Promise<void> {
72 try {
73 const ran = await $.process.run(PROBE, {
74 cwd: root || undefined,
75 timeoutMs: PROBE_TIMEOUT_MS,
76 })
77 // Exit 1 means "something blocks"; both 0 and 1 print the document.
78 const parsed = JSON.parse(ran.stdout) as DeckSnapshot
79 await update($, snapshot, () => parsed)
80 await update($, readError, () => null)
81 } catch (err) {
82 const message = err instanceof Error ? err.message : String(err)
83 await update($, readError, () => `probe.py landing failed: ${message.slice(0, 200)}`)
84 }
85}
86
87async function setToggle(
88 $: EngineInterface,
89 key: string,
90 value: boolean,
91): Promise<void> {
92 await $.store.set(key, value)
93 if (key === STORE_BAND) {
94 await update($, isBandHidden, () => value)
95 } else if (key === STORE_CONTEXT) {
96 await update($, isContextOff, () => value)
97 }
98}
99
100export const register: Register = on => {
101 on('session.start', async ($, e, next) => {
102 await $.command.register({
103 name: 'deck',
104 description: 'Landing blockers, runs in flight and other worktrees (band, context: on|off)',
105 })
106 const bandHidden = (await $.store.get(STORE_BAND)) === true
107 const contextOff = (await $.store.get(STORE_CONTEXT)) === true
108 await update($, isBandHidden, () => bandHidden)
109 await update($, isContextOff, () => contextOff)
110 if ((await $.store.get(STORE_PANE)) === true) {
111 void $.ui.open({ id: PANE, title: 'Deck' })
112 }
113 if (await findProbe($)) {
114 void refresh($)
115 $.clock.every(REFRESH_MS, () => refresh($))
116 }
117
118 return next(e)
119 })
120
121 on('command.run', { command: 'deck' }, async ($, e) => {
122 const [what, value] = e.args.trim().split(/\s+/)
123 if (!what) {
124 const isOpen = (await $.ui.panes()).some(pane => pane.id === PANE)
125 if (isOpen) {
126 await $.ui.close({ id: PANE })
127 await $.store.set(STORE_PANE, false)
128 return { text: 'Deck pane closed.' }
129 }
130 await $.store.set(STORE_PANE, true)
131 await $.ui.open({ id: PANE, title: 'Deck' })
132 void refresh($)
133 return { text: 'Deck pane opened.' }
134 }
135 if (what === 'refresh') {
136 await refresh($)
137 return { text: 'Deck refreshed.' }
138 }
139 if ((what === 'band' || what === 'context') && (value === 'on' || value === 'off')) {
140 const key = what === 'band' ? STORE_BAND : STORE_CONTEXT
141 await setToggle($, key, value === 'off')
142 return { text: `Deck ${what} ${value}.` }
143 }
144 return { text: USAGE }
145 })
146
147 on('ui.close', async ($, e, next) => {
148 if (e.id === PANE && e.origin.kind === 'person') {
149 await $.store.set(STORE_PANE, false)
150 }
151 return next(e)
152 })
153
154 on('prompt.compose', async ($, e, next) => {
155 const composed = await next(e)
156 const snap = await read($, snapshot)
157 if (!snap || snap.blockers.length === 0 || (await read($, isContextOff))) {
158 return composed
159 }
160 return {
161 sections: [
162 ...composed.sections,
163 { id: 'deck:landing-blocked', text: blockedSection(snap.blockers), scope: 'session' },
164 ],
165 }
166 })
167
168 on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => {
169 const snap = await read($, snapshot)
170 if (
171 e.props.hasSurvey ||
172 !snap ||
173 snap.blockers.length === 0 ||
174 (await read($, isBandHidden))
175 ) {
176 return next(e)
177 }
178 const { Box, Button, Text } = $.ui.resolve(e)
179 const more = snap.blockers.length > 1 ? ` (+${snap.blockers.length - 1} more, /deck)` : ''
180
181 return (
182 <Box key="deck-band">
183 <Text key="blocked" color="red" wrap="truncate-end">
184 Landing blocked: {snap.blockers[0]}
185 {more}{' '}
186 </Text>
187 <Button key="hide" label="Hide" onPress={() => setToggle($, STORE_BAND, true)} />
188 </Box>
189 )
190 })
191
192 on('ui.render', { component: 'Pane', requestId: PANE }, async ($, e) => {
193 const { Box, Text } = $.ui.resolve(e)
194 const snap = await read($, snapshot)
195 const error = await read($, readError)
196 const lines: { key: string; text: string; color?: string; dim?: boolean }[] = []
197 const add = (text: string, color?: string, dim?: boolean) =>
198 lines.push({ key: `l${lines.length}`, text, color, dim })
199
200 if (!snap) {
201 add(error ?? 'Reading probe.py landing...', undefined, true)
202 } else {
203 add('Landing blockers', undefined)
204 if (snap.blockers.length === 0) {
205 add(' none', 'green')
206 }
207 for (const b of snap.blockers) {
208 add(` ✗ ${b}`, 'red')
209 }
210 const hold = snap.hold === null ? 'unknown' : snap.hold.sha.slice(0, 8) || 'none'
211 const ci = snap.ci === null ? 'unknown' : `${snap.ci.state} ${snap.ci.sha}`
212 const owed =
213 snap.manual_planes === null ? 'unknown here' : String(snap.manual_planes.length)
214 add(`hold: ${hold} master CI: ${ci} manual planes: ${owed}`, undefined, true)
215 for (const plane of snap.manual_planes ?? []) {
216 add(` owed: ${plane.line}`, 'yellow')
217 }
218 add('')
219 add('In flight')
220 if (snap.runs === null) {
221 add(' unknown', undefined, true)
222 } else if (snap.runs.length === 0) {
223 add(' nothing', undefined, true)
224 }
225 for (const r of snap.runs ?? []) {
226 const what = r.pr ? `PR ${r.pr}` : r.tag.slice(0, 40)
227 add(` ${r.kind} ${what} ${r.elapsed_s}s ${r.verdict ?? ''}`.trimEnd())
228 }
229 if (snap.last_verdict) {
230 add(`Last landing: ${snap.last_verdict.verdict ?? 'no VERDICT line'}`, undefined, true)
231 }
232 add('')
233 add('Worktrees')
234 for (const t of snap.worktrees ?? []) {
235 const claims = t.claims.map(n => `#${n}`).join(' ')
236 add(` ${t.branch || t.path} ${claims}`.trimEnd())
237 }
238 // A claim the probe matched to no worktree above, with findings.py's own reason.
239 const worked = new Set((snap.worktrees ?? []).flatMap(t => t.claims))
240 const unmatched = (snap.claims ?? []).filter(c => !worked.has(c.number))
241 if (unmatched.length > 0) {
242 add('Claims no worktree here works')
243 for (const c of unmatched) {
244 add(` #${c.number} ${c.worktree} ${c.reason}`.trimEnd(), undefined, !c.live)
245 }
246 }
247 for (const err of snap.errors) {
248 add(err, 'yellow', true)
249 }
250 if (error) {
251 add(error, 'yellow', true)
252 }
253 add(`read ${new Date(snap.read_at * 1000).toISOString()} on ${snap.host}`, undefined, true)
254 }
255
256 return (
257 <Box key="deck" flexDirection="column">
258 {lines.map(l => (
259 <Text key={l.key} color={l.color} dimColor={l.dim} wrap="truncate-end">
260 {l.text || ' '}
261 </Text>
262 ))}
263 </Box>
264 )
265 })
266}
267types/index.d.ts 46 lines1// The document `probe.py landing --json` prints. `scripts/diagnostics/probe_lib/landing_blockers.py`
2// owns the shape; a field is null when its source could not be read.
3export type DeckRun = {
4 kind: string
5 pr: string
6 tag: string
7 elapsed_s: number
8 verdict: string | null
9}
10
11export type DeckWorktree = { path: string; branch: string; claims: number[] }
12
13// One row of `findings.py claims --json`.
14export type DeckClaim = {
15 number: number
16 worktree: string
17 live: boolean
18 reason: string
19 age_days: number
20}
21
22export type DeckSnapshot = {
23 host: string
24 read_at: number
25 hold: { sha: string; planes: string[] } | null
26 manual_planes: { role: string; line: string }[] | null
27 ci: { state: string; sha: string; url: string } | null
28 runs: DeckRun[] | null
29 last_verdict: { log: string; verdict: string | null } | null
30 worktrees: DeckWorktree[] | null
31 claims: DeckClaim[] | null
32 blockers: string[]
33 errors: string[]
34}
35
36declare module 'claude-code' {
37 interface PluginState {
38 deck: {
39 snapshot: DeckSnapshot | null
40 readError: string | null
41 isBandHidden: boolean
42 isContextOff: boolean
43 }
44 }
45}
46