magpie-utilities
Apache Magpie — framework meta-skills: write-skill, optimize-skill, skill-reconciler, list-skills.
newcommandprocess
Apache Magpie — framework meta-skills: write-skill, optimize-skill, skill-reconciler, list-skills.
<!-- SPDX-License-Identifier: Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 -->
<!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> Table of Contents generated with DocToc
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- SPDX-License-Identifier: Apache-2.0 https://www.apache.org/legal/release-policy.html -->
Adopt a Magpie. Magpie is a curated set of skills for the AI agent you already use — Claude Code, Codex, Gemini CLI, Copilot and others — that take on the repetitive parts of running an open-source project: triaging issues and pull requests, reviewing code, mentoring contributors, drafting fixes, and handling security reports end to end.
Three things make it different from a folder of prompts:
Magpie is currently in development for ASF projects + Python Core team friendlies. Testers welcome!
One security report, start to finish — installed from the marketplace, the agent sandboxed and guarded, then the report imported, triaged against the project's history, fixed, and published as a CVE:
Illustrative — a written transcript, rendered deterministically, not a recording. The interactive version on the project site steps through the same story with the full output of each command.
[!IMPORTANT] The motivation, scope, and design commitments behind this work live in
MISSION.md— the founding mission of the Apache Magpie Top-Level Project, originally filed as its establishment proposal. Read that for the why; this README is the how once you've decided to install.
Start here → Quick start — a handful of commands, in the agent you already use. Nothing is written to your repository.
/plugin marketplace add apache/magpie # Claude Code
/plugin install magpie-setup@apache-magpie # install this one first
/plugin install magpie-agent-guard@apache-magpie # the baseline:
/plugin install magpie-utilities@apache-magpie # take all three
/plugin install magpie-pr-management@apache-magpie # + the families you needThose first three are the baseline — and the same set a project commits as its floor when it adopts Magpie, so taking them is taking what a project would recommend to every contributor. After that, install one plugin per family you actually want: every installed skill costs context on every turn, used or not (~0.2–2.0k tokens a family), so there is deliberately no install-everything plugin.
Once magpie-setup is in, you never type these again — ask in plain language ("install the Magpie families for PR review") and the setup skill runs the installs for you.
Codex, VS Code / Copilot, and Gemini are one-liners too — the quick start has all four, plus what to run next to put the agent in its sandbox. Nothing is committed to your repository.
Fallback — the pinned snapshot install. Use it when a marketplace is not an option or not enough: your agent has no plugin mechanism, you need the signed ASF source release rather than a git clone, or the project wants every contributor and CI job pinned to one committed version with drift detection.
docs/setup/install-recipes.md/magpie-setup:setup installThe two are complementary, not exclusive.
Magpie is used by interacting with your AI agents. You'll use plain-language prompts like
review PR #5193
or
triage the latest security reports
or skill calls starting with a slash, like
/magpie-repo-health:dependency-audit
(the family-plugin form, assuming the recommended marketplace install above — see Skill names differ by install method if you're on the pinned-snapshot fallback instead).
Marketplace install (the recommended path above):
/plugin marketplace update apache-magpie then /plugin update <plugin>@apache-magpie — refresh the marketplace metadata, then bump the installed plugin(s) to its latest. Auto-update does the same in the background once enabled./magpie-setup:setup reconcile. See Reconciliation when a skill first runs after an upgrade./magpie-setup:isolated-setup-update after an upgrade that changed its files, and weekly otherwise. Run it directly to check now. See Automatic reminders from the pre-flight.Pinned-snapshot install (the fallback):
/magpie-setup upgrade — refresh the snapshot to a newer framework version + reconcile any overrides against the new framework structure./magpie-setup verify — read-only health check (snapshot intact, symlinks live, .gitignore correct, etc.)./magpie-setup:setup override <framework-skill> — open or scaffold an override file for a framework skill.The following skill families ship in the framework, all at experimental or stable, and each skill declares its family in a family: frontmatter key. On the recommended marketplace install, you choose families by which per-family plugin(s) you install (see Install above) — install or uninstall a plugin at any time to add or drop a family. On the pinned-snapshot fallback, /magpie-setup offers the opt-in families — and the optional MCP servers (ponymail, apache-projects, gmail-plaintext) — in a single install choice, and symlinks for the picked families land in the skill directory. Either way, the two always-on families (setup, utilities) are wired unconditionally and never prompted for.
The Modes column maps each family to the MISSION agent-assistance taxonomy — see docs/modes.md for what each mode means and which modes are still proposed vs. shipping today.
| Family | Type | Modes | Purpose | Detail |
|---|---|---|---|---|
| setup | always-on | (infra) | Isolated agent setup, framework install + maintenance, shared-config sync. The prerequisite — at minimum the setup skill itself runs out of this family. | 11 skills, docs/setup/ |
| utilities | always-on | (meta) | Framework meta-skills: author skills (write-skill), restructure them (optimize-skill), reconcile skill state (skill-reconciler), report framework issues (report-framework-issue), and print a live index (list-skills). | 5 skills |
| security | opt-in | Triage, Drafting | 16-step security-issue handling lifecycle — from security@ import through CVE publication, including state sync — plus producing, verifying, and maintaining the project's own security model. Maintainer-only. | 15 skills, docs/security/ |
| pr-management | opt-in | Triage | Maintainer-facing PR-queue management — triage, stats, deep code review, stack review, express-lane merge, stale-sweep, reviewer routing, and pre-first-PR checks. | 9 skills, docs/pr-management/ |
| issue | opt-in | Triage, Drafting | General-issue lifecycle: triage, reproduction, fix drafting, reassess, stale-sweep, deduplication, and backlog reporting. | 8 skills, docs/issue-management/ |
| release-management | opt-in | Triage, Drafting | 14-step ASF release lifecycle, planning issue, RC cut + sign, [VOTE] thread, tally, promote, [ANNOUNCE], archive, audit log. Agent never holds the RM's signing key and never publishes the release. Experimental, all 10 skills shipped. | 10 skills, docs/release-management/ |
| repo-health | opt-in | Triage | Read-only repository-health audits: obsolete runner labels, Actions workflow security, dependency vulnerabilities, dependency licence review, license/NOTICE compliance, flaky-test patterns, plus audit-finding fixes. | 7 skills, docs/repo-health/ |
| pairing | opt-in | Pairing | Pair a change with a structured self-review or a multi-agent adversarial review before it lands. | 2 skills, docs/pairing/ |
| mentoring | opt-in | Mentoring | Newcomer-facing mentoring — first-contact welcome, newcomer-issue explanations, and good-first-issue authoring + backlog curation. Experimental. | 4 skills, docs/mentoring/ |
| contributor-growth | opt-in | Triage, Mentoring | The path-to-committer track: activity sweeps, nomination briefs, contributor-sentiment signals, readiness tracking, threshold calibration, candidate screening, committer / post-vote onboarding, and contributor identity mapping across Slack, Discord, and social media. | 9 skills, docs/contributor-growth/ |
Skill families or individual skills can be pulled from a trusted external source — a repo other than apache/magpie that ships Magpie-shaped skills (with their evals and tests). Where a skill directory would sit, a skills/<name>/source.md redirect names a pinned, verified source the adopter has vouched for; /magpie-setup fetches it into the gitignored snapshot and wires it in exactly like a framework skill. Nothing is fetched unless the adopter commits the pin — see docs/skill-sources/, PRINCIPLES.md §14, and RFC-AI-0006.
Apache Magpie was first developed and proven inside Apache Airflow, and was maintained for a time as the apache/airflow-steward repository under the Airflow PMC before being renamed and established as its own project. It also incorporates early skill work contributed by way of the Apache Groovy community. All of that code carries the same rightsholder — Copyright The Apache Software Foundation, under the Apache License 2.0 — so it is not a third-party inclusion; the required attribution lines from the originating projects' NOTICE files are reproduced in NOTICE.
MISSION.md — founding mission of the established TLP: motivation, scope, design commitments, initial PMC composition target.docs/setup/agentic-overrides.md — the contract between adopters who write overrides and framework skills that read them.docs/prerequisites.md — what a maintainer needs installed before invoking any framework skill (Claude Code, Gmail MCP, GitHub auth, browser, uv, etc.).docs/source-release-contents.md — what ships in the signed apache-magpie-<version>-source.zip (and what is excluded), with the rationale for the repository-root metadata/config files it keeps.docs/release-management/manual-release-process.md — the concrete, as-executed runbook for cutting a Magpie release by hand on the current hybrid SVN-dist + ATR-vote backend (with the abstract per-backend runbooks and the 14-step lifecycle alongside it in docs/release-management/).AGENTS.md — agent instructions, placeholder convention, framework conventions.CONTRIBUTING.md — for framework contributors.hooks/skills-command.ts 197 lines1// SPDX-License-Identifier: Apache-2.0
2//
3// Licensed to the Apache Software Foundation (ASF) under one
4// or more contributor license agreements. See the NOTICE file
5// distributed with this work for additional information
6// regarding copyright ownership. The ASF licenses this file
7// to you under the Apache License, Version 2.0 (the
8// "License"); you may not use this file except in compliance
9// with the License. You may obtain a copy of the License at
10//
11// http://www.apache.org/licenses/LICENSE-2.0
12//
13// Unless required by applicable law or agreed to in writing,
14// software distributed under the License is distributed on an
15// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16// KIND, either express or implied. See the License for the
17// specific language governing permissions and limitations
18// under the License.
19
20export interface ProcessRunResult {
21 exitCode: number;
22 stdout: string;
23 stderr: string;
24}
25
26export type ProcessRunner = (
27 argv: readonly string[],
28 init?: { cwd?: string; env?: Record<string, string>; timeoutMs?: number }
29) => Promise<ProcessRunResult>;
30
31const PROCESS_TIMEOUT_MS = 30_000;
32
33/**
34 * Joins path segments normalizing slashes without depending on node:path.
35 */
36export function joinPath(...segments: string[]): string {
37 const nonEmpties = segments.filter((s) => s && s.trim().length > 0);
38 if (nonEmpties.length === 0) return '.';
39
40 const isAbsolute = segments[0]?.startsWith('/') || segments[0]?.startsWith('\\');
41 const parts: string[] = [];
42
43 for (const seg of nonEmpties) {
44 const split = seg.split(/[/\\]+/);
45 for (const p of split) {
46 if (p === '..') {
47 if (parts.length > 0 && parts[parts.length - 1] !== '..') {
48 parts.pop();
49 } else if (!isAbsolute) {
50 parts.push('..');
51 }
52 } else if (p !== '.' && p !== '') {
53 parts.push(p);
54 }
55 }
56 }
57
58 const result = parts.join('/');
59 if (isAbsolute) {
60 return '/' + result;
61 }
62 return result || '.';
63}
64
65/**
66 * Executes list_skills.py deterministically via process runner (python3 or python).
67 */
68export async function executeMagpieSkills(
69 runProcess: ProcessRunner,
70 scriptPath: string,
71 workspaceDir: string,
72 rawArgs?: string
73): Promise<{ text: string }> {
74 const args = (rawArgs || '').trim();
75 const isHelp = /(?:--help|-h)(?:\s|$)/i.test(args);
76
77 let scriptArgs: string[];
78 if (isHelp) {
79 scriptArgs = [scriptPath, '--help'];
80 } else {
81 const isVerbose = /(?:--verbose|-v)(?:\s|$)/i.test(args);
82 const rootMatch = args.match(/(?:--root[=\s]+)(?:"([^"]+)"|'([^']+)'|([^\s]+))/i);
83 const rootCandidate = rootMatch
84 ? (rootMatch[1] ?? rootMatch[2] ?? rootMatch[3]).trim()
85 : '';
86 const targetRoot =
87 rootCandidate && !rootCandidate.startsWith('-') ? rootCandidate : workspaceDir;
88
89 scriptArgs = [scriptPath, '--root', targetRoot];
90 if (isVerbose) {
91 scriptArgs.push('--verbose');
92 }
93 }
94
95 const runInit = {
96 cwd: workspaceDir,
97 env: { PYTHONIOENCODING: 'utf-8' },
98 timeoutMs: PROCESS_TIMEOUT_MS,
99 };
100
101 // Try python3 first, fallback to python
102 let result: ProcessRunResult | undefined;
103 let executionError: Error | undefined;
104
105 try {
106 result = await runProcess(['python3', ...scriptArgs], runInit);
107 } catch (err: any) {
108 executionError = err instanceof Error ? err : new Error(String(err));
109 }
110
111 // If python3 failed to run (e.g. command not found) or returned non-zero, try python
112 if (!result || result.exitCode !== 0) {
113 try {
114 const fallbackResult = await runProcess(['python', ...scriptArgs], runInit);
115 if (fallbackResult.exitCode === 0 || !result || result.exitCode === 127) {
116 result = fallbackResult;
117 if (fallbackResult.exitCode === 0) {
118 executionError = undefined;
119 }
120 }
121 } catch (fallbackErr: any) {
122 if (!executionError) {
123 executionError = fallbackErr instanceof Error ? fallbackErr : new Error(String(fallbackErr));
124 }
125 }
126 }
127
128 if (result && result.exitCode === 0) {
129 return { text: result.stdout.trimEnd() };
130 }
131
132 const errorLines = [
133 '⚠️ Apache Magpie: Unable to execute deterministic skills indexer (`list_skills.py`).',
134 ];
135 if (result?.stderr?.trim()) {
136 errorLines.push(`Error: ${result.stderr.trim()}`);
137 } else if (executionError?.message) {
138 errorLines.push(`Error: ${executionError.message}`);
139 }
140 errorLines.push(
141 'Please verify that Python 3 is installed and available on PATH, or invoke the full workflow via `/magpie-utilities list-skills`.'
142 );
143
144 return { text: errorLines.join('\n\n') };
145}
146
147/**
148 * Event hook registration entry point for Claude Code.
149 */
150export function register(on: any): void {
151 // 1. Hook session.start to register slash command
152 on('session.start', async ($: any, e: any, next: any) => {
153 try {
154 await $.command.register({
155 name: 'magpie-skills',
156 description: 'List installed Apache Magpie skills (zero-token)',
157 argumentHint: '[-v|--verbose] [--root <path>] [-h|--help]',
158 immediate: true,
159 });
160 } catch {
161 // Safe no-op on command register exception
162 }
163
164 if (typeof next === 'function') {
165 return next(e);
166 }
167 });
168
169 // 2. Hook command.run on magpie-skills
170 on('command.run', { command: 'magpie-skills' }, async ($: any, e: any) => {
171 try {
172 const workspaceDir = await $.session.root();
173 const pluginDir = $.plugin.root;
174 const scriptPath = joinPath(
175 pluginDir,
176 'skills',
177 'list-skills',
178 'scripts',
179 'list_skills.py'
180 );
181
182 return await executeMagpieSkills(
183 (argv, init) => $.process.run(argv, init),
184 scriptPath,
185 workspaceDir,
186 e.args
187 );
188 } catch (err: any) {
189 return {
190 text: `⚠️ Apache Magpie: Unexpected error executing /magpie-skills: ${err?.message || String(err)}`,
191 };
192 }
193 });
194}
195
196export default register;
197