Godot Stagehand
MCP server, agent skill and CLI for automating a running Godot game
newpanebandguardcommand
MCP server, agent skill and CLI for automating a running Godot game
Playwright, but for game engines.
**Drive your running Godot game from outside the engine.** Click real buttons, read real node state, capture real frames and diff them against saved baselines.
One Go binary. Two frontends over the same live connection:
See it work: examples/minimal-game is one command. Clone it and watch an agent click a button in a real running Godot scene, then assert the result.
Status: beta, pre-1.0. Prebuilt binaries for Linux, macOS (Intel and Apple Silicon) and Windows. Tool schemas and the wire protocol may still change between minor versions.
# 1. Get the binary (Linux shown; macOS and Windows builds are on the same page)
curl -fsSLo godot-stagehand \
https://github.com/mrf/godot-stagehand/releases/latest/download/godot-stagehand-linux-amd64
chmod +x godot-stagehand
# 2. Install the addon into your Godot project (idempotent)
./godot-stagehand setup /path/to/your/godot/project
# 3. Run your game with Stagehand on
godot --path /path/to/your/project --stagehandsetup prints the MCP client config snippet and the command to run your game. Your game then prints a one-session auth token. Keep it private, and give it to godot_connect.
Rather not use a terminal? Enable the addon in Project → Project Settings → Plugins, then click Setup… in the editor toolbar. That wizard downloads the binary, writes the config, and tests the connection. Either path is walked through step by step in the Quickstart.
/plugin marketplace add mrf/godot-stagehand
/plugin install godot-stagehand@godot-stagehandThe plugin brings the MCP server (it downloads the matching release binary on first start), the agent skill, and godot-stagehand on the Bash tool's PATH. Enable it in your Godot project's .claude/settings.json rather than user-wide, and remove any hand-written godot-stagehand MCP entry first, or every tool appears twice.
From an MCP client (Claude Code, Claude Desktop, Cursor, anything speaking MCP):
{
"mcpServers": {
"godot-stagehand": {
"command": "/absolute/path/to/godot-stagehand"
}
}
}From a terminal or a CI job:
export STAGEHAND_AUTH_TOKEN=<the token this Godot session printed>
godot-stagehand find --port 26788 'class:Button' --properties text
godot-stagehand run scenarios/menu-smoke.json --out-dir ci-artifactsrun executes a declarative list of launch, action, wait and assertion steps against a real Godot build and exits nonzero on failure. Exit 5 means a real regression. --out-dir collects report.json, junit.xml, rpc-trace.json, godot.log, screenshots and diff images.
Stagehand binds to 127.0.0.1 and rejects every command until the peer supplies the session token, but it is a dev control plane, not a hardened endpoint. Read the security boundary before you expose anything.
| Quickstart | Install and first command, step by step |
| Tool reference | Every MCP tool, and what to build with them |
| CLI and scenario runner | Commands, scenario format, exit codes, CI recipes |
| Selectors | Targeting nodes by path, name, class, group, text, role |
| Configuration | Flags, env vars, timeouts, running several agents at once |
| Security boundary | Auth, remote binding, unsafe methods |
| Architecture | How the addon, the binary and your client fit together |
| Compatibility | Godot 4.3 to 4.7, and why not 4.2 |
| Troubleshooting | When it won't connect, or the screenshots are black |
| Comparison | Versus editor-automation tools and in-engine test frameworks |
| Visual regression | Baselines, diffing, and the CI gate contract |
| Agent skill | Drop-in skill file that teaches an agent the whole workflow |
| Windows / WSL | Bridging Godot on Windows with a client in WSL |
go vet ./... # lint
go test ./... # Go tests (no Godot needed)
# Scenario runner against a real headless Godot
GODOT_BIN=/path/to/godot go test -tags=godot -run '^TestScenarioRunner' .
# GDScript unit suite (GdUnit4, headless, needs Godot 4.6+)
GODOT_BIN=/path/to/godot ./scripts/run-gdscript-tests.shRead the GDScript testing guide for the suite layout and strict-mode rules, the addon sync contract before editing any copy of the addon, and the error model before adding a handler that can fail.
MIT. See LICENSE.
hooks/register.ts 342 lines1// The godot-stagehand mod (docs/design/claude-code-plugin.md, D7): a status
2// band above the prompt and a /stagehand-view pane with the last game frame.
3//
4// It only observes. The tool.call hook passes every call on unchanged and
5// reads the results of Claude's own Stagehand calls; it never calls the server
6// itself, because $.mcp.call fires tool.call again and, inside a turn, needs a
7// permission grant (design Q7). Only the pane's buttons call the server, from
8// outside a turn. The MCP server and the skill work without this mod.
9import type { EngineInterface, On } from 'claude-code'
10
11const SERVER = 'plugin:godot-stagehand:stagehand'
12const TOOL_PREFIX = 'mcp__plugin_godot-stagehand_stagehand__'
13const PANE_ID = 'stagehand-view'
14// The Image element takes PNGs of at most 2 MiB decoded.
15const IMAGE_LIMIT_BYTES = 2 * 1024 * 1024
16// godot_connect's defaults, for a successful call that left them out.
17const DEFAULT_INSTANCE = 'default'
18const DEFAULT_HOST = '127.0.0.1'
19const DEFAULT_PORT = 26700
20
21interface Instance {
22 id: string
23 state: string
24 host: string
25 port: number
26}
27
28interface Frame {
29 png: string
30 bytes: number
31 width: number
32 height: number
33 capturedAt: Date
34}
35
36// State lives in module variables: the band rebuilds from the next Stagehand
37// call, so nothing has to survive a reload.
38const instances = new Map<string, Instance>()
39let frame: Frame | undefined
40let paneError: string | undefined
41
42// ── Reading tool results ────────────────────────────────────────────────────
43
44function asRecord(value: unknown): Readonly<Record<string, unknown>> | undefined {
45 return typeof value === 'object' && value !== null && !Array.isArray(value)
46 ? (value as Readonly<Record<string, unknown>>)
47 : undefined
48}
49
50function stringField(record: Readonly<Record<string, unknown>> | undefined, key: string): string | undefined {
51 const value = record?.[key]
52 return typeof value === 'string' ? value : undefined
53}
54
55function numberField(record: Readonly<Record<string, unknown>> | undefined, key: string): number | undefined {
56 const value = record?.[key]
57 return typeof value === 'number' && Number.isFinite(value) ? value : undefined
58}
59
60function parseJSON(text: string | undefined): Readonly<Record<string, unknown>> | undefined {
61 if (text === undefined) return undefined
62 try {
63 return asRecord(JSON.parse(text))
64 } catch {
65 // Not every result is JSON (godot_connect answers in prose); not ours to read.
66 return undefined
67 }
68}
69
70// What a successful call left for the model: its text, and its content blocks.
71interface Observed {
72 text: string | undefined
73 blocks: readonly unknown[]
74}
75
76// A tool.call result as core gives it: `{ result, text }`, or `isError` / `deny`.
77function fromToolCall(result: unknown): Observed | undefined {
78 const record = asRecord(result)
79 if (record === undefined || record['isError'] === true || record['deny'] !== undefined) return undefined
80 const inner = record['result']
81 const text = stringField(record, 'text') ?? (typeof inner === 'string' ? inner : undefined)
82 return { text, blocks: Array.isArray(inner) ? inner : [] }
83}
84
85// A $.mcp.call result: `{ content, isError }`.
86function fromMcpCall(result: unknown): Observed | undefined {
87 const record = asRecord(result)
88 if (record === undefined || record['isError'] === true) return undefined
89 const content = record['content']
90 const blocks: readonly unknown[] = Array.isArray(content) ? content : []
91 const texts = blocks.flatMap((block) => {
92 const text = stringField(asRecord(block), 'text')
93 return text === undefined ? [] : [text]
94 })
95 return { text: texts.length > 0 ? texts.join('\n') : undefined, blocks }
96}
97
98function replaceInstances(list: unknown, stateOf: (item: Readonly<Record<string, unknown>>) => string | undefined): void {
99 if (!Array.isArray(list)) return
100 instances.clear()
101 for (const entry of list) {
102 const item = asRecord(entry)
103 const id = stringField(item, 'id')
104 const host = stringField(item, 'host')
105 const port = numberField(item, 'port')
106 const state = item === undefined ? undefined : stateOf(item)
107 if (id !== undefined && host !== undefined && port !== undefined && state !== undefined) {
108 instances.set(id, { id, state, host, port })
109 }
110 }
111}
112
113// PNG width and height from the IHDR chunk: bytes 16-23 of the file, which
114// are base64 characters 0-31.
115function pngSize(png: string): { width: number; height: number } {
116 const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
117 const bytes: number[] = []
118 for (let i = 0; i + 3 < 32 && i + 3 < png.length; i += 4) {
119 const n = [0, 1, 2, 3].reduce((acc, k) => acc * 64 + Math.max(0, alphabet.indexOf(png.charAt(i + k))), 0)
120 bytes.push((n >> 16) & 0xff, (n >> 8) & 0xff, n & 0xff)
121 }
122 const word = (at: number): number =>
123 ((bytes[at] ?? 0) * 0x1000000) + ((bytes[at + 1] ?? 0) << 16) + ((bytes[at + 2] ?? 0) << 8) + (bytes[at + 3] ?? 0)
124 return { width: word(16), height: word(20) }
125}
126
127function decodedLength(base64: string): number {
128 const padding = base64.endsWith('==') ? 2 : base64.endsWith('=') ? 1 : 0
129 return Math.floor((base64.length * 3) / 4) - padding
130}
131
132function frameFrom(blocks: readonly unknown[]): Frame | undefined {
133 for (const entry of blocks) {
134 const block = asRecord(entry)
135 if (stringField(block, 'type') !== 'image') continue
136 // Claude Code hands MCP images on in the Messages API shape
137 // ({ source: { type: 'base64', media_type, data } }); accept the MCP shape too.
138 const source = asRecord(block?.['source'])
139 const png = stringField(source, 'data') ?? stringField(block, 'data')
140 const mime = stringField(source, 'media_type') ?? stringField(block, 'mimeType')
141 if (png === undefined || mime !== 'image/png') continue
142 return { png, bytes: decodedLength(png), ...pngSize(png), capturedAt: new Date() }
143 }
144 return undefined
145}
146
147// Updates the module state from one successful Stagehand call. Returns
148// whether anything a drawing shows changed.
149function observe(tool: string, args: Readonly<Record<string, unknown>>, observed: Observed): boolean {
150 switch (tool) {
151 case 'godot_status': {
152 const report = parseJSON(observed.text)
153 if (report === undefined) return false
154 replaceInstances(report['instances'], (item) => stringField(item, 'state'))
155 return true
156 }
157 case 'godot_list_instances': {
158 const report = parseJSON(observed.text)
159 if (report === undefined) return false
160 replaceInstances(report['instances'], (item) =>
161 item['connected'] === true ? 'connected' : 'disconnected',
162 )
163 return true
164 }
165 case 'godot_launch': {
166 const launched = parseJSON(observed.text)
167 const host = stringField(launched, 'host')
168 const port = numberField(launched, 'port')
169 if (host === undefined || port === undefined) return false
170 const id = stringField(launched, 'instance_id') ?? DEFAULT_INSTANCE
171 instances.set(id, { id, state: 'connected', host, port })
172 return true
173 }
174 case 'godot_connect': {
175 // godot_connect answers in prose, so read the call's own arguments.
176 const id = stringField(args, 'instance_id') ?? DEFAULT_INSTANCE
177 const host = stringField(args, 'host') ?? DEFAULT_HOST
178 const port = numberField(args, 'port') ?? DEFAULT_PORT
179 instances.set(id, { id, state: 'connected', host, port })
180 return true
181 }
182 case 'godot_disconnect': {
183 const id = stringField(args, 'instance_id')
184 return id !== undefined && instances.delete(id)
185 }
186 case 'godot_screenshot': {
187 const captured = frameFrom(observed.blocks)
188 if (captured === undefined) return false
189 frame = captured
190 return true
191 }
192 default:
193 return false
194 }
195}
196
197// ── Drawing ─────────────────────────────────────────────────────────────────
198
199function statusLine(): string | undefined {
200 const connected = [...instances.values()].filter((instance) => instance.state === 'connected')
201 const [only] = connected
202 if (only === undefined) return undefined
203 const address = (instance: Instance): string => `${instance.id} ${instance.host}:${String(instance.port)}`
204 if (connected.length === 1) return `stagehand · ${address(only)} · connected`
205 return `stagehand · ${String(connected.length)} connected · ${connected.map(address).join(', ')}`
206}
207
208function describeSize(bytes: number): string {
209 return bytes >= 1024 * 1024 ? `${(bytes / (1024 * 1024)).toFixed(1)} MiB` : `${String(Math.ceil(bytes / 1024))} KiB`
210}
211
212function clock(date: Date): string {
213 const two = (n: number): string => String(n).padStart(2, '0')
214 return `${two(date.getHours())}:${two(date.getMinutes())}:${two(date.getSeconds())}`
215}
216
217// Terminal cells are about twice as tall as they are wide.
218function imageCells(width: number, height: number, maxColumns: number, maxRows: number): { columns: number; rows: number } {
219 const clamp = (n: number, max: number): number => Math.max(1, Math.min(255, max, Math.round(n)))
220 const aspect = width > 0 && height > 0 ? height / width : 9 / 16
221 let columns = clamp(maxColumns, maxColumns)
222 let rows = clamp((columns * aspect) / 2, maxRows)
223 if ((columns * aspect) / 2 > maxRows) columns = clamp((rows * 2) / aspect, maxColumns)
224 rows = clamp(rows, maxRows)
225 return { columns, rows }
226}
227
228// ── Pane buttons: the only server calls, made outside a turn ────────────────
229
230async function callFromPane($: EngineInterface, tool: 'godot_screenshot' | 'godot_status'): Promise<void> {
231 try {
232 const observed = fromMcpCall(await $.mcp.call(SERVER, tool, {}))
233 paneError = observed === undefined ? `${tool} failed; ask Claude to check the game.` : undefined
234 if (observed !== undefined) observe(tool, {}, observed)
235 } catch (error) {
236 paneError = error instanceof Error ? error.message : String(error)
237 }
238 $.ui.invalidate('ui.render')
239}
240
241// ── Hooks ───────────────────────────────────────────────────────────────────
242
243export function register(on: On): void {
244 on('tool.call', async ($, e, next) => {
245 const result = await next(e)
246 if (e.tool.startsWith(TOOL_PREFIX)) {
247 const observed = fromToolCall(result)
248 const args = asRecord(e) ?? {}
249 if (observed !== undefined && observe(e.tool.slice(TOOL_PREFIX.length), args, observed)) {
250 $.ui.invalidate('ui.render')
251 }
252 }
253 return result
254 })
255
256 on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => {
257 const line = statusLine()
258 if (line === undefined || e.props.hasSurvey) return next(e)
259 const { Box, Text } = $.ui.resolve(e)
260 // Keep what the mods after this one drew.
261 const theirs = await next(e)
262 return Box({
263 flexDirection: 'column',
264 children: [Text({ dimColor: true, wrap: 'truncate-end', children: [line] }), theirs],
265 })
266 })
267
268 on('ui.render', { component: 'Pane' }, async ($, e, next) => {
269 if (e.requestId !== PANE_ID) return next(e)
270 const { Box, Text, Button } = $.ui.resolve(e)
271
272 const rows = [
273 Text({ bold: true, wrap: 'truncate-end', children: [statusLine() ?? 'stagehand · no game connected'] }),
274 ]
275 if (frame === undefined) {
276 rows.push(Text({ dimColor: true, children: ['No frame yet. Press r, or ask Claude for a screenshot.'] }))
277 } else {
278 const summary = `${String(frame.width)}×${String(frame.height)} frame, ${describeSize(frame.bytes)}, captured ${clock(frame.capturedAt)}`
279 if (e.surface === 'terminal' && frame.bytes <= IMAGE_LIMIT_BYTES) {
280 // Only the terminal's element table has Image (the Desktop app has none).
281 const { Image } = $.ui.resolve(e)
282 const cells = imageCells(frame.width, frame.height, e.props.bodyColumns, Math.max(1, e.props.scroll.bodyRows - 4))
283 // The alt text stands in for the picture where the terminal cannot
284 // draw one (inside tmux, for example), right above the summary line.
285 rows.push(Image({ key: 'frame', source: { png: frame.png }, ...cells, alt: 'Last game frame' }))
286 rows.push(Text({ dimColor: true, children: [summary] }))
287 } else {
288 const why = frame.bytes > IMAGE_LIMIT_BYTES ? 'too large to draw here' : 'this app cannot draw it'
289 rows.push(Text({ children: [`Last ${summary}: ${why}; ask Claude for a screenshot to see it.`] }))
290 }
291 }
292 if (paneError !== undefined) rows.push(Text({ color: 'error', children: [paneError] }))
293 rows.push(
294 Box({
295 flexDirection: 'row',
296 columnGap: 3,
297 children: [
298 Button({
299 key: 'refresh',
300 label: 'Refresh frame',
301 hotkey: 'r',
302 plain: true,
303 onPress: () => {
304 void callFromPane($, 'godot_screenshot')
305 },
306 }),
307 Button({
308 key: 'status',
309 label: 'Re-read status',
310 hotkey: 's',
311 plain: true,
312 onPress: () => {
313 void callFromPane($, 'godot_status')
314 },
315 }),
316 ],
317 }),
318 )
319 return Box({ flexDirection: 'column', children: rows })
320 })
321
322 on('command.run', { command: 'stagehand-view' }, async ($) => {
323 await $.ui.open({ id: PANE_ID, title: 'Stagehand', focus: true, closeOnEscape: true })
324 return {}
325 })
326
327 // Last, and guarded: $.command.register throws on a taken name, and the
328 // band above must keep working when it does.
329 on('session.start', async ($, e, next) => {
330 try {
331 await $.command.register({
332 name: 'stagehand-view',
333 description: 'Show the last game frame Claude captured and the Stagehand connection',
334 immediate: true,
335 })
336 } catch {
337 // Another plugin or the user owns /stagehand-view; the band still works.
338 }
339 return next(e)
340 })
341}
342