Story Board tool — design

Story Board — a BugSmash-style review/annotate loop for the storyline & tutorial

Date: 2026-07-23 Status: Design approved, ready for implementation plan Author: Chris + Claude (brainstorm session 2026-07-23)

Problem

Chris needs a way to review scripted-sequence pacing — the tutorial and storyline beats — and hand Claude precise, batched change requests, the way BugSmash does for websites (pin + change-needed + enough context to act accurately). The concrete pain: the tutorial’s ember_intro dialogue crams 7 teaching lines into a single box (movement, aimed bolt, warp, healing, blade, all at once) — it needs to be slowed down, spread out, step by step. Today the only feedback channel is Chris telling Claude one thing at a time, which is too slow for a multi-beat sequence.

The deeper structural reason this is hard: Claude cannot perceive gameplay feel in real time (documented as the lowest-rated capability for this project — see memory bullet-heaven-claude-capability-map). BugSmash works because a webpage is static — you pin onto a frozen thing. The storyline is temporal. So the tool’s job is to freeze the temporal sequence into a discrete, annotatable storyboard that Claude can “see” (screenshots + timing + triggers + the controlling code), turning an unperceivable flow into an inspectable data structure.

Two problems, one surface (A now, B later)

• (A) Scripted-sequence pacing — tutorial/storyline beats. Deterministic and inspectable. This is what we build now.
• (B) Emergent gameplay feel — combat smoothness, difficulty spikes, dead time. Needs Chris’s real-time judgment; a tool can only surface signals. Not built now, but the review surface is architected so B can plug into it later with one new capture stage and zero changes to the web tool or round-trip.

Why this is tractable (verified architecture, 2026-07-23)

• Beats are already observable events. sim/story_director.gd’s _say(sim, id) appends {"kind":"say","name","lines","color"} to the single sim.story.events stream, in order with structural events ({"kind":"room","event":"enter",...}, "meet", "cleared", "region", "complete"). So a replay can watch that one stream and know exactly when each beat fires and what triggered it.
• The crammed feel has an exact cause. ui/dialogue_box.gd queues every line in a say event back-to-back (consume() → _queue), typewriters at CHARS_PER_SEC, holds maxf(LINE_TIME, MIN_HOLD), then auto-advances (_pop_line). Seven lines in one say event = one box, no internal gaps. Fixable by splitting the beat and/or inserting pauses.
• A capture harness already exists. marketing/capture/shoot.gd boots main.tscn into a windowed Metal HDR SubViewport and saves PNGs. We adapt it to drive the sim from a ReplayLoader and screenshot on each beat.
• Deterministic replay exists. RECORD_REPLAY (main.gd) records lossless input + seed to user://replays/; input/replay_loader.gd (load_file, input_at) reconstructs InputStates for a byte-identical re-run.
• A proven local-web-tool pattern exists. tools/sound_map/ (Python server + browser) already does “browse events next to their graphic, act, write back to files” — the exact shape of this tool.

Architecture & data flow

A 6-stage deterministic pipeline; each stage writes a plain file the next stage reads.

1. RECORD   Chris plays the tutorial once (RECORD_REPLAY=true)
            -> user://replays/tutorial.json      (lossless input + seed)

2. CAPTURE  tools/story_board/capture.gd  (adapts marketing/capture/shoot.gd)
            re-ticks the replay in a windowed Metal SubViewport; each tick
            scans sim.story.events; on a `say` beat -> screenshot + log
            {id, tick, t, speaker, lines, trigger, gap_to_next, controls}
            -> tools/story_board/out/storyboard.json  +  beat-NNN.png

3. RESOLVE  (inside capture) each beat -> the exact thing that controls it:
            dialogue text -> data/story.json#dialogue[id]
            within-box pacing -> ui/dialogue_box.gd consts
            trigger -> the story_director rule / room that fired it

4. REVIEW   tools/story_board/server.py + browser (sound_map stack)
            renders a timeline strip + one card per beat (screenshot,
            text, timing, trigger, controls). Chris pins a comment + tag.

5. WRITE    tool writes tools/story_board/annotations.json
            [{beat_id, comment, tag}]                    <- the round-trip

6. APPLY    Claude reads annotations.json; each pin carries its controlling
            file#ref, so the batch applies as precise edits. Chris re-runs
            stages 2-4 on the SAME replay -> before/after on identical inputs.

Key property: stages 1 and 6 close a loop. Because the replay is deterministic and dialogue edits are (mostly) render-side, re-capturing the same recorded inputs after an edit shows the new pacing directly — a before/after on identical player behavior, the thing BugSmash structurally can’t do.

Components

tools/story_board/capture.gd (+ capture.tscn)

• Boots main.tscn into a windowed Metal SubViewport (copy marketing/capture setup — it renders at exact size with HDR regardless of physical display).
• Feeds InputStates from ReplayLoader.input_at(data, tick, pilot_idx) instead of live input, tick by tick, to the replay’s recorded seed.
• A beat = one say event (one box, however many lines) — deliberately NOT per-line, so the “7 lines in one box” problem shows up as one crammed card to annotate, not 7 tidy cards that hide it. The card flags line-count + no-internal-pause.
• Trigger = the most recent structural sim.story.events entry before the beat.
• gap_to_next = ticks between this beat firing and the next, in seconds.
• Screenshot at the beat tick (a few settle frames), saved out/beat-NNN.png.
• Instrumentation add (1 line): _say stamps the dialogue id into the say event ({"kind":"say","id":dialogue_id,...}). dialogue_box.consume() ignores unknown keys, so this is render-safe and determinism-neutral (render-side field). It gives an unambiguous beat -> data/story.json mapping instead of fragile text-matching.
• Output: out/storyboard.json (array of beat cards) + PNGs.

tools/story_board/server.py + index.html (sound_map stack)

• Top: a timeline strip — every beat as a tick on one line, so clustering (boxes too close) is visible at a glance before scrolling. The pacing overview.
• Body: one card per beat — screenshot thumb (click to enlarge), speaker, the lines (count + warn if many), t + gap_to_next (warn if below a threshold), trigger, controls:file#ref, a free-text comment box, and a tag dropdown (split / add-pause / reword / re-trigger / reorder / other).
• Save writes tools/story_board/annotations.json = [{beat_id, comment, tag}].

Round-trip (Claude side)

Read annotations.json, group pins by controls file, apply as one batch, report per-pin what changed. split -> N dialogue entries + trigger points; add-pause -> a pacing field/beat; reword -> a line edit; re-trigger -> a story_director/room condition change. Chris re-runs capture on the same replay to verify the batch.

The B extension seam (open, not built)

storyboard.json is just an array of event-cards; the web tool + round-trip only care about that shape, so they are source-agnostic. For A the source is sim.story.events beats. Later, B adds a capture_gameplay.gd that emits the same card shape from a soak/replay analysis — “notable gameplay events” (a death, a long dead-time stretch, a difficulty spike, a near-miss), each with a screenshot and its own controls ref (a spawn table, a balance const). Same cards, same pins, same apply loop. Nothing in A’s web tool or round-trip changes for B; only a new capture module is added.

Scope

• First cut = the tutorial (Ember Reach). But capture reads any story replay with zero tutorial-specific code, so it generalizes to the whole storyline for free once a longer run is recorded.

Testing

• Beat-detection is pure and unit-tested (GUT) against a fixture replay: correct beat count, monotonic ticks, correct gap_to_next, every beat resolves a controls ref, trigger pairing correct.
• Capture harness headless smoke: fixture replay -> storyboard.json with the expected shape (this may need windowed for real screenshots — the JSON/beat log can be asserted headless; PNG presence checked separately, since MultiMesh colours are unreadable headless).
• Web tool is dev-only, lightly tested like tools/sound_map/.
• No determinism impact: capture is read-only over the sim; the _say id stamp is a render-ignored field (determinism baselines unchanged — verify, no re-pin expected).

Non-goals (YAGNI)

• No video/timeline scrubber (storyboard chosen for discrete, single-edit-mappable pins).
• No auto-driver AI (record-once chosen; the survival AutoPlayer can’t path story rooms).
• No in-engine live overlay.
• No hosted/multi-user surface — local dev tool only.
• B (emergent gameplay-feel capture) is not implemented — only its one seam kept open.

Follow-ups / debt

• New dev-tooling entry in docs/architecture/dev-tools.md (alongside Remote Console + web-demo tooling).
• NEXT_TASKS.md line for the tool if any rough edges ship.
• Consider a new dark-cosmos/ role brief owning “narrative pacing review” if the tool becomes a recurring workflow (parallel to the audio-direction role that owns sound_map).