Skip to content
Dark Cosmos — Dev Docs

Skills usage audit (2026-10-07)

Skills usage audit — why each bh-* skill exists, why some go unused, and the fix

Section titled “Skills usage audit — why each bh-* skill exists, why some go unused, and the fix”

Follow-up to item 17 of 2026-10-07-dev-infrastructure-meta-audit.md. Chris: “look at all the skills and figure out why they were made in the first place, why they haven’t been used, how we can improve it to use them.”

Status (2026-10-07): steps 1, 2, 5 and 6 applied in 73eb996b. The Cloudflare plugin is off for the whole repo, wrangler skill included, since plugin skills can’t be hidden one by one; across 79 transcripts here it had 0 skill calls and 1 tool call. Steps 3 and 4 (deleting skills) are deferred by Chris.

Caveat on the counts: session history only covers 2026-09-24 → 10-07. “0 uses” means “not needed in that window”, not “broken”; rare-task skills (engine upgrade, area backdrop, Android build) are expected to look idle. No change below rests on the counts alone.

Status (2026-10-07) — all six steps shipped

Section titled “Status (2026-10-07) — all six steps shipped”
  • Steps 1, 2, 5, 6: 73eb996b (a parallel session). Overrides now live in the tracked .claude/settings.json (plus the cloudflare plugin disabled, since skillOverrides can’t reach plugin skills); guard test scripts/test/skills-paths.test.mjs. Check next session: if bh-* skills still list without descriptions, set skillListingBudgetFraction: 0.02.
  • Step 3: bh-investigation-loop → bh-overnight-audit-loop/references/{method-catalog, method-effectiveness}.md + references/scripts/; bh-polish-audit → references/polish-categories.md. The audit loop’s description now triggers on “investigation/improvement loop” and “polish pass”; global investigation-loop repointed (lumara-skills 3c583f0).
  • Step 4: bh-live-debug-deploy → bh-deploy § C4, rewritten for runtime DevFlags (the main.gd const flip it described no longer exists).

Headline

Section titled “Headline”
  1. The “15 unused skills” figure was wrong. It counted only Skill-tool calls. Counting direct SKILL.md reads and subagent prompts that name the skill (134 transcripts + compaction backups, 2026-09-24 → 10-07), only ~6 skills are actually dormant. Most of the rest are used, just not through the Skill tool.
  2. Root cause of real non-use: the skill listing drops descriptions, least-used first. The listing has a budget of about 1% of context (skillListingBudgetFraction / SLASH_COMMAND_TOOL_CHAR_BUDGET). When it is over budget, Claude Code drops descriptions from the least-used skills first (docs: code.claude.com/docs/en/skills.md, “Skill descriptions are cut short”). In this session’s listing, every bh- skill with zero Skill-tool calls appears with its name only*, while every used one keeps its description. So it feeds itself: an unused skill loses its description, the model can’t match it to a task, and it stays unused. The budget is eaten by ~150 global/plugin skills that are irrelevant to a Godot game (BK Safetywear, client email, WordPress, Cloudflare plugin, ComfyUI, Remotion, …). [Likely: the mechanism is from the docs, the correlation is observed; I can’t see the exact budget cut-off.]
  3. Skills that work are named at the moment of need, from the CLAUDE.md table, another skill or a dispatch prompt (bh-add-content 17 reads, bh-tunable-batch 9, both named in CLAUDE.md). Skills nothing names only get found through the listing, and the listing hides them.
  4. Six overlapping loop skills (investigation-loop, improvement-loop, overnight-audit-loop, overnight-swarm, overnight-feature-loop, polish-audit), plus 2 global loop skills and 4 saved workflows. Even with descriptions shown, picking one is hard.
  5. Three skills point at deleted files and one depends on a deleted flag. A skill that misleads when loaded is worse than one that’s never loaded.

Per-skill table

Section titled “Per-skill table”

Use = Skill-tool calls / SKILL.md reads (sessions) / dispatch prompts naming it. Listing = whether this session’s listing showed a description.

Skill Made (why) Use Listing Verdict / fix
bh-telemetry 07-17, D1 telemetry queries 16 / 75 (38) / 60 desc Healthy
bh-dev-chunk 07-04, per-change ritual 13 / 19 (9) / 43 desc Healthy
bh-sdd-concurrent 07-06, parallel-session collisions 11 / 16 (8) / 8 desc Healthy
bh-deploy 07-03, device installs 6 / 72 (27) / 29 desc Healthy
bh-appstore-release 07-02, store uploads 3 / 94 (45) / 34 desc Healthy (25 KB: worth trimming)
bh-vfx-iteration 07-22, neon-glow rework 7 / 21 (13) / 29 desc Healthy
bh-worktree-device-install 10-01, installs from worktrees 5 / 19 (14) / 1 desc Healthy
bh-merge-reviewed-branch 10-02, merge + release candidate 4 / 5 (4) / 0 desc Healthy
bh-deploy-mac-dev 10-07 (Droid port), Mac dev loop 2 / 5 (4) / 0 desc Healthy
bh-diagnose-render-timing 07-05, recurring flicker fixes 1 / 3 (3) / 9 desc Healthy (rare task)
bh-android-tester-build 09-30, tester APK pipeline 1 / 1 / 0 desc Healthy (rare task)
bh-add-content 07-02, invisible-entity trap 0 / 17 (12) / 19 name only Used via CLAUDE.md. Stale: cites test_determinism_checksum.gd (gone). Fix refs
bh-determinism-repin 07-01, baseline re-pins 0 / 9 (8) / 16 name only Used. Stale: description + body cite test_determinism_checksum/_crystals.gd; real pins are test_determinism{,_survival,_coop}.gd. Rewrite triggers to match the real failing tests, or it never fires
bh-write-test ~07, GUT conventions 0 / 7 (7) / 12 name only Used by subagents. Name it in bh-implementer’s brief so every new test file gets it
bh-tunable-batch 09-27, const → tunable loop 0 / 9 (9) / 1 name only Used via CLAUDE.md. Fine
bh-godot-engine-upgrade 07-19, half-upgraded toolchain 0 / 0 / 16 name only Rare task, 16 dispatch prompts name it. Fine once the description shows
bh-area-background 07-02, new area + backdrop 0 / 4 (2) / 8 name only Rare task. Stale checksum ref. Add to CLAUDE.md table row
bh-economy-sim 09-27, “ablate before you blame” 0 / 1 / 0 name only Dormant. CLAUDE.md economy row points at scripts and docs, not the skill. Add the skill to that row
bh-live-debug-deploy ~07, live device fix loop 0 / 2 (1) / 8 name only Dormant + stale: built around V01_LOCK_NETWORKED_MP, which no longer exists. Overlaps bh-worktree-device-install. Fold its “one fix per deploy, wait for the report” rule into bh-deploy, then retire it
bh-cloud-offload 09-29, RemoteTrigger offload 0 / 1 / 0 name only Dormant. One experiment (memory bullet-heaven-offload-experiments). Keep user-invocable-only until offload is used again
bh-polish-audit 07-16, polish-issue categories 0 / 2 (2) / 0 name only Reference material, not a workflow. Merge its categories into the loop skill as finder angles
bh-investigation-loop 07-13, find→verify swarm 0 / 1 / 0 name only Superseded by overnight-audit-loop (same job, proven 07-22). Merge, then retire
bh-overnight-audit-loop 07-22, cron find→verify→fix 2 / 1 / 0 desc Keep as THE discovery loop
bh-overnight-swarm 07-14, burn down a fixed punch list 0 / 7 (3) / 0 desc Keep (fixed-list job), or merge into feature-loop (also a fixed list)
bh-overnight-feature-loop 10-07, build a feature queue 0 / 1 / 0 name only New today. Keep
bh-improvement-loop 10-02, Droid-only loop 0 / 1 / 0 name only Droid only. off for Claude (it stays in .factory/)
bh-fresh-install-stall-check 10-07, fresh-install shader stalls 0 / 0 / 0 name only New today. Name it in the shader rows of CLAUDE.md / rendering doc

Saved workflows surfaced as skills (bh-whole-game-cycle, bh-overnight-feature-cycle, bh-overnight-perf-quality-cycle, bh-legacy-elemental-and-code-quality-audit) also have no descriptions. They are what the loop skills run, so they don’t need to be found directly.

Proposed fix (structural, not “remember to”)

Section titled “Proposed fix (structural, not “remember to”)”
  1. Free the listing budget in this repo’s tracked .claude/settings.json: skillOverrides set to "off" for the global and plugin skills that never apply to this game (BK/GMC/supplier product skills, client email/handover/bulk-email, WordPress-era, ComfyUI/Remotion/logo/photo sourcing, Cloudflare plugin skills except wrangler, rank-radar sweep, …). This is the same pattern settings.local.json already uses for ~250 web skills, just made shared and extended. Check with /context (Skills row) that every bh-* skill shows a description afterwards. If not, set skillListingBudgetFraction to 0.02.
  2. Fix the stale skills: update the checksum file refs in determinism-repin, add-content and area-background, and rewrite determinism-repin’s trigger to name the real test files.
  3. Collapse the loops to three, by job: discover (overnight-audit-loop, which absorbs investigation-loop’s method catalogue and polish-audit’s categories), build a list (overnight-feature-loop), burn down a punch list (overnight-swarm). Retire investigation-loop and polish-audit as stand-alone skills. Droid’s improvement-loop is set to off for Claude.
  4. Retire bh-live-debug-deploy, after moving its one-fix-per-deploy rule into bh-deploy.
  5. Name skills at the moment of need: CLAUDE.md economy row → bh-economy-sim; shader row → bh-fresh-install-stall-check; bh-implementer agent → bh-write-test + bh-add-content.
  6. A guard so this can’t drift silently again: a node test in scripts/test/ asserting every `tests/…gd` / `scripts/…` path cited in .claude/skills/**/SKILL.md exists (stale paths were the second failure mode here).

Steps 3 and 4 delete skills, so they need Chris’s OK. Each change goes through node scripts/port-claude-to-factory.mjs so .factory/ stays in parity.