Shop economy simulator (design)

Shop economy simulator — design

Date: 2026-09-27 · Status: approved in brainstorm (Chris), awaiting spec review Release item: docs/RELEASE-0.1.md §2, “Meta-shop pricing from a simulated economy”

1 · Problem

Meta-shop prices were set by hand. Maxing all 66 items costs 22,951 gold (recomputed from bible.json 2026-09-27; the first count said 22,943) (Pilot 3,636 · Utility 527 · Arsenal 2,640 · Drones 15,340 · Ships 800), and a median human Survival run pays ~200-400 gold (GOLD_PER_KILL=1, +2 per heavy enemy, BOSS_GOLD=40, all × area_reward_mult; sim/sim.gd:2202, :2619-2671; since 2026-10-04 those consts are gone: gold is per-type bible enemies.<id>.gold x Sim.GOLD_MULT, see docs/architecture/remote-config.md § Gold economy. The economy sim reads only the soak report’s run_gold, so it follows bible gold automatically). Nobody knows how many runs each tier takes, how much easier a run gets after a purchase, or which items are over/under-priced.

2 · Goals (Chris’s calls)

Pacing target (tiered): first meaningful purchase after 1-2 runs; core Pilot + Arsenal owned by ~run 20; Drones are the long tail. Targets get finalised per tier AFTER the simulator shows the current curve.
Three player profiles (weak / median / strong), calibrated to human telemetry; prices are set against median, and weak must not be starved.
Two buy policies: shop-tip order (what a real player does) for pacing; value-per-gold for “is this price fair”. Data on every buying pathway is the long-term aim; in practice that is every item’s value at several progression checkpoints (enumerating purchase orders is infeasible).
Output is a report + a price proposal. No price or gold-formula change ships without Chris approving it.

Non-goals: changing gold earning (the report may recommend it; it doesn’t do it), new shop items, anything EVE/Sovereign-shaped.

3 · Components

3.1 One shared run-setup path (`sim/meta_run_setup.gd`, new)

Today main.gd applies the shop in two places (_new_run ~:875 and the reset_upgrades dev command ~:2723, the latter already a partial copy) and soak applies almost none of it: no meta.apply_to, no locked_weapons, no decoy, no drone recharge, no drone loadout. A simulator that skips those measures a different game.

Extract MetaRunSetup.apply(sim, meta, ship, mode) doing exactly the _new_run sequence: ShipBonuses.apply_base → meta.apply_to → ShipBonuses.apply_to → sim.locked_weapons → decoy → drone_recharge_mult → drone loadout (tutorial exceptions preserved). main.gd (both sites) and soak call it. Structural fix: the simulator can’t drift from the real run start. Must not change main.gd behaviour; the determinism baselines don’t touch this path.

3.2 Soak additions (`tools/soak/`)

--meta=<path.json>: a MetaState.to_dict() file; applied via 3.1. Omitted = fresh save (today’s behaviour).
--profile=weak|median|strong|off (default off = current SkilledAutoPlayer, unchanged).
Report gains run_gold, run_time, and meta_spent per run.

3.3 Skill knobs on SkilledAutoPlayer

A SkillProfile (plain RefCounted data object) with, initially:

decide_every: the pilot re-decides movement, dash and aim only every N ticks (reaction time); between decisions it holds its last input;
aim_jitter_rad: uniform ± error on aim;
dash_skill: probability a dash the skilled AI wants (panic, lunge-dodge, engage) is actually taken at a decision tick. Randomness comes from the AI’s own SeededRng (seeded from the run seed + a constant), never from any of Sim’s four streams. off = all zero = byte-identical to today (test-pinned). Attract mode keeps off until Chris picks a profile for it.

3.4 Calibration (`tools/economy_sim/calibrate.gd`)

Targets: all human Survival runs (ruleset='survival', is_auto=0, builds ≥185, ended in a death: death_cause != '', run_time < 1800), p25 / p50 / p85 of run_time and gold → weak / median / strong. (Corrected 2026-09-27 while planning: runs.session is a per-RUN id, main.gd:1280, so “each tester’s first 3 runs” can’t be selected.)
Known bias: those humans owned an unknown amount of shop upgrades, but profiles are fitted on a fresh save, so profiles come out slightly STRONGER than real players at equal shop state, and the sim will make prices look slightly cheaper than they feel. The report states this.
Sample is small (n=18 after filters, checked 2026-09-27); tolerance ±20% on both metrics, mean over 20 seeds.
Fitted knob values are written back as constants in SkillProfile with the date + targets.
Re-calibrate later against runs.meta_spent/meta_levels (live from 2026-09-27, populated once a build ships) to compare like with like.

3.5 Strong-profile autopilot work

Only as far as needed for the strong profile to reach p85. Known starting gap: flee logic was never made loadout-aware (ai/skilled_auto_player.gd:38). Changes stay behind the profile where they would alter off behaviour, OR are accepted as an attract-mode change Chris signs off.

3.6 Career simulator (`tools/economy_sim/career.gd`)

Loop: run (soak SoakRun with --meta state) → bank run_gold → buy per policy → next run.

Policy A, shop-tip: repeatedly buy the cheapest affordable row of ShopAdvice.recommend() (ui/shop_advice.gd); when its rows are exhausted, cheapest affordable remaining item. Ship: flies Obsidian once bought.
Policy B, value-per-gold: buy the affordable item with the highest measured value/cost from the value sweep (3.7) at the nearest checkpoint.
Per run it logs: index, gold earned, run_time, level, death cause, purchases, gold banked, total spent. Runs: 3 profiles × 5 seeds × up to 150 runs (stop early when everything is owned).
Output: reports/economy/career-<profile>-<seed0>.json (one file per career, so parallel jobs never collide). Derived: runs to first purchase, to “core Pilot + Arsenal owned”, to 50% / 100% spend, per profile.

3.7 Item value sweep (`tools/economy_sim/value_sweep.gd`)

Checkpoints: fresh save; the policy-A median career’s save at run 20; at run 60.
For each of the 66 items (skipping maxed ones at that checkpoint): +1 level vs. baseline over 10 paired seeds (same seeds both arms), median profile. Metrics: Δ run_time, Δ gold.
Value/gold = Δ gold ÷ price (gold is the pacing currency; Δ run_time reported alongside).
Weak + strong re-check only the top and bottom 10 items.
Unlock items (weapons, drone classes) are measured the same way. The +1 makes them offerable or owned, and the autopilot picks what’s offered.

3.8 Report

A shareable page (Artifact) for Chris + Toby: career curves per profile against the tier targets, runs-to-tier table, item value-per-gold ranking per checkpoint, and a price proposal (per-item new base_cost/cost_growth, OR a gold-rate change if prices are uniformly off). Rules: sample sizes next to every number; calibration targets labelled rough; items whose 90% interval crosses zero shown as “no measurable effect”, not ranked.

4 · Compute

Headless soak ≈ 12× real time per process (docs/overnight-swarm-runbook-2026-07-16.md:108); run N processes in parallel (cap = cores − 2). Estimates [likely, unmeasured]: careers 1-2 h wall, value sweep ~2,000 runs ≈ 1-2 h. Longer runs later in a career make this grow; measure on step 5 and adjust seeds before committing to the full sweep. All jobs headless, overnight-safe.

5 · Build order (each step: gate.sh green → merge → push)

MetaRunSetup extraction + soak --meta/run_gold (tests: meta file changes player stats; locked weapons honoured; off path unchanged; gold recorded).
SkillProfile knobs + --profile (tests: off byte-identical trace; knobs never draw from Sim RNG streams; determinism baselines green).
Calibration run → fitted constants.
Strong-profile improvements to hit p85.
Career sim, policy A → first curve shown to Chris → he sets per-tier targets.
Value sweep at 3 checkpoints; policy B career.
Report + price proposal → Chris approves → data/bible.json edit (separate commit) → re-run career to confirm targets hit.

Implementers: bh-implementer (Opus); review: bh-reviewer. Never Haiku for code.

6 · Testing

GUT unit tests for: MetaRunSetup (matches the old inline sequence field by field), buy policies (given defs + gold → expected purchases), checkpoint save builder, report maths (percentiles, paired deltas, interval). Sim tools verified end-to-end on a hand-checked 3-run career. gate.sh before every merge; soak sweep stays clean.

7 · Risks

Calibration data is thin. Mitigated by labelling, and re-calibrating once meta_spent rows arrive.
Autopilot ≠ human. It may be good at some things (perfect aim) and bad at others (bosses). Matching on run_time + gold only checks the aggregate. Death-cause mix is shown in the report as a sanity check.
Unlock values depend on the autopilot’s upgrade picking (pick_upgrade prefers weapons), so a new weapon’s value is “as used by the autopilot”, flagged in the report.
Attract-mode drift from 3.5: kept behind the profile unless Chris accepts it.