Soak Death-Cause + Level Tracking — Design

Soak Death-Cause + Level Tracking — Design

Problem

Chris asked (2026-07-17): “we need to include what killed ace and also track his level as a potentially better metric to evaluate against rather than survive time, i think he may be reachign a point where his ship flying ability is his downfall, proabbly agansnt chargers, he needs to learn how to orbit them and use warp at the right time.”

Two separate needs:

1. Soak reports don’t currently surface WHAT killed the pilot, so there’s no way to test the “chargers are the bottleneck” hypothesis with real data.
2. level_reached is already captured per-run but never aggregated across a sweep the way death_tick_distribution is — level is a plausible alternate/complementary skill metric to survival time (a pilot that dies fast but at a high level played the early game very well and died to something else; survival time alone conflates “died early because bad build” with “died early because a specific mechanic beat them”).

What already exists (verified against source, not assumed)

• Sim.death_cause: String (sim/sim.gd:344) is already latched at the FIRST fatal blow — sim/sim.gd:2716-2717: if pilot.hp <= 0.0 and death_cause == "" and source != "": death_cause = source. This is exactly “what killed the pilot,” not “highest total damage source” — no new /sim code needed, just plumbing an existing field through to the soak report.
• source strings are the enemy type name (EnemyPool.type_name(...), sim/sim.gd:2753) for contact hits, or "ranged" for projectile hits (sim/sim.gd:2737) — “charger” is TYPE_ELITE’s display name (sim/enemy_pool.gd:46), so a death_cause of "charger" is literally what we’re looking for.
• level_reached is already in SoakRun.execute()’s report dict (tools/soak/soak_run.gd:85).
• A fresh Sim is constructed per SoakRun.execute() call (tools/soak/soak_run.gd:37), so death_cause starts "" for every run — no reset/staleness risk between runs in a sweep.
• “Warp” (Chris’s term) is not a real ability — it’s the area-transition screen effect in main.gd. The real evasion tool is dash (PlayerState.dash_cd, SkilledAutoPlayer.wants_dash). Out of scope for this doc — noted here so a future dash-timing design doesn’t re-litigate it.

Non-goals

• No change to /sim — death_cause already does exactly what’s needed.
• No AI behavior change (orbit/dodge/dash-timing) in this pass — Chris explicitly chose “instrument first, then design” so any charger-specific AI fix is designed AFTER this data exists, not guessed at now.
• Not retrofitting the already-delivered combination-map artifact (docs/superpowers/plans/ 2026-07-17-ace-loadout-aware-weapon-range.md’s companion sweep) to use level instead of survival time — that was built to Chris’s own explicit spec at the time. Level becomes available for future sweeps, not a mandate to redo past ones.
• Co-op nuance: death_cause is a single field on Sim, latched at the FIRST pilot to reach hp<=0, which may not be the pilot whose death ends a co-op run (_all_pilots_down(), sim/sim.gd:2757). Soak’s --player=skilled --coop= runs are the target use case here and co-op skilled soaks are rare in practice; a per-pilot death_cause is a real future gap but out of scope for this pass — noted, not fixed.

Design

1. tools/soak/soak_run.gd

Add one field to the report dict returned by execute():

"death_cause": sim.death_cause,

Placed alongside "level_reached": sim.player.level, (soak_run.gd:85). Empty string means the run hit max_ticks without dying (mortal cap runs) or is an invuln coverage run that never took a fatal hit.

2. tools/soak/soak_runner.gd’s _analyze_fun_signals()

Three new aggregates, computed only over the existing skilled array (already filtered to player == "skilled" runs, soak_runner.gd:72-77):

• level_distribution: Array[int], one level_reached per skilled run — parallel to the existing death_tick_distribution, same shape, so it can be read the same way (mean/median/ spread) without new tooling.
• death_cause_breakdown: Dictionary[String, int] — count of each non-empty death_cause across skilled runs that actually ended in game_over. Runs that hit the tick cap without dying are excluded from this dict entirely (not bucketed as ""), since “didn’t die” isn’t a cause.
• mean_level_at_death_by_cause: Dictionary[String, float] — for each cause in the breakdown above, the mean level_reached across the runs that died to it. This is the number that actually tests Chris’s hypothesis: if "charger" (or TYPE_ELITE’s other names, if any reach the player via "ranged" too) shows a meaningfully higher mean level than other causes, that’s real evidence chargers specifically beat higher-level/later-game builds rather than being a generic early threat.

Testing

• tests/test_soak_run.gd: SoakRun.execute() IS GUT-tested today, so the new death_cause field gets real coverage — a new test runs a long enough mortal soak that the pilot actually dies and asserts report.death_cause is a non-empty string matching one of the sources _hurt_player can pass (an enemy type name or "ranged"); a second test asserts a short max_ticks run that never dies reports death_cause == "".
• tools/soak/soak_runner.gd’s _analyze_fun_signals() has no test file today — verified: no tests/test_soak_runner*.gd exists, and the script’s own header comment states the CLI runner “isn’t GUT-tested” (only SoakRun, the core it wraps, is). Every existing aggregate in that function (anticlimax_ratio, build_diversity, upgrade_dud_rate_ge_90pct, etc.) already follows this convention. The 3 new aggregates follow it too — no new test file. Verification is a real sweep run (scripts/soak.sh --runs=N --player=skilled ...) with the resulting JSON’s fun_signals block inspected directly for correctness on real data, same as how the existing fun-signals feature was itself verified when built.