Weapon FX v2 batch 1 — perf comparison

Weapon FX v2 batch 1 — perf comparison (2026-09-29)

Verdict: gpu_ms FAILS the ≤ 1.10 criterion. With all five looks on screen and only clean samples counted, v2 costs a real +0.23 ms per frame on the M5 (paired median over 48 clean pairs, v2 higher in 38/48). The criterion as the plan states it (v2 median ≤ v1 median × 1.10) gives 2.53 / 2.26 ms = 1.117. The noise control at L3 shows −0.04 ms, so the gap is real. None of the cost cuts tried in fix round 2 reduced it (below), so no shader change ships in that round. The render-thread cpu_ms shows no difference, but main-thread CPU is not shown to pass: the vsync-bound frame times cannot see it, and a direct microbenchmark puts v2’s bullet sync at ~1.5x legacy (+0.5 ms per frame at 1500 bullets on the M5; see “Main-thread bullet sync cost” below). Bolt GPU and CPU cost at scale was never loaded by this scene (0.56 live bolts). Draws are not resolvable at the ±5 criterion by this method (see Verdict). Fix round 3 reverted round 1’s look-changing orbit swirl cut, which bought no measurable saving (below). This needs Chris’s call: accept ~+0.23 ms on the M5 (and check it on the Apple TV), or cut look quality (the one lever that showed any effect was removing the orbit ring’s void accent).

Setup: survival, blaster + orbit + blade + scythe + nova all active at level 5 and evolved (active=blaster*,orbit*,blade*,scythe*,nova*), MacBook Pro M5, 1920x1080 window, --rendering-method mobile, perf_probe --no-breakdown, Godot 4.7.1 stable.

Method

1. GPU time via Vulkan. viewport_get_measured_render_time_gpu returns 0 on every frame under Metal 4.0 / Forward Mobile, so GPU time is measured with --rendering-driver vulkan (MoltenVK on the same M5). Draws and cpu_ms are also reported for Metal.

2. In-run A/B. The sim ticks on wall time, so separate launches never see the same scene. The probe’s --ab=N toggles VisualLook.weapon_fx_v2 inside one run, in ABBA order, waiting 0.90 s after each toggle (the longest look life, evolved Supernova 0.70 s, + 0.2 s) so the previous arm’s event looks have expired. Each arm samples 240 frames of draw counts, then 240 frames of timers.

3. Every look on screen. --targets=8 keeps 8 unkillable, stationary dev targets (Sim.dev_spawn_custom, the existing dev spawn hook) on a 70 px ring round the player, so the hold-fire-when-nothing-in-range weapons (blade, scythe, nova) fire throughout. No sim code changed.

4. Level-ups suppressed (fix round 2). A level-up panel freezes the sim (0.8 s auto-pick each, often chained). In round 1 that hit about half of many arms, so those samples measured a frozen scene with the event looks expired. Each auto-pick also changed the weapon build mid-comparison. During --ab the probe now holds every pilot’s XP at 0 each frame (dev-only, like the targets), and forces main._auto on (a Mac’s mouse-aim poll otherwise turns autopilot off). This suppresses most level-ups, not all: XP collected within a single frame can still cross the threshold before the probe zeroes it. The rejection rule (step 5) catches the frozen arms. The headline batches (c0, ctl) were fully clean (0 rejections, blaster 36-39 fires per arm throughout). Some cut-experiment runs still froze or auto-picked upgrades (e_noacc_ring, c1/cutA, c3, c4, e_band02, e_band02b); in e_noacc_ring seed 1235 blaster jumped from ~38 to 102-217 fires, so its later pairs compared a changed build. Those rows are experiments, not the headline figure.

5. Explicit rejection rule (fix round 2), printed by the probe as ab RULE. An arm is clean only if, over its whole sample:

   • main was never frozen on a level-up or dimension panel (frozen_frames == 0);
   • the sim advanced at least 0.90 s per wall second (sim_rate >= 0.90);
   • every look had live instances during the timer window (v2: bolts, orbit, blade, scythe, nova, flash; v1: projectiles, orbit shards, slash, nova, ring).

   A pair counts only if both arms are clean. Rejected pairs are re-run (up to 4× the pair count), and the MEDIAN lines cover clean pairs only; MEDIAN-ALL keeps the unfiltered view. In the two headline L0 batches, 0 of 48 attempts were rejected.

6. Independent samples. Three seeds (1234, 1235, 1236) per configuration, so the launches are different runs, not replicas. Pairs are 5 s of play apart (--ab-gap, default 5), so each launch’s 8 pairs sample 8 different moments of its run. The final L0 figure pools two 3-seed batches taken hours apart: 48 clean pairs from 6 launches.

7. CPU cost is measured, not filtered. Every arm reports cpu_ms (render-thread submit) and frame_ms (main-thread frame time), because a slower v2 frame would slow the wall-clock sim.

Why round 1’s 1.092 was not a pass

The re-review showed that every round-1 pair where v2 was not higher was a v2 arm in which the sim had stalled. Blaster fired about 18 times instead of about 38, and blade and scythe were often not drawing. The stalls fell on the same arms in every launch (a fixed seed plus ABBA). The new probe confirms the cause: rejected arms carry frozen= counts of 13 to 339 frames (level-up panels). On an unfiltered run, 11 v2 arms and 10 v1 arms were frozen, so v2 does not stall more by itself: the round-1 asymmetry came from where the level-ups fell. frame_ms is 8.33-8.38 ms (the 120 Hz vsync) for both arms in every sample, and the paired cpu_ms median is +0.00. That shows v2 adds no render-thread submit cost here and does not push the frame past the 8.33 ms vsync budget. It does not show v2 costs no main-thread CPU: a vsync-capped frame time hides any cost below the budget. The direct measurement is in “Main-thread bullet sync cost” below.

Evidence all five looks were drawn (final L0 data, 48 clean pairs)

Look	v2 events per sample (median / min)	v2 mean live instances	v2 samples with look live	v1 samples with look live
blaster (bolts + muzzle flash)	39 / 36	0.56 bolts, 0.71 flash	48/48, 48/48	projectiles 48/48
orbit	persistent	7.0 (ring + 6)	48/48	48/48
blade	8 / 7	0.34	48/48	slash 48/48
scythe	3 / 3	0.20	48/48	(slash, above)
nova	3 / 3	0.29	48/48	nova 48/48, ring 48/48

The fire counts are now the same in both arms (v1 blaster 38.5 / min 36).

Results — Vulkan (MoltenVK), L0, clean pairs only

Config	clean pairs	draws	gpu_ms	cpu_ms	frame_ms
v1 (legacy looks)	48	126.0	2.26	0.22	8.33
v2	48	168.8	2.53	0.22	8.33

• gpu: ratio of medians 1.117; median of the per-pair ratios 1.097; paired v2 − v1 median +0.23 ms, mean +0.20, sd 0.22; v2 higher in 38/48.
• Batches: batch 1 (c0) 1.125 (per-pair 1.081, paired +0.21, 18/24); batch 2 (ctl) 1.125 (per-pair 1.106, paired +0.24, 20/24).
• Draws: paired median +1.7, paired mean +29 ± 15 (se; sd 101, IQR −18 to +75); v2 higher in 27/48.

Results — L3 noise control (Vulkan, 3 seeds, 24 clean pairs)

At L3 the tier forces the legacy looks whichever way the flag is set. The v2 arm’s SHOWN line lists the legacy keys (proj orbit slash nova ring, 8/8 each), which confirms the fallback engaged.

Config	draws	gpu_ms	cpu_ms
v1 at L3	84.8	1.21	0.21
v2 flag at L3 (fallback)	126.7	1.15	0.22

• gpu: paired median −0.04 ms (v2 higher in 9/24), ratio 0.950.
• draws: paired median +0.15, paired mean +2 ± 19 (se; sd 93, IQR −37 to +61).
• Rejections: 26 of 50 attempts, mostly dead:ring. The legacy ring effect is not always live at L3; rejected pairs were re-run.

Results — Metal (the shipping driver), L0, 3 seeds, 24 clean pairs, no GPU timer

Config	draws	cpu_ms	frame_ms
v1	150.7	0.27	8.35
v2	188.9	0.27	8.35

Paired draws median +1.45, paired mean +22 ± 13 (se; sd 65, IQR −30 to +90), v2 higher in 12/24; paired cpu −0.01. 0 of 24 attempts rejected.

Per-look cost (clean attribution)

--hide-v2=<looks> zeroes those v2 layers just before each frame is drawn, so the A/B shows what the rest of v2 costs against full v1. Vulkan L0, seeds 1234+1235, 16 clean pairs each. These runs were taken with the interim ring shader from cut rows 1-2 below, which measured the same as the final one.

v2 looks hidden	v1 gpu	v2 gpu	paired median	v2 higher
none (final, for reference)	2.26	2.53	+0.23	38/48
orbit (ring + shards)	2.30	2.33	−0.01	7/16
bolts	2.28	2.55	+0.26	15/16
blade + scythe + nova + muzzle flash	2.24	2.35	+0.13	15/16

Reading:

• The orbit look carries essentially the whole gap. With it hidden, v2 matches v1, even though v1 still draws its own orbit sprites.
• Bolt cost at scale was NOT measured. Hiding them moved nothing, but the scene averaged only 0.56 live bolts (8 stationary targets at 70 px, so bullets die almost at once). The most-instanced look was never loaded; see “Main-thread bullet sync cost” below for its CPU side.
• The event looks cost roughly +0.1 ms. This is not additive with the orbit row, because hiding a v2 look does not remove v1’s equivalent, and the ±0.05 ms noise per 16-pair row is the same size.

Cost cuts tried in fix round 2 (none shipped)

All rows are Vulkan L0, clean pairs. Each row is measured against the same-session v1, so rows can be compared by their paired median.

Change (on top of the round-1 shaders)	pairs	ratio of medians	paired median	v2 higher	kept?
none: round-1 shaders, batch 1	24	1.125	+0.21	18/24	(baseline)
ring: per-shard trail loop only within 0.25 of the orbit (lossless)	24	1.113	+0.25	23/24	no, no effect
+ ring: lens fbm only on the bright core, mean elsewhere	24	1.099	+0.28	20/24	no, no effect
ring: band shaded to |d−1| < 0.25 (was 0.42), lens width 0.14 → 0.11	24	1.129	+0.29	23/24	no, no effect
experiment: ring with no fbm at all	16	1.135	+0.31	14/16	not shippable
experiment: ring band 0.2	16	1.036	+0.08	9/16	irreproducible: repeat gave +0.24 (13/16)
experiment: ring element/evolve accents off	16	1.081	+0.18	12/16	not shipped: removes the Event Horizon void motif
step 2 (bolt accents)	—	—	—	—	skipped: hiding the bolts entirely changes nothing
step 3: nova ember loop only on the ember annulus (lossless)	24	1.198	+0.44	24/24	no, worse
step 3 again, as an if around the constant-bound loop	24	1.208	+0.44	24/24	no, worse
none: round-1 shaders re-run as a same-session control (batch 2)	24	1.125	+0.24	20/24	(final)

Notes:

• Every look in this probe is evolved, so each carries an evolve accent code. Round 1’s “skip accents for slot 0” cuts are therefore never taken here, which is why they measured nothing.
• The ember-annulus cut is lossless on paper (every ember is < 1e-7 beyond 0.15 of its centre) but measured +0.2 ms worse in two forms, 24/24 pairs each. The branching presumably defeats a compiler optimisation of the fixed 21-iteration loop on this GPU. It is reverted.
• The ring’s cost did not respond to its per-pixel maths (fbm, trail loop) or to its band width. The only lever with any effect was the element accent motif (void bite, one fbm per pixel). That motif is the evolved look’s element identity, so cutting it is Chris’s call at the Task 16 sign-off, not a perf-pass decision.

Orbit swirl cut reverted (fix round 3)

Round 1 (commit 4a517b20) replaced the ring’s lens swirl, a two-fbm cross-fade around the whole circle, with a single fbm plus a ±π seam blend. That was a look change: a higher-contrast Event Horizon swirl. No measurement supports a saving from it. e_nofbm, which removed all ring fbm, gave paired +0.31 (16 pairs), no better than the baselines with fbm (+0.21 / +0.24; run-to-run spread about ±0.15), and the lens-fbm-on-core cut (row 2 above) had no effect. Removing one fbm therefore cannot have bought a measurable saving, so fix round 3 restored the original two-fbm swirl.

Kept, because they do not change the look: the discard outside the ring band (|d − 1| > 0.42 or past the ring extent), the lens gate (fbm skipped where the lens term is < 1e-3, which contributes < 0.0007 either way), and the slot-0 accent skips (never taken in this probe, since every look is evolved).

The headline figures above were measured with the round-1 swirl. No new perf run was taken after the revert. The expected effect is within noise, bounded by e_nofbm. The reverted shader was compile-checked in a real renderer on Metal and Vulkan (0 shader errors, orbit look live at 7.0 instances).

Verdict

• gpu_ms (v2 median ≤ v1 median × 1.10): FAIL. 2.53 vs 2.26 ms, ratio 1.117 over 48 clean pairs (paired +0.23 ms, 38/48; L3 control −0.04). The median of the per-pair ratios, 1.097, sits just under the line, but the two batches straddle it (1.081 and 1.106). A true gap of ~+10 % is not a pass under either reading.

• Draws (v2 ≤ v1 + 5): not resolvable by this method. Paired v2 − v1, against the L3 control:

  	paired median	paired mean ± se	sd
  Vulkan L0 (48 pairs)	+1.7	+29 ± 15	101
  Metal L0 (24 pairs)	+1.45	+22 ± 13	65
  L3 control (24 pairs)	+0.15	+2 ± 19	93

  • With a standard error of about 14 draws, a ±5 criterion cannot be decided either way. The mean sits about 2 se above zero on both drivers while the control sits at +2, so v2 may add draws; the medians alone do not show it.
  • The per-arm medians (126.0 vs 168.8 Vulkan, 150.7 vs 188.9 Metal) are unpaired scene noise: at L3 both arms draw identical legacy looks, yet the per-arm medians differ by a similar margin (84.8 vs 126.7). Draw counts swing from about 70 to 600 between moments of a run.
  • Resolving it needs a structural count (the layer draw calls v2 adds or removes per look, from the renderer code or one frozen-scene toggle), or Chris’s call on these numbers.

• cpu_ms (render thread): no difference. Paired cpu +0.00 (Vulkan) and −0.01 (Metal).

• frame_ms (main thread): NOT a pass. It is identical (8.33-8.35 ms), but it is vsync-bound, so it cannot show any main-thread cost below the frame budget. The direct measurement below shows v2’s bullet sync is ~1.5x legacy per bullet.

• L3 fallback: PASS. The L3 v2 arm reports the legacy looks live and times like v1 (paired −0.04). From round 0: reports/fxv2/l3-vulkan.png (v2 flag on, L3) shows the legacy looks.

Caveats:

• Hardware. This is an M5 laptop GPU measured through MoltenVK, not the shipping Metal driver. +0.23 ms on a ~2.3 ms frame says nothing reliable about the Apple TV’s budget. The orbit look is the thing to watch there.
• Run-to-run variation. The same configuration’s paired median varied between about +0.1 and +0.4 ms across launches. 16-pair experiments are indicative at best; one of them, “ring band 0.2”, did not reproduce.

Main-thread bullet sync cost (final review, 2026-09-29)

Added after the final review found the CPU “PASS” above rested on vsync-bound frame times and a scene with ~0.56 live bolts. Method, as the reviewer ran it: a headless microbenchmark on the M5, 100 calls each, timing PlayerBoltRenderer.sync_bolts (v2) against the legacy SwarmRenderer.sync on the same projectile pool.

Bullets	v2	legacy
100	0.098 ms	0.064 ms
500	0.480 ms	0.305 ms
1500	1.447 ms	0.923 ms

• v2 is about 1.5x legacy, or +0.5 ms per frame at 1500 bullets on the M5. PROJ_CAP is 4000, and the Apple TV CPU will be several times slower.
• The main per-bullet extras are in fx/weapons/player_bolt_renderer.gd (the per-bullet loop): the _slot() dictionary lookup on the context string and the accent_slot() linear scan. A possible fix is to cache the slot per context for the frame, keyed on kind (an int) rather than the string.
• Bolt GPU cost at scale is still unmeasured. The Apple TV check needs a high-bullet-count probe config (for example Auto-Cannon + scatter against a far target ring) so bolts are actually on screen.
• Not re-measured here: this fix round only corrected the claims (re-measurement is Task 16 work).

Bug found on the way (round 0)

The first probe run logged 12 l-value required "material" shader compile errors:

• player_bolt.gdshader’s uniform float dim (Task 13) shared its name with a local dim inside element_accents.gdshaderinc’s accents_all().
• Godot resolves the local to the uniform, so the v2 bolt shader failed to compile on the real renderer. Headless GUT never compiles shaders.

Fixed by renaming the local to acc_dim. Guard 7 in tests/test_weapon_fx_guards.gd now fails on any uniform/include-local name clash. All numbers here were taken after the fix, with zero shader errors in the logs.

Reproduce

for s in 1234 1235 1236; do
  godot --path . --rendering-method mobile --rendering-driver vulkan --always-on-top \
    res://tools/perf_probe/perf_probe.tscn -- --mode=survival \
    --weapons=blaster,orbit,blade,scythe,nova --evolve --v2=1 --level=0 --seed=$s \
    --no-breakdown --ab=8 --targets=8
  git diff --quiet project.godot || git checkout -- project.godot
done
# attribution: add --hide-v2=orbit (or bolts, or blade,scythe,nova,blaster); L3 control: --level=3;
# Metal: --rendering-driver metal (draws/cpu only). Only `clean=1` lines count.