Fidelity: the D1 device model (Apple TV GPU cost bins and per-device CPU multipliers)

Fidelity: the D1 device model

What the Apple TV (and the other test devices) actually cost, read from production telemetry instead of from a Mac run. It exists because a Mac run cannot predict Apple TV cost: in real play the Apple TV 4K (A12) is GPU-bound (mean GPU time 14.96 ms inside frames that average 55.7 fps), and the Mac’s GPU has far more fill rate. The numbers are aggregate SELECTs over the telemetry samples table, committed as two small JSON files; the game and the tests never read live D1.

Written 2026-10-22 from telemetry received up to 2026-10-22T01:49:46Z (samples taken 2026-10-11 to 2026-10-18 for the GPU bins, 2026-10-09 to 2026-10-18 for the CPU multipliers). Follow-up that needs a real GPU capture: Issue #84 (“ATV GPU fill-rate pass + 4.8 sim-CPU profile”).

The short version

• About a fifth of sampled Apple TV frames are already over budget. In human play on the Apple TV 4K, 22.0% of sampled frames take more than the 16.7 ms a 60 Hz frame allows on the GPU (95% CI 2.4%-38.7%, wide because the data are 60 sessions from 6 installs, nearly all of them Dev play). Mean GPU time is 14.96 ms.
• Cost rises with draw calls and with primitives: 13.4 ms (4.3% over budget) below 100 draws up to 20.8 ms (82.8% over) at 600 and more; binned by primitives instead, 13.4 ms (3.5% over) below 5,000 prims up to 23.7 ms (97.8% over) at 45,000 and more. Only the two lowest bins of each kind are high confidence; every draws bin from 200 up and every prims bin from 10,000 up is low confidence.
• A change that adds draws or prims gets priced before it ships (AtvGpuModel.delta, printed by perf_probe.gd): the extra GPU ms and the expected extra share of sampled ATV frames over 16.7 ms, each a one-sided point estimate with a 95% CI, never a bound and never a prediction. A prims-only rise (more instances in an existing draw call) is priced too. Draws and prims are two readings of ONE load: read either, never add them. A small shift usually cannot be told apart from zero (+80 draws from 300 is +11.1 percentage points, 95% CI -0.6 to +13.4), and the output says so.
• “Draws did not move” never means “safe”. The bins are blind to fill, overdraw and full-screen passes: an evolved Supernova zone adds about +1.5 ms of GPU time when it appears (558 same-session flips) while moving draws by +4 on average.
• “The Mac says fine” is not evidence for the Apple TV. Mac Dev builds never report gpu_ms (0 of 32,919 rows), and an M-series GPU is not a fill-rate proxy for an A12.
• CPU: the Apple TV’s sim tick is about 2x the M5’s: 1.99x at 0-24 enemies (ratio of medians, 95% CI 1.95-6.87) and 2.09x at 150 and more (95% CI 1.67-2.56). Only 4 of the 12 device cells have enough data for a ratio at all (2 sessions and 10 rows on each side; the other 8 publish none), and none of the 4 is high confidence because the Mac side rests on at most 8 sessions. Treat them as order of magnitude.

This is a regression tripwire and a ranking aid. It is NOT a frame-time prediction.

What is in the box

File	What it is
tools/device_model/atv_gpu_bins.json	Schema atv-gpu-bins/1. Six draw-call bins and six primitive bins for the Apple TV 4K (AppleTV11,1): frames, sessions, mean and p90 GPU ms, share of frames over 16.7 ms, each with a session-cluster 95% interval and a confidence, plus the two within-session slopes with intervals, 1,000 stored bootstrap replicates of the bin shares (the intervals for a shift), correlations and the Supernova blind-spot numbers
tools/device_model/device_mult.json	Schema device-mult/1. Sim tick_ms multipliers vs the M5 Mac for profiles tv, iphone, ipad, a12_phone, keyed by enemy bucket (0-24, 25-99, 100-149, 150+), never one scalar per device. A cell is ok (a ratio of medians with a session-bootstrap interval) or insufficient (no ratio of any kind)
tools/device_model/atv_gpu_model.gd	class_name AtvGpuModel (RefCounted). estimate(draws, prims), delta(...), paired_delta(...), and the printable estimate_lines / delta_lines / paired_delta_lines / notes_lines
tools/perf_probe/perf_probe.gd	Samples draws AND prims at the Apple TV’s own 1280x720 logical view, prints the estimate next to every measured draws/prims, the caveat, the scene-resemblance statement and the “Mac says fine” trap, and prices the --ab change pair by pair
tools/device_model/fit_atv_gpu.py, fit_device_mult.py, refresh.py	The fits and the one-command refresh (stdlib only), with an offline replay from a recording
tools/device_model/doc_tables.py	Prints this page’s tables from the committed JSON; a test fails while the page and the JSON disagree
tools/device_model/sql/*.sql	Every query that produced a number: one read-only statement per file, one shared population filter
tools/device_model/fidelity_check.py	Offline check of the committed JSON (the d1-device-model entry in tools/device_sim/registry.d/)
tests/test_atv_gpu_model.gd, tools/device_model/tests/fid_device_model.test.py	GUT tests of the model and unittest of the fits and the committed files. They pin the committed JSON, never live D1

How to ask the model

From the perf probe

scripts/dev/godot-window-lock.sh godot --path . --rendering-method mobile --always-on-top \
  res://tools/perf_probe/perf_probe.tscn -- --mode=survival --no-breakdown
git checkout -- project.godot audio/bus_layout.tres      # always, after any windowed run

The window is 1920x1080, which is what the Apple TV renders, but the probe’s LOGICAL view (the world area on screen) is the project’s 1280x720, the same world area an Apple TV player sees. The first version of the probe ran a 1920x1080 logical view, which is 2.25x that world area and can put more of the scene in frame than an Apple TV player sees; --wide-view restores it for captures that need it, and the scene note then calls it synthetic load.

One real output (a Mac, survival mode, 192 enemies, first 30 sampled frames):

[perf_probe] mode=survival webs=0 enemies=192 TOTAL draws=121.8 prims=6091
[perf_probe] est ATV GPU ms / share of frames over 16.7 ms: 13.5 ms (95% CI 12.1-14.2) / 11.5% (95% CI 0.9-19.7%), draws=121.8 in bin 100-199
[perf_probe]   by prims (the same load read a second way, never added): 13.3 ms (95% CI 12.2-14.2) / 9.4% (95% CI 2.6-16.3%), prims=6090.7 in prims bin 5000-9999, confidence high
[perf_probe]   basis: 1187 sampled frames from 39 sessions on 3 installs; share = share of sampled frames whose GPU time exceeds 16.7 ms; confidence high
[perf_probe]   caveat: draws and prims are blind to overdraw and full-screen passes (the Supernova example), so this is a regression tripwire and ranking aid, NOT a frame-time prediction
[perf_probe]   scene: this estimate only transfers if the scene you measured resembles the ATV scene the bins were fitted on (human-played runs on the Apple TV, builds 303-335, single frames sampled about every 5 s) and was viewed at the ATV's own 1280x720 logical size; a quiet or synthetic probe scene, or a wider probe view, is not that scene
[perf_probe]   trap: 'Mac says fine' is not evidence for the Apple TV: the Mac editor's Metal gpu_ms reads 0.00 and the M-series GPU has far more fill rate than the Apple TV's A12, so a fill-bound cost that drops ATV frames can be invisible on the Mac. Only an ATV measurement can say a change is cheap; this model prices only ADDED draws or prims (point estimates with intervals), so it can say a change is NOT cheap, never that it is
[perf_probe]   share: an extra-share figure compares bins filled by different sessions, and the busy bins also carry more Supernova-zone frames (fill cost draws and prims cannot see), so it can overstate or understate: weigh its interval and the lower confidence, not the point (zone frames are 4% of the 0-99 draws bin and 84% of the 600+ one)
[perf_probe]   probe scene: mode=survival enemies=192 view=1280x720 window=1920x1080 seen=1280x720; no synthetic load injected, but ATV samples are human play, so compare the enemy count and weapon set before trusting the bin
[perf_probe] v2=true level=0 weapons= evolve=false gpu_ms=0.00 cpu_ms=0.21
[perf_probe]   gpu_ms=0.00 means NOT MEASURED here (stock Godot 4.7.1 Metal on the Mac editor reports no GPU time; add --rendering-driver vulkan). It does not mean the GPU is free: see the est ATV line above.
[perf_probe] DONE

Read that output in this order: the confidence and the interval, then the basis, then the caveat and the scene statement. The probe prints the one load twice, once from its draws bin and once from its primitives bin (here 11.5% and 9.4% of frames over budget, both high confidence); the two lines are alternative readings and are never added. The scene line shows view=1280x720 window=1920x1080 seen=1280x720: seen is the world area the viewport really shows (a window the OS shrinks, or a shape that is not 16:9, widens it under aspect expand), and “no synthetic load injected” appears only when seen is the Apple TV’s 1280x720 and none of --webs, --zones, --targets, --hide-v2 or --weapons is set. With --wide-view the same line reads:

[perf_probe]   probe scene: mode=survival enemies=200 view=1920x1080 window=1920x1080 seen=1920x1080; SYNTHETIC load (logical view 1920x1080 is 2.25x the Apple TV's 1280x720 world area): the ATV bins were not fitted on this scene, so read the estimate as a ranking aid only

--ab=N prints a compact estimate per arm, the estimate of each median, and a DELTA block that prices the PAIRED v1 to v2 change: the median of the per-pair differences, with the pair count printed. It is not the difference of the two arm medians, which moves whenever the scene drifts between pairs: pairs 30 to 40, 60 to 62 and 100 to 110 have a median paired difference of +10 draws while their arm medians differ by only 2. Here is the model fed 300 draws before and 380 after, to show the wording (a worked example, not a measurement). The bins’ own intervals overlap, so the extra share’s interval includes zero:

ATV cost of this change (one-sided point estimates, each with a 95% CI; not a prediction):
  by draws: +80.0 draws (300.0 -> 380.0) x 0.00929 ms/draw is about +0.74 ms (95% CI +0.23 to +0.80 ms), a draws-only point estimate that may overstate (load that climbs with run time) or understate (fill, overdraw and full-screen passes add no draw)
    expected extra share of sampled frames whose GPU time exceeds 16.7 ms: +11.1 percentage points (95% CI -0.6 to +13.4 pp) along the bins' trend line; this cannot be told apart from zero
    share read from bin 200-299 (share 20.5%, 95% CI 3.4-41.8%, confidence low), bin 300-399 (share 32.4%, 95% CI 4.4-45.7%, confidence low), bin 400-599 (share 61.8%, 95% CI 5.0-64.2%, confidence low); lower confidence: low (200-299: one session supplies 63% of the rows (limit 50%); 300-399: only 14 sessions (need 15); 300-399: one session supplies 64% of the rows (limit 50%); 400-599: only 12 sessions (need 15); 400-599: one session supplies 90% of the rows (limit 50%)); 22.0% of sampled ATV frames already exceed 16.7 ms
  by prims: not given
  draws: 80 extra draws are priced at the within-session slope; the figure cannot see any fill, overdraw or full-screen cost the change also adds; prims were not given, so a rise in prims (more instances in an existing draw call) was not checked: pass the measured prims too
  caveat: draws and prims are blind to overdraw and full-screen passes (the Supernova example), so this is a regression tripwire and ranking aid, NOT a frame-time prediction

A real clean --ab=2 run on a Mac (--mode=survival --weapons=orbit --v2=1 --level=0 --seed=1234 --ab-gap=1) had v2 draw fewer calls but more primitives (an instanced look): a draws-only model would price that at zero. The model prices the prims rise and says in words that the extra share cannot be told apart from zero:

[perf_probe] ab DELTA v2=false->true ATV cost of this change (one-sided point estimates, each with a 95% CI; not a prediction):
[perf_probe] ab DELTA v2=false->true   paired A/B: 2 clean pairs; median paired difference -30.9 draws (0 of 2 pairs rose) and +984.6 prims (1 of 2 pairs rose); before = median of the v2=false arm (143.9 draws, 3384.1 prims), after = before + that median paired difference (113.1 draws, 4368.7 prims); only 2 pairs, so the median difference has no spread to judge
[perf_probe] ab DELTA v2=false->true   by draws: 30.9 fewer draws (143.9 -> 113.1): no saving can be claimed from draws alone
[perf_probe] ab DELTA v2=false->true   by prims: +984.6 prims (3384.1 -> 4368.7) x 0.2147 ms per 1000 prims is about +0.21 ms (95% CI +0.12 to +0.22 ms), a prims-only point estimate that may overstate (load that climbs with run time) or understate (fill, overdraw and full-screen passes add no primitive)
[perf_probe] ab DELTA v2=false->true     expected extra share of sampled frames whose GPU time exceeds 16.7 ms: +1.1 percentage points (95% CI -0.1 to +2.3 pp) along the bins' trend line; this cannot be told apart from zero
[perf_probe] ab DELTA v2=false->true     share read from bin 0-4999 (share 3.5%, 95% CI 1.4-6.5%, confidence high), bin 5000-9999 (share 9.4%, 95% CI 2.6-16.3%, confidence high); lower confidence: high; 22.0% of sampled ATV frames already exceed 16.7 ms
[perf_probe] ab DELTA v2=false->true   draws fell by 31; no saving is claimed: the model is one-sided and only prices ADDED load, and the same change may add fill or full-screen cost it cannot see; prims rose by 985 and are priced at the within-session prims slope
[perf_probe] ab DELTA v2=false->true   caveat: draws and prims are blind to overdraw and full-screen passes (the Supernova example), so this is a regression tripwire and ranking aid, NOT a frame-time prediction

From code

var model := AtvGpuModel.new()                         # loads tools/device_model/atv_gpu_bins.json
var e: Dictionary = model.estimate(draws, prims)       # bucket, gpu_ms, p_over_budget, n, caveat + CI, confidence, prims bin
for line in model.estimate_lines(draws, prims): print(line)
for line in model.delta_lines(draws_before, draws_after, prims_before, prims_after): print(line)
# per-pair arrays from an A/B run (the pair index lines the two arms up)
for line in model.paired_delta_lines(before_draws, after_draws, before_prims, after_prims, "the old look"): print(line)

estimate returns ok, error, bucket, gpu_ms, gpu_ms_ci95, gpu_ms_p90, p_over_budget, p_over_budget_ci95, n, n_sessions, n_installs, top_session_share, confidence, confidence_reasons, budget_ms, caveat, prims_note, draws, prims, and, when prims are given, the second reading: has_prims_bin, prims_bucket, prims_gpu_ms, prims_gpu_ms_ci95, prims_p_over_budget, prims_p_over_budget_ci95, prims_confidence. A fractional draw count (the probe averages frames) is floored into its bin. A missing or malformed bins file gives ok = false and a message, never a push_error, and the caveat and the Mac trap still print. The loader refuses a document with an empty bin (every statistic in it would be null) and names the bin.

delta returns ok, error, verdict (added, flat or removed), two readings draws and prims (each with its own verdict, priced, extra, extra_gpu_ms and extra_gpu_ms_ci95, extra_share_over_budget and extra_share_over_budget_ci95, extra_share_indistinct_from_zero, the bins it read with their share, interval and confidence, and share_confidence), claims_safe (always false), note and caveat. There is deliberately no combined ms or share field. paired_delta takes the per-pair arrays, prices the median of the per-pair differences and returns the pair count and how many pairs rose beside the same delta.

The one-sided player-benefit translation

delta(draws_before, draws_after, prims_before, prims_after) answers “what does this change cost a player on the Apple TV, as far as draws and prims can tell”. It is one-sided on purpose, and it reads draws and prims as two separate readings of the same load:

• Added load is priced as a point estimate with an interval. Draws added: N extra draws at the within-session slope of 0.00929 ms per draw (95% CI 0.00292-0.00999), so +100 draws is about +0.93 ms (95% CI +0.29 to +1.00). Prims added: P extra prims at 0.2147 ms per 1,000 prims (95% CI 0.1177-0.2244). Each is printed as a draws-only or prims-only point estimate that may overstate (part of each slope is late-run load that climbs with run time) or understate (fill, overdraw and full-screen passes add no draw and no primitive). Neither is a bound in either direction.
• The extra share of frames over budget is read off the bins’ trend line (piecewise linear through the bin centres, flat beyond the first and last centre so it never extrapolates) and carries its own 95% CI from the 1,000 stored whole-session bootstrap replicates: for each replicate the trend is evaluated at the two points, and the interval is the 2.5% to 97.5% percentile of the differences. When the lower end is at or below zero the output says “this cannot be told apart from zero”. Beyond the last bin centre the trend is flat, so the share is zero by construction; the output says so and points at the ms figure.
• Every bin the trend reads is named with its share, its interval and its confidence, followed by the lowest confidence among them and the baseline (“22.0% of sampled ATV frames already exceed 16.7 ms”).
• A prims-only rise is priced. Instancing (fewer draws, more instances per draw) raises primitives while draws stay flat or fall. The prims reading prices the rise on its own, and a draws reading that fell books no saving.
• Draws and prims are never added. They climb together (r 0.668 and 0.676 with GPU time), so each reading already includes most of the other’s load.
• Flat or removed: nothing is claimed. claims_safe is always false; a removal never books a saving, because the draws removed may have been the cheap ones and fill cost can move without draws.
• The slope and the bin shift are not added to each other either. The slope is the within-session estimate, the bins span sessions, so the output prints the slope as milliseconds and the bin trend as percentage points, side by side.
• ms_per_draw and ms_per_1000_prims in the JSON are point estimates (the within-session slopes) with intervals ms_per_draw_ci95 and ms_per_1000_prims_ci95. Neither is a statistical bound, and neither is described as one anywhere the model prints.
• A/B pairs are priced by paired_delta: before is the median of the before arm, after is before plus the median of the per-pair differences. The probe prints the pair count and how many pairs rose, and warns when there is a single pair (a median difference with no spread).

The Apple TV GPU bins

Population: Apple TV 4K (A12, AppleTV11,1), builds 303 to 335 (the 4.8-dev fork builds, the first that report gpu_ms), human play (autopilot excluded), not frozen. One row is one single-frame sample taken about every 5 s of play, so every share below is the share of SAMPLED FRAMES whose GPU time exceeds 16.7 ms (all 51 ATV launches on builds 303-335 report 60 Hz, so 16.7 ms is the right budget; that count is from the pre-fit review described under “Facts corrected alongside”). samples rows: 8,541 from 60 sessions on 6 installs.

Draw calls	Sampled frames	Sessions	Mean GPU ms (95% CI)	p90 GPU ms	Share over 16.7 ms (95% CI)	FPS	Top session’s share of rows	Confidence
0-99	4,761	58	13.4 (12.9-13.7)	14.7	4.3% (2.0%-8.0%)	59.3	22%	high
100-199	1,187	39	13.5 (12.1-14.2)	17.2	11.5% (0.9%-19.7%)	57.9	38%	high
200-299	537	21	14.7 (12.9-15.8)	19.5	20.5% (3.4%-41.8%)	56.5	63%	low
300-399	275	14	15.8 (12.3-16.5)	21.7	32.4% (4.4%-45.7%)	53.3	64%	low
400-599	652	12	18.5 (12.6-18.7)	25.7	61.8% (5.0%-64.2%)	49.0	90%	low
600+	1,129	10	20.8 (13.1-20.9)	27.5	82.8% (9.5%-83.8%)	42.2	97%	low
All bins	8,541	60	14.96 (13.02-16.47)		22.0% (2.4%-38.7%)	55.7	33%

Intervals are 95% session-cluster bootstrap intervals (see Method). The rule for confidence is: low when fewer than 15 sessions contribute or one session supplies more than half the rows.

Reading the bins

• The ordering is the robust finding. Mean GPU time climbs from 13.4 to 20.8 ms across the bins, and the share over budget from 4% to 83%. The exact level of any bin from 200 draws up is not robust: 21, 14, 12 and 10 sessions, and a single session supplies 63%, 64%, 90% and 97% of those rows. Build 324 alone supplies 66%, 67%, 92% and 98%.
• A row-level interval is far too narrow: 4 to 17 times narrower than the session-aware one per bin (20 times for the all-rows total), because the rows inside a session are strongly correlated. The same bins with and without respecting sessions:

Draw calls	Mean GPU ms: row-level CI (too narrow)	Mean GPU ms: session-cluster CI	Share over 16.7 ms: row-level CI	Share over 16.7 ms: session-cluster CI
0-99	13.3-13.5	12.9-13.7	4%-5%	2%-8%
100-199	13.4-13.7	12.1-14.2	10%-13%	1%-20%
200-299	14.4-15.0	12.9-15.8	17%-24%	3%-42%
300-399	15.4-16.3	12.3-16.5	27%-38%	4%-46%
400-599	18.2-18.8	12.6-18.7	58%-66%	5%-64%
600+	20.6-21.1	13.1-20.9	81%-85%	10%-84%

• The store is barely in the sample. 98.7% of rows are Dev installs; 108 of the 8,541 rows come from App Store installs. Do not quote the 22.0% as a store-player number; it describes the team’s Dev play on the Apple TV.
• The p90 column is a descriptive percentile (linear interpolation between the bracketing ranks), without an interval.

The same frames binned by primitives

The same 8,541 frames, re-binned on RENDER_TOTAL_PRIMITIVES_IN_FRAME instead of draw calls (the Mean draws column shows how the two readings line up):

Primitives	Sampled frames	Sessions	Mean GPU ms (95% CI)	Share over 16.7 ms (95% CI)	Mean draws	Top session’s share of rows	Confidence
0-4999	4,240	58	13.4 (12.8-13.6)	3.5% (1.4%-6.5%)	33	25%	high
5000-9999	1,227	40	13.3 (12.2-14.2)	9.4% (2.6%-16.3%)	111	20%	high
10000-19999	987	17	14.5 (12.8-15.1)	18.1% (3.0%-27.8%)	231	53%	low
20000-29999	742	9	17.1 (14.1-18.1)	47.2% (9.5%-72.5%)	432	88%	low
30000-44999	1,069	5	19.8 (15.4-20.8)	76.4% (42.9%-88.1%)	677	98%	low
45000+	276	4	23.7 (18.6-23.8)	97.8% (97.8%-100.0%)	965	99%	low

• It is the same story: 13.4 to 23.7 ms and 3.5% to 97.8% over budget as primitives climb, with the same confidence rule. These are alternative readings of ONE load, never added to the draws bins.
• The top bins are single sessions. The 30,000-44,999 and 45,000+ bins hold 5 and 4 sessions, and one session supplies 98% and 99% of their rows, so read them as “that session”, not as a population. The 45,000+ bin’s interval (97.8%-100.0%) only reflects which of its 4 sessions land in a resample.
• Instancing is where this reading earns its place. A change that adds instances to an existing draw call leaves the draws reading flat while primitives rise; only the prims bins and the prims slope see it.

Why draw calls are not enough

Correlation of gpu_ms with each recorded predictor over the 8,541 rows (descriptive only, rows inside a session are not independent):

Predictor	Pearson r with gpu_ms (rows)
prims	0.676
draws	0.668
tick_ms	0.619
webs	0.636
run_time	0.623
zones	0.576
enemies	0.040

• Draws are partly a proxy for late-run FX load. webs (.636) and run_time (.623) correlate with GPU time about as strongly as draws do, while enemies barely does (.040). A change that removes draws late in a run may not remove the load the draws were standing in for, and a late-run feature raises draws, webs and cost together.
• The within-session draw slope is 0.929 ms per 100 draws (95% CI 0.292-0.999, r 0.547, from 23 sessions with at least 20 rows, 7,248 rows), against 0.965 pooled across sessions. The within-session prims slope is 0.215 ms per 1,000 prims (95% CI 0.118-0.224, r 0.557, the same sessions and rows), against 0.192 pooled. Within a session the slope removes differences between sessions (a draw-heavy session is also a fill-heavy one) but not the co-movement inside a session, so part of it is late-run load rather than the price of one draw.
• The Supernova blind spot. In this data zones is 0 or 1 and only the evolved Nova (Supernova) produces it: 2,298 rows from 7 sessions. Those rows average 512 draws against 102 without, so the top draw bins are confounded with the zone:

Draw calls	Frames	Frames with a Supernova zone	Sessions with one	Mean GPU ms, zone on / off	Share over 16.7 ms, zone on / off
0-99	4,761	193 (4%)	5	15.8 / 13.3	26% / 3%
100-199	1,187	303 (26%)	5	14.9 / 13.1	24% / 7%
200-299	537	235 (44%)	4	15.2 / 14.3	25% / 17%
300-399	275	142 (52%)	2	16.9 / 14.7	41% / 23%
400-599	652	481 (74%)	2	19.5 / 15.7	73% / 32%
600+	1,129	944 (84%)	1	21.4 / 17.9	87% / 62%

A zone appearing is 558 same-session up-flips (adjacent telemetry windows 7 s or less apart): mean GPU change +1.53 ms (median +0.52, 95% CI 0.91-5.72 over sessions, and 87% of the flips come from one session). Pooling both directions gives 1,116 flips at +2.63 ms. The mean draws change across those flips is only +4.4 (median -2), so the bins predict about +0.029 ms (0.045 ms for the pooled flips). The zone costs GPU time that no draw count can see. Inside the 0-99 draws bin alone, zone frames cost 15.8 ms (26% over budget) against 13.3 ms (3% over budget) without. The same confound is why an extra-share figure compares bins that different sessions filled, and why the busy bins carry more zone frames than the quiet ones.

Per-device CPU multipliers

device_mult.json answers “how much slower is the sim tick on this device than on the M5 Mac, at this enemy count”. The metric is tick_ms, the mean sim tick time over a ~5 s sampling window (CPU only; it says nothing about GPU). The primary statistic is the ratio of medians of per-sample tick_ms (device over Mac) because Apple TV 150+ is heavy tailed (mean 2.52 ms, median 1.69 ms), with a 95% session-bootstrap interval (device and Mac sessions resampled independently). Each cell also carries the ratio of means (a labelled comparison, with its own interval), the ratio of worst-tick (tick_max_ms) medians, the per-Mac-session range, and the same ratio with autopilot rows included. Filter: ticks above 0, not frozen, build 283 or later (the first with tick_ms), human rows for the primary and every row for the “incl. autopilot” variant.

A cell needs at least 2 sessions and 10 rows on each side to publish a ratio at all. Under that its status is insufficient, every ratio field is null, and only the raw medians are shown.

Profile	Enemies	Median tick ms, device / M5	Ratio of medians (95% CI)	Ratio of means	Worst-tick ratio (medians)	Device rows / sessions	Mac rows / sessions	Same ratio incl. autopilot rows	Confidence
tv	0-24	0.989 / 0.496	1.99 (1.95-6.87)	2.48	2.19	7,119 / 43	12,290 / 4	1.97	low
tv	25-99	1.403 / -	insufficient	-	-	1,872 / 27	0 / 0	-	low
tv	100-149	2.623 / 0.492	insufficient	-	-	226 / 31	1 / 1	-	low
tv	150+	1.689 / 0.809	2.09 (1.67-2.56)	2.95	2.74	247 / 46	15 / 8	2.14	low
iphone	0-24	0.429 / 0.496	0.86 (0.78-3.02)	1.00	0.87	417 / 24	12,290 / 4	0.92	low
iphone	25-99	0.686 / -	insufficient	-	-	40 / 6	0 / 0	-	low
iphone	100-149	0.894 / 0.492	insufficient	-	-	9 / 5	1 / 1	-	low
iphone	150+	0.817 / 0.809	1.01 (0.81-1.20)	1.01	0.94	32 / 11	15 / 8	2.05	low
ipad	0-24	0.566 / 0.496	insufficient	-	-	7 / 1	12,290 / 4	-	low
ipad	25-99	- / -	insufficient	-	-	0 / 0	0 / 0	-	low
ipad	100-149	- / 0.492	insufficient	-	-	0 / 0	1 / 1	-	low
ipad	150+	- / 0.809	insufficient	-	-	0 / 0	15 / 8	-	low
a12_phone	0-24	no data				0 / 0			low
a12_phone	25-99	no data				0 / 0			low
a12_phone	100-149	no data				0 / 0			low
a12_phone	150+	no data				0 / 0			low

Eight of the 12 device cells are insufficient: the Mac has no rows at 25-99 enemies and one row at 100-149, and the iPad has one session of 7 rows (0-24 enemies) and nothing else. A cell with no human device rows has no ratio and says so in its JSON note; the autopilot-inclusive ratio is shown only beside a human one. The 4 cells with a ratio are all low confidence (rule: high needs 15 or more sessions on each side, neither side dominated by one session, human and all-rows ratios within 25%, and comparable mean enemy counts inside the bucket); each cell lists its reasons in the JSON. The a12_phone profile has no cells at all.

Reading the multipliers

• Apple TV at 0-24 enemies: 2.0x (median, 95% CI 1.95-6.87) to 2.5x (mean, 95% CI 1.98-5.08). One Mac session supplies 99.96% of the Mac rows (mean 0.0 enemies) and the other three share the remaining 0.04%, so the interval is lopsided: its low end is what the big session says, its high end is what the small ones allow. Using each Mac session as the baseline in turn gives 2.5x to 8.4x (ratio of means). A one-vote-per-session version (the median over the 43 Apple TV sessions of each session’s mean tick, divided by the same median over the Mac’s sessions) gives 2.63x. The Apple TV rows average 6.4 enemies against 0.0 on the Mac, so part of this ratio is more enemies, not slower silicon.
• Apple TV at 150 and more enemies: 2.1x (median, 95% CI 1.67-2.56), 2.9x (mean, 95% CI 1.94-4.47), 2.7x (worst-tick median). 15 Mac rows from 8 sessions, which keeps this cell low confidence even with the tighter interval.
• Apple TV at 25-99 and 100-149 enemies: no ratio. The Mac has no rows at 25-99 and one row at 100-149; a single row is not a baseline.
• iPhone16,2 (A17 Pro): about 1.0x at 0-24 (0.86 median with 95% CI 0.78-3.02, 1.00 mean) and 1.01x at 150 and more with human rows (95% CI 0.81-1.20; 32 rows against the Mac’s 15), but 2.05x if autopilot rows are included, so that cell is low confidence on its own disagreement.
• iPad13,8: one session, 7 rows, at 0-24 enemies. Below the minimum, so no ratio. Nothing else.
• a12_phone: no data. There is no A12 phone in the lab and no telemetry rows with a measured sim tick from one, so no multiplier is claimed.
• Part of every ratio is the engine build. Mac Dev builds run stock Godot 4.7.1 while the Apple TV and iPhone builds run the 4.8-dev fork, about 8% slower on sim CPU (docs/research/2026-10-11-godot-48-gpu-ab-and-fill-rate.md).

Cross-check against the sim benchmark

For a high-enemy Mac baseline do not lean on 15 field rows: tools/sim_bench (headless, seed 1234, blaster only, late half of a 3,600-tick run) measured 695 microseconds per tick on this Mac in one run on a shared machine, so indicative only. The Apple TV 150+ field median is 1.689 ms, which is 2.4x that figure, against 2.1x using the field Mac median of 0.809 ms. The bench scene is not the field scene, so this only says the order of magnitude agrees. Re-run the bench on a quiet machine when a multiplier matters.

Error bars, in one place

1. The unit of replication is the session, not the row. 60 sessions from 6 installs. The store population is about 13 installs in 60 days (lead audit figure, not re-derived here), so even the whole store population is a small sample. A row-level interval would be 4 to 17 times too narrow per bin.
2. The lab is four devices: the Apple TV 4K (A12), an iPhone16,2 (A17 Pro), an iPad13,8 and the M5 Mac. Only one Apple TV model is modeled, and there is no A12 phone.
3. The Mac-versus-device multiplier depends on the enemy bucket and on which Mac session is the baseline (2.5x to 8.4x for the Apple TV at idle), and a ratio needs 2 sessions and 10 rows on each side. Never use one scalar per device.
4. The GPU bins are blind to fill, overdraw and full-screen passes (the Supernova example). They can say a change is NOT cheap; nothing in them can say it is.
5. Draws are partly a proxy for late-run FX load (webs .636, run_time .623 against gpu_ms), and so are prims. A priced figure may overstate for that reason and understate for the fill reason in 4; the two do not cancel in any known way.
6. The data skew to the team’s Dev play on a few builds: 98.7% Dev rows, and build 324 supplies most of the rows from 200 draws up. Later builds drift; refresh (below) rather than trust the age of this fit.
7. Engine build differences: Mac Dev builds are stock Godot 4.7.1, Apple TV and iPhone are the 4.8-dev fork (about 8% slower sim CPU).
8. A sample is a single frame every ~5 s, not a frame-time distribution. The shares are shares of sampled frames.
9. The estimate only transfers if the scene resembles the ATV scene (human play, builds 303-335) and was viewed at the Apple TV’s 1280x720 logical size. A quiet or synthetic probe scene is not that scene; the probe prints its enemy count and its view so you can compare.
10. The “Mac says fine” trap. The Mac editor’s Metal gpu_ms reads 0.00 (stock Godot 4.7.1 on Metal does not measure GPU time) and the M-series GPU has far more fill rate than the A12. Only an Apple TV measurement can say a change is cheap.
11. An extra share compares bins that different sessions filled. Its interval comes from whole-session replicates and is often wide enough to include zero; the printed words say when it does. The busy bins also carry more Supernova-zone frames, so the point can overstate or understate: weigh the interval and the lower confidence, not the point.

Method

• Population filter (shared by every ATV query, enforced by a test): dev='tvOS/AppleTV11,1' AND build>=303 AND build<9000 AND IFNULL(auto,0)=0 AND IFNULL(frozen,0)=0 AND gpu_ms IS NOT NULL AND draws IS NOT NULL, plus a received-time cutoff (ts <=) so a fit can be reproduced. build<9000 drops 254 synthetic build-9001 rows (4 sessions from 2026-09-23, gpu_ms NULL, flagged human) that would otherwise push the row count to 8,795. In samples the flags are auto and frozen (is_auto belongs to runs), and dev is the device string (tvOS/AppleTV11,1, macOS/Mac17,2), not a developer flag.
• What is measured. Each sample (net/telemetry.gd, _send_sample, every INTERVAL of 5 s) reads one frame’s RENDER_TOTAL_DRAW_CALLS_IN_FRAME and RENDER_TOTAL_PRIMITIVES_IN_FRAME, the same monitors the probe reads, and that frame’s measured GPU time.
• Aggregate queries only. Session and install keys are replaced by anonymous dense ranks inside D1 and never leave it; a bucket-by-session group returns a few hundred rows of counts and sums, and the multiplier histogram returns per-session counts of tick_ms rounded to three decimals. The p90 comes from the bracketing ranks, linearly interpolated. No raw ids are persisted or printed anywhere; the tests scan both JSON files, the SQL, this page and the registry entry for 32- and 16-hex strings.
• Bootstrap. Whole sessions are resampled (2,000 resamples, fixed seed 20261022, Python’s stdlib random); intervals are the 2.5% to 97.5% percentiles. For the bins the statistic is the ratio of sums over the resampled sessions. For the CPU multipliers it is the ratio of medians of per-sample tick_ms, computed from per-session histograms, with device and Mac sessions resampled independently; a side with fewer than 2 sessions or 10 rows has no interval and no ratio.
• Trend replicates. The fit also stores 1,000 whole-session replicates in which every draws bin’s share and every prims bin’s share is recomputed from the same resampled sessions (null when no resampled session has a frame in that bin; the bin centres stay at the fitted values). A shift in share read off two bins’ separate intervals would be far too wide or too narrow, because sessions that visit both bins move them together; these rows let the model put an interval on the shift itself.
• Seed values. The lead audit’s per-bin values (mean 13.4, 13.5, 14.7, 15.8, 18.5, 20.8 ms; share 4%, 12%, 21%, 32%, 62%, 83%) are reproduced by the fit (means within 5%, shares within 5% or one percentage point because the audit rounded them to whole percent) and pinned in the Python test (SEEDS). A refresh that lands further away has hit a different population or a broken query, so investigate before changing the seeds.
• Source of truth: the SQL in tools/device_model/sql/ (atv_gpu_cells, atv_gpu_prims_cells, atv_gpu_quantiles, atv_gpu_corr, atv_gpu_session_fit, atv_zone_flips, atv_zone_confound, mult_cells, mult_medians, mult_hist).

Refresh procedure

python3 tools/device_model/refresh.py                           # live D1 via wrangler (read-only); writes both JSON files
python3 tools/device_model/refresh.py --cutoff-ts 1792633786780   # reproduce the committed fit exactly
python3 tools/device_model/refresh.py --raw-dir DIR             # live run that also records the raw rows (DIR must be outside the repository)
python3 tools/device_model/refresh.py --replay-dir DIR --cutoff-ts 1792633786780   # re-fit offline from such a recording
python3 tools/device_model/refresh.py --sqlite samples.db       # offline dry run on a local samples table
python3 tools/device_model/refresh.py --out-dir /tmp/dm         # write elsewhere, then diff against the committed files

• A live run reads about 2.6 million D1 rows; the free-plan daily limit is 5 million. Run it once with --raw-dir, then iterate with --replay-dir, which costs nothing and refuses a recording whose SQL has since changed (the recording’s manifest pins each file to the hash of its query).
• It needs the wrangler OAuth login on this Mac (skill bh-telemetry: CLOUDFLARE_API_TOKEN is unset for the call because the secrets-file token gets error 7403 on D1). Queries are read-only; the runner refuses anything that is not a single SELECT.
• The two files are written together or not at all. A fit that finds an empty draws or prims bin refuses (exit 1, nothing written) rather than publish a bin of nulls, and a file that fails its validator is never written.
• --raw-dir saves raw wrangler output and must be OUTSIDE the repository (raw rows must never be committed).
• After a refresh: python3 tools/device_model/tests/fid_device_model.test.py and scripts/check-test-count.sh. The tests pin the committed JSON: shape, bucket coverage, a confidence on every bin and cell, the seed values within 5%, no id-shaped strings.
• Then regenerate this page’s tables with python3 tools/device_model/doc_tables.py and paste them in (the Python test fails while a table here differs from the JSON), and re-read the prose that quotes the same numbers: the short version, the reading sections, the slopes, the zone paragraph and the CPU headline. The model-fed example under “From the perf probe” is pinned to the model’s output by a GUT test, so regenerate it too if that test fails.
• Refresh when the fidelity check warns (the committed fit is older than 60 days), after a build that changes the Apple TV render cost, or when a bin’s session count changes the confidence story.

The fidelity check

python3 tools/device_model/fidelity_check.py --out DIR writes DIR/result.json (fidelity-result/1, tool d1-device-model, scope global) and prints the headline. It reads only the committed JSON, never the network or D1, so it cannot see field-data drift. fail means a file is malformed or leaks an id-shaped string, warn means the fit is older than the refresh window, pass otherwise. Low-confidence bins and cells do not make it warn (most are low confidence today); they are counted in its metrics (atv_bins_low_confidence, atv_prims_bins_low_confidence, mult_cells_with_ratio, mult_cells_insufficient, mult_cells_low_confidence) and its caveat. The registry entry is tools/device_sim/registry.d/d1-device-model.json (tier d1, offline). The Python tests are not wired into scripts/gate.sh; run them by hand until the device-sim glob step lands.

Where this fit differs from the lead audit

The lead audit’s numbers were the starting point; these did not reproduce under the stated filter, and the committed values are the reproduced ones:

Lead audit	This fit
n = 8,539	n = 8,541 (live count; the audit was an earlier snapshot)
Supernova zone “+2.44 ms / 658 flips” on the ATV, “+0.02 ms on the M5”	Not reproduced. With the committed definition (adjacent windows 7 s or less apart) a zone appearing is 558 flips at +1.53 ms, both directions pooled 1,116 at +2.63 ms. The Mac cannot show a GPU change at all: its Dev builds report no gpu_ms
Within-session slope 0.93 ms per 100 draws from 25 sessions	0.929 from 23 sessions (a session needs at least 20 rows and some spread in draws)
Apple TV 0-24 enemies about 2.5x (n 6,875 vs 12,286)	1.99x ratio of medians (95% CI 1.95-6.87), 2.48x ratio of means (n 7,119 vs 12,290)
Apple TV 100-199 enemies about 2.6x	The audit’s 100-199 and 150+ buckets overlapped, so the buckets are 0-24, 25-99, 100-149, 150+. 100-149 has one Mac row, so no ratio is published for it
Apple TV 150+ about 2.1x mean and 2.4x worst-tick	Human rows: 2.088 median, 2.947 mean, 2.740 worst-tick median. Rows including autopilot: 2.144 median, 2.356 mean, 1.752 worst-tick median, 2.373 worst-tick mean. The audit’s 2.1x matches the all-rows median (2.144, not a mean) and its 2.4x matches the all-rows worst-tick MEAN (2.373), not the worst-tick median (1.752)
iPhone 150+ about 2.1x mean and 1.8x worst-tick	Human rows: 1.009 median, 1.006 mean, 0.935 worst-tick median (32 rows vs 15). Rows including autopilot: 2.053 median, 2.045 mean, 1.783 worst-tick median, 1.683 worst-tick mean. The audit’s 2.1x matches the all-rows median (2.053) and its 1.8x the all-rows worst-tick median (1.783)
Mac-session ratio range 1.9x to 6.3x at Apple TV idle	2.5x to 8.4x (ratio of means, each Mac session as the baseline, human rows)

Facts corrected alongside

Provenance: the telemetry counts below (the 51 launches, which builds carry gpu_ms, the Mac row count) come from the live D1 review of 2026-10-22 that preceded this fit and were not re-queried for it. The repository facts (the commit, the display_payload code, the comment sites listed further down, the arithmetic behind 2880x1620) were checked in the tree.

• The Apple TV renders 1920x1080, and has since BUILD 211 (commit 4dc6d572, 2026-09-29, fork branch tvos-render-1080p, carried to the 4.8 fork as ffb6b6ffe5; the fork is a separate repository, so that last commit is taken from the review that preceded this fit and was not checked here). It is NOT build 234, which is the late-run perf-fix build. Its multiplier against the 1280x720 design size is x2.25, not x9. Fixed in docs/architecture/rendering-and-performance.md.
• Telemetry does record the output size. events rows of type launch carry win_w/win_h 1920x1080, screen_scale 1, refresh_hz 60 and renderer mobile on all 51 Apple TV launches on builds 303-335 (net/lifecycle_telemetry.gd, display_payload). Only the single BUILD 211 launch was 3840x2160 at scale 2.
• samples.vp_w/vp_h are not render sizes. They equal the window times window over canvas: 2880x1620 on the Apple TV, 11520x6480 on that one 4K launch.
• gpu_ms is populated on every Apple TV row from build 303, and not fleet-wide. iPhone16,2 has it on the 4.8 builds (306, 310-332) and NULL on 300, 301, 307 and 309, and a Mac row has never had it (0 of 32,919 rows). tools/perf_probe/perf_probe.gd now says so: on the Mac editor (stock Godot 4.7.1) Metal gpu_ms still reads 0.00, so use --rendering-driver vulkan; the Apple TV and iPhone fork builds populate samples.gpu_ms in D1.

Stale “Metal can’t measure GPU time” sites (not edited here)

These still claim that Metal has no GPU timing or that the Apple TV resolution is unknown; each sits in a file this change does not own (telemetry has its own deploy hashes and migrations). Correct them with the fact set above:

• telemetry/schema.sql:27 (the gpu_ms column comment)
• net/telemetry.gd:12-13 and :195-196 (header and render_times comments)
• docs/architecture/dev-tools.md:452
• docs/RELEASE-1.0.1.md:342 and :654
• docs/superpowers/specs/2026-09-29-telemetry-v2-design.md:133
• docs/research/2026-09-29-general-performance-review.md:43, :65 and :414
• tests/test_telemetry_reporters_queue.gd:189
• net/lifecycle_telemetry.gd, display_payload: the comment calls vp_* “the root viewport’s render size” (see above for what it is)

The look_bench, chalice_perf and web_lite_bench headers say the Mac cannot measure GPU time under Metal; on the Mac that is still true, so they were left alone.

What this does not tell you

Nothing here measures fill cost directly. The missing instrument is a real GPU capture of the Apple TV (per-pass GPU time, overdraw), which needs a physical device: that is Issue #84. Until then a change that is fill-bound (a big translucent layer, a full-screen pass, a wide additive glow) can only be judged by running it on the Apple TV; the model’s job is to stop a draws-and-prims change from shipping unpriced, and to stop “the Mac says fine” from standing in for a device.