Telemetry gaps and options (2026-09-28)

Telemetry gaps and options (2026-09-28)

Question (Chris): do we need to ramp telemetry up, is there a better way to do any of it, and what more could we collect to work out what is going on?

Answer: capture is already always-on with no sampling to raise. The problem is loss and blind spots, not volume. About half of sessions never produce an outcome, nothing survives offline or a crash, and the questions we actually got stuck on (tester crashes, Halo Chain 0 XP, the “invincible” run) needed data we never send.

What exists today (verified 2026-09-28)

Reporter	Sends	Loss behaviour
net/telemetry.gd → /sample	perf row every 5s	single HTTPRequest; dropped if previous still in flight
net/hitch_tracker.gd → /hitch	frames ≥60ms, batched 3s	MAX_PER_SESSION := 40, keeps the FIRST 40
net/gameplay_telemetry.gd → /gameplay	one run summary	only from _bank_run() (main.gd game over / story complete / tutorial hand-off)
same → /feedback	pause-menu note	fire-and-forget
net/dash_encounter_telemetry.gd → /encounter	per charger/spider dash	own per-boot session id, can’t join to runs

No disk queue, no retry, anywhere. Crash marker (net/remote_config_state.gd) only arms for a non-default remote-config id, so it is effectively off (355/359 runs on default).

Live D1 (2026-09-28): 13.4 MB; samples 71,946, runs 359, encounters 1,311, hitches 447, feedback 2. ~3k rows written/day, far below the D1 free cap (100k/day).

Gaps, ranked by how much they limit diagnosis

1. No outcome for ~half of sessions. 62/129 sessions in 14 days had samples but no runs row (40 of them ≥60s). Every TestFlight-only iPhone model sent samples and never a run row. Quit to menu, backgrounding, an iOS kill and a crash all look identical (nothing).
2. No disk queue. Offline or killed = data gone, including the single run summary.
3. No crash reporting. The iPad “Crashed or glitched” feedback and the Apple TV HD launch crash (found by App Review) were invisible to us.
4. No launch/funnel events or install id. Can’t count players, retention, tutorial drop-off. install_id exists (net/remote_config.gd) but only goes to /config, and the Worker ignores it.
5. No mid-run timeline. Level/XP by source, pick timestamps, per-weapon damage, boss time-to-kill. Would have caught Halo Chain 0 XP and explained the “invincible” run without code archaeology.
6. Hitch cap keeps the first 40: 14,192 hitch frames counted in samples, 447 rows stored (3%).
7. Mislabelled perf metrics: proc_ms and render_ms both read Performance.TIME_PROCESS (whole-frame time). The dashboard’s “high proc = our code” reading doesn’t hold; no GPU time.
8. Human vs attract-mode perf mixed (no is_auto on samples); silent truncation of build_picks (400 chars) and weapons_list (120).

External options

• No third-party crash or analytics SDK supports tvOS from Godot. sentry-godot: iOS/web/desktop, not tvOS (sentry-cocoa could be linked natively into the tvOS Xcode shell: native crashes only, no GDScript context). GameAnalytics Godot SDK: no tvOS. Crashlytics: tvOS community-only, no Godot integration. Backtrace: Godot 3 only.
• MetricKit is not on tvOS (iOS/iPadOS/macOS/visionOS only). iOS-only via a ~100-line plugin.
• Free now: Xcode Organizer crash logs (TestFlight testers share automatically; App Store only from opted-in users). Needs dSYMs from the fork-built archive to symbolicate the engine.
• Cloudflare: D1 is fine at current volume. If perf samples ever pressure the write cap, Workers Analytics Engine suits them; keep runs/feedback/hitches in D1.
• What roguelite devs track (Slay the Spire GDC 2019): pick rate when offered, presence in winning/long runs, time/cause of death, first-session funnel, D1/D7 return. Heatmaps need hundreds of samples per bin, too many for our tester base.

Sources: docs.sentry.io/platforms/godot, github.com/getsentry/sentry-godot, docs.gameanalytics.com (Godot SDK), developer.apple.com (MetricKit, app-privacy-details), developers.cloudflare.com/d1/platform/limits, gdcvault.com/play/1025731.

Privacy

marketing/appstore/compliance.md declares Performance Data + Product Interaction, not linked, “no identifiers”, “optional analytics”. Code has no opt-out; free-text feedback isn’t declared. A persistent install id → Device ID; crash capture → Diagnostics/Crash Data. UK: first-party analytics exemption (Data (Use and Access) Act, from Feb 2026) still expects clear info and an easy opt-out, so add a “Share diagnostics” toggle alongside any expansion.

Recommended order

1. Run lifecycle: send a run_end for every exit (quit, background, kill-on-resume) with a reason, and an always-armed crash marker reported on next boot with last state/fx.
2. On-disk event queue (JSONL in user://, event ids for Worker dedupe, flush on boot/background/N). Best-effort on tvOS (user:// is Caches).
3. Install id + launch/funnel events (launch, menu, tutorial step, shop, run start).
4. Mid-run timeline in the run row: level-ups with time, picks offered vs taken, XP by source, per-weapon damage, boss TTK.
5. Hitch: keep worst-N plus a reservoir sample instead of first-40. Fix proc_ms/render_ms.
6. is_auto on samples; one session id across all reporters.
7. Privacy label + opt-out toggle updated in the same release.
8. Later/optional: dSYMs for Organizer now; Sentry on iOS/web; MetricKit iOS plugin.