Autopilot vs human skill gap (2026-10-13)

Autopilot vs human skill gap

Question (Chris): why can’t the autopilot get as good as Toby, even with the learning autopilot?

Short answer: the learner only tunes multipliers on the hand-written SkilledAutoPlayer, and the dodge/positioning logic, which is where the bot loses, is not evolvable. Telemetry shows the gap is survival in the first 1-3 minutes, not damage, and not shop upgrades.

Telemetry (runs table, Apple TV, BUILD >= 250, survival, solo, all with 0 shop spend)

	Runs (>= 30 s)	Mean survival	>= 10 min	Longest
Human	19	392 s	4	1581 s
Autopilot (attract + personas)	209	104 s	3	805 s

Same run-length buckets:

Bucket	Human n / lvl / kills / dps	Bot n / lvl / kills / dps
30-90 s	4 / 4.0 / 192 / 30	116 / 4.9 / 237 / 41
90-180 s	5 / 7.6 / 345 / 31	80 / 8.4 / 410 / 41
180-400 s	3 / 17.7 / 707 / 56	5 / 15.6 / 611 / 37
400 s+	6 / 49.2 / 1946 / 628	3 / 27 / 829 / 120

• For the first ~3 min the bot out-damages humans, yet ~80% of its runs die there to plain swarmer / charger / shooter-shot waves. The early gap is dodging and positioning.
• Past ~7 min humans reach ~5x the bot’s dps, so build scaling matters too, but n is small (6 vs 3).
• Meta-progression does not explain it: every human run here had meta_spent = 0, meta_levels = ''.

Caveats: telemetry can’t tell Toby from Chris (shared ATV, install ids change per reinstall), and these are Dev builds, so dev powers may inflate the human runs.

Why the learner can’t close it

1. The genome is build weights plus multipliers on SkilledAutoPlayer consts. Escape rings, reaction time and warp landing search are not genes, so evolution can’t change how it dodges.
2. The planner looks ~0.5 s ahead and steers locally; humans survive on whole-run positioning (keeping an exit open, kiting loops, not getting boxed in).
3. Score noise: survival sd is 50-70% of the mean per genome, so 12 samples only detect large gains.
4. Fitness covers a fresh save’s first ~1-3 min only (no late game).

Humanize is presentation, not a handicap (Chris, 2026-10-13): it exists so the ship faces the way it warps and doesn’t make odd-looking moves. The skill handicaps are SkillProfile’s decide_every / aim_jitter_rad / dash_skill. Humanize’s effect on survival has never been measured; some knobs (idle stick-release pauses HZ_PAUSE_CHANCE, calm push 0.82-1.0, held calm commits) could cost survival as a side effect.

Plan (in order)

1. Identify Toby’s runs (ask) and record his inputs with the input-replay harness. Diff his movement against the bot’s in the 1-3 min window where the bot dies: positioning vs build.
2. Humanize A/B: the same genome, humanize on vs off, on the benchmark’s fixed seeds. If it costs survival, make the look-preserving parts cheaper (e.g. pauses/wander only when nothing is near). Don’t evolve it away; it stays a presentation constraint.
3. Fixed benchmark seeds for scoring (open call #3 in memory bullet-heaven-learning-autopilot), so small tweaks are measurable.
4. Split SkilledAutoPlayer consts into three groups:
   • Strategy: ESCAPE_RINGS, ESCAPE_MISALIGN_PX, STEP_LOOK_S, WARP_CLEAR_RADIUS, ESCAPE_MAX_S, LATE_REACT_S, boss/elite/missile/pull weights, orbit/kite shape. These become evolvable.
   • Skill handicaps: SkillProfile knobs and REACTION_S. Fixed per skill level.
   • Presentation: humanize HZ_*, MAX_TURN_PER_TICK, ENGAGE_ALIGN_DOT. Not evolved.
5. Gene bounds from Toby’s recordings, so the bot can’t learn a play style no human could fly.
6. Optimise movement separately from builds: one shared movement genome (CMA-ES on the fixed seeds, e.g. the Python cma package; confirm before committing). MAP-Elites keeps evolving builds per role cell, and all cells fly the best movement genome. Dodging isn’t role-specific, and splitting it across 12 cells would make each cell relearn it from a quarter of the samples.
7. Strategy consts become Remote Console tunables too (project rule), so a value can be tried by hand on device and pushed by the learner through the same mechanism.

Ceiling caveat: steps 4-7 only tune existing heuristics. If step 1 shows Toby doing something the bot has no code for (e.g. herding the swarm), the fix is new behaviour, not tuning. Step 1 decides.