Skip to content
Dark Cosmos — Dev Docs

QA build-out (v0.1 RC Phase 1) — implementation plan

QA Build-out Implementation Plan

Section titled “QA Build-out Implementation Plan”

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking. Toby may be pushing to main concurrently — follow the bh-sdd-concurrent defenses (pull/rebase + re-verify quoted code before dispatching each task).

Goal: Build the automated issue-finding machinery for the v0.1 RC: a headless autopilot soak harness with damage/economy telemetry, a 2-pilot determinism baseline with drone state in the checksum, an input-replay recorder/replayer, and a windowed UI screenshot regression sweep.

Architecture: All sim-side hooks are read-only, optional, and injected (a null-by-default stats_sink object field + a transient damage_context string on Sim) so /sim purity and determinism are untouched. The soak core (SoakRun) is a pure RefCounted unit GUT can test; the CLI runner is a thin SceneTree shell. The UI sweep reuses the marketing/capture/shoot.gd SubViewport pattern (windowed — MultiMesh colours are unreadable headless).

Tech Stack: Godot 4.6.3 typed GDScript, GUT 9.6.0, ImageMagick (magick compare), zsh wrappers in scripts/.

Spec: docs/superpowers/specs/2026-07-14-v0.1-release-candidate-process-design.md (Phase 1).

Global Constraints

Section titled “Global Constraints”
  • /sim files stay pure: extends RefCounted, NO Node/render/Input/Engine/Time APIs. An injected plain-object callback field is allowed; file/JSON I/O is not.
  • Determinism is the keystone: no new rng/upgrade_rng draws, no tick-order changes. The pinned baseline lives in tests/test_determinism_survival.gd (read the literal assertions, currently snapshot_string().hash()==4217109746, state_checksum()==3312602286). Task 4 deliberately re-pins via the bh-determinism-repin skill; no other task may move it.
  • Every task ends with the bh-dev-chunk ritual: full suite (godot --headless --path . -s res://addons/gut/gut_cmdln.gd -gdir=res://tests -ginclude_subdirs -gexit), scripts/check-test-count.sh, boot smoke (godot --headless --path . --quit-after 60 + grep stderr for SCRIPT ERROR), commit. Judge GUT by its printed Run Summary, NOT the exit code (known exit-134 teardown SIGABRT).
  • Narrow a test run with -gselect=<script_name_without_.gd> (-gtest/-gdir are silently overridden by .gutconfig.json).
  • tools/ and scripts/ additions are dev-only: never referenced from game code, tests, or exports (same convention as tools/glow_bake/, marketing/capture/).
  • New docs need title: frontmatter (docs-site hard requirement).

Task 1: Determinism-safe stats sink + damage attribution on Sim

Section titled “Task 1: Determinism-safe stats sink + damage attribution on Sim”

Files:

  • Modify: sim/sim.gd (new fields near the top var block; context tags around the weapon-update loop at ~line 791 and the drone loop at ~line 744; kill hook in _sweep_dead)
  • Modify: sim/elemental_system.gd:135 (damage_enemy reports to the sink)
  • Create: tools/soak/soak_stats.gd
  • Test: tests/test_soak_stats.gd

Interfaces:

  • Produces: Sim.stats_sink: Object (null default), Sim.damage_context: String (“unattributed” default). Sink duck-type: on_damage(context: String, amount: float) -> void, on_kill(enemy_type: int) -> void. SoakStats (class_name, RefCounted) implementing both with var damage_by_context: Dictionary = {} and var kills_by_type: Dictionary = {}.

  • Consumed by: Task 3’s SoakRun.

  • Step 1: Write the failing test — tests/test_soak_stats.gd:

extends GutTest


# The optional stats sink: read-only telemetry hooks for the soak harness.
# MUST be a pure observer — attaching it cannot change the sim trace.


func _run(seed_v: int, ticks: int, sink) -> Sim:
  var sim := Sim.new(seed_v, SimContentFixture.db())
  sim.enable_survival()
  sim.stats_sink = sink
  for i in range(ticks):
    var dir := Vector2(cos(float(i) * 0.05), sin(float(i) * 0.03)).normalized()
    sim.tick_single(InputState.new(dir))
  return sim


func test_sink_records_attributed_weapon_damage() -> void:
  var stats := SoakStats.new()
  _run(1234, 600, stats)
  var total := 0.0
  for ctx in stats.damage_by_context:
    total += stats.damage_by_context[ctx]
  assert_gt(total, 0.0, "600 ticks of survival deals SOME damage")
  assert_true(stats.damage_by_context.has("blaster"),
    "the starter Blaster's damage is attributed to its weapon id, not 'unattributed'")


func test_sink_records_kills_by_type() -> void:
  var stats := SoakStats.new()
  _run(1234, 600, stats)
  var kills := 0
  for t in stats.kills_by_type:
    kills += stats.kills_by_type[t]
  assert_gt(kills, 0, "600 ticks of survival kills something")


func test_attached_sink_does_not_change_the_trace() -> void:
  var with_sink := _run(1234, 600, SoakStats.new())
  var without := _run(1234, 600, null)
  assert_eq(with_sink.state_checksum(), without.state_checksum(),
    "sink attach/detach is trace-invisible (determinism-safe)")
  assert_eq(with_sink.snapshot_string(), without.snapshot_string())

  • Step 2: Run it, verify it fails — godot --headless --path . -s res://addons/gut/gut_cmdln.gd -gselect=test_soak_stats -gexit → expect parse failure (“SoakStats not declared”) / all three failing.

  • Step 3: Create tools/soak/soak_stats.gd:

class_name SoakStats
extends RefCounted


# Telemetry accumulator for the soak harness. Injected as Sim.stats_sink —
# a pure observer; the sim never reads anything back from it.


var damage_by_context: Dictionary = {}   # context id (weapon id / "drone" / "burn" / "unattributed") -> total damage
var kills_by_type: Dictionary = {}       # enemy type int -> kill count


func on_damage(context: String, amount: float) -> void:
  damage_by_context[context] = damage_by_context.get(context, 0.0) + amount


func on_kill(enemy_type: int) -> void:
  kills_by_type[enemy_type] = kills_by_type.get(enemy_type, 0) + 1
  • Step 4: Add the Sim fields — in sim/sim.gd, next to the other non-state runtime fields (near var game_over: bool ~line 322), add:
# --- QA telemetry (soak harness) ---------------------------------------------
# Optional read-only stats sink (duck-typed: on_damage(context, amount),
# on_kill(enemy_type)). Null in normal play. NEVER part of state_checksum/
# snapshot_string; attaching it MUST NOT change the trace.
var stats_sink: Object = null
# Transient label for "who is dealing damage right now" — set/reset around the
# weapon/drone update loops each tick; read only by damage_enemy's sink report.
var damage_context: String = "unattributed"
  • Step 5: Tag the contexts. In the per-pilot weapon-firing loop (~line 791), wrap the update call:
      damage_context = wid
      pilot.arsenal.weapon_by_id[wid].update(self, pilot, dt)
      damage_context = "unattributed"

In the drone update dispatch (the _pi_dr loop ~line 744), set damage_context = "drone" before the drone director call and reset after, same pattern. Grep sim.gd for where drone_director is invoked inside tick() to find the exact call. If burn/DoT damage is applied inside sim/elemental_system.gd (grep burn there), set sim.damage_context = "burn" / reset around that block too — anything untagged safely reports as “unattributed”, so missing a site is a telemetry gap, not a bug.

  • Step 6: Report from the two hooks. In sim/elemental_system.gd’s damage_enemy (line 135), first line of the body:
  if sim.stats_sink:
    sim.stats_sink.on_damage(sim.damage_context, amount)

In sim/sim.gd’s _sweep_dead(), at the point an enemy with hp <= 0 is confirmed dead (before remove_at), add:

      if stats_sink:
        stats_sink.on_kill(enemies.data[i])

(Verify enemies.data[i] is the type id by reading sim/entity_pool.gd/enemy_pool.gd first; if type lives in a different array, use that one.)

  • Step 7: Run the new test → PASS, then the FULL suite + determinism (-gselect=test_determinism_survival and test_determinism_checksum must hold the pinned values — this task must NOT move them), boot smoke, scripts/check-test-count.sh.

  • Step 8: Commit — git commit -m "feat(sim): optional determinism-safe stats sink + damage attribution context".

Task 2: AutoPlayer per-pilot control + dash/decoy edges

Section titled “Task 2: AutoPlayer per-pilot control + dash/decoy edges”

Files:

  • Modify: ai/auto_player.gd
  • Test: tests/test_auto_player.gd (extend)

Interfaces:

  • Produces: compute_move_for(sim: Sim, pilot: PlayerState) -> Vector2 (existing compute_move(sim) becomes a wrapper for sim.player); wants_dash(sim: Sim, pilot: PlayerState) -> bool; wants_decoy(sim: Sim, pilot: PlayerState) -> bool. All pure functions of sim state — NO randomness (soak input must be deterministic per seed).

  • Consumed by: Task 3’s SoakRun (and existing attract mode keeps working unchanged via the wrapper).

  • Step 1: Write failing tests (append to tests/test_auto_player.gd):

func test_compute_move_for_reads_the_given_pilot() -> void:
  var sim := Sim.new(1, _content())
  sim.add_player2()
  sim.player.pos = Vector2(-400, 0)
  sim.pilots[1].pos = Vector2.ZERO
  sim.enemies.add(Vector2(100, 0), Vector2.ZERO, 14.0, 5.0)  # threat right of P2 only
  sim.hash.rebuild(sim.enemies)
  var move := AutoPlayer.new().compute_move_for(sim, sim.pilots[1])
  assert_lt(move.x, 0.0, "P2 flees ITS OWN nearby enemy, not P1's surroundings")


func test_wants_dash_under_close_threat() -> void:
  var sim := Sim.new(1, _content())
  sim.player.pos = Vector2.ZERO
  sim.enemies.add(Vector2(40, 0), Vector2.ZERO, 14.0, 5.0)   # inside DASH_PANIC_RADIUS
  sim.hash.rebuild(sim.enemies)
  assert_true(AutoPlayer.new().wants_dash(sim, sim.player), "dashes when an enemy is on top of it")


func test_no_dash_when_clear() -> void:
  var sim := Sim.new(1, _content())
  sim.player.pos = Vector2.ZERO
  assert_false(AutoPlayer.new().wants_dash(sim, sim.player), "no threats → no dash")


func test_wants_decoy_only_when_charged_and_threatened() -> void:
  var sim := Sim.new(1, _content())
  sim.player.pos = Vector2.ZERO
  sim.enemies.add(Vector2(100, 0), Vector2.ZERO, 14.0, 5.0)
  sim.hash.rebuild(sim.enemies)
  sim.player.drone_charge = 1.0
  assert_true(AutoPlayer.new().wants_decoy(sim, sim.player))
  sim.player.drone_charge = 0.2
  assert_false(AutoPlayer.new().wants_decoy(sim, sim.player), "not charged → hold")

  • Step 2: Run -gselect=test_auto_player → new tests FAIL (methods missing).

  • Step 3: Implement. In ai/auto_player.gd: rename the body of compute_move to compute_move_for(sim: Sim, pilot: PlayerState) replacing var p := sim.player.pos with var p := pilot.pos; re-add the wrapper. Add:

const DASH_PANIC_RADIUS: float = 70.0    # an enemy this close → dash away
const DECOY_THREAT_RADIUS: float = 220.0 # deploy the decoy when pressed and charged


func compute_move(sim: Sim) -> Vector2:
  return compute_move_for(sim, sim.player)


func wants_dash(sim: Sim, pilot: PlayerState) -> bool:
  var near := sim.hash.query_circle(pilot.pos, DASH_PANIC_RADIUS, sim.enemies)
  return not near.is_empty()


func wants_decoy(sim: Sim, pilot: PlayerState) -> bool:
  if pilot.drone_charge < 1.0:
    return false
  var near := sim.hash.query_circle(pilot.pos, DECOY_THREAT_RADIUS, sim.enemies)
  return not near.is_empty()

(Check PlayerState’s actual charge field name — it moved onto PlayerState in the 2026-07-11 P2 drone work; grep drone_charge in sim/player_state.gd and use the real name.)

  • Step 4: -gselect=test_auto_player → ALL PASS (including the pre-existing tests — the wrapper keeps them green). Full suite + determinism pins hold (AutoPlayer is render-side control, no sim change). Boot smoke (attract mode uses compute_move — must still run).

  • Step 5: Commit — git commit -m "feat(ai): AutoPlayer drives any pilot + deterministic dash/decoy decisions".

Task 3: Soak core (SoakRun) + CLI runner + shell wrapper

Section titled “Task 3: Soak core (SoakRun) + CLI runner + shell wrapper”

Files:

  • Create: tools/soak/soak_run.gd (class_name SoakRun — pure, GUT-testable)
  • Create: tools/soak/soak_runner.gd (extends SceneTree — CLI sweep shell)
  • Create: scripts/soak.sh
  • Test: tests/test_soak_run.gd

Interfaces:

  • Consumes: SoakStats, Sim.stats_sink (Task 1); AutoPlayer.compute_move_for/wants_dash/wants_decoy (Task 2); ContentLoader.load_from_path("res://data/bible.json"); ShipBonuses.apply_base/apply_to/apply_base_stats_to/apply_stat_bonus_to; sim.upgrade_system.roll_upgrade_choices(sim, pilot, 3) / apply_upgrade(sim, pilot, id); sim.pending_dimension_choice + sim.enter_area(id); sim.game_over.

  • Produces: SoakRun.new(cfg: Dictionary).execute() -> Dictionary where cfg = {seed: int, ship: String, coop: bool, max_ticks: int, content: ContentDB} and the report = {seed, ship, coop, ticks_run, game_over, invariant_failures: Array[String], damage_by_context: Dictionary, kills_by_type: Dictionary, level_reached: int, dimensions_visited: Array}.

  • Step 1: Write the failing test — tests/test_soak_run.gd:

extends GutTest


# The soak core: one configured autopilot run, headless, pure. The CLI runner
# (tools/soak/soak_runner.gd) is a thin shell around this and isn't GUT-tested.


func _cfg(overrides: Dictionary = {}) -> Dictionary:
  var cfg := {"seed": 1234, "ship": "cobalt", "coop": false, "max_ticks": 1200,
    "content": SimContentFixture.db()}
  cfg.merge(overrides, true)
  return cfg


func test_run_completes_and_reports() -> void:
  var report := SoakRun.new(_cfg()).execute()
  assert_eq(report.ticks_run, 1200, "runs to max_ticks (or a legitimate game_over)")
  assert_eq(report.invariant_failures.size(), 0,
    "clean short run: %s" % str(report.invariant_failures))
  assert_true(report.damage_by_context.size() > 0, "telemetry captured")


func test_run_is_deterministic_per_seed() -> void:
  var a := SoakRun.new(_cfg()).execute()
  var b := SoakRun.new(_cfg()).execute()
  assert_eq(a.final_checksum, b.final_checksum, "same cfg → identical run")


func test_coop_run_ticks_two_pilots() -> void:
  var report := SoakRun.new(_cfg({"coop": true})).execute()
  assert_eq(report.invariant_failures.size(), 0, str(report.invariant_failures))
  assert_eq(report.pilots, 2)


func test_nan_position_is_caught_by_invariants() -> void:
  var run := SoakRun.new(_cfg({"max_ticks": 10}))
  run.execute()
  run.sim.player.pos = Vector2(NAN, 0.0)
  run.check_invariants()
  assert_gt(run.invariant_failures.size(), 0, "a NaN position is flagged")

  • Step 2: -gselect=test_soak_run → FAIL (SoakRun not declared).

  • Step 3: Implement tools/soak/soak_run.gd:

class_name SoakRun
extends RefCounted


# One configured, fully-headless autopilot run of the pure sim. Used by
# tools/soak/soak_runner.gd (CLI sweep) and tests/test_soak_run.gd.
# Everything here must stay deterministic per cfg — AutoPlayer decisions are
# pure functions of sim state, and upgrade picks go through upgrade_rng's
# normal roll path.


const INVARIANT_EVERY: int = 60          # check cadence (ticks)
const SOFTLOCK_WINDOW_TICKS: int = 10800 # 3 min of game-time with zero kills → flag


var sim: Sim
var stats: SoakStats
var auto: AutoPlayer
var cfg: Dictionary
var invariant_failures: Array[String] = []
var _last_kills: int = 0
var _last_kill_tick: int = 0


func _init(config: Dictionary) -> void:
  cfg = config


func execute() -> Dictionary:
  sim = Sim.new(int(cfg.seed), cfg.content)
  sim.enable_survival()
  stats = SoakStats.new()
  sim.stats_sink = stats
  auto = AutoPlayer.new()
  var meta := MetaState.new()
  ShipBonuses.apply_base(String(cfg.ship), sim, meta)
  ShipBonuses.apply_to(String(cfg.ship), sim.player, sim.mods)
  if bool(cfg.coop):
    sim.add_player2()
    ShipBonuses.apply_base_stats_to(String(cfg.ship), sim.pilots[1])
    ShipBonuses.apply_stat_bonus_to(String(cfg.ship), sim.pilots[1])
  var dimensions_visited: Array = []
  var t := 0
  var max_ticks := int(cfg.max_ticks)
  while t < max_ticks and not sim.game_over:
    sim.tick_single(_input_for(sim.player), _input_for(sim.pilots[1]) if sim.pilots.size() > 1 else null)
    _consume_levelups()
    if not sim.pending_dimension_choice.is_empty():
      var pick := auto.pick_dimension(sim.pending_dimension_choice)
      dimensions_visited.append(pick)
      sim.enter_area(pick)
    if t % INVARIANT_EVERY == 0:
      check_invariants()
    t += 1
  return {
    "seed": cfg.seed, "ship": cfg.ship, "coop": cfg.coop,
    "pilots": sim.pilots.size(), "ticks_run": t, "game_over": sim.game_over,
    "invariant_failures": invariant_failures,
    "damage_by_context": stats.damage_by_context,
    "kills_by_type": stats.kills_by_type,
    "level_reached": sim.player.level,
    "dimensions_visited": dimensions_visited,
    "final_checksum": sim.state_checksum(),
  }


func _input_for(pilot: PlayerState) -> InputState:
  if pilot == null or pilot.hp <= 0.0:
    return InputState.new()
  var move := auto.compute_move_for(sim, pilot)
  # Aim at the nearest enemy so the manual right-stick weapon is exercised.
  var aim := Vector2.ZERO
  var near := sim.hash.query_circle(pilot.pos, 400.0, sim.enemies)
  if not near.is_empty():
    aim = (sim.enemies.pos[near[0]] - pilot.pos).normalized()
  return InputState.new(move, aim, auto.wants_decoy(sim, pilot), auto.wants_dash(sim, pilot))


func _consume_levelups() -> void:
  for pilot in sim.pilots:
    var guard := 0
    while pilot.pending_levelups > 0 and guard < 10:
      var ids := sim.upgrade_system.roll_upgrade_choices(sim, pilot, 3)
      if ids.is_empty():
        invariant_failures.append("tick %d: level-up pending but ZERO upgrade choices offered" % _tick())
        break
      sim.upgrade_system.apply_upgrade(sim, pilot, auto.pick_upgrade(ids))
      guard += 1


func check_invariants() -> void:
  for pilot in sim.pilots:
    if not (is_finite(pilot.pos.x) and is_finite(pilot.pos.y)):
      invariant_failures.append("tick %d: pilot pos not finite: %s" % [_tick(), pilot.pos])
    if pilot.hp > pilot.max_hp + 0.01 or is_nan(pilot.hp):
      invariant_failures.append("tick %d: pilot hp out of bounds: %f/%f" % [_tick(), pilot.hp, pilot.max_hp])
  for i in range(sim.enemies.count):
    if not (is_finite(sim.enemies.pos[i].x) and is_finite(sim.enemies.pos[i].y)):
      invariant_failures.append("tick %d: enemy %d pos not finite" % [_tick(), i])
      break
  if sim.kills > _last_kills:
    _last_kills = sim.kills
    _last_kill_tick = _tick()
  elif _tick() - _last_kill_tick > SOFTLOCK_WINDOW_TICKS and not sim.game_over:
    invariant_failures.append("tick %d: SOFTLOCK? zero kills for 3+ min of game time" % _tick())
    _last_kill_tick = _tick()   # don't spam the same stall every check


func _tick() -> int:
  return int(round(sim.run_time / Sim_Const.DT))

Verification notes for the implementer (read the real code, adjust exactly): MetaState.new() must be I/O-free (it’s in /sim, so it should be — confirm by reading sim/meta_state.gd); tick_single’s second-param null handling (read its signature in sim/sim.gd); is_finite/is_nan are @GlobalScope in 4.6; sim.kills field name (used in snapshot_string); if roll_upgrade_choices returns Array[String] typing quirks arise, match its declared return type.

  • Step 4: -gselect=test_soak_run → PASS. Fix real-API mismatches by reading the code, not by weakening tests.

  • Step 5: Implement tools/soak/soak_runner.gd (thin shell — not GUT-tested):

extends SceneTree


# Headless soak sweep CLI. DEV ONLY — never referenced by game code or tests.
#   godot --headless --path . -s res://tools/soak/soak_runner.gd -- \
#     --runs=8 --ticks=36000 --ships=cobalt,obsidian --coop=both --seed0=1000
# Writes reports/soak-<unix>.json + a human summary to stdout. Exit 0 = clean,
# 1 = invariant failures or anomalies found (scripts/soak.sh relies on this).


func _init() -> void:
  var args := _parse(OS.get_cmdline_user_args())
  var content := ContentLoader.load_from_path("res://data/bible.json")
  if content == null:
    push_error("soak: failed to load bible.json"); quit(2); return
  var ships: Array = String(args.get("ships", "cobalt")).split(",")
  var coop_modes: Array = [false, true] if args.get("coop", "solo") == "both" \
    else [args.get("coop", "solo") == "coop"]
  var runs := int(args.get("runs", "4"))
  var ticks := int(args.get("ticks", "36000"))   # 10 min game-time default
  var seed0 := int(args.get("seed0", "1000"))
  var reports: Array = []
  var failures := 0
  for ship in ships:
    for coop in coop_modes:
      for r in range(runs):
        var rep := SoakRun.new({"seed": seed0 + r, "ship": ship, "coop": coop,
          "max_ticks": ticks, "content": content}).execute()
        reports.append(rep)
        failures += rep.invariant_failures.size()
        print("[soak] ship=%s coop=%s seed=%d ticks=%d lvl=%d fails=%d" % [
          ship, coop, rep.seed, rep.ticks_run, rep.level_reached,
          rep.invariant_failures.size()])
  var anomalies := _find_anomalies(reports)
  DirAccess.make_dir_recursive_absolute("res://reports")
  var path := "res://reports/soak-%d.json" % Time.get_unix_time_from_system()
  var f := FileAccess.open(path, FileAccess.WRITE)
  f.store_string(JSON.stringify({"reports": reports, "anomalies": anomalies}, "  "))
  f.close()
  print("\n[soak] %d runs, %d invariant failures, %d anomalies -> %s" % [
    reports.size(), failures, anomalies.size(), path])
  for a in anomalies:
    print("[soak] ANOMALY: %s" % a)
  quit(1 if (failures > 0 or anomalies.size() > 0) else 0)


# Cross-run analysis: a damage context that stayed at ~zero across EVERY run,
# or an enemy type that spawned but was never killed anywhere, is exactly the
# "silently inert" bug class (Disruptor slow / Halo Chain buff) made visible.
func _find_anomalies(reports: Array) -> Array:
  var out: Array = []
  var totals: Dictionary = {}
  for rep in reports:
    for ctx in rep.damage_by_context:
      totals[ctx] = totals.get(ctx, 0.0) + rep.damage_by_context[ctx]
  for ctx in totals:
    if totals[ctx] <= 0.0:
      out.append("damage context '%s' dealt ZERO damage across all %d runs" % [ctx, reports.size()])
  for rep in reports:
    if not rep.game_over and rep.ticks_run < int(rep.get("max_ticks", rep.ticks_run)):
      out.append("seed %d ended early without game_over" % rep.seed)
  return out


func _parse(argv: PackedStringArray) -> Dictionary:
  var out := {}
  for a in argv:
    if a.begins_with("--") and "=" in a:
      var kv := a.substr(2).split("=", true, 1)
      out[kv[0]] = kv[1]
  return out

(reports/ must be gitignored — add a reports/ line to .gitignore in this step.)

  • Step 6: Create scripts/soak.sh (executable, chmod +x):
#!/bin/zsh
# Soak sweep wrapper: runs the headless soak, fails loud on SCRIPT ERRORs in
# stderr (engine-level breakage the runner itself can't see) or a nonzero
# runner exit (invariant failures / anomalies). Do NOT use set -e around the
# godot call — godot's teardown SIGABRT (exit 134 upstream bug) would mask
# results; judge from the captured output + the runner's own printed exit line.
cd "$(dirname "$0")/.."
err=$(mktemp)
godot --headless --path . -s res://tools/soak/soak_runner.gd -- "$@" 2>"$err"
code=$?
script_errors=$(grep -c "SCRIPT ERROR" "$err" || true)
if [ "$script_errors" -gt 0 ]; then
  echo "soak.sh: $script_errors SCRIPT ERROR(s) in stderr:" >&2
  grep -A2 "SCRIPT ERROR" "$err" | head -40 >&2
  rm -f "$err"; exit 1
fi
rm -f "$err"
# 134 = known clean-teardown SIGABRT; treat as the runner's own verdict lost — rerun cheaply.
if [ "$code" -eq 134 ]; then echo "soak.sh: exit 134 teardown crash (known); check printed summary above"; exit 0; fi
exit "$code"

  • Step 7: Smoke the CLI end-to-end: scripts/soak.sh --runs=2 --ticks=3600 --ships=cobalt --coop=both → expect 4 run lines, a report file under reports/, exit 0. Then full suite + determinism pins hold + check-test-count.sh + boot smoke.

  • Step 8: Commit — git commit -m "feat(tools): headless autopilot soak harness with invariants + telemetry".

Task 4: Drone state in checksum + 2-pilot determinism baseline — ❌ SUPERSEDED, DO NOT EXECUTE

Section titled “Task 4: Drone state in checksum + 2-pilot determinism baseline — ❌ SUPERSEDED, DO NOT EXECUTE”

Superseded 2026-07-14 mid-execution: a concurrent session committed an approved design for this exact gap (docs/superpowers/specs/2026-07-14-coop-determinism-coverage-design.md, commit bf69800) with a strictly better approach — per-drone snapshot segments instead of an unconditional dr=%d count, so the existing single-pilot baseline stays byte-identical and no re-pin happens at all (this plan’s Global Constraints note about a deliberate re-pin is moot). That spec owns this work. The task text below is retained for history only.

Files:

  • Modify: sim/sim.gd:2564-2607 (state_checksum, snapshot_string)
  • Create: tests/test_determinism_coop.gd
  • Modify (only if the pins move): tests/test_determinism_survival.gd, tests/test_determinism_checksum.gd via the bh-determinism-repin skill

Interfaces:

  • Produces: checksum/snapshot now cover sim.drones; a pinned 2-pilot baseline in tests/test_determinism_coop.gd that future changes must hold (same role as the single-pilot pins).

  • Step 1: Write the failing test — tests/test_determinism_coop.gd:

extends GutTest


# 2-pilot determinism: the co-op trace must be byte-identical for a given seed,
# and drone state must be VISIBLE to the checksum (a P2 drone desync must move it).


func _trace(seed_v: int, ticks: int) -> Sim:
  var sim := Sim.new(seed_v, SimContentFixture.db())
  sim.enable_survival()
  sim.add_player2()
  for i in range(ticks):
    var d1 := Vector2(cos(float(i) * 0.05), sin(float(i) * 0.03)).normalized()
    var d2 := Vector2(cos(float(i) * 0.04 + 1.7), sin(float(i) * 0.06 + 0.4)).normalized()
    # Force-charge both decoys periodically so drones actually DEPLOY inside
    # the window (organic charge is slower than 600 ticks) — applied
    # identically to every trace, so determinism comparisons stay valid.
    if i % 120 == 0:
      for pilot in sim.pilots:
        pilot.drone_charge = 1.0
    sim.tick_single(InputState.new(d1, Vector2.ZERO, i % 120 == 0, false),
      InputState.new(d2, Vector2.ZERO, i % 120 == 0, false))
  return sim


func test_coop_trace_is_stable() -> void:
  var a := _trace(1234, 600)
  var b := _trace(1234, 600)
  assert_eq(a.state_checksum(), b.state_checksum(), "co-op is deterministic (a==b)")
  assert_eq(a.snapshot_string(), b.snapshot_string())
  assert_gt(a.drones.size(), 0, "the trace actually fielded at least one drone slot")


func test_checksum_sees_drone_state() -> void:
  var a := _trace(1234, 600)
  var before := a.state_checksum()
  var moved := false
  for d in a.drones:
    if d.active:
      d.pos += Vector2(1, 0)
      moved = true
      break
  assert_true(moved, "an active drone existed to perturb")
  assert_ne(a.state_checksum(), before, "perturbing a drone MOVES the checksum")


# Absolute 2-pilot baseline (seed 1234, 600 ticks, the exact input streams above).
# Pinned like the single-pilot baseline in test_determinism_survival.gd — a move
# here means the co-op trace changed; re-pin deliberately via bh-determinism-repin.
func test_coop_baseline_pins() -> void:
  var a := _trace(1234, 600)
  assert_eq(a.snapshot_string().hash(), 0, "PIN-ME snapshot hash")
  assert_eq(a.state_checksum(), 0, "PIN-ME checksum")

  • Step 2: Run -gselect=test_determinism_coop — expect test_checksum_sees_drone_state to FAIL (drones invisible today) and the PIN-ME pins to fail (placeholder zeros). test_coop_trace_is_stable should already pass; if drone deployment via decoy=true doesn’t fire, read how Sim.tick() consumes the decoy edge + drone_charge and fix the trace setup, not the sim.

  • Step 3: Add drones to state_checksum() (after the gems loop, sim/sim.gd ~line 2588):

  for d in drones:
    parts.append(d.active)
    parts.append(d.pos)
    parts.append(d.hp)
    parts.append(d.charge)
    parts.append(d.life)
    parts.append(pilots.find(d.owner))

And to snapshot_string() (before the xp/lv loop, keeping every existing segment byte-identical in position):

  var live_drones := 0
  for d in drones:
    if d.active:
      live_drones += 1
  s += " dr=%d" % live_drones

  • Step 4: Re-run the SINGLE-pilot determinism tests. The checksum only gains parts if sim.drones holds entries in the baseline window; the snapshot string gains dr=0 unconditionally → the snapshot-hash pin WILL move; this is the planned, deliberate re-pin. Invoke the bh-determinism-repin skill: confirm a==b still holds, update the pinned literals in tests/test_determinism_survival.gd + tests/test_determinism_checksum.gd with a dated comment (“Re-pinned 2026-07-XX: drone state added to snapshot/checksum (QA build-out Task 4). Prior: 4217109746 / 3312602286.”), and update the CLAUDE.md Current-status line.

  • Step 5: Pin the co-op baseline. Run -gselect=test_determinism_coop, read the two actual values from the failure output, replace the PIN-ME zeros, re-run → ALL PASS.

  • Step 6: Full suite + count guard + boot smoke. Commit — git commit -m "test(sim): 2-pilot determinism baseline; drone state joins checksum/snapshot (deliberate re-pin)".

Task 5: Input-replay recorder + replayer

Section titled “Task 5: Input-replay recorder + replayer”

Files:

  • Create: input/replay_recorder.gd (class_name ReplayRecorder — render-side, does file I/O, NOT in /sim)
  • Create: input/replay_loader.gd (class_name ReplayLoader — parses a replay back into InputStates; pure enough for tests but file I/O keeps it out of /sim)
  • Modify: main.gd (dev-flag hook at the sim.tick_single(input, input2) call, line ~1043; seed capture where Sim.new is constructed, ~line 228/534)
  • Test: tests/test_replay.gd

Interfaces:

  • Produces: ReplayRecorder.new(seed) with record(input: InputState, input2: InputState) per tick and save(path: String); ReplayLoader.load_file(path) -> Dictionary returning {seed: int, ticks: Array} where each tick entry rebuilds via ReplayLoader.input_at(data, i, pilot_idx) -> InputState. JSON schema: {"seed": int, "ticks": [[mx,my,ax,ay,dec,dash, mx2,my2,ax2,ay2,dec2,dash2], ...]} (flat arrays — compact; second pilot block all-zero when absent).

  • Step 1: Write the failing test — tests/test_replay.gd:

extends GutTest


# Record → save → load → re-run must land on the SAME final checksum: the
# replay layer is lossless. This is the integration net the synthetic
# AutoPlayer soak can't provide (real human input streams become fixtures).


const TMP := "user://test_replay_roundtrip.json"


func test_roundtrip_reproduces_the_run() -> void:
  var rec := ReplayRecorder.new(777)
  var live := Sim.new(777, SimContentFixture.db())
  live.enable_survival()
  for i in range(300):
    var inp := InputState.new(
      Vector2(cos(float(i) * 0.07), sin(float(i) * 0.02)).normalized(),
      Vector2.RIGHT if i % 30 < 5 else Vector2.ZERO,
      i % 90 == 0, i % 47 == 0)
    rec.record(inp, null)
    live.tick_single(inp)
  rec.save(TMP)
  var data := ReplayLoader.load_file(TMP)
  assert_eq(int(data.seed), 777)
  assert_eq((data.ticks as Array).size(), 300)
  var replay := Sim.new(int(data.seed), SimContentFixture.db())
  replay.enable_survival()
  for i in range(300):
    replay.tick_single(ReplayLoader.input_at(data, i, 0))
  assert_eq(replay.state_checksum(), live.state_checksum(), "replay is lossless")
  DirAccess.remove_absolute(ProjectSettings.globalize_path(TMP))

  • Step 2: -gselect=test_replay → FAIL (classes missing).

  • Step 3: Implement the two classes. input/replay_recorder.gd:

class_name ReplayRecorder
extends RefCounted


# Dev-only input-stream recorder (main.gd RECORD_REPLAY flag). Buffers every
# tick's InputStates + the run seed, saves JSON on demand. Render-side: file
# I/O is fine here, NEVER in /sim.


var seed_value: int
var ticks: Array = []


func _init(seed_v: int) -> void:
  seed_value = seed_v


func record(input: InputState, input2: InputState) -> void:
  var row := [input.move_dir.x, input.move_dir.y, input.aim_dir.x, input.aim_dir.y,
    1 if input.decoy else 0, 1 if input.dash else 0]
  if input2 != null:
    row.append_array([input2.move_dir.x, input2.move_dir.y, input2.aim_dir.x,
      input2.aim_dir.y, 1 if input2.decoy else 0, 1 if input2.dash else 0])
  else:
    row.append_array([0, 0, 0, 0, 0, 0])
  ticks.append(row)


func save(path: String) -> void:
  var f := FileAccess.open(path, FileAccess.WRITE)
  if f == null:
    push_error("ReplayRecorder: cannot write %s" % path)
    return
  f.store_string(JSON.stringify({"seed": seed_value, "ticks": ticks}))
  f.close()
  print("[replay] saved %d ticks -> %s" % [ticks.size(), path])

input/replay_loader.gd:

class_name ReplayLoader
extends RefCounted


static func load_file(path: String) -> Dictionary:
  var f := FileAccess.open(path, FileAccess.READ)
  if f == null:
    return {}
  var data: Variant = JSON.parse_string(f.get_as_text())
  f.close()
  return data if data is Dictionary else {}


static func input_at(data: Dictionary, tick: int, pilot_idx: int) -> InputState:
  var row: Array = (data.ticks as Array)[tick]
  var o := pilot_idx * 6
  return InputState.new(Vector2(row[o], row[o + 1]), Vector2(row[o + 2], row[o + 3]),
    int(row[o + 4]) == 1, int(row[o + 5]) == 1)

  • Step 4: -gselect=test_replay → PASS.

  • Step 5: Wire the main.gd dev hook. Add near the other dev consts: const RECORD_REPLAY := false and var _replay_rec: ReplayRecorder = null. Where the run’s Sim is constructed (grep Sim.new( in main.gd, ~line 534 area — capture the seed variable used there): if RECORD_REPLAY: _replay_rec = ReplayRecorder.new(<the seed var>). At the tick site (line ~1043): if _replay_rec: _replay_rec.record(input, input2) immediately before sim.tick_single(input, input2). On game-over handling (grep game_over in main.gd render path): if _replay_rec: _replay_rec.save("user://replays/replay-%d.json" % Time.get_unix_time_from_system()) (plus DirAccess.make_dir_recursive_absolute("user://replays")). If main.gd derives the seed inline rather than via a variable, hoist it into a local first — behaviour-identical.

  • Step 6: Boot smoke with the flag OFF (default) and a manual editor run with it ON (flip, play 10 s, die/quit, confirm a replay JSON lands in user://replays/, flip back). Full suite + determinism pins + count guard.

  • Step 7: Commit — git commit -m "feat(input): dev-flag input-replay recorder + lossless replayer".

Task 6: UI screenshot regression sweep

Section titled “Task 6: UI screenshot regression sweep”

Files:

  • Create: tools/ui_sweep/sweep.gd + tools/ui_sweep/sweep.tscn (Node scene, script attached — same shape as marketing/capture/shoot.tscn)
  • Create: scripts/ui-sweep.sh
  • Create: tools/ui_sweep/baseline/.gitkeep (baselines committed after first blessed run)
  • Test: none (visual tool; verified by running it — GUT can’t see MultiMesh/panel rendering anyway)

Interfaces:

  • Consumes: main.tscn, the start-menu/panel signals (ship_config_requested, shop_requested, bestiary_requested), main._on_mode_chosen("survival"), the Sim dev seams marketing/capture/shoot.gd already uses (copy its _prepare() staging approach).

  • Produces: tools/ui_sweep/out/<screen>.png captures; scripts/ui-sweep.sh [--bless] → RMSE-diff vs tools/ui_sweep/baseline/, exit 1 on regression/new-unblessed screen.

  • Step 1: Write tools/ui_sweep/sweep.gd (start from marketing/capture/shoot.gd’s SubViewport + _frames() + _capture() helpers — copy those verbatim, then replace the scene table):

extends Node
# UI screenshot sweep — DEV ONLY. Run WINDOWED (not headless: MultiMesh
# per-instance colours don't render headless):
#   godot --path . --rendering-method mobile res://tools/ui_sweep/sweep.tscn
# Captures every menu/panel at TV size into tools/ui_sweep/out/ for
# scripts/ui-sweep.sh to diff against tools/ui_sweep/baseline/.


const SIZE := Vector2i(1920, 1080)
const SETTLE_FRAMES := 90   # staggered card-entrance tweens need a GENEROUS wait


# id -> a Callable staging the screen. Order matters: menu screens first
# (pre-run), then in-run overlays.
var screens: Array = []


var vp: SubViewport
var main


func _ready() -> void:
  vp = SubViewport.new()
  vp.size = SIZE
  vp.render_target_update_mode = SubViewport.UPDATE_ALWAYS
  vp.handle_input_locally = false
  add_child(vp)
  main = preload("res://main.tscn").instantiate()
  vp.add_child(main)
  await _frames(10)
  DirAccess.make_dir_recursive_absolute("res://tools/ui_sweep/out")
  screens = [
    ["start_menu", func(): pass],
    ["ship_config", func(): main.start_menu.ship_config_requested.emit()],
    ["shop", func(): main.start_menu.shop_requested.emit()],
    ["bestiary", func(): main.start_menu.bestiary_requested.emit()],
    ["hud_in_run", func(): main._on_mode_chosen("survival")],
    ["pause", func(): main._toggle_pause()],
  ]
  for entry in screens:
    var id: String = entry[0]
    (entry[1] as Callable).call()
    await _frames(SETTLE_FRAMES)
    await _capture("res://tools/ui_sweep/out/%s.png" % id)
    print("[ui_sweep] captured %s" % id)
    await _reset_between(id)
  print("[ui_sweep] DONE")
  get_tree().quit()

The implementer must read main.gd/ui/start_menu.gd for the REAL close/reset calls (_reset_between) and the exact member names (main.start_menu, _toggle_pause) — verify each staging call exists before using it; where a panel needs has_progressed or a populated codex, copy the pre-seed technique from shoot.gd’s _prepare(). Add the level-up panel + reset-shop dialog + dimension popup screens the same way if a clean staging seam exists; skip (with a printed SKIPPED line) any that would need invasive new seams — no silent caps.

  • Step 2: Write scripts/ui-sweep.sh:
#!/bin/zsh
# UI screenshot regression: run the windowed sweep, RMSE-diff vs baselines.
#   scripts/ui-sweep.sh           # diff mode: exit 1 on regression or unblessed screen
#   scripts/ui-sweep.sh --bless   # accept current captures as the new baseline
cd "$(dirname "$0")/.."
OUT=tools/ui_sweep/out BASE=tools/ui_sweep/baseline THRESH=0.06
godot --path . --rendering-method mobile res://tools/ui_sweep/sweep.tscn
if [ "$1" = "--bless" ]; then mkdir -p "$BASE"; cp "$OUT"/*.png "$BASE/"; echo "blessed $(ls "$OUT" | wc -l | tr -d ' ') baselines"; exit 0; fi
fail=0
for f in "$OUT"/*.png; do
  name=$(basename "$f")
  if [ ! -f "$BASE/$name" ]; then echo "NEW (unblessed): $name — eyeball it, then --bless"; fail=1; continue; fi
  rmse=$(magick compare -metric RMSE "$BASE/$name" "$f" null: 2>&1 | sed 's/.*(\(.*\))/\1/')
  if (( $(echo "$rmse > $THRESH" | bc -l) )); then echo "REGRESSION: $name RMSE=$rmse (> $THRESH)"; fail=1
  else echo "ok: $name ($rmse)"; fi
done
exit $fail

(tools/ui_sweep/out/ gets a .gitignore entry; baseline/ PNGs are committed once blessed. Verify magick exists — brew install imagemagick if not.)

  • Step 3: Run the sweep windowed on the Mac, eyeball every capture, --bless, commit the baselines. Then run diff mode → all ok. Deliberately break one screen locally (e.g. tint a panel), re-run → REGRESSION detected, revert.

  • Step 4: Full suite + count guard + boot smoke (nothing game-side changed; this confirms no stray scene references).

  • Step 5: Commit — git commit -m "feat(tools): windowed UI screenshot regression sweep + bless/diff wrapper".

Self-review notes

Section titled “Self-review notes”
  • Spec coverage: 1a soak ✅ (Tasks 1–3), 1b screenshot sweep ✅ (Task 6), 1c input replay ✅ (Task 5), 1d co-op determinism gap ✅ (Task 4). 1e (overnight-review re-run) is an ops action, not code — runs after this plan lands.
  • Deliberate exclusions (YAGNI): per-enemy-mechanic trigger counters (outcome stats via kills/damage cover the bug class at far lower invasiveness); soak parallelism (one headless process is already ~50-100× real-time; sweep overnight).
  • Type consistency check: SoakStats.on_damage/on_kill names match Task 1’s sink calls; compute_move_for/wants_dash/wants_decoy match Task 3’s _input_for; ReplayLoader.load_file/input_at match Task 5’s test.
  • Known verify-before-trusting points for implementers: PlayerState.drone_charge field name; enemies.data[i] as type id; tick_single null second param; MetaState.new() I/O-free; exact main.gd seed variable; real panel-staging seams in Task 6.