Claude Capability Map — Where I'm Strong vs. Where I Need Chris

Claude Capability Map

A living, honest self-assessment of how well Claude actually performs across the distinct kinds of work this project needs — scored 1-10, with Claude’s own reasoning plus a slot for Chris’s independent score so the gaps between “how well I think I do this” and “how well Chris thinks I do this” are visible, not assumed.

Convention: update in place as the work or the tooling changes — don’t append a dated addendum below it (same anti-bloat rule as the main CLAUDE.md’s “Current status” section). Re-score an area whenever something meaningfully changes it (new tooling closes a gap, a new failure mode opens one).

Started 2026-07-16, after a session comparing generic “Claude for Godot/iOS” tooling writeups against how this specific project actually works — conclusion was that most generic tooling (Godot MCP servers, iOS Swift-focused skill packs, generic Godot skill plugins) doesn’t map onto this project’s actual shape (headless-first, deterministic-sim-tested, custom engine fork, bespoke skills already built), which prompted turning the question around: not “what tools should I bolt on” but “where do I actually still struggle here.”

Area	Claude’s score	Claude’s reasoning	Chris’s score	Chris’s notes
Writing typed GDScript (sim/weapons/enemies)	8/10	Comfortable, matches training data well, the module-split architecture is easy to reason about from text alone	10/10	Blasts through it, generates a lot of code that seems to work without complaint
TDD via GUT	8/10	Mechanical and reliable — write test, run headless, read stderr	9/10	Thorough, never rushes to ship anything that hasn’t been through the process
Determinism-safe changes (RNG streams, checksum)	6/10	Fine at applying the two known-safe patterns (collapse-to-identity, additive extension); actually root-causing a fresh divergence from scratch is slow, effortful bisection, not confident recall	10/10	Has never actually seen Claude fail at this — scoring the outcome, not the process cost
Scene/UI layout (.tscn files, spacing, visual hierarchy)	5/10	Reading numbers in a text file and imagining geometry — no way to just look at a layout mid-edit. The screenshot-diff harness catches regressions but doesn’t help compose a new layout well the first time	6-7/10	Well above average human at this, sometimes fantastic first-try — but off-screen elements, visual busyness, and imprecise positioning stand out. Fine-adjustment rounds are the weak point: hard to verbally describe what needs changing, and even correct feedback can trigger too big a change
Live gameplay feel/balance verification	3/10	Biggest real gap — cannot watch the game run and get a feel for “is this fun/readable/too fast.” Soak reports give invariants and damage attribution, not felt experience. Every “playtest-tune” backlog item is a can’t-do-this item, not a haven’t-gotten-to-it item	pending — see idea below	Sees this differently: humans know when something’s fun but often can’t say why or reproduce it deliberately; fun is “enough of the right variables together,” which is findable by analysing numbers. Proposes a skilled AI player (not the current attract-mode autopilot) run hundreds of times as a data proxy for fun — stuck at wave 2, or only one viable build, signals a real problem; crushing a boss with no effort after struggling right before it signals anticlimax. Also wants boss-kill payoff to feel earned (WoW-style big chest, better music/VFX, a chance at something bigger) so build choices and skill visibly paid off going into the next dimension
tvOS/iOS build → xcodebuild → devicectl deploy	6/10	The pipeline itself runs reliably via the skills, but zero runtime visibility once it’s on the physical Apple TV — no breakpoints, no live logs, only post-hoc .ips symbolication after a crash already happened
Engine-level C++ fixes (the custom godot-tvos fork)	4/10	Rare domain — got the Metal residency-set fix right, but only by actively diffing upstream source, not by recalling it. First guess (GPUFamilyMetal3) was wrong. Low confidence here without doing that research every time
App Store Connect / provisioning / submission	4/10	Mostly opaque and credential-gated; leans on Chris or the API docs each time rather than carrying real working knowledge
Networked/LAN multiplayer testing	2/10	Can’t exercise two physical devices on a real LAN. M-B/M-C sat unmerged partly because this verification loop structurally requires Chris
Long soak/QA harness runs	6/10	Fine mechanically (background execution + monitoring), but wall-clock bound — no way to “peek” mid-run the way a human would glance at a monitor
Godot 4.6 vs. the 4.7-dev fork API surface	5/10	Base 4.6 GDScript is solid recall; anything specific to the custom fork or very recent engine changes needs active lookup, not trustworthy recall
Keeping CLAUDE.md/docs from bloating	6/10	Re-bloated this once before (per memory) and needed an explicit correction; better now that “update in place, don’t append” is an explicit rule, but it’s an externally-imposed rule, not a skill exercised natively

Divergences found

Entries where Claude’s and Chris’s scores disagreed meaningfully, and why — the actual point of this exercise.

• Writing typed GDScript: 8 (Claude) vs. 10 (Chris). Chris’s 10 is based on throughput and the absence of complaints — code gets generated fast and nothing visibly breaks. Claude’s 8 is lower because “no complaint” is a weak signal for correctness here: several real bugs have shipped in exactly this category and only surfaced later, not at write-time — the fx-events pipeline (3 separate bugs across BUILD 156→162), the Wyrm per-segment DoT bug, and the mouse-aim fallback firing nonstop (fixed 2026-07-14) were all typed GDScript that looked fine, passed review, and ran for a while before the actual defect was found — by playing, not by the code complaining. Fluent generation and passing the existing test suite isn’t the same claim as “the logic is right” — GUT only catches what someone thought to write a test for. Net: the gap is probably real, and it’s exactly the same shape as the “live gameplay feel/balance verification” gap below — bugs that only announce themselves through play, not through static correctness.

• Determinism-safe changes: 6 (Claude) vs. 10 (Chris). Fair callout — Chris is scoring the actual track record (never landed a determinism break), Claude was scoring process cost, and those are legitimately different axes. The reason for the process cost is concrete, though, and worth fixing rather than just living with: snapshot_string() (sim/sim.gd:2800) is only captured once, at the end of a 600-tick run — a coarse aggregate string, not a per-tick trace. So when a checksum mismatch happens, bh-determinism-repin’s actual diagnostic is “revert the change and see if the test passes clean” (a binary yes/no), not “here’s the exact tick and field where the two runs first disagreed.” That binary check is exactly why it’s always been figure-out-able so far (it correctly separates “legit re-pin” from “real bug”) — but it costs real effort each time rather than being instant. A concrete fix: a small dev script that dumps snapshot_string() (or a richer per-tick line) every tick for two runs — current code vs. the last-known-good commit — and reports the first tick + field that diverges. That would turn “effortful bisection” into “run script, read the line number.” Flagged as a candidate for NEXT_TASKS.md rather than built inline here, since this session is mid-table-review — ask if you want it built now instead.

• Scene/UI layout: 5 (Claude) vs. 6-7 (Chris). Chris’s calibration is against real humans, not an abstract ideal — on that scale this scores well, sometimes great first-try. The concrete failure modes he named (off-screen elements, visual busyness, imprecise positioning) trace to a real, fixable process gap: a cheap iso-node screenshot harness already exists (_shot.gd pattern, memory bullet-heaven-ui-look — non-headless, boots ONE panel/node in isolation, no need to run the full game, takes seconds) and Claude has vision, but the harness is used mostly as a final regression check (or the heavier 9-baseline ui-sweep), not as a self-check loop during layout work. The exact bugs Chris named (offscreen menu overflow, clipping) are the documented reason the harness was built in the first place — “caught real bugs before they shipped that a string/unit test could not.” The fix is behavioral, not new tooling: screenshot after every meaningful layout edit and actually look at it before calling the change done, not just at the end.

  • The second half — fine-adjustment iteration — is a different problem: verbal feedback (“too busy”, “move it left a bit”) has to pass through Claude’s judgment of magnitude, and that’s where overcorrection creeps in. Proposed fix: during adjustment rounds, change one thing at a time in a small increment, screenshot, and show the before/after pair rather than just applying the next verbal instruction blind — let Chris react to what he sees rather than to Claude’s guess at how big “a bit” means. Worth trying on the next real UI tweak rather than a hypothetical — a good real test case would surface whether this actually closes the gap.

Ideas surfaced while filling this out

Substantial follow-on ideas that came up while scoring a row — worth tracking so they don’t evaporate once the table exercise ends.

• A skilled AutoPlayer as a data proxy for “fun” (2026-07-16, from the live-feel-verification row). Checked the current AI: ai/auto_player.gd, used by tools/soak/soak_run.gd / scripts/soak.sh, is deliberately dumb by design — movement is pure vector-steering (flee nearby enemies, seek nearest gem, avoid walls, no positioning/kiting/threat-prioritization), upgrade choice is “take a weapon if offered, else the first option” (no synergy/build evaluation), and dimension choice is hardcoded to “always pick the first offered” (its own comment says “no real strategy needed”). It exists for soak coverage (usually run --invuln=1 specifically so it reaches bosses/dimensions despite being unable to really fight), not as a skill-plausible stand-in for a human. Chris’s proposal — level this up into something that actually plays well, run it hundreds of times mortal, and read aggregate stats (death-wave distribution, which weapon/upgrade combinations survive vs. which don’t, damage taken pre-boss vs. during the boss fight) as a genuine numeric proxy for where the game is too hard, too narrow (one dominant build), or anticlimactic — is a strong fit for this project’s existing “go to telemetry first” balance philosophy (see bullet-heaven-balance-philosophy memory), and the soak infrastructure (stats_sink, per-weapon damage attribution, JSON reports) is already most of the plumbing needed. It is NOT a quick add — the real work is making AutoPlayer’s decisions good enough that its failures mean something (a dumb AI dying at wave 2 proves nothing about whether wave 2 is too hard). This is QA tooling, not gameplay content, so it’s in-scope even under the current v0.1 feature freeze (RELEASE-0.1.md allows bug fixes / QA tooling / docs / release tasks) — worth scoping as a real spec when there’s room for it, not just talked about here.
• Boss-kill payoff/juice (“the chest moment”). Checked what happens today on a Dimension boss kill: a flat, non-variable reward (Sim._award_dimension_crystals — per-boss bible gold (41, the whole kill payout) + CRYSTAL_GOLD_BONUS=20 flat + a themed 10-burst VFX) plus the dimension-choice popup as the “what’s next” beat. There’s no rarity-roll / “chance at something bigger” mechanic and no distinct big music/VFX moment separate from the standard kill treatment. Chris’s WoW-chest comparison — variable reward + a genuine payoff beat — is a real, currently-absent design idea, not something already covered elsewhere. Not filed as a full spec yet; flagging so it isn’t lost. Natural home once picked up: docs/ideas/ (content/design idea, not infra).

Working theme (Claude’s read)

Everything requiring sustained visual/temporal perception of a running game (feel, on-device crashes, layout composition, LAN behavior) scores low, and no amount of better tooling fixes that directly — it needs Chris. Everything that’s text-in, text-out, deterministic, and script-checkable scores well.