diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/39-RE-WORKFLOW-REVIEW.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/39-RE-WORKFLOW-REVIEW.md | 678 |
1 files changed, 678 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/39-RE-WORKFLOW-REVIEW.md b/debug/risc-v-gsp-rm-vm/docs/39-RE-WORKFLOW-REVIEW.md new file mode 100644 index 0000000..2b94746 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/39-RE-WORKFLOW-REVIEW.md @@ -0,0 +1,678 @@ +# 39 — Reverse-engineering workflow review: tools, real usage, time sinks, stale assumptions and pragmatic upgrades + +> **Status:** review (2026-10-03). Written by a read-only analysis pass over the +> tool tree, the public state page, the private project session logs (cited by +> PART/ADDENDUM and date only; no private paths appear here) and the +> engineering-ops framework. It is a documented review, not a product change; +> nothing in it authorises hardware, module, guest, mdev or RM 610 work. +> +> **Scope of the review:** every reverse-engineering tool/harness/debugger this +> project has built or used, how each has actually been used, where wall-clock +> time is lost, which tool-design assumptions are now stale, how the +> documentation can be made cheaper to maintain, and the smallest set of +> upgrades that would move the dominant cost first. +> +> **Method for reading it:** claims are labelled **[E]** (evidence: a file, +> line, session or recorded event) or **[I]** (inference: the analyst's +> judgement). Section references use `docs/NN` for the tool's documentation set +> and `open-iov/...` for tracked project files. Private project session records +> are cited as `[log PART nn/Add. nn, date]`; they live outside the tracked +> tree and are not named here. + +--- + +## §1 Method and sources + +### 1.1 Question + +The review answers one question for the risc-v-gsp-rm-debugger program: **what +actually reduces the time from "a new GSP/driver release or a live product +wall" to "a supported `vgsp`"** — and where is that time being lost today? + +It is deliberately not a feature wish-list. Every recommendation in §8 carries +the engineering-ops admission fields (scenario, measurable outcome, baseline, +threshold, failure signal) in the shape required by +`open-iov/docs/eng-ops.md` §2. + +### 1.2 Sources read + +| Class | Source | What it provided | +|---|---|---| +| Public live state | `open-iov/STATE-CURRENT.md` (snapshot 2026-10-02 22:12) | milestone ladder, frontier walls, physical-backend record, tool status paragraph | +| Tool documentation | `debug/risc-v-gsp-rm-vm/docs/00`–`38`, README, roadmap | design, accuracy contract, audit, value demos, porting/CI, new simulator work (30–38) | +| Code and harnesses | `greeneye/`, `sim/`, `harness/` (`*.py`, `value_demos/`, `qemu_line/`, `re_demo/`), `plugins/`, `tests/`, `qemu/`, `ghidra/`, `rmharness/`, `gpgpu-sim/` overlay, `sdk/` | what exists and what its tests exercise | +| Recorded evidence | 458 session directories under `dumps/` (counted 2026-10-03), `harness/qemu_line/sim-qemu-line.json`, `harness/re_demo/result.json`, value-demo artifact JSON | what has actually been run, not just built | +| Project ops | `open-iov/docs/eng-ops.md`, `future-eng-ops.md`, `RUNBOOK.md`, `docs/05-operations/03`, `04`, `docs/07-developers/02` | admission discipline, unused-capability catalogue, scenario runbooks | +| Private project logs | the session-part logs and the tool-program ledger (2026-09-24 → 2026-10-03), the progress-analysis log (2026-09-29 → 2026-10-02) | real walls, real costs, real tool usage (cited by PART/date only) | +| Project tooling | Ghidra corpora + headless helper scripts, CUDA porting tools, guest diagnostics, host RM oracle, physical claim harness | the incumbent methods the tool must beat | + +### 1.3 Method and limits + +* **Behaviour framing.** The review describes components and behaviour; it + names no source locations, vendor file paths or filenames. +* **Evidence over recollection.** Cost claims are anchored to recorded numbers + (docs/25 §3–§6, `STATE-CURRENT.md`, PART/date) rather than estimates, except + where explicitly marked as an estimate. +* **No new runs were performed.** No hardware, GPU, module, mdev, guest or + reboot was touched; no test suite was re-run. Test-count claims are therefore + quoted from their source with the source named, and the count discrepancy + found in §5 is reported, not resolved. +* **Sampling.** The 458 session directories were classified by name and + timestamp; individual sessions were sampled, not all read. +* **Fidelity rules apply.** All simulator evidence is L0/L1 unless a source + says otherwise (`docs/06`); every "blocked" real-firmware projection in this + review is marked as such. + +--- + +## §2 Complete RE/tooling inventory + +### 2.1 The tool tree (`open-iov/debug/risc-v-gsp-rm-vm/`) + +Status vocabulary follows `docs/27` §0: **verified useful** (exercised and/or +wired into tests/demos/docs), **in development** (owner-directed work landed +but unfinished; docs 30–38 and the untracked `sim/devices`/`gpgpu-sim` trees), +**pilot** (unadmitted, per `future-eng-ops` F2–F4), **built, not used yet** +(no consumer). + +| # | Tool | Location | Purpose | Who uses it | Evidence of use | Status | +|---|---|---|---|---|---|---| +| 1 | Launcher + CLI (`info/demo/run/attach/launch/probe/record/analyze/compare/symbols/plugins/inject/ghidra`) | `risc-v-gsp-rm-debugger.sh`, `greeneye/cli.py` | entry surface; run/inspect real or demo firmware | agents, owner | audit smokes (`docs/27` §1); sessions in `dumps/` (`demo`, `probe-*`, `inject`) | verified useful | +| 2 | Session/scripting API (`launch`, `attach`, `Session`, hooks, watchpoints, `call`, `run_until`, memory/structs/snapshots) | `greeneye/session.py`, `memory.py`, `structs.py`, `record.py`, `analyze.py` | Frida-like core; sessions as evidence | scripts, tests, value demos | `docs/27` §2; s01–s10 | verified useful | +| 3 | Pure-Python RSP client (QEMU backend) | `greeneye/rsp.py`, `target.py` | transport; no GDB dependency | default backend | demo/probe live-verified (`docs/01` §3) | verified useful | +| 4 | Native RV64IMAC backend (M1) | `sim/decode.py`, `sim/machine.py`, `greeneye/sim_backend.py` | deterministic in-process backend; QEMU parity | tests, demos, new sim work | differential test (`docs/16` §1.1); `sim_diff.py` `PARITY: OK` | verified useful (reference core) | +| 5 | Native exec/access trace (M1.5/R4) | `sim/trace.py` | full instruction/access trace as session blob | record/analyze, value demos | `docs/16` §1.1a; trace tests | verified useful | +| 6 | Modeled cycle counter | `sim/cycles.py` | table-driven modeled cycles (explicitly not silicon timing) | 21 offline tests | `docs/32`; `results.json` of `test_sim_cycles` | in development (increment 0) | +| 7 | GSP device model + profiles + bounded boot runner | `sim/devices/`, `sim/profiles/`, `sim/gsp_boot.py`, `sim/gsp_profile.py` | real 570.144 image past the QEMU first fault (B0–B2) | S1 workstream | `dumps/gsp-boot-b1-run1/2`, `b2-run1/2`; 2-run determinism asserted (`docs/36` §4) | in development (S1) | +| 8 | Declarative probe specs | `greeneye/probes.py`, `examples/probes/` | JSON hooks/conditions without scripts | audit smoke; documented pattern | `docs/27` §3 (3 probes fired) | verified useful | +| 9 | Plugin loader + example plugin | `greeneye/plugins.py`, `plugins/example_plugin.py` | extension point | audit smoke | `plugins` CLI lists all three | verified useful | +| 10 | MCP server | `plugins/mcp_server.py` | agent/CI machine surface (JSON-RPC over stdio/loopback) | no live agent consumer found | 55 tests (`docs/20` §9); tool catalogue 31 (`docs/20` §5) | implemented; **unused in real workflow** (F1 `partially used`) | +| 11 | Injection SDK + `inject` | `sdk/`, `greeneye/inject.py` | position-independent RV64 service load/call, direct + mailbox | tests, s06 | `docs/11`/`12`; audit smoke `0xbeae` both paths | verified useful | +| 12 | Ghidra bridge (symbol export, hit annotation, control) | `ghidra/*.java`, `greeneye/ghidra.py`, `gdbridge.py` | static↔dynamic address join; run-until from decompiler | Ghidra headless (program ledger wave 2) | 25-command bridge tested (`docs/27` §2); Java verified in the tool ledger | verified useful | +| 13 | TCG full-trace plugin | `qemu/greeneye_trace.c` | complete exec/access JSONL without stops | audit live run | 8,012,129 events in ~1 s (`docs/27` §0) | verified useful | +| 14 | Mutation harness | `harness/mutation_harness.py` | bounded deterministic one-field mutation; outcome classes | tests, s08 | seed replay byte-identical (`docs/18` §2) | verified useful | +| 15 | Portability matrix | `harness/portability_matrix.py` | probe+scenario+compare across targets | tests | self-comparison all-zero deltas | verified useful | +| 16 | Release triage pipeline | `harness/version_diff.py`, `harness/ci_release.py` | old/new image diff + hashed bundle; RM610 gate | tests; CI docs | self-diff zero; gate refusals tested (`docs/21`) | verified useful | +| 17 | Control discovery + reachability | `harness/control_discovery.py`, `harness/reach.py` | unknown-call sweep; "did it run / what args" | demo sweep only | 189 probes ≈ 36 s (`docs/19` §3.4); `--surface rm` exits clean | `--surface demo` verified; real-RM surface **built, not used yet** (F5) | +| 18 | Simulator-vs-QEMU A/B harness | `harness/sim_qemu_line.py` | frozen-threshold evaluation on the real image | S4 workstream | verdict `above_lenient_only`, `device_model_used=false` (`harness/qemu_line/sim-qemu-line.json`) at review time; **same-day post-review**: S1 profile discovery wired, verdict `above`, native B2 (docs/37 §4.3) | in development (S4) | +| 19 | Real-RE proof harness | `harness/re_demo.py` | U51 demonstration: entry→first-fault question | S4 workstream | `harness/re_demo/result.json`; baseline floor estimate 600 s vs 0.13 s emulator | in development | +| 20 | Driver-version sweep | `harness/port_driver_versions.py` | 540→current per-version evidence | none in a real sweep yet | 23 tests; dry-run default; versioned inputs absent locally | pilot (unadmitted F2) | +| 21 | New-device scaffold | `harness/port_new_device.py` | read-only device profile capture + pairing | none yet | tests; no profile captured in a workflow | pilot (unadmitted F3) | +| 22 | Trial packaging | `harness/package_sim.py` | deterministic sim+injection bundle for review | none yet | 6 tests; `--pack-image` pre-injected demo | pilot (unadmitted F4) | +| 23 | Value demos s01–s10 | `harness/value_demos/` | measured analog time-to-value vs recorded real baselines | tests; cited in docs 25/27/eng-ops | `artifacts/results.json` (last local run: 3 demos, s01/s04/s05, 1.72 s) | implemented; artifact currently partial (§5) | +| 24 | Host RM oracle harness | `rmharness/`, wrappers for `tools/rmtool`, `rmtop`, `rmq` | read-only host RM identity/memory/engine captures into sessions | audit capture smoke | 3 tests outside CI; read-only allowlist (`docs/17`) | working with caveats (device tables permission-dependent) | +| 25 | GPGPU-Sim overlay | `gpgpu-sim/` | Phase G full-board substrate (gated) | W1 build agents | CUDA 12 build + `vectorAdd` functional/perf (program ledger §13.3); `BUILD-PORT.md` | in development, deferred behind Phase S | +| 26 | Test suite + CI | `tests/` (22 files), `.github/workflows/risc-v-gsp-rm-debugger.yml` | regression net | CI, every change | 155 at audit; 194 claimed at flag removal; 298 `def test_` statically countable and 298 collected in the 2026-10-03 R2 run (live QEMU present) | verified useful; count claim drifted (§5) | +| 27 | Documentation set | `docs/00`–`38`, README, roadmap | design/usage/proof/portability | agents | link checker 61 files OK at wave 8 | mixed: 00–29 audited; 30–38 unindexed elsewhere | + +### 2.2 Project tooling outside the tool tree (incumbent methods) + +| Tool | Location (public ref) | Purpose | Evidence of use | Relationship to this tool | +|---|---|---|---|---| +| Ghidra corpus + headless helper scripts (`DumpFuncs`, `Xrefs`, `Disasm`, `ScanBranches`, `MemCheck`) | `open-iov/tools/ghidra/` | decompile/disassemble/xref the firmware | `[log PART 92/Add.25, 2026-10-02]` decode of the GR commit assert; `[log PART 91/Add.9]` GPFIFO fix | static half of the loop; the bridge joins it to runtime | +| CUDA/userspace porting tools (`cudatool.py`, `diff_corpus.py`, `annotate_corpus.py`, symbol exports) | `open-iov/userspace/cuda/tools/` | userspace version porting | `docs/22` §2; F2 in production use for 570.144 | separate axis (CUDA userspace); the tool handles the firmware axis | +| Guest diagnostics (`tools/guest-diag/`: ioctl tracer, kprobe/tracker scripts, pagemap tools, VM restart scripts) | `open-iov/tools/guest-diag/` | what the stock driver requested/where it blocks | `STATE-CURRENT.md` (guest-wall fixes); `[log PARTs 29–64]` | complements the firmware question; the tool does not replace it | +| Kernel-module debug planes (default-off params, `/sys/kernel/debug/vgsp/*`, `gsp-phys-state`) | `open-iov/kernel/`, `open-iov/docs/eng-ops.md` §10 | live emulation/physical state | every recorded physical cycle | supplies the real-A/B reference for M1–M6 comparisons | +| Supervised physical claim harness | `tools/guest-diag/phys_claim_cycle.sh` (per `STATE-CURRENT.md`) | safe GPU ownership/boot cycles | 3/3 green pre-lock cycles; all later cycles | the expensive oracle the simulator exists to reduce | +| x86 QEMU/guest harness (`vgpu_vm.py`, `arcd`, guest agent) | `open-iov/scripts/` | drive the stock guest | smoke runs (`ALL OK` 64/64) | produces the payload/failure stimulus | +| Evidence archives (per-cycle logs, rings, decodes) | local evidence archive (untracked) | manual evidence packs | cited per milestone in `STATE-CURRENT.md` | the manual method that sessions/analysis should replace | +| Hourly check-back loop | `checkback/hourly.sh` (per progress analysis §1aq) | commits/pushes `STATE-CURRENT.md` monthly-minutes | running since 2026-09-30 | process automation, not an RE tool | + +### 2.3 Actual usage versus build inventory + +**[E]** Of 458 session directories in `dumps/` (2026-10-03): 413 are +`unittest-*` runner sessions, 5 are `opencode-*` agent sessions, 5 are +`gsp-boot-b*` device-model runs, and the rest are demo/probe/inject/record +sessions from self-verification (list sampled). No session directory in the +tree is tagged with a live product-wall question (e.g. a GR-promote or +UVM-push question). The only real-image sessions are `probe-*` first-fault +runs and the `gsp-boot` B1/B2 ladder runs. + +**[I]** The tool has been *used* overwhelmingly to validate itself and to +characterise the real image's first fault; it has not yet been used as the +daily instrument on a live product wall. That is consistent with the tool's own +honesty (`docs/25` §2.1 marks every real case `blocked_on_device_model`) and +with the private logs: the one place the tool is named as a next route for a +live wall — the GR engine-context promote/commit wall — records it as route 3, +unused so far `[log PART 92/Add.25, 2026-10-02; STATE-CURRENT.md:122-125]`. + +--- + +## §3 Usage history: recurring RE tasks, tools, costs, bottlenecks + +The project's real work decomposes into eight recurring task classes. Each row +gives the recorded cost and the bottleneck that made the cost. + +### 3.1 Task class A — guest-initiation walls (userspace driving the emulated vGPU) + +**Typical form:** the stock guest driver/client blocks or returns an opaque +error; find the first unsatisfied expectation. + +**Recorded instances:** `cuCtxCreate` rc=999 and the 20 per-channel +completion counters (five days, `[log PARTs 29–64, 2026-09-24…29]`); UVM +tracker/semaphore waits; the kernel UVM map wait `[log PART 22]`; clock-domain +probe decode (`STATE-CURRENT.md:520-558`). + +**Tools used:** guest diagnostics (strace on ioctls, guest kprobes, pagemap, +counter/tracker samplers), kernel-module debug planes (channel dump, map +add/probe), QMP `gva2gpa`, serial console, and the inline CUDA driver-API smoke +test. **Recorded cost:** ~5 days for the counter campaign; multiple +days-scale walls before that. **Bottleneck [E]:** guest-side observability — +ptrace was broken in the guest (`[log PART 22]`), so every question cost a +custom sampler; the host's VA→PA map was rebuilt/rescanned under live traffic +and the decisive page (counter page, pool page) had no translation until a raw +live dump proved it (`docs/25` §5 s02). + +### 3.2 Task class B — RM control/RPC gaps and opaque server failures + +**Typical form:** the server faults or reports a generic status; the first +error is invisible. + +**Recorded instances:** the 0x59 RM-init chase (cycles 2–13; **~5 h 10 m**, +10+ supervised boots, reply was a generic `0x59`; `docs/25` §3 s01 and §5 s01); +the missing pre-boot messages (fn72/fn73) found only after the NOCAT storm +collapsed 851→5 records `[log PART 91/Adds.2–6]`; the RM log-ring id bug that +hid the first error (`STATE-CURRENT.md:439-457`). + +**Tools used:** raw host-memory reads of message buffers, the server's own +journal ring (once mapped), static source review, supervised cycles; the +Ghidra corpus + headless scripts for the commit path `[log PART 92/Add.25]`. +**Bottleneck [E]:** one experiment per supervised boot and an invisible first +error; static review closed host-side leads but the decisive field was only +visible in a raw live page. + +### 3.3 Task class C — GR engine/channel bring-up and execution + +**Typical form:** allocate channel, commit context, submit work, get a +fault or a soft `INVALID_STATE`. + +**Recorded instances:** the GR ring layout and 20-channel autopin +(`[log PART 30 addenda]`); the first GPFIFO submit fault +(`docs/25` s04/s09: cycles 18–25, **~1 h 41 m**, 8 supervised boots); the +engine-context promote/commit campaign of 2026-10-02 (modes 4–22, **a full +day**, 730+ lines of firmware RE, three wedge-reboot cycles +`[log PART 92/Adds.13–25]`, and the campaign continued into +`STATE-CURRENT.md:98-131`). + +**Tools used:** the module's own ladder modes and debugfs, the decoded status +queue, Ghidra headless (commit assert `0x52a8b04` classified as a *secondary* +failure), host reads. **Bottleneck [E]:** every hypothesis cost one full +boot+cycle (~5–6 min) and the failing path is server-internal; the one +hypothesis with 730+ lines of static RE produced an accepted root cause +(externally-owned VAS) but the next gate still wedged and required a reboot. + +### 3.4 Task class D — physical boot chain and backend bring-up + +**Typical form:** FWSEC → booter → RISC-V → RM init → clean release on the +real GPU. + +**Recorded instances:** FWSEC-only carve proved non-recoverable (fuse-locked), +requiring an approved reboot and retiring the one-way stage +(`STATE-CURRENT.md:466-519`); the first real boot and first GSP-RM message +(`STATE-CURRENT.md:645-705`); cycles 4–13 to init-done +(`STATE-CURRENT.md:72`, `:495-519`); every cycle ran under the supervised +harness. + +**Tools used:** the module-owned claim/boot path, the power-domain D0 harness, +debugfs state (`gsp-phys-state`), pstore inspection, the fail-safe. **Cost:** +10+ supervised cycles for one question; each failure originally needed a +reboot, until the server's own shutdown supplied the missing handoff bit and +made clean releases possible (`STATE-CURRENT.md:672-705`). +**Bottleneck [E]:** the release path: fused parts cannot be host-recovered, so +any wedge converts a 6-minute experiment into a reboot that kills every other +agent on the host. + +### 3.5 Task class E — guest work forwarding and hardware execution (P5/P6) + +**Typical form:** guest-originated pushes must execute on the physical engine +with correct data and exactly-once releases. + +**Recorded instances:** P6a guest-page DMA (`STATE-CURRENT.md:242-270`), P6b +waves 1/1.5/2, P6c relay, all on 2026-10-02; the P6a→P6c span was +**~02:20–08:30 (~6 h)** with one supervised cycle per rung on average +(`docs/25` §3 s06). Two false walls: the guest-RAM file opened without the +large-file flag (root-caused in-run) and the stale shared-memory image that +poisoned cross-run state and cost **7 cycles** +`[log PART 93/Add.29, 2026-10-02]`. + +**Tools used:** the module's forwarding parameters/counters, a guest pagemap +probe, offline reconstruction scripts against the shared guest-RAM image +`[log PART 93/Add.30]`, and the CUDA smoke as the acceptance gate. +**Bottleneck [E]:** cross-run state contamination and the absence of a +running, symbol-aware firmware view; the decisive 12.7 KB push was decoded +*offline from a dead run's memory image*, not from a live session. + +### 3.6 Task class F — firmware portability/version questions + +**Typical form:** does firmware version X behave differently from anchor Y? + +**Recorded instances:** 570.144 vs 570.211 pairing disproven at one full boot +each (`docs/25` s07; `STATE-CURRENT.md:485-490`); CUDA/userspace porting via +the corpus tools (`docs/22` §2, F2). **Tools used:** manual probe + boot +comparison; `version_diff.py` exists but has no admitted real sweep; `run_all` +s07 demonstrates the shape on the analog. **Bottleneck [E]:** each version +question needs hardware or a hand diff; symbols/corpus transfer warnings +(~1% wrapper re-resolution) are a known trap (`future-eng-ops.md` F2). + +### 3.7 Task class G — evidence assembly and status upkeep + +**Typical form:** turn a cycle into a reviewable claim. + +**Recorded instances:** every milestone's evidence pack is assembled by hand +from per-cycle directories (multi-MB logs/rings/decodes) plus the session log; +the value demo s10 packages the same contract in 4.92 s on the analog, and its +"real case" is explicitly device-model-blocked (`docs/25` §3 s10). Status +upkeep is continuous: an hourly check-back loop commits `STATE-CURRENT.md`, +and the session logs are append-only — the main log alone is ~15k lines +(measured 2026-10-03) with the newest entries prepended. +**Bottleneck [E]:** there is no machine-readable cross-cycle evidence index; +every claim is re-narrated in prose, and prose drifts (see §5). + +### 3.8 Task class H — self-verification and audit + +**Typical form:** prove the tool works so its claims can be trusted. + +**Recorded instances:** two adversarial passes (AR1/AR2/AR3), the 67-surface +audit, flag removal, value demos, 194-test claims; the current wave is +device-model B1/B2 with 2-run determinism (`docs/36` §4) and the A/B harness +verdict `above_lenient_only` (`docs/37` §4.1). **Cost:** substantial but +front-loaded; it is the reason the honest `blocked_on_device_model` labels +exist. **Bottleneck [I]:** self-verification consumes the sessions that the +live workflow should be producing, and there is no separate "live-question" +session tag (§2.3). + +--- + +## §4 Time-sink analysis: top 10 recurring costs, ranked + +Ranking is by recorded project minutes lost × frequency × recoverability. +Each item names the evidence and the tool-side lever (if any). + +| # | Time sink | Recorded evidence | Why it costs | Tool-side lever | +|---|---|---|---|---| +| 1 | **Supervised hardware cycle as the only oracle** | ~5–6 min/cycle (`docs/25` §6); 10+ cycles ≈ 5 h 10 m for the 0x59 question; 8 cycles ≈ 1 h 41 m for the ladder; 10 NOCAT cycles; P6a–c ≈ 6 h | every hypothesis = one full boot + fail-safe discipline; cross-question batching is manual | finish device model (S1) so hypotheses resolve off-hardware | +| 2 | **Server wedge → reboot loop** | 22 reboot-request lines in the main log since 2026-10-01 (counted); 8 host-freeze mentions; 3 wedge-reboots in one afternoon `[log PART 92/Adds.13–24]`; a user-paused reboot at 10:25 2026-10-02 | fused-part WPR2 cannot be cleared; a reboot kills all other agents' sessions and breaks continuity | none in-tool today; a recoverable wedge or snapshot-resume would collapse this | +| 3 | **Guest-side observability gaps** | ~5 days for the 20-counter wall; ptrace broken in guest; custom samplers per question `[log PARTs 29–64]` | host emulation state and guest expectations are only visible through ad-hoc probes | pair each guest wall with a firmware-side session instead of a new sampler | +| 4 | **Opaque server-internal failures** | 0x59 invisible until the log ring was mapped (id bug); promote `0x40`/CTXNOTVALID; 730+ lines of static RE for one root cause | first error hidden behind generic statuses and unmapped rings | hooks/watchpoints on the rehosted failure path; session compare | +| 5 | **Work loss / tree hygiene** | PART 44/45 fixes lost from the tree, PART 47 re-root-cause `[log PART 47]`; modes 4–22 uncommitted at the campaign's end; progress analysis notes repeated "highest-loss-risk" moments | long uncommitted experiments die with a reboot/host reset | commit-per-mode discipline; sessions preserve the evidence even when code is lost | +| 6 | **Documentation sprawl and contradiction** | main log ~15k lines + progress analysis ~4.8k + tool ledger ~1k + state page ~0.9k; ≥8 contradictory live status claims found (§5) | every agent re-derives state; wrong assumptions send work down dead ends | single generated status/index; archive superseded logs | +| 7 | **Cross-run state contamination** | stale shared guest-RAM image cost 7 cycles `[log PART 93/Add.29]`; stale-PTE data-check residual (`STATE-CURRENT.md:195-205`); evidence dirs reused | runs are not hermetic; a phantom wall looks like a firmware bug | pre-run state hash + hermetic temp state; session records inputs | +| 8 | **Manual evidence assembly/comparison** | per-cycle multi-MB dirs; s10 shows the same contract in 4.92 s on the analog but is device-model-blocked; M-tables hand-filled | every claim re-narrated; comparisons by eye | one cycle → session → `compare` pipeline (rmharness pattern, extended read-only) | +| 9 | **Long-run guest stalls and timeouts** | agent/SSH timeouts from a trace reader flooding the ring; UVM spins; one no-watcher guest kernel crash (`STATE-CURRENT.md:577-582`) | half-finished runs and VM restarts | timeout budgets in the harness; no-watcher runs marked unsafe | +| 10 | **Planning churn** | owner directives U24→U31–U45→U46–U58 within 24 h (program ledger §13–§14), superseding §5 phases twice | plans and docs rewritten; in-flight work re-scoped | one live plan with an append-only directive ledger, not a rewritten one | + +**[I]** Rank 1 is the root cause; ranks 3, 4, 6, 8 are amplifiers. If only one +thing improves, it must be the number of supervised cycles per question — by +resolving hypotheses off-hardware (rank 1) or by extracting more per cycle +(rank 8). + +--- + +## §5 Design-assumption audit: stale or contradicted claims + +The tool's design and status text contains assumptions that its own newer work +has invalidated. Each row names the document section, the stale claim, the +contradicting evidence, and the correction. These are documentation defects, +not fidelity claims: the underlying work is honest; the text lagged it. + +| # | Stale claim (exact location) | Contradicting evidence | Correction | +|---|---|---|---| +| 1 | "**No TU104 device model**: no boot beyond the first unmapped access" — `docs/06` §8 gap 1; "does not boot the real GSP image to its message loop yet (**no TU104 device model**)" — agent brief, field-guide status text (2026-10-02); "Native M2+ device model — roadmap only" — `docs/00` status table line 56 | `sim/devices/`, `sim/profiles/`, `sim/gsp_boot.py` exist (untracked, 2026-10-03); B1: "**0 unknown MMIO, 0 unhandled in windows**" and B2 reaches the relocation copy; sessions `gsp-boot-b1/b2` (`docs/36` §3–§4) | say: "a partial GSP device model exists (B1/B2); the image does not yet reach the message loop; B3/B4 are the targets" | +| 2 | "L2 — **not modelled** (do not claim)" — `docs/06` §2; "It is not: a time, interrupt or device model (L2/L3 absent)" — `docs/26` §1; "L3 absent" — `docs/16` §3.1 table | `docs/32` models per-instruction cycles (labelled modeled, not calibrated); `docs/35` §1.2 states the standalone simulator "targets L0–L2 … with L2 explicitly labelled *modeled*"; `docs/36` models device windows, IRQ aggregation and PTIMER (partial L3) | the canonical accuracy doc should add a "modelled, uncalibrated" state between L2 absent and L2 calibrated, and all status text should use it | +| 3 | Documentation indexes stop at **docs 29**: README map lines 75–109, `docs/00` map, agent field-guide index/table | docs 30–38 exist and are untracked/in development (git status 2026-10-03): GPGPU-Sim integration, Turing/Ampere/Blackwell dossiers, cycle model, simulator design, device model, sim-QEMU line, GSP-Em API | add 30–38 to every map with an "in development" tag; the new docs are currently discoverable only by directory listing | +| 4 | "**24-tool** JSON-RPC surface" — agent brief, field-guide MCP rows; F1 "24 tools" — `future-eng-ops` F1 | `docs/20` §5 lists **31** tools and §9 says 55 tests (the doc itself is modified in the working tree) | pick one number from the code (`tools/list`) and generate it into all status text | +| 5 | Test-count claims: **194** (`docs/00` line 47; agent brief 194-test references; `eng-ops` register; project doc §6), **155** at audit (`docs/27` §0), **215** in the simulator ledger §13.3; 272 `def test_` statically countable 2026-10-03 | three different live claims plus a fourth countable value; no source states which suites were skipped in which environment | one generated badge from a recorded `make test` output; claims must name the environment (QEMU present?) | +| 6 | "Not implemented: device model (R1), custom simulator (R2), **MCP plugin (R3)** — all roadmap" — `open-iov/docs/05-operations/03` §5 line 97 | `docs/00` line 54: MCP "working (20)"; device model in development (`docs/36`); MCP 54 tests | project page: MCP implemented; device model in development; only custom simulator M2+ remains roadmap | +| 7 | Roadmap gates: "Items here are **not** v0.1. Do not start them ad hoc; each has a gate" and R1 "starts only when a portability task or an accuracy measurement requires it" — `docs/08` lines 5–8, 31–37; `docs/16` lines 2, 12–15 "nothing here exists yet" | owner-directed S1/B1/B2 device-model work and the Phase S program started 2026-10-03 (private program ledger §13–§14; `docs/35`, `docs/36`) | mark R1/R3/R4 started/superseded with the directive citation; keep the gate text for M2+ | +| 8 | R4 trace "**roadmap** … available once the QEMU plugin headers are vendored" — `docs/08` R4; MCP R3 framed as original design sketch even while "implemented" | audit: trace plugin verified live (8M events); native M1.5 trace implemented (`docs/16` §1.1a) | R4: implemented in both forms; session folding remains the only roadmap piece | +| 9 | `STATE-CURRENT.md` self-contradiction: line 160 "(2026-10-02 08:36, **in development**)" vs lines 841–844 "in-development notices were removed 2026-10-02 after … 194-test suite" | same file, same day, two statuses | the dated frontier bullets should be labelled historical and superseded inline | +| 10 | Value-demo numbers: "`run_all` aggregate (**~18.4 s total**)" — `docs/25` §3; "value demos 10/10 in **22.5 s**" — tool ledger wave-8 final; local `artifacts/results.json` records only **3 demos, 1.72 s** | the artifact is gitignored and currently a partial `--only s01,s04,s05` run; doc says "do not quote a stale number without re-running it" (`docs/25` §10) | regenerate before quoting; make `run_all.py` record the selection list and date into the doc via a generator | +| 11 | Private tool ledger header declares v0.1 acceptance frozen and "MCP is roadmap-only / explicitly not v0.1 … no custom simulator", while waves 2–8 and the simulator program superseded it | the ledger's own change log (waves 2–8) and `docs/35` §13 | treat the ledger as historical; point readers at README + docs 00/27 instead | +| 12 | Design assumption: "QEMU is just the first execution backend" and "the whole stack scriptable" — `docs/00`; "the only production backend is QemuTarget + RspClient" — `docs/16` §1 | M1 native backend is a peer with a differential test; device model only exists natively, not in QEMU | state that native is the only backend with device semantics today; QEMU remains the only one able to host the real image meaningfully? (Neither boots it; QEMU gives the first fault) | +| 13 | Design assumption in `docs/25` §2.1: projections become real "once H2.2 boots the image" | the program replaced H2.2-first sequencing with Phase S (standalone simulator) and Plan B; H2.2 is a legacy-handoff concept not mapped to the public docs 30–38 | map legacy H-numbers to docs 30–38 in one migration table | + +**[I]** The pattern is consistent: docs 00–29 describe the audited v0.1 +surface correctly, but the 2026-10-03 simulator wave (docs 30–38, untracked +`sim/devices`, `gpgpu-sim/`, new harnesses) has not been propagated into the +status claims. The cheapest durable fix is a generated status block plus a +claim scanner (§6, §8-U3). + +--- + +## §6 Documentation efficiency + +### 6.1 What is duplicated + +1. **Documentation maps (5 copies).** README §Documentation map (`README.md` + lines 75–109), `docs/00` §Document map, the agent field-guide 22-row + discoverability index, the project user guide §10, and the project master + log's "when to use it" table. All five enumerate the same files; only some + include docs 28/29, none include 30–38. **[E]** +2. **Status blocks (≈20 copies).** Nearly every doc opens with the same + "Status: implemented, documented and tested (audit docs/27 …; value demos + docs/25 …)" line. A status change requires ~20 edits; the test-count drift + (§5 #5) proves the mechanism fails. +3. **The "it is / it is not" framing** appears verbatim in `docs/26` §1, the + project user guide §1 and (paraphrased) the project page §2. +4. **Gating text** for RM 610 / no-hardware appears in README, docs 06/11/12/ + 14/15/19/20/21/22/23/24/25/28/29/35/36/38 and the project docs. Honest, but + repeated in full rather than linked. + +### 6.2 What is stale + +The 13 rows of §5. The highest-impact three: device model existence (#1), +L2/L3 fidelity wording (#2), and maps/status stopping at 29 (#3). + +### 6.3 What is unlinked + +| Item | Where it should be linked | Why | +|---|---|---| +| docs 30–38 | README map, `docs/00` map, agent field-guide index, project page §5 | currently invisible to a cold reader | +| `SIMULATOR-WORK-BOUNDARY.txt` | README layout + docs/35 §6 | the scope contract for the current wave | +| `gpgpu-sim/` overlay (`ADMISSION.md`, `BUILD-PORT.md`) | docs/30 and README | the only place the pinned methodology is recorded | +| `harness/qemu_line/`, `harness/re_demo*` artifacts | docs/37 (it references them) | evidence exists but is outside any rendered report someone will find | + +### 6.4 What is too verbose + +* **The main session log** (~15k lines, newest-first addenda) — excellent + evidence, poor orientation. It answers "what happened" but not "what is + true now". +* **The agent field guide** — a 22-row index plus a 12-row matrix plus F–K + examples, duplicating docs/00 and the user guide. +* **The private simulator handoff** — §4/§5 (the original plan) are superseded + by §13/§14 in the same file; readers must internalise two plans and a + supersession order. **[E]** +* **The historical tool-program ledger** — a wave-by-wave journal whose + header claims to be "the source of truth for what is being built"; now + history. **[E]** + +### 6.5 Concrete trim/re-link plan (cheap, ordered) + +| Step | Action | Files touched | Measurable effect | +|---|---|---|---| +| T1 | Generate `docs/STATUS.md` from the test runner output + `git describe` + harness JSON (`qemu_line`, `re_demo`, `results.json`); replace the ≈20 status blocks with one link and a date | tool docs (mechanical) | status edits: 20 → 1; no stale count survives a test run | +| T2 | Add a claim scanner to the link checker: flag numeric claims (`\d+-test`, `\d+-tool`, `L2 not modelled`, `roadmap only`) whose source value differs from the generated status | `scripts/check-docs-links.py` | contradictions detectable in CI | +| T3 | Add docs 30–38 (and the boundary file) to README, docs/00 map, agent field-guide index, project page; tag each "in development (Phase S)" | 4 files | cold-start discovery | +| T4 | Mark superseded sections in the private simulator handoff (§5 superseded by §13; §4 phases by §14) and the tool ledger (historical header); keep them as evidence | 2 private files | halves the reading needed to find the live plan | +| T5 | Collapse the four doc maps into README as the single map; other maps link to it | README, docs/00, user guide, agent brief | one map to maintain | +| T6 | Add a retention rule for `dumps/`: keep named/live-question sessions, prune `unittest-*` older than N days in CI or a script | `.gitignore` already excludes; add script | 107 MB today, >90 % runner noise; keeps real evidence findable | +| T7 | Replace hand-assembled M-tables with a generator over session pairs (the `compare` output + M1–M6 skeleton already exist) | harness | evidence pack becomes machine-produced | +| T8 | One "where to start" flow: README → user guide → STATUS; remove the duplicate tables | maps | orientation time down; this review's §5 defects become non-recurring | + +--- + +## §7 First-principles re-derivation: what actually minimises time + +For each dominant problem in §4, the minimum-time approach, including methods +not currently used. + +### 7.1 Problem: hypotheses are evaluated at 5–6 minutes per experiment + +**Current method:** each candidate fix/hypothesis is compiled into the module +and paid for with a supervised hardware cycle; failures can wedge the server +and cost a reboot. + +**First principles:** the marginal cost of an experiment should be the cost of +*evaluating the hypothesis*, not the cost of *rebooting the device*. +Two independent moves: + +1. **Move evaluation off-hardware against real firmware.** The only currently + viable route is the S1 device model + rehosted execution (docs/35 §3.3). + Until B4 exists, the tool cannot answer a live firmware question at all — + every session in `dumps/` is either self-verification or a first-fault + probe. +2. **Until then, batch per cycle.** A single boot can be instrumented to + answer many questions (the module already exposes debugfs state, counters + and ladders). The recorded campaigns did one hypothesis per cycle even when + the per-cycle data would have covered several (`docs/25` §5 s01: ten boots + for ten candidate fixes). A "one boot, N recorded observations" harness is + cheaper than N boots and needs no new fidelity. + +**Not currently used:** *surgical rehost* — compiling and running only the +failing function region with recorded inputs, instead of booting the whole +image. The initial failure is localized (a commit helper, a parser), and +rehosting a function does not require the full device model — only its +observed input/output state. This is the shortest path from "730 lines of +static RE" to "watched it decide". + +### 7.2 Problem: the server wedge makes failures catastrophic + +**Current method:** fail-safe holds the device; recovery is a fuse-limited +teardown or an approved host reboot. + +**First principles:** a failure should stop the experiment, not the host. The +tool-side analogue is *deterministic replay from a snapshot* (moments, F7): +experiment on a state image, never on the only copy. That is gated on M2/M3 +today, but a weaker version is available now: record the full stimulus and +state hash so a wedge cannot lose the evidence and the next cycle can +reproduce the exact pre-failure state by construction (the session already +carries snapshots; the missing piece is the capture path for the physical +cycle). + +### 7.3 Problem: opacity (first error invisible) + +**Current method:** map the journal ring, read raw host memory, static review. + +**First principles:** the first error is a data question. Watchpoints on the +reply/state object plus before/after diff answer it directly — the same shape +as value demos s01/s05, which are measured in ~1 s on the analog. The tool +has these primitives; what it lacks is a running firmware to attach them to. +Again: the device model is the unblocker, and *surgical rehost* can provide it +locally for one failure path. + +### 7.4 Problem: documentation and status re-derivation + +**First principles:** state should be produced by the system that changes it, +not retyped by hand. Test results, session manifests, harness JSON and git +describe already exist; a generated status page costs one small script and +removes an entire class of stale claims (§5). The session format is already +the evidence contract; the status page should consume it. + +### 7.5 Problem: phantom walls from state contamination + +**First principles:** make runs hermetic and make inputs visible. A pre-run +state manifest (image hashes, shared-RAM file identity/hash, mdev/module +versions, boot id) recorded in the session would have made the stale +shared-memory image immediately obvious instead of seven cycles later +`[log PART 93/Add.29]`. This is cheap, local and applies to both the physical +and simulator paths. + +### 7.6 What not to re-derive + +Do not rebuild what already works: QEMU execution, the session format, hooks/ +watchpoints, the Ghidra join, the injection SDK. The review found no +evidence that reimplementing these would pay; the evidence points entirely at +evaluation cost and evidence plumbing. + +--- + +## §8 Ranked pragmatic upgrade list + +Ranking rule: expected reduction of §4 rank-1/2/8 costs per engineering week, +with the eng-ops admission fields (`open-iov/docs/eng-ops.md` §2). Thresholds +are stated for the record and must be frozen before the first measurement run. + +### U1 (top) — Land the S1 device model to B4 (message loop) on the real image + +* **Scenario (dynamic-RE loop):** "what does the real 570.144 GSP firmware do + between entry and its message loop, deterministically, without hardware?" +* **Measurable outcome:** boot-ladder rung reached, session hash, device state + hash, unknown-MMIO count, first host message; `sim_qemu_line` verdict field. +* **Baseline (recorded):** QEMU `probe` first fault at fw `0x4009014` + (`docs/06` §3) plus static reading (floor estimate 600 s in `re_demo.py`); + every real-firmware value demo marked `blocked_on_device_model`. +* **Threshold:** reach B3 (first CSB/TRANSCOM poll with read-back) and B4 + (message loop) twice with byte-identical key fields; one scripted host + message served end-to-end with a recorded reply (B5); `device_model_used = + true` and `above_qemu_line = true` in `harness/sim_qemu_line.py`; N=3 runs, + one host. +* **Failure signal:** two consecutive image/profile versions require + unabsorbed code changes (profile seam failed), or replay key fields differ + after three attempts (scheduler/determinism failed); or lenient MMIO is + still load-bearing in a rung claimed as modelled. +* **Why first:** every measured value projection in `docs/25` and every + blocked question in §4 is gated on this one capability. + +### U2 — Prove the tool on the live wall: surgical rehost of the GR +engine-context promote/commit path (and the UVM large-push parser regression) + +* **Scenario (dynamic-RE):** the project's current product wall is a + server-internal commit failure; answer its open hypotheses (the V6 + soft-`INVALID_STATE` site; the V5 commit assert's *first* failure) from + rehosted execution with hooks/watchpoints, not from another 730-line static + read or another reboot. +* **Measurable outcome:** questions resolved per session; supervised cycles + and reboot requests avoided; recorded session pair per hypothesis. +* **Baseline (recorded):** modes 4–22 across 2026-10-02 (a full day), three + wedge-reboot cycles, one accepted root cause after 730+ lines of static RE + `[log PART 92/Adds.13–25; STATE-CURRENT.md:98-131]`. +* **Threshold:** ≥2 open hypotheses answered with A/B sessions and **zero** + hardware cycles; the answer agrees with at least one live/session artefact; + total effort ≤2 engineering days on the rehost path. N=1 path, 2 sessions. +* **Failure signal:** after two bounded attempts the failing code cannot be + driven (required device semantics absent) or rehosted outcomes disagree with + live evidence; then stop, record the gap, and fall back to static+physical. +* **Why:** it converts the largest single-day cost on record into a + device-free artifact and produces the first non-probe real-firmware sessions, + which is exactly the missing usage in §2.3. + +### U3 — One-command physical-cycle evidence capture → session → `compare` + +* **Scenario (correctness/evidence loop):** every supervised cycle leaves a + machine-readable session and an automatic delta against the previous cycle. +* **Measurable outcome:** human minutes per cycle evidence pack; M1–M6 + skeleton rows auto-filled; unexpected-delta list. +* **Baseline (recorded):** manual multi-MB per-cycle directories, rings + decoded by hand, M-tables filled in prose; s10 takes 4.92 s on the analog + (`docs/25` §6). +* **Threshold:** one cycle's read-only captures (module state, counters, + ring/journal excerpts, pstore status) become a session + `compare` report in + ≤2 wall-clock minutes, N=3 cycles; a field change between cycles is + attributed or listed as unexplained. +* **Failure signal:** the capture misses the decisive field twice (tool does + not see what the failure needs) or deltas are non-reproducible; retire to a + checklist and keep sessions for the simulator only. +* **Why:** it attacks rank-1/rank-8 cost directly without waiting for the + device model, is read-only by construction, and reuses `rmharness`'s proven + recording pattern (`docs/17`). + +### Runners-up (below the top 3, kept for the next review) + +* Get one agent to actually use the MCP server end-to-end on a release-triage + rehearsal (F1 threshold) or retire the claim of agent-driven monitoring. +* Extend value demos to the rehosted path (F6: ≥6/10 on rehosted code or + stop calling them value validation). +* Session retention/prune + a session index (`dumps` is >90 % test noise). +* Fold `port_driver_versions` into `ci_release` rather than maintaining a + third orchestrator (`future-eng-ops` F2 adjudication). +* Calibrate cycles (C1–C6) only when a supervised capture window is approved; + until then keep the modeled label and claim nothing. + +### Do-not-build list (with the trigger that would reopen each) + +| Do not build | Why | Reopen trigger | +|---|---|---| +| GPGPU-Sim board model (GMMU/FBIF/DMA/vidmem/VBIOS), Phase G | Phase S is the owner-locked scope; designed-only | Phase S pause + a funded board question | +| Ampere/Blackwell architecture models | parameter dossiers exist; no consumer | a device port actually scheduled | +| Verilog/FPGA soft processor (F12/F13) | multi-quarter, no funded timing claim | a funded cycle-accuracy claim | +| SIMD/kernel execution on device (F11) | unbounded research; hardware executes kernels today | funded thesis/cycle-accuracy use | +| Moments/time-travel before B4 | deferred by the pragmatic scope lock; convenience, not support latency | message loop reached + a real replay need | +| TCG-trace session folding (R4 remainder) | trace already exists; folding is convenience | trace consumers ask for coverage reports | +| Ghidra heat-maps/bidirectional nav (B2–B4) | bridges work; polish | a real navigation complaint | +| MCP port-chain automation (F1) until a release rehearsal runs | no consumer has exercised the chain | first agent-driven release run | +| Packaging/trialing (F4) as a workflow | no reviewer demand yet | a reviewer asks for a replayable bundle | +| A second multi-version orchestrator | duplicates `ci_release` | F2 threshold unmet **without** a bundle precondition | +| RM 610 anything | hard human gate (`docs/23`) | explicit named-human approval | +| SBOM/observation inventory (F15) | legal/review gates; no consumer | SBOM review asks for per-image rows | +| New MCP transports / network exposure | loopback/stdio is the safe surface | never (security rule) | + +--- + +## §9 Frida-like capability gap analysis + +### 9.1 What exists + +**[E]** hooks (enter/leave on RV64 ABI, `Session.hook`), breakpoints (sw/hw), +watchpoints (write/read/access with clear/step/re-arm to match QEMU stop +semantics), memory read/write/hexdump/search/snapshot/diff, schema struct +decode, recorded sessions, `run_until` (run to cursor), `call` (remote +function call with scratch stack and ABI save/restore), injection of +position-independent services (direct and firmware-mailbox paths), exec/access +trace (native and TCG), plugins, MCP, and Ghidra in/out bridges. Source: +`docs/00` §Status, `docs/01` §4, `docs/11`–`13`, `docs/27`. + +### 9.2 What is missing, and whether it matters day-to-day + +| Frida-class capability | In the tool? | Day-to-day matter | Verdict | +|---|---|---|---| +| Backtrace / stack unwind on a hit | no (only `ra`-based leave hooks) | high: "who called this handler" is a daily RE question | **build later** — an `ra`/frame-pointer walk is bounded; pairs with `run_until` | +| Semantic call tracing (Stalker-like call graph) | partial: full exec trace, no call graph/summary | medium-high: call ordering at scale | **extend trace** with a call/extent fold when B4 exists | +| Catchpoints (trap/exception class selection) | partial: traps stop as plain stops; catchpoints planned (S2) | high for error-path RE (fault handling, asserts) | **build now (thin)** — approved in the pragmatic scope table | +| Hart/thread selection | no (single hart) | low today; needed for multi-core libos | defer with H2.4 | +| Interceptor *replace* (patch a function body, not just insert) | partial: SDK services + debugger writes; no atomic replace | medium: fault injection/return-value forcing | keep injection SDK; add a documented return-patch helper only if a scenario needs it | +| Module/symbol enumeration at runtime | n/a (stripped bare-metal; symbol DB from static exports) | n/a | not applicable | +| DWARF type introspection | no (manual schemas) | medium: manual schema per structure is a cost | acceptable; schemas are versioned data by design | +| Memory allocation tracking | no | low for firmware servers | do not build | +| Remote/network attach, ASLR/module-load handling, Java/ObjC bridges, debugger-side allocation hooks | no (deliberately) | none | not applicable; keep out | +| Allocation/protection changes (mprotect-like) | no; MPU is not enforced (`docs/36` §6.8) | medium for fault experiments | defer; documented gap | + +### 9.3 The real gap + +**[I]** The Frida-like feature list is not the constraint. The constraint is +that none of it can yet run against the real firmware to the point where a +live question exists (B4). Every missing feature above is worth less than the +device model. The Frida-like work that *is* worth doing before B4 is small and +already approved: crash/trap catchpoints, hart/core metadata, and memory +region metadata (S2 "thin"); a backtrace would be the next candidate because +it converts the most common live question into one call. + +**[I]** Conversely, several Frida-ish ideas have no firmware use case: +network attach, module enumeration, allocation tracking, OS-level process +injection, Java/ObjC bridges. They should stay out of the roadmap's active +list to keep the feature-count pressure off the critical path. + +--- + +## §10 What to scrap or disable + +Nothing is deleted by this review; these are recorded recommendations for the +next audit (`docs/27`) and the `future-eng-ops` §4 prune/demote review. + +| Item | Action | Evidence / rule | +|---|---|---| +| `Session.symbol_at` (no caller) | **prune at next audit** unless a caller appears | already the first prune candidate (`future-eng-ops` §4) | +| "Value validation" framing of s01–s10 | **demote** to "tooling regression suite" until ≥6/10 class transfer on rehosted code | F6 kill criterion: fewer than four classes transfer | +| `--surface rm` control discovery | keep code, **remove from advertised next steps** until H1.3/H2.2 exposes the table | F5: `built, not used yet`; exits clean | +| `harness/port_driver_versions.py` as a third orchestrator | **fold into `ci_release.py`** unless the F2 threshold is met without a bundle precondition | F2 adjudication note | +| `harness/port_new_device.py` / `package_sim.py` without a consumer | keep as pilots, **do not advertise as admitted**; expire per eng-ops pilot rule | F3/F4 unadmitted | +| Trace session folding / heat-maps / bidirectional nav | remove from "next" lists; keep issues recorded | pragmatic scope lock (§14.2 of the private program ledger) | +| MCP tools that are QEMU-only on the native backend | keep the structured `not_available`, but stop counting them toward "native parity" | `docs/20` §8; parity matrix in `docs/35` §5 | +| `dumps/` growth | add retention/prune for runner sessions; never track | 458 dirs / 107 MB, >90 % `unittest-*` (measured 2026-10-03) | +| GPGPU-Sim overlay further work | freeze until Phase S pause | owner scope lock; overlay build only | +| Historical tool ledgers | mark historical; stop treating as live plan | §5 #11; §6.4 | +| Test-count claims | replace with generated value; delete hand-written numbers | §5 #5 | + +**Disable-now candidates (if a hard choice is needed):** stop advertising +`--surface rm`, port/packaging pilots and trace folding as available +workflows, and stop quoting hand-maintained test/tool counts. Each currently +consumes review attention without a consumer. + +--- + +## §11 Review conclusion (one paragraph) + +**[I]** The tooling is real, honest about fidelity, and already proven on its +own terms, but the evidence shows it has not yet paid for itself on the +project's dominant cost: the supervised hardware cycle. The single +highest-value move is to finish the device model to the message loop so the +hundreds of blocked questions become sessions; the second is to prove the tool +on the current product wall by rehosting the failing path; the third is to +capture every physical cycle as a session so evidence stops being hand-built. +Everything else — more features, more docs, more architectures — should wait +until those three produce recorded, measurable time savings. + +--- + +*Change note: new document; scope: review only; benefit: consolidates the +2026-10-03 tool/simulator/wall evidence into actionable, gated upgrades; +no product, hardware or guest action authorised.* |