risc-v-gsp-rm-debugger: land docs 45-53 + harness/sim tool set with tests

rm_capture (chunked decode), launch_audit, state_seed/state_decode,
valprobe, rehost, contracts+run_contracts, host_traffic, gmmu_walk,
msgq_attach, libos_port, log_decode, server_snapshot; doc index updates
in the tool overview, operations pages and the repo README.

1 files changed, 301 insertions, 0 deletions

diff --git a/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md b/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md
new file mode 100644
index 0000000..24ee3ce
--- /dev/null
+++ b/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md
@@ -0,0 +1,301 @@
+# 50 — Wall workflow and contract registry (in development)
+
+> **Status: in development (2026-10-06, wall-directed wave).** Documents the
+> wall-hell escape workflow (observe → classify → model → falsify → contract),
+> the six wall classes and which tool may be used for each, the capture-request
+> discipline, and the contract registry (`harness/contracts.py`,
+> `harness/run_contracts.py`) that turns every solved wall into a re-runnable
+> regression.
+>
+> Development-only: the registry re-runs offline tooling; it never touches a
+> GPU, kernel module, mdev, guest or network, and it adds no measurement
+> capability of its own. Fidelity stays what each wrapped tool claims (L0/L1;
+> `06-ACCURACY.md`).
+>
+> **See also:** `47-VALIDATION-PREDICATE-PROBE.md` (`valprobe`),
+> `46-SURGICAL-REHOST.md` (`rehost`), `48-GMMU-WALK-AND-LAUNCH-AUDIT.md`
+> (`gmmu_walk`, `launch_audit`), `45-RM-RPC-CAPTURE-AND-REPLAY.md`
+> (`rm_capture`), `19-CONTROL-DISCOVERY.md`, `40-NEW-VERSION-TRIAGE.md`,
+> `25-VALUE-VALIDATION.md`, `27-TOOL-AUDIT.md`, `49-SIMULATION-SURFACE-MAP.md`,
+> the private simulator handoff
+> [`../../../../handoff-simulator.md`](../../../../handoff-simulator.md)
+> §30–33 (read it when available; agents without private access skip it), and
+> `open-iov/docs/eng-ops.md` §2/§6 (admission and automation safety).
+
+## 1. Why this exists: wall-hell has a shape
+
+The GR campaign's dominant cost is **wall-hell**: one blocking problem after
+another, each paying supervised hardware cycles (5–6 min each, plus fail-safe
+discipline) to learn one fact. The toolset now solves several wall classes
+offline and deterministically (`docs/25` §3 records the measured value), but
+knowledge kept only as prose is re-learned by the next agent and re-tested on
+silicon by the next campaign. This document fixes two things:
+
+1. **A wall is classified before it is worked.** Each wall belongs to exactly
+   one of six classes; only the tool that measures that class may answer it.
+   Most expensive wall errors are *domain-crossing inferences* (section 3).
+2. **A wall is not closed until it is a contract.** The answer becomes a
+   declarative, re-runnable check over the existing tool entry point
+   (section 6), so the next firmware bump or tool change is caught by
+   `make test` instead of another cycle.
+
+The registry is a **consumer** of proven tools, not a new probing capability:
+it wraps `valprobe`, `rehost`, `gmmu_walk`, `launch_audit` and `rm_capture`
+entry points as-is (docs/49 §4 lists their proven value).
+
+## 2. Wall taxonomy — six classes
+
+| Class | The question it covers | Measures it DIRECTLY | Must NOT be used to infer it | Artifact a correct answer leaves |
+|---|---|---|---|---|
+| **a. firmware boot/status** | Did the firmware reach rung Bx? Where is it stuck, with what stop reason/status word? | `sim/gsp_boot.py` rungs B0–B3, `harness/triage.py` (identity/probe/native_boot), probe sessions, `greeneye probe` first-fault report | engine memory tools (`gmmu_walk`, `launch_audit`) and predicate probes (`valprobe`) | `greeneye-session/1` with the rung, stop PC/status and first-fault address |
+| **b. RM server behaviour** | What does the server read/write/wait on for this input; which branch/gate decides accept vs refuse; what state does a handler need? | `valprobe` (one firmware predicate branch with crafted state), `rehost` (one whole function; first missing state is the finding), `rm_capture` decode for what the client actually sent | engine-execution claims from hooks/predicate verdicts; `gmmu_walk`/`launch_audit` for server logic | valprobe/rehost report + session: entry, seeded state, stubs, labelled outcome, stop/fault |
+| **c. engine memory contract** | Can the engine reach each leg of this submission (ring, QMD destination, param/program, release slot, semaphore) with the required access, and at which PA? | `launch_audit` (per-leg verdict against the engine's table view), `gmmu_walk` (the leaf mapping behind one VA) | host shadow bookkeeping as proof of engine reachability; server-side hooks; `valprobe` | `greeneye-launch-audit/1` spec+report with per-leg `ok`/`pa_mismatch`/`unmapped`/`access_denied` |
+| **d. engine execution/dispatch** | Did the front-end/engine consume the entry, dispatch the method, write the release/semaphore? | Supervised capture + post-mortem decode (the only authority); `launch_audit` answers only its *address* leg | anything in this tool: no GR/PBDMA model exists by design (`docs/49` §6); page tables cannot prove method dispatch; a silent ring cannot be inferred from an all-`ok` audit | capture with the engine-side consumption evidence (GPFIFO GET/PUT movement, method/QMD execution, release/semaphore state) + post-mortem |
+| **e. guest-driver/protocol interface** | What did the guest driver send (fn/cmd/params), and does a replay/another version differ? | `rm_capture` decode/compare/stimulus (send-side logs), guest diagnostics (`open-iov/tools/guest-diag`) | firmware predicate/branch results as a substitute for the driver's wire traffic; `gsp_boot` rungs | `greeneye-rm-rpc-capture/1` + `greeneye-rm-rpc-compare/1` and a stimulus JSON |
+| **f. version skew/portability** | Did the image/code/behaviour move between versions, and which change explains it? | `harness/triage.py` (full chain), `harness/version_diff.py` (sections + symbols + M1–M6 skeleton), `./risc-v-gsp-rm-debugger.sh probe <image>` first-touch diff | one version's green run as portability proof; a `skipped` step read as a pass | triage report + version diff + one recorded `compare` session pair (self-diff zero) |
+
+Two rules fall out of the table:
+
+* **Class (d) has no offline tool, by design.** Do not attempt to simulate it:
+  the simulator makes no L2/L3 claims (`docs/06`). The offline contribution is
+  the *address leg* (class c) and the *server leg* (class b).
+* **A wall usually names its own class in one sentence.** If two classes seem
+  to fit (e.g. "the launch is silent"), split the question: address leg →
+  (c), method execution → (d), server prewrite → (b). One wall = one class =
+  one decidable question.
+
+## 3. The evidence ladder and the no-cross-domain rule
+
+### 3.1 Observe → classify → model → falsify → contract
+
+| Stage | What happens | Exit test |
+|---|---|---|
+| **1. Observe** | Collect only evidence that can discriminate: one bounded session/probe/capture, named stimulus, recorded artifacts | The observed fact is replayable and has a session/report path |
+| **2. Classify** | Assign exactly one wall class (section 2) and name the metric the question is really asking (M1–M6, `docs/06` §5) | The class's DIRECT tool exists and is reachable offline; if not, record the gap (section 10) |
+| **3. Model** | Run the class's tool with crafted/captured state (predicate branch, rehost, walk, audit, decode) | A result that could have come out the other way (a verdict, not a confirmation) |
+| **4. Falsify** | Try to break the model: inverse variant, mutated spec, second run for determinism, one-bit differential (`docs/14`) | The falsifier fails to overturn the model; the determinism `compare` is empty |
+| **5. Contract** | Pin the decisive run as a registry contract with an explicit expectation | `python3 harness/run_contracts.py --fast` passes and the contract can fail if the tool changes |
+
+The ladder is the OODA loop of `docs/15` made runnable: model is Orient,
+falsify is Decide, contract is Act/regress. Skipping stage 4 is how a
+misread wall survives (the FECS refusal was initially misread as a V6
+limitation until the predicate matrix falsified it; handoff PART 95).
+
+### 3.2 Never infer across domains
+
+The rule is absolute for claims that would change the product:
+
+* **Host bookkeeping cannot prove engine execution.** The engine may read a
+  different PA than the one the post-mortem checks — that is exactly what
+  `launch_audit`'s `pa_mismatch` exists to expose. Only a capture shows what
+  the engine did (class d).
+* **Page tables cannot prove method dispatch.** `gmmu_walk` proves a mapping
+  exists; a silent launch with all legs `ok` means the silence is downstream
+  of translation (front-end/engine/context), not that the method ran
+  (`docs/48` §3).
+* **A predicate verdict cannot prove the whole handler path.** `valprobe` runs
+  one branch with the minimum state; the full handler may early-out earlier
+  or additional gates may exist downstream (`docs/47` §2; `docs/46` §5).
+* **A decoded client record cannot prove what the server replied.**
+  `rm_capture` is send-side; captures currently have no reply sequence
+  (`docs/45` §7).
+* **A demo-surface result is not RM state.** `control_discovery`/`reach` on
+  the demo surface classify demo dispatch; the RM surface waits for H1.3/H2.2
+  (`docs/19` §9).
+* **One version's behaviour is not portability.** A new image requires the
+  class-(f) chain, even when one scenario passes.
+
+Every claim in a handoff/write-up carries its class and its tool so a reviewer
+can see the domain it came from.
+
+## 4. Per-tool honesty table
+
+Wraps the audits in `docs/27` and the limits in each tool doc. "L0/L1" means
+instruction/memory fidelity only; no timing (L2) and no device-environment
+(L3) claim (`docs/06` §2).
+
+| Tool | What it MEASURES | What it does NOT measure | Known failure modes |
+|---|---|---|---|
+| `valprobe` (docs/47) | The branch one firmware predicate takes with crafted register/memory state plus at most one recorded stub; the verdict is a labelled outcome address | Whole-handler behaviour, RM object state, boot state, timing | Wrong entry/state ⇒ `fault` (first missing state) or `unlabelled` outcome; a stub can mask a dependency that would change the branch; crafted state is not what a live caller supplies |
+| `rehost` (docs/46) | One bounded firmware function with crafted args/memory/snapshots; the first missing state is the finding | Service loops, interrupts/scheduling, MMIO semantics, timing | `budget`/timeout usually means entry is mid-function or loops; argument conventions must be caller-supplied; lenient unknown MMIO can mask environment dependence |
+| `gmmu_walk` (docs/48) | The leaf PA/page-size/aperture/access for a VA through a raw table view; exact fault otherwise | Access ordering, execution, timing; video-memory high bits (`pa_complete: false`) | Wrong root/view ⇒ fail-closed faults; 64K dual path is modelled, not live-validated; incomplete view reports `unmapped`, never "unreachable" |
+| `launch_audit` (docs/48) | Per-leg class of a submission's engine-visible memory contract vs host bookkeeping (`ok`/`pa_mismatch`/`unmapped`/`access_denied`) | Whether the engine actually executed/dispatched; anything downstream of the address path | Needs a real capture spec; missing table pages read as `unmapped` (capture error, not engine truth); no fixture ⇒ demo only |
+| `rm_capture` (docs/45) | Decode of real client RPC log tags into records; compare of two captures; replay stimulus preparation | Server replies/event ordering; handles/addresses (omitted from stimuli); anything >192 B payloads until the helper chunks | Unknown tags/truncated payloads (`truncated: true`) are not replayable; send-side only; caller-address symbolisation deferred |
+| `control_discovery` / `reach` (docs/19) | Unknown command-id classification on a surface; a reachability route with bounded timeout and required args | RM object state/behaviour (surface waits for H1.3/H2.2); dead-code proofs | A timeout is "not reached under this stimulus", never unreachability; long handlers may classify by entry hit with empty effects; trap/fault cause mapping is a documented fidelity gap |
+| `triage` / `version_diff` (docs/21, docs/40) | Image identity/sections/symbols/probe/first-touch and M1–M6 diff between two versions | Silicon behaviour; a raw `.fwimage` boots only to its early fault by design; `skipped` is not `pass` | Missing baseline ⇒ `version_diff` step `skipped`; RM 610 refuses without human approval (exit 3); first-touch differences need attribution to profile vs logic before any conclusion |
+
+## 5. Capture-request discipline: one capture, one decidable question
+
+Supervised captures are the scarcest resource; a vague request burns a cycle
+and still leaves ambiguity. The rules:
+
+1. **State the question as one falsifiable sentence.** "Does the engine's PDB
+   resolve the inline-QMD destination VA to the host shadow PA?" — not
+   "capture more launch data".
+2. **Write the acceptance test before asking.** Name the tool and the exact
+   report field that will decide it (e.g. `launch_audit run --spec` leg
+   verdict; `rm_capture compare` row deltas). If no existing consumer parses
+   the data, the capture is not ready.
+3. **List the exact pages/registers, byte-exact.** Prefixes without
+   pretty-printing: PDB root, each level (PDE2/PDE1/PDE0), the leaf table(s),
+   the submission pointers (GPFIFO ring, `SET_INLINE_QMD_ADDRESS_A/B`, QMD
+   param/program/release-slot, report semaphore), and the shadow/chunk pages
+   the engine might write. A well-formed example: the launch-audit request
+   that listed the channel PDB tree + the `rmcall_gr_shadow` chunk DMAs
+   (simulator handoff §32; the private request holds the concrete PA list).
+4. **Include the comparison basis.** The host/shadow bookkeeping for the same
+   VAs, so `pa_mismatch` is decidable; a second consumer's view for the same
+   object if the question is interface alignment.
+5. **One question per capture.** Two questions ⇒ two lists; bundling makes a
+   partial answer unreadable.
+6. **Prefer the existing ingestion format.** Raw pages → `greeneye-launch-
+   audit-spec/1`; RPC logs → the `rm_capture` tag grammar
+   (`harness/rm_capture.py --print-capture-hints`). Do not invent a format
+   that no tool reads.
+7. **On a failed capture, record the fidelity gap, not a guess.** Missing
+   pages or truncated payloads become a new request or an unknowns-ledger row
+   (section 8), never an attribution.
+8. **Gates.** The capture itself is supervised, human-run physical work
+   (`SIMULATOR-WORK-BOUNDARY.txt`); simulator tooling only prepares the spec
+   and reads the artifact afterwards.
+
+Anti-patterns: "capture more logs" (no acceptance test); pretty-printed
+tables instead of raw bytes (breaks byte-exact ingestion); one dump for two
+questions; capturing without the host-side bookkeeping map (nothing to
+classify against); re-capturing because the previous request lacked the
+release slot PA.
+
+## 6. The contract registry
+
+### 6.1 Concept
+
+Every solved wall becomes a **contract**: a declarative record that pins the
+proven tool entry point and the expectation observed when the wall was
+solved. New walls run the registry first - a failure localises the regression
+to one class in seconds; only then is a new probe/capture justified.
+
+| Contract field | Meaning |
+|---|---|
+| `name` | stable id, e.g. `fecs-predicate-matrix` |
+| `wall_class` | one of a–f (section 2) |
+| `doc` | the tool/wall documentation reference |
+| `entry` | the exact existing entry point wrapped, e.g. `harness.valprobe:run_spec(fecs_spec())` |
+| `expect` | the pass condition in words (falsifiable) |
+| `speed` | `fast` (runs in `make test`) or `slow` (only `--all`/`--name`) |
+| `requires` | input paths; absent ⇒ `skip`, never `pass` |
+
+Rules for adding one: wrap an **existing** entry point (no new measurement
+logic); the expectation must be able to fail; keep fast contracts small
+(seconds total); one wall class per contract; link the doc; and if the wall
+has a public write-up, link it too. A contract without a falsifier is prose.
+
+### 6.2 Commands
+
+```bash
+python3 harness/run_contracts.py                     # fast contracts (make test path)
+python3 harness/run_contracts.py --all               # fast + slow
+python3 harness/run_contracts.py --json dumps/contracts.json
+python3 harness/run_contracts.py --fast --name gmmu-golden-vector   # single contract
+python3 -m unittest tests.test_contracts -v          # fast contracts as a test
+```
+
+The runner prints pass/fail per contract and per-wall-class coverage (classes
+with zero contracts are shown, so gaps stay visible); exit status is nonzero
+if any selected contract fails; a `skip` is a missing prerequisite, not a
+pass. Schema: `greeneye-contract-run/1`.
+
+### 6.3 Shipped contracts (2026-10-06)
+
+| Contract | Class | Wraps | Expectation | Speed |
+|---|---|---|---|---|
+| `fecs-predicate-matrix` | b | `harness.valprobe.run_spec(fecs_spec())` | 15/15 PROMOTE_CTX predicate variants labelled, 0 failed | fast |
+| `fecs-predicate-determinism` | b | `harness.valprobe.run_demo()` | matrix twice, identical verdicts | slow |
+| `rehost-demo` | b | `harness.rehost.run_demo()` | demo function returns the expected sum twice | fast |
+| `gmmu-golden-vector` | c | `sim.gmmu_walk.run_demo()` | 5/5 captured VAs → logged DMA addresses | fast |
+| `launch-audit-captured-contract` | c | `harness.launch_audit.run_demo()` | captured contract all `ok`; QMD hypotheses yield `pa_mismatch` + `unmapped` | fast |
+| `rm-capture-decode-demo` | e | `harness.rm_capture.run_demo()` | synthetic capture decodes/compares/yields stimulus | fast |
+
+Coverage gaps, stated honestly: class **a** and **f** have tools (`triage`,
+`version_diff`, boot rungs) but no contract yet because a meaningful fast
+expectation needs a pinned baseline pair (that is the queued golden-matrix
+build, handoff §30); class **d** has no offline contract by design (section
+2). Classes b/c/e are covered by the five fast contracts; `make test` runs
+them.
+
+## 7. Admission note (eng-ops §2)
+
+The registry/runner is admitted per `open-iov/docs/eng-ops.md` §2:
+
+| Field | Value |
+|---|---|
+| **Scenario** | Wall-directed loop: after a wall is solved offline, the answer must survive as a regression; before a new wall is worked, the known walls are re-checked; and every firmware/tool change must show whether a solved wall moved. |
+| **Measurable outcome** | `greeneye-contract-run/1` summary: pass/fail/skip per contract + per-wall-class coverage; exit code and JSON artifact; `tests/test_contracts.py` in the suite. |
+| **Baseline** | Re-running the same demos by hand per wall, or not re-running them at all; per-tool costs are recorded in `docs/25` §3 (0.2–6.0 s per demo path vs 5–6 min per supervised cycle). |
+| **Evaluation threshold** | Keep if: (1) every fast contract passes in ≤ 10 s total in `make test`; (2) each newly solved wall gets exactly one contract within the same session; (3) a deliberate tool change can make a contract fail (falsifier proven by a synthetic failing case in `tests/test_contracts.py`). |
+| **Failure signal** | A contract that cannot pin a concrete expectation; two contracts measuring the same thing; a fast contract exceeding the time budget; contracts added faster than walls are solved; or a wall re-opened with the registry green (meaning the contract did not capture the decisive run). On the first signal, prune or rewrite the contract before adding new ones. |
+
+## 8. Unknowns ledger (practice)
+
+Open questions that no current tool answers live in an **unknowns ledger**:
+one row per question, with a discriminating experiment. The ledger itself is
+maintained by the campaign **outside this tool**; this document describes
+only the practice. Row shape:
+
+| Field | Meaning |
+|---|---|
+| Question | one falsifiable sentence |
+| Discriminating experiment | the run that decides it, with the class (a–f) |
+| Tool / capture | the registry entry, probe spec, or supervised capture request (section 5) |
+| Status | `open` → `probe-ready` (spec exists) → `answered` → `contract` (registry) or `closed` (no action) |
+| Evidence | session/report path once answered |
+
+Rules: a row moves to `answered` only with an artifact, and to `contract`
+when the decisive run can be pinned (section 6); if the experiment needs a
+class-(d) fact, the row says so and waits for a supervised capture rather
+than a simulator guess; when a tool that already exists answers the row, a
+new tool is not built (`eng-ops.md` §2 core rule).
+
+## 9. Worked examples
+
+* **FECS promote wall (2026-10-06, class b).** Live refusals (`0x1f`) looked
+  like a V6 channel limitation. Observe: refusal reproduced in logs.
+  Classify: firmware predicate (b). Model: `valprobe` crafted variants —
+  refuse iff the shared slot is non-NULL, regardless of aperture/size.
+  Falsify: `slot NULL` and sysmem-attr variants accept; a second matrix run
+  is identical. Contract: `fecs-predicate-matrix` +
+  `fecs-predicate-determinism`. Result: the refusal is the shared one-shot
+  rule, triggered by the campaign's own earlier promote; FECS is exonerated
+  (handoff PART 95; `docs/47` §3).
+* **Launch silence (`qmd_rel=5`, class c → d).** Classify: the address leg is
+  (c), method execution is (d). Model: `launch_audit` over the captured
+  channel fixture partitions `ok`/`pa_mismatch`/`unmapped`; the downstream
+  question is not simulatable. Falsify: pending the real
+  `greeneye-launch-audit-spec/1` raw-page capture (section 5). Contract:
+  `launch-audit-captured-contract` covers the fixture today; the real-spec
+  validation is handoff §31 item V1.
+* **Construct/state walls (`0x40`/`0x52a8b04`, class b).** `rehost` runs the
+  function and stops at the first missing state — the finding, not a defect.
+  The missing piece is the RM-object state-seeding fixture (handoff §30 ask
+  #1); until it exists those walls have no contract, and the registry run
+  before starting them shows exactly that gap (class b has only demo/known
+  inputs).
+* **Engine dispatch (class d).** No offline contract exists by design. The
+  registry's coverage line for class d is the reminder that the answer must
+  come from a supervised capture + post-mortem, with `launch_audit` used
+  only for its address leg.
+
+## 10. Limits, status and open items
+
+* **Status (2026-10-06).** Registry, runner and tests implemented; 6
+  contracts (5 fast, 1 slow) at `greeneye-contract-run/1`; fast run ≈ 2 s;
+  `tests/test_contracts.py` green. Coverage: b (3), c (2), e (1); a/f/d
+  open as described in §6.3.
+* The registry measures nothing new: if a wrapped tool's own demo/test is
+  wrong, the contract inherits that error. The `docs/25` demos and the
+  per-tool test modules remain the deeper checks.
+* No timing, security or device-environment claim (L0/L1 only); injected
+  code, modified images and dumps stay development-only.
+* Open items: pin the class-a/f contracts once a stable baseline pair exists
+  (golden matrices, handoff §30/§31); re-validate the launch-audit contract
+  against the first real `greeneye-launch-audit-spec/1`; add a contract only
+  when the wall it closes has an artifact.