debug: green-eye wave 3 - RM harness, local simulator, device map, MCP plugin, value-validation demos, CI pipeline, toolchain docs; add host RM tools rmtool/rmtop

1 files changed, 445 insertions, 0 deletions

diff --git a/debug/risc-v-gsp-rm-vm/docs/19-CONTROL-DISCOVERY.md b/debug/risc-v-gsp-rm-vm/docs/19-CONTROL-DISCOVERY.md
new file mode 100644
index 0000000..0f9afbe
--- /dev/null
+++ b/debug/risc-v-gsp-rm-vm/docs/19-CONTROL-DISCOVERY.md
@@ -0,0 +1,445 @@
+# 19 — Control discovery and hot-path reachability
+
+> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status and
+> [`GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md) H4 for the program.
+
+Two harness tools close the loop between "a new firmware image does something
+we do not model" and "here is the code path, its trigger and its effects":
+
+| Tool | What it answers |
+|---|---|
+| `harness/control_discovery.py` | Which command IDs exist, which code path each one takes, and what it changes. New/undiscovered controls fall out of the sweep instead of waiting for documentation. |
+| `harness/reach.py` | Given one decompiled symbol/address: is it reachable, what triggers it, what arguments does it need, what happens. |
+
+Both are simulator-only, bounded, evidence-producing, and depend only on the
+public `Session`/hook/watch primitives (stdlib-only Python). They are the
+practical start of the "Sandsifter for RM" idea in
+[`GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md) H4 and reuse the contracts
+in [06-ACCURACY](06-ACCURACY.md) and [14-ADVERSARIAL-INTEROP-GUIDE](14-ADVERSARIAL-INTEROP-GUIDE.md).
+
+---
+
+## 1. The problem
+
+The GSP's RM is a message/method system: services own objects and answer
+controls dispatched by ID. NVIDIA ships new images on a hidden schedule,
+stripped and signed, and new services, messages and controls appear between
+versions. Static diffing (docs/18 portability matrix, docs/14 section 4.1)
+tells us the code changed; it does not tell us which input activates the new
+path, what the path needs, or what it changes. A sweep of the input surface
+plus dynamic hooks answers exactly those questions, and it is safe to run in a
+simulator where the firmware cannot be damaged.
+
+## 2. The discovery model
+
+### 2.1 Sweep spaces
+
+A "call surface" is a finite input space with an observable effect:
+
+| Surface (today / future) | Space | Probe action |
+|---|---|---|
+| demo mailbox (`--surface demo`) | command IDs `1..--max-id`, with argument patterns | write `gsp_ctrl.arg0..2`, then `gsp_ctrl.mailbox` |
+| RM control/method entry (`--surface rm`, H1.3) | control IDs, classes and method opcodes | call the RM method entry with `(control, a0, a1, a2)` |
+| future CSB/TRANSCOM message functions | message function IDs | write the message buffer and ring the doorbell |
+
+ID 0 is the demo's idle mailbox sentinel (the polling loop ignores it), so the
+sweep starts at 1; surface classes with a meaningful 0 can start there instead.
+
+The same machinery applies to any of them because the sweep is described by a
+:class:`Surface` subclass: a probe function, an effect observer, and an
+entry-point table.
+
+### 2.2 Deterministic argument patterns
+
+Every ID is probed with `--patterns` (default all three):
+
+* `zero` — `(0, 0, 0)` (the "no arguments" baseline);
+* `sentinel` — three distinct pseudo-random 64-bit values from a stream keyed
+  by `(seed, id)`, so an ID that misuses an argument is caught;
+* `boundary` — `(2^64-1, 1, 2^63-1)`, the usual off-by-one/size boundaries.
+
+Patterns are deterministic: the same `--seed`/`--max-id` replays the same
+findings byte-for-byte (verified by `tests/test_discovery.py`).
+
+### 2.3 Entry-point modelling
+
+Before probing, the tool reads the surface's declared entry points:
+
+* the dispatcher symbol (`rpc_dispatch` on the demo) is hooked for every
+  probe, capturing the command and the raw `arg0..arg2` at dispatch entry;
+* every non-zero handler-table slot (`gsp_ctrl.handlers[0..7]`) is hooked so a
+  hit can be attributed to a function even when the command is not one of the
+  documented switch cases.
+
+On real RM the same role is played by the RM-core dispatch table / object
+class table: hook the dispatcher, then resolve the handler the simulator
+routes to.
+
+### 2.4 Effect capture
+
+Each probe observes:
+
+* control-block fields before/after (`state`, `rpc_count`, `last_rpc`,
+  `result`, `buf` hash) plus write watchpoints on the scalar fields, which
+  also record the *store PC* (`effects.watch`);
+* the new QEMU exception window (`-d guest_errors,unimp,int`), parsed with
+  `greeneye.analyze.parse_qemu_faults`, so traps (`cause 3`) and faults are
+  distinguished from "no progress";
+* stop signals as a backend-independent fallback (a native backend has no
+  QEMU log: SIGTRAP maps to trap, SIGSEGV/SIGBUS/SIGILL to fault; budget
+  SIGINTs are deliberately excluded from findings).
+
+### 2.5 Classification
+
+One probe yields one outcome, in priority order:
+
+| Outcome | Meaning |
+|---|---|
+| `trap` | breakpoint exception (`ebreak`) or trap loop below `0x100` |
+| `fault` | load/store/fetch/illegal exception |
+| `method-return` | an RM-style call entry returned a value |
+| `no-progress` | the mailbox was never consumed in the budget |
+| `injected-slot` | command routed through the injected-service handler table |
+| `known-handler` | a declared handler ran |
+| `default-path` | the dispatcher catch-all ran (no dedicated handler) |
+| `settle-failed` / `error` | the pre-probe state or the probe itself failed |
+
+`default-path`, `injected-slot` and `trap/fault` are the interesting classes:
+they are where new behaviour, injection wiring and input bugs live.
+
+---
+
+## 3. `control_discovery.py` — unknown-call discovery
+
+### 3.1 Usage
+
+Full demo mailbox sweep (189 probes: IDs 1..63 times three patterns):
+
+```bash
+python3 harness/control_discovery.py --out dumps/discovery --seed 7
+```
+
+A quick sweep for CI or bring-up (IDs 1..32, one pattern):
+
+```bash
+python3 harness/control_discovery.py --surface demo --out dumps/discovery-min \
+    --max-id 32 --patterns zero --seed 5 --timeout 0.25
+```
+
+Other surfaces / budgets:
+
+```bash
+# RM surface (documented future interface; see section 5)
+python3 harness/control_discovery.py --surface rm --out dumps/discovery-rm
+
+# cap the probe count and keep every recorded session
+python3 harness/control_discovery.py --out dumps/discovery-cap --iterations 40 --record-all
+```
+
+Flags: `--max-id` (1..4096), `--patterns zero,sentinel,boundary`, `--seed`,
+`--iterations` (0 = all probes), `--timeout` (per-probe observation window),
+`--settle` (self-test budget), `--backend qemu|native` (QEMU is the supported
+default; native drops the QEMU fault log and uses signals), `--kernel`
+(bundled demo by default; a foreign ELF needs `--allow-foreign-elf`),
+`--record-classes` and `--record-all`, `--max-log-bytes`.
+
+Each probe uses a **fresh session**, settles through the self-test, then
+applies one command. Fresh sessions are the safe strategy: a trap or fault can
+leave the demo in a state that would corrupt the next probe.
+
+### 3.2 Output
+
+```text
+<out>/
+  findings.jsonl             one compact finding per probe (deterministic order)
+  discovery-report.json      format + options + per-ID aggregate + findings
+  control-discovery.md       rendered report grouped by class
+  sessions/<idx>-id<NNN>-<pattern>/   kept evidence for interesting classes
+```
+
+Finding shape (stable keys, no wall-clock data):
+
+```json
+{"probe": 15, "cmd": 16, "pattern": "zero", "args": [0, 0, 0],
+ "outcome": "injected-slot",
+ "handler": "rpc_ping", "handler_entry": 2147483720, "documented_handler": null,
+ "dispatch_seen": true, "dispatch_pc": 2147483912, "dispatch_args": [16, 0, 0, 0],
+ "rpc_count_delta": 1,
+ "effects": {"changed": [{"field": "result", "before": 48, "after": 48879}],
+             "watch": [{"field": "result", "pc": 2147484066, "symbol": "rpc_dispatch"}]},
+ "fault": null, "signals": [], "settle_ok": true,
+ "session": "sessions/0015-id016-zero", "reason": null}
+```
+
+### 3.3 Reading the report
+
+The markdown report opens with the outcome counts and an "Ids by class" line,
+then sections that matter:
+
+* **Known handler ids** — the documented dispatch table (by the firmware's own
+  control block, not the docs);
+* **Injected-slot dispatch ids** — commands whose path is the injected-service
+  table, with the handler each aliases;
+* **Unknown / undiscovered ids** — the explicit list that answers "what is
+  not handled here yet";
+* **Traps and faults** — IDs that reach error paths under some input;
+* **All findings by id** — the join key for `findings.jsonl`.
+
+### 3.4 What the demo sweep already shows
+
+Measured on 2026-10-02, demo ELF sha256 `c038a04a...`, 189 probes in ~36 s:
+
+* IDs 1..6 hit `rpc_ping`/`rpc_echo`/`rpc_fill`/`rpc_sum`/`rpc_state`/
+  `rpc_fault`; the ID 6 path ends in the `ebreak` trap and never writes
+  `result` (the dispatcher's post-handler stores do not run);
+* **discovered class:** IDs 16..23 fall through the switch into the
+  injected-service handler table. IDs 16..21 alias the *built-in* handlers
+  (`16 -> rpc_ping`, ..., `21 -> rpc_fault`); IDs 22..23 are NULL slots and
+  return `~id`. This is exactly the kind of wiring a new firmware can add
+  silently;
+* IDs 7..15 and 24..63 take the catch-all and return `~id` with
+  `rpc_count + 1` — the unknown-ID space;
+* argument patterns change the class where the handler uses the argument:
+  `rpc_echo` and `rpc_sum` fault on sentinel/boundary pointers, while
+  `rpc_ping`/`rpc_state`/`rpc_fill` stay on the handler path.
+
+---
+
+## 4. `reach.py` — hot-path reachability
+
+### 4.1 Usage
+
+```bash
+# Directly callable target, with an argument
+python3 harness/reach.py --target rpc_ping --method call --args 0x41 \
+    --out dumps/reach-ping
+
+# Natural boot/self-test path
+python3 harness/reach.py --target rpc_sum --method run --out dumps/reach-sum
+
+# Only reachable through a mailbox command: discover the trigger from the
+# handler table and apply it
+python3 harness/reach.py --target rpc_fault --method trigger --out dumps/reach-fault
+
+# Replay a saved probe spec; emit one for CI
+python3 harness/reach.py --probe dumps/reach-fault/probe.json --emit-probe /tmp/fault-probe.json
+```
+
+Flags: `--method auto|stimulus|trigger|run|call`, `--all-methods`, `--args`,
+`--arg-count` with `--arg-fill zero|sentinel`, `--timeout`, `--settle`,
+`--stimulus script.py`, `--probe spec.json`, `--emit-probe`, `--record [DIR]`,
+`--max-prior`, `--kernel`, `--backend`, `--allow-foreign-elf`.
+
+### 4.2 Methods
+
+`auto` tries methods in this order and stops at the first hit (use
+`--all-methods` to compare routes):
+
+1. `stimulus` — run a green-eye script (or a probe spec's stimulus) with the
+   target hook and watchpoints armed. The script drives `session.run` itself.
+2. `trigger` — read the surface's handler table, find the command ID(s) that
+   resolve to the target, settle the firmware, apply that command and run
+   bounded. This is the trigger discovery for IDs 1..6 and 16..23 on the demo.
+3. `run` — `Session.run_until(target, timeout)` from launch, which catches the
+   natural boot/self-test path.
+4. `call` — `Session.call(target, args)`, the Frida-like direct call, for
+   unit-testing a function in isolation.
+
+A method reports a hit when the target-entry hook fires, even if execution
+immediately traps (`rpc_fault`): reachability and success are different
+questions, and the report distinguishes them.
+
+### 4.3 Output
+
+```text
+<out>/
+  reach-report.json      machine-readable report (stable keys)
+  reach-report.md        rendered report
+  probe.json             suggested repeatable probe spec (on a hit)
+  sessions/<method>/     optional recorded session (--record)
+```
+
+The report records:
+
+* `reachable`, `method`, and every `methods_tried` with its miss reason;
+* `entry`: the register state at the target (args `a0..a3`, `ra`, `sp`) and
+  `dispatch_args` — the authoritative command arguments when the target was
+  reached through the dispatcher;
+* `prior_events`: the ordered hook events that led to the hit (bounded by
+  `--max-prior`);
+* `required_args`: discovered arguments plus their provenance (`call-args` or
+  `firmware-dispatch`) and the caller `ra`;
+* `effects`: field deltas, watchpoint stores, the return value, and whether a
+  trap/fault followed (`rpc_fault`: trap `ebreak` at `pc 0`);
+* `trace_window`: hook counts and, when recorded, session event counts;
+* `probe_spec`: the exact spec to replay the route.
+
+### 4.4 Demo results
+
+Measured on 2026-10-02 (each run ~0.5 s, QEMU):
+
+| Target | Method | Result |
+|---|---|---|
+| `rpc_ping` | `call` with `--args 0x41` | reachable; returns `0x41 ^ 0xbeef = 0xbeae` |
+| `rpc_sum` | `run` | reachable in the self-test; entry args `(buf, 16, 32, ...)`; path passes through `rpc_ping` -> `rpc_echo` -> `rpc_fill` first |
+| `rpc_fault` | `trigger` | reachable via mailbox command 6 (also aliased at 21); dispatch args `cmd=6, a0=0, a1=0`; immediate `ebreak` trap |
+
+The self-test path is a useful reminder that compile-time inlining can bypass
+the dispatcher: the `run` route reaches `rpc_sum` from an inlined call site,
+so there are no `rpc_dispatch` hits before it. The `trigger` route does pass
+through the dispatcher and exposes the control arguments.
+
+---
+
+## 5. Sandsifter for RM (the future surface)
+
+A Sandsifter-style tool for RM needs three more capabilities than the demo
+has today, in this order.
+
+### 5.1 The `Surface` abstraction (already in place)
+
+`harness/control_discovery.Surface` separates:
+
+* `probe(session, id, args)` — how one call is applied (mailbox write today;
+  RM method call / message write later);
+* `observer()` — the effect model (control-block fields today; RM reply
+  buffer and object state later);
+* `handler_map()` / `command_for_addr()` — how IDs map to code, which drives
+  both discovery and reach's `trigger` method.
+
+Adding a real RM surface is a new subclass, not a rewrite of the sweep,
+classification or reporting.
+
+### 5.2 RM sweep design
+
+1. **Control/method inventory.** Start from `rmtool`/`rmq` (docs/17) for the
+   host-visible control list and class IDs; import the Ghidra RM-core dispatch
+   table into the symbol DB. The sweep ranges are then control IDs by class
+   (`--rm-class`) and method opcodes within a control, mirroring the
+   `nv2080 method call` surface.
+2. **Entry-point modelling.** Hook the RM dispatch routine, the object
+   constructor/method tables and the status/notifier writes. For each swept
+   ID, capture the handler entry, the object it selects and the status it
+   returns.
+3. **Effect observation.** Point `--rm-reply` at the reply buffer for the
+   control and hash/snapshot it; add watchpoints on the object handle table
+   and status queue. Compare the reply field-by-field against the host oracle
+   (`rmtool` JSON) for M2 (docs/06).
+4. **Branch exploration.** The native simulator's snapshots (H3) make it cheap
+   to fork at the dispatcher and sweep IDs without re-booting; the QEMU
+   surface uses fresh sessions exactly like today.
+5. **Corpus and gates.** Findings become an inventory corpus (`findings.jsonl`)
+   that CI diffs between firmware versions; new IDs, new handlers, or changed
+   default-path results are the regression signal (docs/21).
+
+### 5.3 Rollout gates
+
+* Entry gate: H2.2/H1.3 — a simulator session reaches the RM message loop and
+  exposes the dispatch table (otherwise `--surface rm` exits cleanly with the
+  message below).
+* Exit gate: sweep a real control space, reach one previously unknown control,
+  and replay its probe spec through `reach.py` with a recorded session.
+
+When unavailable the tool is explicit, not mysterious:
+
+```text
+$ python3 harness/control_discovery.py --surface rm --out dumps/discovery-rm
+[discovery] skipped: surface 'rm' unavailable: no RM control/method entry point is exposed
+by the simulator or a live session yet (roadmap H1.3). Pass --rm-entry SYMBOL once the
+RM method dispatcher exists; the demo surface works today with --surface demo.
+```
+
+---
+
+## 6. Feeding the simulator/device model and CI
+
+* **Device model.** Each discovered class is a candidate row in the
+  simulator's dispatch/control model: the command ID, the entry symbol, the
+  argument pattern that reaches it, and the effect snapshot are exactly the
+  data the H2/H3 device map needs. Unknown IDs (`default-path`) document the
+  fail-closed/default behaviour the emulation must reproduce.
+* **Accuracy metrics.** The `dispatch_pc`/`handler_entry`/effects fields feed
+  M1 (inventory coverage) and M4/M6 (state and memory side effects,
+  docs/06); compare two firmware versions by diffing `discovery-report.json`
+  (new handler IDs, moved handler entries, changed default results).
+* **CI (docs/21).** The intended gates are:
+  1. `control_discovery.py --max-id 32 --patterns zero` as a fast inventory
+     smoke test: known handlers must resolve, `rpc_fault` must trap, and the
+     default-path result must stay `~id`;
+  2. `reach.py` on a pinned list of targets, asserting `reachable` and the
+     method (`call` for `rpc_ping`, `run` for `rpc_sum`, `trigger` for
+     `rpc_fault`) so a compiler/inlining or dispatch change is caught;
+  3. diff `findings.jsonl`/`discovery-report.json` against a golden run.
+  Findings contain no wall-clock data, so the diff is stable for a fixed
+  seed/params (the same property the mutation harness documents in docs/18).
+* **Probe specs are the interface.** `reach.py --emit-probe` and
+  `--probe` turn a discovered route into an artifact that any agent or CI job
+  can replay without re-deriving it.
+
+---
+
+## 7. Tests
+
+```bash
+python3 -m unittest tests.test_discovery -v
+make test        # includes the discovery tests with the rest of the suite
+```
+
+`tests/test_discovery.py` covers the pure functions (pattern determinism,
+probe ordering/caps, classification priority), a 32-probe demo sweep that
+must attribute IDs 1..6, find an unknown ID and an injected-slot alias and
+write the report, byte-identical replay for a fixed seed, the clean RM skip,
+and the three reach demonstrations (`rpc_ping` call, `rpc_sum` run,
+`rpc_fault` trigger). It skips cleanly when QEMU or the demo ELF is absent.
+The suite completes in well under a minute.
+
+---
+
+## 8. Safety and ethics
+
+* **Simulator only.** The tools launch their own QEMU (or the native
+  simulator with `--backend native`) against the bundled demo or an
+  explicitly allowed re-hosted ELF. They never attach to hardware, a guest, a
+  kernel module or an existing gdbstub. Ctrl-C leaves no target running: each
+  session is closed in a `finally`.
+* **Bounded probes.** Every probe has a settle budget and an observation
+  window; `--max-id`/`--iterations` cap the sweep; QEMU log reads are capped.
+* **No hardware writes.** All memory writes go through the debug stub into
+  emulated RAM; the only host writes are under `--out`.
+* **Evidence, not truth.** A `default-path` result is evidence about this
+  firmware at this fidelity (L0/L1, docs/06), not a statement about silicon.
+  Never use a simulator observation to bypass a hardware protection; escalate
+  anything that would change the product to a physical capture (docs/14
+  sections 6-7).
+* **No injected payloads here.** Discovery only *observes* injection slots; it
+  never loads a blob. Injection itself stays behind `greeneye inject` and the
+  docs/12 contract.
+
+---
+
+## 9. Limits and fidelity
+
+* QEMU wall-clock slicing bounds when a handler is interrupted, so a very long
+  handler may be classified from its entry hit without its final result; the
+  finding says `known-handler` and leaves `effects` empty rather than guessing.
+* The demo is not RM: there is no object model, no status queue and no
+  per-control reply. The RM design in section 5 is deliberately unbuilt until
+  the simulator exposes an RM dispatch table (H1.3/H2.2).
+* `trap` vs `fault` uses the guest exception cause; QEMU and the native
+  simulator agree on the demo, but x86/RISC-V mapping differences are a
+  documented fidelity gap (docs/06 section 8).
+* The sweep is single-threaded and deterministic; concurrent/multi-core
+  dispatch interleavings (H2.4) are future work.
+
+---
+
+## 10. Roadmap mapping
+
+| Item | Contribution |
+|---|---|
+| **H4.adversarial navigation** | unknown-call discovery and trigger discovery over control surfaces |
+| **H4.1 semantic diffing** | `discovery-report.json` is a version-diffable inventory (new/changed IDs, handlers, default paths) |
+| **H4.2 capability inference** | `effects.watch` store PCs are access evidence for the capability table |
+| **H4.5 reimplementation loop** | `reach.py` output plus probe spec is the observe -> reproduce artifact for a new behaviour |
+| **H3 native simulator** | `--backend native` reuses the same sweep once parity lands; snapshots will remove the fresh-session cost |
+| **V1 golden data / docs/21 CI** | stable findings and the fast 32-probe gate are the golden corpus |