debug: green-eye wave 3 - RM harness, local simulator, device map, MCP plugin, value-validation demos, CI pipeline, toolchain docs; add host RM tools rmtool/rmtop

1 files changed, 207 insertions, 0 deletions

diff --git a/debug/risc-v-gsp-rm-vm/docs/24-DEVICE-MAP.md b/debug/risc-v-gsp-rm-vm/docs/24-DEVICE-MAP.md
new file mode 100644
index 0000000..4430bf9
--- /dev/null
+++ b/debug/risc-v-gsp-rm-vm/docs/24-DEVICE-MAP.md
@@ -0,0 +1,207 @@
+# 24 — Device map: first-touch MMIO and lenient native execution
+
+> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status.
+> This is the H2.1 data layer and the first H2.2 building block, not a device
+> model. It carries the same fidelity rules as `06-ACCURACY.md`: L3 remains
+> **absent**; nothing here is a hardware, timing or security claim.
+
+## 1. Purpose and status
+
+`greeneye-device-map/1` records which MMIO addresses a firmware image touched
+first, at which width and direction, with observed values, hit counts and
+provenance. It is JSON data, versioned with the firmware that produced it, and
+consumed by the native simulator's lenient MMIO mode. It exists so the R1
+device work (H2.2) starts from measured addresses instead of guesses, and so
+the real GSP image can be stepped past its first unmapped access to collect
+the *next* ones.
+
+| Piece | Where |
+|---|---|
+| map model, harvest, query, CLI | `greeneye/device_map.py` |
+| lenient / fault MMIO policy | `sim/machine.py` (`Machine(device_map=..., unknown_mmio=...)`) |
+| backend plumbing | `greeneye/sim_backend.py` (`launch_native(..., device_map=..., unknown_mmio=..., learn_mmio=..., bias=...)`) |
+| tests | `tests/test_device_map.py` |
+
+## 2. The model
+
+An entry is one `(address, width, direction)` observation:
+
+| field | meaning |
+|---|---|
+| `address` | the MMIO address as accessed (64-bit; 61-bit GSP addresses fit without allocating memory) |
+| `width` | access size in bytes (1/2/4/8) |
+| `kind` | `"r"` or `"w"` |
+| `values` | observed values (first-seen order, deduplicated); empty for probe-harvested reads because the QEMU log records the fault, not the value |
+| `hits` | number of times observed at this key |
+| `first_epc` | the first observing PC as executed (relocated) |
+| `first_epc_fw` | `first_epc - bias`: the firmware's own virtual address; a PC of 0 (trap-vector fetch) stays 0 |
+| `source` | `probe` (harvested), `manual` (hand-written), `trace` (learned at runtime) |
+| `notes` | fault/exception provenance, e.g. `fault_load reason=rejected` |
+
+The file carries `meta` with the schema version, harvest/learn source, the
+relocation `bias`, `entry`, the firmware path, its `sha256` (when the file is
+readable) and `gsp_version.txt` (sanitised), plus `qemu_fault_count` when
+harvested. Entries are always sorted by `(address, width, kind)` and the file
+is written with `sort_keys=True`, so harvesting the same log twice is
+byte-identical; there are no timestamps.
+
+`DeviceMap.lookup(addr, size, write)` returns a behaviour spec:
+
+* `value:0x...` — a read observed to return this value;
+* `zero` — the address is known (or the direction is unknown) and reads back 0;
+* `unimplemented` — no evidence at this address.
+
+Exact `(address, width, kind)` wins; a different width at the same address is
+still evidence for the register. A write to a read-observed address is
+absorbed; a read of a write-only register reads back zero.
+
+## 3. Harvesting from `greeneye probe`
+
+`greeneye probe` (unchanged) records `blobs/qemu.log` and the parsed
+`qemu_first_fault` in `session.json`. Harvest reads either:
+
+```bash
+# 1) run a bounded probe against the real image (QEMU path, existing command)
+./greeneye.sh probe                      # writes dumps/<stamp>-probe-<image>/
+
+# 2) turn that session into a device map
+python3 -m greeneye.device_map harvest dumps/<stamp>-probe-<image> \
+    --out dumps/570.144-device-map.json
+
+# 3) inspect / merge maps later, e.g. across firmware versions
+python3 -m greeneye.device_map show dumps/570.144-device-map.json
+python3 -m greeneye.device_map merge a.json b.json --out merged.json
+```
+
+Harvesting the recorded TU104 570.144 probe session
+(`dumps/20261002-091222-probe-gsp-570.144.fwimage`) gives two entries — the
+first-touch MMIO read and the `mtvec=0` fetch loop QEMU falls into afterwards:
+
+```json
+{"schema": "greeneye-device-map/1", "entries": [
+  {"address": 0, "width": 2, "kind": "r", "hits": 367551, "first_epc": 0,
+   "first_epc_fw": 0, "source": "probe", "notes": "fault_fetch reason=rejected"},
+  {"address": 2305843009213696512, "width": 4, "kind": "r", "hits": 1,
+   "first_epc": 2214629396, "first_epc_fw": 67145748, "source": "probe",
+   "notes": "fault_load reason=rejected"}]}
+```
+
+`2305843009213696512 == 0x2000000000000A00`, `67145748 == 0x4009014`; `meta`
+also carries the firmware sha256
+(`2d1bb5c6…`), `firmware_version: 570.144W` and `qemu_fault_count: 367552`.
+Programmatic forms: `harvest_probe_session(session_dir)` and
+`harvest_from_text(log_text, bias=...)` for arbitrary logs; `DeviceMap.save`,
+`DeviceMap.load`, `merge`, `lookup`.
+
+## 4. Lenient MMIO in the native simulator
+
+`sim.machine.Machine` gained three knobs (all default to today's behaviour):
+
+```python
+Machine(memory_mb=256, ram_base=0x80000000,
+        device_map=None, unknown_mmio="fault", learn_mmio=False, bias=0)
+```
+
+* `unknown_mmio="fault"` (default) — **byte-for-byte M1 parity**: an access
+  that is neither RAM nor UART faults exactly as before, and nothing is logged
+  or counted.
+* `device_map=...` — mapped entries are served in *either* mode; a mapped
+  address that is unknown to the map is still `unimplemented`.
+* `unknown_mmio="lenient"` — an unmodelled access returns the mapped
+  behaviour, else zero; it is appended to `machine.mmio_log` as
+  `{pc, addr, size, rw}` and counted in `machine.mmio_unknown`. No logging
+  occurs when lenient mode is off. Debugger `read_mem`/`write_mem` stay
+  strict.
+* `learn_mmio=True` — lenient observations are written back into the map
+  (`source: "trace"`, read value `0`, write value as written); with no map
+  given, one is created lazily. Values recorded with `pc` and `bias`.
+
+Through the backend:
+
+```python
+session = ge.launch(kernel=..., backend="native",
+                    device_map="dumps/570.144-device-map.json",
+                    unknown_mmio="lenient")
+session.target.mmio_log       # [{pc, addr, size, rw}, ...]
+session.target.mmio_unknown   # int
+```
+
+`launch_native(..., device_map=..., unknown_mmio=..., learn_mmio=..., bias=...)`
+takes the same options; `device_map` may be a `DeviceMap` or a saved path, and
+`bias` relocates images linked for GPU physical addresses. The lenient path is
+only entered on a miss, so RAM/UART/trace throughput is unchanged; recording
+is a plain list append and the map lookup is a dict hit.
+
+**What lenient proves, honestly:** an unmodelled address can be served a
+mapped/zero value and execution continues deterministically; a bounded run
+lists every unmodelled access and their PCs. It does **not** prove what a real
+device returns, does not model read-back, side effects, interrupts or timing,
+and a zero that keeps the firmware running is a hypothesis to be checked
+against the H2.2 device model (and eventually hardware), not a finding about
+silicon.
+
+## 5. Real-image demonstration (bounded, measured 2026-10-02)
+
+```bash
+python3 -m greeneye.device_map demo --instructions 200000
+```
+
+```text
+image: /home/rmcore/Git/integdev_gpu_drv/open-iov/firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin
+sha256: 2d1bb5c60b208cfd115f015b17dd04a6fece05fd2c0dcba3625d9d48cce3ceea
+elf: entry=0x4009000 vaddr=[0x0,0x5bb9000)
+bias: 0x80000000  relocated entry: 0x84009000
+budget: 200000 instructions
+stopped: reason=budget pc=0x84009070 (fw 0x4009070)
+unknown mmio accesses: 1
+  first unknown: pc=0x84009014 (fw 0x4009014) addr=0x2000000000000a00 size=4 rw=r
+[device-map] qemu probe first fault: fw vaddr 0x4009014 reading 0x2000000000000a00 (load access fault)
+[device-map] verdict: first-touch address reproduced; lenient execution continued for 200000 instructions (199994 past the qemu trap)
+```
+
+Comparison against the QEMU probe (`docs/06 §3`, still current): QEMU executes
+six instructions and traps at firmware vaddr `0x4009014` on the 4-byte read of
+`0x2000000000000A00`, then loops on `mtvec=0` fetch faults. The lenient native
+run reproduces the same first access at the same PC (the `pc` in `mmio_log` is
+the accessing instruction, not the run start), returns 0, and continues
+through the entire 200 000-instruction budget, ending at firmware vaddr
+`0x4009070` with **exactly one** unmodelled access logged. It does not fault.
+
+**61-bit addresses.** `0x2000000000000A00` is a valid 64-bit value in the
+interpreter's address computation; no memory is allocated for it. RAM is
+bounds-checked by `addr - ram_base`, the UART window by range, and everything
+else goes to the device map / lenient handler. (`1 << 61` is well inside the
+RV64 address width; the only care needed is that no map or device code ever
+treats MMIO addresses as host indices.) `--out map.json` saves the learned
+map; `python3 -m greeneye.device_map show map.json` renders it.
+
+## 6. How this feeds H2.2
+
+* The harvested entries are the first test vectors for the `falcon`/`CSB`/
+  `pmu_stub` windows in the docs/16 profile DSL: group entries by page
+  (`address & ~0xFFF`), take `lookup()` behaviour as the initial read-back,
+  and replace `zero` with real semantics one window at a time.
+* `learn_mmio=True` is the bring-up loop: run lenient, inspect the learned
+  map + `mmio_log`, promote hand-verified entries to `source: "manual"`, and
+  re-run strict to see the next unmapped wall. Each promotion should carry a
+  note and the firmware hash, so the map is a reviewable artifact.
+* The map's `bias`, firmware sha256/version and schema make it profile data
+  keyed by `(device, firmware-family, profile-version)`, as required by
+  docs/16 §7.
+* The same maps are the YAML/JSON seed for the QEMU `-device` model, keeping
+  the two backends on one source of truth.
+
+## 7. API and commands
+
+| Call | Purpose |
+|---|---|
+| `harvest_probe_session(dir)` | session dir -> `DeviceMap` |
+| `harvest_from_text(text, bias=0)` | arbitrary QEMU log -> `DeviceMap` |
+| `DeviceMap.lookup(addr, size, write)` | `zero` / `value:0x...` / `unimplemented` |
+| `DeviceMap.record(...)`, `merge(...)`, `save(path)`, `load(path)` | authoring/persistence |
+| `python3 -m greeneye.device_map harvest\|show\|merge\|demo` | CLI |
+| `python3 -m pytest tests/test_device_map.py` | 20 unit/parity/real-image tests |
+
+Testing honesty: the unit tests are synthetic (no QEMU); the real-image test
+is bounded to 200 000 instructions and skips when the fwimage is absent.
+`tests/test_sim_backend.py` remains the M1 parity guard and passes unchanged.