diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md | 254 |
1 files changed, 254 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md b/debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md new file mode 100644 index 0000000..affd350 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md @@ -0,0 +1,254 @@ +# 44 — GMMU/FBIF oracle and workflow (in development) + +> **Status: in development (2026-10-03, G3 work package).** This page +> documents two stdlib-only harness tools: +> +> * `harness/gmmu_oracle.py` — offline comparison of a human oracle capture +> against a simulator GMMU/FBIF trace (per-register agreement, coverage, +> unmatched lists, fidelity labels); +> * `harness/gmmu_debug.py` — static↔dynamic join: a trace plus a symbol +> map becomes a per-function register-write report and a +> `greeneye_hits.tsv` the existing Ghidra annotator consumes. +> +> Development-only: these tools read files and write reports. They never +> touch a GPU, kernel module, mdev, guest or network, and they make no +> cycle/timing claim. Physical work (the oracle capture) is supervised-only +> per RUNBOOK P1/P7 and is never run by an agent. +> +> **See also:** `06-ACCURACY.md` (fidelity levels, M1–M6), +> `05-GHIDRA-BRIDGE.md` (address spaces, hit annotation), +> `17-RM-HARNESS.md` (read-only RM oracle policy), +> `25-VALUE-VALIDATION.md` (measured-value discipline), +> `36-GSP-DEVICE-MODEL.md` (the simulator device model whose traces feed +> these tools). + +## 1. What this is and why + +The device model (`sim/devices/`, docs/36) records every modelled MMIO +access with its PC, width, direction and value. Two questions follow: + +1. **Accuracy** — does the value the model serves agree with what a human + can observe on the host (read-only RM/sysfs readbacks, supervised + captures)? (`gmmu_oracle.py`) +2. **Static analysis** — which firmware function programs which + GMMU/FBIF register, in what order, and where is it in Ghidra? + (`gmmu_debug.py`) + +Both are offline. The oracle file is produced by a human; the tools ingest +it and compare semantically. Nothing here executes a physical command. + +## 2. The accuracy loop (oracle vs simulator) + +``` +human (supervised, read-only) simulator (agent-safe) + rmtool / rmtop / sysfs / debugfs greeneye-gmmu-fbif-trace/1 + -> greeneye-gmmu-oracle/1 (from sim/ devices + trace) + \ / + \ harness/gmmu_oracle.py compare / + \____________________________________/ + | + gmmu-oracle-report.{json,md} + agreement %, coverage %, unmatched lists, + fidelity label per row +``` + +Rules: + +* The oracle is **observed** (human capture). The simulator value is + **modelled** (device model / recorded trace). Every row carries that + distinction; when a value is missing on either side the row's fidelity + label says the value is **unknown** instead of guessing. +* Agreement is computed **per register on the matched intersection** — it + is never a single global "accuracy %". Coverage is reported separately. +* An oracle-only register means the simulator did not touch it under this + stimulus; a simulator-only register means no oracle counterpart exists + yet. Neither is proof of a gap. +* No timing, ordering or silicon claim. Fidelity is L0/L1 (docs/06). + +### 2.1 Oracle schema `greeneye-gmmu-oracle/1` + +An object with a `captures` list (a bare JSON list is accepted and reported +as `unlabeled-list`): + +```json +{ + "format": "greeneye-gmmu-oracle/1", + "captures": [ + {"name": "FBIF_CTL", "window": "falcon", "offset": "0x624", + "value": "0x1", "source": "rm-readback", "captured_at": "2026-10-03"} + ] +} +``` + +`name`, `window`, `offset`, `value`, `source`, `captured_at` are the +documented fields. `offset`/`value`/`addr` accept ints or `0x…` strings. +A different explicit `format` is a clean error. + +### 2.2 Simulator schema `greeneye-gmmu-fbif-trace/1` + +The producer (`sim/devices/observe.py:build_trace`, docs/36) emits a +nested document: accesses are grouped per window and per declared register, +with an `unclassified` bucket for observed offsets the profile does not +name. The `schema` key carries the format tag: + +```json +{ + "schema": "greeneye-gmmu-fbif-trace/1", + "image_sha256": "…", "profile": "tu104-570.144", + "summary": {"windows": 1, "accesses": 2}, + "windows": [ + { + "label": "falcon-observe", "base": "0x300110000", "size": "0x1000", + "mode": "observe-only", "evidence": "header", + "registers": [ + {"offset": "0x624", "name": "FBIF_CTL", "access": "rw", + "counts": {"r": 0, "w": 1, "served": 0}, + "accesses": [ + {"addr": "0x300110624", "offset": "0x624", "size": 4, "rw": "w", + "value": "0x1", "pc": "0x4002040", "vt": 123, + "window": "falcon-observe", "register": "FBIF_CTL", + "served": false, "pc_symbol": "fbif_setup"} + ]} + ], + "unclassified": {"counts": {"r": 1, "w": 0, "served": 0}, + "accesses": [{"addr": "0x300110008", "offset": "0x8", + "rw": "r", "value": null, "pc": "0x4002050"}]} + } + ] +} +``` + +`load_sim_trace` flattens this shape (and accepts a flat `events` list, a +bare list, JSONL, and aliases) so the harness survives producer naming +choices: `kind`/`rw`/`direction`/`type`/`t`; `name`/`register`/`reg`; +`window`/`block`/`device`; `offset`/`reg_offset`; `addr`/`address`/`pa`; +`value`/`data`; `pc`; `function`/`symbol`/`pc_symbol`; `handled`/`served`. +Nested documents are flattened per window/register: per-register access +order is preserved, but global interleaving across registers is not +recoverable from that grouping. The producer's top-level `summary` — +including `retained_accesses` / `dropped_accesses` — is carried into +`inputs.trace.source.summary`, so a capped (truncated) access log stays +visible downstream. A session directory is accepted too: +`session.json` pointers (`gmmu_fbif_trace`, `gmmu_fbif_trace_blob`, +`trace_blob`) or a `blobs/*.json[l]` file carrying the format tag. A raw +`read`/`write` JSONL trace without register names still parses (registers +are then reported by address). + +## 3. Static-analysis debugging loop (`gmmu_debug.py`) + +``` +greeneye-gmmu-fbif-trace/1 + symbols (greeneye-symbols/1 | addr->name map | TSV) + | + v +per-function register-write report gmmu-debug-report.{json,md} + function, register, window, offset, + sequence of written values, + first/last pc, write/read counts + | + v +greeneye_hits.tsv (addr hits symbol annotation) + | + v +analyzeHeadless ... -postScript MarkGreenEyeHits.java <greeneye_hits.tsv> +``` + +`greeneye_hits.tsv` is byte-compatible with +`greeneye/ghidra.py:export_hits` and `ghidra/MarkGreenEyeHits.java`: header +`addr\thits\tsymbol\tannotation`, hex address, one row per function, `hits` += register accesses attributed to the function. Function attribution uses +the symbol DB (containing symbol when sizes exist, the toolchain's +nearest-preceding fallback otherwise); a pc outside every symbol becomes +`sub_<pc>`. An explicit `function` field in the trace wins over resolution. + +## 4. Exact commands + +All commands run from `open-iov/debug/risc-v-gsp-rm-vm`. They are offline +and safe for an agent. + +```bash +# Accuracy loop — compare a human oracle with a simulator trace/session. +python3 harness/gmmu_oracle.py compare \ + --oracle oracle.json --sim dumps/<session-or-trace.json> \ + --out dumps/gmmu-oracle-report + +# Print (do NOT execute) the supervised, read-only capture instructions. +python3 harness/gmmu_oracle.py --print-capture-hints + +# Static↔dynamic join — report + Ghidra-ready hits TSV. +python3 harness/gmmu_debug.py report \ + --trace dumps/<session-or-trace.json> --symbols symbols.json \ + --out dumps/gmmu-debug-report + +# Synthetic end-to-end fixture (no firmware, no hardware). +python3 harness/gmmu_debug.py demo --out harness/gmmu_debug_fixture + +# Tests (synthetic only; no live oracle capture, no hardware). +python3 -m unittest tests.test_gmmu_oracle tests.test_gmmu_debug -v +``` + +`--print-capture-hints` only prints. It names the read-only RM wrapper +commands (`./rmharness/rmtool.sh gpu list/engine list/mem stats`, +`rmtop --once`), sysfs/proc/debugfs readbacks, and the optional +first-touch step, with the hard warning that physical work is +supervised-only and an agent must never run those commands. The safe +simulator observation (`python3 -m sim.gsp_boot … --record …`) is the only +capture an agent may produce. + +Generated fixture/report directories under `harness/gmmu_*` are gitignored. + +## 5. Worked example (synthetic, measured) + +`python3 harness/gmmu_debug.py demo` builds a 13-event synthetic trace with +three named functions plus one unattributed pc, runs the whole loop, and +prints a `RESULT_JSON` line. On 2026-10-03 (this host, CPython 3.12) the +loop measured **≈1 ms**, against a manual static-reading baseline frozen +in `harness/gmmu_debug.py` **before** the first measurement: + +| field | value | +|---|---| +| manual baseline (frozen) | `1200 s` — a **conservative estimate**, not measured on a human | +| threshold fraction (frozen) | `0.05` → `60 s` | +| measured (fresh run required to quote) | ≈0.001 s | +| verdict | pass | + +The baseline and fraction are constants in `harness/gmmu_debug.py`; moving +them requires a new recorded run and a note here. The demo proves the +**shape** of the loop (trace → per-function report → hits TSV → +Ghidra-ready output), never a hardware fact. + +Synthetic oracle comparison check (three oracle items, three trace events): +one agree, one mismatch, one oracle-only, one simulator-only → agreement +50 % on the two-item intersection, coverage 66.67 %. These are fixture +numbers, not an accuracy claim about any firmware. + +## 6. Limits and not-verified list + +* **No live oracle capture was taken.** The tools were exercised only with + synthetic fixtures. A real oracle requires a supervised human session + per §4 and docs/17 §6. +* **No real firmware trace was compared.** The loader is exercised against + the real `sim/devices/observe.py` producer (read-only integration test) + and against synthetic documents; no captured firmware run was compared, + and no live oracle capture was taken. +* **L0/L1 only.** The trace records execution and register accesses, not + time; no L2/L3 claim is made. Absence of a register in a trace is a fact + about that run's stimulus, not proof of dead code. +* **Agreement is per-register on the intersection**, with coverage and both + unmatched lists reported; never a single global percentage. +* **Function attribution depends on the symbol map.** Zero-size symbols use + the toolchain's nearest-preceding fallback; supply sized symbols + (`greeneye-symbols/1`) for strict boundaries. +* **Product changes are not implied.** Any emulation change still needs the + accuracy loop plus physical reproduction per docs/06 §6. +* The demo baseline is an **estimate**; no human was timed. + +## 7. Tests + +`tests/test_gmmu_oracle.py` and `tests/test_gmmu_debug.py` cover: oracle +ingest (object/list/aliases/bad format), comparison (agreement, mismatch, +oracle-only, simulator-only, name fallback, no-value, empty), session +directory ingest, report shape and sequence order, explicit-function +override, unknown-pc handling, hits-TSV compatibility with +`MarkGreenEyeHits.java`, empty traces, the frozen-threshold demo, and the +CLI error paths (exit 2, no traceback). All fixtures are synthetic; no +hardware, module, guest or network is touched. |