diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-06 19:28:39 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-06 19:28:39 -0400 |
| commit | cd1f2528e8243fed3ce80d5170ef93928e4e7284 (patch) | |
| tree | 208c326a109347baf6c04917dca23ef60c73eabc /debug/risc-v-gsp-rm-vm | |
| parent | 595dcded851dc45d78cc76280137100ea8b3d0ad (diff) | |
| download | open-iov-cd1f2528e8243fed3ce80d5170ef93928e4e7284.tar.gz open-iov-cd1f2528e8243fed3ce80d5170ef93928e4e7284.zip | |
risc-v-gsp-rm-debugger: land docs 45-53 + harness/sim tool set with tests
rm_capture (chunked decode), launch_audit, state_seed/state_decode,
valprobe, rehost, contracts+run_contracts, host_traffic, gmmu_walk,
msgq_attach, libos_port, log_decode, server_snapshot; doc index updates
in the tool overview, operations pages and the repo README.
Diffstat (limited to 'debug/risc-v-gsp-rm-vm')
38 files changed, 16623 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/README.md b/debug/risc-v-gsp-rm-vm/README.md index d730660..3c71442 100644 --- a/debug/risc-v-gsp-rm-vm/README.md +++ b/debug/risc-v-gsp-rm-vm/README.md @@ -127,6 +127,14 @@ SIMULATOR-WORK-BOUNDARY.txt scope contract for the 2026-10-03 simulator wave | [42-GMMU-FBIF-OBSERVABILITY](docs/42-GMMU-FBIF-OBSERVABILITY.md) | observe-only GMMU/FBIF windows and `greeneye-gmmu-fbif-trace/1` (in development) | | [43-GMMU-PTE-DECODER](docs/43-GMMU-PTE-DECODER.md) | clean-room TU104 GMMU PTE/PDE decoder (in development) | | [44-GMMU-ORACLE-AND-WORKFLOW](docs/44-GMMU-ORACLE-AND-WORKFLOW.md) | oracle-vs-simulator comparison and static<->dynamic register join (in development) | +| [45-RM-RPC-CAPTURE-AND-REPLAY](docs/45-RM-RPC-CAPTURE-AND-REPLAY.md) | real-client GSP RPC capture: decode, compare, replay prep (`harness/rm_capture.py`, in development) | +| [46-SURGICAL-REHOST](docs/46-SURGICAL-REHOST.md) | run one bounded firmware function on the emulator with crafted state (`harness/rehost.py`, in development) | +| [47-VALIDATION-PREDICATE-PROBE](docs/47-VALIDATION-PREDICATE-PROBE.md) | run one firmware validation branch with crafted state and labelled outcomes (`harness/valprobe.py`, in development) | +| [48-GMMU-WALK-AND-LAUNCH-AUDIT](docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md) | functional GMMU page-table walk + launch memory-contract audit (`sim/gmmu_walk.py`, `harness/launch_audit.py`, in development) | +| [49-SIMULATION-SURFACE-MAP](docs/49-SIMULATION-SURFACE-MAP.md) | emulator coverage map and value-to-campaign proof (in development) | +| [50-WALL-WORKFLOW](docs/50-WALL-WORKFLOW.md) | wall-class taxonomy, capture discipline, contract registry (`harness/contracts.py`, `harness/run_contracts.py`, in development) | +| [51-HOST-TRAFFIC-LOOP](docs/51-HOST-TRAFFIC-LOOP.md) | emulated command/status transport, RPC framing, stub dispatch, `rm_capture` replay (`sim/host_traffic.py`, in development) | +| [52-RM-OBJECT-STATE-SEEDING](docs/52-RM-OBJECT-STATE-SEEDING.md) | declarative RM-object state seeding for rehost/valprobe (`harness/state_seed.py`, in development) | | [RISC-V-GSP-RM-DEBUGGER-ROADMAP](RISC-V-GSP-RM-DEBUGGER-ROADMAP.md) | the long-term program (H1–H7) | Scope contract for the current simulator wave: diff --git a/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md b/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md index a5f14ab..ebbc20a 100644 --- a/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md +++ b/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md @@ -131,6 +131,14 @@ Interactively: | [42-GMMU-FBIF-OBSERVABILITY](42-GMMU-FBIF-OBSERVABILITY.md) | observe-only GMMU/FBIF windows and trace (in development) | | [43-GMMU-PTE-DECODER](43-GMMU-PTE-DECODER.md) | TU104 GMMU PTE/PDE decoder (in development) | | [44-GMMU-ORACLE-AND-WORKFLOW](44-GMMU-ORACLE-AND-WORKFLOW.md) | GMMU/FBIF oracle comparison and static<->dynamic join (in development) | +| [45-RM-RPC-CAPTURE-AND-REPLAY](45-RM-RPC-CAPTURE-AND-REPLAY.md) | real-client GSP RPC capture decode/compare/replay prep (in development) | +| [46-SURGICAL-REHOST](46-SURGICAL-REHOST.md) | bounded function rehost on the emulator with crafted state (in development) | +| [47-VALIDATION-PREDICATE-PROBE](47-VALIDATION-PREDICATE-PROBE.md) | firmware validation branch with crafted state + labelled outcomes (in development) | +| [48-GMMU-WALK-AND-LAUNCH-AUDIT](48-GMMU-WALK-AND-LAUNCH-AUDIT.md) | functional GMMU walk + launch memory-contract audit (in development) | +| [49-SIMULATION-SURFACE-MAP](49-SIMULATION-SURFACE-MAP.md) | emulator coverage map and value-to-campaign proof (in development) | +| [50-WALL-WORKFLOW](50-WALL-WORKFLOW.md) | wall taxonomy, capture discipline, contract registry (in development) | +| [51-HOST-TRAFFIC-LOOP](51-HOST-TRAFFIC-LOOP.md) | host-traffic transport, RPC framing, stub dispatch, replay (in development) | +| [52-RM-OBJECT-STATE-SEEDING](52-RM-OBJECT-STATE-SEEDING.md) | RM-object state seeding schema + loader for rehost/valprobe (in development) | | [RISC-V-GSP-RM-DEBUGGER-ROADMAP](../RISC-V-GSP-RM-DEBUGGER-ROADMAP.md) | long-term program H1–H7 | ## Where this fits in the Open-IOV workflow diff --git a/debug/risc-v-gsp-rm-vm/docs/45-RM-RPC-CAPTURE-AND-REPLAY.md b/debug/risc-v-gsp-rm-vm/docs/45-RM-RPC-CAPTURE-AND-REPLAY.md new file mode 100644 index 0000000..023689a --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/45-RM-RPC-CAPTURE-AND-REPLAY.md @@ -0,0 +1,143 @@ +# 45 — GSP RPC capture ingestion and replay prep (in development) + +> **Status: in development (2026-10-06, capture-ingestion wave).** This page +> documents `harness/rm_capture.py`, a stdlib-only harness that turns the +> **supervised real-client GSP RPC capture** (the tag contract produced by the +> send-path instrumentation runbook) into structured, replayable artifacts: +> a decoded capture document, a per-record comparison report, and a replay +> **stimulus** document for the simulator's B4/B5 CSB/message-queue seam. +> +> Development-only: the tool reads files and writes reports. It never touches +> a GPU, kernel module, mdev, guest or network, and it never runs or schedules +> the capture itself. The capture is a supervised, human-run physical action +> (RUNBOOK P1/P7); the tool only ingests its log. +> +> **See also:** `06-ACCURACY.md` (fidelity levels, M1–M6), +> `17-RM-HARNESS.md` (read-only RM oracle policy), +> `36-GSP-DEVICE-MODEL.md` (the S1 device model and boot ladder), +> `38-GSP-EM-API.md` (TLV boundary), +> `44-GMMU-ORACLE-AND-WORKFLOW.md` (the same offline compare discipline for +> GMMU/FBIF registers). + +## 1. What this is and why + +The open GR question is *what the real host client actually sends* while it +creates and commits a GR context (the golden-image sequence and its promote +controls). The authored capture runbook obtains that sequence from an +instrumented reference kernel driver; this harness consumes the resulting log +and prepares it for two workflows: + +1. **Compare** — the real client's captured sequence vs. the open-iov host + ladder's replay (or two driver/firmware versions), per record, with explicit + unmatched lists. +2. **Replay** — an ordered stimulus document that the simulator's next rung + (B4/B5: host-traffic doorbell + first CSB/TRANSCOM poll served + one + scripted RM message end-to-end) can feed into the modelled firmware, so the + server side of the captured sequence can be observed off-hardware and + deterministically. + +The simulator consumes captured truth; it does not replace the capture. + +Known limitation up front: the capture runbook's hex helper currently caps a +payload dump at 192 bytes. Payloads larger than that (for example a 560-byte +`PROMOTE_CTX` control) decode as **`truncated`** and are **not replayable** +until the helper is extended to emit chunked blocks. The decoder flags this +per record and in the summary; it never pads a payload to look complete. + +## 2. The capture log contract + +Lines are matched anywhere in the line, so dmesg/NVRM/timestamp prefixes are +fine. Tags (exact grammar: `python3 harness/rm_capture.py --print-capture-hints`): + +``` +GE-GOLDEN-TRIGGER gspclient=<0|1> mig=<0|1> +GE-GOLDEN-BEGIN +GE-GOLDEN-END status=0x... +GE-SEND fn=0x... len=0x... ret=0x... +GE-CTRL cmd=0x... hClient=0x... hObject=0x... size=0x... paramp=0x... ret=0x... +GE-ALLOC cls=0x... hClient=0x... hParent=0x... hObject=0x... size=0x... paramp=0x... ret=0x... +GE-CTRL-DONE cmd=0x... rpc_status=0x... params_status=0x... +GE-<TAG> <offset-hex>: <hex bytes...> # attaches to the newest record of that tag +``` + +* Hex blocks attach to the most recent record **of their own tag**; an orphan + block is reported as a warning, never guessed onto another record. +* A `GE-SEND` payload's named id is derived when no named control line was + captured: function `76` (`GSP_RM_CONTROL`) carries the control id at payload + offset 8; function `103` (`GSP_RM_ALLOC`) carries the class id at offset 12. +* `ret=` is the caller return address: recorded as-is, capture-specific. + +## 3. Schemas + +* `greeneye-rm-rpc-capture/1` — `{format, source, sha256, records[], summary, + warnings, limits}`. Each record: `{index, kind, line, ts?, fn?, length?, + size?, cmd?, hclass?, hclient?, hobject?, hparent?, caller?, caller_raw?, + marker?, gspclient?, mig?, status?, rpc_status?, params_status?, + params_hex?, params_bytes, params_size_expected?, truncated?, anomalies?}`. +* `greeneye-rm-rpc-compare/1` — inputs + `summary` (matched / agree / + shape-agree / partial / mismatch / A-only / B-only / agreement % / + coverage % / mean byte agreement %), `rows[]` (per aligned record, with + byte agreement and first-difference offset), `a_only[]`, `b_only[]`, + `fidelity`, `limits`, `warnings`. +* `greeneye-rm-rpc-stimulus/1` — `markers[]` (golden slice boundaries) and + `messages[]` (`fn`/`cmd`/`hclass`/`wire_id`, `params_hex`, sizes, truncation + flag). Handles, physical addresses and caller addresses are **omitted** on + purpose: a replay must allocate/re-map them. + +## 4. Commands + +```bash +python3 harness/rm_capture.py --print-capture-hints # prints only +python3 harness/rm_capture.py decode --log capture.log --out dumps/rm-capture +python3 harness/rm_capture.py compare --a real.json --b replay.json --out dumps/rm-compare +python3 harness/rm_capture.py stimulus --capture real.json --out dumps/rm-stimulus.json +python3 harness/rm_capture.py demo --out /tmp/rm-capture-demo # synthetic, measured +``` + +Inputs to `compare`/`stimulus` may be decoded JSON documents or raw logs; the +loader auto-detects. A non-capture JSON format is a clean error. + +## 5. Simulator linkage (B4/B5) — not wired yet + +The stimulus document is the intended input for the simulator's +host-traffic rung: once the doorbell path and the first CSB/TRANSCOM poll are +served and a scripted message can be delivered end-to-end (handoff-simulator +§17.5), the captured golden-image sequence is the natural first real message +set. Until then this tool stays offline: `stimulus` states the pending wiring +in its `limits`, and nothing in `sim/` imports it yet. + +Ordering honesty: the capture is **send-side only**; server replies were not +captured by the probes. A replay therefore reproduces the stimulus, not the +reply sequence. + +## 6. Admission record (draft — pending maintainer filing) + +Per the project admission rule (`docs/eng-ops.md` §2): all five fields. This +record is a draft, like the simulator program's `gpgpu-sim/ADMISSION.md` +records; a maintainer files it. + +| Field | Value | +|---|---| +| Scenario | RE/porting loop: ingest a supervised real-client GSP RPC capture into the workflow — decode the golden-image sequence, compare it with the open-iov ladder's replay or another version, and prepare a replay stimulus for the simulator's message-loop rung. No hardware access. | +| Measurable outcome | Decode/compare/stimulus wall time vs. manual baseline; record and parameter integrity (no silent loss; truncation flagged); deterministic outputs (two runs byte-identical apart from timing). | +| Baseline | Manual decode-by-eye of one golden sequence from a capture log: grep tags, count records, walk each parameter blob against the RPC structs, transcribe a replay table. Conservative estimate **1800 s** (not measured on a human). | +| Evaluation threshold | Tool path ≤ **5%** of the baseline (≤90 s); demo `ok`; deterministic reports; new tests plus the full suite green. | +| Failure signal | Silent record/parameter loss; truncation unflagged; non-deterministic reports; demo above threshold; or any physical action executed/scheduled by the tool → disable the feature and record it as a failed experiment. | + +Measured proof (2026-10-06, synthetic demo on this host): 15/14 records, +13 agree + 1 shape-agree (flipped parameter byte) + 1 A-only (dropped alloc), +**wall ≈ 0.02 s** vs the 90 s threshold; artifacts under the `demo --out` +directory; 27 new tests, full suite 444 green (twice). + +## 7. Limits and deferrals + +* No hardware, module, mdev, guest or network action; the tool never runs the + capture. +* The 192-byte helper cap: extend the runbook helper to chunk before any large + payload (promote entries) is considered replayable. +* Handles/addresses are capture-specific and explicitly omitted from stimuli. +* Send-side only; no reply/event ordering. +* B4/B5 stimulus wiring, a captured-sequence vs. simulated-firmware accuracy + loop, and symbol/`addr2line` mapping of captured caller addresses are + deferred (the `gmmu_debug.py` static↔dynamic join is the pattern to reuse). +* L0/L1 fidelity only; no timing claim. diff --git a/debug/risc-v-gsp-rm-vm/docs/46-SURGICAL-REHOST.md b/debug/risc-v-gsp-rm-vm/docs/46-SURGICAL-REHOST.md new file mode 100644 index 0000000..e64be25 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/46-SURGICAL-REHOST.md @@ -0,0 +1,91 @@ +# 46 — Surgical rehost of firmware functions (in development) + +> **Status: in development (2026-10-06, rehost wave).** Documents +> `harness/rehost.py`: run one bounded function of a real/modelled firmware +> image on the native emulator, starting at a chosen entry with a crafted +> register/memory state, and record a session + report. This is the workflow +> the RE review ranks first for the live GR wall (docs/39 U2): answer +> server-side hypotheses from rehosted execution with hooks/watchpoints/ +> memory tools instead of another static read or another hardware cycle. +> +> Development-only: executes firmware on the simulator. Never touches a GPU, +> kernel module, mdev, guest or network. No timing/cycle claim (L0/L1). +> +> **See also:** `39-RE-WORKFLOW-REVIEW.md` (U2), `44-GMMU-ORACLE-AND-WORKFLOW.md` +> (the same evidence discipline), `03-SCRIPTING.md` (hooks/watch/memory), +> `16-CUSTOM-SIMULATOR.md` (backend design), `45-RM-RPC-CAPTURE-AND-REPLAY.md` +> (the capture whose sequence a rehost can seed). + +## 1. What it does + +* Loads an ELF image into the native machine exactly like `sim.gsp_boot` + (device registry from the profile, optional boot shim, libos syscall model, + lenient MMIO). +* Writes caller-supplied memory (`--mem addr=hex`), sets `a0..a7` + (`--arg`), installs a return trampoline (`ra` + `ebreak` sentinel) and a + private stack, and starts at `--entry` (hex address or ELF symbol). +* Runs under `--budget` instructions and a wall-clock `--timeout`, stopping on + return (`R_BP` at the sentinel), fault, unhandled ecall, timeout or budget. +* Snapshots regions before/after (`--snap addr:size`) with per-region changed + bytes, first-difference offset and sha256s; dumps nonzero registers on entry + and exit; records a `greeneye-session/1` directory and a + `greeneye-rehost/1` report. + +``` +python3 harness/rehost.py run --entry 0x5b01210 --arg 0 --budget 50000 \ + --snap 0x1000000:64 --out dumps/rehost-gate +python3 harness/rehost.py demo # synthetic self-test (demo firmware) +``` + +## 2. Report schema (`greeneye-rehost/1`) + +`status` is one of `return | fault | ecall | timeout | budget`; `fault` +carries the trap cause and `tval`; `retval` is `a0` at the stop; snapshots +carry the before/after diff. The report states its limits explicitly: it is +**not a boot** (the function ran with only the supplied state), a fault is the +**first missing state observed**, not a product defect, and there is no +timing claim. + +## 3. First probes on the 570.144 image (2026-10-06) + +All offline, one bounded call each, `a0=0`, budget 300k (numeric vaddrs from +the campaign analysis): + +| entry | status | stop pc | `a0` at stop | insns | fault | +|---|---|---|---|---|---| +| GR per-buffer commit gate `0x5b01210` | fault | `0x0` | `0x40` | 24 | illegal @ `0x0` | +| per-buffer commit `0x5b00d2c` | budget | `0x5b0110c` | `0x40` | 189 | — | +| commit caller `0x5b00dd8` | budget | `0x5b0120c` | `0x40` | 110 | — | +| engine-ctx memdesc install `0x514cab8` | fault | `0x0` | `0x429b450` | 97 | illegal @ `0x0` | +| promote handler `0x526a530` | budget | `0x5244134` | `0x40` | 167 | — | +| per-entry commit `0x5290a48` | budget | `0x5897fdc` | `0x40` | 258 | — | +| engine-class commit `0x5350078` | budget | `0x5350484` | `0x40` | 197 | — | + +Reading: with no engine-context state, the whole gate family bottoms out in +`NV_ERR_INVALID_STATE (0x40)` — the same family the live campaign observes — +and the memdesc-install path faults on a NULL callback early. That is a +reproduction of known behaviour, not an answer: the next step is to **seed the +context objects** (from the B4 message-loop state or from crafted structures) +and A/B the promote entry shapes against these entries. + +## 4. Admission record (draft — pending maintainer filing) + +Per the project admission rule (docs/eng-ops.md §2), all five fields: + +| Field | Value | +|---|---| +| Scenario | dynamic-RE loop: run the GR promote/commit functions of the real 570.144 image off-hardware at a chosen entry with crafted state, to answer server-side hypotheses (gate/entry/context requirements) without hardware cycles. | +| Measurable outcome | sessions per hypothesis; wall time per probe; zero hardware cycles; rehost status/fault map. | +| Baseline | the live campaign's modes 4–22 day (docs/39: 730+ lines of static RE, three wedge-reboot cycles for one accepted root cause). | +| Evaluation threshold | ≥1 wall hypothesis answered with a session pair and **zero** hardware cycles; probe loop ≤ 2 minutes per entry (measured: ≈0.1 s per probe); deterministic status/`a0` across runs. | +| Failure signal | a probe cannot be driven because required device semantics are absent, and crafted-state seeding cannot reach the failing branch after two bounded attempts → record the gap and fall back to static + physical. | + +## 5. Limits + +* Not a boot: entry-state dependence is the whole difficulty; a `budget` stop + usually means the entry is mid-function or the path loops without returning. +* The first probes use caller-supplied `a0`; the real argument conventions and + context layouts are to be supplied by seeding (or by B4/B5 state). +* Single hart, flat memory, no interrupts/scheduling; unknown MMIO is lenient + and counted, and can mask environment dependence. +* No cycle/timing/security claim; L0/L1 fidelity only. diff --git a/debug/risc-v-gsp-rm-vm/docs/47-VALIDATION-PREDICATE-PROBE.md b/debug/risc-v-gsp-rm-vm/docs/47-VALIDATION-PREDICATE-PROBE.md new file mode 100644 index 0000000..8d17859 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/47-VALIDATION-PREDICATE-PROBE.md @@ -0,0 +1,172 @@ +# 47 — Validation-predicate probe (`valprobe`) — in development + +> **Status: in development (2026-10-06, wall-directed wave).** Documents +> `harness/valprobe.py`: run one firmware **validation branch** of the real +> image with crafted register/memory state (plus, where needed, one recorded +> inline stub for a dependency outside the predicate) and record the branch +> the firmware took. Born from the live GR wall's question class: *which +> control/entry shapes does 570.144 GSP-RM accept or refuse, and why?* +> +> Development-only: executes firmware on the simulator. Never touches a GPU, +> kernel module, mdev, guest or network. No timing/cycle claim (L0/L1). +> +> **See also:** `46-SURGICAL-REHOST.md` (whole-function rehost — the parent +> tool), `44-GMMU-ORACLE-AND-WORKFLOW.md` (evidence discipline), +> `39-RE-WORKFLOW-REVIEW.md` (time-to-value ranking), `06-ACCURACY.md` +> (fidelity levels). + +## 1. Why it exists + +The live GR campaign submits `GPU_PROMOTE_CTX` shapes to the real GSP-RM and +reads accept/refuse per supervised cycle. The decisive questions are +firmware-side predicates, but a whole-function rehost (`docs/46`) early-outs +on missing RM object state (`0x40`) before the predicate executes. This +harness executes the predicate branch itself: + +* **entry**: the validation branch point (not the full handler); +* **state**: exactly the registers/memory the branch reads, crafted in RAM; +* **stub**: at most one `lui/addi/ret` stub for a dependency that is *not* + the predicate (e.g. a lock/table walk), recorded in the report; +* **outcome**: execution stops on labelled outcome addresses; the verdict is + the branch taken, with the firmware's own arithmetic/indexing. + +## 2. Invocation + +```bash +# Bundled matrix: the 570.144 PROMOTE_CTX per-buffer validation (FECS etc.) +python3 harness/valprobe.py sweep # crafted-state, ~2 s +python3 harness/valprobe.py sweep --boot-to B3 # boot the image first +python3 harness/valprobe.py demo # matrix twice (determinism) +python3 harness/valprobe.py run --spec my-spec.json # generic spec +``` + +A spec is `greeneye-valprobe-spec/1` with a `target`, an `outcomes` map +(address → label), optional `boot_to`, and `variants` carrying `registers`, +`memory` (`hex|u64|u32|u16|u8`), `stubs` (`addr` + `return`), `reads` +(post-run) and `expect`. Report schemas: `greeneye-valprobe/1` per variant, +`greeneye-valprobe-matrix/1` for the run (`valprobe-matrix.json`), with one +`greeneye-session/1` per variant when recording is on. + +Outcome statuses: `outcome` (a labelled address was reached — the verdict), +`return` (`ra` sentinel), `fault` (first missing state), `ecall`, `timeout`, +`budget`. `pass` compares `outcome` to `expect`; `sweep` exits 3 on any fail. + +## 3. The current wall, answered offline (570.144, GR PROMOTE_CTX) + +Live facts this reproduces (V6 KMD-managed channel, supervised cycles): +`PM`/`PATCH` init accepted, **FECS_EVENT init refused `0x1f`**, FECS VA-only +accepted; engine dispatch still absent. The handoff's suggested next hardware +experiment was "find a FECS init form V6 accepts (e.g. a sysmem attr 0x6 / +different size / different bufferId)". + +The probe enters the per-buffer validation branch `0x5290dd4` +(`FUN_ram_05290a48`) and executes the firmware's own logic. Firmware facts +(disassembly, numeric addresses only): + +| # | Fact | +|---|---| +| 1 | `bInitialize=1`, `bufferId==9` (FECS) takes the branch at `0x5290e3c` | +| 2 | that branch reads the **global GR-context memDesc slot**: `instance = pGpu[(idx+0x37c)]`, `table = *(u64*)(instance+0x190)`, slot index from the firmware's own `0x531b944` jump table (**FECS → 6**), and returns `NV_ERR_INVALID_ARGUMENT` (`0x1f`) **iff `slot != 0`** | +| 3 | `physAttr`, `size`, `gpuPhysAddr`, `gpuVirtAddr` are **not read** in that branch | +| 4 | `bInitialize=0`: `bNonmapped && !bInitialize` → `0x1f`; otherwise the entry passes | +| 5 | init is allowed for ids 0..2, 8..11; refused for 3..7 and 12; ids > 12 refused a block earlier | + +So **if the server has already populated the FECS slot, no init form exists** +— the aperture/size variants cannot change the verdict, and the only accepted +FECS shape is `bInitialize=0` (a VA booking, matching the live VA-only +accept). The campaign can redirect from "find a FECS init form" to +"prevent/clear the server's FECS slot" or "accept VA-only and look at +dispatch elsewhere" without spending a cycle on the aperture test. + +Matrix (15/15, deterministic; run twice with identical verdicts; 1.9 s +crafted-state, 13 s with `--boot-to B3`): + +| variant | verdict | +|---|---| +| FECS init vidmem, slot non-NULL | `refuse-fecs-slot-exists-0x1f` | +| FECS init vidmem, slot NULL | accept (commit path) | +| FECS init **sysmem attr 0x6**, slot non-NULL | `refuse-fecs-slot-exists-0x1f` | +| FECS init **sysmem attr 0x6**, slot NULL | accept | +| FECS init size 0x1000, slot non-NULL | `refuse-fecs-slot-exists-0x1f` | +| FECS no-init mapped, slot non-NULL | accept (skip path) | +| FECS no-init nonmapped | `refuse-invalid-argument-0x1f` | +| PM init / PATCH init | accept | +| GFXP_CTRL_BLK (8) init | accept (skip path) | +| PRIV (10) / UNR (11) init | accept (skip path) | +| BUNDLE_CB (3) / GLOBAL_PRIV (12) init | `refuse-invalid-argument-0x1f` | +| unknown id 13 | `refuse-invalid-argument-0x1f` | + +Each variant's report also carries the firmware-mapped slot index read back +from the run (`6` for FECS) and the applied stub, so the data flow is in the +evidence, not in prose. + +### 3.1 The FECS buffer is global and one-shot (live-confirmed 2026-10-06) + +The probe's refusal condition was then joined to a live backend run and the +semantics are now settled: + +* The FECS memDesc lives in a **global GR context-buffer table shared across + channels** (per GPU/client, per gfid); the table's only writer is the + init-promote path, and a successful init leaves the slot set so that + **every later `bInitialize=1` FECS promote is refused `0x1f`** — independent + of aperture, size, address or channel. +* A live run showed the decisive ordering: the backend's own **golden-channel + 9-entry atomic promote** (including `FECS_EVENT bInitialize=1`) was accepted, + and the per-entry FECS promote on a *different channel of the same client* + was refused `0x1f` **~0.1–0.3 s later**. The refusal was the shared + one-shot rule, not a channel deficiency. +* The per-channel requirement for FECS is therefore only the **VA booking** + (`bInitialize=0`), which the same live channel already accepted. No client + "FECS init form" exists once any earlier promote initialized the shared + buffer; an atomic multi-entry set containing FECS init fails as a whole + after that point. + +This is what the probe class is for: it turns a supervised-cycle guessing loop +into a seconds-long offline verdict plus one log join. + +## 4. Honesty rules + +* A probe is **not a boot** and **not a model**: only the seeded state exists; + the report lists entry, registers, memory, stubs, outcomes and limits. +* A stub replaces an explicitly named dependency outside the predicate; the + predicate, its id→slot mapping call and its memory reads are the firmware's + own code. Prefer entering after the dependency when that is safe. +* A fault is the **first missing state observed**, not a product defect; a + timeout/budget is not a verdict. +* `--boot-to <rung>` runs the real image first so predicates that need + initialized globals can be probed with real state; the report records the + observed rung and whether the request was reached. +* No timing/cycle claims; sessions and matrix JSON are the evidence. + +## 5. Tests + +`tests/test_valprobe.py` — stub encoding decodes as `lui/addi/jalr`, memory +spec widths, spec shape, scratch layout bounds, a live reduced matrix with +determinism, the firmware slot-index read-back (asserts `6`), boot-to-B3, and +CLI exit codes. Live tests skip without the shipped image. + +```bash +python3 -m unittest tests.test_valprobe -v +python3 harness/valprobe.py demo +``` + +## 6. Problem classes and time-to-value + +Use this tool when the question can be stated as *"what will this firmware +branch do with input/state X?"* — control-entry validation, accept/refuse +rules, error-path predicates, field-dependency questions. It answers offline, +in seconds, with labelled outcomes and a session; the alternative is static +reading (rule known, cause not), or a supervised hardware cycle per variant +(minutes to an hour plus teardown risk). + +The reference case (2026-10-06): a wall that had consumed five-plus supervised +cycles and was still misdiagnosed as a channel limitation was closed by a +15-variant probe (≈2 s, run twice, identical) plus one log join. The rule to +copy: phrase the question as a predicate → probe it offline → join the verdict +to live logs → spend the supervised cycle only on what the probe cannot see +(the engine/silicon side). + +Companion tools: `harness/rehost.py` (docs/46) for whole-function calls with +crafted state and snapshots; `sim.gsp_boot` for boot-state evidence (rung and +state); `harness/triage.py` / `version_diff.py` for release-to-release +questions. diff --git a/debug/risc-v-gsp-rm-vm/docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md b/debug/risc-v-gsp-rm-vm/docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md new file mode 100644 index 0000000..c412f21 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md @@ -0,0 +1,179 @@ +# 48 — GMMU walk & launch memory-contract audit (in development) + +> **Status: in development (2026-10-06, engine-path wave).** Documents the +> functional page-table walker (`sim/gmmu_walk.py`) and the launch +> memory-contract auditor (`harness/launch_audit.py`). Together they answer, +> offline and deterministically, *where the engine would read/write each +> address of a submission* — the class of question behind "the front-end never +> wrote the QMD" and "the release never fired". +> +> Development-only tooling: never a shipped component, never in a guest, +> never touching hardware. Fidelity is the address view (L1); no timing or +> engine-behaviour claim. +> +> **See also:** `43-GMMU-PTE-DECODER.md` (entry decoding), `42`/`44` +> (observe-only GMMU/FBIF windows and the oracle workflow), `46`/`47` +> (surgical rehost and predicate probes), `06-ACCURACY.md` (fidelity levels). + +## 1. What the two tools are + +* **Walker** (`sim.gmmu_walk`): given a page-directory base PA and a sparse + physical-memory view of the table pages, it follows VER2 entries through all + five levels and reports the leaf mapping — PA, page size, aperture, + writable/read-only — or the exact fault (`table_missing`, + `invalid_aperture`, `pte_invalid`, `unsupported`, `va_out_of_range`). + Every level's raw entry and decode is kept in the report. +* **Auditor** (`harness/launch_audit.py`): takes a submission's **memory + contract** — the labelled VAs a launch requires (ring, inline-QMD + destination, parameter/program pages, release slot, report semaphore, ctx + buffers) — walks each leg, and classifies the result against what the host + believes: + +| verdict | meaning | +|---|---| +| `ok` | the engine can reach the leg with the required access | +| `pa_mismatch` | the walk resolves to a different PA than the host's bookkeeping (the engine writes the shadow while you check the guest page, or vice versa) | +| `unmapped` | no mapping/valid leaf: the engine's access is dropped or faults | +| `access_denied` | the leaf is read-only for a write leg | + +Schemas: `greeneye-gmmu-walk/1`, `greeneye-launch-audit-spec/1`, +`greeneye-launch-audit/1`, `greeneye-launch-audit-check/1`. + +```bash +python3 -m sim.gmmu_walk demo # captured channel-PDB vector +python3 -m sim.gmmu_walk run --pdb-pa 0x12f16e000 \ + --va 0x100000 --mem 0x12f16e000=<hex> … # ad-hoc walk +python3 harness/launch_audit.py demo # captured contract + fork classes +python3 harness/launch_audit.py run --spec audit.json +python3 harness/launch_audit.py check --spec partial.json # pages to dump next +python3 harness/launch_audit.py capture-hints # exact live dump list + skeleton +``` + +## 2. Evidence and validation + +* Geometry, field positions and encodings come from the same evidence base as + the PTE decoder (`docs/43`); the walker composes the decoder rather than + re-implementing it. +* The 4K and 2M paths are **validated against a live captured channel PDB**: + `sim.gmmu_walk demo` walks five VAs and must reproduce the DMA addresses the + capture logged (5/5, deterministic). The capture lives as numeric entries in + the module (`channel_pdb_fixture`). +* The 64K dual path is **modelled** from the format definitions and reported + with `evidence: "modeled"` until a capture validates it. Video-memory leaf + addresses are reported with `pa_complete: false` (the decoder does not claim + the high bits there). +* Unsupported encodings fail closed; an incomplete table view reports + `unmapped`, never "unreachable". + +## 3. Using it on the live launch wall + +`harness/launch_audit.py capture-hints` prints the capture recipe plus the +exact page list and a copy-paste `greeneye-launch-audit-spec/1` skeleton for +the current wall. In short: the channel's PDB base PA, a raw PA→bytes dump of +the table pages reachable from it for the contract VAs, the submitted stream's +pointers (GPFIFO ring, `SET_INLINE_QMD_ADDRESS_A/B` destination, QMD +parameter/program/release-slot addresses, report semaphore), and the host's +shadow bookkeeping for the same VAs. Put them in one spec and run the audit. + +### 3.1 `check` — iterate the capture with an empty or partial dump + +`check` runs the same walks but reports, per contract leg, the FIRST table +page the `memory[]` view is missing — the page to dump next — with the +level/index path that reached it, and a consolidated `needed_pages[]` list +(sorted, de-duplicated, each entry naming the legs that need it). A leg whose +walk faults on entries that are present (invalid aperture/PTE, unsupported +encoding) is `fault` and needs no more pages; `complete` means every table +page the walk needed is in the spec (not that any mapping is correct). + +Start with only `pdb_pa` + `contract` (empty `memory`) and let the tool drive +one dump level per iteration; then run the verdict audit on the completed +spec: + +```bash +python3 harness/launch_audit.py check --spec m7-launch.json # -> needed_pages +python3 harness/launch_audit.py check --spec m7-launch.json --out ./out +python3 harness/launch_audit.py run --spec m7-launch.json +``` + +### 3.2 Current live wall dump list (m7 evidence, 2026-10-06) + +The m7 launch silence needs these raw pages (`capture-hints` prints the same +list with the spec skeleton): + +* host-side chain (the validated 5/5 fixture): root `0x12f16e000`, pd2 + `0x12f16f000`, pd1 `0x11065e000`, pd0 `0x110663000`, pt `0x15908d000`; +* forward-window PT pages `0x1b8706000` (pdeIndex 4128) and `0x1197a6000` + (pdeIndex 4131); the geometry rule `pdeIndex = (pd1_idx << 8) | pd0_idx` + places the forward windows at pd1 index 16, and their pd1/pd0-level chain + entries are server-written (in no host log) — dump them too if they fall + outside the host pages above; +* the 16 guest-scratch-PT page DMAs (`vaBase=0x200000`, capacity 512) — the + Tier-2 shadow legs resolve through them; +* the `rmcall_gr_shadow` chunk DMAs — the engine-write shadow pages for the + high-VA legs. + +Contract VAs: QMD destination `0x204608000` (write), release slot +`0x204027f7c` (write), low-window ring/push-buffer legs `0x100000..0x104000`, +and the high-VA param/program legs, which are NOT directly mapped — they +resolve through the shadow aliases (`0x210000`/`0x220000`/`0x260000`) whose +PTEs come from the guest scratch pool. + +Reading the result: + +* `pa_mismatch` on the inline-QMD destination or the release slot means the + engine operates on a different PA than the one the post-mortem checks — fix + the bookkeeping/observation point, not the engine path. +* `unmapped` means the engine's access is dropped at translation; the fault + names the exact missing level/table — `check` names the page to dump. +* all-`ok` with a silent launch means the address path is complete: the + silence is downstream (front-end/engine/context), and the next suspect is + the context program itself (see docs/46/47 for the commit-gate probes). + +### 3.3 Demo scenarios and expected verdicts (pinned by tests) + +`demo` audits the fork classes and prints every verdict: + +| scenario | mapping (VA → PA) | host bookkeeping | expected verdict | +|---|---|---|---| +| captured channel contract | 5 low-window legs (validated fixture) | matches capture | all `ok` | +| `qmd-dest-shadow-trusted` | 0x204608000 → shadow 0x15e8fa000 | shadow PA | `ok` | +| `qmd-dest-host-checks-guest` | 0x204608000 → shadow | guest page 0x204608000 | `pa_mismatch` | +| `qmd-dest-unmapped` | 0x204608000 → unmapped | n/a | `unmapped` | +| `engine-write-shadow-host-checks-guest` | alias 0x210000 → shadow PA | guest PA | `pa_mismatch` | +| `engine-write-shadow-bookkeeping-aligned` | alias 0x210000 → shadow PA | shadow PA | `ok` | + +The last two are the next wall class after dispatch clears: CUDA output +buffers at high VAs, where Tier-1 is refused (0x1f) and the leg resolves +through the guest scratch PT pool to a shadow page. The shadow/guest PAs in +the demo are synthetic (documented in the `run_demo` docstring), not m7 +captures. + +## 4. Reuse across firmware/driver version upgrades + +Both tools are data-driven and fail closed; nothing about them encodes a +specific firmware version except the entry-encoding family: + +1. On a new version, re-run `python3 -m sim.gmmu_walk demo` and the test + module. If the encoding family changed, the walker refuses with + `unsupported` rather than guessing. +2. Extend the evidence base (decoder field tables/encoders) and the LEVELS + geometry in one place (`sim/gmmu_pte.py` / `sim/gmmu_walk.py`), add one + captured golden vector per new format, and pin it in tests. +3. Keep `firmware`/`gmmu` fields in audit specs so reports are self-describing + across versions; the audit refuses an unknown `gmmu` kind. +4. Re-capture the contract data from the new version (the capture recipe is + version-independent) and re-run; compare the two audit reports for + behaviour drift. + +## 5. Tests + +`tests/test_gmmu_walk.py` (15) and `tests/test_launch_audit.py` (20): +captured golden vector, synthetic round-trips, 2M/64K paths, all fault +classes, read-only leaves, mismatch/unmapped/access-denied verdicts, +check-mode page needs (empty memory, partial memory, complete+fault legs, +needed-pages dedup/ordering, CLI exit codes/artifact), engine-write shadow +demo verdicts, determinism, spec errors and CLI exit codes. + +```bash +python3 -m unittest tests.test_gmmu_walk tests.test_launch_audit -v +``` diff --git a/debug/risc-v-gsp-rm-vm/docs/49-SIMULATION-SURFACE-MAP.md b/debug/risc-v-gsp-rm-vm/docs/49-SIMULATION-SURFACE-MAP.md new file mode 100644 index 0000000..f3de0c9 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/49-SIMULATION-SURFACE-MAP.md @@ -0,0 +1,80 @@ +# 49 — Simulation Surface Map and Handoff-vGPU Value Case + +Status: session 12 assessment (2026-10-06). Scope: the simulator workstream +(`open-iov/debug/risc-v-gsp-rm-vm/`) and what it offers the GR campaign. +Rule: nothing goes to `Handoff-vGPU-IO.md` unless a past wall proves it saved +time. This document is that proof table and the coverage map for future walls. + +## 1. What the emulator is for (and is not) + +It boots the **real 570.144 blob** to `B3` (474k insns, deterministic, device +model + LIBOS + MPU, `greeneye-session/1` recordings) and provides +dump/inspect/inject around it. It is **not** a GR/PBDMA engine model: engine +questions are answered by hardware captures + the GMMU/launch audit, never by +simulation (L2/L3 absent, docs/06). Fidelity claims are L0/L1 only. + +## 2. Coverage map — what exists per subsystem + +| Subsystem (future-bug surface) | Tool / entry | Standard format | Status | +|---|---|---|---| +| Image/boot/portability | `sim/gsp_boot.py`, `harness/probe*`, `triage.py`, `version_diff.py` | `greeneye-session/1` | built, B3 ceiling | +| Device windows / MMIO / BARs | `sim/devices`, `harness/gmmu_debug.py`, device-map | device-map JSON + sessions | built | +| LIBOS/RTOS services, MPU | `sim/libos.py`, `sim/mpu.py` | sessions | built | +| GMMU PTE/PDE (VER2/4K/2M/64K) | `sim/gmmu_pte.py`, `gmmu_walk.py`, `gmmu_oracle.py` | walk reports | built, live 5/5 fixture | +| Launch memory contract (QMD/param/program/sem legs) | `harness/launch_audit.py` | `greeneye-launch-audit-spec/1` | built; awaiting supervised trace | +| Validation predicates / gates | `harness/valprobe.py` | valprobe spec + labelled outcomes | built (FECS 15/15, seconds) | +| Whole-function exec with crafted state | `harness/rehost.py` + snapshots | sessions | built; queued for construct gate | +| Client RPC traffic (decode/replay-prep) | `harness/rm_capture.py` | capture JSON + compare | built; decode from logs | +| Client↔server live traffic loop | **B4/B5 — missing** | (rm_capture stimulus) | strategic build | +| RM object state (clients/channels/contexts) | **state-seeding fixture — missing** | (sessions) | campaign ask #1 | +| Engine method execution (GR/PBDMA) | **not simulated by design** | captures + `launch_audit` | out of scope, L2/L3 absent | +| Instrumentation/injection | hooks/watchpoints/snapshots, SDK, MCP, Ghidra bridge | `events.jsonl`, blobs, hits TSV | built | + +## 3. Standard capture formats (already unified) + +* `greeneye-session/1`: `session.json` + `events.jsonl` + `snapshots/` + + `blobs/` + `summary.md`; render with `analyze`, diff with `compare`. +* `rm_capture` JSON: real client RPC decode/compare/stimulus preparation. +* `greeneye-launch-audit-spec/1`: `pdb_pa` + `contract[]` + (label/va/access/expect_pa) + raw table pages → per-leg verdicts + (`ok`/`pa_mismatch`/`unmapped`/`access_denied`). +* valprobe spec: crafted state + labelled outcomes → accepted/refused matrix. +* symbols DB (export) and device map (harvest/query) schemas. + +## 4. Value-to-Handoff proof (do not add unproven tooling) + +| Past wall | Emulator contribution | Measured value | Verdict | +|---|---|---|---| +| FECS `0x1f` one-shot rule | `valprobe` predicate matrix, offline | matrix in seconds vs hours of static/log work; closed lane (a) | **proven, adopted** | +| Fetch-state hypothesis (userd ctl/GET/PUT) | whole-image access scan | zero accesses proven exhaustively without a hardware cycle | **proven, closed §6** | +| GMMU/QMD mapping forks | live 5/5 channel-PDB fixture + walker | fixture and verdict classes without silicon | **proven for audit input** | +| Launch silence (`qmd_rel=5`) | `launch_audit` fork partitioned (`ok`/`pa_mismatch`/`unmapped`) | single spec run classifies the fork once trace exists | **ready, pending supervised spec** | +| Golden-channel construct `-110`/`0x52a8b04` | rehost/probe queued; static analysis answered first | honest counter-example: analysis beat emulator here | **no claim yet** | +| Engine execution/shape mismatches | none | captures + audit answer it | **never promise emulation** | + +Rule: a capability is added to Handoff-vGPU only with a row in §4 and an +eng-ops admission (scenario + metric + baseline + threshold + failure signal). + +## 5. When the campaign should consult the emulator (and when not) + +| Wall type | Consult simulator? | Tool | +|---|---|---| +| Validation/refusal predicate, unknown gate | yes, first (seconds) | `valprobe` | +| Address translation / PDB / launch contract | yes | `gmmu_walk`, `launch_audit` | +| "Does the server read/write address X?" | yes | access scan / watchpoints | +| Full client traffic interaction with server | only after B4/B5 | rm_capture + sessions | +| RM `-110` deadlock with live objects | rehost if state can be crafted; else source | `rehost` | +| Engine/silicon timing, PBDMA behaviour, real dispatch | no | hardware captures + audit | + +## 6. Queued build order (each needs the §4 admission before Handoff mention) + +1. **B4/B5 host-traffic message loop** — replay `rm_capture` stimulus into the + real blob, dump every reply/write: the generic tool for the wall class the + campaign keeps hitting ("what does the server do with our traffic"). +2. **RM-object state-seeding fixture** — execute whole construct/handler paths + with crafted channels/contexts (campaign ask #1). +3. **Per-version golden matrices** — promote stable probe/audit runs to CI so + firmware bumps are regression-checked. + +Do **not** build: GR/PBDMA engine model, speculative backends, or any tool +without a wall scenario it demonstrably solves faster. diff --git a/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md b/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md new file mode 100644 index 0000000..24ee3ce --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/50-WALL-WORKFLOW.md @@ -0,0 +1,301 @@ +# 50 — Wall workflow and contract registry (in development) + +> **Status: in development (2026-10-06, wall-directed wave).** Documents the +> wall-hell escape workflow (observe → classify → model → falsify → contract), +> the six wall classes and which tool may be used for each, the capture-request +> discipline, and the contract registry (`harness/contracts.py`, +> `harness/run_contracts.py`) that turns every solved wall into a re-runnable +> regression. +> +> Development-only: the registry re-runs offline tooling; it never touches a +> GPU, kernel module, mdev, guest or network, and it adds no measurement +> capability of its own. Fidelity stays what each wrapped tool claims (L0/L1; +> `06-ACCURACY.md`). +> +> **See also:** `47-VALIDATION-PREDICATE-PROBE.md` (`valprobe`), +> `46-SURGICAL-REHOST.md` (`rehost`), `48-GMMU-WALK-AND-LAUNCH-AUDIT.md` +> (`gmmu_walk`, `launch_audit`), `45-RM-RPC-CAPTURE-AND-REPLAY.md` +> (`rm_capture`), `19-CONTROL-DISCOVERY.md`, `40-NEW-VERSION-TRIAGE.md`, +> `25-VALUE-VALIDATION.md`, `27-TOOL-AUDIT.md`, `49-SIMULATION-SURFACE-MAP.md`, +> the private simulator handoff +> [`../../../../handoff-simulator.md`](../../../../handoff-simulator.md) +> §30–33 (read it when available; agents without private access skip it), and +> `open-iov/docs/eng-ops.md` §2/§6 (admission and automation safety). + +## 1. Why this exists: wall-hell has a shape + +The GR campaign's dominant cost is **wall-hell**: one blocking problem after +another, each paying supervised hardware cycles (5–6 min each, plus fail-safe +discipline) to learn one fact. The toolset now solves several wall classes +offline and deterministically (`docs/25` §3 records the measured value), but +knowledge kept only as prose is re-learned by the next agent and re-tested on +silicon by the next campaign. This document fixes two things: + +1. **A wall is classified before it is worked.** Each wall belongs to exactly + one of six classes; only the tool that measures that class may answer it. + Most expensive wall errors are *domain-crossing inferences* (section 3). +2. **A wall is not closed until it is a contract.** The answer becomes a + declarative, re-runnable check over the existing tool entry point + (section 6), so the next firmware bump or tool change is caught by + `make test` instead of another cycle. + +The registry is a **consumer** of proven tools, not a new probing capability: +it wraps `valprobe`, `rehost`, `gmmu_walk`, `launch_audit` and `rm_capture` +entry points as-is (docs/49 §4 lists their proven value). + +## 2. Wall taxonomy — six classes + +| Class | The question it covers | Measures it DIRECTLY | Must NOT be used to infer it | Artifact a correct answer leaves | +|---|---|---|---|---| +| **a. firmware boot/status** | Did the firmware reach rung Bx? Where is it stuck, with what stop reason/status word? | `sim/gsp_boot.py` rungs B0–B3, `harness/triage.py` (identity/probe/native_boot), probe sessions, `greeneye probe` first-fault report | engine memory tools (`gmmu_walk`, `launch_audit`) and predicate probes (`valprobe`) | `greeneye-session/1` with the rung, stop PC/status and first-fault address | +| **b. RM server behaviour** | What does the server read/write/wait on for this input; which branch/gate decides accept vs refuse; what state does a handler need? | `valprobe` (one firmware predicate branch with crafted state), `rehost` (one whole function; first missing state is the finding), `rm_capture` decode for what the client actually sent | engine-execution claims from hooks/predicate verdicts; `gmmu_walk`/`launch_audit` for server logic | valprobe/rehost report + session: entry, seeded state, stubs, labelled outcome, stop/fault | +| **c. engine memory contract** | Can the engine reach each leg of this submission (ring, QMD destination, param/program, release slot, semaphore) with the required access, and at which PA? | `launch_audit` (per-leg verdict against the engine's table view), `gmmu_walk` (the leaf mapping behind one VA) | host shadow bookkeeping as proof of engine reachability; server-side hooks; `valprobe` | `greeneye-launch-audit/1` spec+report with per-leg `ok`/`pa_mismatch`/`unmapped`/`access_denied` | +| **d. engine execution/dispatch** | Did the front-end/engine consume the entry, dispatch the method, write the release/semaphore? | Supervised capture + post-mortem decode (the only authority); `launch_audit` answers only its *address* leg | anything in this tool: no GR/PBDMA model exists by design (`docs/49` §6); page tables cannot prove method dispatch; a silent ring cannot be inferred from an all-`ok` audit | capture with the engine-side consumption evidence (GPFIFO GET/PUT movement, method/QMD execution, release/semaphore state) + post-mortem | +| **e. guest-driver/protocol interface** | What did the guest driver send (fn/cmd/params), and does a replay/another version differ? | `rm_capture` decode/compare/stimulus (send-side logs), guest diagnostics (`open-iov/tools/guest-diag`) | firmware predicate/branch results as a substitute for the driver's wire traffic; `gsp_boot` rungs | `greeneye-rm-rpc-capture/1` + `greeneye-rm-rpc-compare/1` and a stimulus JSON | +| **f. version skew/portability** | Did the image/code/behaviour move between versions, and which change explains it? | `harness/triage.py` (full chain), `harness/version_diff.py` (sections + symbols + M1–M6 skeleton), `./risc-v-gsp-rm-debugger.sh probe <image>` first-touch diff | one version's green run as portability proof; a `skipped` step read as a pass | triage report + version diff + one recorded `compare` session pair (self-diff zero) | + +Two rules fall out of the table: + +* **Class (d) has no offline tool, by design.** Do not attempt to simulate it: + the simulator makes no L2/L3 claims (`docs/06`). The offline contribution is + the *address leg* (class c) and the *server leg* (class b). +* **A wall usually names its own class in one sentence.** If two classes seem + to fit (e.g. "the launch is silent"), split the question: address leg → + (c), method execution → (d), server prewrite → (b). One wall = one class = + one decidable question. + +## 3. The evidence ladder and the no-cross-domain rule + +### 3.1 Observe → classify → model → falsify → contract + +| Stage | What happens | Exit test | +|---|---|---| +| **1. Observe** | Collect only evidence that can discriminate: one bounded session/probe/capture, named stimulus, recorded artifacts | The observed fact is replayable and has a session/report path | +| **2. Classify** | Assign exactly one wall class (section 2) and name the metric the question is really asking (M1–M6, `docs/06` §5) | The class's DIRECT tool exists and is reachable offline; if not, record the gap (section 10) | +| **3. Model** | Run the class's tool with crafted/captured state (predicate branch, rehost, walk, audit, decode) | A result that could have come out the other way (a verdict, not a confirmation) | +| **4. Falsify** | Try to break the model: inverse variant, mutated spec, second run for determinism, one-bit differential (`docs/14`) | The falsifier fails to overturn the model; the determinism `compare` is empty | +| **5. Contract** | Pin the decisive run as a registry contract with an explicit expectation | `python3 harness/run_contracts.py --fast` passes and the contract can fail if the tool changes | + +The ladder is the OODA loop of `docs/15` made runnable: model is Orient, +falsify is Decide, contract is Act/regress. Skipping stage 4 is how a +misread wall survives (the FECS refusal was initially misread as a V6 +limitation until the predicate matrix falsified it; handoff PART 95). + +### 3.2 Never infer across domains + +The rule is absolute for claims that would change the product: + +* **Host bookkeeping cannot prove engine execution.** The engine may read a + different PA than the one the post-mortem checks — that is exactly what + `launch_audit`'s `pa_mismatch` exists to expose. Only a capture shows what + the engine did (class d). +* **Page tables cannot prove method dispatch.** `gmmu_walk` proves a mapping + exists; a silent launch with all legs `ok` means the silence is downstream + of translation (front-end/engine/context), not that the method ran + (`docs/48` §3). +* **A predicate verdict cannot prove the whole handler path.** `valprobe` runs + one branch with the minimum state; the full handler may early-out earlier + or additional gates may exist downstream (`docs/47` §2; `docs/46` §5). +* **A decoded client record cannot prove what the server replied.** + `rm_capture` is send-side; captures currently have no reply sequence + (`docs/45` §7). +* **A demo-surface result is not RM state.** `control_discovery`/`reach` on + the demo surface classify demo dispatch; the RM surface waits for H1.3/H2.2 + (`docs/19` §9). +* **One version's behaviour is not portability.** A new image requires the + class-(f) chain, even when one scenario passes. + +Every claim in a handoff/write-up carries its class and its tool so a reviewer +can see the domain it came from. + +## 4. Per-tool honesty table + +Wraps the audits in `docs/27` and the limits in each tool doc. "L0/L1" means +instruction/memory fidelity only; no timing (L2) and no device-environment +(L3) claim (`docs/06` §2). + +| Tool | What it MEASURES | What it does NOT measure | Known failure modes | +|---|---|---|---| +| `valprobe` (docs/47) | The branch one firmware predicate takes with crafted register/memory state plus at most one recorded stub; the verdict is a labelled outcome address | Whole-handler behaviour, RM object state, boot state, timing | Wrong entry/state ⇒ `fault` (first missing state) or `unlabelled` outcome; a stub can mask a dependency that would change the branch; crafted state is not what a live caller supplies | +| `rehost` (docs/46) | One bounded firmware function with crafted args/memory/snapshots; the first missing state is the finding | Service loops, interrupts/scheduling, MMIO semantics, timing | `budget`/timeout usually means entry is mid-function or loops; argument conventions must be caller-supplied; lenient unknown MMIO can mask environment dependence | +| `gmmu_walk` (docs/48) | The leaf PA/page-size/aperture/access for a VA through a raw table view; exact fault otherwise | Access ordering, execution, timing; video-memory high bits (`pa_complete: false`) | Wrong root/view ⇒ fail-closed faults; 64K dual path is modelled, not live-validated; incomplete view reports `unmapped`, never "unreachable" | +| `launch_audit` (docs/48) | Per-leg class of a submission's engine-visible memory contract vs host bookkeeping (`ok`/`pa_mismatch`/`unmapped`/`access_denied`) | Whether the engine actually executed/dispatched; anything downstream of the address path | Needs a real capture spec; missing table pages read as `unmapped` (capture error, not engine truth); no fixture ⇒ demo only | +| `rm_capture` (docs/45) | Decode of real client RPC log tags into records; compare of two captures; replay stimulus preparation | Server replies/event ordering; handles/addresses (omitted from stimuli); anything >192 B payloads until the helper chunks | Unknown tags/truncated payloads (`truncated: true`) are not replayable; send-side only; caller-address symbolisation deferred | +| `control_discovery` / `reach` (docs/19) | Unknown command-id classification on a surface; a reachability route with bounded timeout and required args | RM object state/behaviour (surface waits for H1.3/H2.2); dead-code proofs | A timeout is "not reached under this stimulus", never unreachability; long handlers may classify by entry hit with empty effects; trap/fault cause mapping is a documented fidelity gap | +| `triage` / `version_diff` (docs/21, docs/40) | Image identity/sections/symbols/probe/first-touch and M1–M6 diff between two versions | Silicon behaviour; a raw `.fwimage` boots only to its early fault by design; `skipped` is not `pass` | Missing baseline ⇒ `version_diff` step `skipped`; RM 610 refuses without human approval (exit 3); first-touch differences need attribution to profile vs logic before any conclusion | + +## 5. Capture-request discipline: one capture, one decidable question + +Supervised captures are the scarcest resource; a vague request burns a cycle +and still leaves ambiguity. The rules: + +1. **State the question as one falsifiable sentence.** "Does the engine's PDB + resolve the inline-QMD destination VA to the host shadow PA?" — not + "capture more launch data". +2. **Write the acceptance test before asking.** Name the tool and the exact + report field that will decide it (e.g. `launch_audit run --spec` leg + verdict; `rm_capture compare` row deltas). If no existing consumer parses + the data, the capture is not ready. +3. **List the exact pages/registers, byte-exact.** Prefixes without + pretty-printing: PDB root, each level (PDE2/PDE1/PDE0), the leaf table(s), + the submission pointers (GPFIFO ring, `SET_INLINE_QMD_ADDRESS_A/B`, QMD + param/program/release-slot, report semaphore), and the shadow/chunk pages + the engine might write. A well-formed example: the launch-audit request + that listed the channel PDB tree + the `rmcall_gr_shadow` chunk DMAs + (simulator handoff §32; the private request holds the concrete PA list). +4. **Include the comparison basis.** The host/shadow bookkeeping for the same + VAs, so `pa_mismatch` is decidable; a second consumer's view for the same + object if the question is interface alignment. +5. **One question per capture.** Two questions ⇒ two lists; bundling makes a + partial answer unreadable. +6. **Prefer the existing ingestion format.** Raw pages → `greeneye-launch- + audit-spec/1`; RPC logs → the `rm_capture` tag grammar + (`harness/rm_capture.py --print-capture-hints`). Do not invent a format + that no tool reads. +7. **On a failed capture, record the fidelity gap, not a guess.** Missing + pages or truncated payloads become a new request or an unknowns-ledger row + (section 8), never an attribution. +8. **Gates.** The capture itself is supervised, human-run physical work + (`SIMULATOR-WORK-BOUNDARY.txt`); simulator tooling only prepares the spec + and reads the artifact afterwards. + +Anti-patterns: "capture more logs" (no acceptance test); pretty-printed +tables instead of raw bytes (breaks byte-exact ingestion); one dump for two +questions; capturing without the host-side bookkeeping map (nothing to +classify against); re-capturing because the previous request lacked the +release slot PA. + +## 6. The contract registry + +### 6.1 Concept + +Every solved wall becomes a **contract**: a declarative record that pins the +proven tool entry point and the expectation observed when the wall was +solved. New walls run the registry first - a failure localises the regression +to one class in seconds; only then is a new probe/capture justified. + +| Contract field | Meaning | +|---|---| +| `name` | stable id, e.g. `fecs-predicate-matrix` | +| `wall_class` | one of a–f (section 2) | +| `doc` | the tool/wall documentation reference | +| `entry` | the exact existing entry point wrapped, e.g. `harness.valprobe:run_spec(fecs_spec())` | +| `expect` | the pass condition in words (falsifiable) | +| `speed` | `fast` (runs in `make test`) or `slow` (only `--all`/`--name`) | +| `requires` | input paths; absent ⇒ `skip`, never `pass` | + +Rules for adding one: wrap an **existing** entry point (no new measurement +logic); the expectation must be able to fail; keep fast contracts small +(seconds total); one wall class per contract; link the doc; and if the wall +has a public write-up, link it too. A contract without a falsifier is prose. + +### 6.2 Commands + +```bash +python3 harness/run_contracts.py # fast contracts (make test path) +python3 harness/run_contracts.py --all # fast + slow +python3 harness/run_contracts.py --json dumps/contracts.json +python3 harness/run_contracts.py --fast --name gmmu-golden-vector # single contract +python3 -m unittest tests.test_contracts -v # fast contracts as a test +``` + +The runner prints pass/fail per contract and per-wall-class coverage (classes +with zero contracts are shown, so gaps stay visible); exit status is nonzero +if any selected contract fails; a `skip` is a missing prerequisite, not a +pass. Schema: `greeneye-contract-run/1`. + +### 6.3 Shipped contracts (2026-10-06) + +| Contract | Class | Wraps | Expectation | Speed | +|---|---|---|---|---| +| `fecs-predicate-matrix` | b | `harness.valprobe.run_spec(fecs_spec())` | 15/15 PROMOTE_CTX predicate variants labelled, 0 failed | fast | +| `fecs-predicate-determinism` | b | `harness.valprobe.run_demo()` | matrix twice, identical verdicts | slow | +| `rehost-demo` | b | `harness.rehost.run_demo()` | demo function returns the expected sum twice | fast | +| `gmmu-golden-vector` | c | `sim.gmmu_walk.run_demo()` | 5/5 captured VAs → logged DMA addresses | fast | +| `launch-audit-captured-contract` | c | `harness.launch_audit.run_demo()` | captured contract all `ok`; QMD hypotheses yield `pa_mismatch` + `unmapped` | fast | +| `rm-capture-decode-demo` | e | `harness.rm_capture.run_demo()` | synthetic capture decodes/compares/yields stimulus | fast | + +Coverage gaps, stated honestly: class **a** and **f** have tools (`triage`, +`version_diff`, boot rungs) but no contract yet because a meaningful fast +expectation needs a pinned baseline pair (that is the queued golden-matrix +build, handoff §30); class **d** has no offline contract by design (section +2). Classes b/c/e are covered by the five fast contracts; `make test` runs +them. + +## 7. Admission note (eng-ops §2) + +The registry/runner is admitted per `open-iov/docs/eng-ops.md` §2: + +| Field | Value | +|---|---| +| **Scenario** | Wall-directed loop: after a wall is solved offline, the answer must survive as a regression; before a new wall is worked, the known walls are re-checked; and every firmware/tool change must show whether a solved wall moved. | +| **Measurable outcome** | `greeneye-contract-run/1` summary: pass/fail/skip per contract + per-wall-class coverage; exit code and JSON artifact; `tests/test_contracts.py` in the suite. | +| **Baseline** | Re-running the same demos by hand per wall, or not re-running them at all; per-tool costs are recorded in `docs/25` §3 (0.2–6.0 s per demo path vs 5–6 min per supervised cycle). | +| **Evaluation threshold** | Keep if: (1) every fast contract passes in ≤ 10 s total in `make test`; (2) each newly solved wall gets exactly one contract within the same session; (3) a deliberate tool change can make a contract fail (falsifier proven by a synthetic failing case in `tests/test_contracts.py`). | +| **Failure signal** | A contract that cannot pin a concrete expectation; two contracts measuring the same thing; a fast contract exceeding the time budget; contracts added faster than walls are solved; or a wall re-opened with the registry green (meaning the contract did not capture the decisive run). On the first signal, prune or rewrite the contract before adding new ones. | + +## 8. Unknowns ledger (practice) + +Open questions that no current tool answers live in an **unknowns ledger**: +one row per question, with a discriminating experiment. The ledger itself is +maintained by the campaign **outside this tool**; this document describes +only the practice. Row shape: + +| Field | Meaning | +|---|---| +| Question | one falsifiable sentence | +| Discriminating experiment | the run that decides it, with the class (a–f) | +| Tool / capture | the registry entry, probe spec, or supervised capture request (section 5) | +| Status | `open` → `probe-ready` (spec exists) → `answered` → `contract` (registry) or `closed` (no action) | +| Evidence | session/report path once answered | + +Rules: a row moves to `answered` only with an artifact, and to `contract` +when the decisive run can be pinned (section 6); if the experiment needs a +class-(d) fact, the row says so and waits for a supervised capture rather +than a simulator guess; when a tool that already exists answers the row, a +new tool is not built (`eng-ops.md` §2 core rule). + +## 9. Worked examples + +* **FECS promote wall (2026-10-06, class b).** Live refusals (`0x1f`) looked + like a V6 channel limitation. Observe: refusal reproduced in logs. + Classify: firmware predicate (b). Model: `valprobe` crafted variants — + refuse iff the shared slot is non-NULL, regardless of aperture/size. + Falsify: `slot NULL` and sysmem-attr variants accept; a second matrix run + is identical. Contract: `fecs-predicate-matrix` + + `fecs-predicate-determinism`. Result: the refusal is the shared one-shot + rule, triggered by the campaign's own earlier promote; FECS is exonerated + (handoff PART 95; `docs/47` §3). +* **Launch silence (`qmd_rel=5`, class c → d).** Classify: the address leg is + (c), method execution is (d). Model: `launch_audit` over the captured + channel fixture partitions `ok`/`pa_mismatch`/`unmapped`; the downstream + question is not simulatable. Falsify: pending the real + `greeneye-launch-audit-spec/1` raw-page capture (section 5). Contract: + `launch-audit-captured-contract` covers the fixture today; the real-spec + validation is handoff §31 item V1. +* **Construct/state walls (`0x40`/`0x52a8b04`, class b).** `rehost` runs the + function and stops at the first missing state — the finding, not a defect. + The missing piece is the RM-object state-seeding fixture (handoff §30 ask + #1); until it exists those walls have no contract, and the registry run + before starting them shows exactly that gap (class b has only demo/known + inputs). +* **Engine dispatch (class d).** No offline contract exists by design. The + registry's coverage line for class d is the reminder that the answer must + come from a supervised capture + post-mortem, with `launch_audit` used + only for its address leg. + +## 10. Limits, status and open items + +* **Status (2026-10-06).** Registry, runner and tests implemented; 6 + contracts (5 fast, 1 slow) at `greeneye-contract-run/1`; fast run ≈ 2 s; + `tests/test_contracts.py` green. Coverage: b (3), c (2), e (1); a/f/d + open as described in §6.3. +* The registry measures nothing new: if a wrapped tool's own demo/test is + wrong, the contract inherits that error. The `docs/25` demos and the + per-tool test modules remain the deeper checks. +* No timing, security or device-environment claim (L0/L1 only); injected + code, modified images and dumps stay development-only. +* Open items: pin the class-a/f contracts once a stable baseline pair exists + (golden matrices, handoff §30/§31); re-validate the launch-audit contract + against the first real `greeneye-launch-audit-spec/1`; add a contract only + when the wall it closes has an artifact. diff --git a/debug/risc-v-gsp-rm-vm/docs/51-HOST-TRAFFIC-LOOP.md b/debug/risc-v-gsp-rm-vm/docs/51-HOST-TRAFFIC-LOOP.md new file mode 100644 index 0000000..e56094c --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/51-HOST-TRAFFIC-LOOP.md @@ -0,0 +1,253 @@ +# 51 — Host-traffic message loop: doorbell + msgq transport + replay (in development) + +> **Status: in development (2026-10-06, B4/B5 transport wave).** This page +> documents `sim/host_traffic.py`, the transport half of boot-ladder rungs +> B4/B5 (docs/35-GREENEYE-SIMULATOR-DESIGN.md §3.3): a modelled host +> doorbell, a command/status message-queue pair laid out with the real queue +> arithmetic, queue-element framing with the real checksum, a stub dispatcher +> and deterministic replay of an `rm_capture` stimulus (docs/45). The **real +> 570.144 firmware is not run by this module**: there is no real message loop, +> no RM semantics and no firmware-behaviour claim. +> +> Development-only, offline: no GPU, kernel module, mdev, guest, network, +> QEMU, `/dev` access or reboot. Nothing here is a product dependency and +> nothing is injected into a guest. +> +> **See also:** `35-GREENEYE-SIMULATOR-DESIGN.md` §3.3 (ladder), `36` (device +> model; the falcon window this doorbell overlays), `45` (capture decode and +> stimulus), `49` (surface map; B4/B5 is queued build #1), `06-ACCURACY.md` +> (L0/L1 only), `open-iov/docs/eng-ops.md` §2 (admission rule). + +## 1. What this is and why + +The campaign keeps hitting one wall class: *what does the server do with the +traffic we send it, and what does it read/write/wait on for this control*. +The strategic tool for that class is B4/B5: replay a captured real RPC +sequence into the emulated GSP through an emulated transport and record the +replies and memory writes as evidence (handoff-simulator §27, §30, §31 item +B1; docs/49 §6 build order #1). + +This wave lands the **transport and framing** so the replay plumbing exists +and is tested before the firmware loop is connected: + +* a doorbell register model at the real hardware offset; +* command/status msgq rings in simulated memory, laid out with the arithmetic + the real host queue code uses (including the `MSGQ_FLAGS_SWAP_RX` + cross-link of the two rx headers); +* queue-element and RPC framing with the real header layouts and checksum; +* a 2-handler stub dispatcher plus `NV_ERR_NOT_SUPPORTED` for real RM + functions, so a message can travel host → command ring → dispatch → status + ring → host and be recorded; +* replay of a `greeneye-rm-rpc-stimulus/1` document (or a capture document + converted by `harness/rm_capture.py`), message by message, in order. + +The dispatcher is deliberately a **stub**: answers are labelled +`stub:echo`, `stub:status_query`, `stub:unsupported` in the transcript. A +stub verdict is never presented as firmware behaviour. + +## 2. Admission record (draft — pending maintainer filing) + +Copied from the queued execution item (private live contract §31 item B1), +expanded to the `open-iov/docs/eng-ops.md` §2 five-field format; a maintainer +files the final record. One scenario, one axis (axis 4, bug-analysis time). + +```text +TOOL ADMISSION RECORD id: TOOL-2026-XX (draft) +tool: sim/host_traffic.py (new; B4/B5 transport + replay wave) +loop: dynamic-RE (campaign traffic-interaction walls) +axis: 4 — bug-analysis time (supervised cycles avoided / TTFA) +owner/decider: <maintainer>; executor: agent +scenario: a campaign traffic-interaction wall (e.g. the V6 launch + silence) asks what the server does with a given control or + reply traffic; replay the captured stimulus into the modelled + transport and read the server-side evidence offline. +measurable: wall resolved from emulator evidence alone or with < 1 + supervised cycle; per wall: a session/transcript pair with + every submitted message, dispatch handler and reply recorded. +baseline: today = static decompiler dig + one or more supervised + hardware cycles per hypothesis (unrecorded as a single + number; docs/49 §4). +threshold: keep if 2 of the next 3 wall-class instances are answered + offline; N >= 3 wall-class instances; environment: this tool + checkout, simulator only. +failure signal: replies diverge from captured server behaviour, or the tool + cannot represent the control/queue shape -> record the + fidelity gap and stop (no emulation edit from simulator-only + evidence). +evidence: tests/test_host_traffic.py (36 tests) + `demo`/`replay` + session directories under the `--out` directory. +decision: admit-pilot +review-by: one quarter / five sessions trials-dir: dumps/ +``` + +Queued-item exit criterion (handoff §31 B1): *one real 512-byte RPC control +replayed end-to-end with the reply captured in a session*. This wave's demo +replays a **real-shaped** 512-byte `GSP_RM_CONTROL` (built to the documented +control shape, from no capture) end-to-end through the transport; its reply +is the stub dispatcher's framed `NV_ERR_NOT_SUPPORTED`, captured in a session. +`replay` consumes a real stimulus document the moment a supervised capture +provides one — the capture itself is still pending, so no captured-sequence +claim is made yet. + +## 3. What the transport is based on (real offsets and structures) + +All facts below come from public OpenRM register/queue header material (the +same class of evidence the tracked device model already uses). The capture +tool exposes the overlapping facts as constants (`RPC_HEADER_BYTES`, the +`GSP_RM_*` ids, the payload offsets), not as struct types, so this module +mirrors them and `test_rpc_constants_match_capture_tool` fails if they drift. +The models are L0/L1 framing and ordering only; no timing claim. + +| Fact | Value | Use in the model | +|---|---|---| +| GSP queue-head register | `NV_PGSP_QUEUE_HEAD(i) = 0x110c00 + i*8`, 32-bit RW, i < 8 | doorbell window `PRIV+0x110c00`, 0x40 bytes; reached by the firmware through the PRIV aperture | +| Host kick | the 570.144 submit path writes the queue head after queueing (observed value 0) | `HostTraffic.submit` rings queue 0 with 0; the recorded value is the firmware-visible read-back | +| Falcon mailbox | `NV_PGSP_FALCON_MAILBOX0/1` 0x110040/0x110044 | already modelled by `sim/devices/falcon.py`; not duplicated here | +| `msgqTxHeader` | 8 x u32: version, size, msgSize, msgCount, writePtr, flags, rxHdrOff, entryOff | `MsgqTxHeader` / `MsgqQueue` | +| `msgqRxHeader` | 1 x u32: readPtr | `MsgqQueue.set_read` / `.read_ptr` | +| msgq arithmetic | `rxHdrOff = align32(16) = 32`, `entryOff = align36(16) = 48`, `msgCount = (size - entryOff) / msgSize` | `MsgqQueue.format` reproduces it exactly | +| Queue flag | `MSGQ_FLAGS_SWAP_RX = 1`; the real command queue sets it and `msgqRxLink` cross-links the rx headers between the command and status buffers | `SharedQueues` places the command rx header in the status buffer and vice versa | +| `GSP_MSG_QUEUE_ELEMENT` | 16 B authTag + 16 B aad + checksum + seqNum + elemCount + 4 B reserved + 32 B `rpc_message_header_v` (payload at 0x50) | `build_queue_element` / `parse_queue_element` | +| Element/queue sizes | element min 0x1000, max 0x10000; default command and status queue 0x40000 each; header size 0x1000; header align 4, element align 12 | defaults in the module; a message stages contiguously and splits across ring slots exactly as the real producer copies it | +| Multi-element | elemCount in the first element; the contiguous staged stream is split into 0x1000-byte slots (tail zero-padded); checksum covers the staged stream | `split_element_stream` + `SharedQueues._place`/`_take` (wrap-safe); max 16 elements | +| Checksum | XOR of 64-bit words over `0x30 + rpc.length` bytes, then hi32 ^ lo32; the stored field makes the receiver's recomputation zero | `compute_message_checksum` / `build_queue_element` (verified) | +| RPC header | 32 bytes: header_version (0x03000000), signature ("VRPC" 0x43505256), length (= 32 + payload), function, rpc_result, rpc_result_private, sequence, spare | `build_rpc_message` / `parse_rpc_message` | +| RM ids | `GSP_RM_CONTROL = 76`, `GSP_RM_ALLOC = 103`, `GSP_GET_GSP_STATIC_INFO = 65` (same as the capture tool) | replay mapping for `control`/`alloc` records | +| NV status | `NV_OK = 0`, `NV_ERR_NOT_SUPPORTED = 0x56` | stub replies | + +Modelled rather than proven, and therefore labelled: + +* the **shared memory address and queue sizes** are fixture defaults; the real + driver learns them from the `GSP_ARGUMENTS_CACHED` / message-queue init + arguments. Nothing here reads a real `MESSAGE_QUEUE_INIT_ARGUMENTS`. +* the **doorbell read-back** is storage of the last written value (the header + says RW); the firmware's read use of this register has **not** been observed. +* the **command/status direction** is the documented one (host writes the + command queue, GSP writes the status queue), but no real GSP has yet been + observed consuming or producing through this model. +* the **MSI-X/event path** by which the real GSP signals a status-queue write + is not modelled; replies are read by polling (`read_reply`). + +## 4. Files and API + +| File | Contents | +|---|---| +| `sim/host_traffic.py` | framing (`build_rpc_message`, `parse_rpc_message`, `build_queue_element`, `parse_queue_element`, `compute_message_checksum`), `MsgqQueue`, `SharedQueues`, `DoorbellModel`, `DoorbellBlock` (a `sim.devices` overlay), `HostTraffic`, `load_stimulus`, `replay_stimulus`, `run_demo`, CLI | +| `tests/test_host_traffic.py` | 36 offline tests: framing/checksum, queue arithmetic + cross-link, doorbell model + guest instruction access, submit/dispatch/reply, determinism, replay (stimulus + capture doc), recorder events, demo + CLI | +| `docs/51-HOST-TRAFFIC-LOOP.md` | this page | + +`HostTraffic` surface: + +* `submit(message)` / `submit_rpc(function, params, sequence=None)` — frame, + write one command element, advance `writePtr`, ring the doorbell (recorded). +* `poll()` — GSP-side drain of the command ring through the handler table, + writing framed replies to the status ring; returns the dispatched count. + Bounded by construction (each iteration consumes one element). +* `read_reply()` — host-side consumption of one status element. +* `run_cycles(budget)` — run the attached `sim.machine.Machine` for a bounded + budget (cap 2M) and then `poll`; without a machine it only polls. +* `transcript()` / `state()` / `close()` — deterministic transcript, queue and + doorbell state; `close()` writes `blobs/host-traffic.json` when a recorder + is attached. +* `doorbell_block()` — a `DoorbellBlock` sharing the same `DoorbellModel`, so + the guest-visible register and the host model are one state. Attach it with + `machine.devices.add(host.doorbell_block())`; the narrow window overrides + the wider falcon window only for `0x110c00..0x110c3f`. + +Records: `greeneye-host-traffic-{demo,replay,log,reply}/1`. The transcript has +no host timestamps and is byte-identical across runs; the optional +`greeneye-session/1` recorder events (`host_traffic_config`, `doorbell`, +`host_submit`, `host_dispatch`, `host_reply_read`, `host_replay_skip`, +`host_transcript`) follow the standard event shape, where `t` is host-timed +and non-semantic. + +## 5. How to run it + +```bash +cd open-iov/debug/risc-v-gsp-rm-vm + +# Deterministic end-to-end demo (echo + status query + real-shaped 512-byte +# GSP_RM_CONTROL; stub replies), with a recorded session: +python3 -m sim.host_traffic demo --out dumps/host-traffic-demo + +# Replay a stimulus document (or a capture document / raw capture log): +python3 -m sim.host_traffic replay --stimulus dumps/rm-stimulus.json --out dumps/host-traffic-replay +python3 -m sim.host_traffic replay --stimulus capture.json --allow-truncated + +# Tests: +python3 -m unittest tests.test_host_traffic -v +``` + +`replay` maps stimulus records as follows: `send` carries the wire function +and payload directly; `control` and `alloc` records carry a named id and are +mapped with the capture tool's constants (76 / 103) using the captured +parameter block as the payload. That identity is an **assumption** until a +capture with full send-side payloads exists (the current runbook helper also +truncates payloads above 192 bytes; truncated messages are refused by default +and skipped with `--allow-truncated`). + +## 6. Real vs stubbed — the honest table + +| Piece | State | +|---|---| +| RPC header + queue element framing, checksum | **real** (mirrored; drift-tested against `harness/rm_capture.py` constants) | +| msgq header arithmetic + `SWAP_RX` cross-link | **real** (mirrored from the public queue header and `msgqTxCreate`/`msgqRxLink`) | +| Doorbell register offset/width/access | **real** (public register header); read-back is storage of the last write, firmware read use unobserved | +| Guest access to the doorbell window through `Machine` | **real** (device window + recorded store PC); exercised by instruction-level tests | +| Command/status rings in simulated memory | **real layout**, fixture addresses and sizes | +| Multi-element messages | **real split** (contiguous staged stream across ring slots, wrap-safe, checksum over the stream) | +| Dispatcher (`echo`, `status_query`) | **stub** (ids 0x40/0x41 are not RM functions) | +| Reply to real RM functions | **stub** `NV_ERR_NOT_SUPPORTED` — framed and transported exactly; not firmware behaviour | +| Real firmware message loop | **not run** (B5 remains open) | +| RM semantics (controls, allocs, objects, events) | **absent** | +| MSI-X / GSP→host interrupt | **not modelled** (polling) | +| Timing/cycles | **not claimed** (L2 absent; `run_cycles` is an instruction budget) | + +## 7. What this does NOT do yet (and the next step to B5) + +Not done: booting the real image and serving its message-loop poll; reading +the real `GSP_ARGUMENTS_CACHED`/queue-init parameters; event/MSI-X delivery; +any RM handler. + +Next step for B5, concretely (each point is bounded and session-recorded): + +1. Boot the real 570.144 image to B3 (`sim.gsp_boot`, 474k instructions, + stop at the message-loop port wait, `ecall` at `0x4018038`). +2. Attach `host.doorbell_block()` to the boot registry before the run, so the + firmware's first queue-head access is served by the model and recorded, + and place the command/status rings at addresses the firmware is told + (stub: the fixture block; real: the addresses from a captured + `GSP_ARGUMENTS_CACHED`). +3. Ring the doorbell from the host model and continue the machine with + `run_cycles` until it polls the rings; classify the first poll: which + address does it read (queue header? Tx/Rx header? entry?) and what + read-back semantics does it need. +4. Compare the firmware's observed ring arithmetic against this module's + (field-by-field); where the real init used `MSGQ_FLAGS_SWAP_RX`, the + cross-link must already match; fix only with a recorded session pair + (docs/06 accuracy loop), never from a decompiler reading alone. +5. Then replay one captured stimulus message with the real image and record + the reply/writes as the B5 session. Until a supervised capture exists, + this stops at step 3–4 with the stub dispatcher. + +The seam for step 3 already exists: `HostTraffic(machine=..., shared_base=...)` +uses the machine's RAM for the rings and `run_cycles` steps the core; the +doorbell block overlays the falcon window without touching +`sim/devices/falcon.py` or the profile. + +## 8. Limits, deferrals and verification + +Limits: L0/L1 only; no capture has been consumed yet (no supervised runbook +execution); control/alloc payload identity is an assumption; truncated +payloads are refused or skipped, never padded; a stub reply is never a +firmware claim; no real queue-init parameters. + +Deferred: real-firmware B4/B5 wiring (above), event/MSI-X delivery, RM-object +state seeding (docs/49 build #2), golden matrices (#3). + +Verification of this wave: `python3 -m unittest tests.test_host_traffic -v` +→ 36 tests green; full suite `make test` green; `demo` `ok: true` with all +checks (`echo` payload round-trip, status magic, 512-byte control replay, +framed stub verdict, one doorbell ring per submit, all replies read); +`ruff check` clean on the new files. Two-run determinism is asserted by +tests (transcript and replies byte-equal). diff --git a/debug/risc-v-gsp-rm-vm/docs/52-RM-OBJECT-STATE-SEEDING.md b/debug/risc-v-gsp-rm-vm/docs/52-RM-OBJECT-STATE-SEEDING.md new file mode 100644 index 0000000..1dc6203 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/52-RM-OBJECT-STATE-SEEDING.md @@ -0,0 +1,297 @@ +# 52 — RM-object state seeding (`state_seed`) — in development + +> **Status: in development (2026-10-06, state-seeding wave).** Documents +> `harness/state_seed.py`: a declarative **seed** of RM object state +> (`greeneye-state-seed/1`) that is placed into the same native machine +> `harness/rehost.py` uses, then run at a chosen entry under a bounded budget. +> The tool records which seeded state the firmware reads, in order, and the +> **first gate** it stops at: the address and the state the returning/faulting +> path checked. This is the state-seeding fixture the campaign asked for: +> rehost early-outs on missing objects; a seed supplies the +> objects so the next blocker is named instead of guessed. +> +> Development-only: executes firmware on the simulator. Never touches a GPU, +> kernel module, mdev, guest or network. No timing/cycle claim (L0/L1). +> +> **See also:** `46-SURGICAL-REHOST.md` (the runner this reuses), +> `47-VALIDATION-PREDICATE-PROBE.md` (the predicate probe the mechanism proof +> replays), `45-RM-RPC-CAPTURE-AND-REPLAY.md` (the capture whose objects a +> seed can mirror), `06-ACCURACY.md` (fidelity levels), `39-RE-WORKFLOW-REVIEW.md` +> (time-to-value rankings). + +## 1. Why it exists + +A cold rehost of a construct/promote entry on the 570.144 image bottoms out in +the error family the live campaign observes (`0x40` invalid state, or an +illegal instruction at `pc 0x0` after a NULL indirect call), because the RM +object state the handler walks -- the client/device object, the channel or +context object, the graphics instance and its memDesc table -- does not exist. +The first rehost probes could only record the stop (`docs/46` §3); they could +not say *which object* was missing, so the next step was still a guess. + +`state_seed.py` makes the missing state explicit: + +* a **seed** declares symbolic objects (clients, channels, graphics contexts, + memdescs/context buffers, generic RM objects), their fields, an allocator + that places them at chosen addresses, and pointer fields that reference each + other by name; +* a **loader** compiles the seed into byte writes and register values and + applies them to the native machine through the same wiring `rehost.py` + uses; +* a **runner** executes the entry under a bounded budget with watchpoints on + every seeded field and records the ordered first-touch state reads, then + reports the first gate: the last seeded state read before the stop plus the + stop reason. + +A field whose value is not known is left at the literal `"unknown"` marker: +the seeder does not write it and the report lists it. A gate that reads an +`unknown` field is a finding (the minimal object set is incomplete), not a +seed bug. + +## 2. Schema `greeneye-state-seed/1` + +```json +{ + "format": "greeneye-state-seed/1", + "name": "construct-gate-minimal", + "entry": "0x52a70ec", + "allocator": {"base": "0x10010000", "align": "0x1000"}, + "stack": "0x1003e000", + "registers": {"a0": "@client", "a1": "@channel", "a2": "0x1003f000"}, + "stubs": [{"name": "engine_stub", "return": 0}], + "objects": [ + {"name": "client", "type": "client", "size": "0x2000", + "fields": {"0x1de8": "@engine_node", "0x1dd8": {"value": 0, "size": 8}}}, + {"name": "engine_node", "type": "rm_object", "size": "0x1000", + "fields": {"0x600": "unknown"}} + ], + "memory": [{"pa": "0x200000", "u32": 7}], + "outcomes": {"0x5290ebc": "refuse-fecs-slot-exists-0x1f"}, + "watch": [{"pa": "0x200000", "size": 4, "label": "raw-state"}], + "reads": [{"pa": "0x200000", "size": 4, "label": "raw-after"}], + "provenance": {"source": "construct-path disassembly", "date": "2026-10-06"}, + "notes": ["minimal object set; +0x600 callback unknown"] +} +``` + +Object `type` is one of `client`, `channel`, `graphics_context` (an alias of +`channel`), `graphics_instance`, `memdesc`, `ctx_buffer` or the generic +`rm_object` (no field map; every field is explicit). + +Field value forms (object `fields`): +| Form | Meaning | +|---|---| +| number / numeric string | raw value; the width is the field-map width, else 8 bytes | +| `{"value": v, "size": n}` | raw value with an explicit width (1/2/4/8) | +| `"@name"` / `"@name+0xNN"` | pointer to another object's or stub's base (+offset) | +| `"unknown"` / `{"unknown": true}` | explicit unknown: not written, reported in `unknown_fields` | +| `{"ptr": "@name+0xNN"}` | pointer form in a dict (same as the string form) | + +Object placement (the allocator/pointer story): + +* objects without `pa` (or its alias `va`) are placed in declaration order + from `allocator.base`, each aligned to `align` (object-level `align` + overrides) and advanced by `size`; +* every object name becomes a symbol, so `"@client+0x10"` resolves to the + object's address plus `0x10` when the writes are compiled; +* stubs are the same: a named stub without `addr` is allocated after the + objects and can be used as a pointer target (`"@engine_stub"`); +* in this machine memory is flat and the MPU is inactive, so PA and VA are + identical; `va` is accepted as an alias of `pa` and recorded as such. + +Other top-level keys: + +| Key | Meaning | +|---|---| +| `entry` | code entry (required unless `registers.pc` is given) | +| `registers` | any `x0..x31`/ABI register name; `pc` is an entry alias, not a register | +| `stack` | initial `sp` when the seed does not set `sp` explicitly | +| `memory` | raw page entries: `pa` (or `addr`) plus one of `hex/u64/u32/u16/u8` | +| `stubs` | `{addr?, name?, return}` -> a `lui/addi/jalr ra` returning `return` | +| `outcomes` | address -> label; reaching one stops the run as `outcome` | +| `watch` | extra watch ranges `{pa/addr, size, label}` for the state-read log | +| `reads` | post-run reads included in the report | +| `provenance`, `notes` | free-form evidence/provenance carried into the report | + +## 3. Runner and report `greeneye-state-seed-report/1` + +```bash +python3 harness/state_seed.py demo +python3 harness/state_seed.py emit --kind fecs --out /tmp/fecs.json +python3 harness/state_seed.py emit --kind target-minimal --out /tmp/target.json +python3 harness/state_seed.py emit --kind target-advanced --out /tmp/target-adv.json +python3 harness/state_seed.py validate --seed /tmp/target.json +python3 harness/state_seed.py run --seed /tmp/target.json --out dumps/state-seed-target +``` + +`run` loads the seed, boots the native machine exactly like `rehost` (device +registry from the profile, boot shim, libos syscall model, lenient MMIO), +applies the plan, installs a return trampoline, watchpoints every seeded field +and runs under `--budget`/`--timeout`. It stops on `outcome` (a labelled +address), `return` (the trampoline), `fault`, unhandled `ecall`, `timeout` or +`budget`, and records a `greeneye-session/1` directory plus +`state-seed-report.json`. + +Report highlights: + +* `objects[]`: the compiled layout (name, type, PA, size); +* `unknown_fields[]`: every field the seed marked `unknown`, with its mapped + role where one exists; +* `state_reads[]`: the ordered first touch of each watched field + (`pc`, `addr`, `label`, `value`, `width`) -- the state the path checked; +* `gate`: for a stop, the last read before it and the verdict: + +| gate kind | meaning | +|---|---| +| `null_indirect_call` | the stop is `illegal @ pc 0x0` and the last read was a 0 function-pointer field; `pc` is the load site, `addr`/`label` the pointer read | +| `fault` | a fault with a different last read | +| `<status>_before_gate` | timeout/budget before any early-out; last read shown | + +## 4. Known object field map (derived so far) + +Status `known` = read by a documented path; `unknown` = the offset is real +(from the same disassembly) but its semantics/value are not established. +Every entry is numeric evidence, no source paths. + +| Type | Offset | Width | Status | Role | Evidence | +|---|---|---|---|---|---| +| client | `0x1be0` | 8 | known | graphics-instance pointer array base (index `0x37c + idx`) | promote predicate (`docs/47`) | +| client | `0x1dd8` | 8 | known | engine-context pointer; `0` selects the no-engine join | construct-path disassembly (this doc) | +| client | `0x1de8` | 8 | known | sub-device object; `+0x600` callback is invoked | construct-path disassembly | +| client | `0x1aa8` | 8 | unknown | sub-object consumed later (`+0x24c` callback) | construct-path disassembly | +| client | `0x1cc8` | 8 | unknown | sub-object consumed later (`+0x150`, `+0x1d0`) | construct-path disassembly | +| client | `0x1e48` | 8 | unknown | sub-object on the no-engine path (`+0x110` callback) | construct-path disassembly | +| channel / context | `0x0d28` | 4 | known | graphics-instance index (u32) | promote predicate (`docs/47`) | +| channel / context | `0x0898` | 8 | unknown | callback invoked inside `0x5285aa8` | construct-path disassembly | +| channel / context | `0x0d01` | 1 | unknown | u8 flag: `0` takes the panic path on a failed lookup | construct-path disassembly | +| channel / context | `0x0cd1` | 1 | unknown | u8 flag before the promote calls | construct-path disassembly | +| channel / context | `0x0648` | 8 | unknown | callback on the invalid-state path | construct-path disassembly | +| channel / context | `0x03b8` | 8 | unknown | identity marker compared across objects | construct-path disassembly | +| channel / context | `0x00ee` | 8 | unknown | sub-object (`+0xe8` read as an index/gfid) | construct-path disassembly | +| graphics_instance | `0x0190` | 8 | known | pointer to the per-buffer memDesc slot table | promote predicate (`docs/47`) | +| graphics_instance | `0x0161` | 1 | unknown | u8 flag gating the `0x529bd44` call | construct-path disassembly | +| graphics_instance | `0x03b8` | 8 | unknown | identity marker compared across objects | construct-path disassembly | +| memdesc | `0x0000`, `0x0008` | 8 | unknown | slot value semantics outside the promote predicate | promote predicate only checks non-NULL | +| ctx_buffer | `0x0000` | 8 | unknown | context-buffer (memdesc) object; field semantics unknown | promote predicate only checks non-NULL | + +## 5. What the demonstration proves (2026-10-06) + +`python3 harness/state_seed.py demo` runs three things, all offline, +deterministic and bounded: + +1. **Mechanism proof on a path that needs no RM objects.** The + `fecs-promote` seed replays the `docs/47` predicate through this loader: + `FECS init, slot non-NULL` -> `refuse-fecs-slot-exists-0x1f` (stop + `0x5290ebc`, 75 instructions) and `slot NULL` -> `accept-commit-path` + (`0x5290cd0`). The only difference between the two seeds is the seeded slot + value, so the seed data is driving the firmware's own predicate. Two runs + of the same seed are identical (status, outcome, instructions, reads). +2. **First-gate attempt on `FUN_ram_052a70ec` (entry `0x52a70ec`).** The + minimal seed places a client, a channel/context, a graphics instance and a + memdesc table, sets `a0`/`a1`/`a2`, and walks the real prologue. The path + reads, in order: `client+0x1dd8`, `client+0x1cc8`, `client+0x1aa8`, + `client+0x1de8`, `client+0x1e48`, then (no-engine join) `channel+0xd28`, + then the gate: at `0x52a724c` it loads the `client+0x1de8` sub-object's + `+0x600` callback (=0, marked `unknown`) and calls it at `0x52a7268`; the + call faults `illegal @ pc 0x0` after 106 instructions. **First gate: + `engine_node+0x600` (the indirect call target), read at `0x52a724c`.** The + report's `unknown_fields` names it, and the ordered read list shows the + client fields the seed satisfied. +3. **Gate advance.** The same seed with `+0x600` pointed at a recorded + return-0 stub walks past the first gate and stops at the next one: inside + `0x5285aa8`, `channel+0x898` is loaded at `0x5285af4` and called at + `0x5285b0c`; the target is 0 (marked `unknown`), fault after 143 + instructions. This is the loop the fixture is for: seed the named state, + re-run, name the next gate. + +Timing on this machine (same environment for all rows): FECS single run +0.08 s, construct single run 0.07 s, whole `demo` 0.8 s. Zero hardware +cycles. + +## 6. Admission record + +Per the project admission rule (`open-iov/docs/eng-ops.md` §2), all five +fields: + +| Field | Value | +|---|---| +| Scenario | dynamic-RE loop: run state-gated RM firmware paths (construct/promote family) off-hardware with crafted client/channel/context objects so the first missing state is named without a hardware cycle; trigger = a wall where rehost stops with no state attribution. | +| Measurable outcome | per seed: one `greeneye-state-seed-report/1` with layout, `unknown_fields[]`, ordered `state_reads[]` and a `gate` (address + state checked); sessions per attempt; wall time per run. | +| Baseline | cold rehost of the same entry with no objects (docs/46 §3): stop with `0x40`/illegal-at-0 and no attribution, so the next object is a guess; the alternative for an attributed answer is a supervised construct cycle ([live campaign log PART 96 Add. D, 2026-10-06]: `-110` + NOCAT, FLR recovery). | +| Evaluation threshold | ≥1 wall-class question answered with the gate identified in ≤1 session per attempt, two-run deterministic, zero hardware cycles; seed loop ≤2 min per object. Measured: 0.07–0.08 s per run, 0.8 s demo. | +| Failure signal | a seeded run cannot reach any gate (faults in the loader itself), or the same gate reproduces with two different seeds and the read state cannot be attributed -> record the gap here and fall back to static + supervised capture. | + +## 7. Proven vs hypothesised + +**Proven by this wave (sessions/reports are the evidence):** + +* the loader places objects, pointers, raw memory, stubs and registers + byte-exact and deterministically (unit tests and reports); +* the FECS predicate verdicts are reproduced through the seed with the slot + value as the only variable; +* the construct prologue's ordered state reads are real (watchpoint + first-touch on the seeded fields), and the first gate for the minimal seed + is the `client+0x1de8` -> `+0x600` indirect call; +* stubbing that callback advances execution to the `channel+0x898` gate, + which proves the fixture can iterate the object set. + +**Hypothesised / not claimed:** + +* the object **type names** `client`, `channel`, `graphics_context` are + inferred from the promote predicate and the construct call shape, not from a + live RM object dump; +* fields marked `unknown` in §4 have no established semantics, and their + values in the demo seeds are 0 (or a stub), not real object state; +* the real function behind `engine_node+0x600` and `channel+0x898` is not + known; the return-0 stubs are synthetic probes, never product code; +* no timing, scheduling or engine-side behaviour is claimed (L0/L1 only). + +## 8. Minimal object set still missing + +To walk further than the second gate, the fixture still needs (in order): + +1. the **`engine_node` object** (`client+0x1de8`) with a real or recorded + `+0x600` callback (first gate); +2. the **`channel+0x898` callback** reached inside `0x5285aa8` (second gate); +3. the **`obj_1cc8`** object with `+0x150` (a valid object pointer) and + `+0x1d0` flags, and **`obj_1aa8`** with its `+0x24c` callback (next gates + after the lookup); +4. **`obj_1e48`** with its `+0x110` callback on the no-engine path; +5. **memdesc field semantics** (`+0x0`, `+0x8`, ...) for paths that actually + read a memdesc rather than only checking non-NULL; +6. the **`channel+0x0d01` / `+0x0cd1` / `+0x0648` / `+0x03b8` / `+0x00ee`** + fields once a path reaches the lookup-result and invalid-state branches. + +The function-pointer fields are the load-bearing gap: they must come from a +live/B4 object dump or a capture, because a seeder must not invent code +addresses. Recorded return-0 stubs are the bounded way to cross a gate and +name the next one, and are always labelled as stubs in the report. + +## 9. How to author a new seed + +1. Start from the target's entry and the known field map (§4); write the + smallest object set the entry's prologue reads, marking everything else + `unknown`. +2. Give each object a name; wire pointer fields with `"@name"`; set the entry + registers (`a0`/`a1`/...) and `stack`. +3. `validate` the seed to see the layout and the `unknown_fields` list; check + nothing overlaps and every pointer resolved. +4. `run` it (add `--out` to keep the session + report). Read `state_reads[]` + to confirm each object field was actually consumed, then read `gate` to get + the blocker. +5. If the gate is a callback, re-run with that field pointed at a named stub + to reveal the next gate; repeat until the required object cannot be + authored without a capture. The remaining `unknown_fields` are the + minimal missing object set for the next session. + +## 10. Limits + +* Not a boot and not an RM model: only the seeded state exists; the report + lists it and every limit inline. +* A fault is the first missing state observed, not a product defect; a + timeout/budget is not a verdict. +* Watchpoints report the first touch of each watched field (repeated accesses + are not replayed); the ordered first-touch list is the gate evidence. +* PA = VA here (flat memory, inactive MPU); a seed that needs a translated + view must say so explicitly. +* No timing/cycle/security claim; L0/L1 fidelity only. diff --git a/debug/risc-v-gsp-rm-vm/docs/53-FIRMWARE-OBSERVABILITY-WORKFLOW.md b/debug/risc-v-gsp-rm-vm/docs/53-FIRMWARE-OBSERVABILITY-WORKFLOW.md new file mode 100644 index 0000000..4106244 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/53-FIRMWARE-OBSERVABILITY-WORKFLOW.md @@ -0,0 +1,245 @@ +# Firmware Observability Workflow: using the emulator in the present problem space + +> **Status: 2026-10-06.** This guide says where the emulator belongs in the +> Open-IOV engineering loop, what it can and cannot answer, and exactly which +> commands to run for each question. It introduces four tools added this +> session (`state_decode`, `log_decode`, `server_snapshot`, `msgq_attach`) and +> wires them to the existing ones (`valprobe`, `rehost`, `state_seed`, +> `launch_audit`, `gmmu_walk`, `host_traffic`, `rm_capture`, `contracts`, +> `triage`, `version_diff`). It is offline-only: no GPU, module, guest or +> network. Fidelity is L0/L1 (instruction/memory); never quote timing or +> device-environment results from it. +> +> Index registration in `docs/00-OVERVIEW.md`/`README.md` is intentionally +> left to the in-flight docs edit owned by the simulator workstream. + +--- + +## 1. The layer map: what belongs where + +The project sits on three layers; each layer has one honest oracle. + +| Layer | What it decides | Oracle | Emulator role | +|---|---|---|---| +| **Firmware / server** (GSP-RM: controls, objects, context commit, logs) | "What will the firmware do with this control/state?" | the emulator (predicates, rehosts, seeded handlers, message loop) + a live log ring | first-class: dumps, probes, replay | +| **Front-end / engine** (PBDMA fetch, inline-QMD install, GR dispatch, SMs) | "Does the engine execute this work?" | **silicon only** — the emulator has no PBDMA/GR model by design | evidence preparation and diffing only; *never* a verdict | +| **Guest / host software** | "Did the guest see correct completion?" | the live guest plus the acceptance strings | offline decode of what happened | + +**Consequence:** the emulator is the right tool for *server decisions* and +*evidence reduction*; it is the wrong tool for *engine dispatch*. When a wall +is engine-side, the emulator's job is to (a) exonerate the server half and +(b) reduce the live search space, not to declare a fix. + +The **differential loop** that makes this practical: + +``` + live cycle / captured evidence ──► snapshot + decode + log join + ▲ │ + │ diff / query + │ ▼ + one hardware test ◄── ranked candidates ◄── emulator probes/rehost (server half) +``` + +--- + +## 2. The four new tools + +All four are stdlib-only, deterministic, session-log friendly, and each has +tests in `tests/` (84 tests across the four as of 2026-10-06). Run everything +from the tool root. + +### 2.1 `harness/state_decode.py` — turn channel state into named fields + +Decodes INSTANCE / RAMFC / context-buffer words into field tables and diffs +two snapshots. The bundled table is deliberately small and evidence-only +(INSTANCE magic and GPFIFO VA; RAMFC_TARGET bits 16/17 = ENG/CE context valid); +everything else comes from an extensible `--map` (`greeneye-state-map/1`). + +```bash +# Decode a captured ladder dump (finds INSTANCE/RAMFC blocks automatically) +python3 harness/state_decode.py decode --input <evidence>/06-ladder-<run>.txt --json out.json + +# Diff two decoded states (or two raw inputs) -> greeneye-state-diff/1 +python3 harness/state_decode.py diff a.json b.json +``` + +Use it whenever a run's verdict is "silence": the field table shows context +validity bits and the words that correlate with success, and the diff shows +what a cycle changed. + +### 2.2 `harness/log_decode.py` — the server's stderr, readable + +Normalizes the two real dump shapes (raw ring dumps with `LOGINIT-raw:`/ +`LOGRM-raw:` headers and hexdump rows; textual NOCAT/record lines incl. +continuation call stacks and aggregate entries) into `greeneye-log-decode/1`, +with histograms, filters, and optional nearest-symbol resolution. + +```bash +python3 harness/log_decode.py decode <evidence>/21-logdump-<run>.txt \ + --rec-type 5 --json log.json # just the asserts +python3 harness/log_decode.py decode <log> --symbols symbols.json +``` + +Use it to read *why* the server refused or asserted: recover addresses become +function names, and record histograms expose repeated failure signatures. + +### 2.3 `harness/server_snapshot.py` — canonical server-state snapshot + diff + +Collects chunk dumps and record lines from evidence directories/files into one +`greeneye-server-snapshot/1` JSON (labels → words + records), and diffs two +snapshots into `greeneye-server-snapshot-diff/1`. + +```bash +python3 harness/server_snapshot.py collect --file <evidence>/06-ladder-a.txt \ + --created 2026-10-06T00:00:00Z --json snap-a.json +python3 harness/server_snapshot.py collect --file <evidence>/06-ladder-b.txt \ + --created 2026-10-06T00:00:00Z --json snap-b.json +python3 harness/server_snapshot.py diff --a snap-a.json --b snap-b.json +``` + +This is the A/B instrument for the differential loop: two cycles, one diff, +sorted per-chunk added/removed/changed words. Record differences (e.g. dmesg +timestamps) are visible and honest rather than silently merged. + +Note: `20-gsp-phys-state-*.txt` is an aggregate key:value state, not a memory +chunk; it yields no chunks by design. Pair it with `state_decode` only after +converting its contents to a chunk form. + +### 2.4 `sim/msgq_attach.py` — boot the real firmware to its message loop + +Implements docs/51 §7 steps 1–3: boots the real 570.144 image to the B3 +message-loop port wait, attaches the host doorbell/queue rings before the run, +rings the doorbell, continues under a bounded budget, and classifies the first +poll (`greeneye-msgq-attach/1`). It reports honestly: loop reached or first +divergence, with the reason. + +```bash +python3 -m sim.msgq_attach run --doorbell --json attach.json --record <sess-dir> +python3 -m sim.msgq_attach demo # bounded run + honest skip if no image +``` + +Measured today: **B3 reached** (~474k instructions), doorbell write performed +and checksum-verified; the **libos port model** then completes the +external-interrupt receive handoff (`wait @ 0x4019df4: completed shuttle 10 +(extintr) size=0`), and the loop task's next syscall is `TASK_STATE_QUERY` at +`0x401a46c` — the libos trap/report path. **First ring poll: still none**, and +the report says exactly why (trap state for the query; the command-queue +consumer task is never scheduled in this single-hart model; the +`GSP_ARGUMENTS_CACHED` address handoff is not delivered). No fabricated poll; +two runs are byte-identical. + +--- + +## 3. Run-to-the-wall procedure (reproducible) + +This is the exact sequence to bring the emulator to the current frontier and +produce its verdict on what it *can* decide. + +```bash +cd open-iov/debug/risc-v-gsp-rm-vm +EVID=<our evidence directory> + +# 1. Server state at the current runs, decoded +python3 harness/state_decode.py decode --input $EVID/06-ladder-<run>.txt + +# 2. Server log records around the promote/launch +python3 harness/log_decode.py decode $EVID/21-logdump-<run>.txt + +# 3. A/B the two runs that differ in launch capture +python3 harness/server_snapshot.py collect --file $EVID/06-ladder-<a>.txt --json snap-a.json +python3 harness/server_snapshot.py collect --file $EVID/06-ladder-<b>.txt --json snap-b.json +python3 harness/server_snapshot.py diff --a snap-a.json --b snap-b.json + +# 4. Real image to the message loop (bounded; honest boundary report) +python3 -m sim.msgq_attach run --doorbell +``` + +What today's run of this procedure established: + +* INSTANCE decodes cleanly (magic `0xface`, GPFIFO VA `0x100000`); the + `0x00030000` marker appears at instance words `0x4c`/`0xac` — the same + success marker the live campaign reads, now a named field candidate rather + than an unexplained word. The RAMFC chunk's own `RAMFC_TARGET` is zero, + consistent with the server programming target bits through the instance map. +* The log dump decodes to 122 records with histograms and frame addresses — + the server's assert stream is now machine-readable and symbolizable. +* The two ladder runs diff cleanly at word level — the A/B instrument works. +* The real firmware boots to its message-loop wait deterministically, and the + first thing it needs is **libos port delivery**, not more MMIO. + +**Emulator verdict on the present wall:** the wall is the *forwarded launch's +inline-descriptor install* (front-end/engine physics). The emulator can and did +exonerate the server half (context validity markers, control acceptance, +mapping), but it **cannot decide engine dispatch** and will not be asked to. +The candidate fixes remain the live-lane tests (restore the proven stream +shape, add the descriptor-virtualization control, relocate the prelude) and, +if those fail, the qmdwin variant — all hardware-side. + +--- + +## 4. The full tool chooser (existing + new) + +| Question you actually have | Tool | Kind | +|---|---|---| +| What fields does this chunk hold? What changed between runs? | `state_decode` | new | +| Why did the firmware assert/refuse? What do its records say? | `log_decode` | new | +| Give me one canonical state snapshot per cycle, and diff it | `server_snapshot` | new | +| Does the real image reach its message loop? What does it poll first? | `msgq_attach` | new | +| Which entry/control forms does the firmware accept, on which fields? | `valprobe` | existing | +| Run one whole handler with crafted state; first missing state is the finding | `rehost` + `state_seed` | existing | +| Where would the engine read/write each launch leg? | `launch_audit` (+ `gmmu_walk`) | existing | +| Walk a captured page table through a VA | `gmmu_walk` | existing | +| Replay captured host traffic against the transport model | `host_traffic` | existing | +| Decode/compare real-client RPC captures | `rm_capture` | existing | +| Re-validate proven tools before a cycle | `contracts` / `run_contracts` | existing | +| New image triage / version delta | `triage` / `version_diff` | existing | + +--- + +## 5. Next emulation iterations (the continuation loop) + +Ordered by what each unblocks; each is bounded and session-recorded. + +1. **libos port delivery — landed (partial):** the port-role model + (`sim/libos_port.py`) now carries the doorbell wake through the + external-interrupt receive contract. Measured next blockers: the + `TASK_STATE_QUERY` trap state (`xcause`/`xepc` from the wait point) and + scheduling the command-queue consumer task (the single-hart model runs one + task). Success for this iteration = the consumer's first ring read + (`addr/width/value`), not a fabricated poll. +2. **`GSP_ARGUMENTS_CACHED` / queue-init parsing**: relocate the rings to the + addresses the host actually gives the firmware instead of the fixture block. +3. **CSB/TRANSCOM read-back semantics** for the loop's poll window. +4. **One served RPC**: dispatcher stub returning a real captured reply + (`rm_capture` stimulus), recorded as the B5 session. +5. **MSI-X/event delivery and IRQ arming** so replies can be pushed, not only + polled. +6. **Front-end/engine model — decision gate, not a default.** Reproducing the + inline-descriptor install (PBDMA method execution, IQ2M) would let the + emulator rank engine-side fixes offline, but the project has explicitly + excluded an engine model (surface map §6; it is silicon physics, and offline + engine claims are forbidden). Treat this as a gated, admitted research item + only if the live lane's next tests leave an unresolved ambiguity that only a + model could separate — never as a routine build. + +--- + +## 6. Fidelity, honesty and safety rules + +* Emulator outputs are L0/L1: instruction and memory semantics of the firmware. + No timing (L2), no device environment (L3), no silicon behaviour. +* Silence is not health: a missing fault does not mean success. Prefer + engine-authored observables; front-end-authored signals are not proof. +* Every run keeps its session/report; a claim without a session is an anecdote. +* Offline only: never a GPU, module, mdev, guest, network or capture from this + tree. Injected code is development-only and stays here. +* The differential loop is only as good as the captured evidence; a verdict on + a partial capture names the next page (or field) to capture, not a cause. + +--- + +## 7. Change log + +| Date | Change | +|---|---| +| 2026-10-06 | Created with the four new tools (`state_decode`, `log_decode`, `server_snapshot`, `msgq_attach`), the layer map, the run-to-the-wall procedure and the continuation loop. | diff --git a/debug/risc-v-gsp-rm-vm/harness/contracts.py b/debug/risc-v-gsp-rm-vm/harness/contracts.py new file mode 100644 index 0000000..ec387fe --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/contracts.py @@ -0,0 +1,396 @@ +#!/usr/bin/env python3 +"""Contract registry: every solved wall becomes a re-runnable regression. + +Why this exists +--------------- +The GR campaign's dominant cost is wall-hell: one blocking firmware problem +after another, each triaged by supervised hardware cycles. Several wall +classes now have offline, deterministic tools (docs/45-48), but their value is +only durable if the *answer* survives as a re-runnable check. This registry +pins the proven entry points of those tools to one explicit expectation each +and re-runs them: fast contracts run in the main suite, slow ones on demand. + +It wraps existing entry points as-is (no measurement logic is reimplemented) +and every contract names its wall class from ``docs/50-WALL-WORKFLOW.md``, so +per-domain coverage is visible - including the classes with no offline +contract by design (engine execution awaits supervised captures). + +Offline only: never touches a GPU, kernel module, mdev, guest or network. +""" +from __future__ import annotations + +import os +import time +from dataclasses import dataclass +from typing import Any, Callable + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + +from harness.rehost import DEFAULT_IMAGE, DEMO_ELF # noqa: E402 + +#: Schema tag of a registry run summary. +RUN_FORMAT = "greeneye-contract-run/1" + +FAST = "fast" +SLOW = "slow" + +#: Wall classes from docs/50 section 2. A contract names exactly one. +WALL_CLASSES = { + "a": "firmware boot/status", + "b": "RM server behaviour", + "c": "engine memory contract", + "d": "engine execution/dispatch", + "e": "guest-driver/protocol interface", + "f": "version skew/portability", +} + + +class ContractError(Exception): + """A contract record or registry request is invalid.""" + + +class ContractSkip(Exception): + """A contract precondition is missing at run time (not a pass/fail).""" + + +@dataclass(frozen=True) +class Contract: + """One re-runnable regression over an existing tool entry point.""" + + name: str + wall_class: str # key into WALL_CLASSES (docs/50 section 2) + doc: str # where the wall and the tool are documented + entry: str # exact wrapped entry point, e.g. "sim.gmmu_walk:run_demo()" + speed: str # FAST runs in the suite; SLOW is --all only + expect: str # the pass condition, in words + run: Callable[[], dict] # no-arg callable returning the tool's report + check: Callable[[dict], tuple[bool, str]] # (passed, detail) + requires: tuple[str, ...] = () # paths that must exist; otherwise skip + + @property + def domain(self) -> str: + return WALL_CLASSES.get(self.wall_class, self.wall_class) + + +@dataclass(frozen=True) +class ContractResult: + """Outcome of one contract run; ``status`` is pass | fail | skip.""" + + name: str + wall_class: str + speed: str + doc: str + entry: str + expect: str + status: str + detail: str + seconds: float + + @property + def domain(self) -> str: + return WALL_CLASSES.get(self.wall_class, self.wall_class) + + @property + def passed(self) -> bool: + return self.status == "pass" + + def to_dict(self) -> dict: + return { + "name": self.name, + "wall_class": self.wall_class, + "domain": self.domain, + "speed": self.speed, + "doc": self.doc, + "entry": self.entry, + "expect": self.expect, + "status": self.status, + "detail": self.detail, + "seconds": round(self.seconds, 4), + } + + +# --------------------------------------------------------------------------- +# Runners: thin wrappers over the proven tool entry points. +# +# Every wrapper only imports, calls and (where the demo writes artifacts) +# suppresses output paths. The expectation lives in the checker next to it. +# --------------------------------------------------------------------------- + +def _run_fecs_matrix() -> dict: + from harness.valprobe import fecs_spec, run_spec + + return run_spec(spec=fecs_spec(), record=False, name="contract-fecs-matrix") + + +def _check_fecs_matrix(report: dict) -> tuple[bool, str]: + variants = report.get("variants", []) + evaluated = report.get("evaluated") + failed = report.get("failed") + ok = ( + report.get("format") == "greeneye-valprobe-matrix/1" + and len(variants) == 15 + and evaluated == 15 + and failed == 0 + and all(row.get("pass") is True for row in variants) + ) + return ok, f"{evaluated}/{len(variants)} PROMOTE_CTX variants labelled, {failed} failed" + + +def _run_fecs_determinism() -> dict: + from harness.valprobe import run_demo + + return run_demo() + + +def _check_fecs_determinism(report: dict) -> tuple[bool, str]: + ok = ( + report.get("format") == "greeneye-valprobe-demo/1" + and report.get("ok") is True + and report.get("deterministic") is True + and report.get("failed") == 0 + ) + return ok, ( + f"evaluated={report.get('evaluated')} failed={report.get('failed')} " + f"deterministic={report.get('deterministic')}" + ) + + +def _run_gmmu_demo() -> dict: + from sim.gmmu_walk import run_demo + + return run_demo() + + +def _check_gmmu_demo(report: dict) -> tuple[bool, str]: + rows = report.get("rows", []) + matched = sum(1 for row in rows if row.get("match") is True) + ok = report.get("format") == "greeneye-gmmu-walk-demo/1" and report.get("ok") is True \ + and matched == len(rows) == 5 + return ok, f"{matched}/{len(rows)} VAs walk to the captured DMA addresses" + + +def _run_launch_audit_demo() -> dict: + from harness.launch_audit import run_demo + + return run_demo() + + +def _check_launch_audit_demo(report: dict) -> tuple[bool, str]: + from harness.launch_audit import VERDICT_OK, VERDICT_PA_MISMATCH, VERDICT_UNMAPPED + + legs = report.get("captured_channel", {}).get("contract", []) + ok_legs = sum(1 for leg in legs if leg.get("verdict") == VERDICT_OK) + classes = { + (scenario.get("report", {}).get("contract") or [{}])[0].get("verdict") + for scenario in report.get("qmd_hypotheses", []) + } + ok = ( + report.get("format") == "greeneye-launch-audit-demo/1" + and report.get("ok") is True + and ok_legs == len(legs) == 5 + and {VERDICT_PA_MISMATCH, VERDICT_UNMAPPED} <= classes + ) + return ok, ( + f"captured channel {ok_legs}/{len(legs)} legs ok; " + f"QMD verdict classes {sorted(c for c in classes if c)}" + ) + + +def _run_rehost_demo() -> dict: + from harness.rehost import run_demo + + return run_demo() + + +def _check_rehost_demo(report: dict) -> tuple[bool, str]: + first = report.get("first", {}) + second = report.get("second", {}) + agree = ( + first.get("status") == second.get("status") + and first.get("retval") == second.get("retval") + ) + ok = ( + report.get("format") == "greeneye-rehost-demo/1" + and report.get("ok") is True + and first.get("status") == "return" + and first.get("retval") == report.get("expected") + ) + return ok, ( + f"status={first.get('status')} retval={first.get('retval')} " + f"(expected {report.get('expected')}); two runs agree={agree}" + ) + + +def _run_rm_capture_demo() -> dict: + from harness.rm_capture import run_demo + + return run_demo() + + +def _check_rm_capture_demo(report: dict) -> tuple[bool, str]: + summary = report.get("summary", {}) + ok = report.get("format") == "greeneye-rm-rpc-capture-demo/1" and report.get("ok") is True + return ok, ( + f"records {report.get('records_a')}/{report.get('records_b')}; " + f"stimulus messages {report.get('messages')}; " + f"agree={summary.get('agree')} shape_agree={summary.get('shape_agree')} " + f"a_only={summary.get('a_only')} b_only={summary.get('b_only')}" + ) + + +# --------------------------------------------------------------------------- +# Registry +# --------------------------------------------------------------------------- + +def _registry() -> list[Contract]: + return [ + Contract( + name="fecs-predicate-matrix", + wall_class="b", + doc="docs/47, docs/50", + entry="harness.valprobe:run_spec(fecs_spec())", + speed=FAST, + expect="15/15 PROMOTE_CTX predicate variants reach their labelled outcome (failed == 0)", + requires=(DEFAULT_IMAGE,), + run=_run_fecs_matrix, + check=_check_fecs_matrix, + ), + Contract( + name="fecs-predicate-determinism", + wall_class="b", + doc="docs/47, docs/25, docs/50", + entry="harness.valprobe:run_demo()", + speed=SLOW, + expect="the bundled matrix runs twice with identical verdicts (ok and deterministic)", + requires=(DEFAULT_IMAGE,), + run=_run_fecs_determinism, + check=_check_fecs_determinism, + ), + Contract( + name="gmmu-golden-vector", + wall_class="c", + doc="docs/48, docs/50", + entry="sim.gmmu_walk:run_demo()", + speed=FAST, + expect="the captured channel-PDB vector walks 5/5 VAs to the logged DMA addresses", + run=_run_gmmu_demo, + check=_check_gmmu_demo, + ), + Contract( + name="launch-audit-captured-contract", + wall_class="c", + doc="docs/48, docs/50", + entry="harness.launch_audit:run_demo()", + speed=FAST, + expect="captured contract all ok; QMD hypotheses classify pa_mismatch and unmapped", + run=_run_launch_audit_demo, + check=_check_launch_audit_demo, + ), + Contract( + name="rehost-demo", + wall_class="b", + doc="docs/46, docs/25, docs/50", + entry="harness.rehost:run_demo()", + speed=FAST, + expect="the demo function returns the expected sum twice (ok == true)", + requires=(DEMO_ELF,), + run=_run_rehost_demo, + check=_check_rehost_demo, + ), + Contract( + name="rm-capture-decode-demo", + wall_class="e", + doc="docs/45, docs/25, docs/50", + entry="harness.rm_capture:run_demo()", + speed=FAST, + expect="synthetic capture decodes, compares and yields a stimulus (ok == true)", + run=_run_rm_capture_demo, + check=_check_rm_capture_demo, + ), + ] + + +def all_contracts() -> list[Contract]: + """Every contract in the registry, fast and slow.""" + return _registry() + + +def fast_contracts() -> list[Contract]: + """Contracts the main suite runs.""" + return [c for c in all_contracts() if c.speed == FAST] + + +def slow_contracts() -> list[Contract]: + """Contracts only ``run_contracts.py --all`` runs.""" + return [c for c in all_contracts() if c.speed == SLOW] + + +def _result(contract: Contract, status: str, detail: str, seconds: float) -> ContractResult: + return ContractResult( + name=contract.name, + wall_class=contract.wall_class, + speed=contract.speed, + doc=contract.doc, + entry=contract.entry, + expect=contract.expect, + status=status, + detail=detail, + seconds=seconds, + ) + + +def run_contract(contract: Contract) -> ContractResult: + """Run one contract, classifying every outcome without raising.""" + missing = [path for path in contract.requires if not os.path.exists(path)] + if missing: + return _result(contract, "skip", "prerequisite missing: " + ", ".join(missing), 0.0) + started = time.monotonic() + try: + report = contract.run() + except ContractSkip as exc: + return _result(contract, "skip", str(exc), time.monotonic() - started) + except Exception as exc: # noqa: BLE001 - a raising entry point is a failed contract + return _result( + contract, "fail", f"{type(exc).__name__}: {exc}", time.monotonic() - started + ) + elapsed = time.monotonic() - started + try: + passed, detail = contract.check(report) + except Exception as exc: # noqa: BLE001 - a broken checker is a failed contract + return _result(contract, "fail", f"checker error: {type(exc).__name__}: {exc}", elapsed) + return _result(contract, "pass" if passed else "fail", str(detail), elapsed) + + +def run_contracts(contracts: list[Contract]) -> list[ContractResult]: + """Run the given contracts in order; never raises for a contract outcome.""" + return [run_contract(contract) for contract in contracts] + + +def summarize(results: list[ContractResult]) -> dict: + """Machine-readable run summary with per-wall-class coverage.""" + counts = {"pass": 0, "fail": 0, "skip": 0} + domains: dict[str, dict[str, Any]] = { + key: { + "wall_class": key, + "domain": label, + "total": 0, + "pass": 0, + "fail": 0, + "skip": 0, + } + for key, label in WALL_CLASSES.items() + } + for result in results: + if result.status not in counts: + raise ContractError(f"unknown contract status {result.status!r}") + counts[result.status] += 1 + bucket = domains[result.wall_class] + bucket["total"] += 1 + bucket[result.status] += 1 + return { + "format": RUN_FORMAT, + "ok": counts["fail"] == 0, + "counts": counts, + "domains": [domains[key] for key in sorted(domains)], + "contracts": [result.to_dict() for result in results], + } diff --git a/debug/risc-v-gsp-rm-vm/harness/launch_audit.py b/debug/risc-v-gsp-rm-vm/harness/launch_audit.py new file mode 100644 index 0000000..ba7a623 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/launch_audit.py @@ -0,0 +1,691 @@ +#!/usr/bin/env python3 +"""Launch memory-contract audit: where would the engine read/write? + +A forwarded submission goes silent when one of the engine-visible memory legs +is wrong: the QMD destination the front-end copies to, the release semaphore +the engine writes, the ring the front-end fetches, the parameter/program +pages the QMD points at. The host's bookkeeping (shadows, guest pages) is not +what the engine sees -- the *page tables* are. This harness walks each +address of a submission's memory contract through a captured/constructed +page-table view (``sim.gmmu_walk``) and reports, per leg, the PA the engine +would use, its access flags, and any divergence from what the host believes. + +It answers the class of question behind "the front-end never wrote the QMD"/ +"the release never fired": *is the mapping the engine uses the mapping we are +checking?* -- offline, deterministically, without hardware. + +``check`` closes the capture loop: run it against a partial or empty +``memory[]`` view and it reports, per contract leg, the FIRST table page the +walk needed but the view does not contain (with the level/index path), plus a +consolidated ``needed_pages[]`` list. Dump exactly those pages, re-run, and +repeat -- one capture iteration per table level. + +Schemas: + +* ``greeneye-launch-audit-spec/1`` -- inputs (page tables + contract). +* ``greeneye-launch-audit/1`` -- the verdict report. +* ``greeneye-launch-audit-check/1`` -- the dump-completeness report. + +Honesty: a verdict is only as good as the captured table view in the spec. +``capture-hints`` prints what to capture for a real audit and never executes +anything. Fidelity is the address view (L1); no timing claim. +""" +from __future__ import annotations + +import argparse +import json +import os +import struct +import sys +from typing import Any + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.gmmu_pte import KIND_VER2, LEVELS, SUPPORTED_KINDS # noqa: E402 +from sim.gmmu_walk import ( # noqa: E402 + FAULT_TABLE_MISSING, + MemoryView, + WalkError, + channel_pdb_fixture, + encode_dual_slot, + encode_pde_pointer, + encode_pte, + walk, +) + +SPEC_FORMAT = "greeneye-launch-audit-spec/1" +AUDIT_FORMAT = "greeneye-launch-audit/1" +CHECK_FORMAT = "greeneye-launch-audit-check/1" +DEMO_FORMAT = "greeneye-launch-audit-demo/1" + +VERDICT_OK = "ok" +VERDICT_UNMAPPED = "unmapped" +VERDICT_PA_MISMATCH = "pa_mismatch" +VERDICT_ACCESS_DENIED = "access_denied" + +#: Walk depth -> display label (root first, leaf page table last). Mirrors +#: the ``LEVELS`` geometry in ``sim.gmmu_pte``; kept here as display names so +#: check reports read like the capture hints (root/pde2/pde1/pde0/pte). +LEVEL_LABELS = ("root", "pde2", "pde1", "pde0", "pte") + + +class AuditError(ValueError): + """A spec or request the tool cannot honour.""" + + +def _parse_int(value: Any) -> int: + if isinstance(value, bool): + raise AuditError(f"bad integer {value!r}") + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value, 0) + except ValueError as exc: + raise AuditError(f"bad integer {value!r}") from exc + raise AuditError(f"bad integer {value!r}") + + +def _parse_hex_bytes(text: str) -> bytes: + data = "".join(str(text).split()) + if len(data) % 2: + raise AuditError("odd hex length") + try: + return bytes.fromhex(data) + except ValueError as exc: + raise AuditError(f"bad hex payload: {exc}") from exc + + +def load_memory(items) -> MemoryView: + mem = MemoryView() + for item in items or (): + if "pa" not in item or "hex" not in item: + raise AuditError(f"memory item needs pa and hex: {item!r}") + mem.add(_parse_int(item["pa"]), _parse_hex_bytes(item["hex"])) + return mem + + +def audit(spec: dict) -> dict: + """Run one audit spec; return a ``greeneye-launch-audit/1`` report.""" + fmt = spec.get("format", SPEC_FORMAT) + if fmt != SPEC_FORMAT: + raise AuditError(f"unsupported spec format {fmt!r} (want {SPEC_FORMAT})") + if "pdb_pa" not in spec or "contract" not in spec: + raise AuditError("spec needs pdb_pa and contract") + kind = spec.get("gmmu", KIND_VER2) + if kind not in SUPPORTED_KINDS: + raise AuditError(f"unsupported gmmu kind {kind!r} (support: {SUPPORTED_KINDS})") + page_mode = spec.get("page_mode", "auto") + root_pa = _parse_int(spec["pdb_pa"]) + mem = load_memory(spec.get("memory")) + + rows = [] + for item in spec["contract"]: + label = item.get("label") or "unnamed" + if "va" not in item: + raise AuditError(f"contract item needs va: {item!r}") + va = _parse_int(item["va"]) + access = item.get("access", "read") + report = walk(va, root_pa=root_pa, mem=mem, kind=kind, page_mode=page_mode) + row: dict[str, Any] = { + "label": label, + "va": f"{va:#x}", + "access": access, + "note": item.get("note"), + "verdict": None, + "walk": report, + } + if report["status"] != "ok": + row["verdict"] = VERDICT_UNMAPPED + row["detail"] = report.get("fault_message") + else: + expect_pa = item.get("expect_pa") + if access == "write" and not report.get("writable", False): + row["verdict"] = VERDICT_ACCESS_DENIED + row["detail"] = "PTE is read-only but the engine must write here" + elif expect_pa is not None and _parse_int(expect_pa) != _parse_int(report["pa"]): + row["verdict"] = VERDICT_PA_MISMATCH + row["detail"] = (f"engine walk -> {report['pa']}; host believes " + f"{_parse_int(expect_pa):#x}") + else: + row["verdict"] = VERDICT_OK + rows.append(row) + + summary = {v: sum(1 for r in rows if r["verdict"] == v) + for v in (VERDICT_OK, VERDICT_UNMAPPED, VERDICT_PA_MISMATCH, + VERDICT_ACCESS_DENIED)} + report = { + "format": AUDIT_FORMAT, + "name": spec.get("name"), + "firmware": spec.get("firmware"), + "gmmu": kind, + "page_mode": page_mode, + "pdb_pa": f"{root_pa:#x}", + "contract": rows, + "summary": summary, + "ok": summary[VERDICT_UNMAPPED] == 0 and summary[VERDICT_PA_MISMATCH] == 0 + and summary[VERDICT_ACCESS_DENIED] == 0, + "limits": [ + "Address view only (L1): a clean audit proves the engine CAN reach these PAs, not that it runs.", + "Verdicts depend on the captured table view in the spec; an incomplete view reports unmapped, not unreachability.", + ], + } + return report + + +# --------------------------------------------------------------------------- +# Check: which table pages does the walk still need? +# --------------------------------------------------------------------------- + +def _path_from_steps(steps: list[dict]) -> list[dict]: + """The completed walk hops, in walk order, as check ``path`` elements.""" + return [ + { + "level": LEVEL_LABELS[step["depth"]], + "depth": step["depth"], + "table_pa": step["table_pa"], + "index": step["index"], + "entry_pa": step["entry_pa"], + "present": True, + } + for step in steps + ] + + +def _missing_table(report: dict, root_pa: int, va: int) -> dict: + """Describe the first table page a ``table_missing`` walk needed. + + ``report["steps"]`` holds the levels that completed before the failing + read; the missing table is the page that read was addressing. The + returned dict carries the page base (``pa`` and ``table_pa``), the level + label, the walk depth, the index inside the table and the exact entry + address. ``report`` must be a ``table_missing`` fault. + """ + depth = len(report["steps"]) + if depth == 0: + table_pa = root_pa + else: + prev = report["steps"][-1] + if prev.get("role") == "dual_pde": + half = prev.get("page_mode_used") + table_pa = (prev["big"]["address"] if half == "64k" + else prev["small"]["address"]) + else: + table_pa = prev["address"] + if depth not in range(len(LEVEL_LABELS)): + # Cannot happen for a walker fault, but never guess a geometry. + return {"pa": f"{table_pa:#x}", "table_pa": f"{table_pa:#x}", + "depth": depth, "level": f"depth{depth}", "index": None, + "entry_pa": None, "present": False} + if depth == 4: + prev = report["steps"][-1] + shift = 16 if prev.get("page_mode_used") == "64k" else 12 + index = (va >> shift) & 0x1FF + entry_size = 8 + else: + geo = LEVELS[depth] + index = (va >> geo["va_bits"][1]) & ((1 << geo["index_bits"]) - 1) + entry_size = geo["entry_size"] + return { + "pa": f"{table_pa:#x}", + "table_pa": f"{table_pa:#x}", + "depth": depth, + "level": LEVEL_LABELS[depth], + "index": index, + "entry_pa": f"{table_pa + index * entry_size:#x}", + "present": False, + } + + +def check(spec: dict) -> dict: + """Report the table pages a partial ``memory[]`` view still needs. + + Runs the same walks as :func:`audit` but, instead of verdicts, reports per + contract leg the FIRST table page the walk could not read (the page the + dump is missing) with the level/index path that reached it, and a + consolidated ``needed_pages`` list. Iterate: run ``check`` with a partial + or empty dump, dump exactly the named pages, repeat until ``complete`` is + true, then run :func:`audit` for verdicts. A leg whose walk faults for a + non-dump reason (invalid aperture/PTE, unsupported encoding) is reported + as ``fault`` and needs no more pages. + + An empty spec memory is valid: every leg then reports the root page as + its first needed page. + """ + fmt = spec.get("format", SPEC_FORMAT) + if fmt != SPEC_FORMAT: + raise AuditError(f"unsupported spec format {fmt!r} (want {SPEC_FORMAT})") + if "pdb_pa" not in spec or "contract" not in spec: + raise AuditError("spec needs pdb_pa and contract") + kind = spec.get("gmmu", KIND_VER2) + if kind not in SUPPORTED_KINDS: + raise AuditError(f"unsupported gmmu kind {kind!r} (support: {SUPPORTED_KINDS})") + page_mode = spec.get("page_mode", "auto") + root_pa = _parse_int(spec["pdb_pa"]) + mem = load_memory(spec.get("memory")) + + rows = [] + needed: dict[int, dict[str, Any]] = {} + for item in spec["contract"]: + label = item.get("label") or "unnamed" + if "va" not in item: + raise AuditError(f"contract item needs va: {item!r}") + va = _parse_int(item["va"]) + report = walk(va, root_pa=root_pa, mem=mem, kind=kind, page_mode=page_mode) + row: dict[str, Any] = { + "label": label, + "va": f"{va:#x}", + "access": item.get("access", "read"), + "expect_pa": None, + "status": None, + "resolved_pa": None, + "writable": None, + "first_missing": None, + "fault": report.get("fault"), + "detail": report.get("fault_message"), + "path": _path_from_steps(report["steps"]), + } + if item.get("expect_pa") is not None: + row["expect_pa"] = f"{_parse_int(item['expect_pa']):#x}" + if report["status"] == "ok": + row["status"] = "complete" + row["resolved_pa"] = report["pa"] + row["writable"] = report.get("writable") + row["fault"] = None + row["detail"] = None + elif report.get("fault") == FAULT_TABLE_MISSING: + missing = _missing_table(report, root_pa, va) + row["status"] = "missing" + row["first_missing"] = missing + row["path"].append(missing) + entry = needed.setdefault(int(missing["pa"], 16), + {"levels": set(), "legs": []}) + entry["levels"].add(missing["level"]) + entry["legs"].append(label) + else: + row["status"] = "fault" + rows.append(row) + + needed_pages = [ + {"pa": f"{pa:#x}", "levels": sorted(value["levels"]), + "legs": sorted(value["legs"])} + for pa, value in sorted(needed.items()) + ] + return { + "format": CHECK_FORMAT, + "name": spec.get("name"), + "firmware": spec.get("firmware"), + "gmmu": kind, + "page_mode": page_mode, + "pdb_pa": f"{root_pa:#x}", + "memory_pages": len(mem.spans()), + "contract": rows, + "needed_pages": needed_pages, + "complete": all(row["status"] != "missing" for row in rows), + "limits": [ + "Dump-completeness view only: `complete` means every table page the walk needed is in the spec, not that any mapping is correct.", + "A `fault` leg needs no more pages: the walk read its entries and refused them (invalid aperture/PTE or unsupported encoding).", + "`needed_pages` is the FIRST missing page per leg; dumping it can reveal the next one, so iterate.", + ], + } + + +# --------------------------------------------------------------------------- +# Demo: the captured channel-PDB contract + the QMD-destination hypotheses +# --------------------------------------------------------------------------- + +def _tree_for_va(root_pa: int, target_va: int, target_pa: int | None, + *, tables_base: int = 0x50000000) -> MemoryView: + """Build a minimal VER2 tree mapping one VA to one PA (None = unmapped).""" + va = target_va + i0 = (va >> 47) & 0x3 + i1 = (va >> 38) & 0x1FF + i2 = (va >> 29) & 0x1FF + i3 = (va >> 21) & 0xFF + i4 = (va >> 12) & 0x1FF + pd2 = tables_base + 0x1000 + pd1 = tables_base + 0x2000 + pd0 = tables_base + 0x3000 + pt = tables_base + 0x4000 + mem = MemoryView() + root = bytearray(0x1000) + struct.pack_into("<Q", root, i0 * 8, encode_pde_pointer(pd2)) + mem.add(root_pa, bytes(root)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3 * 16, encode_dual_slot(None, pt)) + mem.add(pd0, bytes(page)) + page = bytearray(0x1000) + if target_pa is not None: + struct.pack_into("<Q", page, i4 * 8, encode_pte(target_pa)) + mem.add(pt, bytes(page)) + return mem + + +def run_demo(out: str | None = None) -> dict: + """Audit the captured channel contract and the fork classes. + + Expected verdicts (all pinned by tests): + + * captured channel contract -- all 5 low-window legs ``ok``; + * QMD destination (VA ``0x204608000``): engine resolves to the host shadow + while the host checks the guest page -> ``pa_mismatch``; bookkeeping + trusting the shadow -> ``ok``; unmapped leaf -> ``unmapped``; + * engine-write through shadow (the next wall class: CUDA output buffers at + high VAs, where Tier-1 refuses 0x1f and the leg resolves through the + guest scratch PT pool): the write leg resolves to a shadow PA while the + host bookkeeping still expects the guest PA -> ``pa_mismatch``; when the + bookkeeping is aligned to the engine-visible shadow PA -> ``ok``. + The shadow/guest PAs in this scenario are synthetic (documented in + docs/48), not m7 captures. + """ + # 1) Captured channel PDB contract (validated environment probe). + root_pa, mem = channel_pdb_fixture() + captured = { + "format": SPEC_FORMAT, + "name": "captured-channel-pdb", + "firmware": "570.144", + "gmmu": KIND_VER2, + "page_mode": "4k", + "pdb_pa": f"{root_pa:#x}", + "memory": [{"pa": f"{pa:#x}", "hex": b.hex()} for pa, b in _memory_items(mem)], + "contract": [ + {"label": "gpfifo-ring", "va": "0x100000", "access": "read", + "expect_pa": "0x2045d9000", "note": "front-end GPFIFO fetch"}, + {"label": "push-buffer", "va": "0x101000", "access": "write", + "expect_pa": "0x121bd9000"}, + {"label": "copy-src", "va": "0x102000", "access": "read", + "expect_pa": "0x152115000"}, + {"label": "copy-dst", "va": "0x103000", "access": "write", + "expect_pa": "0x15a5f5000"}, + {"label": "semaphore", "va": "0x104000", "access": "write", + "expect_pa": "0x10cf27000"}, + ], + } + captured_report = audit(captured) + + # 2) QMD-destination hypotheses, same VA (0x204608000): + # A: engine resolves to the host shadow (0x15e8fa000) -- host checks + # the guest page (0x204608000) -> pa_mismatch (the "wrong page" bug). + # B: no mapping / invalid leaf -> unmapped (the "copy dropped" bug). + qmd_va = 0x204608000 + shadow_pa = 0x15E8FA000 + scenarios = [] + for label, target, expect in ( + ("qmd-dest-shadow-trusted", shadow_pa, shadow_pa), + ("qmd-dest-host-checks-guest", shadow_pa, 0x204608000), + ("qmd-dest-unmapped", None, None), + ): + root = 0x60000000 + tree = _tree_for_va(root, qmd_va, target) + spec = { + "format": SPEC_FORMAT, "name": label, "firmware": "570.144", + "gmmu": KIND_VER2, "page_mode": "4k", "pdb_pa": f"{root:#x}", + "memory": [{"pa": f"{pa:#x}", "hex": b.hex()} for pa, b in _memory_items(tree)], + "contract": [{"label": "inline-qmd-destination", "va": f"{qmd_va:#x}", + "access": "write", "expect_pa": f"{expect:#x}" if expect else None, + "note": "front-end copies the inline QMD here"}], + } + report = audit(spec) + scenarios.append({"name": label, "report": report}) + + # 3) Engine-write through shadow (the next wall class). A high guest VA + # is refused by Tier-1 and resolved through the guest scratch PT pool + # to a shadow page; the host bookkeeping for the output may still track + # the guest page. Same walk, two bookkeeping states: + # A: host expects the guest PA -> pa_mismatch (the write is invisible). + # B: bookkeeping aligned to the engine-visible shadow PA -> ok. + shadow_va = 0x210000 + shadow_target = 0x13D2A0000 + guest_pa = 0x2045C0000 + shadow_scenarios = [] + for label, expect, want in ( + ("engine-write-shadow-host-checks-guest", guest_pa, VERDICT_PA_MISMATCH), + ("engine-write-shadow-bookkeeping-aligned", shadow_target, VERDICT_OK), + ): + root = 0x70000000 + tree = _tree_for_va(root, shadow_va, shadow_target) + spec = { + "format": SPEC_FORMAT, "name": label, "firmware": "570.144", + "gmmu": KIND_VER2, "page_mode": "4k", "pdb_pa": f"{root:#x}", + "memory": [{"pa": f"{pa:#x}", "hex": b.hex()} for pa, b in _memory_items(tree)], + "contract": [{"label": "high-va-output", "va": f"{shadow_va:#x}", + "access": "write", "expect_pa": f"{expect:#x}", + "note": "Tier-2 shadow alias for a high-VA write leg"}], + } + report = audit(spec) + shadow_scenarios.append({"name": label, "expected_verdict": want, + "report": report}) + + demo = { + "format": DEMO_FORMAT, + "ok": (captured_report["ok"] + and scenarios[0]["report"]["ok"] + and scenarios[1]["report"]["contract"][0]["verdict"] == VERDICT_PA_MISMATCH + and scenarios[2]["report"]["contract"][0]["verdict"] == VERDICT_UNMAPPED + and all(s["report"]["contract"][0]["verdict"] == s["expected_verdict"] + for s in shadow_scenarios)), + "captured_channel": captured_report, + "qmd_hypotheses": scenarios, + "engine_write_shadow": shadow_scenarios, + } + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "launch-audit-demo.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(demo, fh, indent=2, sort_keys=True) + fh.write("\n") + demo["artifacts"] = {"demo_json": os.path.abspath(path)} + return demo + + +def _memory_items(mem: MemoryView): + return [(int(item["pa"], 16), mem.read(int(item["pa"], 16), item["size"])) + for item in mem.spans()] + + +CAPTURE_HINTS = """\ +CAPTURE HINTS (print-only; nothing is executed by this tool) +============================================================ +For a real launch audit you need the *engine's* table view, not the host's +bookkeeping. Capture, for one failing submission: + +1. The channel's page-directory base PA (PDB) -- from the channel's INSTANCE + dump field, or the SET_PAGE_DIRECTORY / GPU_PDB_ADDRESS your stack programs. +2. The table pages reachable from that root for the contract VAs: root, PDE2, + PDE1, PDE0 (dual slot) and the leaf page table(s). A raw PA->bytes dump of + those 4 KB pages is enough; keep them byte-exact (no pretty-printing). +3. The submitted stream's own pointers: GPFIFO ring, inline-QMD destination + (SET_INLINE_QMD_ADDRESS_A/B), the QMD's parameter/program/release-slot + addresses, and the channel report semaphore. +4. Your shadow/guest bookkeeping map for those same VAs (what you currently + check after a timeout). +5. Put 1-4 into a `greeneye-launch-audit-spec/1` JSON (see docs/48). + +ITERATE WITH `check` (one capture round per table level): + # start with only pdb_pa + contract and `"memory": []` (or a partial dump) + python3 harness/launch_audit.py check --spec m7-launch.json + # -> prints the FIRST missing table page per leg + `needed_pages[]` + # dump exactly those pages, append them to memory[], re-run check + # when check says complete: + python3 harness/launch_audit.py run --spec m7-launch.json + +CURRENT LIVE WALL (m7, 2026-10-06) -- exact dump list: + Host-side chain (the validated 5/5 fixture; dump all five raw pages): + root 0x12f16e000 + pd2 0x12f16f000 + pd1 0x11065e000 + pd0 0x110663000 (dual slot pd0[0].small -> pt) + pt 0x15908d000 (pt entries 256..260 = the low-window legs) + Forward-window PT pages (Tier-1 DMA_UPDATE_PDE_2 windows 4128 / 4131): + 0x1b8706000 (pdeIndex 4128) + 0x1197a6000 (pdeIndex 4131) + Geometry rule: pdeIndex = (pd1_idx << 8) | pd0_idx, so the forward windows + land at pd1 index 16. Their pd1[16] and pd0-level chain entries are + written by the SERVER on DMA_UPDATE_PDE_2 acceptance and are in no host + log -- include them (and any page they point at) if they fall outside the + host pages above; `check` names the exact PA. + Tier-2 shadows (high-VA legs): + * the 16 guest-scratch-PT page DMAs -- module stage-2 chunk list, + built as `guest scratch PT built: pages=16 vaBase=0x200000 + capacity=512`; shadow VA = 0x200000 + (page << 21) + (line << 16); + * the rmcall_gr_shadow chunk DMAs -- engine-write shadow pages for the + high-VA legs (the next wall class after dispatch clears). + Key contract VAs: + qmd-dest 0x204608000 write (QMD destination prewritten by host; + post-mortem reads the same hpa) + release-slot 0x204027f7c write (engine release; relw never fired) + ring 0x100000..0x104000 read/write (GPFIFO ring, push-buffer, + copy src/dst, channel semaphore) + param-alias 0x210000 read (Tier-2 shadow alias; NOT directly + mapped; PTEs come from guest scratch) + program-alias 0x220000 read (same) + indirect-alias 0x260000 read (shadow slot 6; same) + The high-VA param/program guest VAs (0x7e06f0.../0x735aa0...) are NOT + directly mapped: Tier-1 refuses status 0x1f and they resolve through the + shadow aliases above. + +COPY-PASTE SPEC SKELETON (start with empty memory and let `check` drive; fill +each `hex` with the 4096-byte raw dump of the page -- no pretty-printing): +{ + "format": "greeneye-launch-audit-spec/1", + "name": "m7-launch", + "firmware": "570.144", + "gmmu": "ver2", + "page_mode": "4k", + "pdb_pa": "0x12f16e000", + "memory": [], + "contract": [ + {"label": "qmd-dest", "va": "0x204608000", "access": "write", "expect_pa": "0x204608000"}, + {"label": "release-slot", "va": "0x204027f7c", "access": "write"}, + {"label": "gpfifo-ring", "va": "0x100000", "access": "read"}, + {"label": "push-buffer", "va": "0x101000", "access": "write"}, + {"label": "copy-src", "va": "0x102000", "access": "read"}, + {"label": "copy-dst", "va": "0x103000", "access": "write"}, + {"label": "semaphore", "va": "0x104000", "access": "write"}, + {"label": "param-alias", "va": "0x210000", "access": "read"}, + {"label": "program-alias", "va": "0x220000", "access": "read"}, + {"label": "indirect-alias","va": "0x260000", "access": "read"} + ] +} +Then: python3 harness/launch_audit.py check --spec m7-launch.json + +Reading the result: `pa_mismatch` means the engine reads/writes a different PA +than the one you are checking (look at the shadow/copy bookkeeping); +`unmapped` means the walk cannot reach any PA (the engine's access is dropped +or faults); `access_denied` means the leaf is read-only for a write leg. +`ok` means the engine CAN reach every leg -- the silence is downstream of the +address path (front-end/engine/context), not in the mapping. +""" + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="launch_audit.py", + description="Audit a submission's engine-visible memory contract (offline).", + ) + sub = parser.add_subparsers(dest="command") + run = sub.add_parser("run", help="run an audit spec") + run.add_argument("--spec", required=True) + run.add_argument("--out", help="write the report JSON to this directory") + check_cmd = sub.add_parser( + "check", help="report the first table pages a partial/empty dump is missing") + check_cmd.add_argument("--spec", required=True) + check_cmd.add_argument("--out", help="write the report JSON to this directory") + demo = sub.add_parser("demo", help="captured contract + QMD/shadow hypotheses") + demo.add_argument("--out", help="output directory") + sub.add_parser("capture-hints", help="print what to capture for a real audit") + return parser + + +def _write_out(report: dict, out: str, name: str) -> str: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, name) + with open(path, "w", encoding="utf-8") as fh: + json.dump(report, fh, indent=2, sort_keys=True) + fh.write("\n") + return os.path.abspath(path) + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "run": + if not os.path.isfile(args.spec): + raise AuditError(f"spec not found: {args.spec}") + with open(args.spec, "r", encoding="utf-8") as fh: + spec = json.load(fh) + report = audit(spec) + for row in report["contract"]: + print(f"{row['verdict']:<14} {row['label']:<28} {row['va']}" + + (f" {row['detail']}" if row.get("detail") else "")) + summary = {"format": report["format"], "ok": report["ok"], + "summary": report["summary"]} + if args.out: + summary["report_json"] = _write_out(report, args.out, "launch-audit.json") + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 if report["ok"] else 3 + if args.command == "check": + if not os.path.isfile(args.spec): + raise AuditError(f"spec not found: {args.spec}") + with open(args.spec, "r", encoding="utf-8") as fh: + spec = json.load(fh) + report = check(spec) + for row in report["contract"]: + if row["status"] == "missing": + miss = row["first_missing"] + print(f"missing {row['label']:<28} {row['va']}" + f" need {miss['pa']} ({miss['level']}[{miss['index']}])") + elif row["status"] == "complete": + print(f"complete {row['label']:<28} {row['va']}" + f" -> {row['resolved_pa']}") + else: + print(f"fault {row['label']:<28} {row['va']}" + f" {row['fault']}: {row['detail']}") + if report["needed_pages"]: + print("dump next:") + for page in report["needed_pages"]: + print(f" {page['pa']} {'/'.join(page['levels'])}" + f" legs: {', '.join(page['legs'])}") + summary = { + "format": report["format"], + "complete": report["complete"], + "needed_pages": [page["pa"] for page in report["needed_pages"]], + } + if args.out: + summary["report_json"] = _write_out( + report, args.out, "launch-audit-check.json") + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 if report["complete"] else 3 + if args.command == "demo": + demo = run_demo(args.out) + cap = demo["captured_channel"] + print(f"captured channel contract: ok={cap['ok']} {cap['summary']}") + for scenario in demo["qmd_hypotheses"]: + row = scenario["report"]["contract"][0] + print(f"{row['verdict']:<14} {scenario['name']} {row.get('detail') or ''}") + for scenario in demo["engine_write_shadow"]: + row = scenario["report"]["contract"][0] + print(f"{row['verdict']:<14} {scenario['name']} {row.get('detail') or ''}") + print(json.dumps({"ok": demo["ok"]}, indent=2)) + return 0 if demo["ok"] else 3 + if args.command == "capture-hints": + print(CAPTURE_HINTS) + return 0 + parser.print_help() + return 2 + except (AuditError, WalkError) as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/log_decode.py b/debug/risc-v-gsp-rm-vm/harness/log_decode.py new file mode 100644 index 0000000..e1d351c --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/log_decode.py @@ -0,0 +1,660 @@ +#!/usr/bin/env python3 +"""Decode the two real GSP-RM log dump forms into one normalized record stream. + +The live GSP-RM writes its "server stderr" into an 8 KiB-word libos log ring. +Two dump forms exist in the campaign evidence and they are the inputs here: + +1. Raw ring dumps (debugfs ``gsp-phys-log-dump`` saved to evidence files). + A ``<LABEL>-raw: <n> bytes (...) VA=... dma=...`` header is followed by + hexdump rows ``00000000: 000002d2 00000000 ...`` (little-endian u64 words). + Each header's rows decode into one ``raw-hex`` record whose ``frames`` are + the u64 words and whose ``raw`` is the decoded byte buffer as lowercase + hex. A later ``<LABEL>-decoded:`` marker switches to the offline + liblogdecode text records ``rec 449 ts=0x.. meta=0x.. args=0x..`` which + become ``record`` entries (``args`` -> ``frames``); both the + ``args=[0x.., 0x..]`` and ``args=0x..,0x..`` spellings are accepted. + +2. Live dmesg / evidence lines, e.g.:: + + vgsp-nocat: rec seq=1 recType=5 errorCode=0x57f1344 subsystem=0x0 bugcheck=0x0 payload=1212 source="ASSERT" faultingEngine="" + vgsp-nocat: rec seq=1 assertCallStack count=1 [0x401000,0x402000] (offsets from load base 0x4000000) + vgsp-nocat: entry[0]: count=2 recType=5 errorCode=0x57f1344 firstSeq=1 lastSeq=2 source="ASSERT" + vgsp: WARNING: ... (NOCAT recType=5 errorCode=0x52a8b04) ... + rec 449 ts=0x18dbb29118ec6200 meta=0x0000000005bb9c48 args=0x1,0x2 + + ``key=value`` pairs fill ``rec_type`` / ``error_code`` / ``source_name`` / + ``seq``; ``assertCallStack [..]`` lines attach their bracketed frame list + to the immediately preceding record with the same ``seq``; anything that + matches none of the shapes is counted in ``unparsed`` (never dropped). + Lines starting with ``#``, ``<LABEL>-raw:``/``<LABEL>-decoded:`` headers and + blank lines are structure, not records. + +Report schema ``greeneye-log-decode/1``:: + + {"schema": "greeneye-log-decode/1", + "source": "<basename of the input>", + "count": <records after filters>, + "unparsed": <lines that matched no shape>, + "records": [{"kind": "raw-hex"|"record", "label"?: str, + "rec_type"?: int|str, "error_code"?: "0x...", + "source_name"?: str, "seq"?: int, + "frames": ["0x...", ...], "raw": str, + "resolved"?: [{"frame": "0x...", "symbol": str|None, + "offset": "0x..."|None}, ...]}], + "histogram": {"rec_type": {..}, "error_code": {..}, "source": {..}}} + +``resolved`` appears only when a symbol table is supplied and is aligned 1:1 +with ``frames``. ``--symbols`` accepts the project symbol-DB shape: a +top-level list, or an object with a ``symbols`` list, of entries +``{"name": str, "addr": int | "0x..."}`` (extra keys ignored). Each frame is +matched to the nearest symbol with ``addr <= frame`` and the offset +(``frame - addr``) is reported; an unreadable or malformed symbols file exits +non-zero with a clear message. + +Filtering (``decode`` subcommand): ``--kind`` selects raw-hex/record, +``--rec-type``/``--error-code`` accept decimal or ``0x`` hex, ``--source`` is +a case-insensitive substring on ``source_name`` and ``--limit`` caps the +records; ``count``/``histogram`` describe the filtered stream. ``--json +out.json`` writes the full report. + +Honesty: offline decoder only (L1 at best; no timing claims). Raw words are +exposed, not structurally ring-decoded; ``assertCallStack`` frames are offsets +from the load base named in the line and are symbolized as given. There is no +hardware, module, QEMU or guest interaction. + +Run:: + + python3 harness/log_decode.py decode <dump> [--symbols symbols.json] + python3 harness/log_decode.py demo +""" +from __future__ import annotations + +import argparse +import bisect +import json +import os +import re +import sys +import tempfile +from typing import Any + +SCHEMA = "greeneye-log-decode/1" +DEMO_FORMAT = "greeneye-log-decode-demo/1" + +#: How many ``resolved`` entries the human-readable output prints per record +#: (the JSON report always carries all of them). +RESOLVED_PRINT_LIMIT = 8 + + +class LogDecodeError(ValueError): + """An input or request the decoder cannot honour.""" + + +# --------------------------------------------------------------------------- +# Symbols: the project symbol-DB shape, nearest-below resolution +# --------------------------------------------------------------------------- + +def _parse_sym_addr(value: Any, where: str) -> int: + if isinstance(value, bool): + raise LogDecodeError(f"{where}: bad symbol addr {value!r}") + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value, 0) + except ValueError as exc: + raise LogDecodeError(f"{where}: bad symbol addr {value!r}") from exc + raise LogDecodeError(f"{where}: bad symbol addr {value!r}") + + +class SymbolTable: + """Nearest-below ``frame -> (name, offset)`` resolver. + + Holds the same entries the project symbol DBs carry; only ``addr`` and + ``name`` are used here. ``resolve`` returns ``None`` when no symbol has an + address at or below the frame. + """ + + def __init__(self, entries: list[tuple[int, str]]) -> None: + self._entries = sorted(entries, key=lambda item: item[0]) + self._addrs = [addr for addr, _ in self._entries] + + def __len__(self) -> int: + return len(self._entries) + + def resolve(self, addr: int) -> tuple[str, int] | None: + pos = bisect.bisect_right(self._addrs, addr) - 1 + if pos < 0: + return None + sym_addr, name = self._entries[pos] + return name, addr - sym_addr + + def annotate_frame(self, frame: str) -> dict: + try: + value = int(frame, 0) + except ValueError: + return {"frame": frame, "symbol": None, "offset": None} + hit = self.resolve(value) + if hit is None: + return {"frame": frame, "symbol": None, "offset": None} + name, offset = hit + return {"frame": frame, "symbol": name, "offset": f"{offset:#x}"} + + +def load_symbols(path: str) -> SymbolTable: + """Load the accepted symbol JSON; raise :class:`LogDecodeError` when bad.""" + if not os.path.isfile(path): + raise LogDecodeError(f"symbols file not found: {path}") + try: + with open(path, "r", encoding="utf-8") as fh: + payload = json.load(fh) + except OSError as exc: + raise LogDecodeError(f"cannot read symbols {path}: {exc}") from exc + except json.JSONDecodeError as exc: + raise LogDecodeError(f"symbols JSON {path} is not valid JSON: {exc}") from exc + if isinstance(payload, dict): + entries = payload.get("symbols") + if entries is None: + raise LogDecodeError( + f"symbols JSON {path}: object must have a 'symbols' list") + elif isinstance(payload, list): + entries = payload + else: + raise LogDecodeError( + f"symbols JSON {path}: expected a list or an object with 'symbols'") + if not isinstance(entries, list): + raise LogDecodeError(f"symbols JSON {path}: 'symbols' must be a list") + table: list[tuple[int, str]] = [] + for index, item in enumerate(entries): + where = f"symbols JSON {path}: entry[{index}]" + if not isinstance(item, dict): + raise LogDecodeError(f"{where} is not an object") + if "addr" not in item: + raise LogDecodeError(f"{where} has no 'addr'") + addr = _parse_sym_addr(item["addr"], where) + name = item.get("name") + if name is None: + name = f"sub_{addr:x}" + if not isinstance(name, str): + raise LogDecodeError(f"{where}: 'name' must be a string") + table.append((addr, name)) + return SymbolTable(table) + + +# --------------------------------------------------------------------------- +# Line shapes observed in the real evidence +# --------------------------------------------------------------------------- + +RAW_SECTION_RE = re.compile(r"^(?P<label>[A-Za-z0-9_.+-]+)-raw:\s+\d+\s+bytes\b") +DECODED_SECTION_RE = re.compile(r"^(?P<label>[A-Za-z0-9_.+-]+)-decoded:") +COMMENT_LABEL_RE = re.compile(r"^#\s*(?P<label>[A-Za-z][A-Za-z0-9_.+-]*)\s*$") +HEX_ROW_RE = re.compile( + r"^(?P<offset>[0-9a-fA-F]{6,16}):\s+" + r"(?P<words>[0-9a-fA-F]{1,16}(?:\s+[0-9a-fA-F]{1,16})*)\s*$") +DMSG_PREFIX_RE = re.compile( + r"^(?:\[[^\]]*\]\s*)?(?P<label>[A-Za-z0-9_.+-]+):\s+(?P<body>.*)$") +KV_RE = re.compile(r"(?P<key>[A-Za-z_][A-Za-z0-9_]*)=" + r"(?P<value>\"[^\"]*\"|[^\s,()]+)") +FRAME_LIST_RE = re.compile(r"\[([^\[\]]*)\]") +CONTINUATION_RE = re.compile(r"\brec\s+seq=(?P<seq>\d+)\s+assertCallStack\b") +DECODED_REC_RE = re.compile( + r"^rec\s+(?P<seq>\d+)\s+ts=(?P<ts>0x[0-9a-fA-F]+)\s+" + r"meta=(?P<meta>\S+)\s+args=(?P<args>.*)$") +REC_SEQ_RE = re.compile(r"\brec\s+seq=(?P<seq>\d+)") + + +def _parse_kv(body: str) -> dict[str, str]: + fields: dict[str, str] = {} + for match in KV_RE.finditer(body): + value = match.group("value") + if value.startswith('"') and value.endswith('"') and len(value) >= 2: + value = value[1:-1] + fields[match.group("key")] = value + return fields + + +def _int_or_text(value: str) -> int | str: + try: + return int(value, 0) + except ValueError: + return value + + +def _norm_hex(value: str) -> str: + try: + return f"{int(value, 0):#x}" + except ValueError: + return value + + +def _parse_frame_tokens(text: str) -> list[str]: + text = text.strip() + if text.startswith("[") and text.endswith("]"): + text = text[1:-1] + frames: list[str] = [] + for token in text.replace(",", " ").split(): + try: + frames.append(f"{int(token, 0):#x}") + except ValueError: + continue + return frames + + +def _frames_from_brackets(text: str) -> list[str]: + match = FRAME_LIST_RE.search(text) + if not match: + return [] + return _parse_frame_tokens(match.group(1)) + + +def _looks_like_record(body: str) -> bool: + if DECODED_REC_RE.match(body): + return True + if "recType=" in body or "errorCode=" in body: + return True + if REC_SEQ_RE.search(body): + return True + return False + + +def _record_from_body(body: str, label: str | None, raw: str | None = None) -> dict: + if raw is None: + raw = body + match = DECODED_REC_RE.match(body) + if match: + return { + "kind": "record", + "label": label, + "seq": int(match.group("seq")), + "frames": _parse_frame_tokens(match.group("args")), + "raw": raw, + } + record: dict[str, Any] = {"kind": "record", "label": label, + "frames": [], "raw": raw} + fields = _parse_kv(body) + if "recType" in fields: + record["rec_type"] = _int_or_text(fields["recType"]) + if "errorCode" in fields: + record["error_code"] = _norm_hex(fields["errorCode"]) + if "source" in fields: + record["source_name"] = fields["source"] + if "seq" in fields: + record["seq"] = int(fields["seq"], 0) + return record + + +def _raw_record(label: str | None, words: list[int]) -> dict: + data = b"".join(word.to_bytes(8, "little") for word in words) + return { + "kind": "raw-hex", + "label": label, + "source_name": label, + "frames": [f"{word:#x}" for word in words], + "raw": data.hex(), + } + + +# --------------------------------------------------------------------------- +# Decode +# --------------------------------------------------------------------------- + +def _histogram(records: list[dict]) -> dict[str, dict[str, int]]: + hist: dict[str, dict[str, int]] = {"rec_type": {}, "error_code": {}, + "source": {}} + for record in records: + if "rec_type" in record: + key = str(record["rec_type"]) + hist["rec_type"][key] = hist["rec_type"].get(key, 0) + 1 + if "error_code" in record: + key = str(record["error_code"]) + hist["error_code"][key] = hist["error_code"].get(key, 0) + 1 + if record.get("source_name"): + key = str(record["source_name"]) + hist["source"][key] = hist["source"].get(key, 0) + 1 + return {name: dict(sorted(values.items())) for name, values in hist.items()} + + +def decode_text(text: str, source: str = "<text>", + symbols: SymbolTable | None = None) -> dict: + """Decode one dump/evidence text into a ``greeneye-log-decode/1`` report.""" + records: list[dict] = [] + unparsed = 0 + raw_state: dict[str, Any] | None = None + current_label: str | None = None + + def flush_raw() -> None: + nonlocal raw_state + if raw_state is None: + return + records.append(_raw_record(raw_state["label"], raw_state["words"])) + raw_state = None + + for line in text.splitlines(): + stripped = line.strip() + if not stripped: + continue + if stripped.startswith("#"): + match = COMMENT_LABEL_RE.match(stripped) + if match: + current_label = match.group("label") + continue + match = RAW_SECTION_RE.match(stripped) + if match: + flush_raw() + raw_state = {"label": match.group("label"), "words": []} + current_label = match.group("label") + continue + match = DECODED_SECTION_RE.match(stripped) + if match: + flush_raw() + current_label = match.group("label") + continue + if raw_state is not None: + match = HEX_ROW_RE.match(stripped) + if match: + raw_state["words"].extend( + int(token, 16) for token in match.group("words").split()) + continue + flush_raw() + else: + match = HEX_ROW_RE.match(stripped) + if match: + raw_state = {"label": None, "words": []} + raw_state["words"].extend( + int(token, 16) for token in match.group("words").split()) + continue + + label = current_label + body = stripped + prefix = DMSG_PREFIX_RE.match(stripped) + if prefix: + label = prefix.group("label") + body = prefix.group("body").strip() + + continuation = CONTINUATION_RE.search(body) + if continuation: + seq = int(continuation.group("seq")) + frames = _frames_from_brackets(body) + last = records[-1] if records else None + if (last is not None and last.get("kind") == "record" + and last.get("seq") == seq and not last.get("frames")): + last["frames"] = frames + else: + records.append({"kind": "record", "label": label, "seq": seq, + "frames": frames, "raw": stripped}) + continue + + if _looks_like_record(body): + records.append(_record_from_body(body, label, stripped)) + continue + unparsed += 1 + + flush_raw() + if symbols is not None: + for record in records: + if record.get("frames"): + record["resolved"] = [symbols.annotate_frame(frame) + for frame in record["frames"]] + return { + "schema": SCHEMA, + "source": source, + "count": len(records), + "unparsed": unparsed, + "records": records, + "histogram": _histogram(records), + } + + +def decode_path(path: str, symbols_path: str | None = None) -> dict: + """Decode the file at ``path``; ``source`` in the report is its basename.""" + if not os.path.isfile(path): + raise LogDecodeError(f"input not found: {path}") + symbols = load_symbols(symbols_path) if symbols_path else None + try: + with open(path, "r", encoding="utf-8", errors="replace") as fh: + text = fh.read() + except OSError as exc: + raise LogDecodeError(f"cannot read input {path}: {exc}") from exc + return decode_text(text, source=os.path.basename(path), symbols=symbols) + + +# --------------------------------------------------------------------------- +# Filters +# --------------------------------------------------------------------------- + +def _want_int(value: Any, what: str) -> int: + if isinstance(value, bool): + raise LogDecodeError(f"bad {what} {value!r}") + if isinstance(value, int): + return value + try: + return int(str(value), 0) + except ValueError as exc: + raise LogDecodeError(f"bad {what} {value!r}") from exc + + +def _match_int(record_value: Any, wanted: int) -> bool: + if record_value is None: + return False + if isinstance(record_value, int): + return record_value == wanted + try: + return int(str(record_value), 0) == wanted + except ValueError: + return False + + +def apply_filters(report: dict, *, rec_type: Any = None, source: str | None = None, + error_code: Any = None, kind: str | None = None, + limit: int | None = None) -> dict: + """Return a copy of ``report`` with the requested record filters applied.""" + records = list(report["records"]) + if kind is not None: + if kind not in ("raw-hex", "record"): + raise LogDecodeError(f"bad kind {kind!r} (want raw-hex or record)") + records = [r for r in records if r["kind"] == kind] + if rec_type is not None: + wanted = _want_int(rec_type, "rec-type") + records = [r for r in records if _match_int(r.get("rec_type"), wanted)] + if error_code is not None: + wanted = _want_int(error_code, "error-code") + records = [r for r in records if _match_int(r.get("error_code"), wanted)] + if source is not None: + needle = source.lower() + records = [r for r in records + if needle in str(r.get("source_name") or "").lower()] + if limit is not None: + if limit < 0: + raise LogDecodeError(f"bad limit {limit!r} (must be >= 0)") + records = records[:limit] + filtered = dict(report) + filtered["records"] = records + filtered["count"] = len(records) + filtered["histogram"] = _histogram(records) + return filtered + + +# --------------------------------------------------------------------------- +# Demo: tiny crafted inputs for both real formats (tempdir only) +# --------------------------------------------------------------------------- + +_DEMO_RAW = """\ +LOGINIT-raw: 64 bytes (allocated 64) VA=0xffff0000 dma=0x1000 +00000000: 00000005 00000000 0000000a deadbeef +00000010: 01020304 05060708 090a0b0c 0d0e0f10 +""" + +_DEMO_RECORDS = """\ +[Tue Oct 6 12:40:52 2026] vgsp-nocat: rec seq=1 recType=5 errorCode=0x57f1344 subsystem=0x0 bugcheck=0x0 payload=1212 source="ASSERT" faultingEngine="" +[Tue Oct 6 12:40:52 2026] vgsp-nocat: rec seq=1 assertCallStack count=1 [0x401000,0x402000,0x0] (offsets from load base 0x4000000) +[Tue Oct 6 12:40:53 2026] vgsp-nocat: entry[0]: count=1 recType=5 errorCode=0x57f1344 firstSeq=1 lastSeq=1 source="ASSERT" +an unmatched filler line +rec 7 ts=0x18dbb29118ec6200 meta=0x0000000005bb9c48 args=0x401000,0x402000 +""" + +_DEMO_SYMBOLS = { + "symbols": [ + {"name": "gr_init", "addr": "0x401000"}, + {"name": "gr_done", "addr": int("0x403000", 0)}, + ] +} + + +def run_demo() -> dict: + """Decode crafted inputs for both formats; deterministic, tempdir only.""" + with tempfile.TemporaryDirectory(prefix="log-decode-demo-") as tmp: + raw_path = os.path.join(tmp, "raw.txt") + records_path = os.path.join(tmp, "records.txt") + symbols_path = os.path.join(tmp, "symbols.json") + with open(raw_path, "w", encoding="utf-8") as fh: + fh.write(_DEMO_RAW) + with open(records_path, "w", encoding="utf-8") as fh: + fh.write(_DEMO_RECORDS) + with open(symbols_path, "w", encoding="utf-8") as fh: + json.dump(_DEMO_SYMBOLS, fh, indent=2, sort_keys=True) + fh.write("\n") + raw_report = decode_path(raw_path) + records_report = decode_path(records_path, symbols_path) + + raw_records = raw_report["records"] + record_records = records_report["records"] + first = record_records[0] if record_records else {} + resolved = first.get("resolved") or [] + ok = ( + raw_report["count"] == 1 + and raw_report["unparsed"] == 0 + and len(raw_records) == 1 + and len(raw_records[0]["frames"]) == 8 + and raw_records[0]["frames"][3] == "0xdeadbeef" + and len(raw_records[0]["raw"]) == 2 * 8 * 8 + and records_report["count"] == 3 + and records_report["unparsed"] == 1 + and first.get("rec_type") == 5 + and first.get("error_code") == "0x57f1344" + and first.get("source_name") == "ASSERT" + and first.get("frames") == ["0x401000", "0x402000", "0x0"] + and len(resolved) == 3 + and resolved[0].get("symbol") == "gr_init" + and resolved[1].get("symbol") == "gr_init" + and resolved[1].get("offset") == "0x1000" + and resolved[2].get("symbol") is None + ) + return { + "format": DEMO_FORMAT, + "ok": ok, + "raw": raw_report, + "records": records_report, + } + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def _format_record(record: dict) -> list[str]: + bits = [f"{record['kind']:<8}"] + for key in ("label", "seq", "rec_type", "error_code", "source_name"): + if record.get(key) is not None: + bits.append(f"{key}={record[key]}") + bits.append(f"frames={len(record.get('frames') or [])}") + lines = [" ".join(bits)] + resolved = record.get("resolved") or [] + for item in resolved[:RESOLVED_PRINT_LIMIT]: + if item.get("symbol") is None: + lines.append(f" {item['frame']} <unknown>") + else: + lines.append( + f" {item['frame']} {item['symbol']}+{item['offset']}") + if len(resolved) > RESOLVED_PRINT_LIMIT: + lines.append(f" ... {len(resolved) - RESOLVED_PRINT_LIMIT} more") + return lines + + +def _write_json(report: dict, path: str) -> str: + target = os.path.abspath(path) + parent = os.path.dirname(target) + if parent: + os.makedirs(parent, exist_ok=True) + with open(target, "w", encoding="utf-8") as fh: + json.dump(report, fh, indent=2, sort_keys=True) + fh.write("\n") + return target + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="log_decode.py", + description="Decode GSP-RM log-ring and NOCAT record dumps (offline).", + ) + sub = parser.add_subparsers(dest="command") + decode = sub.add_parser( + "decode", help="decode a raw ring dump or dmesg/evidence log") + decode.add_argument("input", help="dump/log text file to decode") + decode.add_argument("--symbols", help="symbol DB JSON (list or {symbols:[..]})") + decode.add_argument("--rec-type", help="keep records with this recType") + decode.add_argument("--source", help="case-insensitive source substring") + decode.add_argument("--error-code", help="keep records with this errorCode") + decode.add_argument("--kind", choices=("raw-hex", "record"), + help="keep only this record kind") + decode.add_argument("--limit", type=int, help="max records to emit") + decode.add_argument("--json", help="write the full report to this file") + sub.add_parser("demo", help="decode crafted samples for both real formats") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "decode": + report = decode_path(args.input, args.symbols) + report = apply_filters( + report, + rec_type=args.rec_type, + source=args.source, + error_code=args.error_code, + kind=args.kind, + limit=args.limit, + ) + for record in report["records"]: + for line in _format_record(record): + print(line) + summary = { + "schema": report["schema"], + "source": report["source"], + "count": report["count"], + "unparsed": report["unparsed"], + "histogram": report["histogram"], + } + if args.json: + summary["report_json"] = _write_json(report, args.json) + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 + if args.command == "demo": + demo = run_demo() + raw = demo["raw"] + records = demo["records"] + print(f"{raw['source']}: records={raw['count']} " + f"unparsed={raw['unparsed']} " + f"frames={len(raw['records'][0]['frames'])} " + f"raw_bytes={len(raw['records'][0]['raw']) // 2}") + print(f"{records['source']}: records={records['count']} " + f"unparsed={records['unparsed']}") + print(f"histogram: {json.dumps(records['histogram'], sort_keys=True)}") + for item in (records["records"][0].get("resolved") or []): + if item.get("symbol") is None: + print(f" {item['frame']} <unknown>") + else: + print(f" {item['frame']} {item['symbol']}+{item['offset']}") + print(json.dumps({"format": demo["format"], "ok": demo["ok"]}, + sort_keys=True)) + return 0 if demo["ok"] else 3 + parser.print_help() + return 2 + except LogDecodeError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/rehost.py b/debug/risc-v-gsp-rm-vm/harness/rehost.py new file mode 100644 index 0000000..fb554db --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/rehost.py @@ -0,0 +1,428 @@ +#!/usr/bin/env python3 +"""Surgical rehost: run a bounded function of a firmware image on the native emulator. + +The standalone simulator boots the real 570.144 image to the message-loop port +wait (B3). Code *after* that wall (the RM handlers, e.g. the GR promote/commit +chain) is only reachable when host traffic exists, which is the B4/B5 rung. +This harness provides the offline alternative the workflow review ranks first +(docs/39 U2): start the machine at a chosen function entry with a crafted +register/memory state, run forward with a bounded budget, and stop on return, +fault, unhandled ecall or budget. Every run records a session and a report; +snapshots before/after a run show what the function changed. + +Honesty rules: + +* This tool executes firmware code; it never touches a GPU, kernel module, + mdev, guest or network. It is development-only. +* A rehosted run is **not** a boot: the function runs with only the state the + caller supplied. The report says so, and any fault (first missing state) is + recorded as evidence, never papered over. +* No timing/cycle claim; fidelity is L0/L1 (docs/06). +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import sys +import time +from typing import Any + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from greeneye.elf import load_elf # noqa: E402 +from greeneye.errors import GreenEyeError # noqa: E402 +from greeneye.sim_backend import launch_native # noqa: E402 +from sim.devices import load_profile, registry_from_profile # noqa: E402 +from sim.gsp_boot import _apply_boot_shim # noqa: E402 +from sim.libos import LibosSyscalls # noqa: E402 +from sim.machine import R_BP, R_ECALL, R_FAULT, R_WATCH # noqa: E402 + +#: Schema tag of the rehost report. +REHOST_FORMAT = "greeneye-rehost/1" +#: Schema tag of the demo result. +DEMO_FORMAT = "greeneye-rehost-demo/1" + +DEFAULT_IMAGE = os.path.join( + TOOL_ROOT, "..", "..", "firmware", "57X-GSP", "570.144", "gsp-570.144.fwimage.bin" +) +DEFAULT_PROFILE = os.path.join(TOOL_ROOT, "sim", "profiles", "tu104-570.144.json") +DEMO_ELF = os.path.join(TOOL_ROOT, "firmware", "uproc_demo", "build", "uproc_demo.elf") + +#: Wall-clock guard: a rehost that spins in a fault loop must not hang the tool. +DEFAULT_TIMEOUT = 60.0 +#: Instruction budget per default run. +DEFAULT_BUDGET = 5_000_000 +#: Chunk size between stop checks (keeps the loop responsive to timeouts). +RUN_CHUNK = 250_000 +#: Sentinel instruction (``ebreak``) written at the return trampoline. +EBREAK = 0x00100073 + + +class RehostError(GreenEyeError): + """A rehost input, image or argument is invalid.""" + + +def _parse_u64(value: Any) -> int: + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value & 0xFFFF_FFFF_FFFF_FFFF + if isinstance(value, str): + text = value.strip() + if not text: + raise RehostError("empty integer") + return int(text, 0) & 0xFFFF_FFFF_FFFF_FFFF + raise RehostError(f"cannot parse integer from {value!r}") + + +def parse_hex_bytes(spec: str) -> bytes: + """Parse ``aabbcc`` (whitespace tolerated) into bytes.""" + text = "".join(spec.split()) + if len(text) % 2: + raise RehostError(f"hex payload must have an even number of digits: {spec!r}") + try: + return bytes.fromhex(text) + except ValueError as exc: + raise RehostError(f"bad hex payload {spec!r}: {exc}") from exc + + +def parse_mem_spec(spec: str) -> tuple[int, bytes]: + """Parse ``<addr>=<hex>`` into ``(address, bytes)``.""" + if "=" not in spec: + raise RehostError(f"memory spec must be <addr>=<hex>: {spec!r}") + addr_text, _, data_text = spec.partition("=") + return _parse_u64(addr_text), parse_hex_bytes(data_text) + + +def parse_snap_spec(spec: str) -> tuple[int, int]: + """Parse ``<addr>:<size>`` into ``(address, size)``.""" + addr_text, sep, size_text = spec.partition(":") + if not sep: + raise RehostError(f"snapshot spec must be <addr>:<size>: {spec!r}") + size = _parse_u64(size_text) + if size <= 0: + raise RehostError(f"snapshot size must be positive: {spec!r}") + return _parse_u64(addr_text), size + + +def _sha256_file(path: str) -> str: + digest = hashlib.sha256() + with open(path, "rb") as fh: + for chunk in iter(lambda: fh.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def _region_diff(before: bytes, after: bytes) -> dict: + changed = 0 + first = None + for offset, (a, b) in enumerate(zip(before, after)): + if a != b: + changed += 1 + if first is None: + first = offset + return { + "changed_bytes": changed, + "first_diff_offset": first, + "before_sha256": hashlib.sha256(before).hexdigest(), + "after_sha256": hashlib.sha256(after).hexdigest(), + } + + +def _registers(machine) -> dict: + regs = {f"x{i}": machine.x[i] for i in range(32) if i and machine.x[i]} + regs["pc"] = machine.pc + return regs + + +def rehost( + *, + image: str, + entry: int | str, + profile_path: str | None = None, + args: tuple = (), + mem: tuple = (), + snap: tuple = (), + budget: int = DEFAULT_BUDGET, + timeout: float = DEFAULT_TIMEOUT, + out: str | None = None, + record=None, + name: str = "rehost", + apply_shim: bool = True, + unknown_mmio: str = "lenient", +) -> dict: + """Run one bounded function call on the native emulator; return a report.""" + if not os.path.isfile(image): + raise RehostError(f"image not found: {image}") + if budget <= 0: + raise RehostError(f"budget must be positive, got {budget}") + image_obj = load_elf(image) + if not image_obj.is_riscv: + raise RehostError(f"{image}: not a RISC-V ELF image") + + profile = load_profile(profile_path) if profile_path else None + load = (profile or {}).get("load", {}) + ram_base = _parse_u64(load["ram_base"]) if "ram_base" in load else 0x80000000 + memory_mb = int(load.get("memory_mb", 256)) + + session = launch_native( + kernel=image, + memory_mb=memory_mb, + ram_base=ram_base, + record=record if record is not None else bool(out), + name=name, + device_map=None, + unknown_mmio=unknown_mmio, + bias=0, + ) + machine = session.target.machine + try: + if profile is not None: + machine.devices = registry_from_profile(profile, machine=machine) + machine.translator = getattr(machine.devices, "dma", None) + libos = LibosSyscalls() + machine.syscall_handler = libos.handle + if apply_shim: + shim_meta = {"enabled": True, "assumptions": []} + shim = dict(profile.get("boot_shim", {})) + shim["_image_path"] = image + _apply_boot_shim(machine, shim, image_obj, shim_meta) + + # Resolve a symbol name through the ELF symbol table when possible. + if isinstance(entry, str): + try: + entry_addr = session.symbols.lookup(entry) if session.symbols else None + except GreenEyeError: + entry_addr = None + if entry_addr is None: + raise RehostError(f"symbol {entry!r} not found in {os.path.basename(image)}") + else: + entry_addr = _parse_u64(entry) + + for addr, data in mem: + machine.write_mem(addr, data) + + snapshots = [] + for addr, size in snap: + before = machine.read_mem(addr, size) + snapshots.append({"addr": addr, "size": size, "before": before}) + + # Return trampoline well above the image, inside the RAM arena. + sentinel = (machine.ram_base + machine.ram_size - 0x100) & ~0xF + stack = (sentinel - 0x2000) & ~0xF + trampoline = EBREAK.to_bytes(4, "little") + machine.write_mem(sentinel, trampoline) + machine.write_mem(sentinel + 4, trampoline) + for index, value in enumerate(tuple(args)[:8]): + machine.x[10 + index] = _parse_u64(value) + machine.x[1] = sentinel # ra + machine.x[2] = stack # sp + machine.pc = entry_addr + + entered = _registers(machine) + started = time.monotonic() + status = "budget" + fault: dict | None = None + stop_pc = machine.pc + remaining = budget + while remaining > 0: + if time.monotonic() - started > timeout: + status = "timeout" + break + chunk = min(RUN_CHUNK, remaining) + reason, stop_pc, a, b = machine.run(chunk, breakpoints={sentinel}) + remaining -= chunk + if reason == R_BP: + status = "return" + break + if reason == R_FAULT: + status = "fault" + fault = {"cause": a, "tval": f"{b:#x}"} + break + if reason == R_ECALL: + handled, result = machine.syscall_handler(machine, stop_pc) if machine.syscall_handler else (False, None) + if not handled: + status = "ecall" + break + if result is not None: + machine.x[10] = result & 0xFFFF_FFFF_FFFF_FFFF + continue + if reason == R_WATCH: + continue # watchpoints recorded by the caller's own tooling + # R_BUDGET: keep going until the total budget is spent. + if status == "budget" and remaining <= 0: + status = "budget" + + report: dict[str, Any] = { + "format": REHOST_FORMAT, + "image": os.path.abspath(image), + "image_sha256": _sha256_file(image), + "profile": os.path.basename(profile_path) if profile_path else None, + "entry": f"{entry_addr:#x}", + "args": [_parse_u64(v) for v in tuple(args)[:8]], + "status": status, + "stop_pc": f"{stop_pc:#x}", + "fault": fault, + "retval": machine.x[10], + "instructions": 0 if machine.insns is None else machine.insns, + "wall_seconds": round(time.monotonic() - started, 4), + "registers_entered": {k: f"{v:#x}" if isinstance(v, int) else v for k, v in entered.items()}, + "registers_exited": {k: f"{v:#x}" if isinstance(v, int) else v for k, v in _registers(machine).items()}, + "mmio_unknown": machine.mmio_unknown, + "snapshots": [], + "limits": [ + "Not a boot: the function ran with only the supplied state.", + "No timing/cycle claim (L0/L1 fidelity, docs/06).", + "A fault is the first missing state observed, not proof of a product defect.", + ], + } + for item in snapshots: + after = machine.read_mem(item["addr"], item["size"]) + report["snapshots"].append( + { + "addr": f"{item['addr']:#x}", + "size": item["size"], + **_region_diff(item["before"], after), + } + ) + if getattr(session.recorder, "dir", None) is not None: + session.note("rehost", entry=f"{entry_addr:#x}", status=status, stop_pc=f"{stop_pc:#x}") + report["session"] = session.recorder.dir + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "rehost-report.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(report, fh, indent=2, sort_keys=True) + fh.write("\n") + report["report_json"] = os.path.abspath(path) + return report + finally: + session.close() + + +def run_demo(out: str | None = None) -> dict: + """Rehost a known demo function to prove the loop (synthetic, deterministic).""" + if not os.path.isfile(DEMO_ELF): + raise RehostError( + f"demo firmware not built: {DEMO_ELF} (run `make demo` in the tool dir)" + ) + buffer_addr, payload = 0x80012000, bytes([1, 2, 3, 4]) + expected = sum(payload) + first = rehost( + image=DEMO_ELF, + entry="rpc_sum", + args=(buffer_addr, len(payload)), + mem=((buffer_addr, payload),), + snap=(), + budget=100_000, + timeout=20.0, + record=False, + name="rehost-demo", + ) + second = rehost( + image=DEMO_ELF, + entry="rpc_sum", + args=(buffer_addr, len(payload)), + mem=((buffer_addr, payload),), + snap=(), + budget=100_000, + timeout=20.0, + record=False, + name="rehost-demo", + ) + ok = ( + first["status"] == "return" + and first["retval"] == expected + and first["status"] == second["status"] + and first["retval"] == second["retval"] + ) + payload_result = { + "format": DEMO_FORMAT, + "ok": ok, + "expected": expected, + "first": { + "status": first["status"], + "retval": first["retval"], + "wall_seconds": first["wall_seconds"], + }, + "second": { + "status": second["status"], + "retval": second["retval"], + }, + } + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "rehost-demo.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(payload_result, fh, indent=2, sort_keys=True) + fh.write("\n") + payload_result["artifacts"] = {"demo_json": os.path.abspath(path)} + return payload_result + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="rehost.py", + description=( + "Run one bounded firmware function on the native emulator " + "(surgical rehost). Offline only; never touches hardware." + ), + ) + sub = parser.add_subparsers(dest="command") + run_cmd = sub.add_parser("run", help="rehost a function from an image") + run_cmd.add_argument("--image", default=DEFAULT_IMAGE, help="firmware ELF") + run_cmd.add_argument("--profile", default=DEFAULT_PROFILE, help="device profile JSON") + run_cmd.add_argument("--entry", required=True, help="function entry (hex address or symbol)") + run_cmd.add_argument("--arg", action="append", default=[], help="a0..a7 value (repeatable)") + run_cmd.add_argument("--mem", action="append", default=[], help="<addr>=<hex> pre-write (repeatable)") + run_cmd.add_argument("--snap", action="append", default=[], help="<addr>:<size> snapshot (repeatable)") + run_cmd.add_argument("--budget", type=int, default=DEFAULT_BUDGET) + run_cmd.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT) + run_cmd.add_argument("--no-shim", action="store_true", help="do not apply the profile boot shim") + run_cmd.add_argument("--out", help="output directory (report + session)") + demo_cmd = sub.add_parser("demo", help="synthetic self-test on the demo firmware") + demo_cmd.add_argument("--out", help="output directory") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "run": + report = rehost( + image=args.image, + profile_path=args.profile or None, + entry=args.entry, + args=tuple(args.arg), + mem=tuple(parse_mem_spec(spec) for spec in args.mem), + snap=tuple(parse_snap_spec(spec) for spec in args.snap), + budget=args.budget, + timeout=args.timeout, + out=args.out, + apply_shim=not args.no_shim, + ) + elif args.command == "demo": + report = run_demo(args.out) + else: + parser.print_help() + return 2 + except RehostError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + summary_keys = ( + "format", "ok", "entry", "status", "stop_pc", "retval", "fault", + "instructions", "wall_seconds", "report_json", "session", "artifacts", + ) + print(json.dumps({k: report[k] for k in summary_keys if k in report}, indent=2, sort_keys=True)) + if args.command == "demo" and not report.get("ok"): + return 3 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/rm_capture.py b/debug/risc-v-gsp-rm-vm/harness/rm_capture.py new file mode 100644 index 0000000..4aa459b --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/rm_capture.py @@ -0,0 +1,1251 @@ +#!/usr/bin/env python3 +"""Real-client GSP RPC capture ingestion, comparison and replay prep. + +Ingests the text output of the supervised host-client capture (the GSP RPC +send-path instrumentation described by the CAPTURE-RUNBOOK) and turns it into +structured, replayable artifacts: + +* **decode** — a raw capture log (dmesg text with the documented ``GE-SEND``, + ``GE-CTRL``, ``GE-ALLOC``, ``GE-GOLDEN-*`` and ``GE-CTRL-DONE`` tags) into a + ``greeneye-rm-rpc-capture/1`` document: one ordered record per RPC/alloc, + with the wire function id, the named control/class id, handles, caller + return address, and the captured parameter bytes. +* **compare** — two capture documents (for example, the real host client's + sequence vs. the open-iov host ladder's replay, or two firmware/driver + versions) aligned by record identity, with per-record parameter agreement, + coverage and explicit unmatched lists. +* **stimulus** — a ``greeneye-rm-rpc-stimulus/1`` ordered message list for the + simulator's B4/B5 CSB/message-queue seam (not wired yet). + +Honesty rules: + +* Truncated payloads are flagged, never silently padded. The runbook's hex + helper caps a dump at 192 bytes; a capture produced by that helper will mark + every larger payload ``truncated`` so a reader can see what is missing. +* Handles, physical addresses and caller addresses are capture-specific; the + compare report labels them, and the stimulus output says so explicitly. +* This tool only reads files and writes reports. It never touches a GPU, + kernel module, mdev, guest or network. The capture it ingests is a + supervised, human-run physical action (RUNBOOK P1/P7); this tool never runs + or instructs it beyond printing the tag grammar. +* No timing/cycle claim is made anywhere; fidelity is L0/L1 (docs/06). +""" +from __future__ import annotations + +import argparse +import difflib +import hashlib +import json +import os +import re +import sys +import time +from dataclasses import dataclass, field +from typing import Any + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from greeneye.errors import GreenEyeError # noqa: E402 + +#: Schema tag of the decoded capture document. +CAPTURE_FORMAT = "greeneye-rm-rpc-capture/1" +#: Schema tag of the comparison report. +COMPARE_FORMAT = "greeneye-rm-rpc-compare/1" +#: Schema tag of the replay stimulus document. +STIMULUS_FORMAT = "greeneye-rm-rpc-stimulus/1" +#: Schema tag of the demo result line. +DEMO_FORMAT = "greeneye-rm-rpc-capture-demo/1" + +#: Size of ``rpc_message_header_v``: the wire payload starts after it. +RPC_HEADER_BYTES = 32 + +# Wire function ids (``rpc_global_enums.h``): used to derive the embedded +# control/class id from a ``GE-SEND`` payload when no named ``GE-CTRL`` line +# was captured for it. +GSP_RM_CONTROL = 76 +GSP_RM_ALLOC = 103 +# Payload offsets inside the flat parameter blob (runbook §4). +CONTROL_CMD_OFFSET = 8 +ALLOC_CLASS_OFFSET = 12 + +KINDS = ("send", "control", "alloc", "golden", "ctrl-done") +HEX_KIND_TAG = {"send": "SEND", "control": "CTRL", "alloc": "ALLOC"} +TAG_KIND = {v: k for k, v in HEX_KIND_TAG.items()} + +# FROZEN BEFORE THE FIRST MEASUREMENT (2026-10-06). Do not move silently: +# a threshold change requires a new recorded run and a note in docs/45. +#: Conservative ESTIMATE (not measured on a human) of decoding one golden +#: sequence by eye from a capture log: grep the tags, count records, walk each +#: parameter blob against the RPC structs, transcribe into a replay table. +MANUAL_DECODE_BASELINE_SECONDS = 1800 +MANUAL_BASELINE_LABEL = "conservative estimate (not measured on a human)" +#: The tool path must stay at or below this fraction of the estimate. +WORKFLOW_THRESHOLD_FRACTION = 0.05 + +_HEX_RE = re.compile( + r"GE-(SEND|CTRL|ALLOC)\s+([0-9a-fA-F]{1,4})\s*:\s*((?:[0-9a-fA-F]{2}\s*)+)" +) +_KV_RE = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=(\S+)") +_TS_RE = re.compile(r"^\[([^\]]+)\]\s*") +_GOLDEN_TRIGGER_RE = re.compile( + r"GE-GOLDEN-TRIGGER\s+gspclient=(\d+)(?:\s+mig=(\d+))?" +) +_GOLDEN_END_RE = re.compile(r"GE-GOLDEN-END\s+status=(\S+)") +_GOLDEN_BEGIN_RE = re.compile(r"GE-GOLDEN-BEGIN") +_CTRL_DONE_RE = re.compile(r"GE-CTRL-DONE\b") + + +class RmCaptureError(GreenEyeError): + """A capture input is absent, unreadable or not in a supported schema.""" + + +def _parse_int(value: Any, default: int | None = None) -> int | None: + """Parse ints, decimal strings and ``0x`` strings; ``default`` on failure.""" + if value is None: + return default + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value + if isinstance(value, float): + return int(value) + if isinstance(value, str): + text = value.strip() + if not text: + return default + try: + return int(text, 0) + except ValueError: + try: + return int(text, 16) + except ValueError: + return default + return default + + +def _sha256_bytes(data: bytes) -> str: + return hashlib.sha256(data).hexdigest() + + +def _sha256_file(path: str) -> str: + digest = hashlib.sha256() + with open(path, "rb") as fh: + for chunk in iter(lambda: fh.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def canonical_json(payload: Any) -> str: + """Deterministic JSON text (sorted keys, fixed indent, trailing newline).""" + return json.dumps(payload, indent=2, sort_keys=True) + "\n" + + +@dataclass +class CaptureRecord: + """One decoded capture record (an RPC, an alloc, or a golden marker).""" + + index: int + kind: str + line: int + ts: str = "" + fn: int | None = None + length: int | None = None + size: int | None = None + cmd: int | None = None + hclass: int | None = None + hclient: int | None = None + hobject: int | None = None + hparent: int | None = None + caller: int | None = None + caller_raw: str = "" + marker: str = "" + gspclient: int | None = None + mig: int | None = None + status: int | None = None + rpc_status: int | None = None + params_status: int | None = None + chunks: dict[int, bytes] = field(default_factory=dict) + anomalies: list[str] = field(default_factory=list) + + # -- parameter assembly ------------------------------------------------- + @property + def params(self) -> bytes: + """Captured parameter bytes, assembled from the hex chunks in order. + + A gap between chunks is zero-filled and recorded as an anomaly; the + raw chunk map stays available on the record. + """ + if not self.chunks: + return b"" + end = max(offset + len(data) for offset, data in self.chunks.items()) + out = bytearray(end) + for offset, data in sorted(self.chunks.items()): + out[offset : offset + len(data)] = data + return bytes(out) + + @property + def expected_params_size(self) -> int | None: + """Declared parameter size, or ``None`` when the tag does not carry one.""" + if self.kind == "send": + if self.length is None or self.length < RPC_HEADER_BYTES: + return None + return self.length - RPC_HEADER_BYTES + if self.kind in ("control", "alloc"): + return self.size + return None + + @property + def truncated(self) -> bool: + expected = self.expected_params_size + return expected is not None and len(self.params) < expected + + @property + def overrun(self) -> bool: + expected = self.expected_params_size + return expected is not None and len(self.params) > expected + + @property + def wire_id(self) -> int | None: + """Named id: ``cmd``/``hclass`` when captured, else derived from payload.""" + if self.kind == "control": + return self.cmd + if self.kind == "alloc": + return self.hclass + if self.kind == "send": + payload = self.params + if self.fn == GSP_RM_CONTROL and len(payload) >= CONTROL_CMD_OFFSET + 4: + return int.from_bytes( + payload[CONTROL_CMD_OFFSET : CONTROL_CMD_OFFSET + 4], "little" + ) + if self.fn == GSP_RM_ALLOC and len(payload) >= ALLOC_CLASS_OFFSET + 4: + return int.from_bytes( + payload[ALLOC_CLASS_OFFSET : ALLOC_CLASS_OFFSET + 4], "little" + ) + return None + + @property + def signature(self) -> str: + """Identity used for compare alignment (kind + id + declared size).""" + if self.kind == "golden": + return f"golden:{self.marker}" + if self.kind == "ctrl-done": + return f"ctrl-done:{self.cmd:#x}" if self.cmd is not None else "ctrl-done:?" + identifier = self.wire_id + id_text = f"{identifier:#x}" if identifier is not None else "?" + fn_text = f"{self.fn:#x}" if self.fn is not None else "?" + size = self.expected_params_size + size_text = f"{size:#x}" if size is not None else "?" + return f"{self.kind}:{id_text}:{fn_text}:{size_text}" + + @property + def label(self) -> str: + """Short human label for reports.""" + if self.kind == "golden": + return f"GOLDEN {self.marker}" + ( + f" status={self.status:#x}" if self.status is not None else "" + ) + identifier = self.wire_id + id_text = f"{identifier:#x}" if identifier is not None else "?" + if self.kind == "send": + return f"SEND fn={self.fn:#x} id={id_text}" if self.fn is not None else "SEND" + if self.kind == "control": + return f"CTRL cmd={id_text}" + if self.kind == "alloc": + return f"ALLOC cls={id_text}" + return f"CTRL-DONE cmd={id_text}" + + +@dataclass +class RmCapture: + """A decoded capture document.""" + + path: str + sha256: str + records: list[CaptureRecord] = field(default_factory=list) + source: str = "" + format: str = CAPTURE_FORMAT + warnings: list[str] = field(default_factory=list) + + @property + def truncated_records(self) -> list[CaptureRecord]: + return [r for r in self.records if r.truncated] + + def kind_counts(self) -> dict[str, int]: + counts: dict[str, int] = {} + for record in self.records: + counts[record.kind] = counts.get(record.kind, 0) + 1 + return counts + + def fn_counts(self) -> dict[str, int]: + counts: dict[str, int] = {} + for record in self.records: + if record.kind == "send" and record.fn is not None: + key = f"0x{record.fn:x}" + counts[key] = counts.get(key, 0) + 1 + return counts + + def golden_status(self) -> int | None: + for record in self.records: + if record.kind == "golden" and record.marker == "end": + return record.status + return None + + +# --------------------------------------------------------------------------- +# Decode +# --------------------------------------------------------------------------- + + +def _kv_fields(line: str) -> dict[str, str]: + return {key: value for key, value in _KV_RE.findall(line)} + + +def _new_record(index: int, kind: str, line_no: int, ts: str) -> CaptureRecord: + return CaptureRecord(index=index, kind=kind, line=line_no, ts=ts) + + +def decode_text(text: str, source: str = "<text>") -> RmCapture: + """Decode capture-log text into a capture document. + + Lines are matched anywhere in the line (dmesg/NVRM prefixes are fine). + Hex lines attach to the most recent record of their own tag; a hex line + with no open record of that tag is recorded as an ``orphan_hex`` warning + instead of being guessed onto some other record. + """ + records: list[CaptureRecord] = [] + warnings: list[str] = [] + open_records: dict[str, CaptureRecord] = {} + + for line_no, raw_line in enumerate(text.splitlines(), start=1): + ts_match = _TS_RE.search(raw_line) + ts = ts_match.group(1) if ts_match else "" + + hex_match = _HEX_RE.search(raw_line) + if hex_match: + tag = hex_match.group(1) + kind = TAG_KIND[tag] + offset = int(hex_match.group(2), 16) + data = bytes.fromhex(re.sub(r"\s+", "", hex_match.group(3))) + record = open_records.get(kind) + if record is None: + warnings.append(f"line {line_no}: orphan {tag} hex block (no header)") + continue + if offset in record.chunks: + record.anomalies.append(f"duplicate hex offset {offset:#x}") + record.chunks[offset] = data + continue + + if _GOLDEN_TRIGGER_RE.search(raw_line): + match = _GOLDEN_TRIGGER_RE.search(raw_line) + record = _new_record(len(records), "golden", line_no, ts) + record.marker = "trigger" + record.gspclient = int(match.group(1)) if match else None + record.mig = int(match.group(2)) if match and match.group(2) else None + records.append(record) + continue + + if _GOLDEN_BEGIN_RE.search(raw_line): + record = _new_record(len(records), "golden", line_no, ts) + record.marker = "begin" + records.append(record) + continue + + if _GOLDEN_END_RE.search(raw_line): + match = _GOLDEN_END_RE.search(raw_line) + record = _new_record(len(records), "golden", line_no, ts) + record.marker = "end" + record.status = _parse_int(match.group(1) if match else None) + records.append(record) + continue + + if _CTRL_DONE_RE.search(raw_line): + fields = _kv_fields(raw_line) + record = _new_record(len(records), "ctrl-done", line_no, ts) + record.cmd = _parse_int(fields.get("cmd")) + record.rpc_status = _parse_int(fields.get("rpc_status")) + record.params_status = _parse_int(fields.get("params_status")) + records.append(record) + continue + + header_kind = None + for tag, kind in TAG_KIND.items(): + if re.search(rf"GE-{tag}\b", raw_line): + header_kind = kind + break + if header_kind is None: + continue + + fields = _kv_fields(raw_line) + record = _new_record(len(records), header_kind, line_no, ts) + if header_kind == "send": + record.fn = _parse_int(fields.get("fn")) + record.length = _parse_int(fields.get("len")) + elif header_kind == "control": + record.cmd = _parse_int(fields.get("cmd")) + record.hclient = _parse_int(fields.get("hClient")) + record.hobject = _parse_int(fields.get("hObject")) + record.size = _parse_int(fields.get("size")) + elif header_kind == "alloc": + record.hclass = _parse_int(fields.get("cls")) + record.hclient = _parse_int(fields.get("hClient")) + record.hparent = _parse_int(fields.get("hParent")) + record.hobject = _parse_int(fields.get("hObject")) + record.size = _parse_int(fields.get("size")) + caller_raw = fields.get("ret", "") + record.caller_raw = caller_raw + record.caller = _parse_int(caller_raw) + records.append(record) + open_records[header_kind] = record + + # Post-pass: chunk-gap and size anomalies. + for record in records: + expected = record.expected_params_size + if record.chunks and 0 not in record.chunks: + record.anomalies.append("hex chunks do not start at offset 0") + if record.overrun: + record.anomalies.append( + f"captured {len(record.params)} bytes > declared {expected} bytes" + ) + + return RmCapture( + path=os.path.abspath(source), + sha256=_sha256_bytes(text.encode("utf-8", "replace")), + records=records, + source=source, + warnings=warnings, + ) + + +def decode_file(path: str) -> RmCapture: + if not os.path.isfile(path): + raise RmCaptureError(f"capture log not found: {path}") + try: + with open(path, "r", encoding="utf-8", errors="replace") as fh: + text = fh.read() + except OSError as exc: + raise RmCaptureError(f"cannot read capture log {path}: {exc}") from exc + return decode_text(text, source=path) + + +def load_capture(path: str) -> RmCapture: + """Load a ``greeneye-rm-rpc-capture/1`` document or decode a raw log.""" + if not os.path.isfile(path): + raise RmCaptureError(f"capture not found: {path}") + with open(path, "r", encoding="utf-8", errors="replace") as fh: + head = fh.read(4096) + if not head.lstrip().startswith("{"): + return decode_file(path) + try: + with open(path, "r", encoding="utf-8", errors="replace") as fh: + payload = json.load(fh) + except (OSError, json.JSONDecodeError) as exc: + raise RmCaptureError(f"cannot read capture {path}: {exc}") from exc + if not isinstance(payload, dict): + raise RmCaptureError(f"{path}: capture must be an object") + fmt = str(payload.get("format") or payload.get("schema") or "") + if fmt != CAPTURE_FORMAT: + raise RmCaptureError(f"{path}: format {fmt!r} is not {CAPTURE_FORMAT!r}") + raw_records = payload.get("records") + if not isinstance(raw_records, list): + raise RmCaptureError(f"{path}: missing 'records' list") + records: list[CaptureRecord] = [] + for index, raw in enumerate(raw_records): + if not isinstance(raw, dict): + continue + record = _new_record( + index, str(raw.get("kind") or "unknown"), int(raw.get("line") or 0), + str(raw.get("ts") or ""), + ) + record.fn = _parse_int(raw.get("fn")) + record.length = _parse_int(raw.get("length")) + record.size = _parse_int(raw.get("size")) + record.cmd = _parse_int(raw.get("cmd")) + record.hclass = _parse_int(raw.get("hclass")) + record.hclient = _parse_int(raw.get("hclient")) + record.hobject = _parse_int(raw.get("hobject")) + record.hparent = _parse_int(raw.get("hparent")) + record.caller = _parse_int(raw.get("caller")) + record.caller_raw = str(raw.get("caller_raw") or "") + record.marker = str(raw.get("marker") or "") + record.gspclient = _parse_int(raw.get("gspclient")) + record.mig = _parse_int(raw.get("mig")) + record.status = _parse_int(raw.get("status")) + record.rpc_status = _parse_int(raw.get("rpc_status")) + record.params_status = _parse_int(raw.get("params_status")) + params_hex = raw.get("params_hex") + if isinstance(params_hex, str) and params_hex: + try: + record.chunks[0] = bytes.fromhex(re.sub(r"\s+", "", params_hex)) + except ValueError as exc: + raise RmCaptureError(f"{path}: record {index}: bad params hex: {exc}") from exc + records.append(record) + capture = RmCapture( + path=os.path.abspath(path), + sha256=_sha256_file(path), + records=records, + source=str(payload.get("source") or path), + ) + warnings = payload.get("warnings") + if isinstance(warnings, list): + capture.warnings = [str(w) for w in warnings] + return capture + + +# --------------------------------------------------------------------------- +# Compare +# --------------------------------------------------------------------------- + + +def _param_agreement(a: CaptureRecord, b: CaptureRecord) -> dict: + pa, pb = a.params, b.params + matched = sum(1 for x, y in zip(pa, pb) if x == y) + first_diff = None + for offset, (x, y) in enumerate(zip(pa, pb)): + if x != y: + first_diff = offset + break + if first_diff is None and len(pa) != len(pb): + first_diff = min(len(pa), len(pb)) + denominator = max(len(pa), len(pb)) + return { + "a_bytes": len(pa), + "b_bytes": len(pb), + "matched_bytes": matched, + "byte_agreement_pct": ( + round(100.0 * matched / denominator, 2) if denominator else None + ), + "first_diff_offset": first_diff, + "a_truncated": a.truncated, + "b_truncated": b.truncated, + } + + +def _pair_status(a: CaptureRecord, b: CaptureRecord) -> tuple[str, dict]: + if a.kind == "golden": + if ( + a.status == b.status + and a.gspclient == b.gspclient + and a.mig == b.mig + ): + return "agree", {} + return "mismatch", { + "a_status": a.status, + "b_status": b.status, + "a_gspclient": a.gspclient, + "b_gspclient": b.gspclient, + "a_mig": a.mig, + "b_mig": b.mig, + } + if a.kind == "ctrl-done": + if a.rpc_status == b.rpc_status and a.params_status == b.params_status: + return "agree", {} + return "mismatch", { + "a_rpc_status": a.rpc_status, + "b_rpc_status": b.rpc_status, + "a_params_status": a.params_status, + "b_params_status": b.params_status, + } + agreement = _param_agreement(a, b) + if a.truncated or b.truncated: + return "partial", agreement + if agreement["a_bytes"] == agreement["b_bytes"] and agreement["first_diff_offset"] is None: + return "agree", agreement + return "shape-agree", agreement + + +def compare(a: RmCapture, b: RmCapture) -> dict: + """Ordered, per-record comparison of two captures; no timing claims.""" + sig_a = [record.signature for record in a.records] + sig_b = [record.signature for record in b.records] + matcher = difflib.SequenceMatcher(a=sig_a, b=sig_b, autojunk=False) + rows: list[dict] = [] + a_only: list[dict] = [] + b_only: list[dict] = [] + counts = { + "agree": 0, + "shape-agree": 0, + "partial": 0, + "mismatch": 0, + "a-only": 0, + "b-only": 0, + } + for opcode, i1, i2, j1, j2 in matcher.get_opcodes(): + if opcode == "equal": + for offset in range(i2 - i1): + record_a = a.records[i1 + offset] + record_b = b.records[j1 + offset] + status, detail = _pair_status(record_a, record_b) + counts[status] += 1 + rows.append( + { + "a_index": record_a.index, + "b_index": record_b.index, + "a_label": record_a.label, + "b_label": record_b.label, + "signature": record_a.signature, + "status": status, + "detail": detail, + "a_truncated": record_a.truncated, + "b_truncated": record_b.truncated, + "a_caller": record_a.caller, + "b_caller": record_b.caller, + } + ) + else: + for index in range(i1, i2): + counts["a-only"] += 1 + record = a.records[index] + a_only.append( + { + "index": record.index, + "label": record.label, + "signature": record.signature, + "params_bytes": len(record.params), + "line": record.line, + } + ) + for index in range(j1, j2): + counts["b-only"] += 1 + record = b.records[index] + b_only.append( + { + "index": record.index, + "label": record.label, + "signature": record.signature, + "params_bytes": len(record.params), + "line": record.line, + } + ) + matched = counts["agree"] + counts["shape-agree"] + counts["partial"] + counts["mismatch"] + byte_rows = [ + row["detail"].get("byte_agreement_pct") + for row in rows + if isinstance(row.get("detail"), dict) + and row["detail"].get("byte_agreement_pct") is not None + ] + return { + "format": COMPARE_FORMAT, + "inputs": { + "a": { + "path": a.path, + "sha256": a.sha256, + "source": a.source, + "records": len(a.records), + "kinds": a.kind_counts(), + "fn_counts": a.fn_counts(), + "golden_status": a.golden_status(), + "truncated_records": len(a.truncated_records), + }, + "b": { + "path": b.path, + "sha256": b.sha256, + "source": b.source, + "records": len(b.records), + "kinds": b.kind_counts(), + "fn_counts": b.fn_counts(), + "golden_status": b.golden_status(), + "truncated_records": len(b.truncated_records), + }, + }, + "summary": { + "a_records": len(a.records), + "b_records": len(b.records), + "matched": matched, + "agree": counts["agree"], + "shape_agree": counts["shape-agree"], + "partial": counts["partial"], + "mismatch": counts["mismatch"], + "a_only": counts["a-only"], + "b_only": counts["b-only"], + "agreement_pct": round(100.0 * counts["agree"] / matched, 2) if matched else None, + "coverage_pct": round(100.0 * matched / len(a.records), 2) if a.records else None, + "mean_byte_agreement_pct": ( + round(sum(byte_rows) / len(byte_rows), 2) if byte_rows else None + ), + "claim": ( + "per-record identity and parameter agreement on the aligned intersection; " + "not a timing, silicon or full-coverage claim" + ), + }, + "rows": rows, + "a_only": a_only, + "b_only": b_only, + "fidelity": { + "a": f"file-ingested capture ({a.source or 'unlabelled'})", + "b": f"file-ingested capture ({b.source or 'unlabelled'})", + "levels": "L0/L1 only; L2 timing and L3 device environment not claimed (docs/06)", + }, + "limits": [ + "This tool never ran the capture: a supervised human produced both inputs.", + "Handles, physical addresses and caller return addresses are capture-specific and are labelled, not normalised.", + "A truncated payload is reported as partial/truncated, never padded to look complete.", + "A-only/b-only records are ordering or presence differences under this alignment; they are not proof of a wrong value.", + ], + "warnings": (a.warnings if a.warnings else []) + (b.warnings if b.warnings else []), + } + + +def render_compare_markdown(report: dict) -> str: + summary = report["summary"] + lines = [ + "# GSP RPC capture comparison", + "", + f"> Format `{report['format']}`. Fidelity: {report['fidelity']['levels']}", + "> No cycle/timing claim. Underclaim every number (docs/06, docs/45).", + "", + "## Inputs", + "", + f"* A: `{report['inputs']['a']['path']}` " + f"({report['inputs']['a']['records']} records, " + f"sha256 `{report['inputs']['a']['sha256'][:16]}…`" + + ( + f", golden status `{report['inputs']['a']['golden_status']:#x}`" + if report["inputs"]["a"]["golden_status"] is not None + else "" + ) + + ")", + f"* B: `{report['inputs']['b']['path']}` " + f"({report['inputs']['b']['records']} records, " + f"sha256 `{report['inputs']['b']['sha256'][:16]}…`" + + ( + f", golden status `{report['inputs']['b']['golden_status']:#x}`" + if report["inputs"]["b"]["golden_status"] is not None + else "" + ) + + ")", + "", + "## Summary", + "", + "| matched | agree | shape-agree | partial | mismatch | A-only | B-only | agreement % | coverage % | mean byte agreement % |", + "|---|---|---|---|---|---|---|---|---|---|", + f"| {summary['matched']} | {summary['agree']} | {summary['shape_agree']} | " + f"{summary['partial']} | {summary['mismatch']} | {summary['a_only']} | {summary['b_only']} | " + f"{summary['agreement_pct'] if summary['agreement_pct'] is not None else 'n/a'} | " + f"{summary['coverage_pct'] if summary['coverage_pct'] is not None else 'n/a'} | " + f"{summary['mean_byte_agreement_pct'] if summary['mean_byte_agreement_pct'] is not None else 'n/a'} |", + "", + f"* {summary['claim']}", + "", + "## Per-record rows", + "", + "| A | B | status | bytes A/B | byte agreement % | first diff | truncated A/B |", + "|---|---|---|---|---|---|---|", + ] + for row in report["rows"]: + detail = row.get("detail") if isinstance(row.get("detail"), dict) else {} + a_bytes = detail.get("a_bytes", "") + b_bytes = detail.get("b_bytes", "") + byte_pct = detail.get("byte_agreement_pct") + first_diff = detail.get("first_diff_offset") + lines.append( + f"| `{row['a_label']}` | `{row['b_label']}` | {row['status']} | " + f"{a_bytes}/{b_bytes} | {byte_pct if byte_pct is not None else ''} | " + f"{first_diff if first_diff is not None else ''} | " + f"{'Y' if row.get('a_truncated') else '-'}/{'Y' if row.get('b_truncated') else '-'} |" + ) + lines += ["", "## A-only records", ""] + if report["a_only"]: + for item in report["a_only"]: + lines.append( + f"* #{item['index']} `{item['label']}` ({item['params_bytes']} B, line {item['line']})" + ) + else: + lines.append("* none") + lines += ["", "## B-only records", ""] + if report["b_only"]: + for item in report["b_only"]: + lines.append( + f"* #{item['index']} `{item['label']}` ({item['params_bytes']} B, line {item['line']})" + ) + else: + lines.append("* none") + lines += ["", "## Limits", ""] + for limit in report["limits"]: + lines.append(f"* {limit}") + if report.get("warnings"): + lines += ["", "## Decode warnings", ""] + for warning in report["warnings"]: + lines.append(f"* {warning}") + lines.append("") + return "\n".join(lines) + + +def run_decode(log_path: str, out_dir: str | None) -> dict: + capture = decode_file(log_path) + payload = capture_to_dict(capture) + if out_dir: + os.makedirs(out_dir, exist_ok=True) + json_path = os.path.join(out_dir, "rm-capture.json") + with open(json_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(payload)) + md_path = os.path.join(out_dir, "rm-capture.md") + with open(md_path, "w", encoding="utf-8") as fh: + fh.write(render_capture_markdown(capture)) + payload["artifacts"] = { + "capture_json": os.path.abspath(json_path), + "capture_md": os.path.abspath(md_path), + } + return payload + + +def run_compare(a_path: str, b_path: str, out_dir: str) -> dict: + capture_a = load_capture(a_path) + capture_b = load_capture(b_path) + report = compare(capture_a, capture_b) + os.makedirs(out_dir, exist_ok=True) + json_path = os.path.join(out_dir, "rm-capture-compare.json") + md_path = os.path.join(out_dir, "rm-capture-compare.md") + with open(json_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(report)) + with open(md_path, "w", encoding="utf-8") as fh: + fh.write(render_compare_markdown(report)) + report["artifacts"] = { + "report_json": os.path.abspath(json_path), + "report_md": os.path.abspath(md_path), + } + return report + + +def run_stimulus(capture_path: str, out_path: str | None) -> dict: + capture = load_capture(capture_path) + payload = stimulus_from_capture(capture) + if out_path: + os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True) + with open(out_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(payload)) + payload["artifacts"] = {"stimulus_json": os.path.abspath(out_path)} + return payload + + +# --------------------------------------------------------------------------- +# Serialisation / stimulus +# --------------------------------------------------------------------------- + + +def record_to_dict(record: CaptureRecord) -> dict: + out: dict[str, Any] = { + "index": record.index, + "kind": record.kind, + "line": record.line, + } + if record.ts: + out["ts"] = record.ts + for key in ( + "fn", + "length", + "size", + "cmd", + "hclass", + "hclient", + "hobject", + "hparent", + "caller", + ): + value = getattr(record, key) + if value is not None: + out[key] = value + if record.caller_raw: + out["caller_raw"] = record.caller_raw + if record.marker: + out["marker"] = record.marker + if record.gspclient is not None: + out["gspclient"] = record.gspclient + if record.mig is not None: + out["mig"] = record.mig + for key in ("status", "rpc_status", "params_status"): + value = getattr(record, key) + if value is not None: + out[key] = value + params = record.params + if params: + out["params_hex"] = params.hex() + out["params_bytes"] = len(params) + out["params_size_expected"] = record.expected_params_size + out["truncated"] = record.truncated + if record.anomalies: + out["anomalies"] = list(record.anomalies) + return out + + +def capture_to_dict(capture: RmCapture) -> dict: + return { + "format": CAPTURE_FORMAT, + "source": capture.source, + "sha256": capture.sha256, + "records": [record_to_dict(record) for record in capture.records], + "summary": { + "records": len(capture.records), + "kinds": capture.kind_counts(), + "fn_counts": capture.fn_counts(), + "truncated_records": len(capture.truncated_records), + "golden_status": capture.golden_status(), + }, + "warnings": list(capture.warnings), + "limits": [ + "Decoded from the documented capture tags only (GE-SEND/GE-CTRL/GE-ALLOC/GE-GOLDEN-*/GE-CTRL-DONE).", + "Handles and addresses are capture-specific; do not reuse them across boots.", + "Truncated payloads are flagged (the runbook hex helper caps at 192 bytes unless extended).", + "This tool never ran the capture; the log is a supervised human artifact.", + ], + } + + +def render_capture_markdown(capture: RmCapture) -> str: + summary = capture_to_dict(capture)["summary"] + lines = [ + "# GSP RPC capture", + "", + f"> Format `{CAPTURE_FORMAT}`; source `{capture.source}`; " + f"sha256 `{capture.sha256[:16]}…`. Fidelity L0/L1; no timing claim.", + "", + "## Summary", + "", + f"* records: {summary['records']} {summary['kinds']}", + f"* wire fn counts: {summary['fn_counts']}", + f"* truncated records: {summary['truncated_records']}", + f"* golden end status: {summary['golden_status']}", + "", + "## Records", + "", + "| # | kind | ts | id | size | captured B | expected B | truncated | caller |", + "|---|---|---|---|---|---|---|---|---|", + ] + for record in capture.records: + identifier = record.wire_id + id_text = f"{identifier:#x}" if identifier is not None else "" + expected = record.expected_params_size + lines.append( + f"| {record.index} | {record.kind} | {record.ts} | {id_text} | " + f"{record.size if record.size is not None else ''} | {len(record.params)} | " + f"{expected if expected is not None else ''} | " + f"{'Y' if record.truncated else '-'} | " + f"{f'{record.caller:#x}' if record.caller is not None else record.caller_raw} |" + ) + if capture.warnings: + lines += ["", "## Decode warnings", ""] + for warning in capture.warnings: + lines.append(f"* {warning}") + lines += ["", "## Limits", ""] + for limit in capture_to_dict(capture)["limits"]: + lines.append(f"* {limit}") + lines.append("") + return "\n".join(lines) + + +def stimulus_from_capture(capture: RmCapture) -> dict: + messages: list[dict] = [] + markers: list[dict] = [] + for record in capture.records: + if record.kind == "golden": + markers.append( + { + "line": record.line, + "marker": record.marker, + "status": record.status, + "gspclient": record.gspclient, + } + ) + continue + if record.kind == "ctrl-done": + continue + message: dict[str, Any] = { + "seq": len(messages), + "kind": record.kind, + "params_hex": record.params.hex(), + "params_size": len(record.params), + "params_size_expected": record.expected_params_size, + "truncated": record.truncated, + } + if record.fn is not None: + message["fn"] = record.fn + if record.cmd is not None: + message["cmd"] = record.cmd + if record.hclass is not None: + message["hclass"] = record.hclass + if record.kind == "send" and record.wire_id is not None: + message["wire_id"] = record.wire_id + messages.append(message) + return { + "format": STIMULUS_FORMAT, + "source": { + "path": capture.path, + "sha256": capture.sha256, + "records": len(capture.records), + }, + "markers": markers, + "messages": messages, + "limits": [ + "Handles, physical addresses and caller addresses are capture-specific and deliberately omitted; a replay must re-map or allocate them.", + "Truncated messages (params_size < params_size_expected) are not replayable as-is; extend the capture dump first.", + "Consumption by the simulator B4/B5 CSB/message-queue seam is not wired yet (docs/45).", + "No ordering claim beyond the captured log order; server replies were not captured by the send-side probes.", + ], + } + + +# --------------------------------------------------------------------------- +# Demo (synthetic; measured value contract) +# --------------------------------------------------------------------------- + + +def _hex_lines(tag: str, payload: bytes, chunk: int = 192) -> list[str]: + lines: list[str] = [] + for start in range(0, len(payload), chunk): + piece = payload[start : start + chunk] + lines.append(f"GE-{tag} {start:03x}:" + "".join(f" {b:02x}" for b in piece)) + return lines + + +def _synthetic_log() -> str: + allocs = [ + ("0x90f1", 0x30), + ("0x003e", 0x100), + ("0x50a0", 0x40), + ("0x0040", 0x1000), + ("0xc46f", 0x180), + ("0xc597", 0x0), + ] + controls = [ + ("0x2080012b", 560), + ("0x20801201", 64), + ("0x20800a2a", 32), + ("0x20809001", 16), + ] + lines = [ + "[ 100.000000] NVRM: GE-GOLDEN-TRIGGER gspclient=1 mig=0", + "[ 100.000010] NVRM: GE-GOLDEN-BEGIN", + ] + step = 1 + for hclass, size in allocs: + hclass_value = int(hclass, 16) + params = bytes((i * 7 + hclass_value) & 0xFF for i in range(size)) + lines.append( + f"[ 100.{step:06d}] NVRM: GE-ALLOC cls={hclass} hClient=0xbaba0001 " + f"hParent=0x0 hObject=0xbaba10{step:02x} size={size:#x} " + f"paramp=0xffffffffc0a00000 ret=0xffffffffc0b00000" + ) + lines.extend(f"[ 100.{step:06d}] NVRM: {line}" for line in _hex_lines("ALLOC", params)) + step += 1 + for cmd, size in controls: + cmd_value = int(cmd, 16) + params = bytes((i * 3 + cmd_value) & 0xFF for i in range(size)) + lines.append( + f"[ 100.{step:06d}] NVRM: GE-CTRL cmd={cmd} hClient=0xc2000006 " + f"hObject=0xabcd0080 size={size:#x} paramp=0xffffffffc0a10000 " + f"ret=0xffffffffc0b00000" + ) + lines.extend(f"[ 100.{step:06d}] NVRM: {line}" for line in _hex_lines("CTRL", params)) + step += 1 + payload = ( + (0xC2000006).to_bytes(4, "little") + + (0xABCD0080).to_bytes(4, "little") + + (0x2080012B).to_bytes(4, "little") + + bytes(range(60)) + ) + lines.append( + f"[ 100.{step:06d}] NVRM: GE-SEND fn=0x0000004c len={len(payload) + RPC_HEADER_BYTES:#x} " + f"ret=0xffffffffc0b00000" + ) + lines.extend(f"[ 100.{step:06d}] NVRM: {line}" for line in _hex_lines("SEND", payload)) + step += 1 + lines.append( + f"[ 100.{step:06d}] NVRM: GE-CTRL-DONE cmd=0x2080012b rpc_status=0x0 params_status=0x0" + ) + lines.append("[ 100.000999] NVRM: GE-GOLDEN-END status=0x0") + return "\n".join(lines) + "\n" + + +def _mutate_log(text: str) -> str: + """Replay-side variant: one parameter byte differs, one alloc is dropped.""" + kept: list[str] = [] + dropping = False + dropped = False + for line in text.splitlines(): + if not dropped and "GE-ALLOC cls=0x003e" in line: + dropped = True + dropping = True + continue + if dropping: + if re.search(r"GE-ALLOC\s+[0-9a-fA-F]{1,4}:", line): + continue + dropping = False + kept.append(line) + text_b = "\n".join(kept) + "\n" + + def flip(match: re.Match) -> str: + value = int(match.group(2), 16) ^ 0x01 + return f"{match.group(1)}{value:02x}" + + text_b, _ = re.subn(r"(GE-CTRL 000: )([0-9a-fA-F]{2})", flip, text_b, count=1) + return text_b + + +def run_demo(out_dir: str | None = None) -> dict: + started = time.perf_counter() + capture_a = decode_text(_synthetic_log(), source="demo-capture-a") + capture_b = decode_text(_mutate_log(_synthetic_log()), source="demo-replay-b") + report = compare(capture_a, capture_b) + stimulus = stimulus_from_capture(capture_a) + wall = time.perf_counter() - started + threshold_seconds = MANUAL_DECODE_BASELINE_SECONDS * WORKFLOW_THRESHOLD_FRACTION + ok = ( + wall <= threshold_seconds + and report["summary"]["agree"] + report["summary"]["shape_agree"] >= 1 + and report["summary"]["a_only"] + report["summary"]["b_only"] >= 1 + and not capture_a.truncated_records + ) + payload = { + "format": DEMO_FORMAT, + "ok": ok, + "wall_seconds": round(wall, 4), + "baseline_seconds": MANUAL_DECODE_BASELINE_SECONDS, + "baseline_label": MANUAL_BASELINE_LABEL, + "threshold_fraction": WORKFLOW_THRESHOLD_FRACTION, + "threshold_seconds": threshold_seconds, + "records_a": len(capture_a.records), + "records_b": len(capture_b.records), + "messages": len(stimulus["messages"]), + "summary": report["summary"], + } + if out_dir: + os.makedirs(out_dir, exist_ok=True) + capture_path = os.path.join(out_dir, "demo-capture-a.json") + replay_path = os.path.join(out_dir, "demo-replay-b.json") + with open(capture_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(capture_to_dict(capture_a))) + with open(replay_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(capture_to_dict(capture_b))) + compare_path = os.path.join(out_dir, "demo-compare.json") + with open(compare_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(report)) + stimulus_path = os.path.join(out_dir, "demo-stimulus.json") + with open(stimulus_path, "w", encoding="utf-8") as fh: + fh.write(canonical_json(stimulus)) + payload["artifacts"] = { + "capture_a": os.path.abspath(capture_path), + "replay_b": os.path.abspath(replay_path), + "compare": os.path.abspath(compare_path), + "stimulus": os.path.abspath(stimulus_path), + } + return payload + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +CAPTURE_HINTS = """\ +================================================================================ +GSP RPC CAPTURE TAG GRAMMAR — SUPERVISED, HUMAN-RUN PHYSICAL ACTION +================================================================================ +This command prints the tag contract the decoder understands; it executes +NOTHING and never runs the capture. + +The capture itself is supervised-only (RUNBOOK P1/P7, SIMULATOR-WORK- +BOUNDARY.txt): driver swap on the physical GPU, one attempt per boot, stop on +Zid/WPR2/pstore; the authored procedure lives in the capture runbook +(authored, not executed). This tool only ingests the resulting log. + +Tags (anywhere in the line; dmesg/NVRM prefixes are fine): + GE-GOLDEN-TRIGGER gspclient=<0|1> mig=<0|1> + GE-GOLDEN-BEGIN + GE-GOLDEN-END status=0x... + GE-SEND fn=0x... len=0x... ret=0x... + GE-CTRL cmd=0x... hClient=0x... hObject=0x... size=0x... paramp=0x... ret=0x... + GE-ALLOC cls=0x... hClient=0x... hParent=0x... hObject=0x... size=0x... paramp=0x... ret=0x... + GE-CTRL-DONE cmd=0x... rpc_status=0x... params_status=0x... + Hex lines attach to the most recent record of their own tag: + GE-<TAG> <offset-hex>: <hex bytes...> + +NOTE — the runbook's hex helper currently caps a dump at 192 bytes; payloads +larger than that (e.g. a 560-byte PROMOTE_CTX) decode as `truncated` and are +not replayable until the helper is extended to chunk. The decoder flags this +instead of padding. + +Offline use (safe): + python3 harness/rm_capture.py decode --log capture.log --out dumps/rm-capture + python3 harness/rm_capture.py compare --a real.json --b replay.json --out dumps/rm-compare + python3 harness/rm_capture.py stimulus --capture real.json --out dumps/rm-stimulus.json + python3 harness/rm_capture.py demo --out /tmp/rm-capture-demo +================================================================================ +""" + + +def print_capture_hints(out=None) -> None: + """Print the supervised capture tag grammar; execute nothing.""" + print(CAPTURE_HINTS, file=out or sys.stdout) + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="rm_capture.py", + description=( + "Decode, compare and replay-prep a supervised GSP RPC capture. " + "Offline only; never executes physical commands." + ), + ) + parser.add_argument( + "--print-capture-hints", + action="store_true", + help="print the capture tag grammar (executes nothing)", + ) + sub = parser.add_subparsers(dest="command") + decode_cmd = sub.add_parser("decode", help="decode a raw capture log") + decode_cmd.add_argument("--log", required=True, help="capture log (text)") + decode_cmd.add_argument("--out", help="output directory (optional)") + compare_cmd = sub.add_parser("compare", help="compare two captures") + compare_cmd.add_argument("--a", required=True, help="capture JSON or raw log") + compare_cmd.add_argument("--b", required=True, help="capture JSON or raw log") + compare_cmd.add_argument("--out", required=True, help="output directory") + stimulus_cmd = sub.add_parser("stimulus", help="emit a replay stimulus document") + stimulus_cmd.add_argument("--capture", required=True, help="capture JSON or raw log") + stimulus_cmd.add_argument("--out", help="output JSON path (optional)") + demo_cmd = sub.add_parser("demo", help="synthetic measured end-to-end demo") + demo_cmd.add_argument("--out", help="output directory (optional)") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + if args.print_capture_hints: + print_capture_hints() + return 0 + try: + if args.command == "decode": + payload = run_decode(args.log, args.out) + elif args.command == "compare": + payload = run_compare(args.a, args.b, args.out) + elif args.command == "stimulus": + payload = run_stimulus(args.capture, args.out) + elif args.command == "demo": + payload = run_demo(args.out) + else: + parser.print_help() + return 2 + except RmCaptureError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + summary = payload.get("summary", {}) + print( + json.dumps( + { + "format": payload.get("format"), + "ok": payload.get("ok"), + "wall_seconds": payload.get("wall_seconds"), + "summary": summary, + "artifacts": payload.get("artifacts"), + }, + indent=2, + sort_keys=True, + ) + ) + if args.command == "demo" and not payload.get("ok"): + return 3 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/run_contracts.py b/debug/risc-v-gsp-rm-vm/harness/run_contracts.py new file mode 100644 index 0000000..1ea8474 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/run_contracts.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +"""Run the wall-contract registry (``harness/contracts.py``, docs/50). + +Every solved wall is pinned as a contract over an existing tool entry point; +this CLI runs them and prints pass/fail per contract plus per-wall-class +coverage. Fast contracts are the suite default; slow ones run with ``--all`` +(or by name). + +Offline only: never touches a GPU, kernel module, mdev, guest or network. +Exit codes: 0 = every selected contract passed (or skipped for a missing +prerequisite), 1 = at least one failure, 2 = usage error. +""" +from __future__ import annotations + +import argparse +import json +import os +import sys + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.contracts import ( # noqa: E402 + Contract, + all_contracts, + fast_contracts, + run_contracts, + summarize, +) + + +def _select(args: argparse.Namespace) -> tuple[list[Contract], list[str]]: + if args.name: + wanted = set(args.name) + chosen = [c for c in all_contracts() if c.name in wanted] + unknown = sorted(wanted - {c.name for c in chosen}) + return chosen, unknown + return (all_contracts() if args.all else fast_contracts()), [] + + +def _table(rows: list[tuple[str, ...]], headers: tuple[str, ...]) -> list[str]: + widths = [ + max(len(headers[index]), *(len(row[index]) for row in rows)) + for index in range(len(headers)) + ] + lines = [" ".join(head.ljust(widths[i]) for i, head in enumerate(headers))] + lines.append(" ".join("-" * width for width in widths)) + for row in rows: + lines.append(" ".join(cell.ljust(widths[i]) for i, cell in enumerate(row))) + return lines + + +def _render(results, summary: dict) -> list[str]: + rows = [ + ( + result.name, + result.wall_class, + result.speed, + result.status.upper(), + f"{result.seconds:.2f}s", + result.detail, + ) + for result in results + ] + lines = _table(rows, ("contract", "class", "speed", "status", "time", "detail")) + lines.append("") + lines.append("wall-class coverage (selected):") + for domain in summary["domains"]: + lines.append( + f" {domain['wall_class']} {domain['domain']:<28} " + f"{domain['total']} contracts: {domain['pass']} pass, " + f"{domain['fail']} fail, {domain['skip']} skip" + ) + counts = summary["counts"] + lines.append("") + lines.append( + f"total: {counts['pass']} pass, {counts['fail']} fail, {counts['skip']} skip " + f"-> {'OK' if summary['ok'] else 'FAIL'}" + ) + return lines + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="run_contracts.py", + description=( + "Run the wall-contract registry: fast contracts by default, all " + "with --all (offline; never touches hardware)." + ), + ) + group = parser.add_mutually_exclusive_group() + group.add_argument( + "--fast", action="store_true", help="run fast contracts only (default)" + ) + group.add_argument("--all", action="store_true", help="run fast and slow contracts") + parser.add_argument( + "--name", + action="append", + default=[], + metavar="NAME", + help="run one named contract (repeatable); overrides --fast/--all selection", + ) + parser.add_argument("--json", metavar="PATH", help="write the run summary as JSON here") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + selected, unknown = _select(args) + if unknown: + parser.error("unknown contract name(s): " + ", ".join(unknown)) + results = run_contracts(selected) + summary = summarize(results) + for line in _render(results, summary): + print(line) + if args.json: + out_path = os.path.abspath(args.json) + parent = os.path.dirname(out_path) + if parent: + os.makedirs(parent, exist_ok=True) + with open(out_path, "w", encoding="utf-8") as fh: + json.dump(summary, fh, indent=2, sort_keys=True) + fh.write("\n") + print(f"json: {out_path}") + return 0 if summary["ok"] else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/server_snapshot.py b/debug/risc-v-gsp-rm-vm/harness/server_snapshot.py new file mode 100644 index 0000000..1099fb0 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/server_snapshot.py @@ -0,0 +1,698 @@ +#!/usr/bin/env python3 +"""Collect a canonical server-state snapshot from cycle evidence directories. + +The differential loop (live run vs emulated run, run N vs run N+1) needs one +machine-readable view of the server state a cycle produced. Cycles already +write that state as text: labelled field variables (``LABEL[offset]=0xVALUE``), +labelled hexdump blocks (an ``INSTANCE[0000]: w w w w``-style row, or a header +line naming the label followed by ``OFFSET: w w w w`` rows -- e.g. +``LOGINIT-raw`` / ``LOGRM-raw`` raw task-log dumps), and ``vgsp-nocat`` record +lines carrying ``recType=`` / ``errorCode=``. This harness discovers those +files under evidence/session directories, normalizes every dump to +little-endian u32 words, and emits one JSON snapshot that two runs can be +compared with ``diff``. + +Schemas +------- + +``greeneye-server-snapshot/1``:: + + {"schema": "greeneye-server-snapshot/1", + "source": "<string>", + "created": "<ISO-8601>", + "chunks": [{"label": "INSTANCE", "base": "0x0", + "words": ["0x00000000", ...]}], + "records": [{"kind": "record", "raw": "...", + "rec_type": 5, "error_code": 0x57f1344, + "source_name": "ASSERT"}]} + +``greeneye-server-snapshot-diff/1``:: + + {"schema": "greeneye-server-snapshot-diff/1", + "a": {"source": ..., "created": ...}, "b": {...}, + "chunks": [{"label": "INSTANCE", "status": "changed|added|removed|unchanged", + "base": {"a": "0x0", "b": "0x0"}, + "words": {"a": 1024, "b": 1024}, + "added": [{"index": 8, "value": "0x00000000"}], + "removed": [{"index": 8, "value": "0x00000001"}], + "changed": [{"index": 8, "a": "0x...", "b": "0x..."}]}], + "records": {"added": [...], "removed": [...]}, + "summary": {"identical": false, "chunks_added": 0, ...}} + +Normalization rules (the consumer depends on them): + +* words are 8-hex-digit lowercase strings of little-endian u32 values; +* a ``LABEL[offset]=0xVALUE`` value wider than 32 bits becomes two words + (low word first); its offset orders the entries and selects ``base``, and + the values are appended in offset order (the offset is a table index there, + not a byte address); +* hexdump rows are placed by byte offset (``(offset - base) // 4``) when + 4-byte aligned, so a row missing in one run still aligns the rest; +* ``base`` is the smallest offset seen for the chunk (``0x0`` when none); +* chunks are sorted by label; a repeated label is merged only when its base + and word list are byte-identical, otherwise it is kept as ``LABEL#2``, + ``LABEL#3`` ... and the collision is reported in the run stats; +* non-empty records are deduplicated by raw line and sorted by raw line; +* ``created`` is the only variable field: pin it with ``--created`` or + ``SOURCE_DATE_EPOCH`` for deterministic output. + +Honesty: this is an evidence normalizer, not a state oracle. It preserves +what the cycle wrote -- offsets are honoured, identical duplicates are merged, +differing duplicates are renamed, and dump-looking lines that fail to parse +are counted (``unparsed``) instead of being silently dropped. It never +executes anything, touches no GPU and reads only files given on the command +line. +""" +from __future__ import annotations + +import argparse +import fnmatch +import json +import os +import re +import sys +import tempfile +import time +from typing import Any, Iterable + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +SNAPSHOT_FORMAT = "greeneye-server-snapshot/1" +DIFF_FORMAT = "greeneye-server-snapshot-diff/1" +DEMO_FORMAT = "greeneye-server-snapshot-demo/1" + +#: File-name globs discovered under each ``--dir`` unless extended with +#: ``--include``. fnmatch's ``*`` matches leading dots, so the rolling host +#: dmesg captures (``.dmesg-*.rolling``) are covered by ``*.rolling``. +DEFAULT_INCLUDES = ("*.txt", "*.log", "*.rolling") + +#: ``INSTANCE[0000]: w w w w`` -- one labelled dump row inside an arbitrary +#: prefix (dmesg timestamp, subsystem, case id). Offsets are hexadecimal. +_INLINE_ROW_RE = re.compile( + r"(?P<label>[A-Za-z_][A-Za-z0-9_.]*)\[(?P<offset>(?:0x)?[0-9A-Fa-f]+)\]:" + r"[ \t]*(?P<tail>[0-9A-Fa-f]{1,8}(?:[ \t]+[0-9A-Fa-f]{1,8})*)[ \t]*$" +) + +#: ``LABEL[offset]=0xVALUE`` (optionally ``LABEL[0].suffix=``), several per +#: line allowed (``root[0]=... pd2[0]=... pd0[0].small=...``). +_LABELED_VALUE_RE = re.compile( + r"(?P<label>[A-Za-z_][A-Za-z0-9_.]*)\[(?P<offset>(?:0x)?[0-9A-Fa-f]+)\]" + r"(?P<suffix>\.[A-Za-z_][A-Za-z0-9_]*)?[ \t]*=[ \t]*" + r"(?P<value>0[xX][0-9A-Fa-f]+)\b" +) + +#: A bare hexdump row (``00000010: w w w w``) inside a labelled block. +_BLOCK_ROW_RE = re.compile( + r"^(?P<offset>[0-9A-Fa-f]{4,16}):[ \t]+" + r"(?P<tail>[0-9A-Fa-f]{1,8}(?:[ \t]+[0-9A-Fa-f]{1,8})*)[ \t]*$" +) + +#: A block header naming the label for the rows that follow: ``LOGINIT-raw:``, +#: ``LOGRM-raw:``, ``RAMFC:`` ... Used only as one-line lookahead. +_BLOCK_HEADER_RE = re.compile(r"^(?P<label>[A-Za-z_][A-Za-z0-9_.-]*)[ \t]*:") + +#: A GSP-RM error record line (``rec seq=1 recType=5 errorCode=0x...``). +_RECORD_RE = re.compile( + r"\brecType=(?P<rec_type>\d+)\b.*?\berrorCode=(?P<error_code>0[xX][0-9A-Fa-f]+)" +) + +_SOURCE_RE = re.compile(r'\bsource="(?P<source_name>[^"]*)"') + +#: Loose shape check for a line that looks like a dump but failed to parse; +#: these are counted so silent data loss is visible in the run stats. +_UNPARSED_HINT_RE = re.compile( + r"\[[0-9A-Fa-fx]+\][ \t]*[=:]|^[0-9A-Fa-f]{4,}:|\brecType=" +) + + +class SnapshotError(ValueError): + """An input file or snapshot the tool cannot honour.""" + + +def _new_stats() -> dict[str, Any]: + return { + "files": 0, + "lines": 0, + "chunks": 0, + "words": 0, + "records": 0, + "unparsed": 0, + "merged": 0, + "collisions": 0, + "conflicts": 0, + "misaligned": 0, + "collision_list": [], + } + + +class _Builder: + """Words captured for one label occurrence before normalization. + + Entries are ``(offset, [u32 word, ...])`` in encounter order. Hexdump + builders are positional: entries are placed at ``(offset - base) // 4`` + when the byte offset is 4-byte aligned; otherwise they are appended after + the highest placed word. Value-line builders (``LABEL[offset]=0xVALUE``) + are sequential: the offset only orders the entries and selects ``base`` + (its unit is a table entry index there, not a byte), so each entry's words + are appended in offset order. A later value that would overwrite a + different value at the same slot is dropped and counted as a + ``conflict``. + """ + + __slots__ = ("label", "positional", "entries", "conflicts", "misaligned") + + def __init__(self, label: str, positional: bool = True): + self.label = label + self.positional = positional + self.entries: list[tuple[int, list[str]]] = [] + self.conflicts = 0 + self.misaligned = 0 + + def add(self, offset: int, words: list[str]) -> None: + if not words: + return + for existing_offset, existing_words in self.entries: + if existing_offset == offset and existing_words == words: + return + self.entries.append((offset, words)) + + def build(self) -> tuple[int, list[str]]: + base = min(offset for offset, _ in self.entries) + placed: dict[int, str] = {} + append_at = 0 + conflicts = 0 + misaligned = 0 + for offset, words in sorted(self.entries, key=lambda entry: entry[0]): + if not self.positional: + index = append_at + elif (offset - base) % 4 == 0: + index = (offset - base) // 4 + else: + index = append_at + misaligned += 1 + for step, word in enumerate(words): + slot = index + step + if slot in placed and placed[slot] != word: + conflicts += 1 + elif slot not in placed: + placed[slot] = word + append_at = max(append_at, index + len(words)) + self.conflicts = conflicts + self.misaligned = misaligned + return base, [placed[index] for index in sorted(placed)] + + +def _word(value: int) -> str: + return f"0x{value & 0xFFFFFFFF:08x}" + + +def _value_words(value: int) -> list[str]: + """Split a value into little-endian u32 words (low word first).""" + words = [] + while True: + words.append(_word(value)) + value >>= 32 + if value == 0: + return words + + +def _split_words(tail: str) -> list[str]: + return [f"0x{int(token, 16):08x}" for token in tail.split()] + + +def _row_offset(text: str) -> int: + return int(text, 16) + + +def _labeled_offset(text: str) -> int: + if text.lower().startswith("0x"): + return int(text, 16) + try: + return int(text, 10) + except ValueError: + return int(text, 16) + + +def _match_record(line: str) -> dict[str, Any] | None: + match = _RECORD_RE.search(line) + if not match: + return None + record: dict[str, Any] = { + "kind": "record", + "raw": line, + "rec_type": int(match.group("rec_type")), + "error_code": int(match.group("error_code"), 16), + } + source = _SOURCE_RE.search(line) + if source: + record["source_name"] = source.group("source_name") + return record + + +def _parse_text(text: str, stats: dict[str, Any]) -> tuple[list[_Builder], list[dict]]: + occurrences: list[_Builder] = [] + records: list[dict] = [] + by_value_label: dict[str, _Builder] = {} + current_block: _Builder | None = None + inline_block: _Builder | None = None + pending_header: str | None = None + + for line in text.splitlines(): + stripped = line.strip() + if not stripped: + current_block = pending_header = inline_block = None + continue + + record = _match_record(stripped) + if record is not None: + records.append(record) + current_block = pending_header = inline_block = None + continue + + match = _INLINE_ROW_RE.search(stripped) + if match: + label = match.group("label") + words = _split_words(match.group("tail")) + if inline_block is not None and inline_block.label == label: + inline_block.add(_row_offset(match.group("offset")), words) + else: + inline_block = _Builder(label) + inline_block.add(_row_offset(match.group("offset")), words) + occurrences.append(inline_block) + current_block = pending_header = None + continue + + match = _BLOCK_ROW_RE.match(stripped) + if match: + if current_block is None: + if pending_header is None: + stats["unparsed"] += 1 + continue + current_block = _Builder(pending_header) + occurrences.append(current_block) + current_block.add(int(match.group("offset"), 16), + _split_words(match.group("tail"))) + pending_header = inline_block = None + continue + + value_matches = list(_LABELED_VALUE_RE.finditer(stripped)) + if value_matches: + for value_match in value_matches: + label = value_match.group("label") + if value_match.group("suffix"): + label += value_match.group("suffix") + builder = by_value_label.get(label) + if builder is None: + builder = _Builder(label, positional=False) + by_value_label[label] = builder + occurrences.append(builder) + builder.add(_labeled_offset(value_match.group("offset")), + _value_words(int(value_match.group("value"), 16))) + current_block = pending_header = inline_block = None + continue + + if _UNPARSED_HINT_RE.search(stripped): + stats["unparsed"] += 1 + + current_block = inline_block = None + header = _BLOCK_HEADER_RE.match(stripped) + pending_header = header.group("label") if header else None + + return occurrences, records + + +def _chunk_sort_key(label: str) -> tuple[str, int, str]: + stem, separator, tail = label.rpartition("#") + if separator and tail.isdigit(): + return stem, int(tail), label + return label, 0, label + + +def _finalize(occurrences: Iterable[_Builder], + stats: dict[str, Any]) -> list[dict[str, Any]]: + chunks: dict[str, dict[str, Any]] = {} + for builder in occurrences: + base, words = builder.build() + stats["conflicts"] += builder.conflicts + stats["misaligned"] += builder.misaligned + if not words: + continue + key = builder.label + suffix = 1 + merged = False + while key in chunks: + existing = chunks[key] + if existing["base"] == f"{base:#x}" and existing["words"] == words: + stats["merged"] += 1 + merged = True + break + suffix += 1 + key = f"{builder.label}#{suffix}" + if merged: + continue + chunks[key] = {"label": key, "base": f"{base:#x}", "words": words} + if suffix > 1: + stats["collisions"] += 1 + stats["collision_list"].append( + {"label": builder.label, "renamed": key, + "base": f"{base:#x}", "words": len(words)}) + return [chunks[key] for key in sorted(chunks, key=_chunk_sort_key)] + + +def discover(dirs: Iterable[str] | None, files: Iterable[str] | None, + includes: Iterable[str] | None = None) -> list[str]: + """Return the sorted absolute paths matched by the dirs/files/includes.""" + patterns = tuple(DEFAULT_INCLUDES) + tuple(includes or ()) + found: set[str] = set() + for directory in dirs or (): + if not os.path.isdir(directory): + raise SnapshotError(f"not a directory: {directory}") + for root, _subdirs, names in os.walk(directory): + for name in names: + if any(fnmatch.fnmatch(name, pattern) for pattern in patterns): + found.add(os.path.abspath(os.path.join(root, name))) + for path in files or (): + if not os.path.isfile(path): + raise SnapshotError(f"not a file: {path}") + found.add(os.path.abspath(path)) + return sorted(found) + + +def _default_source(dirs: Iterable[str] | None, + files: Iterable[str] | None) -> str: + roots = sorted(os.path.abspath(path) + for path in list(dirs or ()) + list(files or ())) + return ",".join(roots) if roots else "empty" + + +def _created_iso() -> str: + """Build timestamp; stable when ``SOURCE_DATE_EPOCH`` is set.""" + raw = os.environ.get("SOURCE_DATE_EPOCH") + try: + epoch = int(raw) if raw else int(time.time()) + except ValueError: + epoch = 0 + return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(epoch)) + + +def collect(dirs: Iterable[str] | None = None, + files: Iterable[str] | None = None, + includes: Iterable[str] | None = None, + source: str | None = None, + created: str | None = None) -> tuple[dict[str, Any], dict[str, Any]]: + """Collect every dump in ``dirs``/``files`` into one snapshot. + + Returns ``(snapshot, stats)``; ``stats`` carries the parse accounting + (files, lines, chunks, words, records, unparsed candidates, merged + duplicates, renamed collisions, conflicts, misaligned offsets) and is + printed by the CLI but is deliberately not part of the consumed schema. + """ + paths = discover(dirs, files, includes) + if not paths: + raise SnapshotError( + "no input files matched the requested --dir/--file/--include set") + stats = _new_stats() + occurrences: list[_Builder] = [] + records: dict[str, dict] = {} + for path in paths: + try: + with open(path, "r", encoding="utf-8", errors="replace") as handle: + text = handle.read() + except OSError as exc: + raise SnapshotError(f"cannot read {path}: {exc}") from exc + stats["files"] += 1 + stats["lines"] += len(text.splitlines()) + file_occurrences, file_records = _parse_text(text, stats) + occurrences.extend(file_occurrences) + for record in file_records: + records.setdefault(record["raw"], record) + chunks = _finalize(occurrences, stats) + stats["chunks"] = len(chunks) + stats["words"] = sum(len(chunk["words"]) for chunk in chunks) + stats["records"] = len(records) + snapshot = { + "schema": SNAPSHOT_FORMAT, + "source": source if source is not None else _default_source(dirs, files), + "created": created if created is not None else _created_iso(), + "chunks": chunks, + "records": sorted(records.values(), key=lambda record: record["raw"]), + } + return snapshot, stats + + +def diff_snapshots(a: dict[str, Any], b: dict[str, Any]) -> dict[str, Any]: + """Compare two ``greeneye-server-snapshot/1`` snapshots word by word. + + Emits a ``greeneye-server-snapshot-diff/1`` report: per shared chunk the + added/removed/changed word lists (with word indices), whole-chunk + added/removed rows, per-chunk and total counts, and record deltas. Output + is sorted by label and by word index, so two diffs of the same inputs are + byte-identical. + """ + for name, snapshot in (("a", a), ("b", b)): + if not isinstance(snapshot, dict) or snapshot.get("schema") != SNAPSHOT_FORMAT: + raise SnapshotError(f"snapshot {name} is not {SNAPSHOT_FORMAT}") + a_chunks = {chunk["label"]: chunk for chunk in a.get("chunks", [])} + b_chunks = {chunk["label"]: chunk for chunk in b.get("chunks", [])} + rows: list[dict[str, Any]] = [] + counts = { + "chunks_added": 0, "chunks_removed": 0, "chunks_changed": 0, + "chunks_unchanged": 0, "words_added": 0, "words_removed": 0, + "words_changed": 0, + } + for label in sorted(set(a_chunks) | set(b_chunks), key=_chunk_sort_key): + left = a_chunks.get(label) + right = b_chunks.get(label) + if left is None: + added = [{"index": index, "value": word} + for index, word in enumerate(right["words"])] + row = {"label": label, "status": "added", + "base": {"a": None, "b": right["base"]}, + "words": {"a": 0, "b": len(right["words"])}, + "added": added, "removed": [], "changed": []} + counts["chunks_added"] += 1 + counts["words_added"] += len(added) + elif right is None: + removed = [{"index": index, "value": word} + for index, word in enumerate(left["words"])] + row = {"label": label, "status": "removed", + "base": {"a": left["base"], "b": None}, + "words": {"a": len(left["words"]), "b": 0}, + "added": [], "removed": removed, "changed": []} + counts["chunks_removed"] += 1 + counts["words_removed"] += len(removed) + else: + a_words, b_words = left["words"], right["words"] + added, removed, changed = [], [], [] + for index in range(max(len(a_words), len(b_words))): + if index >= len(a_words): + added.append({"index": index, "value": b_words[index]}) + elif index >= len(b_words): + removed.append({"index": index, "value": a_words[index]}) + elif a_words[index] != b_words[index]: + changed.append({"index": index, "a": a_words[index], + "b": b_words[index]}) + unchanged = (not added and not removed and not changed + and left["base"] == right["base"]) + row = {"label": label, + "status": "unchanged" if unchanged else "changed", + "base": {"a": left["base"], "b": right["base"]}, + "words": {"a": len(a_words), "b": len(b_words)}, + "added": added, "removed": removed, "changed": changed} + if unchanged: + counts["chunks_unchanged"] += 1 + else: + counts["chunks_changed"] += 1 + counts["words_added"] += len(added) + counts["words_removed"] += len(removed) + counts["words_changed"] += len(changed) + rows.append(row) + + a_records = {record.get("raw", "") for record in a.get("records", [])} + b_records = {record.get("raw", "") for record in b.get("records", [])} + records_added = sorted(b_records - a_records) + records_removed = sorted(a_records - b_records) + counts["records_added"] = len(records_added) + counts["records_removed"] = len(records_removed) + identical = (counts["chunks_added"] == 0 and counts["chunks_removed"] == 0 + and counts["chunks_changed"] == 0 + and not records_added and not records_removed) + return { + "schema": DIFF_FORMAT, + "a": {"source": a.get("source"), "created": a.get("created")}, + "b": {"source": b.get("source"), "created": b.get("created")}, + "chunks": rows, + "records": {"added": records_added, "removed": records_removed}, + "summary": {"identical": identical, **counts}, + } + + +# --------------------------------------------------------------------------- +# Demo: two tiny snapshots in a tempdir, collected and diffed +# --------------------------------------------------------------------------- + +DEMO_A = """\ +[demo] MOCK[0000]: 00000001 00000002 00000003 00000004 +alpha[0]=0x00000011 +alpha[4]=0x00000022 +""" + +DEMO_B = """\ +[demo] MOCK[0000]: 00000001 000000ff 00000003 00000004 00000005 +alpha[0]=0x00000011 +alpha[4]=0x00000033 +""" + + +def _write_text(path: str, text: str) -> str: + with open(path, "w", encoding="utf-8") as handle: + handle.write(text) + return os.path.abspath(path) + + +def _write_json(path: str, payload: dict[str, Any]) -> str: + with open(path, "w", encoding="utf-8") as handle: + json.dump(payload, handle, indent=2, sort_keys=True) + handle.write("\n") + return os.path.abspath(path) + + +def run_demo(workdir: str) -> dict[str, Any]: + """Craft two tiny evidence files in ``workdir``, collect both and diff.""" + a_path = _write_text(os.path.join(workdir, "run-a.txt"), DEMO_A) + b_path = _write_text(os.path.join(workdir, "run-b.txt"), DEMO_B) + snapshot_a, _stats_a = collect(files=[a_path], source="demo-a", + created="2026-01-01T00:00:00Z") + snapshot_b, _stats_b = collect(files=[b_path], source="demo-b", + created="2026-01-01T00:00:00Z") + return { + "schema": DEMO_FORMAT, + "workdir": os.path.abspath(workdir), + "snapshot_a": snapshot_a, + "snapshot_b": snapshot_b, + "diff": diff_snapshots(snapshot_a, snapshot_b), + } + + +def _load_json(path: str) -> dict[str, Any]: + try: + with open(path, "r", encoding="utf-8") as handle: + return json.load(handle) + except (OSError, json.JSONDecodeError) as exc: + raise SnapshotError(f"cannot read {path}: {exc}") from exc + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="server_snapshot.py", + description="Collect and diff canonical server-state snapshots " + "(offline evidence normalizer).", + ) + sub = parser.add_subparsers(dest="command") + + collect_cmd = sub.add_parser( + "collect", help="normalize evidence files into one snapshot") + collect_cmd.add_argument("--dir", action="append", default=[], + help="evidence/session directory (repeatable)") + collect_cmd.add_argument("--file", action="append", default=[], + help="explicit evidence file (repeatable)") + collect_cmd.add_argument( + "--include", action="append", default=[], + help="additional file-name glob under --dir (repeatable); " + f"defaults: {', '.join(DEFAULT_INCLUDES)}") + collect_cmd.add_argument("--source", help="override the snapshot source") + collect_cmd.add_argument( + "--created", help="override the ISO-8601 timestamp (SOURCE_DATE_EPOCH " + "is honoured otherwise)") + collect_cmd.add_argument("--json", help="write the snapshot here instead " + "of stdout") + + diff_cmd = sub.add_parser("diff", help="compare two snapshot JSON files") + diff_cmd.add_argument("--a", required=True, help="first snapshot JSON") + diff_cmd.add_argument("--b", required=True, help="second snapshot JSON") + diff_cmd.add_argument("--json", help="write the diff here instead of stdout") + + demo_cmd = sub.add_parser( + "demo", help="craft two tiny snapshots in a tempdir and diff them") + demo_cmd.add_argument("--out", help="keep the crafted files in this directory") + return parser + + +def _collect_command(args) -> int: + if not args.dir and not args.file: + raise SnapshotError("collect needs at least one --dir or --file") + snapshot, stats = collect(dirs=args.dir, files=args.file, + includes=args.include, source=args.source, + created=args.created) + if args.json: + path = _write_json(args.json, snapshot) + summary = { + "schema": snapshot["schema"], + "source": snapshot["source"], + "created": snapshot["created"], + "chunks": len(snapshot["chunks"]), + "words": stats["words"], + "records": len(snapshot["records"]), + "unparsed": stats["unparsed"], + "collisions": stats["collisions"], + "snapshot_json": path, + } + print(json.dumps(summary, indent=2, sort_keys=True)) + else: + print(json.dumps(snapshot, indent=2, sort_keys=True)) + print(f"server_snapshot: {stats['files']} file(s), {stats['lines']} line(s), " + f"{stats['chunks']} chunk(s), {stats['words']} word(s), " + f"{stats['records']} record(s), {stats['unparsed']} unparsed, " + f"{stats['merged']} merged, {stats['collisions']} collision(s), " + f"{stats['conflicts']} conflict(s)", file=sys.stderr) + return 0 + + +def _diff_command(args) -> int: + report = diff_snapshots(_load_json(args.a), _load_json(args.b)) + if args.json: + path = _write_json(args.json, report) + print(json.dumps({"schema": report["schema"], **report["summary"], + "diff_json": path}, indent=2, sort_keys=True)) + else: + print(json.dumps(report, indent=2, sort_keys=True)) + return 0 if report["summary"]["identical"] else 3 + + +def _demo_command(args) -> int: + if args.out: + os.makedirs(args.out, exist_ok=True) + demo = run_demo(args.out) + else: + with tempfile.TemporaryDirectory(prefix="server-snapshot-demo-") as workdir: + demo = run_demo(workdir) + for row in demo["diff"]["chunks"]: + if row["status"] != "unchanged": + print(f"{row['status']:<9} {row['label']:<16} " + f"+{len(row['added'])} -{len(row['removed'])} " + f"~{len(row['changed'])}") + print(json.dumps(demo["diff"]["summary"], indent=2, sort_keys=True)) + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "collect": + return _collect_command(args) + if args.command == "diff": + return _diff_command(args) + if args.command == "demo": + return _demo_command(args) + parser.print_help() + return 2 + except SnapshotError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/state_decode.py b/debug/risc-v-gsp-rm-vm/harness/state_decode.py new file mode 100644 index 0000000..351515a --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/state_decode.py @@ -0,0 +1,1082 @@ +#!/usr/bin/env python3 +"""Decode engine-visible channel state words into named fields, and diff captures. + +During GR bring-up the answer often hides in a handful of 32-bit words inside +the INSTANCE block, the RAMFC chunk and context-buffer headers: one bit turns +"nothing happens" into an engine that runs or refuses. Raw hex hides that; +this tool turns a captured dump into a named field table and makes two +captures differ field-by-field. + +Schemas: + +* ``greeneye-state-decode/1`` -- the decoded report produced by ``decode``. +* ``greeneye-state-diff/1`` -- the word/field diff produced by ``diff``. +* ``greeneye-server-snapshot/1`` -- an input snapshot emitted elsewhere:: + + {"schema": "greeneye-server-snapshot/1", "source": "...", + "chunks": [{"label": "INSTANCE", "base": "0x0", + "words": ["0x...", "..."]}]} + + (one hex string or integer per 32-bit word; accepted exactly). +* ``greeneye-state-map/1`` -- an optional field map (``--map``), see below. + +Input formats (auto-detected per ``--input``): + +(a) text lines ``LABEL[OFFSET]=VALUE`` -- label case-insensitive, offset and + value hex (``0x..``, letters, or leading-zero padded) or decimal; every + other line is ignored; +(b) a dump row with a bracketed chunk offset and words, e.g. + ``... INSTANCE[0040]: 00100000 00030000 ...`` -- this is the shape live + ladder traces print (8 words per row; any count accepted); +(c) plain hexdump rows ``00000000: 000002d2 00000000 ...`` named with + ``--label``/``--base``; +(d) raw little-endian u32 words (``--bin``, or auto-detected when a text parse + finds nothing and the bytes are control-heavy and 4-byte aligned); +(e) the snapshot JSON above. + +Offsets inside a chunk are byte offsets relative to the chunk start (all +inputs above are u32 word dumps, so offsets are multiples of 4). ``base`` is +the engine-visible address of chunk-relative offset 0: it is recorded as +provenance and for the diff, but it does not shift map lookup. + +Bundled map (deliberately small; extend or override it from a map file, never +by guessing here). Provenance is live bring-up observation (2026-10 GSP +ladder runs): + +* ``INSTANCE`` 0x010: magic ``0xface``. +* ``INSTANCE`` 0x048: GPFIFO VA, observed as ``0x100000``. +* ``RAMFC`` 0x048: ``RAMFC_TARGET`` -- bits 16/17 ``ENG_CTX_VALID`` / + ``CE_CTX_VALID``; the observed live success word ``0x00030000`` sets both. + The bit assignment is observation-driven and fully configurable via the map. + +Map file:: + + {"format": "greeneye-state-map/1", "mode": "extend" | "replace", + "entries": [ + {"chunk": "RAMFC", "label": "RAMFC_TARGET", "offset": "0x48", + "width": 4, + "bits": [{"name": "ENG_CTX_VALID", "lo": 16, "hi": 16, + "values": {"0": "clear", "1": "set"}}], + "note": "..."} + ]} + +A user entry with the same ``chunk`` and ``offset`` replaces the bundled +entry; new chunk/offset pairs extend the table. ``mode: replace`` drops the +bundled table first. An entry without ``bits`` decodes as a named value. + +Honesty: this tool names observed words; it does not prove their meaning. An +unmapped word is silence, not absence. Field offsets without evidence stay +unnamed until a map supplies them. Offline only: no hardware, no module, no +guest, no network. +""" +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import sys +import tempfile +from typing import Any + +DECODE_FORMAT = "greeneye-state-decode/1" +DIFF_FORMAT = "greeneye-state-diff/1" +SNAPSHOT_FORMAT = "greeneye-server-snapshot/1" +MAP_FORMAT = "greeneye-state-map/1" +DEMO_FORMAT = "greeneye-state-decode-demo/1" + +WORD_BYTES = 4 +WORD_MASK = 0xFFFFFFFF +WORD_WIDTHS = (4, 8) + + +class StateDecodeError(ValueError): + """An input, map or request the tool cannot honour.""" + + +# --------------------------------------------------------------------------- +# Bundled field table +# --------------------------------------------------------------------------- +# Every entry is an observation from live GSP bring-up traces (2026-10). The +# table is intentionally tiny: unnamed offsets stay unnamed until evidence +# earns them a name, and the map file carries everything else. +BUILTIN_MAP: dict[str, Any] = { + "format": MAP_FORMAT, + "mode": "extend", + "entries": [ + { + "chunk": "INSTANCE", + "label": "MAGIC_0FACE", + "offset": "0x010", + "width": 4, + "note": "Instance magic observed as 0xface at chunk byte 0x010.", + }, + { + "chunk": "INSTANCE", + "label": "GPFIFO_VA", + "offset": "0x048", + "width": 4, + "note": "GPFIFO VA observed as 0x100000 at chunk byte 0x048.", + }, + { + "chunk": "RAMFC", + "label": "RAMFC_TARGET", + "offset": "0x048", + "width": 4, + "bits": [ + {"name": "ENG_CTX_VALID", "lo": 16, "hi": 16}, + {"name": "CE_CTX_VALID", "lo": 17, "hi": 17}, + ], + "note": ("Engine/CE context-valid bits; the observed live success " + "word 0x00030000 sets both. Bits are configurable via " + "--map."), + }, + ], +} + +# --------------------------------------------------------------------------- +# Number and token parsing +# --------------------------------------------------------------------------- + +_EQ_RE = re.compile( + r"^\s*(?P<label>[A-Za-z_][A-Za-z0-9_.-]*)\s*" + r"\[\s*(?P<off>[0-9a-fA-FxX]+)\s*\]\s*=\s*(?P<val>[0-9a-fA-FxX]+)\s*$") + +_ROW_RE = re.compile( + r"(?P<label>[A-Za-z_][A-Za-z0-9_.-]*)\s*" + r"\[\s*(?P<off>[0-9a-fA-FxX]+)\s*\]\s*:\s*" + r"(?P<words>[0-9a-fA-FxX]+(?:\s+[0-9a-fA-FxX]+)*)\s*$") + +_HEXDUMP_RE = re.compile( + r"^\s*(?P<off>[0-9a-fA-FxX]{2,16})\s*:\s*" + r"(?P<words>[0-9a-fA-FxX]+(?:\s+[0-9a-fA-FxX]+)*)\s*$") + +_HEX_RE = re.compile(r"[0-9a-fA-F]+") + +_TEXT_BYTES = frozenset(b"\t\n\r") | frozenset(range(0x20, 0x7F)) + + +def parse_number(value: Any, *, why: str = "number") -> int: + """Parse an int, a ``0x`` string, or a bare token. + + Bare tokens with a hex letter (``face``) or a leading zero and more + digits (``0040``) are read as hex, matching dump padding; other bare + digit strings are decimal. Write ``0x..`` when in doubt. + """ + if isinstance(value, bool): + raise StateDecodeError(f"bad {why} {value!r}") + if isinstance(value, int): + return value + if not isinstance(value, str): + raise StateDecodeError(f"bad {why} {value!r}") + text = value.strip() + if not text: + raise StateDecodeError(f"bad {why} {value!r}") + sign = 1 + if text[0] in "+-": + if text[0] == "-": + sign = -1 + text = text[1:] + low = text.lower() + if low.startswith("0x"): + base, digits = 16, text[2:] + elif re.fullmatch(r"[0-9]+", text): + base = 16 if text.startswith("0") and len(text) > 1 else 10 + digits = text + elif _HEX_RE.fullmatch(text): + base, digits = 16, text + else: + raise StateDecodeError(f"bad {why} {value!r}") + try: + return sign * int(digits, base) + except ValueError as exc: + raise StateDecodeError(f"bad {why} {value!r}") from exc + + +def _parse_word(value: Any, *, where: str) -> int: + number = parse_number(value, why=where) + if not 0 <= number <= WORD_MASK: + raise StateDecodeError(f"{where} {value!r} does not fit in 32 bits") + return number + + +def _hex(value: int, width: int = 4) -> str: + return f"{value:#0{2 + width * 2}x}" + + +def _parse_base(value: Any) -> int: + if value is None: + return 0 + return parse_number(value, why="base") + + +# --------------------------------------------------------------------------- +# Map loading (bundled <- override/extension) +# --------------------------------------------------------------------------- + +def _normalize_entry(raw: Any, *, where: str) -> dict[str, Any]: + if not isinstance(raw, dict): + raise StateDecodeError(f"{where} must be an object") + chunk = raw.get("chunk") + label = raw.get("label") + if not isinstance(chunk, str) or not chunk: + raise StateDecodeError(f"{where} needs a non-empty chunk") + if not isinstance(label, str) or not label: + raise StateDecodeError(f"{where} needs a non-empty label") + if raw.get("offset") is None: + raise StateDecodeError(f"{where} needs an offset") + offset = parse_number(raw["offset"], why=f"{where} offset") + width = raw.get("width", 4) + width = parse_number(width, why=f"{where} width") + if width not in WORD_WIDTHS: + raise StateDecodeError(f"{where} width {width!r} is not 4 or 8") + if offset < 0 or offset % WORD_BYTES: + raise StateDecodeError(f"{where} offset {offset:#x} is not word-aligned") + bits_raw = raw.get("bits") or [] + if not isinstance(bits_raw, list): + raise StateDecodeError(f"{where} bits must be a list") + nbits = width * 8 + bits = [] + for index, bit in enumerate(bits_raw): + if not isinstance(bit, dict): + raise StateDecodeError(f"{where} bits[{index}] must be an object") + name = bit.get("name") + if not isinstance(name, str) or not name: + raise StateDecodeError(f"{where} bits[{index}] needs a name") + lo = parse_number(bit.get("lo"), why=f"{where} bits[{index}].lo") + hi = parse_number(bit.get("hi"), why=f"{where} bits[{index}].hi") + if not 0 <= lo <= hi < nbits: + raise StateDecodeError( + f"{where} bits[{index}] range {lo}..{hi} outside width {width}") + values = bit.get("values") + norm_values: dict[int, str] = {} + if values is not None: + if not isinstance(values, dict): + raise StateDecodeError( + f"{where} bits[{index}].values must be an object") + for key, text in values.items(): + norm_values[parse_number(key, why=f"{where} values key")] = str(text) + bits.append({"name": name, "lo": lo, "hi": hi, "values": norm_values}) + note = raw.get("note") + return { + "chunk": chunk, + "label": label, + "offset": offset, + "width": width, + "bits": bits, + "note": note if isinstance(note, str) else None, + } + + +def load_map(path: str | None = None) -> tuple[dict[tuple[str, int], dict], str]: + """Build the field table; returns ``(table, source)``. + + ``table`` is keyed by ``(chunk.lower(), offset)``. Without ``path`` the + bundled table is used. With ``path``, entries merge over the bundled + table (same chunk+offset replaces, new pairs extend) unless the file sets + ``"mode": "replace"``. + """ + entries: list[Any] = list(BUILTIN_MAP["entries"]) + source = "builtin" + if path: + if not os.path.isfile(path): + raise StateDecodeError(f"map not found: {path}") + with open(path, "r", encoding="utf-8") as handle: + doc = json.load(handle) + source = os.path.abspath(path) + if isinstance(doc, list): + raw_entries, mode = doc, "extend" + elif isinstance(doc, dict): + fmt = doc.get("format", MAP_FORMAT) + if fmt != MAP_FORMAT: + raise StateDecodeError( + f"unsupported map format {fmt!r} (want {MAP_FORMAT})") + raw_entries = doc.get("entries") + mode = doc.get("mode", "extend") + else: + raise StateDecodeError(f"map {path} must be an object or a list") + if mode not in ("extend", "replace"): + raise StateDecodeError(f"map mode {mode!r} is not extend/replace") + if not isinstance(raw_entries, list): + raise StateDecodeError(f"map {path} needs an entries list") + if mode == "replace": + entries = [] + entries.extend(raw_entries) + table: dict[tuple[str, int], dict] = {} + for index, raw in enumerate(entries): + entry = _normalize_entry(raw, where=f"map entry {index}") + table[(entry["chunk"].lower(), entry["offset"])] = entry + return table, source + + +# --------------------------------------------------------------------------- +# Input loading (auto-detected per input) +# --------------------------------------------------------------------------- + +def _read_bytes(path: str) -> bytes: + if not os.path.isfile(path): + raise StateDecodeError(f"input not found: {path}") + with open(path, "rb") as handle: + return handle.read() + + +def _try_json(data: bytes) -> Any: + """Return parsed JSON when the bytes look like JSON, else ``None``.""" + try: + text = data.decode("utf-8") + except UnicodeDecodeError: + return None + stripped = text.lstrip() + if not stripped.startswith("{"): + return None + try: + return json.loads(text) + except json.JSONDecodeError as exc: + raise StateDecodeError(f"invalid JSON input: {exc}") from exc + + +def _try_text(data: bytes) -> str | None: + try: + return data.decode("utf-8") + except UnicodeDecodeError: + return None + + +def _looks_binary(data: bytes) -> bool: + if not data or len(data) % WORD_BYTES: + return False + return any(byte not in _TEXT_BYTES for byte in data) + + +def _chunk_from_binary(data: bytes, label: str, base: int, source: str) -> dict: + if not data: + raise StateDecodeError(f"binary input {source} is empty") + if len(data) % WORD_BYTES: + raise StateDecodeError( + f"binary input {source} size {len(data)} is not a multiple of 4") + words = {offset: int.from_bytes(data[offset:offset + WORD_BYTES], "little") + for offset in range(0, len(data), WORD_BYTES)} + return {"label": label, "base": base, "words": words, + "format": "binary", "source": source} + + +def _document(path: str, fmt: str, label: str | None, base: int, + chunks: list[dict]) -> dict: + return { + "path": os.path.abspath(path), + "format": fmt, + "label": label, + "base": base, + "chunks": chunks, + } + + +def parse_text(text: str, *, default_label: str | None = None, + default_base: int = 0, source: str | None = None) -> list[dict]: + """Parse the text/hexdump/row formats into chunks. + + Chunk names are case-insensitive: ``INSTANCE`` and ``inst`` land in one + chunk, displayed with the first spelling seen. Every line that matches + none of the three patterns is ignored. + """ + chunks: dict[str, dict] = {} + + def chunk_for(label: str) -> dict: + key = label.lower() + chunk = chunks.get(key) + if chunk is None: + chunk = {"label": label, "base": default_base, "words": {}, + "format": "text", "source": source} + chunks[key] = chunk + return chunk + + def put(label: str, offset: int, value: int, where: str) -> None: + if offset < 0 or offset % WORD_BYTES: + raise StateDecodeError(f"{where}: offset {offset:#x} is not word-aligned") + if not 0 <= value <= WORD_MASK: + raise StateDecodeError(f"{where}: value {value:#x} does not fit in 32 bits") + chunk_for(label)["words"][offset] = value + + for lineno, raw_line in enumerate(text.splitlines(), 1): + line = raw_line.strip() + if not line: + continue + where = f"{source or '<text>'}:{lineno}" + match = _EQ_RE.match(line) + if match: + offset = parse_number(match.group("off"), why=f"{where} offset") + value = parse_number(match.group("val"), why=f"{where} value") + put(match.group("label"), offset, value, where) + continue + match = _ROW_RE.search(line) + if match: + offset = parse_number(match.group("off"), why=f"{where} offset") + for index, token in enumerate(match.group("words").split()): + put(match.group("label"), offset + index * WORD_BYTES, + int(token, 16), where) + continue + match = _HEXDUMP_RE.match(line) + if match: + label = default_label or "RAW" + offset = int(match.group("off"), 16) + for index, token in enumerate(match.group("words").split()): + put(label, offset + index * WORD_BYTES, int(token, 16), where) + continue + return [chunks[key] for key in chunks] + + +def load_input(spec: dict) -> dict: + """Load one input spec dict (path/label/base/binary) into a document.""" + path = spec.get("path") + if not path: + raise StateDecodeError("input spec needs a path") + data = _read_bytes(path) + label = spec.get("label") + base = _parse_base(spec.get("base")) + if spec.get("binary"): + chunk = _chunk_from_binary(data, label or "RAW", base, path) + return _document(path, "binary", label, base, [chunk]) + doc = _try_json(data) + if doc is not None: + return _document_from_json(path, doc, label, base) + text = _try_text(data) + if text is not None: + chunks = parse_text(text, default_label=label, default_base=base, + source=path) + if chunks: + return _document(path, "text", label, base, chunks) + if _looks_binary(data): + chunk = _chunk_from_binary(data, label or "RAW", base, path) + return _document(path, "binary", label, base, [chunk]) + raise StateDecodeError( + f"cannot decode {path}: not text fields, not {SNAPSHOT_FORMAT} JSON, " + f"not raw u32 words") + + +def _document_from_json(path: str, doc: Any, label: str | None, + base: int) -> dict: + if isinstance(doc, dict) and doc.get("format") == DECODE_FORMAT: + raise StateDecodeError(f"{path} is a decoded report; use `diff` on it") + if not isinstance(doc, dict) or doc.get("schema") != SNAPSHOT_FORMAT: + got = doc.get("schema") if isinstance(doc, dict) else type(doc).__name__ + raise StateDecodeError( + f"unsupported JSON input {path}: schema={got!r} " + f"(want {SNAPSHOT_FORMAT})") + raw_chunks = doc.get("chunks") + if not isinstance(raw_chunks, list) or not raw_chunks: + raise StateDecodeError(f"snapshot {path} has no chunks") + chunks = [] + for index, raw in enumerate(raw_chunks): + if not isinstance(raw, dict): + raise StateDecodeError(f"{path}:chunks[{index}] must be an object") + chunk_label = raw.get("label") or label or "RAW" + chunk_base = (_parse_base(raw["base"]) if raw.get("base") is not None + else base) + words_raw = raw.get("words") + if not isinstance(words_raw, list): + raise StateDecodeError( + f"{path}:chunks[{index}] needs a words list") + words = {} + for position, word in enumerate(words_raw): + words[position * WORD_BYTES] = _parse_word( + word, where=f"{path}:chunks[{index}].words[{position}]") + chunks.append({"label": str(chunk_label), "base": chunk_base, + "words": words, "format": "snapshot-json", + "source": path}) + return _document(path, "snapshot-json", label, base, chunks) + + +# --------------------------------------------------------------------------- +# Decode +# --------------------------------------------------------------------------- + +def _read_field_value(words: dict[int, int], offset: int, + width: int) -> int | None: + """Little-endian read of a 4/8-byte field; ``None`` when words are absent.""" + data = b"" + for step in range(0, width, WORD_BYTES): + word = words.get(offset + step) + if word is None: + return None + data += word.to_bytes(WORD_BYTES, "little") + return int.from_bytes(data, "little") + + +def _bit_state(bit: dict, value: int) -> str: + values = bit.get("values") or {} + if value in values: + return values[value] + mask = (1 << (bit["hi"] - bit["lo"] + 1)) - 1 + if value == 0: + return "clear" + if value == mask: + return "set" + return f"{value:#x}" + + +def _decode_field(entry: dict, value: int) -> dict: + width = entry["width"] + field: dict[str, Any] = { + "name": entry["label"], + "offset": f"{entry['offset']:#x}", + "width": width, + "kind": "bits" if entry["bits"] else "value", + "raw": _hex(value, width), + "value": value, + } + if entry.get("note"): + field["note"] = entry["note"] + if entry["bits"]: + bitfields = [] + for bit in entry["bits"]: + raw_bit = (value >> bit["lo"]) & ((1 << (bit["hi"] - bit["lo"] + 1)) - 1) + bitfields.append({ + "name": bit["name"], + "lo": bit["lo"], + "hi": bit["hi"], + "value": raw_bit, + "state": _bit_state(bit, raw_bit), + }) + field["bitfields"] = bitfields + return field + + +def decode_chunk(chunk: dict, entries: dict[tuple[str, int], dict]) -> dict: + """Decode one internal chunk into the ``greeneye-state-decode/1`` shape.""" + label = chunk["label"] + words: dict[int, int] = chunk["words"] + matched = sorted( + (entry for (chunk_key, _off), entry in entries.items() + if chunk_key == label.lower()), + key=lambda entry: (entry["offset"], entry["label"])) + fields = [] + covered: set[int] = set() + for entry in matched: + value = _read_field_value(words, entry["offset"], entry["width"]) + if value is None: + continue + fields.append(_decode_field(entry, value)) + covered.update(range(entry["offset"], + entry["offset"] + entry["width"], WORD_BYTES)) + unmapped = [{"offset": f"{offset:#x}", "raw": _hex(words[offset])} + for offset in sorted(words) if offset not in covered] + return { + "label": label, + "base": f"{chunk['base']:#x}", + "source": chunk.get("source"), + "input_format": chunk.get("format"), + "word_width": WORD_BYTES, + "word_count": len(words), + "words": {f"{offset:#x}": _hex(value) + for offset, value in sorted(words.items())}, + "fields": fields, + "unmapped": unmapped, + "unmapped_count": len(unmapped), + } + + +def decode_specs(specs: list[dict], entries: dict[tuple[str, int], dict], + map_source: str = "builtin") -> dict: + """Load and decode input specs into a ``greeneye-state-decode/1`` report.""" + sources = [] + chunks = [] + for spec in specs: + document = load_input(spec) + sources.append({ + "path": document["path"], + "format": document["format"], + "label": document["label"], + "base": f"{document['base']:#x}", + "chunk_labels": [chunk["label"] for chunk in document["chunks"]], + }) + for chunk in document["chunks"]: + chunks.append(decode_chunk(chunk, entries)) + chunks.sort(key=lambda chunk: (chunk["label"].lower(), chunk["label"], + int(chunk["base"], 16))) + return { + "format": DECODE_FORMAT, + "map_source": map_source, + "sources": sources, + "chunks": chunks, + "limits": [ + "Field names come from the bundled/extended map; an unmapped word " + "is silence, not absence.", + "Offsets are chunk-relative bytes; base is recorded provenance and " + "does not shift field lookup.", + "No timing (L2) or device-environment (L3) claim: this is a field " + "decoder over captured words.", + ], + } + + +# --------------------------------------------------------------------------- +# Diff +# --------------------------------------------------------------------------- + +def _validate_report(doc: Any, side: str) -> None: + if not isinstance(doc, dict) or doc.get("format") != DECODE_FORMAT: + raise StateDecodeError(f"{side} is not a {DECODE_FORMAT} report") + if not isinstance(doc.get("chunks"), list): + raise StateDecodeError(f"{side} report has no chunks list") + + +def _words_to_int(chunk: dict) -> dict[int, int]: + return {int(offset, 16): int(raw, 16) + for offset, raw in chunk["words"].items()} + + +def _field_index(chunk: dict | None) -> dict[tuple[int, str], dict]: + if chunk is None: + return {} + return {(int(field["offset"], 16), field["name"]): field + for field in chunk["fields"]} + + +def _field_summary(field: dict | None) -> str | None: + if field is None: + return None + if field.get("kind") == "bits": + parts = [f"{bit['name']}={bit['state']}" + for bit in field.get("bitfields", [])] + return ", ".join(parts) if parts else field["raw"] + return f"value={field['raw']}" + + +def _changed_fields(left: dict | None, right: dict | None) -> list[dict]: + left_fields = _field_index(left) + right_fields = _field_index(right) + rows = [] + for offset, name in sorted(set(left_fields) | set(right_fields)): + left_field = left_fields.get((offset, name)) + right_field = right_fields.get((offset, name)) + if (left_field is not None and right_field is not None + and left_field["raw"] == right_field["raw"]): + continue + row: dict[str, Any] = { + "offset": f"{offset:#x}", + "name": name, + "old": left_field["raw"] if left_field else None, + "new": right_field["raw"] if right_field else None, + "old_summary": _field_summary(left_field), + "new_summary": _field_summary(right_field), + } + if (left_field is not None and right_field is not None + and left_field.get("kind") == right_field.get("kind") == "bits"): + left_bits = {bit["name"]: bit for bit in left_field["bitfields"]} + right_bits = {bit["name"]: bit for bit in right_field["bitfields"]} + bits = [] + for bit_name in sorted(set(left_bits) | set(right_bits)): + bits.append({ + "name": bit_name, + "old": (left_bits[bit_name]["state"] + if bit_name in left_bits else None), + "new": (right_bits[bit_name]["state"] + if bit_name in right_bits else None), + }) + row["bits"] = bits + rows.append(row) + return rows + + +def _word_rows(words: dict[int, int]) -> list[dict]: + return [{"offset": f"{offset:#x}", "raw": _hex(raw)} + for offset, raw in sorted(words.items())] + + +def _diff_chunk(left: dict | None, right: dict | None) -> dict: + if left is None or right is None: + present = right if left is None else left + words = _words_to_int(present) + rows = _word_rows(words) + row = { + "label": present["label"], + "base": present["base"], + "left_base": left["base"] if left else None, + "right_base": right["base"] if right else None, + "status": "right_only" if left is None else "left_only", + "added": rows if left is None else [], + "removed": rows if right is None else [], + "changed": [], + "changed_fields": _changed_fields(left, right), + } + return row + left_words = _words_to_int(left) + right_words = _words_to_int(right) + added = _word_rows({offset: right_words[offset] + for offset in right_words if offset not in left_words}) + removed = _word_rows({offset: left_words[offset] + for offset in left_words if offset not in right_words}) + changed = [{"offset": f"{offset:#x}", "old": _hex(left_words[offset]), + "new": _hex(right_words[offset])} + for offset in sorted(set(left_words) & set(right_words)) + if left_words[offset] != right_words[offset]] + changed_fields = _changed_fields(left, right) + status = "changed" if (added or removed or changed or changed_fields) \ + else "identical" + return { + "label": left["label"], + "base": left["base"], + "left_base": left["base"], + "right_base": right["base"], + "status": status, + "added": added, + "removed": removed, + "changed": changed, + "changed_fields": changed_fields, + } + + +def diff_reports(left: dict, right: dict) -> dict: + """Deterministic word/field diff of two decoded reports.""" + _validate_report(left, "left") + _validate_report(right, "right") + left_groups: dict[str, list[dict]] = {} + right_groups: dict[str, list[dict]] = {} + for chunk in left["chunks"]: + left_groups.setdefault(chunk["label"].lower(), []).append(chunk) + for chunk in right["chunks"]: + right_groups.setdefault(chunk["label"].lower(), []).append(chunk) + labels = sorted(set(left_groups) | set(right_groups)) + chunks = [] + for label in labels: + left_list = left_groups.get(label, []) + right_list = right_groups.get(label, []) + for index in range(max(len(left_list), len(right_list))): + left_chunk = left_list[index] if index < len(left_list) else None + right_chunk = right_list[index] if index < len(right_list) else None + chunks.append(_diff_chunk(left_chunk, right_chunk)) + summary = { + "chunks": len(chunks), + "changed_chunks": sum(1 for chunk in chunks + if chunk["status"] != "identical"), + "added": sum(len(chunk["added"]) for chunk in chunks), + "removed": sum(len(chunk["removed"]) for chunk in chunks), + "changed_words": sum(len(chunk["changed"]) for chunk in chunks), + "changed_fields": sum(len(chunk["changed_fields"]) for chunk in chunks), + } + return { + "format": DIFF_FORMAT, + "left": {"format": left["format"], "map_source": left.get("map_source"), + "chunks": len(left["chunks"])}, + "right": {"format": right["format"], "map_source": right.get("map_source"), + "chunks": len(right["chunks"])}, + "chunks": chunks, + "summary": summary, + "identical": all(chunk["status"] == "identical" for chunk in chunks), + "limits": [ + "Word/field diff only; an unchanged word does not prove untested " + "behaviour.", + "Chunks pair by label and occurrence order; a differing base is " + "recorded but does not re-pair.", + ], + } + + +# --------------------------------------------------------------------------- +# Rendering +# --------------------------------------------------------------------------- + +def render_decode(report: dict) -> str: + """Readable field table for humans; the JSON report stays canonical.""" + lines = [] + for chunk in report["chunks"]: + lines.append( + f"chunk {chunk['label']} base={chunk['base']} " + f"words={chunk['word_count']} mapped={len(chunk['fields'])} " + f"unmapped={chunk['unmapped_count']}") + for field in chunk["fields"]: + if field["kind"] == "bits": + detail = " ".join(f"{bit['name']}={bit['state']}" + for bit in field["bitfields"]) + else: + detail = f"value={field['raw']}" + lines.append(f" {field['offset']:>6} {field['raw']:<12} " + f"{field['name']:<20} {detail}") + nonzero = [word for word in chunk["unmapped"] + if int(word["raw"], 16)] + if nonzero: + shown = " ".join(f"{word['offset']}={word['raw']}" + for word in nonzero[:12]) + more = "" if len(nonzero) <= 12 else f" (+{len(nonzero) - 12} more)" + lines.append(f" unmapped nonzero: {shown}{more}") + return "\n".join(lines) + + +def render_diff(report: dict) -> str: + """Readable diff for humans; the JSON report stays canonical.""" + lines = [] + for chunk in report["chunks"]: + lines.append(f"chunk {chunk['label']} base={chunk['base']}: " + f"{chunk['status']}") + for word in chunk["added"]: + lines.append(f" added {word['offset']} {word['raw']}") + for word in chunk["removed"]: + lines.append(f" removed {word['offset']} {word['raw']}") + for word in chunk["changed"]: + lines.append(f" changed {word['offset']} " + f"{word['old']} -> {word['new']}") + for field in chunk["changed_fields"]: + lines.append(f" field {field['name']} @ {field['offset']}: " + f"{field['old']} -> {field['new']} " + f"({field['old_summary']} -> {field['new_summary']})") + for bit in field.get("bits", []): + lines.append(f" bit {bit['name']}: " + f"{bit['old']} -> {bit['new']}") + lines.append(f"summary: {json.dumps(report['summary'], sort_keys=True)} " + f"identical={report['identical']}") + return "\n".join(lines) + + +# --------------------------------------------------------------------------- +# Demo +# --------------------------------------------------------------------------- + +def _demo_words(word48: int) -> list[str]: + words = ["0x0"] * 20 # chunk words 0x00..0x4c + words[0x10 // WORD_BYTES] = "0xface" + words[0x48 // WORD_BYTES] = _hex(word48) + return words + + +def _demo_snapshot(name: str, *, gpfifo_va: int, ramfc_target: int) -> dict: + return { + "schema": SNAPSHOT_FORMAT, + "source": name, + "chunks": [ + {"label": "INSTANCE", "base": "0x0", + "words": _demo_words(gpfifo_va)}, + {"label": "RAMFC", "base": "0x0", + "words": _demo_words(ramfc_target)}, + ], + } + + +def run_demo(out: str | None = None) -> dict: + """Craft two snapshots in a tempdir, decode+diff them, and report. + + Expected: INSTANCE word 0x048 moves 0x100000 -> 0x200000 and RAMFC word + 0x048 moves 0x00030000 -> 0x00010000 (CE_CTX_VALID clears), i.e. two + changed words and two changed named fields. + """ + demo_dir = tempfile.mkdtemp(prefix="greeneye-state-decode-demo-") + try: + left_path = os.path.join(demo_dir, "left.json") + right_path = os.path.join(demo_dir, "right.json") + left_snapshot = _demo_snapshot("demo-left", gpfifo_va=0x100000, + ramfc_target=0x00030000) + right_snapshot = _demo_snapshot("demo-right", gpfifo_va=0x200000, + ramfc_target=0x00010000) + _write_json(left_path, left_snapshot) + _write_json(right_path, right_snapshot) + entries, map_source = load_map(None) + left = decode_specs( + [{"path": left_path, "label": None, "base": None, "binary": False}], + entries, map_source) + right = decode_specs( + [{"path": right_path, "label": None, "base": None, "binary": False}], + entries, map_source) + diff = diff_reports(left, right) + ok = (not diff["identical"] + and diff["summary"]["changed_words"] == 2 + and diff["summary"]["changed_fields"] == 2 + and diff["summary"]["added"] == 0 + and diff["summary"]["removed"] == 0) + demo = { + "format": DEMO_FORMAT, + "ok": ok, + "left": left, + "right": right, + "diff": diff, + } + if out: + os.makedirs(out, exist_ok=True) + artifacts = {} + for name, doc in (("state-decode-left.json", left), + ("state-decode-right.json", right), + ("state-diff.json", diff)): + artifacts[name] = _write_json(os.path.join(out, name), doc) + for name, path in (("state-snapshot-left.json", left_path), + ("state-snapshot-right.json", right_path)): + target = os.path.join(out, name) + shutil.copyfile(path, target) + artifacts[name] = os.path.abspath(target) + demo["artifacts"] = artifacts + return demo + finally: + shutil.rmtree(demo_dir, ignore_errors=True) + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +class _InputAction(argparse.Action): + """Collect repeatable ``--input``/``--bin`` specs in command order.""" + + def __init__(self, option_strings, dest, binary=False, **kwargs): + self.binary = binary + super().__init__(option_strings, dest, **kwargs) + + def __call__(self, parser, namespace, values, option_string=None): + specs = getattr(namespace, self.dest, None) + if specs is None: + specs = [] + setattr(namespace, self.dest, specs) + spec = {"path": values, "label": None, "base": None, + "binary": self.binary} + specs.append(spec) + setattr(namespace, "_current_spec", spec) + + +class _InputMetaAction(argparse.Action): + """Attach ``--label``/``--base`` to the preceding input, or set a default.""" + + def __call__(self, parser, namespace, values, option_string=None): + current = getattr(namespace, "_current_spec", None) + if current is not None: + current[self.dest] = values + setattr(namespace, self.dest, values) + + +def _apply_defaults(specs: list[dict], label: str | None, + base: str | None) -> list[dict]: + resolved = [] + for spec in specs: + item = dict(spec) + if item.get("label") is None: + item["label"] = label + if item.get("base") is None: + item["base"] = base + resolved.append(item) + return resolved + + +def _peek_report(path: str) -> dict | None: + data = _read_bytes(path) + doc = _try_json(data) + if isinstance(doc, dict) and doc.get("format") == DECODE_FORMAT: + _validate_report(doc, path) + return doc + return None + + +def _load_side(spec: dict, entries: dict[tuple[str, int], dict], + map_source: str) -> dict: + report = _peek_report(spec["path"]) + if report is not None: + return report + return decode_specs([spec], entries, map_source) + + +def _write_json(path: str, doc: Any) -> str: + absolute = os.path.abspath(path) + parent = os.path.dirname(absolute) + if parent: + os.makedirs(parent, exist_ok=True) + with open(absolute, "w", encoding="utf-8") as handle: + json.dump(doc, handle, indent=2, sort_keys=True) + handle.write("\n") + return absolute + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="state_decode.py", + description="Decode engine-visible channel state words into named " + "fields, and diff two captures (offline).") + sub = parser.add_subparsers(dest="command") + + decode = sub.add_parser( + "decode", help="decode one or more raw inputs into a field table") + decode.add_argument("--input", action=_InputAction, dest="inputs", + default=None, metavar="FILE", + help="input file; repeatable, auto-detected format") + decode.add_argument("--bin", action=_InputAction, dest="inputs", + default=None, metavar="FILE", + help="raw little-endian u32 input; repeatable") + decode.add_argument("--label", action=_InputMetaAction, default=None, + help="chunk label for the preceding --input/--bin " + "(default for the rest)") + decode.add_argument("--base", action=_InputMetaAction, default="0x0", + help="engine address of chunk offset 0 for the " + "preceding input (default 0x0)") + decode.add_argument("--map", dest="map_path", default=None, + help=f"{MAP_FORMAT} override/extension file") + decode.add_argument("--json", dest="json_path", default=None, + help="write the decoded report JSON here") + + diff = sub.add_parser( + "diff", help="diff two decoded reports or two raw inputs") + diff.add_argument("left", help=f"{DECODE_FORMAT} report or raw input") + diff.add_argument("right", help=f"{DECODE_FORMAT} report or raw input") + diff.add_argument("--left-label", dest="left_label", default=None, + help="chunk label for a raw left input") + diff.add_argument("--left-base", dest="left_base", default="0x0", + help="base for a raw left input") + diff.add_argument("--left-bin", dest="left_bin", action="store_true", + help="force the left input to raw u32 words") + diff.add_argument("--right-label", dest="right_label", default=None, + help="chunk label for a raw right input") + diff.add_argument("--right-base", dest="right_base", default="0x0", + help="base for a raw right input") + diff.add_argument("--right-bin", dest="right_bin", action="store_true", + help="force the right input to raw u32 words") + diff.add_argument("--map", dest="map_path", default=None, + help=f"{MAP_FORMAT} override/extension file") + diff.add_argument("--json", dest="json_path", default=None, + help="write the diff report JSON here") + + demo = sub.add_parser( + "demo", help="craft two snapshots, decode+diff them, print the result") + demo.add_argument("--out", default=None, + help="write demo snapshots and reports to this directory") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "decode": + specs = _apply_defaults(list(args.inputs or []), args.label, + args.base) + if not specs: + raise StateDecodeError("decode needs at least one --input/--bin") + entries, map_source = load_map(args.map_path) + report = decode_specs(specs, entries, map_source) + print(render_decode(report)) + if args.json_path: + _write_json(args.json_path, report) + return 0 + if args.command == "diff": + entries, map_source = load_map(args.map_path) + left = _load_side({"path": args.left, "label": args.left_label, + "base": args.left_base, + "binary": args.left_bin}, entries, map_source) + right = _load_side({"path": args.right, "label": args.right_label, + "base": args.right_base, + "binary": args.right_bin}, entries, map_source) + report = diff_reports(left, right) + print(render_diff(report)) + if args.json_path: + _write_json(args.json_path, report) + return 0 if report["identical"] else 3 + if args.command == "demo": + demo = run_demo(args.out) + print(f"demo: {demo['format']} ok={demo['ok']}") + print(render_decode(demo["left"])) + print(render_diff(demo["diff"])) + print(json.dumps({"ok": demo["ok"], + "summary": demo["diff"]["summary"]}, + indent=2, sort_keys=True)) + return 0 if demo["ok"] else 3 + parser.print_help() + return 2 + except StateDecodeError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/state_seed.py b/debug/risc-v-gsp-rm-vm/harness/state_seed.py new file mode 100644 index 0000000..e4d9a42 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/state_seed.py @@ -0,0 +1,990 @@ +#!/usr/bin/env python3 +"""RM-object state seeding: declarative state for rehost/valprobe runs. + +Why this exists +--------------- +``harness/rehost.py`` runs one firmware function with caller-supplied bytes and +``harness/valprobe.py`` runs one validation branch with crafted registers and +memory. Both hit the same wall on RM handlers: the function early-outs (or +faults on a NULL callback) before the branch of interest because the RM object +state it walks -- clients, channels, graphics contexts, memdesc/context-buffer +objects -- does not exist. This harness adds the missing layer: a declarative +**state seed** (``greeneye-state-seed/1``) that places symbolic objects at +chosen addresses, fills their fields and pointers, sets registers/entry, and +runs the function under a bounded budget while recording which seeded state the +firmware actually reads and where it first stops. + +Every run reports the ordered first-touch state reads and a ``gate``: the +address and the state the returning/faulting path checked. A clean early-out +with the gate identified is the success criterion; a fault is the first missing +state observed, never a product defect. + +Honesty rules: + +* Not a boot and not a model: only the seeded state exists. The report lists + the layout, every field it wrote, and every field marked ``unknown``. +* The object field map below is partial and cites its evidence per field; a + field without evidence is marked ``unknown`` and must not be invented. +* No timing/cycle claim (L0/L1 fidelity, docs/06). + +Offline only: never touches a GPU, kernel module, mdev, guest or network. +""" +from __future__ import annotations + +import argparse +import json +import os +import sys +import time +from typing import Any + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from greeneye.elf import load_elf # noqa: E402 +from greeneye.errors import GreenEyeError # noqa: E402 +from greeneye.sim_backend import launch_native # noqa: E402 +from sim.devices import load_profile, registry_from_profile # noqa: E402 +from sim.gsp_boot import _apply_boot_shim # noqa: E402 +from sim.libos import LibosSyscalls # noqa: E402 +from sim.machine import Fault, R_BP, R_BUDGET, R_ECALL, R_FAULT, R_WATCH # noqa: E402 + +from harness.rehost import ( # noqa: E402 + DEFAULT_IMAGE, + DEFAULT_PROFILE, + EBREAK, + RUN_CHUNK, + _parse_u64, + _registers, + _sha256_file, +) +from harness.valprobe import ( # noqa: E402 + BUFFER_ID, + INSTANCE_GETTER, + OUTCOMES, + PROMOTE_VALIDATION, + decode_memory_spec, + encode_return_stub, + reg_index, +) + +#: Schema tag of a state seed. +SEED_FORMAT = "greeneye-state-seed/1" +#: Schema tag of a single seeded run. +REPORT_FORMAT = "greeneye-state-seed-report/1" +#: Schema tag of the bundled demo. +DEMO_FORMAT = "greeneye-state-seed-demo/1" + +DEFAULT_BUDGET = 50_000 +DEFAULT_TIMEOUT = 20.0 +DEFAULT_ALLOC_BASE = 0x10000000 +DEFAULT_ALLOC_ALIGN = 16 + +#: Object types that carry a documented field map (evidence per field below). +_CHANNEL_FIELDS = { + 0x0D28: {"name": "instance_index", "width": 4, "status": "known", + "role": "graphics-instance index (u32); selects client+0x37c+idx", + "evidence": "promote-validation predicate (docs/47)"}, + 0x0D01: {"name": "flag_d01", "width": 1, "status": "unknown", + "role": "u8 flag on the failed-lookup path (0 takes the panic path)"}, + 0x0CD1: {"name": "flag_cd1", "width": 1, "status": "unknown", + "role": "u8 flag before the promote calls"}, + 0x0648: {"name": "callback_648", "width": 8, "status": "unknown", + "role": "indirect callback on the invalid-state path"}, + 0x0898: {"name": "callback_898", "width": 8, "status": "unknown", + "role": "indirect callback first reached after the +0x600 gate"}, + 0x03B8: {"name": "marker_3b8", "width": 8, "status": "unknown", + "role": "identity marker compared across objects"}, + 0x00EE: {"name": "obj_ee", "width": 8, "status": "unknown", + "role": "sub-object; +0xe8 read as the gfid/index"}, +} + +_GRAPHICS_INSTANCE_FIELDS = { + 0x0190: {"name": "memdesc_table", "width": 8, "status": "known", + "role": "pointer to the per-buffer memDesc slot table", + "evidence": "promote-validation predicate (docs/47)"}, + 0x0161: {"name": "flag_161", "width": 1, "status": "unknown", + "role": "u8 flag gating the 0x529bd44 call"}, + 0x03B8: {"name": "marker_3b8", "width": 8, "status": "unknown", + "role": "identity marker compared across objects"}, +} + +FIELD_MAP: dict[str, dict[int, dict]] = { + "client": { + 0x1BE0: {"name": "instance_table", "width": 8, "status": "known", + "role": "graphics-instance pointer array base (index 0x37c + idx)", + "evidence": "promote-validation predicate (docs/47)"}, + 0x1AA8: {"name": "obj_1aa8", "width": 8, "status": "unknown", + "role": "sub-object consumed by the construct path (+0x24c callback)"}, + 0x1CC8: {"name": "obj_1cc8", "width": 8, "status": "unknown", + "role": "sub-object consumed by the construct path (+0x150, +0x1d0)"}, + 0x1DD8: {"name": "engine_ctx", "width": 8, "status": "known", + "role": "engine-context pointer; 0 selects the no-engine path", + "evidence": "construct-path disassembly (docs/52)"}, + 0x1DE8: {"name": "subdevice", "width": 8, "status": "known", + "role": "sub-device object whose +0x600 callback is called", + "evidence": "construct-path disassembly (docs/52)"}, + 0x1E48: {"name": "obj_1e48", "width": 8, "status": "unknown", + "role": "sub-object consumed on the no-engine path (+0x110 callback)"}, + }, + "channel": _CHANNEL_FIELDS, + "graphics_context": _CHANNEL_FIELDS, + "graphics_instance": _GRAPHICS_INSTANCE_FIELDS, + "memdesc": { + 0x0000: {"name": "unknown_0", "width": 8, "status": "unknown", + "role": "slot value semantics outside the promote predicate"}, + 0x0008: {"name": "unknown_8", "width": 8, "status": "unknown", + "role": "unknown"}, + }, + "ctx_buffer": { + 0x0000: {"name": "unknown_0", "width": 8, "status": "unknown", + "role": "context-buffer (memdesc) object; field semantics unknown"}, + }, + "rm_object": {}, +} + +#: Object types accepted in a seed. ``rm_object`` carries no field map. +KNOWN_TYPES = tuple(FIELD_MAP) +MISSING_MARKER = "unknown" + + +class StateSeedError(GreenEyeError): + """A state seed, image or scenario is invalid.""" + + +def _require_u64(value: Any, what: str = "value") -> int: + try: + return _parse_u64(value) + except GreenEyeError as exc: + raise StateSeedError(f"{what}: {exc}") from exc + + +def _field_width(obj_type: str, offset: int) -> int: + spec = FIELD_MAP.get(obj_type, {}).get(offset) + return int(spec["width"]) if spec else 8 + + +def _field_spec(obj_type: str, offset: int) -> dict | None: + return FIELD_MAP.get(obj_type, {}).get(offset) + + +def _parse_offset(value: Any) -> int: + text = str(value).strip() + if text.startswith("+"): + text = text[1:] + try: + return int(text, 0) + except ValueError as exc: + raise StateSeedError(f"bad field offset {value!r}") from exc + + +def _encode_int(value: int, width: int) -> bytes: + if width not in (1, 2, 4, 8): + raise StateSeedError(f"unsupported field width {width}") + return (value & ((1 << (8 * width)) - 1)).to_bytes(width, "little") + + +def _is_symbol(value: Any) -> bool: + return isinstance(value, str) and value.strip().startswith("@") + + +def _symbol_expr(value: str) -> tuple[str, int]: + text = value.strip()[1:] + name, sep, offset = text.partition("+") + if not name: + raise StateSeedError(f"bad pointer expression {value!r}") + if sep: + try: + delta = int(offset, 0) + except ValueError as exc: + raise StateSeedError(f"bad pointer expression {value!r}") from exc + else: + delta = 0 + return name, delta + + +class _Symbols: + """Name -> address table for symbolic object/stub pointers.""" + + def __init__(self) -> None: + self._by_name: dict[str, int] = {} + + def add(self, name: str, addr: int, owner: str) -> None: + if name in self._by_name: + raise StateSeedError(f"duplicate symbol {name!r} ({owner})") + self._by_name[name] = addr + + def resolve(self, value: str, *, owner: str) -> int: + name, delta = _symbol_expr(value) + if name not in self._by_name: + raise StateSeedError(f"{owner}: unknown symbol {name!r}") + return (self._by_name[name] + delta) & 0xFFFF_FFFF_FFFF_FFFF + + +def load_seed(seed: dict) -> dict: + """Validate a ``greeneye-state-seed/1`` seed and compile an application plan.""" + if not isinstance(seed, dict): + raise StateSeedError(f"seed must be an object, got {type(seed).__name__}") + fmt = seed.get("format", SEED_FORMAT) + if fmt != SEED_FORMAT: + raise StateSeedError(f"unsupported seed format {fmt!r} (want {SEED_FORMAT})") + name = str(seed.get("name") or "state-seed") + seeded_registers = seed.get("registers") or {} + if not isinstance(seeded_registers, dict): + raise StateSeedError("registers must be an object") + entry_value = seed.get("entry", seeded_registers.get("pc")) + if entry_value is None: + raise StateSeedError("seed needs an entry address (or registers.pc)") + entry = _require_u64(entry_value) + + allocator = seed.get("allocator") or {} + cursor = _require_u64(allocator.get("base", DEFAULT_ALLOC_BASE)) + default_align = _require_u64(allocator.get("align", DEFAULT_ALLOC_ALIGN)) + if default_align <= 0 or default_align & (default_align - 1): + raise StateSeedError("allocator align must be a positive power of two") + + objects = seed.get("objects") or [] + if not isinstance(objects, list): + raise StateSeedError("objects must be a list") + layout: list[dict] = [] + symbols = _Symbols() + for obj in objects: + if not isinstance(obj, dict): + raise StateSeedError(f"object must be an object, got {obj!r}") + obj_type = str(obj.get("type") or "") + if obj_type not in KNOWN_TYPES: + raise StateSeedError(f"unknown object type {obj_type!r} (known: {', '.join(KNOWN_TYPES)})") + obj_name = str(obj.get("name") or "") + if not obj_name: + raise StateSeedError(f"{obj_type} object needs a name") + align = _require_u64(obj.get("align", default_align)) + if align <= 0 or align & (align - 1): + raise StateSeedError(f"{obj_name}: align must be a positive power of two") + fields = obj.get("fields") or {} + if not isinstance(fields, dict): + raise StateSeedError(f"{obj_name}: fields must be an object") + extent = 0 + for key in fields: + offset = _parse_offset(key) + extent = max(extent, offset + _field_width(obj_type, offset)) + size = _require_u64(obj.get("size", max(extent, default_align))) + if size <= 0: + raise StateSeedError(f"{obj_name}: size must be positive") + if "pa" in obj and "va" in obj: + raise StateSeedError(f"{obj_name}: use pa or va, not both") + explicit = obj.get("pa", obj.get("va")) + if explicit is not None: + pa = _require_u64(explicit) + else: + cursor = (cursor + align - 1) & ~(align - 1) + pa = cursor + cursor += (size + align - 1) & ~(align - 1) + record = {"name": obj_name, "type": obj_type, "pa": pa, "size": size, + "fields": {}, "unknown": []} + layout.append(record) + symbols.add(obj_name, pa, obj_name) + for key, value in fields.items(): + record["fields"][_parse_offset(key)] = value + + for i, first in enumerate(layout): + for second in layout[i + 1:]: + if first["pa"] < second["pa"] + second["size"] and second["pa"] < first["pa"] + first["size"]: + raise StateSeedError( + f"objects {first['name']!r} and {second['name']!r} overlap" + ) + + writes: list[dict] = [] + unknown_fields: list[dict] = [] + + stubs = seed.get("stubs") or [] + if not isinstance(stubs, list): + raise StateSeedError("stubs must be a list") + stub_records: list[dict] = [] + for stub in stubs: + if not isinstance(stub, dict) or "return" not in stub: + raise StateSeedError(f"stub needs 'return': {stub!r}") + ret = _require_u64(stub["return"]) + if "addr" in stub: + addr = _require_u64(stub["addr"]) + else: + cursor = (cursor + default_align - 1) & ~(default_align - 1) + addr = cursor + cursor += default_align + stub_name = stub.get("name") + if stub_name: + symbols.add(str(stub_name), addr, f"stub {stub_name}") + code = encode_return_stub(ret) + stub_records.append({"name": stub_name, "addr": addr, "return": ret, + "code_hex": code.hex()}) + writes.append({"pa": addr, "hex": code.hex(), "label": f"stub:{stub_name or hex(addr)}", + "kind": "stub"}) + + def resolve_field(owner: str, obj_type: str, offset: int, value: Any) -> tuple[int | None, int, str]: + width = _field_width(obj_type, offset) + if value == MISSING_MARKER or (isinstance(value, dict) and value.get("unknown")): + return None, width, MISSING_MARKER + if isinstance(value, dict): + if "unknown" in value and value["unknown"]: + return None, width, MISSING_MARKER + if "ptr" in value: + value = value["ptr"] + elif "value" in value: + raw = value["value"] + if raw == MISSING_MARKER: + return None, _require_u64(value.get("size", width)), MISSING_MARKER + width = _require_u64(value.get("size", width)) + return _require_u64(raw), width, "value" + else: + raise StateSeedError(f"{owner}+{offset:#x}: field dict needs value/ptr/unknown") + if _is_symbol(value): + return symbols.resolve(str(value), owner=f"{owner}+{offset:#x}"), width, "ptr" + return _parse_u64(value), width, "value" + + for record in layout: + pa = record["pa"] + for offset in sorted(record["fields"]): + value = record["fields"][offset] + resolved, width, kind = resolve_field(record["name"], record["type"], offset, value) + spec = _field_spec(record["type"], offset) + label = f"{record['name']}+{offset:#x}" + if resolved is None: + unknown_fields.append({ + "object": record["name"], "type": record["type"], "off": f"{offset:#x}", + "name": spec["name"] if spec else None, + "role": spec["role"] if spec else "not documented", + "reason": "seed marks this field unknown; left at zero", + }) + continue + writes.append({"pa": pa + offset, "hex": _encode_int(resolved, width).hex(), + "label": label, "kind": kind}) + + memory = seed.get("memory") or [] + if not isinstance(memory, list): + raise StateSeedError("memory must be a list") + for item in memory: + if not isinstance(item, dict): + raise StateSeedError(f"memory item must be an object: {item!r}") + normalized = dict(item) + if "pa" in normalized and "addr" in normalized: + raise StateSeedError(f"memory item: use pa or addr, not both: {item!r}") + if "pa" in normalized: + normalized["addr"] = normalized.pop("pa") + try: + addr, data = decode_memory_spec(normalized) + except GreenEyeError as exc: + raise StateSeedError(str(exc)) from exc + writes.append({"pa": addr, "hex": data.hex(), + "label": item.get("label") or f"mem+{addr:#x}", "kind": "memory"}) + + registers: dict[str, int] = {} + for reg_name, value in seeded_registers.items(): + if str(reg_name).strip().lower() == "pc": + continue + try: + reg_index(str(reg_name)) + except GreenEyeError as exc: + raise StateSeedError(str(exc)) from exc + if _is_symbol(value): + registers[str(reg_name)] = symbols.resolve(str(value), owner=f"register {reg_name}") + else: + registers[str(reg_name)] = _require_u64(value) + + stack = seed.get("stack") + outcomes = {_require_u64(a): str(label) + for a, label in (seed.get("outcomes") or {}).items()} + reads = [] + for item in seed.get("reads") or (): + if "pa" in item and "addr" not in item: + item = dict(item) + item["addr"] = item.pop("pa") + if "addr" not in item: + raise StateSeedError(f"read item needs addr/pa: {item!r}") + reads.append(item) + watch: list[dict] = [] + seen_labels: dict[str, int] = {} + + def _add_watch(addr: int, size: int, label: str) -> None: + count = seen_labels.get(label, 0) + 1 + seen_labels[label] = count + if count > 1: + label = f"{label}~{count}" + watch.append({"addr": addr, "size": size, "label": label}) + + for item in seed.get("watch") or (): + if "pa" in item and "addr" not in item: + item = dict(item) + item["addr"] = item.pop("pa") + if "addr" not in item: + raise StateSeedError(f"watch item needs addr/pa: {item!r}") + _add_watch(_require_u64(item["addr"]), _require_u64(item.get("size", 8)), + str(item.get("label") or f"{_require_u64(item['addr']):#x}")) + for record in layout: + for offset in record["fields"]: + _add_watch(record["pa"] + offset, _field_width(record["type"], offset), + f"{record['name']}+{offset:#x}") + + return { + "format": SEED_FORMAT, + "name": name, + "entry": entry, + "stack": _require_u64(stack) if stack is not None else None, + "registers": registers, + "writes": writes, + "layout": layout, + "unknown_fields": unknown_fields, + "stubs": stub_records, + "outcomes": outcomes, + "reads": reads, + "watch": watch, + "provenance": seed.get("provenance") or {}, + "notes": seed.get("notes") or [], + } + + +def apply_seed(machine, plan: dict) -> dict: + """Apply a compiled plan to a machine; return the same plan (chainable).""" + for item in plan["writes"]: + machine.write_mem(item["pa"], bytes.fromhex(item["hex"])) + for reg_name, value in plan["registers"].items(): + machine.x[reg_index(reg_name)] = value + return plan + + +def _read_after(machine, reads) -> list[dict]: + out = [] + for item in reads or (): + addr = _require_u64(item["addr"]) + size = _require_u64(item.get("size", 8)) + data = bytes(machine.read_mem(addr, size)) + record = {"addr": f"{addr:#x}", "size": size, "hex": data.hex()} + if item.get("label"): + record["label"] = item["label"] + record["u64"] = int.from_bytes(data.ljust(8, b"\x00"), "little") + out.append(record) + return out + + +def _gate_from_run(status: str, stop_pc: int, fault: dict | None, + state_reads: list[dict]) -> dict | None: + if not state_reads: + return None + last = state_reads[-1] + if status == "fault" and stop_pc == 0 and last.get("value") == 0: + return { + "kind": "null_indirect_call", + "pc": last["pc"], + "addr": last["addr"], + "label": last["label"], + "value": 0, + "detail": ( + f"indirect call target {last['label']} is NULL; execution faulted at " + f"pc 0x0 (fault={fault})" + ), + } + if status == "fault": + return { + "kind": "fault", + "pc": f"{stop_pc:#x}", + "addr": last["addr"], + "label": last["label"], + "value": last.get("value"), + "detail": f"fault before the next seeded-state read (fault={fault})", + } + if status == "outcome": + return None + return { + "kind": f"{status}_before_gate", + "pc": f"{stop_pc:#x}", + "addr": last["addr"], + "label": last["label"], + "value": last.get("value"), + "detail": f"run stopped as {status}; last seeded state read shown", + } + + +def run_seed( + seed: dict, + *, + image: str = DEFAULT_IMAGE, + profile_path: str | None = DEFAULT_PROFILE, + entry: int | str | None = None, + out: str | None = None, + record: bool | None = None, + budget: int = DEFAULT_BUDGET, + timeout: float = DEFAULT_TIMEOUT, + name: str = "state-seed", +) -> dict: + """Load a seed into the native machine, run it bounded, return a report.""" + if not os.path.isfile(image): + raise StateSeedError(f"image not found: {image}") + if budget <= 0: + raise StateSeedError(f"budget must be positive, got {budget}") + image_obj = load_elf(image) + if not image_obj.is_riscv: + raise StateSeedError(f"{image}: not a RISC-V ELF image") + plan = load_seed(seed) + entry_addr = _require_u64(entry) if entry is not None else plan["entry"] + + profile = load_profile(profile_path) if profile_path else None + load = (profile or {}).get("load", {}) + ram_base = _require_u64(load.get("ram_base", 0)) + memory_mb = int(load.get("memory_mb", 256)) + ram_top = ram_base + memory_mb * 1024 * 1024 + + if record is None: + record = bool(out) + session = launch_native( + kernel=image, + memory_mb=memory_mb, + ram_base=ram_base, + record=record, + name=f"{name}-{plan['name']}"[:80], + device_map=None, + unknown_mmio="lenient", + bias=0, + ) + machine = session.target.machine + try: + if profile is not None: + machine.devices = registry_from_profile(profile, machine=machine) + machine.translator = getattr(machine.devices, "dma", None) + machine.syscall_handler = LibosSyscalls().handle + shim = dict(profile.get("boot_shim", {})) + shim["_image_path"] = image + _apply_boot_shim(machine, shim, image_obj, {"enabled": True, "assumptions": []}) + + sentinel = (ram_top - 0x100) & ~0xF + default_sp = (ram_top - 0x8000) & ~0xF + machine.write_mem(sentinel, EBREAK.to_bytes(4, "little")) + + apply_seed(machine, plan) + machine.x[1] = sentinel + if machine.x[2] == 0: + machine.x[2] = plan["stack"] if plan["stack"] is not None else default_sp + machine.pc = entry_addr + + breakpoints = set(plan["outcomes"]) | {sentinel} + watchers = [] + for item in plan["watch"]: + addr = _require_u64(item["addr"]) + size = _require_u64(item.get("size", 8)) + label = str(item.get("label") or f"{addr:#x}") + watchers.append((addr, addr + size, label, 3)) + + entered = {f"x{i}": machine.x[i] for i in range(32) if i and machine.x[i]} + entered["pc"] = entry_addr + state_reads: list[dict] = [] + started = time.monotonic() + status = "budget" + stop_pc = entry_addr + fault: dict | None = None + outcome: str | None = None + remaining = budget + while remaining > 0: + if time.monotonic() - started > timeout: + status = "timeout" + break + chunk = min(RUN_CHUNK, remaining) + reason, stop_pc, a, b = machine.run( + chunk, breakpoints=breakpoints, watchers=tuple(watchers) + ) + if reason == R_WATCH: + width = 8 + for watcher in watchers: + if watcher[2] == b: + width = watcher[1] - watcher[0] + break + try: + value = int.from_bytes(bytes(machine.read_mem(a, width)), "little") + except (GreenEyeError, Fault): + value = None + state_reads.append({"pc": f"{stop_pc:#x}", "addr": f"{a:#x}", + "label": b, "value": value, "width": width}) + watchers = [watcher for watcher in watchers if watcher[2] != b] + continue + remaining -= chunk + if reason == R_BP: + if stop_pc == sentinel: + status = "return" + break + outcome = plan["outcomes"].get(stop_pc, "unlabelled") + status = "outcome" + break + if reason == R_FAULT: + status = "fault" + fault = {"cause": a, "tval": f"{b:#x}"} + break + if reason == R_ECALL: + handled, result = (machine.syscall_handler(machine, stop_pc) + if machine.syscall_handler else (False, None)) + if not handled: + status = "ecall" + break + if result is not None: + machine.x[10] = result & 0xFFFF_FFFF_FFFF_FFFF + continue + if reason == R_BUDGET: + status = "budget" + + gate = _gate_from_run(status, stop_pc, fault, state_reads) + report: dict[str, Any] = { + "format": REPORT_FORMAT, + "seed": plan["name"], + "seed_format": plan["format"], + "seed_provenance": plan["provenance"], + "image": os.path.abspath(image), + "image_sha256": _sha256_file(image), + "profile": os.path.basename(profile_path) if profile_path else None, + "entry": f"{entry_addr:#x}", + "status": status, + "outcome": outcome, + "stop_pc": f"{stop_pc:#x}", + "fault": fault, + "retval": machine.x[10], + "instructions": machine.insns, + "wall_seconds": round(time.monotonic() - started, 4), + "objects": [ + {"name": item["name"], "type": item["type"], "pa": f"{item['pa']:#x}", + "size": item["size"]} + for item in plan["layout"] + ], + "unknown_fields": plan["unknown_fields"], + "stubs": plan["stubs"], + "registers_entered": {k: f"{v:#x}" if isinstance(v, int) else v + for k, v in entered.items()}, + "registers_exited": {k: f"{v:#x}" if isinstance(v, int) else v + for k, v in _registers(machine).items()}, + "state_reads": state_reads, + "gate": gate, + "reads": _read_after(machine, plan["reads"]), + "limits": [ + "Seeded run, not a boot: only the supplied objects/memory exist.", + "A fault is the first missing state observed, not a product defect.", + "No timing/cycle claim (L0/L1 fidelity, docs/06).", + "A field marked unknown was left at zero and reported.", + ], + } + if getattr(session.recorder, "dir", None) is not None: + session.note("state-seed", seed=plan["name"], status=status, + outcome=outcome, stop_pc=f"{stop_pc:#x}", + gate=(gate or {}).get("kind")) + report["session"] = session.recorder.dir + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "state-seed-report.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(report, fh, indent=2, sort_keys=True) + fh.write("\n") + report["report_json"] = os.path.abspath(path) + return report + finally: + session.close() + + +# --------------------------------------------------------------------------- +# Bundled seeds. +# +# 1. fecs_seed: the promote-validation predicate seeded through this loader +# (the mechanism proof -- it works without RM objects). It reproduces the +# valprobe variant memory/registers/stub exactly, so a slot non-NULL seed +# must refuse 0x1f and a slot NULL seed must take the commit path. +# 2. target_seed: the construct target 0x52a70ec with a minimal crafted +# client/channel/graphics-context/memdesc set. The first gate is reported +# (see docs/52); with the +0x600 callback stubbed it advances to the next. +# --------------------------------------------------------------------------- + + +def _fecs_seed(*, slot: int, label: str) -> dict: + scratch = 0x10000000 + layout = { + "scratch": scratch, + "sp": scratch + 0x4000, + "s0": scratch + 0x5000, + "pgpu": scratch + 0x1000, + "getter_out": scratch + 0x1800, + "instance": scratch + 0x3800, + "entry": scratch + 0x6000, + } + slot_addr = layout["instance"] + 0x300 + 6 * 8 + memory = [ + {"pa": f"{layout['entry'] + 0x30:#x}", "u64": 0x1F4150000}, + {"pa": f"{layout['entry'] + 0x38:#x}", "u64": 0x204608000}, + {"pa": f"{layout['entry'] + 0x40:#x}", "u64": 0x10000}, + {"pa": f"{layout['entry'] + 0x48:#x}", "u32": 4}, + {"pa": f"{layout['entry'] + 0x4c:#x}", "u16": BUFFER_ID["FECS_EVENT"]}, + {"pa": f"{layout['entry'] + 0x4e:#x}", "u8": 1}, + {"pa": f"{layout['entry'] + 0x4f:#x}", "u8": 0}, + {"pa": f"{layout['pgpu'] + 0x1be0:#x}", "u64": layout["instance"]}, + {"pa": f"{layout['instance'] + 0x190:#x}", "u64": layout["instance"] + 0x300}, + {"pa": f"{slot_addr:#x}", "u64": slot, "label": "fecs-slot"}, + ] + return { + "format": SEED_FORMAT, + "name": label, + "entry": f"{PROMOTE_VALIDATION:#x}", + "outcomes": dict(OUTCOMES), + "registers": { + "s0": f"{layout['s0']:#x}", + "s2": f"{layout['pgpu']:#x}", + "s5": f"{layout['entry']:#x}", + "sp": f"{layout['sp']:#x}", + "a3": BUFFER_ID["FECS_EVENT"], + "t1": 0xC, + }, + "stubs": [{"addr": f"{INSTANCE_GETTER:#x}", "return": f"{layout['getter_out']:#x}", + "name": "instance_getter"}], + "memory": memory, + "watch": [{"pa": f"{slot_addr:#x}", "size": 8, "label": "fecs-slot"}], + "reads": [{"pa": f"{slot_addr:#x}", "size": 8, "label": "fecs-slot-value"}], + "provenance": { + "source": "promote-validation predicate state (docs/47), rebuilt as a seed", + "slot": slot, + }, + "notes": [ + "Mechanism proof: a path that needs no RM objects, seeded end to end.", + ], + } + + +def fecs_seed(*, slot: int = 1) -> dict: + """Build the FECS promote-validation seed (slot 1 refuses, slot 0 accepts).""" + label = "fecs-promote-slot-nonnull" if slot else "fecs-promote-slot-null" + return _fecs_seed(slot=slot, label=label) + + +def target_seed(*, advance: bool = False) -> dict: + """Build the minimal client/channel/context seed for the 0x52a70ec target. + + ``advance=False`` leaves the +0x600 callback unknown (the first gate). + ``advance=True`` points it at a return-0 stub so the next gate is reached. + """ + callback = "@engine_stub" if advance else MISSING_MARKER + name = "construct-gate-advanced" if advance else "construct-gate-minimal" + return { + "format": SEED_FORMAT, + "name": name, + "entry": "0x52a70ec", + "allocator": {"base": "0x10010000", "align": "0x1000"}, + "stubs": [{"name": "engine_stub", "return": 0}] if advance else [], + "registers": { + "a0": "@client", + "a1": "@channel", + "a2": "0x1003f000", + }, + "stack": "0x1003e000", + "objects": [ + {"name": "client", "type": "client", "size": "0x2000", "fields": { + "0x1be0": "@graphics_instance", + "0x1aa8": "@obj_1aa8", + "0x1cc8": "@obj_1cc8", + "0x1dd8": {"value": 0, "size": 8}, + "0x1de8": "@engine_node", + "0x1e48": "@obj_1e48", + }}, + {"name": "channel", "type": "channel", "size": "0x1000", "fields": { + "0xd28": {"value": 0, "size": 4}, + "0xd01": {"value": 1, "size": 1}, + "0xcd1": {"value": 0, "size": 1}, + "0x898": MISSING_MARKER, + }}, + {"name": "graphics_instance", "type": "graphics_instance", "size": "0x1000", + "fields": { + "0x190": "@memdesc_table", + "0x161": {"value": 0, "size": 1}, + }}, + {"name": "engine_node", "type": "rm_object", "size": "0x1000", "fields": { + "0x600": callback, + }}, + {"name": "obj_1cc8", "type": "rm_object", "size": "0x400", "fields": { + "0x150": MISSING_MARKER, + "0x1d0": {"value": 0x20, "size": 4}, + }}, + {"name": "obj_1aa8", "type": "rm_object", "size": "0x400", "fields": { + "0x24c": MISSING_MARKER, + }}, + {"name": "obj_1e48", "type": "rm_object", "size": "0x400", "fields": { + "0x110": MISSING_MARKER, + }}, + {"name": "memdesc_table", "type": "rm_object", "size": "0x100", "fields": {}}, + {"name": "fecs_memdesc", "type": "memdesc", "size": "0x100", "fields": { + "0x0": MISSING_MARKER, + "0x8": MISSING_MARKER, + }}, + ], + "reads": [{"pa": "0x1003f000", "size": 8, "label": "a2-out-scratch"}], + "provenance": { + "source": "construct-path disassembly (docs/52, this wave)", + "callback": callback, + }, + "notes": [ + "Only the fields the known firmware path reads are seeded; a field marked", + "unknown is left zero and reported so the gate is not mistaken for a seed bug.", + ], + } + + +def run_demo(out: str | None = None) -> dict: + """Prove the seeder on the FECS predicate, then attempt the construct target.""" + if not os.path.isfile(DEFAULT_IMAGE): + raise StateSeedError(f"firmware image absent: {DEFAULT_IMAGE}") + profile = DEFAULT_PROFILE + + fecs_nonnull = run_seed(fecs_seed(slot=1), profile_path=profile, record=False, + name="state-seed-fecs") + fecs_nonnull_again = run_seed(fecs_seed(slot=1), profile_path=profile, record=False, + name="state-seed-fecs") + fecs_null = run_seed(fecs_seed(slot=0), profile_path=profile, record=False, + name="state-seed-fecs") + target = run_seed(target_seed(advance=False), profile_path=profile, record=False, + name="state-seed-target") + advanced = run_seed(target_seed(advance=True), profile_path=profile, record=False, + name="state-seed-target") + + verdicts = [ + {"label": "FECS init slot non-NULL", "outcome": fecs_nonnull["outcome"], + "expect": "refuse-fecs-slot-exists-0x1f", + "pass": fecs_nonnull["outcome"] == "refuse-fecs-slot-exists-0x1f"}, + {"label": "FECS init slot NULL", "outcome": fecs_null["outcome"], + "expect": "accept-commit-path", + "pass": fecs_null["outcome"] == "accept-commit-path"}, + ] + deterministic = ( + fecs_nonnull["outcome"] == fecs_nonnull_again["outcome"] + and fecs_nonnull["gate"] == fecs_nonnull_again["gate"] + ) + result = { + "format": DEMO_FORMAT, + "ok": all(row["pass"] for row in verdicts) and deterministic, + "deterministic": deterministic, + "mechanism": { + "seed": "fecs-promote", + "verdicts": verdicts, + "wall_seconds": round(fecs_nonnull["wall_seconds"] + fecs_null["wall_seconds"], 4), + }, + "target": { + "entry": "0x52a70ec", + "status": target["status"], + "stop_pc": target["stop_pc"], + "gate": target["gate"], + "state_reads": target["state_reads"], + "unknown_fields": target["unknown_fields"], + }, + "target_advanced": { + "entry": "0x52a70ec", + "status": advanced["status"], + "stop_pc": advanced["stop_pc"], + "gate": advanced["gate"], + "state_reads": advanced["state_reads"], + }, + } + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "state-seed-demo.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(result, fh, indent=2, sort_keys=True) + fh.write("\n") + result["artifacts"] = {"demo_json": os.path.abspath(path)} + return result + + +def _load_seed_file(path: str) -> dict: + if not os.path.isfile(path): + raise StateSeedError(f"seed not found: {path}") + with open(path, "r", encoding="utf-8") as fh: + return json.load(fh) + + +def _emit_seed(kind: str) -> dict: + if kind == "fecs": + return fecs_seed() + if kind == "target-minimal": + return target_seed(advance=False) + if kind == "target-advanced": + return target_seed(advance=True) + raise StateSeedError(f"unknown seed kind {kind!r}") + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="state_seed.py", + description=( + "Seed declarative RM object state into a bounded firmware run " + "(offline; never touches hardware)." + ), + ) + sub = parser.add_subparsers(dest="command") + run_cmd = sub.add_parser("run", help="run one seed file") + run_cmd.add_argument("--seed", required=True, help="greeneye-state-seed/1 JSON") + run_cmd.add_argument("--entry", default=None, help="override the seed entry (hex)") + run_cmd.add_argument("--image", default=DEFAULT_IMAGE, help="firmware ELF") + run_cmd.add_argument("--profile", default=DEFAULT_PROFILE, help="device profile JSON") + run_cmd.add_argument("--budget", type=int, default=DEFAULT_BUDGET) + run_cmd.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT) + run_cmd.add_argument("--out", help="output directory (session + report JSON)") + run_cmd.add_argument("--no-record", action="store_true", help="skip session recording") + validate = sub.add_parser("validate", help="validate a seed and print its layout") + validate.add_argument("--seed", required=True) + emit = sub.add_parser("emit", help="write a bundled seed") + emit.add_argument("--kind", required=True, + choices=["fecs", "target-minimal", "target-advanced"]) + emit.add_argument("--out", required=True) + demo = sub.add_parser("demo", help="FECS mechanism proof + construct-gate attempt") + demo.add_argument("--out", help="output directory") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "run": + seed = _load_seed_file(args.seed) + report = run_seed(seed, image=args.image, profile_path=args.profile or None, + entry=args.entry, budget=args.budget, timeout=args.timeout, + out=args.out, record=False if args.no_record else None) + summary = {k: report[k] for k in + ("format", "seed", "entry", "status", "outcome", "stop_pc", + "fault", "gate", "instructions", "wall_seconds", + "report_json", "session") if k in report} + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 + if args.command == "validate": + seed = _load_seed_file(args.seed) + plan = load_seed(seed) + print(json.dumps({ + "format": plan["format"], + "name": plan["name"], + "entry": f"{plan['entry']:#x}", + "objects": [{"name": item["name"], "type": item["type"], + "pa": f"{item['pa']:#x}", "size": item["size"]} + for item in plan["layout"]], + "unknown_fields": plan["unknown_fields"], + "writes": len(plan["writes"]), + "stubs": plan["stubs"], + }, indent=2, sort_keys=True)) + return 0 + if args.command == "emit": + seed = _emit_seed(args.kind) + os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True) + with open(args.out, "w", encoding="utf-8") as fh: + json.dump(seed, fh, indent=2, sort_keys=True) + fh.write("\n") + print(os.path.abspath(args.out)) + return 0 + if args.command == "demo": + result = run_demo(args.out) + print(json.dumps({k: result[k] for k in + ("format", "ok", "deterministic", "mechanism", + "target", "target_advanced", "artifacts") + if k in result}, indent=2, sort_keys=True)) + return 0 if result["ok"] else 3 + parser.print_help() + return 2 + except StateSeedError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/harness/valprobe.py b/debug/risc-v-gsp-rm-vm/harness/valprobe.py new file mode 100644 index 0000000..55952b9 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/harness/valprobe.py @@ -0,0 +1,651 @@ +#!/usr/bin/env python3 +"""Validation-predicate probe: run a firmware predicate branch with crafted state. + +Why this exists +--------------- +The live GR campaign keeps burning supervised hardware cycles to ask the same +question class: *which control/entry shapes does the 570.144 GSP-RM firmware +accept or refuse, and why?* The answer lives in firmware branches that are +buried behind host traffic (B4/B5) and RM object state, so a plain rehost of +the full handler early-outs before the predicate. This harness instead runs +one **predicate branch** of the real image with the minimum crafted state it +needs: registers, memory, and (where a branch calls out to a lock/table walk +that is not the predicate) one recorded inline stub. Each run stops on a +labelled outcome address, so the verdict is the branch the firmware took, not +an inference. + +It is deliberately not a boot and not a model: the report carries the exact +entry, seeded state, stubs and outcome map, and every limitation in `limits`. +Fidelity is L0/L1 (docs/06); no timing claim. + +Bundled target: ``sweep --target fecs-promote`` reproduces the live V6 FECS +matrix (`PM/PATCH/PRIV/UNR accepted, FECS init refused 0x1f, FECS VA-only +accepted`) and evaluates the untested shapes offline, including the +sysmem-aperture FECS-init variant the handoff proposed paying a hardware cycle +for. + +Offline only: never touches a GPU, kernel module, mdev, guest or network. +""" +from __future__ import annotations + +import argparse +import json +import os +import struct +import sys +import time +from typing import Any + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from greeneye.elf import load_elf # noqa: E402 +from greeneye.errors import GreenEyeError # noqa: E402 +from greeneye.sim_backend import launch_native # noqa: E402 +from sim.devices import load_profile, registry_from_profile # noqa: E402 +from sim.gsp_boot import _apply_boot_shim, _run_with_milestones, _rung_from_run # noqa: E402 +from sim.libos import LibosSyscalls # noqa: E402 +from sim.machine import R_BP, R_BUDGET, R_ECALL, R_FAULT # noqa: E402 + +from harness.rehost import ( # noqa: E402 + DEFAULT_IMAGE, + DEFAULT_PROFILE, + RUN_CHUNK, + _parse_u64, + _sha256_file, + parse_hex_bytes, +) + +#: Schema tag of a single probe run. +PROBE_FORMAT = "greeneye-valprobe/1" +#: Schema tag of a probe spec. +SPEC_FORMAT = "greeneye-valprobe-spec/1" +#: Schema tag of a matrix run (a spec with variants). +MATRIX_FORMAT = "greeneye-valprobe-matrix/1" +#: Schema tag of the bundled demo. +DEMO_FORMAT = "greeneye-valprobe-demo/1" + +DEFAULT_BUDGET = 50_000 +DEFAULT_TIMEOUT = 20.0 +BOOT_INSTRUCTIONS = 2_000_000 +EBREAK = 0x00100073 + +REGISTERS = { + "zero": 0, "ra": 1, "sp": 2, "gp": 3, "tp": 4, + "t0": 5, "t1": 6, "t2": 7, "s0": 8, "fp": 8, "s1": 9, + "a0": 10, "a1": 11, "a2": 12, "a3": 13, "a4": 14, "a5": 15, + "a6": 16, "a7": 17, "s2": 18, "s3": 19, "s4": 20, "s5": 21, + "s6": 22, "s7": 23, "s8": 24, "s9": 25, "s10": 26, "s11": 27, + "t3": 28, "t4": 29, "t5": 30, "t6": 31, +} +for _i in range(32): + REGISTERS.setdefault(f"x{_i}", _i) + +RUNG_RANK = {"B0": 0, "B1": 1, "B2": 2, "B3": 3} + + +class ValprobeError(GreenEyeError): + """A probe spec, image or scenario is invalid.""" + + +def reg_index(name: str) -> int: + key = str(name).strip().lower() + if key not in REGISTERS: + raise ValprobeError(f"unknown register {name!r}") + return REGISTERS[key] + + +def encode_return_stub(value: int) -> bytes: + """Encode ``lui a0,hi; addi a0,a0,lo; jalr zero,0(ra)`` (RV64IMAC).""" + value &= 0xFFFF_FFFF_FFFF_FFFF + hi = (value + 0x800) >> 12 + lo = value - (hi << 12) + lui = ((hi & 0xF_FFFF) << 12) | (10 << 7) | 0x37 + addi = ((lo & 0xFFF) << 20) | (10 << 15) | (10 << 7) | 0x13 + ret = 0x0000_8067 + return struct.pack("<III", lui, addi, ret) + + +def decode_memory_spec(item: dict) -> tuple[int, bytes]: + """Parse one memory item: ``{addr, hex|u64|u32|u8}``.""" + if "addr" not in item: + raise ValprobeError(f"memory item needs addr: {item!r}") + addr = _parse_u64(item["addr"]) + for key, fmt, size in (("hex", None, None), ("u64", "<Q", 8), ("u32", "<I", 4), + ("u16", "<H", 2), ("u8", "<B", 1)): + if key in item: + if key == "hex": + return addr, parse_hex_bytes(str(item["hex"])) + return addr, struct.pack(fmt, _parse_u64(item[key]) & ((1 << (8 * size)) - 1)) + raise ValprobeError(f"memory item needs one of hex/u64/u32/u16/u8: {item!r}") + + +def _load_spec(path: str) -> dict: + if not os.path.isfile(path): + raise ValprobeError(f"spec not found: {path}") + with open(path, "r", encoding="utf-8") as fh: + spec = json.load(fh) + fmt = spec.get("format", SPEC_FORMAT) + if fmt != SPEC_FORMAT: + raise ValprobeError(f"unsupported spec format {fmt!r} (want {SPEC_FORMAT})") + if not spec.get("variants"): + raise ValprobeError("spec has no variants") + return spec + + +def _apply_state(machine, *, registers: dict, memory, stubs, default_sp: int) -> list[dict]: + """Write stubs, memory and registers; return the applied stub notes.""" + stub_notes = [] + for stub in stubs or (): + addr = _parse_u64(stub["addr"]) + if "return" not in stub: + raise ValprobeError(f"stub needs return: {stub!r}") + code = encode_return_stub(_parse_u64(stub["return"])) + machine.write_mem(addr, code) + stub_notes.append({"addr": f"{addr:#x}", "return": f"{_parse_u64(stub['return']):#x}", + "code_hex": code.hex()}) + for item in memory or (): + addr, data = decode_memory_spec(item) + machine.write_mem(addr, data) + for name, value in (registers or {}).items(): + machine.x[reg_index(name)] = _parse_u64(value) + if machine.x[2] == 0: + machine.x[2] = default_sp + return stub_notes + + +def _read_outcomes(machine, reads) -> list[dict]: + out = [] + for item in reads or (): + addr = _parse_u64(item["addr"]) + size = int(item.get("size", 8)) + data = bytes(machine.read_mem(addr, size)) + record = {"addr": f"{addr:#x}", "size": size, "hex": data.hex()} + if item.get("label"): + record["label"] = item["label"] + if size == 8: + record["u64"] = int.from_bytes(data, "little") + elif size == 4: + record["u32"] = int.from_bytes(data, "little") + out.append(record) + return out + + +def run_probe( + *, + image: str = DEFAULT_IMAGE, + profile_path: str | None = DEFAULT_PROFILE, + spec: dict, + variant: dict, + out: str | None = None, + record: bool | None = None, + name: str = "valprobe", +) -> dict: + """Run one variant of a probe spec in a fresh machine; return its report.""" + if not os.path.isfile(image): + raise ValprobeError(f"image not found: {image}") + image_obj = load_elf(image) + if not image_obj.is_riscv: + raise ValprobeError(f"{image}: not a RISC-V ELF image") + + profile = load_profile(profile_path) if profile_path else None + load = (profile or {}).get("load", {}) + ram_base = _parse_u64(load.get("ram_base", 0)) + memory_mb = int(load.get("memory_mb", 256)) + ram_top = ram_base + memory_mb * 1024 * 1024 + + entry = _parse_u64(variant.get("entry", spec.get("target"))) + outcomes = {_parse_u64(a): label for a, label in + (variant.get("outcomes") or spec.get("outcomes") or {}).items()} + if not outcomes: + raise ValprobeError("probe needs at least one outcome address") + + budget = int(variant.get("budget", spec.get("budget", DEFAULT_BUDGET))) + timeout = float(variant.get("timeout", spec.get("timeout", DEFAULT_TIMEOUT))) + label = variant.get("label") or f"variant-{entry:#x}" + boot_to = spec.get("boot_to") or variant.get("boot_to") + + if record is None: + record = bool(out) + session = launch_native( + kernel=image, + memory_mb=memory_mb, + ram_base=ram_base, + record=record, + name=f"{name}-{label}"[:80], + device_map=None, + unknown_mmio="lenient", + bias=0, + ) + machine = session.target.machine + shim_meta: dict[str, Any] = {"enabled": False, "assumptions": []} + libos = LibosSyscalls() + boot: dict | None = None + try: + if profile is not None: + machine.devices = registry_from_profile(profile, machine=machine) + machine.translator = getattr(machine.devices, "dma", None) + machine.syscall_handler = libos.handle + shim = dict(profile.get("boot_shim", {})) + shim["_image_path"] = image + _apply_boot_shim(machine, shim, image_obj, shim_meta) + shim_meta["enabled"] = True + + if boot_to: + milestones: list[str] = [] + reason, pc, _a, _b = _run_with_milestones(machine, BOOT_INSTRUCTIONS, milestones) + rung = _rung_from_run(machine, shim_meta, milestones, libos) + boot = { + "requested": boot_to, + "rung": rung, + "instructions": machine.insns, + "stop_pc": f"{pc:#x}", + "milestones": milestones, + "reached": RUNG_RANK.get(rung, -1) >= RUNG_RANK.get(str(boot_to), 99), + } + + sentinel = (ram_top - 0x100) & ~0xF + default_sp = (ram_top - 0x8000) & ~0xF + machine.write_mem(sentinel, EBREAK.to_bytes(4, "little")) + breakpoints = set(outcomes) + breakpoints.add(sentinel) + + registers = dict(spec.get("registers") or {}) + registers.update(variant.get("registers") or {}) + registers.setdefault("ra", sentinel) + memory = list(spec.get("memory") or ()) + list(variant.get("memory") or ()) + stubs = list(spec.get("stubs") or ()) + list(variant.get("stubs") or ()) + stubs_applied = _apply_state( + machine, registers=registers, memory=memory, stubs=stubs, default_sp=default_sp + ) + machine.pc = entry + + entered = {f"x{i}": machine.x[i] for i in range(32) if i and machine.x[i]} + entered["pc"] = entry + started = time.monotonic() + status = "budget" + stop_pc = entry + fault: dict | None = None + outcome: str | None = None + remaining = budget + while remaining > 0: + if time.monotonic() - started > timeout: + status = "timeout" + break + chunk = min(RUN_CHUNK, remaining) + reason, stop_pc, a, b = machine.run(chunk, breakpoints=breakpoints) + remaining -= chunk + if reason == R_BP: + if stop_pc == sentinel: + status = "return" + break + outcome = outcomes.get(stop_pc, "unlabelled") + status = "outcome" + break + if reason == R_FAULT: + status = "fault" + fault = {"cause": a, "tval": f"{b:#x}"} + break + if reason == R_ECALL: + handled, result = (machine.syscall_handler(machine, stop_pc) + if machine.syscall_handler else (False, None)) + if not handled: + status = "ecall" + break + if result is not None: + machine.x[10] = result & 0xFFFF_FFFF_FFFF_FFFF + continue + if reason == R_BUDGET: + status = "budget" + + report: dict[str, Any] = { + "format": PROBE_FORMAT, + "probe": spec.get("name"), + "label": label, + "image": os.path.abspath(image), + "image_sha256": _sha256_file(image), + "profile": os.path.basename(profile_path) if profile_path else None, + "entry": f"{entry:#x}", + "status": status, + "outcome": outcome, + "expect": variant.get("expect"), + "pass": (None if variant.get("expect") is None else outcome == variant.get("expect")), + "stop_pc": f"{stop_pc:#x}", + "fault": fault, + "retval": machine.x[10], + "instructions": machine.insns, + "wall_seconds": round(time.monotonic() - started, 4), + "boot": boot, + "stubs": stubs_applied, + "registers_entered": {k: f"{v:#x}" if isinstance(v, int) else v + for k, v in entered.items()}, + "reads": _read_outcomes(machine, variant.get("reads") or spec.get("reads")), + "limits": [ + "Predicate-branch probe, not a boot: only the seeded state exists.", + "Stubs (if any) replace an out-of-predicate dependency; they are listed.", + "No timing/cycle claim (L0/L1 fidelity, docs/06).", + "A fault is the first missing state, not a product defect.", + ], + } + if getattr(session.recorder, "dir", None) is not None: + session.note("valprobe", label=label, outcome=outcome, status=status) + report["session"] = session.recorder.dir + return report + finally: + session.close() + + +def run_spec( + *, + spec: dict, + image: str | None = None, + profile_path: str | None = None, + out: str | None = None, + record: bool | None = None, + name: str | None = None, +) -> dict: + """Run every variant of a spec; return a matrix report.""" + image = image or spec.get("image") or DEFAULT_IMAGE + profile_path = profile_path or spec.get("profile") or DEFAULT_PROFILE + started = time.monotonic() + rows = [] + for variant in spec["variants"]: + rows.append(run_probe( + image=image, profile_path=profile_path, spec=spec, variant=variant, + out=out, record=record, name=name or (spec.get("name") or "valprobe"), + )) + matrix: dict[str, Any] = { + "format": MATRIX_FORMAT, + "spec": spec.get("name"), + "spec_format": spec.get("format"), + "image": os.path.abspath(image), + "image_sha256": _sha256_file(image), + "target": spec.get("target"), + "boot_to": spec.get("boot_to"), + "variants": rows, + "passed": sum(1 for r in rows if r["pass"] is True), + "failed": sum(1 for r in rows if r["pass"] is False), + "evaluated": sum(1 for r in rows if r["pass"] is not None), + "wall_seconds": round(time.monotonic() - started, 4), + } + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "valprobe-matrix.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(matrix, fh, indent=2, sort_keys=True) + fh.write("\n") + matrix["report_json"] = os.path.abspath(path) + return matrix + + +# --------------------------------------------------------------------------- +# Bundled target: the GR PROMOTE_CTX FECS predicate (570.144 firmware). +# +# Firmware facts used (all verified by disassembly of the shipped image): +# * per-buffer validation branch point: 0x5290dd4 (in FUN_ram_05290a48) +# * bInitialize=1 and bufferId==9 -> FECS branch at 0x5290e3c +# * FECS branch: instance = pGpu[(idx+0x37c)], idx = *(u32*)(getter+0xD28); +# table = *(u64*)(instance+0x190); slot = table[mapped(bufferId)*8]; +# refuse (0x1f) iff slot != 0 -- regardless of physAttr/size/VA +# * bufferId -> slot index via the firmware's own 0x531b944 jump table +# (FECS => 6) +# * bInitialize=0: bNonmapped!=0 => 0x1f, else accept +# +# The instance-index getter (0x524a5b8, a lock/table walk outside the +# predicate) is stubbed with a recorded constant return so the branch can run +# with crafted state; the predicate itself, the mapping table call and the +# slot read are the firmware's own code. +# --------------------------------------------------------------------------- + +PROMOTE_VALIDATION = 0x5290DD4 +FECS_INIT_BRANCH = 0x5290E3C +INSTANCE_GETTER = 0x524A5B8 +OUTCOMES = { + "0x5290cd0": "accept-commit-path", + "0x5290ce4": "accept-skip-path", + "0x5290df8": "refuse-invalid-argument-0x1f", + "0x5290ebc": "refuse-fecs-slot-exists-0x1f", +} + +#: Buffer ids (NV2080_CTRL_GPU_PROMOTE_CTX_BUFFER_ID_*), 570.144 interface order. +BUFFER_ID = { + "MAIN": 0, "PM": 1, "PATCH": 2, "BUNDLE_CB": 3, "PAGEPOOL": 4, + "ATTRIBUTE_CB": 5, "RTV_CB_GLOBAL": 6, "GFXP_POOL": 7, "GFXP_CTRL_BLK": 8, + "FECS_EVENT": 9, "PRIV_ACCESS_MAP": 10, "UNRESTRICTED_PRIV_ACCESS_MAP": 11, + "GLOBAL_PRIV_ACCESS_MAP": 12, +} + + +#: Scratch base for crafted predicate state. Must be encodable by the +#: ``lui+addi`` return stub (below 2^31), inside RAM, outside the image +#: (max vaddr ~0x5bb9000), outside the 0x0..0x8000000 DMA window and below +#: the 0x100000000 DMEM aperture the firmware uses after relocation. +FECS_SCRATCH_BASE = 0x10000000 + + +def _fecs_layout() -> dict: + """Deterministic scratch layout (see module doc).""" + scratch = FECS_SCRATCH_BASE + return { + "scratch": scratch, + "sp": scratch + 0x4000, + "s0": scratch + 0x5000, + "pgpu": scratch + 0x1000, + "getter_out": scratch + 0x1800, + "instance": scratch + 0x3800, + "entry": scratch + 0x6000, + } + + +def _fecs_variant(layout: dict, *, label: str, buffer_id: int, b_initialize: int, + b_nonmapped: int, phys_attr: int = 4, size: int = 0x10000, + phys: int = 0x1F4150000, virt: int = 0x204608000, + slot: int | None = None, expect: str | None = None) -> dict: + lay = layout + entry = lay["entry"] + # table = *(u64*)(instance+0x190) = instance+0x300; FECS maps to slot 6. + slot_addr = lay["instance"] + 0x300 + 6 * 8 + memory = [ + {"addr": f"{entry + 0x30:#x}", "u64": b_initialize and phys or 0}, + {"addr": f"{entry + 0x38:#x}", "u64": virt}, + {"addr": f"{entry + 0x40:#x}", "u64": size}, + {"addr": f"{entry + 0x48:#x}", "u32": phys_attr}, + {"addr": f"{entry + 0x4c:#x}", "u16": buffer_id}, + {"addr": f"{entry + 0x4e:#x}", "u8": b_initialize}, + {"addr": f"{entry + 0x4f:#x}", "u8": b_nonmapped}, + # pGpu -> instance pointer; getter result -> idx 0; instance -> table. + {"addr": f"{lay['pgpu'] + 0x1be0:#x}", "u64": lay["instance"]}, + {"addr": f"{lay['instance'] + 0x190:#x}", "u64": lay["instance"] + 0x300}, + ] + if slot is not None: + memory.append({"addr": f"{slot_addr:#x}", "u64": slot}) + return { + "label": label, + "registers": { + "s0": f"{lay['s0']:#x}", + "s2": f"{lay['pgpu']:#x}", + "s5": f"{entry:#x}", + "sp": f"{lay['sp']:#x}", + "a3": buffer_id, + "t1": 0xC, # loop bound set by the function before our entry point + }, + "stubs": [{"addr": f"{INSTANCE_GETTER:#x}", "return": f"{lay['getter_out']:#x}"}], + "memory": memory, + "reads": [{"addr": f"{lay['s0'] - 0x1c8:#x}", "size": 4, + "label": "firmware-mapped-slot-index"}, + {"addr": f"{slot_addr:#x}", "size": 8, "label": "fecs-slot-value"}], + "expect": expect, + } + + +def fecs_spec(*, image: str = DEFAULT_IMAGE, profile_path: str = DEFAULT_PROFILE) -> dict: + """Build the FECS promote-validation matrix spec (bundled target).""" + layout = _fecs_layout() + variants = [ + # Live m3j: FECS init (PA) refused -- slot already populated. + _fecs_variant(layout, label="FECS init vidmem, slot non-NULL", + buffer_id=9, b_initialize=1, b_nonmapped=0, slot=1, + expect="refuse-fecs-slot-exists-0x1f"), + # Predicate inverse: the same shape would be accepted on an empty slot. + _fecs_variant(layout, label="FECS init vidmem, slot NULL", + buffer_id=9, b_initialize=1, b_nonmapped=0, slot=0, + expect="accept-commit-path"), + # The handoff's untested variant (a): does a sysmem aperture change it? + _fecs_variant(layout, label="FECS init sysmem attr 0x6, slot non-NULL", + buffer_id=9, b_initialize=1, b_nonmapped=0, phys_attr=6, slot=1, + expect="refuse-fecs-slot-exists-0x1f"), + _fecs_variant(layout, label="FECS init sysmem attr 0x6, slot NULL", + buffer_id=9, b_initialize=1, b_nonmapped=0, phys_attr=6, slot=0, + expect="accept-commit-path"), + # Size/PA content must not matter to the predicate. + _fecs_variant(layout, label="FECS init size 0x1000, slot non-NULL", + buffer_id=9, b_initialize=1, b_nonmapped=0, size=0x1000, slot=1, + expect="refuse-fecs-slot-exists-0x1f"), + # Live m7: FECS VA-only accepted (bInitialize=0 path). + _fecs_variant(layout, label="FECS no-init mapped, slot non-NULL", + buffer_id=9, b_initialize=0, b_nonmapped=0, slot=1, + expect="accept-skip-path"), + _fecs_variant(layout, label="FECS no-init nonmapped (bookkeeping refusal)", + buffer_id=9, b_initialize=0, b_nonmapped=1, slot=1, + expect="refuse-invalid-argument-0x1f"), + # Live m3j: PM and PATCH init accepted. + _fecs_variant(layout, label="PM init", + buffer_id=1, b_initialize=1, b_nonmapped=0, + expect="accept-commit-path"), + _fecs_variant(layout, label="PATCH init", + buffer_id=2, b_initialize=1, b_nonmapped=0, + expect="accept-commit-path"), + _fecs_variant(layout, label="GFXP_CTRL_BLK init (id 8)", + buffer_id=8, b_initialize=1, b_nonmapped=0, + expect="accept-skip-path"), + _fecs_variant(layout, label="BUNDLE_CB init (id 3)", + buffer_id=3, b_initialize=1, b_nonmapped=0, + expect="refuse-invalid-argument-0x1f"), + _fecs_variant(layout, label="PRIV_ACCESS_MAP init (id 10)", + buffer_id=10, b_initialize=1, b_nonmapped=0, + expect="accept-skip-path"), + _fecs_variant(layout, label="UNRESTRICTED_PRIV init (id 11)", + buffer_id=11, b_initialize=1, b_nonmapped=0, + expect="accept-skip-path"), + _fecs_variant(layout, label="GLOBAL_PRIV_ACCESS_MAP init (id 12)", + buffer_id=12, b_initialize=1, b_nonmapped=0, + expect="refuse-invalid-argument-0x1f"), + ] + # id > 0xc is refused one block earlier (0x5290cc4, still in the per-entry + # loop); enter there with the loop's a4/t1 state. + unknown_id = _fecs_variant(layout, label="unknown buffer id 13", + buffer_id=13, b_initialize=1, b_nonmapped=0, + expect="refuse-invalid-argument-0x1f") + unknown_id["entry"] = "0x5290cc0" + unknown_id["registers"]["a4"] = 13 + variants.append(unknown_id) + return { + "format": SPEC_FORMAT, + "name": "fecs-promote-validation", + "target": f"{PROMOTE_VALIDATION:#x}", + "outcomes": OUTCOMES, + "budget": 20_000, + "timeout": 20.0, + "image": image, + "profile": profile_path, + "note": ( + "570.144 per-buffer PROMOTE_CTX validation; instance-index getter " + f"({INSTANCE_GETTER:#x}, lock/table walk outside the predicate) stubbed " + "to a recorded constant; the predicate, the id->slot mapping call and " + "the memDesc slot read are the firmware's own code." + ), + "variants": variants, + } + + +def run_demo(out: str | None = None) -> dict: + """Run the bundled FECS matrix on the real image, twice for determinism.""" + spec = fecs_spec() + first = run_spec(spec=spec, out=out, record=False, name="valprobe-fecs") + second = run_spec(spec=spec, record=False, name="valprobe-fecs") + verdicts_a = [(r["label"], r["outcome"]) for r in first["variants"]] + verdicts_b = [(r["label"], r["outcome"]) for r in second["variants"]] + result = { + "format": DEMO_FORMAT, + "ok": first["failed"] == 0 and verdicts_a == verdicts_b, + "evaluated": first["evaluated"], + "failed": first["failed"], + "deterministic": verdicts_a == verdicts_b, + "verdicts": [{"label": label, "outcome": outcome} for label, outcome in verdicts_a], + } + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "valprobe-demo.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(result, fh, indent=2, sort_keys=True) + fh.write("\n") + result["artifacts"] = {"demo_json": os.path.abspath(path)} + return result + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="valprobe.py", + description=( + "Run one firmware validation-predicate branch with crafted state " + "(offline; never touches hardware)." + ), + ) + sub = parser.add_subparsers(dest="command") + sweep = sub.add_parser("sweep", help="run a bundled target matrix") + sweep.add_argument("--target", default="fecs-promote", choices=["fecs-promote"]) + sweep.add_argument("--image", default=None) + sweep.add_argument("--profile", default=None) + sweep.add_argument("--boot-to", default=None, help="boot the image to a rung first (e.g. B3)") + sweep.add_argument("--out", help="output directory (matrix JSON + sessions)") + sweep.add_argument("--no-record", action="store_true", help="skip session recording") + run = sub.add_parser("run", help="run a probe spec file") + run.add_argument("--spec", required=True) + run.add_argument("--image", default=None) + run.add_argument("--profile", default=None) + run.add_argument("--out", help="output directory (matrix JSON + sessions)") + run.add_argument("--no-record", action="store_true", help="skip session recording") + demo = sub.add_parser("demo", help="bundled FECS matrix, run twice (determinism)") + demo.add_argument("--out", help="output directory") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "sweep": + spec = fecs_spec(image=args.image or DEFAULT_IMAGE, + profile_path=args.profile or DEFAULT_PROFILE) + if args.boot_to: + spec["boot_to"] = args.boot_to + matrix = run_spec(spec=spec, image=args.image, profile_path=args.profile, + out=args.out, record=False if args.no_record else None) + elif args.command == "run": + spec = _load_spec(args.spec) + matrix = run_spec(spec=spec, image=args.image, profile_path=args.profile, + out=args.out, record=False if args.no_record else None) + elif args.command == "demo": + matrix = run_demo(args.out) + else: + parser.print_help() + return 2 + except ValprobeError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + if matrix.get("format") == MATRIX_FORMAT: + for row in matrix["variants"]: + mark = {True: "PASS", False: "FAIL", None: "----"}[row["pass"]] + print(f"{mark} {row['label']:<48} {row['outcome'] or row['status']}") + print(json.dumps({k: matrix[k] for k in + ("format", "spec", "evaluated", "passed", "failed", + "wall_seconds", "report_json") if k in matrix}, + indent=2, sort_keys=True)) + return 3 if matrix["failed"] else 0 + print(json.dumps(matrix, indent=2, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/sim/gmmu_walk.py b/debug/risc-v-gsp-rm-vm/sim/gmmu_walk.py new file mode 100644 index 0000000..50960ab --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sim/gmmu_walk.py @@ -0,0 +1,488 @@ +#!/usr/bin/env python3 +"""Functional GMMU page-table walker (Turing VER2 geometry). + +The decoder in :mod:`sim.gmmu_pte` decodes one captured entry. This module +follows those entries: given a page-directory base and a sparse physical-memory +view of the table pages, it walks a virtual address to its leaf mapping and +reports every level, the page size, the aperture and the access flags -- or the +exact fault (missing table, invalid aperture, invalid leaf). + +Why it exists: the engine sees memory through the GMMU, not through the host's +bookkeeping. When a forwarded submission goes silent (an engine-side write +never appears), the first question is *where would the engine read/write this +address?* A capture of the channel's page tables plus this walker answers it +offline, deterministically, without hardware. + +Evidence and honesty: + +* Geometry, field positions and the entry encodings come from the same + evidence base as :mod:`sim.gmmu_pte` (see ``docs/43`` and ``docs/48``). +* The 4K and 2M paths are **validated against a live captured channel PDB** + (see :func:`channel_pdb_fixture`); the 64K dual path is modelled from the + format definitions and reported with ``evidence: "modeled"`` until a capture + validates it. +* Unsupported encodings fail closed (``E_UNSUPPORTED``); a video-memory leaf + address is reported with ``pa_complete: false`` because the decoder cannot + claim the high address bits there (never guessed). +* No timing claim; this is the engine's *address* view (L1), not its behaviour. +""" +from __future__ import annotations + +import argparse +import json +import os +import struct +import sys + +TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.gmmu_pte import ( # noqa: E402 + GmmuDecodeError, + KIND_VER2, + KIND_VER2_DUAL, + LEVELS, + SUPPORTED_KINDS, + decode_dual_pde, + decode_entry, + decode_pde, + decode_pte, +) + +#: Schema tag of a walk report. +WALK_FORMAT = "greeneye-gmmu-walk/1" + +#: Page modes the walker accepts. ``auto`` prefers the 64K dual half for a +#: 64K-aligned VA when that half is valid, else the 4K half. +PAGE_MODES = ("4k", "64k", "auto") + +VA_MASK = (1 << 49) - 1 + +#: Fault codes (never a guess; each names a concrete missing/invalid piece). +FAULT_TABLE_MISSING = "table_missing" +FAULT_INVALID_APERTURE = "invalid_aperture" +FAULT_PTE_INVALID = "pte_invalid" +FAULT_UNSUPPORTED = "unsupported" +FAULT_OUT_OF_RANGE = "va_out_of_range" + + +class WalkError(ValueError): + """A walk request the evidence cannot answer.""" + + +class MemoryView: + """Sparse physical memory: ``pa -> bytes`` blobs (table pages, targets).""" + + def __init__(self, blobs: dict[int, bytes] | None = None): + self._blobs = dict(blobs or {}) + + def add(self, pa: int, data: bytes) -> None: + if pa < 0 or not isinstance(data, (bytes, bytearray)): + raise WalkError("add() needs a non-negative pa and bytes") + self._blobs[pa] = bytes(data) + + def has(self, pa: int) -> bool: + return pa in self._blobs + + def read(self, pa: int, size: int) -> bytes | None: + for base, blob in self._blobs.items(): + if base <= pa and pa + size <= base + len(blob): + off = pa - base + return blob[off:off + size] + return None + + def spans(self) -> list[dict]: + return [{"pa": f"{pa:#x}", "size": len(b)} for pa, b in sorted(self._blobs.items())] + + +# --------------------------------------------------------------------------- +# Encoders (fixtures/tests; mirror of the decoder's field positions) +# --------------------------------------------------------------------------- + +def encode_pde_pointer(pa: int, *, volatile: bool = True) -> int: + """Encode a VER2 directory PDE: bit0=0, aperture 3 (sysmem non-coherent).""" + if pa & 0xFFF: + raise WalkError(f"table PA must be 4K aligned, got {pa:#x}") + return ((pa >> 12) << 8) | (3 << 1) | (8 if volatile else 0) + + +def encode_pte(pa: int, *, page_size: str = "4k", read_only: bool = False, + volatile: bool = True) -> int: + """Encode a VER2 leaf PTE: bit0=1, aperture 3 (sysmem non-coherent).""" + if page_size == "4k": + align = 0xFFF + elif page_size == "64k": + align = 0xFFFF + elif page_size == "2m": + align = 0x1F_FFFF + else: + raise WalkError(f"unknown page_size {page_size!r}") + if pa & align: + raise WalkError(f"PA {pa:#x} is not {page_size}-aligned") + return ((pa >> 12) << 8) | (3 << 1) | 1 | (8 if volatile else 0) | (0x40 if read_only else 0) + + +def encode_dual_slot(big_pt_pa: int | None, small_pt_pa: int) -> bytes: + """Encode a VER2 dual PDE0 slot; BIG = 64K table, SMALL = 4K table.""" + big = 0 + if big_pt_pa is not None: + if big_pt_pa & 0xFFF: + raise WalkError(f"big table PA must be 4K aligned, got {big_pt_pa:#x}") + big = (((big_pt_pa >> 8) & ~1) << 4) | (3 << 1) | 8 + small = ((small_pt_pa >> 12) << 8) | (3 << 1) | 8 + return struct.pack("<QQ", big, small) + + +def _fault(code: str, message: str, steps: list, **extra) -> dict: + report = { + "format": WALK_FORMAT, "status": "fault", "fault": code, + "fault_message": message, "steps": steps, + } + report.update(extra) + return report + + +def walk( + va: int, + *, + root_pa: int, + mem: MemoryView, + kind: str = KIND_VER2, + page_mode: str = "4k", +) -> dict: + """Walk ``va`` through the page tables rooted at ``root_pa``. + + Returns a ``greeneye-gmmu-walk/1`` report. ``status`` is ``ok`` with + ``pa``/``page_size``/``writable`` on success, otherwise ``fault`` with a + concrete ``fault`` code. Every level's raw entry and decode is kept in + ``steps`` so the result is auditable. + """ + if kind not in SUPPORTED_KINDS: + return _fault(FAULT_UNSUPPORTED, f"kind {kind!r} not in {SUPPORTED_KINDS}", + [], va=va, root_pa=root_pa) + if page_mode not in PAGE_MODES: + raise WalkError(f"page_mode must be one of {PAGE_MODES}, got {page_mode!r}") + if not (0 <= va <= VA_MASK): + return _fault(FAULT_OUT_OF_RANGE, f"va {va:#x} outside 49-bit space", [], + va=va, root_pa=root_pa) + + steps: list[dict] = [] + pa_complete = True + page_used = page_mode + evidence = "validated" + cur = root_pa + + for depth in range(5): + geo = LEVELS[depth] + _hi_bit, lo_bit = geo["va_bits"] + idx = (va >> lo_bit) & ((1 << geo["index_bits"]) - 1) + table_pa = cur + if depth == 3: + off = idx * 16 + raw = mem.read(table_pa + off, 16) + if raw is None: + return _fault(FAULT_TABLE_MISSING, + f"depth-3 dual slot at pa={table_pa + off:#x} not in view", + steps, va=va, root_pa=root_pa) + value128 = int.from_bytes(raw, "little") + low = value128 & 0xFFFF_FFFF_FFFF_FFFF + step = {"depth": depth, "level": geo["hw"], "table_pa": f"{table_pa:#x}", + "index": idx, "entry_pa": f"{table_pa + off:#x}", + "raw": f"{value128:#x}"} + if low & 1: # 2 MB leaf in the low half + try: + pte = decode_entry(low, kind=KIND_VER2, role="pte", level=3) + except GmmuDecodeError as exc: + return _fault(FAULT_UNSUPPORTED, str(exc), steps + [step], + va=va, root_pa=root_pa) + step.update({"role": "pte", "page_size": "2m", + "aperture_name": pte["aperture_name"], + "valid": pte["valid"], "address": pte["address"]}) + steps.append(step) + if not pte["valid"] or pte["address"] is None: + return _fault(FAULT_PTE_INVALID, + f"2M leaf at {table_pa + off:#x} invalid/unsupported", + steps, va=va, root_pa=root_pa) + if not pte["address_complete"]: + pa_complete = False + pa = pte["address"] + (va & 0x1F_FFFF) + return _ok(va, pa, "2m", pte, steps, root_pa, kind, + page_used="2m", evidence=evidence, pa_complete=pa_complete) + try: + dual = decode_dual_pde(value128, pde_kind=KIND_VER2_DUAL, level=3) + except GmmuDecodeError as exc: + return _fault(FAULT_UNSUPPORTED, str(exc), steps + [step], + va=va, root_pa=root_pa) + chosen = None + if page_mode == "64k": + chosen, page_used, evidence = dual["big"], "64k", "modeled" + elif page_mode == "4k": + chosen, page_used = dual["small"], "4k" + else: # auto: prefer BIG for a 64K-aligned VA when valid + if va % 0x10000 == 0 and dual["big"]["address"] is not None: + chosen, page_used, evidence = dual["big"], "64k", "modeled" + else: + chosen, page_used = dual["small"], "4k" + step.update({"role": "dual_pde", "page_mode_used": page_used, + "big": dual["big"], "small": dual["small"]}) + steps.append(step) + if chosen["address"] is None: + return _fault(FAULT_INVALID_APERTURE, + f"dual PDE at {table_pa + off:#x} {page_used} half has " + f"aperture {chosen['aperture_name']}", steps, + va=va, root_pa=root_pa) + if not chosen["address_complete"]: + pa_complete = False + cur = chosen["address"] + continue + + off = idx * geo["entry_size"] + if depth == 4: + # A 64K-granularity leaf table is indexed by VA bits 24:16. + idx = (va >> (16 if page_used == "64k" else 12)) & 0x1FF + off = idx * geo["entry_size"] + raw = mem.read(table_pa + off, geo["entry_size"]) + if raw is None: + return _fault(FAULT_TABLE_MISSING, + f"level {depth} entry at pa={table_pa + off:#x} not in view", + steps, va=va, root_pa=root_pa) + value = int.from_bytes(raw, "little") + step = {"depth": depth, "level": geo["hw"], "table_pa": f"{table_pa:#x}", + "index": idx, "entry_pa": f"{table_pa + off:#x}", "raw": f"{value:#x}"} + if depth < 3: + try: + pde = decode_pde(value, pde_kind=KIND_VER2, level=depth) + except GmmuDecodeError as exc: + return _fault(FAULT_UNSUPPORTED, str(exc), steps + [step], + va=va, root_pa=root_pa) + step.update({"role": "pde", "aperture_name": pde["aperture_name"], + "address": pde["address"]}) + steps.append(step) + if pde["address"] is None: + return _fault(FAULT_INVALID_APERTURE, + f"level {depth} entry at {table_pa + off:#x} has " + f"aperture {pde['aperture_name']}", steps, + va=va, root_pa=root_pa) + if not pde["address_complete"]: + pa_complete = False + cur = pde["address"] + continue + + # depth 4: leaf page table + try: + pte = decode_pte(value, pte_kind=KIND_VER2, level=4) + except GmmuDecodeError as exc: + return _fault(FAULT_UNSUPPORTED, str(exc), steps + [step], + va=va, root_pa=root_pa) + leaf_mask = 0xFFFF if page_used == "64k" else 0xFFF + step.update({"role": "pte", "page_size": page_used, + "aperture_name": pte["aperture_name"], "valid": pte["valid"], + "address": pte["address"], "writable": pte["writable"]}) + steps.append(step) + if not pte["valid"] or pte["address"] is None: + return _fault(FAULT_PTE_INVALID, + f"leaf PTE at {table_pa + off:#x} invalid/unsupported " + f"(raw {value:#x})", steps, va=va, root_pa=root_pa) + if not pte["address_complete"]: + pa_complete = False + pa = pte["address"] + (va & leaf_mask) + return _ok(va, pa, page_used, pte, steps, root_pa, kind, + page_used=page_used, evidence=evidence, pa_complete=pa_complete) + + return _fault(FAULT_UNSUPPORTED, "walk exceeded 5 levels", steps, + va=va, root_pa=root_pa) + + +def _ok(va, pa, page_size, leaf, steps, root_pa, kind, *, page_used, evidence, + pa_complete) -> dict: + report = { + "format": WALK_FORMAT, + "status": "ok", + "va": f"{va:#x}", + "root_pa": f"{root_pa:#x}", + "kind": kind, + "page_mode_used": page_used, + "page_size": page_size, + "evidence": evidence, + "pa": f"{pa:#x}", + "pa_complete": pa_complete, + "aperture_name": leaf.get("aperture_name"), + "writable": leaf.get("writable", True), + "read_only": leaf.get("read_only", False), + "kind_name": leaf.get("kind_name"), + "steps": steps, + "assumptions": [ + "4K/2M paths are validated against a captured channel PDB; the 64K dual path is modelled and reported with evidence='modeled'.", + "A video-memory leaf reports pa_complete=false (high address bits are not claimed by the decoder).", + ], + } + if page_size == "64k": + report["assumptions"].append("64K page-mode selection for auto walks is an assumption (64K-aligned VA prefers the BIG half).") + return report + + +def walk_many(vas, *, root_pa: int, mem: MemoryView, kind: str = KIND_VER2, + page_mode: str = "4k") -> list[dict]: + return [walk(va, root_pa=root_pa, mem=mem, kind=kind, page_mode=page_mode) + for va in vas] + + +# --------------------------------------------------------------------------- +# Captured golden vector: a live channel PDB (numeric only) +# --------------------------------------------------------------------------- + +#: Captured channel-PDB entries from a live run (five VAs of one channel +#: chunk). Used as the walker's regression vector: the walk must reproduce +#: the DMA addresses the capture logged for these VAs. +_CHANNEL_PDB = { + "root_pa": 0x12F16E000, + "root": {0: 0x0000_0000_12F16F0E}, + "pd2": {0: 0x0000_0000_11065E0E}, + "pd1": {0: 0x0000_0000_1106630E}, + "pd0_small": {0: 0x0000_0000_15908D0E}, + "pt": { + 256: 0x0000_0000_2045D90F, + 257: 0x0000_0000_121BD90F, + 258: 0x0000_0000_1521150F, + 259: 0x0000_0000_15A5F50F, + 260: 0x0000_0000_10CF270F, + }, +} + +_CHANNEL_VAS = { + 0x100000: ("gpfifo", 0x2045D9000), + 0x101000: ("push-buffer", 0x121BD9000), + 0x102000: ("copy-src", 0x152115000), + 0x103000: ("copy-dst", 0x15A5F5000), + 0x104000: ("semaphore", 0x10CF27000), +} + + +def channel_pdb_fixture() -> tuple[int, MemoryView]: + """Rebuild the captured channel PDB as a walkable memory view.""" + mem = MemoryView() + root = bytearray(0x1000) + for idx, value in _CHANNEL_PDB["root"].items(): + struct.pack_into("<Q", root, idx * 8, value) + mem.add(0x12F16E000, bytes(root)) + for name, base in (("pd2", 0x12F16F000), ("pd1", 0x11065E000)): + page = bytearray(0x1000) + for idx, value in _CHANNEL_PDB[name].items(): + struct.pack_into("<Q", page, idx * 8, value) + mem.add(base, bytes(page)) + pd0 = bytearray(0x1000) + for idx, small in _CHANNEL_PDB["pd0_small"].items(): + struct.pack_into("<16s", pd0, idx * 16, encode_dual_slot(None, (small >> 8) << 12)) + mem.add(0x110663000, bytes(pd0)) + pt = bytearray(0x1000) + for idx, value in _CHANNEL_PDB["pt"].items(): + struct.pack_into("<Q", pt, idx * 8, value) + mem.add(0x15908D000, bytes(pt)) + return _CHANNEL_PDB["root_pa"], mem + + +def run_demo(out: str | None = None) -> dict: + """Walk the captured channel-PDB vector; PASS iff every VA matches.""" + root_pa, mem = channel_pdb_fixture() + rows = [] + ok = True + for va, (label, expected) in sorted(_CHANNEL_VAS.items()): + report = walk(va, root_pa=root_pa, mem=mem) + got = int(report["pa"], 16) if report["status"] == "ok" else None + match = got == expected + ok = ok and match + rows.append({"label": label, "va": f"{va:#x}", "expected_pa": f"{expected:#x}", + "walk": report, "match": match}) + demo = {"format": "greeneye-gmmu-walk-demo/1", "ok": ok, "rows": rows, + "root_pa": f"{root_pa:#x}"} + if out: + os.makedirs(out, exist_ok=True) + path = os.path.join(out, "gmmu-walk-demo.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(demo, fh, indent=2, sort_keys=True) + fh.write("\n") + demo["artifacts"] = {"demo_json": os.path.abspath(path)} + return demo + + +def _parse_int(text: str) -> int: + try: + return int(text, 0) + except ValueError as exc: + raise WalkError(f"bad integer {text!r}") from exc + + +def _parse_mem(spec: str) -> tuple[int, bytes]: + if "=" not in spec: + raise WalkError(f"--mem needs <pa>=<hex>: {spec!r}") + pa_text, _, hex_text = spec.partition("=") + data = "".join(hex_text.split()) + if len(data) % 2: + raise WalkError(f"odd hex length in {spec!r}") + return _parse_int(pa_text), bytes.fromhex(data) + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="gmmu_walk.py", + description="Walk a GMMU VA through captured/constructed page tables (offline).", + ) + sub = parser.add_subparsers(dest="command") + run = sub.add_parser("run", help="walk VAs through a page-table view") + run.add_argument("--pdb-pa", required=True, help="page-directory base PA") + run.add_argument("--va", action="append", required=True, help="virtual address (repeatable)") + run.add_argument("--mem", action="append", default=[], help="<pa>=<hex> table page (repeatable)") + run.add_argument("--kind", default=KIND_VER2, choices=list(SUPPORTED_KINDS)) + run.add_argument("--page-mode", default="4k", choices=list(PAGE_MODES)) + run.add_argument("--out", help="write the JSON report here") + demo = sub.add_parser("demo", help="walk the captured channel-PDB vector") + demo.add_argument("--out", help="output directory") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + try: + if args.command == "run": + mem = MemoryView() + for spec in args.mem: + pa, data = _parse_mem(spec) + mem.add(pa, data) + reports = [walk(_parse_int(va), root_pa=_parse_int(args.pdb_pa), mem=mem, + kind=args.kind, page_mode=args.page_mode) for va in args.va] + doc = {"format": "greeneye-gmmu-walk-set/1", "reports": reports} + for report in reports: + if report["status"] == "ok": + print(f"OK {report['va']} -> {report['pa']} " + f"{report['page_size']} {report['aperture_name']} " + f"writable={report['writable']}") + else: + print(f"FAULT {report['va']} {report['fault']}: {report['fault_message']}") + if args.out: + os.makedirs(args.out, exist_ok=True) + path = os.path.join(args.out, "gmmu-walk.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(doc, fh, indent=2, sort_keys=True) + fh.write("\n") + print(json.dumps({"report_json": os.path.abspath(path)}, indent=2)) + return 0 if all(r["status"] == "ok" for r in reports) else 3 + if args.command == "demo": + demo = run_demo(args.out) + for row in demo["rows"]: + report = row["walk"] + got = report.get("pa", report.get("fault_message")) + print(f"{'PASS' if row['match'] else 'FAIL'} {row['label']:<12} " + f"{row['va']} -> {got} (expected {row['expected_pa']})") + print(json.dumps({"ok": demo["ok"]}, indent=2)) + return 0 if demo["ok"] else 3 + parser.print_help() + return 2 + except WalkError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/sim/host_traffic.py b/debug/risc-v-gsp-rm-vm/sim/host_traffic.py new file mode 100644 index 0000000..a9dd55d --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sim/host_traffic.py @@ -0,0 +1,1437 @@ +"""Host-traffic message loop: doorbell + GSP msgq transport + replay (B4/B5). + +Status: **stub wave (in development)**. This module lands the transport half +of the B4/B5 rung (docs/35-GREENEYE-SIMULATOR-DESIGN.md §3.3, docs/45, +handoff-simulator.md §31 item B1): a modelled host doorbell, a command/status +message-queue pair laid out with the real queue arithmetic, queue-element +framing with the real checksum, a small stub dispatcher, and deterministic +replay of an ``rm_capture`` stimulus. The real 570.144 firmware is **not** +run by this module: no real message loop, no RM semantics, no firmware claim. + +Real structures mirrored here (provenance: public OpenRM headers and the +header-derived layouts already used by the project's boot code; every constant +is checked against ``harness/rm_capture.py`` where the two overlap): + +* ``NV_PGSP_QUEUE_HEAD(i) = 0x110c00 + i * 8`` (public Turing ``dev_gsp.h``), + 32-bit RW; the 570.144 host writes it after submitting a command + (``kgspSetCmdQueueHead``), and the command data itself lives in the sysmem + message queues. The simulator reaches the register at ``PRIV+0x110c00``. +* ``msgqTxHeader`` (8 x u32: version/size/msgSize/msgCount/writePtr/flags/ + rxHdrOff/entryOff) and ``msgqRxHeader`` (readPtr), with + ``rxHdrOff = align32(16) = 32`` and ``entryOff = align36(16) = 48`` for the + GSP header alignment of 4 (``msgqTxCreate``). +* ``GSP_MSG_QUEUE_ELEMENT``: 16 B authTag + 16 B aad + checksum/seqNum/ + elemCount/reserved (16 B) + the 32 B ``rpc_message_header_v`` (so the RPC + payload starts at 0x50). Checksum = XOR of 64-bit words, hi32 ^ lo32; the + stored field makes a receiver's recomputation zero (verified). +* The real command queue is created with ``MSGQ_FLAGS_SWAP_RX``; the two rx + headers are cross-linked between the command and status buffers. That + cross-link is modelled, not simplified away. +* The RPC call shape is the one ``harness/rm_capture.py`` decodes + (``rpc_message_header_v`` = 32 bytes, GSP_RM_CONTROL = 76, GSP_RM_ALLOC = + 103); the demo's 512-byte control is real-shaped but comes from no capture. + +What is stubbed: the dispatcher answers only ``RPC_FN_ECHO`` / +``RPC_FN_STATUS_QUERY`` (stub-only function ids) and returns +``NV_ERR_NOT_SUPPORTED`` for every real RM function. That reply is framed and +transported exactly, but it is a stub verdict, never firmware behaviour. + +Determinism: the transport transcript carries no host timestamps and is +byte-identical across runs; only the optional ``greeneye-session/1`` recorder +events carry host ``t`` (documented as non-semantic by ``compare``). Every +run is bounded: a message stages contiguously and splits across at most 16 +ring slots (the wire element maximum; larger messages are refused), ``poll`` +drains only the ring, ``run_cycles`` is capped. No hardware, module, mdev, +guest, network or QEMU is touched. +""" +from __future__ import annotations + +import argparse +import json +import os +import struct +import sys +from dataclasses import dataclass +from typing import Any, Callable + +from .devices.base import Device + +_TOOL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + +#: GSP doorbell register block (public Turing ``dev_gsp.h``, block-relative). +QUEUE_HEAD_OFFSET = 0x110C00 +QUEUE_HEAD_STRIDE = 8 +QUEUE_HEAD_COUNT = 8 +#: PRIV aperture base used by the GSP profiles; doorbell window = PRIV + offset. +PRIV_BASE = 0x2000000000000000 +DOORBELL_BASE = PRIV_BASE + QUEUE_HEAD_OFFSET +DOORBELL_SIZE = QUEUE_HEAD_STRIDE * QUEUE_HEAD_COUNT +#: Host writes the queue head to this register as the kick (observed 570.144 +#: submit path writes 0); the model records the value per queue index. +DOORBELL_KICK_QUEUE = 0 +DOORBELL_KICK_VALUE = 0 + +#: ``msgq_priv.h``. +MSGQ_VERSION = 0 +MSGQ_FLAGS_SWAP_RX = 1 +MSGQ_TX_HEADER_BYTES = 32 +MSGQ_RX_HEADER_BYTES = 4 +MSGQ_HEADER_ALIGN = 4 +MSGQ_RX_HDR_OFF = 32 +MSGQ_ENTRY_OFF = 48 +MSGQ_TX_HEADER_FORMAT = "<8I" +MSGQ_RX_HEADER_FORMAT = "<I" + +#: ``message_queue_priv.h``. +GSP_MSG_QUEUE_ELEMENT_SIZE_MIN = 0x1000 +GSP_MSG_QUEUE_ELEMENT_SIZE_MAX = 0x10000 +GSP_MSG_QUEUE_HEADER_SIZE = 0x1000 +GSP_MSG_QUEUE_ELEMENT_HDR_SIZE = 0x30 + +#: ``g_rpc-message-header.h`` / 570.144 wire constants. +RPC_HEADER_BYTES = 32 +RPC_HEADER_FORMAT = "<8I" +RPC_SIGNATURE = 0x43505256 +RPC_HEADER_VERSION = 0x03000000 + +#: Default shared sysmem queue block and sizes (host ``message_queue_cpu.c``). +DEFAULT_CMD_QUEUE_SIZE = 0x40000 +DEFAULT_STAT_QUEUE_SIZE = 0x40000 +SHARED_MEM_BASE = 0x81000000 + +#: Real RM wire function ids used by the capture fixtures. +GSP_RM_CONTROL = 76 +GSP_RM_ALLOC = 103 +GSP_GET_GSP_STATIC_INFO = 65 + +#: Stub-only dispatcher ids (deliberately not RM function numbers). +RPC_FN_ECHO = 0x40 +RPC_FN_STATUS_QUERY = 0x41 + +#: ``nvstatuscodes.h``. +NV_OK = 0 +NV_ERR_NOT_SUPPORTED = 0x56 + +#: Bounded ``run_cycles`` cap (same ceiling as the boot runner's attempts). +MAX_RUN_CYCLES = 2_000_000 + +STATUS_MAGIC = b"GSTS" +STATUS_VERSION = 1 +STATUS_FORMAT = "<4sIIII" + +DEMO_FORMAT = "greeneye-host-traffic-demo/1" +REPLAY_FORMAT = "greeneye-host-traffic-replay/1" +TRANSCRIPT_FORMAT = "greeneye-host-traffic-log/1" +REPLY_FORMAT = "greeneye-host-traffic-reply/1" +STIMULUS_FORMAT = "greeneye-rm-rpc-stimulus/1" +CAPTURE_FORMAT = "greeneye-rm-rpc-capture/1" + +_TX_HEADER = struct.Struct(MSGQ_TX_HEADER_FORMAT) +_RX_HEADER = struct.Struct(MSGQ_RX_HEADER_FORMAT) +_RPC_HEADER = struct.Struct(RPC_HEADER_FORMAT) +_ELEMENT_PREFIX = struct.Struct("<16s16sIIII") + +_LIMITS = [ + "Stub wave: no real firmware message loop is run; the dispatcher handlers " + "are stubs and real RM functions get NV_ERR_NOT_SUPPORTED.", + "Multi-element messages are split across ring slots up to the wire element " + "maximum (0x10000 bytes); larger messages are refused, not truncated.", + "The shared-memory address and queue sizes are fixture defaults, not " + "values read from a real GSP_ARGUMENTS_CACHED transfer.", + "The MSI-X/event path by which the real GSP signals a status-queue write " + "is not modelled; replies are read by polling.", + "L0/L1 framing and ordering only; no timing (L2) and no device-environment " + "(L3) claim.", +] + + +class HostTrafficError(Exception): + """A framing, queue or replay input that cannot be represented.""" + + +def _align_up(value: int, align: int) -> int: + return (value + align - 1) // align * align + + +# --------------------------------------------------------------------------- +# Framing +# --------------------------------------------------------------------------- + + +@dataclass +class RpcMessage: + """One decoded ``rpc_message_header_v`` plus its payload.""" + + header_version: int + signature: int + length: int + function: int + rpc_result: int + rpc_result_private: int + sequence: int + spare: int + params: bytes = b"" + + def to_bytes(self) -> bytes: + return _RPC_HEADER.pack( + self.header_version, + self.signature, + RPC_HEADER_BYTES + len(self.params), + self.function, + self.rpc_result, + self.rpc_result_private, + self.sequence, + self.spare, + ) + self.params + + +@dataclass +class RpcRequest: + """A command message handed to a dispatcher handler.""" + + function: int + sequence: int + params: bytes + element_index: int + raw: bytes + + +@dataclass +class RpcReply: + """A handler result before framing into a status element.""" + + function: int + sequence: int + params: bytes = b"" + result: int = NV_OK + + +def build_rpc_message( + function: int, + params: bytes = b"", + sequence: int = 0, + *, + result: int = NV_OK, + result_private: int = 0, + header_version: int = RPC_HEADER_VERSION, + signature: int = RPC_SIGNATURE, +) -> bytes: + """Frame one ``rpc_message_header_v`` + payload (capture-tool shape).""" + payload = bytes(params) + return _RPC_HEADER.pack( + header_version, + signature, + RPC_HEADER_BYTES + len(payload), + int(function) & 0xFFFFFFFF, + int(result) & 0xFFFFFFFF, + int(result_private) & 0xFFFFFFFF, + int(sequence) & 0xFFFFFFFF, + 0, + ) + payload + + +def parse_rpc_message(data: bytes) -> RpcMessage: + """Decode and validate one RPC message; raises on a bad frame.""" + if len(data) < RPC_HEADER_BYTES: + raise HostTrafficError( + f"rpc message shorter than {RPC_HEADER_BYTES} bytes: {len(data)}" + ) + ( + header_version, + signature, + length, + function, + rpc_result, + rpc_result_private, + sequence, + spare, + ) = _RPC_HEADER.unpack_from(data, 0) + if signature != RPC_SIGNATURE: + raise HostTrafficError( + f"bad rpc signature {signature:#010x} (expected {RPC_SIGNATURE:#010x})" + ) + if length < RPC_HEADER_BYTES or length > len(data): + raise HostTrafficError( + f"rpc length {length:#x} outside [{RPC_HEADER_BYTES:#x}, {len(data):#x}]" + ) + return RpcMessage( + header_version=header_version, + signature=signature, + length=length, + function=function, + rpc_result=rpc_result, + rpc_result_private=rpc_result_private, + sequence=sequence, + spare=spare, + params=bytes(data[RPC_HEADER_BYTES:length]), + ) + + +def compute_message_checksum(data: bytes) -> int: + """The msgq checksum: XOR of 64-bit words, then hi32 ^ lo32. + + Zero-pads to 8 bytes exactly like ``_checkSum32``; a correctly built queue + element recomputes to 0. + """ + padded = bytes(data) + bytes((-len(data)) % 8) + total = 0 + for offset in range(0, len(padded), 8): + total ^= int.from_bytes(padded[offset : offset + 8], "little") + return ((total >> 32) ^ (total & 0xFFFFFFFF)) & 0xFFFFFFFF + + +def build_queue_element( + rpc_bytes: bytes, *, sequence: int = 0, elem_count: int | None = None +) -> bytes: + """Wrap an RPC message in a ``GSP_MSG_QUEUE_ELEMENT`` with checksum.""" + payload = bytes(rpc_bytes) + if elem_count is None: + elem_count = max(1, -(-(GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + len(payload)) + // GSP_MSG_QUEUE_ELEMENT_SIZE_MIN)) + prefix = _ELEMENT_PREFIX.pack(b"\x00" * 16, b"\x00" * 16, 0, + sequence & 0xFFFFFFFF, + elem_count & 0xFFFFFFFF, 0) + body = prefix + payload + checksum = compute_message_checksum(body) + return _ELEMENT_PREFIX.pack(b"\x00" * 16, b"\x00" * 16, checksum, + sequence & 0xFFFFFFFF, + elem_count & 0xFFFFFFFF, 0) + payload + + +def split_element_stream(stream: bytes, msg_size: int) -> list[bytes]: + """Split a contiguous element stream into ``msg_size`` ring slots. + + The real producer stages the whole message in one buffer and copies + ``GSP_MSG_QUEUE_ELEMENT_SIZE_MIN`` bytes per ring slot + (``GspMsgQueueSendCommand``); the tail slot is zero-padded. + """ + blob = bytes(stream) + if not blob: + raise HostTrafficError("empty element stream") + slots = [] + for start in range(0, len(blob), msg_size): + piece = blob[start : start + msg_size] + slots.append(piece + bytes(msg_size - len(piece))) + return slots + + +def parse_queue_element(data: bytes, *, verify: bool = True) -> tuple[dict, RpcMessage]: + """Decode one queue element stream; raises on a bad frame or checksum.""" + if len(data) < GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + RPC_HEADER_BYTES: + raise HostTrafficError(f"queue element too short: {len(data)} bytes") + auth_tag, aad, checksum, seq_num, elem_count, reserved = _ELEMENT_PREFIX.unpack_from(data, 0) + if not 1 <= elem_count <= GSP_MSG_QUEUE_ELEMENT_SIZE_MAX // GSP_MSG_QUEUE_ELEMENT_SIZE_MIN: + raise HostTrafficError(f"implausible elem_count {elem_count}") + rpc = parse_rpc_message(data[GSP_MSG_QUEUE_ELEMENT_HDR_SIZE:]) + total = GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + rpc.length + if total > len(data): + raise HostTrafficError(f"queue element truncated: need {total:#x}, have {len(data):#x}") + if verify and compute_message_checksum(bytes(data[:total])) != 0: + raise HostTrafficError("bad message checksum") + prefix = { + "auth_tag": auth_tag, + "aad": aad, + "checksum": checksum, + "seq_num": seq_num, + "elem_count": elem_count, + "reserved": reserved, + } + return prefix, rpc + + +# --------------------------------------------------------------------------- +# Memory adapters +# --------------------------------------------------------------------------- + + +class RegionMemory: + """A ``base``/``size`` bytearray window with Machine-like read/write.""" + + def __init__(self, base: int, size: int): + if size <= 0: + raise HostTrafficError(f"region size must be positive, got {size}") + self.base = int(base) + self.size = int(size) + self.data = bytearray(self.size) + + def _check(self, addr: int, size: int) -> int: + offset = addr - self.base + if offset < 0 or offset + size > self.size: + raise HostTrafficError( + f"access outside region: addr={addr:#x} size={size}" + ) + return offset + + def read(self, addr: int, size: int) -> bytes: + offset = self._check(addr, size) + return bytes(self.data[offset : offset + size]) + + def write(self, addr: int, data: bytes) -> None: + blob = bytes(data) + offset = self._check(addr, len(blob)) + self.data[offset : offset + len(blob)] = blob + + +class _MachineMemory: + """Adapter from RegionMemory's surface to ``sim.machine.Machine``.""" + + def __init__(self, machine): + self.machine = machine + + def read(self, addr: int, size: int) -> bytes: + return self.machine.read_mem(addr, size) + + def write(self, addr: int, data: bytes) -> None: + self.machine.write_mem(addr, data) + + +def _make_memory(machine, shared_base: int, size: int): + if machine is None: + return RegionMemory(shared_base, size) + return _MachineMemory(machine) + + +# --------------------------------------------------------------------------- +# Message queues +# --------------------------------------------------------------------------- + + +@dataclass +class MsgqTxHeader: + """``msgqTxHeader``: 8 x u32 written by the queue producer.""" + + version: int + size: int + msg_size: int + msg_count: int + write_ptr: int + flags: int + rx_hdr_off: int + entry_off: int + + def to_bytes(self) -> bytes: + return _TX_HEADER.pack( + self.version, + self.size, + self.msg_size, + self.msg_count, + self.write_ptr, + self.flags, + self.rx_hdr_off, + self.entry_off, + ) + + @classmethod + def from_bytes(cls, data: bytes) -> "MsgqTxHeader": + return cls(*_TX_HEADER.unpack_from(data, 0)) + + +class MsgqQueue: + """One msgq buffer: tx header at its base, entries at ``entryOff``. + + ``rx_hdr_addr`` is the physical address of this queue's rx header. The + real command queue keeps it in the status buffer (``MSGQ_FLAGS_SWAP_RX``), + so the caller passes the cross-linked address. + """ + + def __init__( + self, + mem, + base: int, + size: int, + *, + msg_size: int = GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + header_align: int = MSGQ_HEADER_ALIGN, + flags: int = MSGQ_FLAGS_SWAP_RX, + rx_hdr_addr: int | None = None, + version: int = MSGQ_VERSION, + ): + if size < GSP_MSG_QUEUE_ELEMENT_SIZE_MIN: + raise HostTrafficError(f"queue size {size:#x} below one element") + self.mem = mem + self.tx_base = int(base) + self.size = int(size) + self.msg_size = int(msg_size) + self.header_align = int(header_align) + self.flags = int(flags) + self.version = int(version) + self.rx_hdr_addr = int(rx_hdr_addr) if rx_hdr_addr is not None else base + MSGQ_RX_HDR_OFF + + def format(self) -> MsgqTxHeader: + """Write a fresh tx header and zero the rx header (msgqTxCreate).""" + rx_hdr_off = _align_up(MSGQ_TX_HEADER_BYTES, 1 << self.header_align) + entry_off = _align_up(rx_hdr_off + MSGQ_RX_HEADER_BYTES, 1 << self.header_align) + header = MsgqTxHeader( + version=self.version, + size=self.size, + msg_size=self.msg_size, + msg_count=(self.size - entry_off) // self.msg_size, + write_ptr=0, + flags=self.flags, + rx_hdr_off=rx_hdr_off, + entry_off=entry_off, + ) + self.mem.write(self.tx_base, header.to_bytes()) + self.mem.write(self.rx_hdr_addr, _RX_HEADER.pack(0)) + return header + + def header(self) -> MsgqTxHeader: + return MsgqTxHeader.from_bytes(self.mem.read(self.tx_base, MSGQ_TX_HEADER_BYTES)) + + def _write_header(self, header: MsgqTxHeader) -> None: + self.mem.write(self.tx_base, header.to_bytes()) + + @property + def write_ptr(self) -> int: + return self.header().write_ptr + + @property + def read_ptr(self) -> int: + return _RX_HEADER.unpack_from(self.mem.read(self.rx_hdr_addr, MSGQ_RX_HEADER_BYTES), 0)[0] + + def _entry_addr(self, index: int) -> int: + count = self.header().msg_count + if not 0 <= index < count: + raise HostTrafficError(f"entry index {index} outside 0..{count - 1}") + return self.tx_base + self.header().entry_off + index * self.msg_size + + def write_entry(self, index: int, data: bytes) -> None: + blob = bytes(data) + if len(blob) > self.msg_size: + raise HostTrafficError( + f"entry of {len(blob)} bytes exceeds msgSize {self.msg_size:#x}" + ) + self.mem.write(self._entry_addr(index), blob) + + def read_entry(self, index: int, size: int | None = None) -> bytes: + return self.mem.read(self._entry_addr(index), self.msg_size if size is None else size) + + def publish(self, write_ptr: int) -> int: + header = self.header() + header.write_ptr = int(write_ptr) % header.msg_count + self._write_header(header) + return header.write_ptr + + def set_read(self, read_ptr: int) -> int: + count = self.header().msg_count + value = int(read_ptr) % count + self.mem.write(self.rx_hdr_addr, _RX_HEADER.pack(value)) + return value + + def state(self) -> dict: + header = self.header() + return { + "tx_base": self.tx_base, + "size": self.size, + "msg_size": header.msg_size, + "msg_count": header.msg_count, + "write_ptr": header.write_ptr, + "read_ptr": self.read_ptr, + "flags": header.flags, + "rx_hdr_off": header.rx_hdr_off, + "entry_off": header.entry_off, + "rx_hdr_addr": self.rx_hdr_addr, + } + + +class SharedQueues: + """A command/status msgq pair with the real cross-linked rx headers. + + Layout follows the host's ``GspMsgQueuesInit``: the status buffer starts + ``cmd_size`` bytes after the command buffer, both tx headers carry + ``MSGQ_FLAGS_SWAP_RX``, and each queue's rx header physically lives in the + other buffer (``msgqRxLink``). + """ + + def __init__( + self, + mem, + *, + cmd_base: int = SHARED_MEM_BASE, + stat_base: int | None = None, + cmd_size: int = DEFAULT_CMD_QUEUE_SIZE, + stat_size: int = DEFAULT_STAT_QUEUE_SIZE, + msg_size: int = GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + swap_rx: bool = True, + ): + self.mem = mem + self.cmd_base = int(cmd_base) + self.stat_base = int(stat_base) if stat_base is not None else self.cmd_base + cmd_size + flags = MSGQ_FLAGS_SWAP_RX if swap_rx else 0 + self.command = MsgqQueue( + mem, + self.cmd_base, + cmd_size, + msg_size=msg_size, + flags=flags, + rx_hdr_addr=self.stat_base + MSGQ_RX_HDR_OFF, + ) + self.status = MsgqQueue( + mem, + self.stat_base, + stat_size, + msg_size=msg_size, + flags=flags, + rx_hdr_addr=self.cmd_base + MSGQ_RX_HDR_OFF, + ) + + def format(self) -> None: + self.command.format() + self.status.format() + + def _place(self, queue: MsgqQueue, element: bytes) -> dict: + slots = split_element_stream(element, queue.msg_size) + count = queue.header().msg_count + if len(slots) > count: + raise HostTrafficError( + f"message needs {len(slots)} elements, queue holds {count}" + ) + index = queue.write_ptr + ptr = index + for slot in slots: + queue.write_entry(ptr, slot) + ptr = (ptr + 1) % count + write_ptr = queue.publish(ptr) + return { + "index": index, + "write_ptr": write_ptr, + "bytes": len(element), + "elements": len(slots), + } + + def _take(self, queue: MsgqQueue) -> dict | None: + write_ptr = queue.write_ptr + read_ptr = queue.read_ptr + if read_ptr == write_ptr: + return None + count = queue.header().msg_count + first = queue.read_entry(read_ptr) + elements = int.from_bytes(first[0x28:0x2C], "little") + if not 1 <= elements <= count: + raise HostTrafficError(f"implausible elem_count {elements}") + slots = [] + ptr = read_ptr + for _ in range(elements): + slots.append(queue.read_entry(ptr)) + ptr = (ptr + 1) % count + stream = b"".join(slots) + total = GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + int.from_bytes( + stream[0x38:0x3C], "little" + ) + element = ( + stream[:total] + if GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + RPC_HEADER_BYTES <= total <= len(stream) + else stream + ) + queue.set_read(ptr) + return {"index": read_ptr, "element": element, "elements": elements} + + def submit_element(self, element: bytes) -> dict: + """Host -> GSP: append one element to the command ring.""" + return self._place(self.command, element) + + def take_command(self) -> dict | None: + """GSP side: consume the next command element, or ``None``.""" + return self._take(self.command) + + def publish_element(self, element: bytes) -> dict: + """GSP -> host: append one element to the status ring.""" + return self._place(self.status, element) + + def take_status(self) -> dict | None: + """Host side: consume the next status element, or ``None``.""" + return self._take(self.status) + + def state(self) -> dict: + return {"command": self.command.state(), "status": self.status.state()} + + +# --------------------------------------------------------------------------- +# Doorbell +# --------------------------------------------------------------------------- + + +class DoorbellModel: + """Recorded queue-head writes; the firmware-visible read-back surface.""" + + def __init__(self, *, count: int = QUEUE_HEAD_COUNT, max_writes: int = 1024): + self.count = int(count) + self.max_writes = int(max_writes) + self.heads = [0] * self.count + self.writes: list[dict] = [] + self.dropped = 0 + self.kicks = 0 + + def ring(self, queue: int, value: int = 0, *, pc: int | None = None) -> None: + queue = int(queue) + if not 0 <= queue < self.count: + raise ValueError(f"queue index {queue} outside 0..{self.count - 1}") + value = int(value) & 0xFFFFFFFF + self.heads[queue] = value + self.kicks += 1 + record = {"queue": queue, "value": value} + if pc is not None: + record["pc"] = int(pc) + if len(self.writes) < self.max_writes: + self.writes.append(record) + else: + self.dropped += 1 + + def head(self, queue: int) -> int: + queue = int(queue) + if not 0 <= queue < self.count: + raise ValueError(f"queue index {queue} outside 0..{self.count - 1}") + return self.heads[queue] + + def state(self) -> dict: + return { + "count": self.count, + "heads": list(self.heads), + "kicks": self.kicks, + "writes": [dict(entry) for entry in self.writes], + "dropped": self.dropped, + } + + def reset(self) -> None: + self.heads = [0] * self.count + self.writes = [] + self.dropped = 0 + self.kicks = 0 + + +class DoorbellBlock(Device): + """``NV_PGSP_QUEUE_HEAD`` window over a :class:`DoorbellModel`. + + Evidence: hardware register header (offset/width/access) for the register; + the model's read-back is storage of the last written value, and the + firmware's read use of this register is **not observed** (documented, not + claimed). Guest stores are recorded with their PC. + """ + + name = "gsp-doorbell" + wants_pc = True + + def __init__(self, doorbell: DoorbellModel | None = None, *, + base: int = DOORBELL_BASE, size: int = DOORBELL_SIZE): + super().__init__(base=base, size=size) + self.doorbell = doorbell if doorbell is not None else DoorbellModel() + + def reset(self) -> None: + if getattr(self, "doorbell", None) is None: + self.doorbell = DoorbellModel() + else: + self.doorbell.reset() + + def _queue(self, offset: int, size: int) -> int | None: + if size != 4 or offset % QUEUE_HEAD_STRIDE: + return None + queue = offset // QUEUE_HEAD_STRIDE + if not 0 <= queue < QUEUE_HEAD_COUNT: + return None + return queue + + def read(self, offset: int, size: int, pc: int = 0) -> int | None: + queue = self._queue(offset, size) + if queue is None: + return None + return self.doorbell.head(queue) + + def write(self, offset: int, size: int, value: int, pc: int = 0) -> bool: + queue = self._queue(offset, size) + if queue is None: + return False + self.doorbell.ring(queue, value, pc=pc) + return True + + def peek(self, offset: int, size: int) -> int | None: + return self.read(offset, size) + + def state(self) -> dict: + return {"doorbell": self.doorbell.state()} + + def load_state(self, data: dict) -> None: + state = data.get("doorbell", data) + self.doorbell.count = int(state.get("count", self.doorbell.count)) + self.doorbell.heads = [int(v) for v in state.get("heads", [0] * self.doorbell.count)] + self.doorbell.kicks = int(state.get("kicks", 0)) + self.doorbell.writes = [dict(entry) for entry in state.get("writes", [])] + self.doorbell.dropped = int(state.get("dropped", 0)) + + +# --------------------------------------------------------------------------- +# Recorder compatibility +# --------------------------------------------------------------------------- + + +class _NullRecorder: + """Stand-in for ``greeneye.record.NullRecorder`` (keeps sim standalone).""" + + dir: str | None = None + + def event(self, event_type: str, **fields) -> None: + pass + + def blob(self, name: str, data: bytes) -> str: + return name + + +# --------------------------------------------------------------------------- +# Host traffic loop +# --------------------------------------------------------------------------- + + +class HostTraffic: + """The modelled host-side transport: submit, dispatch (stub), read reply. + + ``machine`` (optional) supplies simulated RAM for the queue block and lets + :meth:`run_cycles` step the core; without it the queues live in a private + :class:`RegionMemory`. ``recorder`` (optional) is any object with + ``event(type, **fields)`` / ``blob(name, data)``; the transport transcript + stays deterministic regardless. + """ + + def __init__( + self, + *, + machine=None, + recorder=None, + shared_base: int = SHARED_MEM_BASE, + cmd_size: int = DEFAULT_CMD_QUEUE_SIZE, + stat_size: int = DEFAULT_STAT_QUEUE_SIZE, + msg_size: int = GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + handlers: dict[int, Callable[..., Any]] | None = None, + ): + if machine is None and shared_base == SHARED_MEM_BASE: + mem = RegionMemory(shared_base, cmd_size + stat_size) + else: + mem = _make_memory(machine, shared_base, cmd_size + stat_size) + self.machine = machine + self.recorder = recorder if recorder is not None else _NullRecorder() + self.shared_base = int(shared_base) + self.queues = SharedQueues( + mem, + cmd_base=shared_base, + cmd_size=cmd_size, + stat_size=stat_size, + msg_size=msg_size, + ) + self.queues.format() + self.doorbell = DoorbellModel() + self._tx_seq = 0 + self._rx_seq = 0 + self._next_sequence = 1 + self._submitted = 0 + self._dispatched = 0 + self._replies = 0 + self._events: list[dict] = [] + self._closed = False + self._handlers: dict[int, tuple[str, Callable[..., Any]]] = {} + self.set_handler(RPC_FN_ECHO, self._handle_echo, name="stub:echo") + self.set_handler(RPC_FN_STATUS_QUERY, self._handle_status_query, + name="stub:status_query") + for function, handler in (handlers or {}).items(): + self.set_handler(function, handler) + self._emit( + "host_traffic_config", + shared_base=self.shared_base, + cmd_base=self.queues.command.tx_base, + stat_base=self.queues.status.tx_base, + cmd_size=self.queues.command.size, + stat_size=self.queues.status.size, + msg_size=self.queues.command.msg_size, + handlers=sorted(self.handler_names().items()), + ) + + # -- handler table ------------------------------------------------------ + + def set_handler(self, function: int, handler: Callable[..., Any], *, + name: str | None = None) -> None: + label = name or getattr(handler, "__name__", "handler") + self._handlers[int(function)] = (label, handler) + + def handler_names(self) -> dict[int, str]: + return {function: label for function, (label, _) in sorted(self._handlers.items())} + + def _handle_echo(self, request: RpcRequest) -> RpcReply: + return RpcReply(function=request.function, sequence=request.sequence, + params=request.params) + + def _handle_status_query(self, request: RpcRequest) -> RpcReply: + payload = struct.pack( + STATUS_FORMAT, + STATUS_MAGIC, + STATUS_VERSION, + self._submitted, + self._dispatched, + self._replies, + ) + return RpcReply(function=request.function, sequence=request.sequence, + params=payload) + + # -- recording ---------------------------------------------------------- + + def _emit(self, event_type: str, **fields) -> None: + self.recorder.event(event_type, **fields) + + def _record(self, event: dict) -> None: + self._events.append(event) + + def note_skip(self, index: int, reason: str) -> None: + """Record a replay message that was not submitted (documented).""" + self._record({"type": "skip", "index": int(index), "reason": reason}) + self._emit("host_replay_skip", index=int(index), reason=reason) + + # -- host side ---------------------------------------------------------- + + def submit(self, message: bytes) -> dict: + """Write one framed RPC message to the command ring, then kick.""" + parsed = parse_rpc_message(message) + element = build_queue_element(message, sequence=self._tx_seq) + if len(element) > GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: + raise HostTrafficError( + f"message of {len(element)} bytes exceeds the wire element " + f"maximum of {GSP_MSG_QUEUE_ELEMENT_SIZE_MAX:#x}" + ) + placed = self.queues.submit_element(element) + self._tx_seq += 1 + self.doorbell.ring(DOORBELL_KICK_QUEUE, DOORBELL_KICK_VALUE) + self._submitted += 1 + self._emit("doorbell", queue=DOORBELL_KICK_QUEUE, + value=DOORBELL_KICK_VALUE, source="host") + result = { + "format": "greeneye-host-traffic-submit/1", + "index": placed["index"], + "write_ptr": placed["write_ptr"], + "function": parsed.function, + "sequence": parsed.sequence, + "msgq_sequence": self._tx_seq - 1, + "length": parsed.length, + "params_bytes": len(parsed.params), + "element_bytes": len(element), + "doorbell": {"queue": DOORBELL_KICK_QUEUE, "value": DOORBELL_KICK_VALUE}, + } + self._record({ + "type": "submit", + "index": placed["index"], + "function": parsed.function, + "sequence": parsed.sequence, + "params_bytes": len(parsed.params), + "element_bytes": len(element), + "write_ptr": placed["write_ptr"], + }) + self._emit( + "host_submit", + function=parsed.function, + sequence=parsed.sequence, + length=parsed.length, + params_bytes=len(parsed.params), + element_index=placed["index"], + write_ptr=placed["write_ptr"], + ) + return result + + def submit_rpc(self, function: int, params: bytes = b"", + *, sequence: int | None = None) -> dict: + """Frame and submit one RPC; ``sequence`` defaults to a local counter.""" + if sequence is None: + sequence = self._next_sequence + self._next_sequence = max(self._next_sequence, int(sequence) + 1) + return self.submit(build_rpc_message(function, params, sequence=sequence)) + + def _normalise_reply(self, produced: Any, request: RpcRequest) -> RpcReply: + if produced is None: + return RpcReply(function=request.function, sequence=request.sequence) + if isinstance(produced, RpcReply): + return produced + if isinstance(produced, (bytes, bytearray)): + return RpcReply(function=request.function, sequence=request.sequence, + params=bytes(produced)) + raise HostTrafficError( + f"handler returned {type(produced).__name__}; expected RpcReply, " + "bytes or None" + ) + + def poll(self) -> int: + """GSP side: drain the command ring through the dispatcher (bounded).""" + drained = 0 + while True: + taken = self.queues.take_command() + if taken is None: + break + prefix, request_message = parse_queue_element(taken["element"]) + label, handler = self._handlers.get( + request_message.function, ("stub:unsupported", None) + ) + self._dispatched += 1 + request = RpcRequest( + function=request_message.function, + sequence=request_message.sequence, + params=request_message.params, + element_index=taken["index"], + raw=bytes(taken["element"]), + ) + if handler is None: + reply = RpcReply( + function=request.function, + sequence=request.sequence, + params=b"", + result=NV_ERR_NOT_SUPPORTED, + ) + else: + reply = self._normalise_reply(handler(request), request) + reply_bytes = build_rpc_message( + reply.function, reply.params, sequence=reply.sequence, + result=reply.result, + ) + element = build_queue_element(reply_bytes, sequence=self._rx_seq) + self._rx_seq += 1 + if len(element) > GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: + raise HostTrafficError( + f"reply of {len(element)} bytes exceeds the wire element maximum" + ) + self.queues.publish_element(element) + self._replies += 1 + drained += 1 + self._record({ + "type": "dispatch", + "index": taken["index"], + "function": request.function, + "sequence": request.sequence, + "handler": label, + "result": reply.result, + "reply_bytes": len(reply.params), + }) + self._emit( + "host_dispatch", + function=request.function, + sequence=request.sequence, + handler=label, + result=reply.result, + reply_bytes=len(reply.params), + element_index=taken["index"], + msgq_sequence=prefix["seq_num"], + ) + return drained + + def read_reply(self) -> dict | None: + """Host side: consume the next status element, or ``None``.""" + taken = self.queues.take_status() + if taken is None: + return None + _prefix, message = parse_queue_element(taken["element"]) + record = { + "format": REPLY_FORMAT, + "index": taken["index"], + "function": message.function, + "sequence": message.sequence, + "result": message.rpc_result, + "params": message.params, + "params_hex": message.params.hex(), + "params_bytes": len(message.params), + } + self._record({ + "type": "reply", + "index": taken["index"], + "function": message.function, + "sequence": message.sequence, + "result": message.rpc_result, + "params_bytes": len(message.params), + }) + self._emit( + "host_reply_read", + function=message.function, + sequence=message.sequence, + result=message.rpc_result, + params_bytes=len(message.params), + element_index=taken["index"], + ) + return record + + def run_cycles(self, budget: int) -> dict: + """Run the attached core for a bounded ``budget``, then ``poll``.""" + if isinstance(budget, bool) or not isinstance(budget, int) or not 0 < budget <= MAX_RUN_CYCLES: + raise ValueError(f"budget must be an int in 1..{MAX_RUN_CYCLES}, got {budget!r}") + reason = pc = None + if self.machine is not None: + reason, pc, _a, _b = self.machine.run(budget) + dispatched = self.poll() + return { + "budget": budget, + "reason": reason, + "pc": pc, + "insns": self.machine.insns if self.machine is not None else 0, + "dispatched": dispatched, + "replies": self._replies, + } + + # -- evidence ----------------------------------------------------------- + + def transcript(self) -> dict: + counts: dict[str, int] = {} + for event in self._events: + counts[event["type"]] = counts.get(event["type"], 0) + 1 + return { + "format": TRANSCRIPT_FORMAT, + "events": [dict(event) for event in self._events], + "counts": counts, + } + + def state(self) -> dict: + return { + "format": TRANSCRIPT_FORMAT, + "shared_base": self.shared_base, + "queues": self.queues.state(), + "doorbell": self.doorbell.state(), + "counts": { + "submitted": self._submitted, + "dispatched": self._dispatched, + "replies": self._replies, + }, + "handlers": self.handler_names(), + } + + def doorbell_block(self) -> DoorbellBlock: + return DoorbellBlock(self.doorbell) + + def close(self) -> None: + if self._closed: + return + self._closed = True + payload = json.dumps(self.transcript(), indent=2, sort_keys=True).encode() + self.recorder.blob("host-traffic.json", payload) + self._emit("host_transcript", events=len(self._events)) + + +# --------------------------------------------------------------------------- +# Stimulus replay +# --------------------------------------------------------------------------- + + +def _stimulus_from_capture(source: str | dict) -> dict: + if _TOOL_ROOT not in sys.path: + sys.path.insert(0, _TOOL_ROOT) + try: + from harness.rm_capture import load_capture, stimulus_from_capture + except ImportError as exc: # pragma: no cover - tool checkout broken + raise HostTrafficError(f"harness.rm_capture unavailable: {exc}") from exc + if isinstance(source, dict): + raise HostTrafficError( + "an inline capture document is not accepted; pass the capture JSON " + "path or a stimulus document" + ) + capture = load_capture(source) + return stimulus_from_capture(capture) + + +def load_stimulus(source: str | dict) -> dict: + """Load a replay stimulus from a doc, a stimulus JSON or a capture file.""" + if isinstance(source, dict): + doc = source + else: + if not os.path.isfile(source): + raise HostTrafficError(f"stimulus not found: {source}") + with open(source, "r", encoding="utf-8", errors="replace") as fh: + head = fh.read(4096) + if not head.lstrip().startswith("{"): + return _stimulus_from_capture(source) + try: + with open(source, "r", encoding="utf-8", errors="replace") as fh: + doc = json.load(fh) + except (OSError, json.JSONDecodeError) as exc: + raise HostTrafficError(f"cannot read stimulus {source}: {exc}") from exc + if not isinstance(doc, dict): + raise HostTrafficError("stimulus document must be an object") + fmt = str(doc.get("format") or "") + if fmt == STIMULUS_FORMAT: + return doc + if fmt == CAPTURE_FORMAT: + return _stimulus_from_capture(source) + raise HostTrafficError( + f"unknown stimulus/capture format {fmt!r}; expected {STIMULUS_FORMAT!r} " + f"or {CAPTURE_FORMAT!r}" + ) + + +def _message_to_rpc(message: dict, index: int) -> tuple[int, bytes]: + """Map one stimulus message to ``(wire function, payload)``. + + ``send`` messages carry the wire function and payload directly. ``control`` + and ``alloc`` records carry a named id instead; the capture tool's mapping + (control -> 76, alloc -> 103) is applied, and the captured parameter block + is used as the payload. That identity is an assumption until a capture + with full send payloads exists (documented in docs/51). + """ + kind = message.get("kind") + function = message.get("fn") + if function is None: + if kind == "control": + function = GSP_RM_CONTROL + elif kind == "alloc": + function = GSP_RM_ALLOC + if function is None: + raise HostTrafficError(f"message {index}: no wire function id") + hex_text = message.get("params_hex") or "" + try: + params = bytes.fromhex(hex_text) + except ValueError as exc: + raise HostTrafficError(f"message {index}: bad params_hex: {exc}") from exc + return int(function), params + + +def replay_stimulus(host: HostTraffic, stimulus: str | dict, *, + strict: bool = True) -> dict: + """Submit a stimulus document's messages in order, then read the replies.""" + doc = load_stimulus(stimulus) + messages = doc.get("messages") + if not isinstance(messages, list): + raise HostTrafficError("stimulus document has no 'messages' list") + pairs: list[dict] = [] + skipped: list[dict] = [] + for index, message in enumerate(messages): + if not isinstance(message, dict): + raise HostTrafficError(f"message {index} is not an object") + if message.get("truncated"): + reason = "truncated payload is not replayable as-is" + if strict: + raise HostTrafficError(f"message {index}: {reason}") + host.note_skip(index, reason) + skipped.append({"index": index, "reason": reason}) + continue + function, params = _message_to_rpc(message, index) + submit = host.submit_rpc(function, params) + dispatched = host.poll() + reply = host.read_reply() + if reply is None or dispatched != 1: + raise HostTrafficError( + f"message {index}: transport did not produce exactly one reply" + ) + pairs.append({ + "seq": message.get("seq", index), + "kind": message.get("kind"), + "function": function, + "submit": submit, + "reply": reply, + }) + return { + "format": REPLAY_FORMAT, + "source": doc.get("source"), + "submitted": len(pairs), + "skipped": skipped, + "pairs": pairs, + "transcript": host.transcript(), + "limits": list(_LIMITS), + } + + +# --------------------------------------------------------------------------- +# Demo +# --------------------------------------------------------------------------- + + +def _new_recorder(out_dir: str | None, name: str): + if out_dir is None: + return _NullRecorder(), False + from greeneye.record import Recorder # deferred: keep sim importable alone + + meta = {"kind": name, "host_traffic": True} + return Recorder(out_dir, name=name, meta=meta), True + + +def _json_reply(reply: dict | None) -> dict | None: + if reply is None: + return None + return { + "format": reply["format"], + "index": reply["index"], + "function": reply["function"], + "sequence": reply["sequence"], + "result": reply["result"], + "params_hex": reply["params_hex"], + "params_bytes": reply["params_bytes"], + "read": True, + } + + +def run_demo(out_dir: str | None = None) -> dict: + """Deterministic end-to-end transport demo; no capture, no firmware.""" + from .machine import Machine # deferred (module import stays light) + + machine = Machine(memory_mb=32) + machine.load_segments( + [(machine.ram_base, bytes.fromhex("73001000"))], entry=machine.ram_base + ) + recorder, owns = _new_recorder(out_dir, "host-traffic-demo") + host = HostTraffic(machine=machine, recorder=recorder) + replies: list[dict] = [] + for sequence, (function, params) in enumerate( + ( + (RPC_FN_ECHO, b"\x01\x02\x03\x04"), + (RPC_FN_STATUS_QUERY, b""), + ), + start=1, + ): + host.submit_rpc(function, params, sequence=sequence) + host.run_cycles(64) + reply = host.read_reply() + if reply is not None: + replies.append(_json_reply(reply)) + + control_cmd = 0x2080012B + control_params = ( + (0xC2000006).to_bytes(4, "little") + + (0xABCD0080).to_bytes(4, "little") + + control_cmd.to_bytes(4, "little") + + bytes(500) + ) + host.submit_rpc(GSP_RM_CONTROL, control_params, sequence=3) + host.run_cycles(64) + control_reply = host.read_reply() + if control_reply is not None: + replies.append(_json_reply(control_reply)) + + checks = { + "echo_reply_matches_payload": ( + replies[0]["function"] == RPC_FN_ECHO + and replies[0]["params_hex"] == b"\x01\x02\x03\x04".hex() + and replies[0]["result"] == NV_OK + ), + "status_reply_has_magic": ( + len(replies) > 1 + and bytes.fromhex(replies[1]["params_hex"])[:4] == STATUS_MAGIC + ), + "control_replay_is_512_bytes": host.state()["counts"]["submitted"] == 3, + "control_reply_is_a_framed_stub_verdict": ( + control_reply is not None + and control_reply["function"] == GSP_RM_CONTROL + and control_reply["result"] == NV_ERR_NOT_SUPPORTED + ), + "doorbell_ring_per_submit": host.doorbell.kicks == 3, + "all_replies_read": len(replies) == 3, + } + control = { + "function": GSP_RM_CONTROL, + "cmd": control_cmd, + "bytes": len(control_params), + "reply_function": control_reply["function"] if control_reply else None, + "reply_result": control_reply["result"] if control_reply else None, + "reply_params_bytes": control_reply["params_bytes"] if control_reply else None, + } + payload = { + "format": DEMO_FORMAT, + "ok": all(checks.values()), + "checks": checks, + "machine": { + "ram_base": machine.ram_base, + "insns": machine.insns, + "cycles": machine.cycles, + }, + "shared_base": host.shared_base, + "queue": host.queues.command.state(), + "control": control, + "replies": replies, + "transcript": host.transcript(), + "limits": list(_LIMITS), + } + host.close() + if owns: + os.makedirs(out_dir, exist_ok=True) + payload["artifacts"] = {"session": os.path.abspath(out_dir)} + with open(os.path.join(out_dir, "host-traffic-demo.json"), "w", + encoding="utf-8") as fh: + json.dump(payload, fh, indent=1, sort_keys=True) + fh.write("\n") + recorder.close() + return payload + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="sim.host_traffic", + description=( + "Modelled host traffic loop for the GSP message queues: doorbell, " + "command/status rings, stub dispatch and rm_capture stimulus " + "replay. Offline only; the simulator runs no real firmware here." + ), + ) + sub = parser.add_subparsers(dest="command") + demo_cmd = sub.add_parser("demo", help="deterministic end-to-end demo") + demo_cmd.add_argument("--out", help="session/output directory (optional)") + demo_cmd.add_argument("--json", action="store_true", help="print full JSON") + replay_cmd = sub.add_parser("replay", help="replay an rm_capture stimulus") + replay_cmd.add_argument("--stimulus", required=True, + help="stimulus JSON, capture JSON or raw capture log") + replay_cmd.add_argument("--out", help="session/output directory (optional)") + replay_cmd.add_argument("--allow-truncated", action="store_true", + help="skip truncated messages instead of refusing") + replay_cmd.add_argument("--json", action="store_true", help="print full JSON") + return parser + + +def _cli_summary(payload: dict) -> dict: + summary = { + "format": payload.get("format"), + "ok": payload.get("ok"), + } + if payload.get("checks") is not None: + summary["checks"] = payload["checks"] + if "submitted" in payload: + summary["submitted"] = payload["submitted"] + summary["skipped"] = len(payload.get("skipped", [])) + if payload.get("artifacts"): + summary["artifacts"] = payload["artifacts"] + return summary + + +def _jsonable(value): + if isinstance(value, bytes): + return value.hex() + if isinstance(value, (list, tuple)): + return [_jsonable(item) for item in value] + if isinstance(value, dict): + return {str(key): _jsonable(item) for key, item in value.items()} + return value + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + if args.command is None: + parser.print_help(sys.stderr) + return 2 + try: + if args.command == "demo": + payload = run_demo(args.out) + else: + recorder, owns = _new_recorder(args.out, "host-traffic-replay") + host = HostTraffic(recorder=recorder) + payload = replay_stimulus( + host, args.stimulus, strict=not args.allow_truncated + ) + host.close() + if owns: + os.makedirs(args.out, exist_ok=True) + payload["artifacts"] = {"session": os.path.abspath(args.out)} + with open(os.path.join(args.out, "host-traffic-replay.json"), "w", + encoding="utf-8") as fh: + json.dump(_jsonable(payload), fh, indent=1, sort_keys=True) + fh.write("\n") + recorder.close() + except HostTrafficError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + if args.json: + print(json.dumps(_jsonable(payload), indent=2, sort_keys=True)) + else: + print(json.dumps(_jsonable(_cli_summary(payload)), indent=2, sort_keys=True)) + if args.command == "demo" and not payload.get("ok"): + return 3 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/debug/risc-v-gsp-rm-vm/sim/libos_port.py b/debug/risc-v-gsp-rm-vm/sim/libos_port.py new file mode 100644 index 0000000..eaa3f95 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sim/libos_port.py @@ -0,0 +1,386 @@ +"""libos port receive-path model (the B5 message-loop blocker, first half). + +Status: **offline development tool (new, additive)**. This module models the +part of the GSP firmware's libos *port receive* machinery that the B3 +message-loop task needs before it can make progress: the argument contract of +``TASK_PORT_SEND_RECV_WAIT`` (syscall 1), the asynchronous receive post, the +synchronous send/reply, the cross-payload handoff performed when a receive +completes, and the external-interrupt port retirement that a host doorbell wake +performs. + +Evidence kind: **firmware use + RTOS port implementation** (the same evidence +class as :mod:`sim.libos`). Every structure below was observed on the real +570.144 image, not inferred from a stub: + +* the message-loop task posts four asynchronous receives through a wrapper at + firmware vaddr ``0x4018000`` that moves its arguments ``a0-a3`` into + ``a4-a7``, zeroes ``a0-a3`` and ``t1``/``t2``, and executes ``ecall`` with + ``t0 = 1`` (an async receive post): + ``(shuttle 10, port 6, payload 0, size 0)`` immediately followed by + ``INTERRUPT_ARM_EXTERNAL(0xFF)`` -- the external-interrupt receive; + ``(shuttle 11, port 7, payload 0x5ba8e10, size 4)``; + ``(shuttle 9, port 8, payload 0x4015070, size 0x808)``; + ``(shuttle 14, port 13, payload 0x5ba8e20, size 0x24)``; +* it then performs one synchronous send on the default shuttle id (1) to port + 12 with a zero-length payload, ``waitId = 1``, ``timeout = -1`` (infinite); +* it blocks in ``PortWait(LIBOS_SHUTTLE_ANY, timeout = -1)`` (ecall at + ``0x4019df4``); +* on an external interrupt the port that was armed retires with + ``error_code = LIBOS_OK`` and ``payload_size = 0`` (the report path), and the + wait returns with ``a0 = status``, ``a4 = completedSize``, + ``a5 = completedRemoteTaskId``, ``a6 = completedShuttle``. + +The argument contract modelled (mirrors the RTOS ``libosPortSendRecvAndWait``): + +=============== ========================================================= +register meaning at ``ecall`` +=============== ========================================================= +``t0`` syscall number (1) +``a0`` send shuttle id (0 = none) +``a1`` send port id +``a2``/``a3`` send payload address / size +``a4`` receive shuttle id (0 = none) +``a5`` receive port id +``a6``/``a7`` receive payload address / capacity +``t1`` wait id (0 = do not block, ``0xFFFF`` = any) +``t2`` timeout in ns (``0xFFFFFFFFFFFFFFFF`` = infinite) +=============== ========================================================= + +On return: ``a0`` = status, ``a4`` = completed size, ``a5`` = originating task +id, ``a6`` = completed shuttle (local handle). + +Honesty rules (mandatory; docs/51 section 6, docs/06-ACCURACY.md): + +* the model is **data-driven by observation**: port roles come from the posted + receive sequence of the real image; nothing infers a port number that was + not observed; +* every modelled event carries ``"modelled": true`` and a label -- the host + doorbell wake is ``shim:host-doorbell-wake``, the boot handshake send is + ``modelled:peer-consumes-on-send``. These are not firmware behaviour; +* **receives are never completed with invented payloads**: a wake only retires + the external-interrupt receive with the RTOS report contract (rare, zero + length). No command, status or event payload is fabricated; +* wait semantics are strictly "return the first completed shuttle or stop"; + the model never returns from a blocking wait that nothing completed; +* L0/L1 only: no timing (L2) and no device-environment (L3) claim; with a + single modelled task there is no scheduler and no other task is run. + +Development-only, offline: no GPU, kernel module, mdev, guest, network, QEMU, +``/dev`` access or reboot. Nothing here is a product dependency and nothing +is injected into a guest. +""" +from __future__ import annotations + +from sim.libos import ( + INTERRUPT_ARM_EXTERNAL, + NAMES, + TASK_PORT_SEND_RECV_WAIT, +) + +#: Report schema tag for :meth:`LibosPortModel.state`. +REPORT_FORMAT = "greeneye-msgq-port/1" + +#: ``LIBOS_SHUTTLE_ANY`` (RTOS port API). +SHUTTLE_ANY = 0xFFFF +#: The task's default shuttle id (RTOS port API). +SHUTTLE_DEFAULT = 1 +#: ``LIBOS_TIMEOUT_INFINITE``. +TIMEOUT_INFINITE = 0xFFFFFFFFFFFFFFFF +#: LIBOS_OK. +LIBOS_OK = 0 + +#: Observed port roles for the real 570.144 message-loop task. The values are +#: the port ids carried by the task's posted receives (resource handles); the +#: labels describe behaviour only. +PORT_ROLES = { + 6: "extintr", # external-interrupt receive (armed with 0xFF) + 7: "notification", # 4-byte notification buffer + 8: "request", # 0x808-byte request/reply buffer + 12: "handshake", # zero-length boot handshake send target + 13: "event", # 36-byte event buffer +} + +#: Port label used when a port was not part of the observed post sequence. +ROLE_DATA = "data" + +#: The wake contract observed for the external-interrupt receive: the RTOS +#: report retires the waiting shuttle with error LIBOS_OK and a zero payload. +EXTINTR_WAKE_CONTRACT = "shim:host-doorbell-wake" +HANDSHAKE_CONTRACT = "modelled:peer-consumes-on-send" + + +def role_for(port: int) -> str: + """Role label for a port id (observed table; ``"data"`` otherwise).""" + return PORT_ROLES.get(int(port), ROLE_DATA) + + +class LibosPortModel: + """Deterministic model of the libos receive path for one GSP task. + + The model is a ``Machine.syscall_handler``: it owns syscall 1 + (``TASK_PORT_SEND_RECV_WAIT``) and the interrupt-arm call, and delegates + every other syscall to ``base`` (:class:`sim.libos.LibosSyscalls`, which + keeps recording). ``machine.x`` is the only machine state touched. + """ + + name = "libos-port-model" + + def __init__(self, base=None, *, task: str = "gsp-rm-task"): + self.base = base + self.task = task + #: Async receive posts in observation order. + self.posts: list[dict] = [] + #: Pending receives by shuttle id. + self.pending: dict[int, dict] = {} + #: Completed handoffs awaiting a wait (FIFO). + self.completed: list[dict] = [] + #: Synchronous sends observed (send payloads are not copied; the peer + #: side is not modelled). + self.sends: list[dict] = [] + #: Blocking waits observed. + self.waits: list[dict] = [] + #: Labelled model events (every one is explicitly ``modelled``). + self.events: list[dict] = [] + #: External interrupt mask last armed (RTOS ``IRQMSET`` semantics). + #: An installed base model may already have observed an arm before this + #: model was attached (the recorded boot arm), so seed from it. + self.armed_mask = int(getattr(base, "armed_mask", 0)) + #: Recorded host doorbell wakes. + self.host_kicks = 0 + #: Host wakes not yet handed to a receive (the firmware may post and + #: arm after the host has rung the doorbell; the pending interrupt is + #: delivered at the first armed receive). + self.kicks_pending = 0 + #: The request that stopped the run (unmodelled or blocking), if any. + self.stopped: dict | None = None + + # -- observation ------------------------------------------------------- + + def _event(self, name: str, **fields) -> dict: + event = {"event": name, "modelled": True, "task": self.task} + event.update(fields) + self.events.append(event) + return event + + def host_kick(self, *, label=EXTINTR_WAKE_CONTRACT, queue=None, value=None, + bit=None) -> dict: + """Record the host doorbell write; deliver it when a receive is armed. + + The host submit path writes ``NV_PGSP_QUEUE_HEAD(0)``; the mapping from + that write to a falcon interrupt bit is **not modelled** (unobserved). + The wake is delivered to the external-interrupt receive once it has + been posted and interrupts are armed (the firmware may do both after + the host has already rung). Returns the event; ``"delivered"`` is the + retired shuttle id, or ``None`` when the wake remains pending. + """ + self.host_kicks += 1 + self.kicks_pending += 1 + delivered = self._try_kick() + return self._event( + "host_kick", + contract=label, + queue=queue, + value=value, + falcon_irq_bit=bit, + delivered=delivered, + pending=bool(self.kicks_pending), + note=("queue-head write -> falcon interrupt bit is unobserved; " + "the wake is the documented shim"), + ) + + def _try_kick(self): + """Deliver one pending host wake to an armed extintr receive, if any.""" + if not self.kicks_pending or not self.armed_mask: + return None + for shuttle in sorted(self.pending): + post = self.pending[shuttle] + if post["role"] != "extintr": + continue + self.kicks_pending -= 1 + self._complete( + shuttle, + payload_size=0, + origin_task=0, + contract=EXTINTR_WAKE_CONTRACT, + reason="external-interrupt report (armed mask " + f"{self.armed_mask:#x}, pending host doorbell wake)", + ) + return shuttle + return None + + def _complete(self, shuttle: int, *, payload_size: int, origin_task: int, + contract: str, reason: str) -> dict: + post = self.pending.pop(int(shuttle)) + entry = { + "shuttle": int(shuttle), + "port": int(post["port"]), + "role": post["role"], + "payload": int(post["payload"]), + "capacity": int(post["capacity"]), + "payload_size": int(payload_size), + "origin_task": int(origin_task), + "contract": contract, + "reason": reason, + "post_pc": int(post["pc"]), + } + self.completed.append(entry) + self._event("receive_complete", **entry) + return entry + + # -- syscall surface --------------------------------------------------- + + def handle(self, machine, pc: int = 0): + """``Machine`` syscall hook: ``(handled, result)``.""" + num = int(machine.x[5]) + if num == TASK_PORT_SEND_RECV_WAIT: + return self._handle_port(machine, pc) + if num == INTERRUPT_ARM_EXTERNAL: + self.armed_mask |= int(machine.x[10]) + self._event("interrupt_arm_external", mask=int(machine.x[10]), pc=pc) + self._try_kick() + if self.base is not None: + handled, result = self.base.handle(machine, pc) + else: + handled, result = (True, LIBOS_OK) if num == INTERRUPT_ARM_EXTERNAL \ + else (False, None) + if not handled and self.stopped is None: + entry = getattr(self.base, "stopped", None) + number = int(entry["num"]) if entry else num + self.stopped = { + "num": number, + "name": (entry or {}).get("name", NAMES.get(number, f"unknown_{number}")), + "pc": int(pc), + "reason": "unmodelled-syscall", + } + return handled, result + + def _handle_port(self, machine, pc: int): + a0, a1, a2, a3 = (int(machine.x[10]), int(machine.x[11]), + int(machine.x[12]), int(machine.x[13])) + a4, a5, a6, a7 = (int(machine.x[14]), int(machine.x[15]), + int(machine.x[16]), int(machine.x[17])) + t1, t2 = int(machine.x[6]), int(machine.x[7]) + + send = a0 != 0 or a1 != 0 + recv = a4 != 0 or a5 != 0 + + if recv: + self._post(shuttle=a4, port=a5, payload=a6, capacity=a7, pc=pc, + wait_id=t1, timeout=t2) + if send: + self._send(shuttle=a0, port=a1, payload=a2, size=a3, wait_id=t1, + timeout=t2, pc=pc, machine=machine) + return True, LIBOS_OK + if recv: + # Async receive post: the kernel returns LIBOS_OK without touching + # the wait outputs (a4-a6 still hold the receive arguments). + return True, LIBOS_OK + + # Pure blocking wait: return the first completed shuttle, or stop. + if not self.completed: + self._try_kick() + if self.completed: + entry = self.completed.pop(0) + machine.x[14] = entry["payload_size"] & 0xFFFFFFFFFFFFFFFF + machine.x[15] = entry["origin_task"] & 0xFFFFFFFFFFFFFFFF + machine.x[16] = entry["shuttle"] & 0xFFFFFFFFFFFFFFFF + machine.x[17] = 0 + record = { + "pc": int(pc), + "wait_id": t1, + "timeout": t2, + "completed": entry["shuttle"], + "payload_size": entry["payload_size"], + "origin_task": entry["origin_task"], + "role": entry["role"], + "contract": entry["contract"], + } + self.waits.append(record) + self._event("wait_complete", **record) + return True, LIBOS_OK + + record = { + "pc": int(pc), + "wait_id": t1, + "timeout": t2, + "reason": "port-wait-blocks", + "pending": sorted(self.pending), + } + self.waits.append(record) + self._event("wait_block", **record) + if self.stopped is None: + self.stopped = { + "num": TASK_PORT_SEND_RECV_WAIT, + "name": "task_port_send_recv_wait", + "pc": int(pc), + "reason": "port-wait-blocks", + "pending": sorted(self.pending), + } + return False, None + + def _post(self, *, shuttle: int, port: int, payload: int, capacity: int, + pc: int, wait_id: int, timeout: int) -> dict: + post = { + "shuttle": int(shuttle), + "port": int(port), + "role": role_for(port), + "payload": int(payload), + "capacity": int(capacity), + "pc": int(pc), + "wait_id": int(wait_id), + "timeout": int(timeout), + } + self.posts.append(post) + self.pending[int(shuttle)] = post + self._event("receive_post", **post) + self._try_kick() + return post + + def _send(self, *, shuttle: int, port: int, payload: int, size: int, + wait_id: int, timeout: int, pc: int, machine) -> dict: + """Model a synchronous send/reply as completed by its peer. + + Only the boot handshake shape was observed (zero-length payload to port + 12 on the default shuttle). The peer is not modelled: consumption is + an explicit, labelled assumption. The wait outputs receive the RTOS + transcribe contract for a completed send (``a4`` = copied size, + ``a5`` = 0, ``a6`` = send shuttle). + """ + record = { + "shuttle": int(shuttle), + "port": int(port), + "role": role_for(port) if port else "reply", + "payload": int(payload), + "size": int(size), + "pc": int(pc), + "wait_id": int(wait_id), + "timeout": int(timeout), + "contract": HANDSHAKE_CONTRACT, + } + self.sends.append(record) + machine.x[14] = int(size) & 0xFFFFFFFFFFFFFFFF + machine.x[15] = 0 + machine.x[16] = int(shuttle) & 0xFFFFFFFFFFFFFFFF + machine.x[17] = 0 + self._event("send_complete", **record) + return record + + # -- report surface ---------------------------------------------------- + + def state(self) -> dict: + """Deterministic serialisable state for the ``greeneye-msgq-attach/1`` + report (no host timestamps).""" + return { + "format": REPORT_FORMAT, + "task": self.task, + "armed_mask": int(self.armed_mask), + "host_kicks": int(self.host_kicks), + "kicks_pending": int(self.kicks_pending), + "posts": [dict(post) for post in self.posts], + "pending": sorted(self.pending), + "completed": [dict(entry) for entry in self.completed], + "sends": [dict(record) for record in self.sends], + "waits": [dict(record) for record in self.waits], + "events": [dict(event) for event in self.events], + "stopped": dict(self.stopped) if self.stopped else None, + } diff --git a/debug/risc-v-gsp-rm-vm/sim/msgq_attach.py b/debug/risc-v-gsp-rm-vm/sim/msgq_attach.py new file mode 100644 index 0000000..f1530bd --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sim/msgq_attach.py @@ -0,0 +1,1122 @@ +"""Message-queue attach: real 570.144 GSP boot to B3 + host ring attach (B3/B4 seam). + +Status: **offline development tool (new, additive)**. This module implements +steps 1--3 of ``docs/51-HOST-TRAFFIC-LOOP.md`` section 7 ("the next step to +B5"): + +1. boot the real 570.144 image to B3 -- the message-loop port wait, the + ``task_port_send_recv_wait`` ecall at ``0x4018038`` -- reusing + :mod:`sim.gsp_boot`'s profile, boot shim and libos service model; +2. attach :class:`sim.host_traffic.DoorbellBlock` and the command/status rings + to the machine **before** the boot run, so the firmware's first queue-head + access is served by the model and recorded with its PC; +3. with ``--doorbell``, submit a real-shaped ``GSP_RM_CONTROL`` from the host + model, wake the port wait exactly once, continue in bounded instructions, + and classify the first ring access the firmware performs (address, width, + value read, and whether it looks like a queue head / Tx header / Rx header / + entry). By default the continuation runs under + :class:`sim.libos_port.LibosPortModel` (the ``--no-port-model`` flag keeps + the first wave's single-wake shim): the model reproduces the observed + async receive posts, the boot handshake send and the host wake's + external-interrupt handoff, and reports the exact unmodelled semantic that + stops the run (``report["missing_semantic"]``) instead of manufacturing a + poll. + +Schema: ``greeneye-msgq-attach/1`` (:func:`assemble_report`). + +Honesty rules (mandatory, docs/51 section 6, docs/06-ACCURACY.md): + +* the real image is executed; nothing here fakes a poll, a reply or a + completion. If the firmware does not reach the wait, faults, or never + touches the rings, the report says so and records the first divergence -- + a run that does not reach the loop is never presented as success; +* the port-wait wake is a documented shim, labelled in + ``report["continuation"]["wake_log"]``; the wake result is not firmware + behaviour and no RM semantics are claimed. Under the port model the wake + uses the RTOS external-interrupt report contract (error 0, zero payload); + without it the first-wave single-wake shim is used; +* the queue layout and arithmetic are the :mod:`sim.host_traffic` models + (real msgq arithmetic), but the queue base addresses are the fixture block; + the real addresses live in ``GSP_ARGUMENTS_CACHED``/queue-init arguments, + which the simulator does not parse yet; +* the dispatcher in :mod:`sim.host_traffic` is a stub and is not driven here: + the tool records what the *firmware* does, not a stub verdict; +* L0/L1 only: no timing (L2) and no device-environment (L3) claim; the cycle + count is the modeled :mod:`sim.cycles` accounting, not a measurement. + +Development-only, offline: no GPU, kernel module, mdev, guest, network, QEMU, +``/dev`` access or reboot. Nothing here is a product dependency and nothing +is injected into a guest. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import sys +from dataclasses import dataclass + +try: # prefer the harness's established image/profile default (docs/51 seam) + from harness.rehost import DEFAULT_IMAGE, DEFAULT_PROFILE +except Exception: # pragma: no cover - harness import is not always on path + from sim.gsp_boot import DEFAULT_IMAGE, DEFAULT_PROFILE + +from greeneye.device_map import firmware_sha256 +from greeneye.elf import load_elf +from greeneye.sim_backend import launch_native +from sim.decode import K_LOAD, LOAD_SIZE, decode16, decode32 +from sim.devices import load_profile, registry_from_profile +from sim.gsp_boot import ( + MAX_ATTEMPT_INSTRUCTIONS, + MILESTONES, + STOP_REASONS, + _apply_boot_shim, + _rung_from_run, +) +from sim.host_traffic import ( + DOORBELL_BASE, + DOORBELL_SIZE, + GSP_MSG_QUEUE_ELEMENT_HDR_SIZE, + GSP_RM_CONTROL, + MSGQ_RX_HDR_OFF, + MSGQ_TX_HEADER_BYTES, + HostTraffic, + parse_queue_element, +) +from sim.libos import LibosSyscalls, TASK_PORT_SEND_RECV_WAIT, TASK_STATE_QUERY +from sim.libos_port import LibosPortModel +from sim.machine import R_BP, R_BUDGET, R_ECALL, R_FAULT, R_STEP, R_WATCH + +#: Report schema tag. Every session/JSON artifact carries it. +REPORT_FORMAT = "greeneye-msgq-attach/1" + +#: ``run_cycles``-style cap: same ceiling as the boot runner's attempts. +MAX_RUN_CYCLES = MAX_ATTEMPT_INSTRUCTIONS +#: Default total instruction budget for one attach run. +DEFAULT_MAX_CYCLES = MAX_ATTEMPT_INSTRUCTIONS +#: Demo budget: boot-to-B3 needs ~474k instructions; leave a small tail. +DEMO_MAX_CYCLES = 600_000 +#: Bounded evidence: the first ring access plus up to 16 subsequent ones. +MAX_POLLS_RECORDED = 17 +#: Internal stop code: the bounded poll list was filled before the run ended. +R_POLL_LIMIT = 199 + +#: Doorbell label (block-relative register name; public Turing dev_gsp.h). +DOORBELL_WINDOW = "PRIV+0x110c00" + +#: The real-shaped control used for the host kick (same shape as the docs/51 +#: demo control; real-shaped, from no capture). +CONTROL_CMD = 0x2080012B + +#: ``msgqTxHeader`` field names by byte offset (public queue header). +_TX_FIELDS = { + 0: "version", + 4: "size", + 8: "msgSize", + 12: "msgCount", + 16: "writePtr", + 20: "flags", + 24: "rxHdrOff", + 28: "entryOff", +} + +_LIMITS = [ + "The real 570.144 image is executed; no poll, reply or completion is " + "faked. A run that does not reach the message-loop wait reports its first " + "divergence instead of success.", + "The port-wait wake is a documented shim (the libos message-port delivery " + "path is not modelled); wake events carry an explicit label.", + "The port model (sim.libos_port) reproduces the observed receive posts, " + "the boot handshake send and the external-interrupt handoff only; it " + "fabricates no message payload and schedules no second task. The exact " + "unmodelled semantic that stops the run is reported in missing_semantic.", + "Queue base addresses are the sim.host_traffic fixture block; the real " + "addresses live in GSP_ARGUMENTS_CACHED/queue-init arguments, which are " + "not parsed yet.", + "No RM semantics: the sim.host_traffic stub dispatcher is not driven by " + "this tool, so no reply is claimed.", + "L0/L1 only: no timing (L2) and no device-environment (L3) claim; cycle " + "numbers are modeled accounting (sim.cycles), not measurements.", +] + + +class MsgqAttachError(Exception): + """An attach input (image, profile or budget) that cannot be run.""" + + +def _as_int(value, default=0): + if value is None: + return default + if isinstance(value, str): + return int(value, 0) + return int(value) + + +def _sha256_text(text: str) -> str: + return hashlib.sha256(text.encode("utf-8")).hexdigest() + + +# --------------------------------------------------------------------------- +# Ring geometry and the pure poll classifier +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class RingGeometry: + """Physical layout of the attached doorbell and command/status rings. + + All fields are plain integers so the classifier is testable without a + machine: ``classify_ring_access`` takes one of these plus an access. + """ + + shared_base: int + cmd_base: int + stat_base: int + cmd_size: int + stat_size: int + msg_size: int + entry_off: int + rx_hdr_off: int + doorbell_base: int + doorbell_size: int + + def to_dict(self) -> dict: + return { + "shared_base": self.shared_base, + "cmd_base": self.cmd_base, + "stat_base": self.stat_base, + "cmd_size": self.cmd_size, + "stat_size": self.stat_size, + "msg_size": self.msg_size, + "entry_off": self.entry_off, + "rx_hdr_off": self.rx_hdr_off, + "doorbell_base": self.doorbell_base, + "doorbell_size": self.doorbell_size, + } + + +def ring_geometry_from_host(host: HostTraffic) -> RingGeometry: + """Read the attached host's queue/doorbell layout into a pure geometry.""" + command = host.queues.command + header = command.header() + return RingGeometry( + shared_base=int(host.shared_base), + cmd_base=int(command.tx_base), + stat_base=int(host.queues.status.tx_base), + cmd_size=int(command.size), + stat_size=int(host.queues.status.size), + msg_size=int(header.msg_size), + entry_off=int(header.entry_off), + rx_hdr_off=int(MSGQ_RX_HDR_OFF), + doorbell_base=int(DOORBELL_BASE), + doorbell_size=int(DOORBELL_SIZE), + ) + + +def classify_ring_access(addr: int, width, value, geometry: RingGeometry) -> dict: + """Classify one firmware access against the ring geometry (pure). + + Returns ``{addr, width, value, in_ring, kind, ...}`` where ``kind`` is one + of ``queue-head`` (the doorbell register), ``tx-header``, ``rx-header``, + ``entry``, ``ring-gap`` or ``other``. The Rx-header note carries the real + ``MSGQ_FLAGS_SWAP_RX`` cross-link: the header physically in the command + buffer at ``+rx_hdr_off`` is the *status* queue's read pointer and vice + versa. + """ + addr = int(addr) + entry = { + "addr": addr, + "width": width, + "value": value, + "in_ring": False, + "kind": "other", + } + if geometry.doorbell_base <= addr < geometry.doorbell_base + geometry.doorbell_size: + offset = addr - geometry.doorbell_base + queue = offset // 8 if offset % 8 == 0 else None + entry.update({ + "in_ring": True, + "kind": "queue-head", + "window": DOORBELL_WINDOW, + "offset": offset, + }) + if queue is not None and 0 <= queue < geometry.doorbell_size // 8: + entry["queue"] = queue + entry["register"] = f"NV_PGSP_QUEUE_HEAD({queue})" + return entry + for buffer_name, base, size in ( + ("command", geometry.cmd_base, geometry.cmd_size), + ("status", geometry.stat_base, geometry.stat_size), + ): + if not base <= addr < base + size: + continue + offset = addr - base + entry.update({"in_ring": True, "buffer": buffer_name}) + if offset < MSGQ_TX_HEADER_BYTES: + entry.update({ + "kind": "tx-header", + "field_offset": offset, + "field": _TX_FIELDS.get(offset, "reserved"), + }) + elif offset == geometry.rx_hdr_off: + owner = "status" if buffer_name == "command" else "command" + entry.update({ + "kind": "rx-header", + "cross_linked_owner": owner, + "semantics": "readPtr", + }) + elif offset >= geometry.entry_off: + relative = offset - geometry.entry_off + slot = relative % geometry.msg_size + entry.update({ + "kind": "entry", + "entry_index": relative // geometry.msg_size, + "slot_offset": slot, + "region": "entry-header" if slot < GSP_MSG_QUEUE_ELEMENT_HDR_SIZE + else "entry-payload", + }) + else: + entry["kind"] = "ring-gap" + return entry + return entry + + +# --------------------------------------------------------------------------- +# Pure report assembly (hermetic-testable) +# --------------------------------------------------------------------------- + + +def _report_sha(report: dict) -> str: + clone = json.loads(json.dumps(report, sort_keys=True, default=str)) + clone.get("determinism", {}).pop("report_sha256", None) + return _sha256_text(json.dumps(clone, sort_keys=True)) + + +def assemble_report(*, image: dict, profile: dict, boot: dict, attach: dict, + doorbell: dict, polls: list, divergence, missing_mmio: dict, + determinism: dict, continuation: dict | None = None, + port_model: dict | None = None, missing_semantic: dict | None = None, + limits=None, notes=None) -> dict: + """Assemble the ``greeneye-msgq-attach/1`` report from plain inputs. + + Pure: no machine, no boot, no I/O. ``polls`` is the ordered list of + classified ring accesses (first = index 0); the report keeps the first and + at most 16 subsequent ones. ``boot["message_loop_wait_reached"]`` and + ``boot.get("skipped")`` select the outcome; a not-reached run keeps its + non-empty divergence reason, never a manufactured success. + + ``port_model`` is the :class:`sim.libos_port.LibosPortModel` state when the + receive path was modelled (``{"enabled": False}`` otherwise), and + ``missing_semantic`` is the exact unmodelled semantic that stopped a + port-model continuation (``None`` when there is none). Both are additive + fields, present on every report. + """ + ordered = [dict(poll) for poll in polls] + first = ordered[0] if ordered else None + subsequent = ordered[1:1 + (MAX_POLLS_RECORDED - 1)] + if boot.get("skipped"): + outcome = "skipped" + elif not boot.get("message_loop_wait_reached"): + outcome = "not-reached" + elif first is not None: + outcome = "wait-reached-poll-classified" + else: + outcome = "wait-reached-no-poll" + ok = outcome in ("wait-reached-poll-classified", "wait-reached-no-poll") + report = { + "format": REPORT_FORMAT, + "ok": ok, + "outcome": outcome, + "image": dict(image), + "profile": dict(profile), + "boot": dict(boot), + "attach": dict(attach), + "doorbell": dict(doorbell), + "continuation": dict(continuation or {}), + "first_poll": first, + "subsequent_polls": subsequent, + "poll_count": len(ordered), + "divergence": divergence, + "missing_mmio": dict(missing_mmio), + "port_model": dict(port_model) if port_model is not None + else {"enabled": False}, + "missing_semantic": dict(missing_semantic) if missing_semantic is not None + else None, + "determinism": dict(determinism), + "limits": list(limits if limits is not None else _LIMITS), + "notes": list(notes or []), + } + report["determinism"]["report_sha256"] = _report_sha(report) + return report + + +def _skipped_report(image: str, profile_path: str, reason: str, + *, doorbell: bool) -> dict: + return assemble_report( + image={"path": os.path.abspath(image), "sha256": None, "size": None}, + profile={"name": os.path.basename(profile_path), + "path": os.path.abspath(profile_path)}, + boot={"ran": False, "skipped": True, "reason": reason, + "message_loop_wait_reached": False, "rung": None, + "instructions_executed": 0}, + attach={}, + doorbell={"requested": bool(doorbell)}, + polls=[], + divergence=None, + missing_mmio={"count": 0, "first": None}, + determinism={}, + ) + + +# --------------------------------------------------------------------------- +# Execution helpers (machine stepping stays in sim.machine) +# --------------------------------------------------------------------------- + + +class _PortWaitWake: + """Host-traffic-gated wake of ``task_port_send_recv_wait`` (documented shim). + + The real libos kernel unblocks the task when a message is delivered through + the port machinery; the simulator does not model that path, so the only + modelled wake is explicit: after the host has rung the doorbell, the first + wait returns LIBOS_OK (0) exactly once and the event is recorded. A + re-entry is recorded and stops the run -- the wait is never silently + repeated as if progress had been made. + """ + + def __init__(self, base_handler, *, max_wakes: int = 1): + self.base_handler = base_handler + self.max_wakes = int(max_wakes) + self.wakes = 0 + self.wake_log: list[dict] = [] + self.reentries: list[dict] = [] + + def handle(self, machine, pc: int = 0): + number = int(machine.x[5]) + if number == TASK_PORT_SEND_RECV_WAIT: + record = { + "num": number, + "name": "task_port_send_recv_wait", + "pc": int(pc), + "a4": int(machine.x[14]), + "a5": int(machine.x[15]), + "a6": int(machine.x[16]), + "a7": int(machine.x[17]), + } + if self.wakes < self.max_wakes: + self.wakes += 1 + record["result"] = "shim:port_wait_wake" + record["returned"] = 0 + self.wake_log.append(record) + return True, 0 + record["result"] = "stop:reentry" + self.reentries.append(record) + return False, None + return self.base_handler.handle(machine, pc) + + def state(self) -> dict: + return { + "wakes": self.wakes, + "wake_log": [dict(entry) for entry in self.wake_log], + "reentries": [dict(entry) for entry in self.reentries], + } + + +def _watchers(geometry: RingGeometry): + """Read-watchpoints over the doorbell and both ring buffers (mask = loads).""" + return ( + (geometry.doorbell_base, geometry.doorbell_base + geometry.doorbell_size, + "doorbell", 2), + (geometry.cmd_base, geometry.cmd_base + geometry.cmd_size, "rings", 2), + (geometry.stat_base, geometry.stat_base + geometry.stat_size, "rings", 2), + ) + + +def _decoded_load(machine, pc: int): + """``(size, rd)`` of the cached load at ``pc``, or ``(None, None)``.""" + decoded = getattr(machine, "_cache", {}).get(pc) + if decoded is None: + try: + fetch = pc + if machine.mpu.active: + translated = machine.mpu.translate(pc) + if translated is not None: + fetch = translated + word = int.from_bytes(machine.read_mem(fetch, 4), "little") + if word & 3 == 3: + decoded = decode32(pc, word) + else: + decoded = decode16(pc, word & 0xFFFF) + except Exception: + return None, None + if decoded[0] != K_LOAD: + return None, None + return int(LOAD_SIZE[decoded[1]]), int(decoded[2]) + + +def _capture_poll(machine, geometry: RingGeometry, pc: int, addr: int) -> dict: + """Record and step over the watched load (a watch fires before execution). + + The load's width/rd come from the decoded instruction; the value is read + from the destination register after the single step, so the recorded value + is the value the firmware actually received. + """ + size, rd = _decoded_load(machine, pc) + step_reason, _spc, fault_cause, fault_tval = machine.step() + value = None + if rd is not None and step_reason != R_FAULT and rd != 0: + value = machine.x[rd] + poll = classify_ring_access(addr, size, value, geometry) + poll["pc"] = int(pc) + poll["executed"] = step_reason == R_STEP + if step_reason == R_FAULT: + poll["step_fault"] = {"cause": int(fault_cause), "tval": int(fault_tval)} + return poll + + +def _run_watching(machine, budget: int, geometry: RingGeometry, + milestones_hit: list, polls: list, + *, max_polls: int = MAX_POLLS_RECORDED): + """Run to ``budget`` (absolute instructions), recording milestone hits and + ring reads; same loop shape as ``sim.gsp_boot._run_with_milestones`` plus + watchpoint step-over. Returns ``(reason, pc, a, b)``. + """ + pending = {addr for name, addr in MILESTONES if name not in milestones_hit} + while True: + remaining = budget - machine.insns + if remaining <= 0: + return R_BUDGET, machine.pc, 0, 0 + watchers = _watchers(geometry) if len(polls) < max_polls else () + reason, pc, a, b = machine.run(remaining, breakpoints=pending, + watchers=watchers) + if reason == R_BP: + for name, addr in MILESTONES: + if addr == pc and name not in milestones_hit: + milestones_hit.append(name) + break + pending = {addr for name, addr in MILESTONES + if name not in milestones_hit} + continue + if reason == R_WATCH: + polls.append(_capture_poll(machine, geometry, pc, a)) + if len(polls) >= max_polls: + return R_POLL_LIMIT, machine.pc, a, b + continue + return reason, pc, a, b + + +def _control_params() -> bytes: + """A real-shaped 512-byte ``GSP_RM_CONTROL`` parameter block (docs/51).""" + return ( + (0xC2000006).to_bytes(4, "little") + + (0xABCD0080).to_bytes(4, "little") + + CONTROL_CMD.to_bytes(4, "little") + + bytes(500) + ) + + +def _verify_command_element(host: HostTraffic, submit: dict) -> dict: + """Parse the just-submitted command element back and verify its checksum.""" + try: + raw = host.queues.command.read_entry(submit["index"], + submit["element_bytes"]) + prefix, message = parse_queue_element(raw) + return { + "checksum_ok": True, + "elem_count": int(prefix["elem_count"]), + "function": int(message.function), + "sequence": int(message.sequence), + "length": int(message.length), + } + except Exception as exc: # pragma: no cover - defensive evidence + return {"checksum_ok": False, "error": str(exc)} + + +def missing_semantic_after_handoff(*, stopped: dict, polls: list) -> dict: + """The exact unmodelled semantics that stopped the port-model continuation. + + Recorded when, after the recorded host wake, the message-loop task + completed the external-interrupt receive, dispatched it, and issued + ``TASK_STATE_QUERY`` (the libos trap/report path) without touching the + command/status rings. Pure: takes the stop record and the classified poll + list; no machine, no I/O. + """ + return { + "stage": "after-extintr-handoff", + "stop": dict(stopped), + "observed": [ + "host doorbell write NV_PGSP_QUEUE_HEAD(0) recorded", + "external-interrupt receive retired with the RTOS report contract " + "(error 0, payload size 0) on the recorded wake", + "completed shuttle dispatched; next action is a task-state query " + "for the reported remote task", + "no command/status ring or doorbell read occurred before the stop", + ], + "missing": [ + "doorbell-write -> falcon interrupt bit mapping: the host submit " + "path writes only NV_PGSP_QUEUE_HEAD; no IRQ register write is " + "observed and the firmware armed mask 0xFF", + "libos interrupt-trap delivery and the interrupted task's saved " + "xcause/xepc consumed by TASK_STATE_QUERY", + "the command-queue consumer (msgq receive) task and the " + "GSP_ARGUMENTS_CACHED/queue-init address handoff; this single-task " + "model schedules no other task", + ], + "next_step": ( + "model TASK_STATE_QUERY's trap state or establish the queue-service " + "task's wake path, then re-run this attach and classify the first " + "ring poll; do not infer a poll from this run"), + "polls": len(polls), + } + + +# --------------------------------------------------------------------------- +# The attach run +# --------------------------------------------------------------------------- + + +def run_attach(*, image: str = DEFAULT_IMAGE, profile_path: str = DEFAULT_PROFILE, + max_cycles: int = DEFAULT_MAX_CYCLES, doorbell: bool = False, + port_model: bool = True, json_out: str | None = None, + record_dir: str | None = None) -> dict: + """Boot to the port wait, attach the rings, optionally kick and continue. + + Bounded and deterministic for a fixed image/profile/budget. Returns the + assembled report; on an absent image/profile it returns a clean + ``outcome: "skipped"`` report instead of raising. Never fakes a poll: the + outcome is the observed one, with the first divergence recorded when the + wait is not reached. + + ``port_model`` (default on) installs + :class:`sim.libos_port.LibosPortModel` for the continuation: the async + receive posts, the boot handshake send and the host wake's + external-interrupt handoff are modelled, so the run advances to the + task-state-query the interrupt dispatch performs (or further). With the + model off, the documented single-wake shim of the first wave is used and + the run reports ``port-wait-reentry`` instead. + """ + if isinstance(max_cycles, bool) or not isinstance(max_cycles, int) \ + or not 0 < max_cycles <= MAX_RUN_CYCLES: + raise MsgqAttachError( + f"max_cycles must be an int in 1..{MAX_RUN_CYCLES}, got {max_cycles!r}") + image = os.path.abspath(image) + profile_path = os.path.abspath(profile_path) + if not os.path.isfile(image): + return _skipped_report(image, profile_path, f"image absent: {image}", + doorbell=doorbell) + if not os.path.isfile(profile_path): + return _skipped_report(image, profile_path, + f"profile absent: {profile_path}", + doorbell=doorbell) + + sha = firmware_sha256(image) + size = os.path.getsize(image) + profile = load_profile(profile_path) + load = profile.get("load", {}) + ram_base = _as_int(load.get("ram_base", 0)) + memory_mb = _as_int(load.get("memory_mb", 12288)) + + session = None + notes: list[str] = [] + divergence = None + polls: list[dict] = [] + continuation: dict = {"ran": False} + port_model_state: dict | None = None + missing_semantic: dict | None = None + try: + session = launch_native( + kernel=image, + memory_mb=memory_mb, + ram_base=ram_base, + record=record_dir if record_dir else False, + name="msgq-attach", + device_map=None, + unknown_mmio="lenient", + bias=0, + ) + machine = session.target.machine + machine.devices = registry_from_profile(profile, machine=machine) + machine.translator = getattr(machine.devices, "dma", None) + libos = LibosSyscalls() + machine.syscall_handler = libos.handle + shim_meta = {"enabled": True, "assumptions": []} + shim = dict(profile.get("boot_shim", {})) + shim["_image_path"] = image + _apply_boot_shim(machine, shim, load_elf(image), shim_meta) + + host = HostTraffic(machine=machine, recorder=session.recorder) + machine.devices.add(host.doorbell_block()) + geometry = ring_geometry_from_host(host) + + submit = None + element_check = None + if doorbell: + submit = host.submit_rpc(GSP_RM_CONTROL, _control_params(), + sequence=1) + element_check = _verify_command_element(host, submit) + + milestones_hit: list[str] = [] + reason, pc, a, b = _run_watching(machine, max_cycles, geometry, + milestones_hit, polls) + stopped = dict(libos.stopped) if libos.stopped else None + wait_reached = bool(stopped) and stopped.get("num") == TASK_PORT_SEND_RECV_WAIT + stop_reason = STOP_REASONS.get(reason, str(reason)) + if reason == R_POLL_LIMIT: + stop_reason = "poll-limit" + fault = {"cause": int(a), "tval": int(b), "pc": int(pc)} \ + if reason == R_FAULT else None + rung = _rung_from_run(machine, shim_meta, milestones_hit, libos) + + if not wait_reached: + if reason == R_FAULT: + divergence = { + "kind": "fault", + "reason": (f"fault before the message-loop port wait: " + f"cause={a:#x} tval={b:#x} pc={pc:#x}"), + } + elif reason == R_POLL_LIMIT: + divergence = { + "kind": "poll-limit", + "reason": (f"{MAX_POLLS_RECORDED} ring accesses observed " + f"before the port wait; run stopped by the " + f"bounded poll limit"), + } + elif reason == R_ECALL: + name = (stopped or {}).get("name", "unhandled ecall") + number = (stopped or {}).get("num") + divergence = { + "kind": "ecall", + "reason": (f"unhandled libos syscall before the " + f"message-loop port wait: {name} " + f"(num={number}) at pc={pc:#x}"), + } + else: + divergence = { + "kind": "budget", + "reason": (f"message-loop port wait not reached within " + f"{max_cycles} instructions (rung {rung}, " + f"stop={stop_reason} at pc={pc:#x})"), + } + + if wait_reached and doorbell and port_model: + model = LibosPortModel(base=libos) + machine.syscall_handler = model.handle + last_kick = host.doorbell.writes[-1] if host.doorbell.writes else None + model.host_kick( + queue=int(last_kick["queue"]) if last_kick else None, + value=int(last_kick["value"]) if last_kick else None, + ) + reason2, pc2, a2, b2 = _run_watching(machine, max_cycles, geometry, + milestones_hit, polls) + stop_reason2 = STOP_REASONS.get(reason2, str(reason2)) + if reason2 == R_POLL_LIMIT: + stop_reason2 = "poll-limit" + port_model_state = {"enabled": True, **model.state()} + continuation = { + "ran": True, + "port_model": True, + "stop_reason": stop_reason2, + "stop_pc": int(pc2), + "instructions_executed": int(machine.insns), + "wakes": int(model.host_kicks), + "wake_log": [ + { + "result": entry["contract"], + "returned": 0, + "delivered": entry.get("delivered"), + "pending": entry.get("pending"), + "queue": entry.get("queue"), + "value": entry.get("value"), + } + for entry in model.events + if entry["event"] == "host_kick" + ], + "reentries": [], + "polls": len(polls), + } + if reason2 == R_FAULT: + divergence = { + "kind": "fault", + "reason": (f"fault after the host wake: cause={a2:#x} " + f"tval={b2:#x} pc={pc2:#x}"), + } + elif reason2 == R_ECALL: + port_stopped = model.stopped or ( + dict(libos.stopped) if libos.stopped else None) + number2 = int(port_stopped["num"]) if port_stopped \ + else int(machine.x[5]) + name2 = (port_stopped or {}).get("name", "unhandled ecall") + if number2 == TASK_STATE_QUERY: + missing_semantic = missing_semantic_after_handoff( + stopped=port_stopped or {"num": number2, "pc": int(pc2)}, + polls=polls) + divergence = { + "kind": "task-state-query", + "reason": ( + "after the recorded host wake the external " + "interrupt receive completed (error 0, size 0) and " + "the message-loop task dispatched it; its next " + "syscall is task_state_query " + f"(num={number2}) at pc={pc2:#x}, the libos " + "trap/report path -- no command/status ring or " + "doorbell read occurred, so no poll is reported"), + "stop": dict(port_stopped or {}), + } + elif number2 == TASK_PORT_SEND_RECV_WAIT: + divergence = { + "kind": "port-wait-reentry", + "reason": ( + "the message-loop task re-entered " + "task_port_send_recv_wait after the recorded host " + "wake; nothing completed the wait, so the firmware " + "produced no queue poll"), + "wait_pc": int(pc2), + } + else: + divergence = { + "kind": "ecall", + "reason": (f"unhandled libos syscall after the host " + f"wake: {name2} (num={number2}) at " + f"pc={pc2:#x}"), + } + elif reason2 == R_BUDGET and not polls: + divergence = { + "kind": "budget", + "reason": (f"no ring access observed within the total " + f"budget of {max_cycles} instructions"), + } + elif reason2 == R_POLL_LIMIT: + notes.append( + f"continuation stopped after {MAX_POLLS_RECORDED} recorded " + "ring accesses (bounded by design)") + elif wait_reached and doorbell: + wake = _PortWaitWake(libos, max_wakes=1) + machine.syscall_handler = wake.handle + reason2, pc2, a2, b2 = _run_watching(machine, max_cycles, geometry, + milestones_hit, polls) + stop_reason2 = STOP_REASONS.get(reason2, str(reason2)) + if reason2 == R_POLL_LIMIT: + stop_reason2 = "poll-limit" + continuation = { + "ran": True, + "port_model": False, + "stop_reason": stop_reason2, + "stop_pc": int(pc2), + "instructions_executed": int(machine.insns), + "wakes": int(wake.wakes), + "wake_log": [dict(entry) for entry in wake.wake_log], + "reentries": [dict(entry) for entry in wake.reentries], + "polls": len(polls), + } + if reason2 == R_FAULT: + divergence = { + "kind": "fault", + "reason": (f"fault after the host wake: cause={a2:#x} " + f"tval={b2:#x} pc={pc2:#x}"), + } + elif reason2 == R_ECALL: + number2 = int(machine.x[5]) + if number2 == TASK_PORT_SEND_RECV_WAIT: + divergence = { + "kind": "port-wait-reentry", + "reason": ( + "the message-loop task re-entered " + "task_port_send_recv_wait after the recorded host " + "wake; libos message-port delivery is not modelled, " + "so the firmware produced no queue poll"), + "wait_pc": int(pc2), + } + else: + divergence = { + "kind": "ecall", + "reason": (f"unhandled libos syscall after the host " + f"wake: num={number2} at pc={pc2:#x}"), + } + elif reason2 == R_BUDGET and not polls: + divergence = { + "kind": "budget", + "reason": (f"no ring access observed within the total " + f"budget of {max_cycles} instructions"), + } + elif reason2 == R_POLL_LIMIT: + notes.append( + f"continuation stopped after {MAX_POLLS_RECORDED} recorded " + "ring accesses (bounded by design)") + elif wait_reached and not doorbell: + notes.append( + "continuation not requested (--doorbell off); the port wait was " + "reached but no host kick was sent and no poll was classified") + + ring_reads = [ + access for access in machine.devices.accesses + if access["rw"] == "r" + and geometry.doorbell_base <= access["addr"] + < geometry.doorbell_base + geometry.doorbell_size + ] + ring_writes = [ + access for access in machine.devices.accesses + if access["rw"] == "w" + and geometry.doorbell_base <= access["addr"] + < geometry.doorbell_base + geometry.doorbell_size + ] + doorbell_write = None + if submit is not None and host.doorbell.writes: + last = host.doorbell.writes[-1] + doorbell_write = { + "window": DOORBELL_WINDOW, + "base": int(DOORBELL_BASE), + "offset": int(last["queue"]) * 8, + "queue": int(last["queue"]), + "value": int(last["value"]), + } + doorbell_report = { + "requested": bool(doorbell), + "write": doorbell_write, + "host_kicks": int(host.doorbell.kicks), + "head_read_back": int(host.doorbell.head(0)), + "guest_reads": len(ring_reads), + "guest_writes": len(ring_writes), + } + + attach = dict(geometry.to_dict()) + attach["command_queue"] = host.queues.command.state() + attach["status_queue"] = host.queues.status.state() + attach["command_element"] = element_check + + device_state = json.dumps(machine.devices.state(), sort_keys=True, + default=str) + queue_state = json.dumps(host.queues.state(), sort_keys=True) + doorbell_state = json.dumps(host.doorbell.state(), sort_keys=True) + determinism = { + "instructions": int(machine.insns), + "cycles_modeled": int(machine.cycles), + "cycle_model": machine.cycle_model.name + if getattr(machine, "cycle_model", None) else None, + "max_cycles": int(max_cycles), + "ram_base": int(machine.ram_base), + "ram_size": int(machine.ram_size), + "device_state_sha256": _sha256_text(device_state), + "access_log_len": len(machine.devices.accesses), + "queue_state_sha256": _sha256_text(queue_state), + "doorbell_state_sha256": _sha256_text(doorbell_state), + } + + boot = { + "ran": True, + "skipped": False, + "rung": rung, + "message_loop_wait_reached": wait_reached, + "instructions_executed": int(machine.insns), + "cycles_modeled": int(machine.cycles), + "cycle_model": determinism["cycle_model"], + "instructions_budget": int(max_cycles), + "stop": {"reason": stop_reason, "pc": int(pc), "fault": fault}, + "wait": stopped if wait_reached else None, + "libos_stopped": stopped, + "milestones": list(milestones_hit), + "mmio_unknown": int(machine.mmio_unknown), + "device_unhandled": int(machine.devices.unhandled), + } + missing_mmio = { + "count": int(machine.mmio_unknown), + "first": dict(machine.mmio_log[0]) if machine.mmio_log else None, + } + report = assemble_report( + image={"path": image, "sha256": sha, "size": size}, + profile={"name": profile.get("name"), "path": profile_path}, + boot=boot, + attach=attach, + doorbell=doorbell_report, + polls=polls, + divergence=divergence, + missing_mmio=missing_mmio, + determinism=determinism, + continuation=continuation, + port_model=port_model_state, + missing_semantic=missing_semantic, + notes=notes, + ) + if getattr(session.recorder, "dir", None) is not None: + session.recorder.meta.update({ + "kind": "msgq-attach", + "outcome": report["outcome"], + "rung": rung, + "instructions_executed": int(machine.insns), + "message_loop_wait_reached": wait_reached, + "port_model": bool(report["port_model"].get("enabled")), + "missing_semantic": bool(report["missing_semantic"]), + "first_poll": None if report["first_poll"] is None + else report["first_poll"]["kind"], + }) + session.recorder.blob( + "msgq-attach.json", + json.dumps(report, indent=1, sort_keys=True).encode(), + ) + session.note( + "msgq_attach", + outcome=report["outcome"], + rung=rung, + instructions=machine.insns, + wait_reached=wait_reached, + polls=report["poll_count"], + ) + if json_out: + parent = os.path.dirname(os.path.abspath(json_out)) + os.makedirs(parent, exist_ok=True) + with open(json_out, "w", encoding="utf-8") as handle: + json.dump(report, handle, indent=1, sort_keys=True) + handle.write("\n") + finally: + if session is not None: + session.close() + return report + + +# --------------------------------------------------------------------------- +# Demo and CLI +# --------------------------------------------------------------------------- + + +def run_demo(*, image: str = DEFAULT_IMAGE, profile_path: str = DEFAULT_PROFILE, + print_json: bool = False, port_model: bool = True) -> dict: + """Bounded attach demo; cleanly skips (exit 0) when the image is absent.""" + report = run_attach(image=image, profile_path=profile_path, + max_cycles=DEMO_MAX_CYCLES, doorbell=True, + port_model=port_model) + if report["outcome"] == "skipped": + print(f"[msgq-attach] skipped: {report['boot'].get('reason')}") + return report + if print_json: + print(json.dumps(report, indent=1, sort_keys=True)) + else: + _print_report(report) + return report + + +def _print_report(report: dict) -> None: + image = report["image"] + boot = report["boot"] + print(f"format: {report['format']}") + print(f"image: {image['path']}") + print(f"sha256: {image['sha256']} size: {image['size']}") + print(f"outcome: {report['outcome']} (ok={report['ok']})") + print(f"rung: {boot.get('rung')} instructions: {boot.get('instructions_executed')}" + f" cycles (modeled): {boot.get('cycles_modeled')}") + wait = boot.get("wait") + if wait: + print(f"message-loop wait: reached ({wait.get('name')} @ " + f"{wait.get('pc'):#x})") + else: + print("message-loop wait: not reached") + write = report["doorbell"].get("write") + if write: + print(f"doorbell: host write {write['window']} offset={write['offset']:#x} " + f"value={write['value']:#x} (guest reads={report['doorbell']['guest_reads']})") + model = report.get("port_model") or {} + if model.get("enabled"): + posts = model.get("posts", []) + roles = ", ".join(f"port {post['port']}:{post['role']}" + for post in posts) + print(f"port model: enabled posts=[{roles}] " + f"host kicks={model.get('host_kicks', 0)}") + for wait_record in model.get("waits", []): + if wait_record.get("completed") is not None: + print(f" wait @ {wait_record['pc']:#x}: completed shuttle " + f"{wait_record['completed']} ({wait_record['role']}) " + f"size={wait_record['payload_size']} " + f"contract={wait_record['contract']}") + else: + print("port model: disabled") + first = report["first_poll"] + if first: + print(f"first poll: addr={first['addr']:#x} width={first['width']} " + f"value={first['value']} kind={first['kind']}") + else: + print("first poll: none observed") + print(f"polls recorded: {report['poll_count']}" + f" (subsequent listed: {len(report['subsequent_polls'])})") + if report["divergence"]: + print(f"divergence: {report['divergence']['reason']}") + missing = report.get("missing_semantic") + if missing: + print(f"missing semantic ({missing.get('stage')}):") + for item in missing.get("missing", []): + print(f" - {item}") + print("note: development-only, offline; L0/L1 evidence, no timing claim") + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="sim.msgq_attach", + description=( + "Boot the real 570.144 GSP image to the message-loop port wait " + "(B3), attach the host doorbell/msgq rings before the run, " + "optionally kick, continue bounded and classify the first ring " + "poll. Offline only; no hardware, module, guest or network."), + ) + sub = parser.add_subparsers(dest="command") + run_cmd = sub.add_parser( + "run", help="bounded boot + attach + optional kick + poll classify") + run_cmd.add_argument("--image", default=DEFAULT_IMAGE, + help="GSP firmware image (default: the 570.144 image)") + run_cmd.add_argument("--profile", default=DEFAULT_PROFILE, + help="greeneye-sim-profile/1 document") + run_cmd.add_argument("--max-cycles", type=int, default=DEFAULT_MAX_CYCLES, + help=f"total instruction budget (1..{MAX_RUN_CYCLES})") + run_cmd.add_argument("--doorbell", action="store_true", + help="submit a host command and wake the port wait once") + run_cmd.add_argument("--no-port-model", action="store_true", + help="disable the sim.libos_port receive-path model " + "(use the first-wave single-wake shim)") + run_cmd.add_argument("--json", metavar="OUT", default=None, + help="write the full report JSON to OUT") + run_cmd.add_argument("--record", metavar="DIR", default=None, + help="write a greeneye-session/1 directory to DIR") + demo_cmd = sub.add_parser( + "demo", help="bounded canned attach run (skips cleanly if image absent)") + demo_cmd.add_argument("--image", default=DEFAULT_IMAGE, + help="GSP firmware image (default: the 570.144 image)") + demo_cmd.add_argument("--profile", default=DEFAULT_PROFILE, + help="greeneye-sim-profile/1 document") + demo_cmd.add_argument("--no-port-model", action="store_true", + help="disable the sim.libos_port receive-path model") + demo_cmd.add_argument("--json", action="store_true", + help="print the full report JSON") + return parser + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + if args.command is None: + parser.print_help(sys.stderr) + return 2 + port_model = not getattr(args, "no_port_model", False) + if args.command == "demo": + run_demo(image=args.image, profile_path=args.profile, + print_json=args.json, port_model=port_model) + return 0 + try: + report = run_attach( + image=args.image, + profile_path=args.profile, + max_cycles=args.max_cycles, + doorbell=args.doorbell, + port_model=port_model, + json_out=args.json, + record_dir=args.record, + ) + except MsgqAttachError as exc: + print(f"[msgq-attach] error: {exc}", file=sys.stderr) + return 1 + if report["outcome"] == "skipped": + print(f"[msgq-attach] skipped: {report['boot'].get('reason')}", + file=sys.stderr) + return 2 + _print_report(report) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/debug/risc-v-gsp-rm-vm/tests/test_contracts.py b/debug/risc-v-gsp-rm-vm/tests/test_contracts.py new file mode 100644 index 0000000..3199b09 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_contracts.py @@ -0,0 +1,171 @@ +#!/usr/bin/env python3 +"""Tests for the wall-contract registry and runner (``harness/contracts.py``). + +The fast contracts execute the proven offline entry points (valprobe matrix, +GMMU golden vector, launch audit, rehost demo, RM-capture demo) and are +asserted to pass; prerequisites that are genuinely absent are reported as +skips, never as passes. No hardware, no module, no guest, no network. + +Run:: + + python3 -m unittest tests.test_contracts -v +""" +from __future__ import annotations + +import contextlib +import importlib +import io +import json +import os +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness import run_contracts as runner # noqa: E402 +from harness.contracts import ( # noqa: E402 + FAST, + SLOW, + WALL_CLASSES, + Contract, + ContractSkip, + all_contracts, + fast_contracts, + run_contracts, + summarize, +) + +MISSING_PATH = os.path.join(TOOL_ROOT, "contract-test-missing-prerequisite") + + +def _entry_target(entry: str) -> tuple[str, str]: + module, _, rest = entry.partition(":") + return module, rest.split("(", 1)[0].strip() + + +def _raise(exc: Exception): + def _run(): + raise exc + + return _run + + +def _fake(name: str, **overrides) -> Contract: + fields = { + "name": name, + "wall_class": "a", + "doc": "docs/50", + "entry": "tests:fake", + "speed": FAST, + "expect": "test expectation", + "run": lambda: {"ok": True}, + "check": lambda report: (report.get("ok") is True, "ok"), + "requires": (), + } + fields.update(overrides) + return Contract(**fields) + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + try: + rc = runner.main(argv) + except SystemExit as exc: + rc = int(exc.code or 0) + return rc, out.getvalue(), err.getvalue() + + +class RegistryShapeTests(unittest.TestCase): + def test_registry_shape(self): + contracts = all_contracts() + names = [contract.name for contract in contracts] + self.assertEqual(len(names), len(set(names)), "contract names must be unique") + self.assertGreaterEqual(len(contracts), 3, "keep at least the proven contracts") + for contract in contracts: + with self.subTest(contract=contract.name): + self.assertIn(contract.wall_class, WALL_CLASSES) + self.assertIn(contract.speed, (FAST, SLOW)) + self.assertTrue(contract.doc.startswith("docs/")) + self.assertTrue(contract.entry) + self.assertTrue(contract.expect) + self.assertTrue(fast_contracts(), "registry must have fast contracts") + + def test_entries_resolve_to_real_callables(self): + for contract in all_contracts(): + with self.subTest(contract=contract.name): + module_name, func_name = _entry_target(contract.entry) + module = importlib.import_module(module_name) + self.assertTrue(callable(getattr(module, func_name, None)), contract.entry) + + +class RunnerSemanticsTests(unittest.TestCase): + def test_classifies_pass_fail_skip(self): + results = run_contracts( + [ + _fake("good"), + _fake("raising", run=_raise(RuntimeError("boom"))), + _fake("missing-prereq", requires=(MISSING_PATH,)), + _fake("declared-skip", run=_raise(ContractSkip("not applicable here"))), + _fake("failed-check", check=lambda report: (False, "checker rejected")), + ] + ) + by_name = {result.name: result for result in results} + self.assertEqual(by_name["good"].status, "pass") + self.assertEqual(by_name["missing-prereq"].status, "skip") + self.assertEqual(by_name["declared-skip"].status, "skip") + self.assertEqual(by_name["raising"].status, "fail") + self.assertIn("boom", by_name["raising"].detail) + self.assertEqual(by_name["failed-check"].status, "fail") + + summary = summarize(results) + self.assertFalse(summary["ok"]) + self.assertEqual(summary["counts"], {"pass": 1, "fail": 2, "skip": 2}) + domains = {domain["wall_class"]: domain for domain in summary["domains"]} + self.assertEqual(set(domains), set(WALL_CLASSES)) + self.assertEqual(domains["a"]["total"], 5) + self.assertEqual(domains["d"]["total"], 0) + + def test_fast_contracts_pass(self): + contracts = fast_contracts() + self.assertGreaterEqual(len(contracts), 3) + results = run_contracts(contracts) + for result in results: + with self.subTest(contract=result.name): + self.assertNotEqual(result.status, "fail", result.detail) + if result.status == "skip": + self.skipTest(result.detail) + self.assertTrue(any(result.status == "pass" for result in results)) + + +class RunnerCliTests(unittest.TestCase): + def test_single_contract_json_summary(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "contracts.json") + rc, out, err = _run_main( + ["--fast", "--name", "gmmu-golden-vector", "--json", path] + ) + self.assertEqual(rc, 0, err) + self.assertIn("PASS", out) + with open(path, "r", encoding="utf-8") as fh: + summary = json.load(fh) + self.assertTrue(summary["ok"]) + self.assertEqual(summary["counts"]["pass"], 1) + self.assertEqual( + [contract["name"] for contract in summary["contracts"]], + ["gmmu-golden-vector"], + ) + self.assertEqual(len(summary["domains"]), len(WALL_CLASSES)) + + def test_unknown_contract_name_is_usage_error(self): + rc, _out, err = _run_main(["--fast", "--name", "no-such-contract"]) + self.assertEqual(rc, 2) + self.assertIn("unknown contract", err) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_gmmu_walk.py b/debug/risc-v-gsp-rm-vm/tests/test_gmmu_walk.py new file mode 100644 index 0000000..f438068 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_gmmu_walk.py @@ -0,0 +1,243 @@ +"""Tests for the functional GMMU walker (sim/gmmu_walk.py, docs/48). + +The golden vector is a captured channel PDB (numeric entries); the synthetic +trees are built with the module's own encoders and must round-trip. No +hardware, no modules, no guest, no network. + +Run:: + + python3 -m unittest tests.test_gmmu_walk -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import struct +import sys +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.gmmu_walk import ( # noqa: E402 + FAULT_INVALID_APERTURE, + FAULT_PTE_INVALID, + FAULT_TABLE_MISSING, + FAULT_UNSUPPORTED, + FAULT_OUT_OF_RANGE, + WALK_FORMAT, + MemoryView, + WalkError, + channel_pdb_fixture, + encode_dual_slot, + encode_pde_pointer, + encode_pte, + main, + walk, +) + + +def _tree(root_pa: int, va: int, target_pa: int | None, *, base: int = 0x50000000, + read_only: bool = False) -> MemoryView: + i0 = (va >> 47) & 0x3 + i1 = (va >> 38) & 0x1FF + i2 = (va >> 29) & 0x1FF + i3 = (va >> 21) & 0xFF + i4 = (va >> 12) & 0x1FF + pd2, pd1, pd0, pt = base + 0x1000, base + 0x2000, base + 0x3000, base + 0x4000 + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0 * 8, encode_pde_pointer(pd2)) + mem.add(root_pa, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3 * 16, encode_dual_slot(None, pt)) + mem.add(pd0, bytes(page)) + page = bytearray(0x1000) + if target_pa is not None: + struct.pack_into("<Q", page, i4 * 8, encode_pte(target_pa, read_only=read_only)) + mem.add(pt, bytes(page)) + return mem + + +def _run_main(argv): + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class GoldenVectorTests(unittest.TestCase): + def test_captured_channel_pdb_walks_to_logged_dmas(self): + root_pa, mem = channel_pdb_fixture() + expected = { + 0x100000: 0x2045D9000, + 0x101000: 0x121BD9000, + 0x102000: 0x152115000, + 0x103000: 0x15A5F5000, + 0x104000: 0x10CF27000, + } + for va, pa in expected.items(): + report = walk(va, root_pa=root_pa, mem=mem) + self.assertEqual(report["status"], "ok", report) + self.assertEqual(report["format"], WALK_FORMAT) + self.assertEqual(int(report["pa"], 16), pa) + self.assertEqual(report["page_size"], "4k") + self.assertTrue(report["writable"]) + self.assertEqual(report["evidence"], "validated") + self.assertEqual(len(report["steps"]), 5) + + def test_walk_is_deterministic(self): + root_pa, mem = channel_pdb_fixture() + first = json.dumps(walk(0x100000, root_pa=root_pa, mem=mem), sort_keys=True) + second = json.dumps(walk(0x100000, root_pa=root_pa, mem=mem), sort_keys=True) + self.assertEqual(first, second) + + +class SyntheticRoundTripTests(unittest.TestCase): + def test_encoder_round_trip_4k(self): + va, pa = 0x204608000, 0x15E8FA000 + mem = _tree(0x60000000, va, pa) + report = walk(va, root_pa=0x60000000, mem=mem) + self.assertEqual(report["status"], "ok") + self.assertEqual(int(report["pa"], 16), pa) + self.assertEqual(report["page_size"], "4k") + self.assertTrue(report["writable"]) + + def test_read_only_leaf_is_reported(self): + va, pa = 0x204608000, 0x15E8FA000 + mem = _tree(0x60000000, va, pa, read_only=True) + report = walk(va, root_pa=0x60000000, mem=mem) + self.assertEqual(report["status"], "ok") + self.assertTrue(report["read_only"]) + self.assertFalse(report["writable"]) + + def test_2m_leaf(self): + va, base_pa = 0x40000000, 0x20000000 + mem = MemoryView() + i0 = (va >> 47) & 0x3 + i1 = (va >> 38) & 0x1FF + i2 = (va >> 29) & 0x1FF + i3 = (va >> 21) & 0xFF + pd2, pd1, pd0 = 0x51001000, 0x51002000, 0x51003000 + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0 * 8, encode_pde_pointer(pd2)) + mem.add(0x51000000, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i3 * 16, encode_pte(base_pa, page_size="2m")) + mem.add(pd0, bytes(page)) + report = walk(va + 0x12345, root_pa=0x51000000, mem=mem) + self.assertEqual(report["status"], "ok") + self.assertEqual(report["page_size"], "2m") + self.assertEqual(int(report["pa"], 16), base_pa + 0x12345) + + def test_auto_page_mode_prefers_prefers_small_when_big_invalid(self): + va = 0x100000 # 64K-aligned; the dual BIG half is invalid (0) + root_pa, mem = channel_pdb_fixture() + report = walk(va, root_pa=root_pa, mem=mem, page_mode="auto") + self.assertEqual(report["status"], "ok") + self.assertEqual(report["page_mode_used"], "4k") + + def test_auto_page_mode_uses_big_when_valid(self): + va, pa = 0x100000, 0x30000000 + # build a tree whose dual BIG half points at a 64K-granularity table + i0 = (va >> 47) & 0x3 + i1 = (va >> 38) & 0x1FF + i2 = (va >> 29) & 0x1FF + i3 = (va >> 21) & 0xFF + pd2, pd1, pd0, pt = 0x52001000, 0x52002000, 0x52003000, 0x52004000 + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0 * 8, encode_pde_pointer(pd2)) + mem.add(0x52000000, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3 * 16, encode_dual_slot(pt, pt)) + mem.add(pd0, bytes(page)) + page = bytearray(0x1000) + # BIG path indexes the 64K table by VA bits 20:16 + struct.pack_into("<Q", page, ((va >> 16) & 0x1FF) * 8, + encode_pte(pa, page_size="64k")) + mem.add(pt, bytes(page)) + report = walk(va, root_pa=0x52000000, mem=mem, page_mode="auto") + self.assertEqual(report["status"], "ok") + self.assertEqual(report["page_mode_used"], "64k") + self.assertEqual(report["evidence"], "modeled") + + +class FaultTests(unittest.TestCase): + def test_missing_table_page(self): + root_pa, mem = channel_pdb_fixture() + report = walk(0x100000, root_pa=root_pa, mem=MemoryView()) + self.assertEqual(report["status"], "fault") + self.assertEqual(report["fault"], FAULT_TABLE_MISSING) + + def test_invalid_leaf(self): + mem = _tree(0x60000000, 0x204608000, None) # leaf entry stays zero + report = walk(0x204608000, root_pa=0x60000000, mem=mem) + self.assertEqual(report["status"], "fault") + self.assertEqual(report["fault"], FAULT_PTE_INVALID) + + def test_invalid_pde_aperture(self): + root_pa = 0x60000000 + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, 0, 0) # aperture 0 = invalid + mem.add(root_pa, bytes(page)) + report = walk(0x100000, root_pa=root_pa, mem=mem) + self.assertEqual(report["status"], "fault") + self.assertEqual(report["fault"], FAULT_INVALID_APERTURE) + + def test_unsupported_kind_fails_closed(self): + root_pa, mem = channel_pdb_fixture() + report = walk(0x100000, root_pa=root_pa, mem=mem, kind="ver9") + self.assertEqual(report["status"], "fault") + self.assertEqual(report["fault"], FAULT_UNSUPPORTED) + + def test_va_out_of_range(self): + root_pa, mem = channel_pdb_fixture() + report = walk(1 << 60, root_pa=root_pa, mem=mem) + self.assertEqual(report["status"], "fault") + self.assertEqual(report["fault"], FAULT_OUT_OF_RANGE) + + def test_bad_page_mode_is_clean_error(self): + root_pa, mem = channel_pdb_fixture() + with self.assertRaises(WalkError): + walk(0x100000, root_pa=root_pa, mem=mem, page_mode="8k") + + +class CliTests(unittest.TestCase): + def test_demo_returns_zero(self): + rc, out, _ = _run_main(["demo"]) + self.assertEqual(rc, 0) + self.assertIn('"ok": true', out) + self.assertIn("PASS", out) + + def test_run_returns_three_on_fault(self): + rc, out, _ = _run_main(["run", "--pdb-pa", "0x60000000", "--va", "0x1000"]) + self.assertEqual(rc, 3) + self.assertIn("FAULT", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_host_traffic.py b/debug/risc-v-gsp-rm-vm/tests/test_host_traffic.py new file mode 100644 index 0000000..93024d2 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_host_traffic.py @@ -0,0 +1,639 @@ +"""Host-traffic message-loop transport tests (sim/host_traffic.py, docs/51). + +All fixtures are synthetic and fully offline: no hardware, no kernel module, +no QEMU, no guest, no network. The tests drive the modelled doorbell, the +msgq rings, the queue-element framing and the stub dispatcher against a small +in-process Machine (or a bare region) and assert framing, ordering and +determinism only. The dispatcher handlers are stubs: nothing here claims +firmware behaviour. + +Run:: + + python3 -m unittest tests.test_host_traffic -v +""" +from __future__ import annotations + +import json +import os +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from greeneye.record import Recorder, load_events # noqa: E402 +from sim.devices import DeviceRegistry, FalconBlock # noqa: E402 +from sim.host_traffic import ( # noqa: E402 + DEMO_FORMAT, + DOORBELL_BASE, + DOORBELL_SIZE, + GSP_MSG_QUEUE_ELEMENT_HDR_SIZE, + GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + GSP_RM_ALLOC, + GSP_RM_CONTROL, + MSGQ_FLAGS_SWAP_RX, + MSGQ_ENTRY_OFF, + MSGQ_RX_HDR_OFF, + NV_ERR_NOT_SUPPORTED, + NV_OK, + REPLAY_FORMAT, + RPC_FN_ECHO, + RPC_FN_STATUS_QUERY, + RPC_HEADER_BYTES, + RPC_HEADER_VERSION, + RPC_SIGNATURE, + SHARED_MEM_BASE, + STATUS_MAGIC, + HostTraffic, + HostTrafficError, + MsgqQueue, + RegionMemory, + SharedQueues, + build_queue_element, + build_rpc_message, + compute_message_checksum, + load_stimulus, + main, + parse_queue_element, + parse_rpc_message, + replay_stimulus, + run_demo, +) +from sim.machine import R_EBREAK, Machine # noqa: E402 + +FALCON_BASE = 0x2000000000110000 +FALCON_DEBUGINFO = 0x94 + +# lui t0,0x20000; slli t0,t0,32; lui t1,0x111; addi t1,t1,-0x400; +# add t0,t0,t1; lw a0,0(t0); ebreak -- read doorbell queue 0. +PROGRAM_READ_DOORBELL = bytes.fromhex( + "b70200209392020237131100130303c0b382620003a5020073001000" +) +# lui t0,0x20000; slli t0,t0,32; lui t1,0x111; addi t1,t1,-0x3f0; +# add t0,t0,t1; sw a1,0(t0); ebreak -- write doorbell queue 2. +PROGRAM_WRITE_DOORBELL = bytes.fromhex( + "b70200209392020237131100130303c1b382620023a0b20073001000" +) + + +def _sample_capture_log() -> str: + payload = ( + (0xC2000006).to_bytes(4, "little") + + (0xABCD0080).to_bytes(4, "little") + + (0x2080012B).to_bytes(4, "little") + + bytes(4) + ) + return "\n".join( + [ + "[ 1.0] NVRM: GE-GOLDEN-BEGIN", + "[ 1.1] NVRM: GE-CTRL cmd=0x2080012b hClient=0xc2000006 " + "hObject=0xabcd0080 size=0x8 paramp=0x0 ret=0x0", + "[ 1.1] NVRM: GE-CTRL 000:" + "".join(f" {b:02x}" for b in bytes(8)), + "[ 1.2] NVRM: GE-SEND fn=0x0000004c len=0x00000028 ret=0x0", + "[ 1.2] NVRM: GE-SEND 000:" + "".join(f" {b:02x}" for b in payload), + "[ 1.3] NVRM: GE-GOLDEN-END status=0x0", + ] + ) + "\n" + + +class FramingTests(unittest.TestCase): + def test_rpc_constants_match_capture_tool(self): + from harness.rm_capture import ( + GSP_RM_ALLOC as capture_alloc, + GSP_RM_CONTROL as capture_control, + RPC_HEADER_BYTES as capture_header_bytes, + ) + + self.assertEqual(RPC_HEADER_BYTES, capture_header_bytes) + self.assertEqual(GSP_RM_CONTROL, capture_control) + self.assertEqual(GSP_RM_ALLOC, capture_alloc) + self.assertEqual(RPC_HEADER_BYTES, 32) + + def test_rpc_message_roundtrip(self): + params = bytes(range(64)) + raw = build_rpc_message(RPC_FN_ECHO, params, sequence=7) + self.assertEqual(len(raw), RPC_HEADER_BYTES + len(params)) + message = parse_rpc_message(raw) + self.assertEqual(message.header_version, RPC_HEADER_VERSION) + self.assertEqual(message.signature, RPC_SIGNATURE) + self.assertEqual(message.function, RPC_FN_ECHO) + self.assertEqual(message.sequence, 7) + self.assertEqual(message.rpc_result, NV_OK) + self.assertEqual(message.length, len(raw)) + self.assertEqual(message.params, params) + self.assertEqual(message.to_bytes(), raw) + + def test_rpc_bad_signature_is_refused(self): + raw = bytearray(build_rpc_message(RPC_FN_ECHO, b"\x01", sequence=1)) + raw[4:8] = (0xDEADBEEF).to_bytes(4, "little") + with self.assertRaises(HostTrafficError): + parse_rpc_message(bytes(raw)) + + def test_rpc_short_and_overlong_length_is_refused(self): + raw = build_rpc_message(RPC_FN_ECHO, b"\x01\x02", sequence=1) + with self.assertRaises(HostTrafficError): + parse_rpc_message(raw[: RPC_HEADER_BYTES - 1]) + overlong = bytearray(raw) + overlong[8:12] = (len(raw) + 16).to_bytes(4, "little") + with self.assertRaises(HostTrafficError): + parse_rpc_message(bytes(overlong)) + + def test_queue_element_layout(self): + params = bytes(range(16)) + rpc = build_rpc_message(RPC_FN_ECHO, params, sequence=3) + element = build_queue_element(rpc, sequence=9) + self.assertEqual(GSP_MSG_QUEUE_ELEMENT_HDR_SIZE, 0x30) + self.assertEqual(element[0x30:0x34], rpc[:4]) + self.assertEqual(element[0x38:0x3C], len(rpc).to_bytes(4, "little")) + self.assertEqual(element[0x34:0x38], RPC_SIGNATURE.to_bytes(4, "little")) + self.assertEqual(element[0x24:0x28], (9).to_bytes(4, "little")) + self.assertEqual(element[0x28:0x2C], (1).to_bytes(4, "little")) + self.assertEqual(element[0x2C:0x30], bytes(4)) + self.assertEqual(element[0x50:], params) + prefix, parsed = parse_queue_element(element) + self.assertEqual(prefix["seq_num"], 9) + self.assertEqual(prefix["elem_count"], 1) + self.assertEqual(parsed.function, RPC_FN_ECHO) + self.assertEqual(parsed.params, params) + + def test_queue_element_checksum_verifies_and_detects_tamper(self): + rpc = build_rpc_message(RPC_FN_ECHO, bytes(range(32)), sequence=1) + element = build_queue_element(rpc, sequence=1) + self.assertEqual(compute_message_checksum(element), 0) + tampered = bytearray(element) + tampered[0x50] ^= 0x01 + self.assertNotEqual(compute_message_checksum(bytes(tampered)), 0) + with self.assertRaises(HostTrafficError): + parse_queue_element(bytes(tampered)) + + def test_multi_element_message_roundtrip(self): + params = bytes((i * 7) & 0xFF for i in range(5000)) + rpc = build_rpc_message(RPC_FN_ECHO, params, sequence=1) + element = build_queue_element(rpc, sequence=1) + self.assertEqual(element[0x28:0x2C], (2).to_bytes(4, "little")) + prefix, parsed = parse_queue_element(element) + self.assertEqual(prefix["elem_count"], 2) + self.assertEqual(parsed.params, params) + host = HostTraffic() + host.submit_rpc(RPC_FN_ECHO, params, sequence=1) + self.assertEqual(host.poll(), 1) + reply = host.read_reply() + self.assertEqual(reply["params"], params) + self.assertEqual(host.queues.status.header().write_ptr, 2) + host.close() + + +class MsgqTests(unittest.TestCase): + def test_queue_header_layout_matches_msgq_arithmetic(self): + mem = RegionMemory(SHARED_MEM_BASE, 0x40000) + queue = MsgqQueue(mem, SHARED_MEM_BASE, 0x40000) + queue.format() + header = queue.header() + self.assertEqual(header.version, 0) + self.assertEqual(header.size, 0x40000) + self.assertEqual(header.msg_size, GSP_MSG_QUEUE_ELEMENT_SIZE_MIN) + self.assertEqual(header.msg_count, (0x40000 - MSGQ_ENTRY_OFF) // 0x1000) + self.assertEqual(header.write_ptr, 0) + self.assertEqual(header.flags, MSGQ_FLAGS_SWAP_RX) + self.assertEqual(header.rx_hdr_off, MSGQ_RX_HDR_OFF) + self.assertEqual(header.entry_off, MSGQ_ENTRY_OFF) + self.assertEqual(MSGQ_RX_HDR_OFF, 32) + self.assertEqual(MSGQ_ENTRY_OFF, 48) + + def test_write_publish_and_read_back_entry(self): + mem = RegionMemory(SHARED_MEM_BASE, 0x40000) + queue = MsgqQueue(mem, SHARED_MEM_BASE, 0x40000) + queue.format() + payload = bytes(range(256)) + queue.write_entry(0, payload) + self.assertEqual(queue.read_entry(0, 256), payload) + queue.publish(1) + self.assertEqual(queue.write_ptr, 1) + queue.set_read(1) + self.assertEqual(queue.read_ptr, 1) + + def test_shared_queues_cross_link_rx_headers(self): + queues = SharedQueues(RegionMemory(SHARED_MEM_BASE, 0x80000)) + queues.format() + self.assertEqual(queues.command.rx_hdr_addr, queues.status.tx_base + MSGQ_RX_HDR_OFF) + self.assertEqual(queues.status.rx_hdr_addr, queues.command.tx_base + MSGQ_RX_HDR_OFF) + queues.command.set_read(5) + raw = queues.mem.read(queues.status.tx_base + MSGQ_RX_HDR_OFF, 4) + self.assertEqual(int.from_bytes(raw, "little"), 5) + queues.status.set_read(2) + raw = queues.mem.read(queues.command.tx_base + MSGQ_RX_HDR_OFF, 4) + self.assertEqual(int.from_bytes(raw, "little"), 2) + + def test_submit_and_take_roundtrip(self): + queues = SharedQueues(RegionMemory(SHARED_MEM_BASE, 0x80000)) + queues.format() + element = build_queue_element( + build_rpc_message(RPC_FN_ECHO, b"\xaa\xbb", sequence=4), sequence=0 + ) + first = queues.submit_element(element) + self.assertEqual(first["index"], 0) + self.assertEqual(queues.command.write_ptr, 1) + taken = queues.take_command() + self.assertIsNotNone(taken) + self.assertEqual(taken["element"], element) + self.assertEqual(queues.take_command(), None) + published = queues.publish_element(element) + self.assertEqual(published["index"], 0) + status = queues.take_status() + self.assertEqual(status["element"], element) + self.assertEqual(queues.take_status(), None) + + +class DoorbellTests(unittest.TestCase): + def test_model_records_writes(self): + host = HostTraffic() + host.doorbell.ring(0, 0) + host.doorbell.ring(0, 0x11) + host.doorbell.ring(2, 0x22) + self.assertEqual(host.doorbell.head(0), 0x11) + self.assertEqual(host.doorbell.head(2), 0x22) + self.assertEqual(host.doorbell.kicks, 3) + state = host.doorbell.state() + self.assertEqual(len(state["writes"]), 3) + self.assertEqual(state["writes"][-1]["queue"], 2) + with self.assertRaises(ValueError): + host.doorbell.ring(8, 0) + host.close() + + def test_block_serves_guest_read(self): + host = HostTraffic() + host.doorbell.ring(1, 0xCAFEF00D) + block = host.doorbell_block() + self.assertEqual(block.read(0, 4), 0) + self.assertEqual(block.read(8, 4), 0xCAFEF00D) + self.assertIsNone(block.read(8 * 8, 4)) + self.assertEqual(block.base, DOORBELL_BASE) + self.assertEqual(block.size, DOORBELL_SIZE) + host.close() + + def test_registry_overlay_beats_wider_falcon_window(self): + registry = DeviceRegistry() + registry.add(FalconBlock(), FALCON_BASE, 0x1000) + host = HostTraffic() + host.doorbell.ring(0, 0x1234) + registry.add(host.doorbell_block()) + handled, value = registry.read(DOORBELL_BASE, 4) + self.assertTrue(handled) + self.assertEqual(value, 0x1234) + handled, value = registry.read(FALCON_BASE + FALCON_DEBUGINFO, 4) + self.assertTrue(handled) + self.assertEqual(value, 0) + host.close() + + def test_guest_load_reads_doorbell(self): + machine = Machine(memory_mb=8, ram_base=0) + machine.load_segments([(0, PROGRAM_READ_DOORBELL)], entry=0) + host = HostTraffic() + host.doorbell.ring(0, 0x5A5A5A5A) + registry = DeviceRegistry() + registry.add(host.doorbell_block()) + machine.devices = registry + reason, _pc, _a, _b = machine.run(100) + self.assertEqual(reason, R_EBREAK) + self.assertEqual(machine.x[10], 0x5A5A5A5A) + host.close() + + def test_guest_store_is_recorded(self): + machine = Machine(memory_mb=8, ram_base=0) + machine.load_segments([(0, PROGRAM_WRITE_DOORBELL)], entry=0) + machine.apply_reset_regs({"a1": 0x77}) + host = HostTraffic() + registry = DeviceRegistry() + registry.add(host.doorbell_block()) + machine.devices = registry + reason, _pc, _a, _b = machine.run(100) + self.assertEqual(reason, R_EBREAK) + self.assertEqual(host.doorbell.head(2), 0x77) + host.close() + + +class TrafficTests(unittest.TestCase): + def test_submit_writes_ring_and_rings_doorbell(self): + host = HostTraffic() + raw = build_rpc_message(RPC_FN_ECHO, b"\x01\x02\x03", sequence=5) + result = host.submit(raw) + self.assertEqual(result["function"], RPC_FN_ECHO) + self.assertEqual(result["sequence"], 5) + self.assertEqual(result["index"], 0) + self.assertEqual(result["write_ptr"], 1) + self.assertEqual(host.queues.command.write_ptr, 1) + self.assertEqual(host.doorbell.kicks, 1) + self.assertEqual(host.doorbell.head(0), 0) + taken = host.queues.take_command() + self.assertIsNotNone(taken) + _prefix, parsed = parse_queue_element(taken["element"]) + self.assertEqual(parsed.params, b"\x01\x02\x03") + host.close() + + def test_poll_dispatches_echo(self): + host = HostTraffic() + host.submit_rpc(RPC_FN_ECHO, b"\xaa\xbb\xcc", sequence=1) + self.assertEqual(host.poll(), 1) + self.assertEqual(host.poll(), 0) + reply = host.read_reply() + self.assertIsNotNone(reply) + self.assertEqual(reply["function"], RPC_FN_ECHO) + self.assertEqual(reply["sequence"], 1) + self.assertEqual(reply["result"], NV_OK) + self.assertEqual(reply["params"], b"\xaa\xbb\xcc") + self.assertEqual(host.read_reply(), None) + host.close() + + def test_unknown_function_gets_not_supported(self): + host = HostTraffic() + host.submit_rpc(0x999, b"\x01", sequence=2) + self.assertEqual(host.poll(), 1) + reply = host.read_reply() + self.assertEqual(reply["function"], 0x999) + self.assertEqual(reply["result"], NV_ERR_NOT_SUPPORTED) + self.assertEqual(reply["params"], b"") + dispatched = [ + event for event in host.transcript()["events"] + if event["type"] == "dispatch" + ] + self.assertEqual(dispatched[0]["handler"], "stub:unsupported") + host.close() + + def test_status_query_reply_is_deterministic(self): + replies = [] + for _ in range(2): + host = HostTraffic() + host.submit_rpc(RPC_FN_ECHO, b"\x01", sequence=1) + host.submit_rpc(RPC_FN_STATUS_QUERY, b"", sequence=2) + host.poll() + first = host.read_reply() + second = host.read_reply() + replies.append((first["params"], second["params"])) + host.close() + self.assertEqual(replies[0], replies[1]) + self.assertEqual(replies[0][1][:4], STATUS_MAGIC) + + def test_empty_poll_and_read_reply(self): + host = HostTraffic() + self.assertEqual(host.poll(), 0) + self.assertIsNone(host.read_reply()) + host.close() + + def test_run_cycles_is_bounded(self): + machine = Machine(memory_mb=32) + host = HostTraffic(machine=machine) + host.submit_rpc(RPC_FN_ECHO, b"\x01", sequence=1) + result = host.run_cycles(64) + self.assertEqual(result["budget"], 64) + self.assertEqual(result["insns"], machine.insns) + self.assertEqual(result["dispatched"], 1) + self.assertEqual(result["replies"], 1) + with self.assertRaises(ValueError): + host.run_cycles(0) + with self.assertRaises(ValueError): + host.run_cycles(-1) + host.close() + + def test_oversized_message_is_refused(self): + host = HostTraffic() + with self.assertRaises(HostTrafficError): + host.submit_rpc(RPC_FN_ECHO, bytes(GSP_MSG_QUEUE_ELEMENT_SIZE_MIN * 16), + sequence=1) + host.close() + + def test_transcript_is_deterministic(self): + def run(): + host = HostTraffic() + host.submit_rpc(RPC_FN_ECHO, b"\x10\x20", sequence=1) + host.submit_rpc(RPC_FN_STATUS_QUERY, b"", sequence=2) + host.submit_rpc(0x999, b"", sequence=3) + host.poll() + while host.read_reply() is not None: + pass + transcript = host.transcript() + host.close() + return transcript + + first, second = run(), run() + self.assertEqual(first, second) + self.assertEqual(first["format"], "greeneye-host-traffic-log/1") + self.assertEqual( + [event["type"] for event in first["events"]], + ["submit", "submit", "submit", "dispatch", "dispatch", "dispatch", + "reply", "reply", "reply"], + ) + + +class ReplayTests(unittest.TestCase): + def test_replay_stimulus_in_order(self): + stimulus = { + "format": "greeneye-rm-rpc-stimulus/1", + "messages": [ + { + "seq": 0, + "kind": "send", + "fn": RPC_FN_ECHO, + "params_hex": "0102", + "truncated": False, + }, + { + "seq": 1, + "kind": "send", + "fn": RPC_FN_ECHO, + "params_hex": "0304", + "truncated": False, + }, + ], + } + host = HostTraffic() + report = replay_stimulus(host, stimulus) + self.assertEqual(report["format"], REPLAY_FORMAT) + self.assertEqual(report["submitted"], 2) + self.assertEqual(report["skipped"], []) + self.assertEqual([pair["reply"]["params"] for pair in report["pairs"]], + [b"\x01\x02", b"\x03\x04"]) + host.close() + + def test_replay_strict_rejects_truncated(self): + stimulus = { + "format": "greeneye-rm-rpc-stimulus/1", + "messages": [ + { + "seq": 0, + "kind": "control", + "params_hex": "00", + "truncated": True, + } + ], + } + host = HostTraffic() + with self.assertRaises(HostTrafficError): + replay_stimulus(host, stimulus, strict=True) + host.close() + + def test_replay_lenient_skips_truncated(self): + stimulus = { + "format": "greeneye-rm-rpc-stimulus/1", + "messages": [ + { + "seq": 0, + "kind": "send", + "fn": RPC_FN_ECHO, + "params_hex": "01", + "truncated": False, + }, + { + "seq": 1, + "kind": "control", + "params_hex": "00", + "truncated": True, + }, + ], + } + host = HostTraffic() + report = replay_stimulus(host, stimulus, strict=False) + self.assertEqual(report["submitted"], 1) + self.assertEqual(len(report["skipped"]), 1) + self.assertIn("truncated", report["skipped"][0]["reason"]) + host.close() + + def test_control_stimulus_maps_to_control_function(self): + stimulus = { + "format": "greeneye-rm-rpc-stimulus/1", + "messages": [ + { + "seq": 0, + "kind": "control", + "params_hex": "01020304", + "truncated": False, + } + ], + } + host = HostTraffic() + report = replay_stimulus(host, stimulus) + self.assertEqual(report["pairs"][0]["reply"]["function"], GSP_RM_CONTROL) + self.assertEqual(report["pairs"][0]["reply"]["result"], NV_ERR_NOT_SUPPORTED) + host.close() + + def test_load_stimulus_from_capture_document(self): + from harness.rm_capture import ( + capture_to_dict, + decode_text, + stimulus_from_capture, + ) + + capture = decode_text(_sample_capture_log(), source="fixture") + self.assertIsNotNone(stimulus_from_capture(capture)) + with tempfile.TemporaryDirectory() as tmp: + capture_path = os.path.join(tmp, "capture.json") + with open(capture_path, "w", encoding="utf-8") as fh: + json.dump(capture_to_dict(capture), fh) + loaded = load_stimulus(capture_path) + self.assertEqual(loaded["format"], "greeneye-rm-rpc-stimulus/1") + self.assertEqual(len(loaded["messages"]), 2) + + def test_load_stimulus_accepts_inline_document(self): + stimulus = {"format": "greeneye-rm-rpc-stimulus/1", "messages": []} + self.assertEqual(load_stimulus(stimulus), stimulus) + with self.assertRaises(HostTrafficError): + load_stimulus({"format": "not-a-stimulus"}) + + +class SessionTests(unittest.TestCase): + def test_recorder_events_and_blob(self): + with tempfile.TemporaryDirectory() as tmp: + recorder = Recorder(tmp, name="host-traffic-test", meta={"kind": "test"}) + host = HostTraffic(recorder=recorder) + host.submit_rpc(RPC_FN_ECHO, b"\x01", sequence=1) + host.poll() + host.read_reply() + host.close() + recorder.close() + events = load_events(tmp) + types = [event["type"] for event in events] + self.assertIn("host_traffic_config", types) + self.assertIn("doorbell", types) + self.assertIn("host_submit", types) + self.assertIn("host_dispatch", types) + self.assertIn("host_reply_read", types) + blob_path = os.path.join(tmp, "blobs", "host-traffic.json") + self.assertTrue(os.path.isfile(blob_path)) + with open(blob_path, "r", encoding="utf-8") as fh: + payload = json.load(fh) + self.assertEqual(payload["format"], "greeneye-host-traffic-log/1") + + +class DemoTests(unittest.TestCase): + def test_demo_end_to_end_and_deterministic(self): + first, second = run_demo(), run_demo() + self.assertTrue(first["ok"], first.get("checks")) + self.assertEqual(first["format"], DEMO_FORMAT) + self.assertEqual(first["replies"], second["replies"]) + self.assertEqual(first["transcript"], second["transcript"]) + self.assertEqual(first["control"]["bytes"], 512) + self.assertEqual(first["control"]["reply_result"], NV_ERR_NOT_SUPPORTED) + self.assertTrue(all(reply["read"] for reply in first["replies"])) + + def test_demo_writes_artifacts(self): + with tempfile.TemporaryDirectory() as tmp: + result = run_demo(out_dir=tmp) + self.assertTrue(result["ok"], result.get("checks")) + self.assertTrue(os.path.isfile(os.path.join(tmp, "blobs", "host-traffic.json"))) + self.assertTrue(os.path.isfile(os.path.join(tmp, "host-traffic-demo.json"))) + self.assertTrue(os.path.isfile(os.path.join(tmp, "session.json"))) + + def test_cli_demo(self): + import contextlib + import io + + with tempfile.TemporaryDirectory() as tmp: + out = io.StringIO() + with contextlib.redirect_stdout(out): + code = main(["demo", "--out", tmp]) + self.assertEqual(code, 0) + payload = json.loads(out.getvalue()) + self.assertEqual(payload["format"], DEMO_FORMAT) + self.assertTrue(payload["ok"]) + + def test_cli_replay(self): + import contextlib + import io + + stimulus = { + "format": "greeneye-rm-rpc-stimulus/1", + "messages": [ + { + "seq": 0, + "kind": "send", + "fn": RPC_FN_ECHO, + "params_hex": "0102", + "truncated": False, + } + ], + } + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "stimulus.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump(stimulus, fh) + out = io.StringIO() + with contextlib.redirect_stdout(out): + code = main(["replay", "--stimulus", path, "--out", tmp]) + self.assertEqual(code, 0) + payload = json.loads(out.getvalue()) + self.assertEqual(payload["format"], REPLAY_FORMAT) + + def test_cli_requires_a_subcommand(self): + import contextlib + import io + + err = io.StringIO() + with contextlib.redirect_stderr(err): + code = main([]) + self.assertEqual(code, 2) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_launch_audit.py b/debug/risc-v-gsp-rm-vm/tests/test_launch_audit.py new file mode 100644 index 0000000..55dce3e --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_launch_audit.py @@ -0,0 +1,477 @@ +"""Tests for the launch memory-contract audit (harness/launch_audit.py, docs/48). + +Offline only: synthetic page-table trees plus the captured channel-PDB vector. + +Run:: + + python3 -m unittest tests.test_launch_audit -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import struct +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.launch_audit import ( # noqa: E402 + AUDIT_FORMAT, + CHECK_FORMAT, + SPEC_FORMAT, + VERDICT_ACCESS_DENIED, + VERDICT_OK, + VERDICT_PA_MISMATCH, + VERDICT_UNMAPPED, + AuditError, + _tree_for_va, + audit, + check, + main, + run_demo, +) +from sim.gmmu_walk import ( # noqa: E402 + MemoryView, + encode_dual_slot, + encode_pde_pointer, + encode_pte, +) + + +def _spec(root_pa: int, mem: MemoryView, contract: list, **extra) -> dict: + spec = { + "format": SPEC_FORMAT, + "name": "test", + "gmmu": "ver2", + "page_mode": "4k", + "pdb_pa": f"{root_pa:#x}", + "memory": [{"pa": f"{pa:#x}", "hex": b.hex()} for pa, b in _blobs(mem)], + "contract": contract, + } + spec.update(extra) + return spec + + +def _blobs(mem: MemoryView): + return [(int(item["pa"], 16), mem.read(int(item["pa"], 16), item["size"])) + for item in mem.spans()] + + +def _drop(mem: MemoryView, drop_pas) -> MemoryView: + """Copy a memory view without the given table pages (a partial dump).""" + out = MemoryView() + for pa, blob in _blobs(mem): + if pa not in drop_pas: + out.add(pa, blob) + return out + + +def _run_main(argv): + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class AuditTests(unittest.TestCase): + def test_shadow_maps_to_expected_pa_is_ok(self): + va, shadow = 0x204608000, 0x15E8FA000 + root = 0x60000000 + report = audit(_spec(root, _tree_for_va(root, va, shadow), [ + {"label": "qmd-dest", "va": f"{va:#x}", "access": "write", + "expect_pa": f"{shadow:#x}"}, + ])) + self.assertEqual(report["format"], AUDIT_FORMAT) + row = report["contract"][0] + self.assertEqual(row["verdict"], VERDICT_OK) + self.assertTrue(report["ok"]) + + def test_host_checks_guest_page_is_pa_mismatch(self): + va, shadow = 0x204608000, 0x15E8FA000 + root = 0x60000000 + report = audit(_spec(root, _tree_for_va(root, va, shadow), [ + {"label": "qmd-dest", "va": f"{va:#x}", "access": "write", + "expect_pa": f"{va:#x}"}, + ])) + row = report["contract"][0] + self.assertEqual(row["verdict"], VERDICT_PA_MISMATCH) + self.assertFalse(report["ok"]) + self.assertIn("0x15e8fa000", row["detail"]) + + def test_unmapped_leaf_is_unmapped(self): + va = 0x204608000 + root = 0x60000000 + report = audit(_spec(root, _tree_for_va(root, va, None), [ + {"label": "qmd-dest", "va": f"{va:#x}", "access": "write"}, + ])) + self.assertEqual(report["contract"][0]["verdict"], VERDICT_UNMAPPED) + self.assertFalse(report["ok"]) + + def test_read_only_leaf_for_write_is_access_denied(self): + va, target = 0x204608000, 0x15E8FA000 + # build the same tree as _tree_for_va but with a read-only leaf + i0, i1 = (va >> 47) & 0x3, (va >> 38) & 0x1FF + i2, i3 = (va >> 29) & 0x1FF, (va >> 21) & 0xFF + i4 = (va >> 12) & 0x1FF + base = 0x50000000 + pd2, pd1, pd0, pt = base + 0x1000, base + 0x2000, base + 0x3000, base + 0x4000 + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0 * 8, encode_pde_pointer(pd2)) + mem.add(base, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3 * 16, encode_dual_slot(None, pt)) + mem.add(pd0, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i4 * 8, encode_pte(target, read_only=True)) + mem.add(pt, bytes(page)) + root = 0x60000000 # separate root alias is not needed; spec uses 'base' + report = audit(_spec(base, mem, [ + {"label": "release-slot", "va": f"{va:#x}", "access": "write"}, + ])) + self.assertEqual(report["contract"][0]["verdict"], VERDICT_ACCESS_DENIED) + self.assertFalse(report["ok"]) + self.assertNotEqual(root, base) + + def test_spec_errors(self): + with self.assertRaises(AuditError): + audit({"format": "nope"}) + with self.assertRaises(AuditError): + audit({"format": SPEC_FORMAT}) + with self.assertRaises(AuditError): + audit(_spec(0x60000000, MemoryView(), [], gmmu="ver9")) + with self.assertRaises(AuditError): + audit(_spec(0x60000000, MemoryView(), [{"label": "x"}])) + + def test_multiple_contract_legs_counted(self): + va, shadow = 0x204608000, 0x15E8FA000 + root = 0x60000000 + report = audit(_spec(root, _tree_for_va(root, va, shadow), [ + {"label": "qmd", "va": f"{va:#x}", "access": "write", + "expect_pa": f"{shadow:#x}"}, + {"label": "missing", "va": f"{va + 0x1000:#x}", "access": "read"}, + ])) + self.assertEqual(report["summary"][VERDICT_OK], 1) + self.assertEqual(report["summary"][VERDICT_UNMAPPED], 1) + self.assertFalse(report["ok"]) + + +class CheckTests(unittest.TestCase): + """The `check` mode: which table pages does a partial dump still need?""" + + def test_check_empty_memory_reports_root_per_leg(self): + spec = { + "format": SPEC_FORMAT, + "name": "empty-memory", + "gmmu": "ver2", + "page_mode": "4k", + "pdb_pa": "0x12f16e000", + "contract": [ + {"label": "qmd", "va": "0x204608000", "access": "write"}, + {"label": "ring", "va": "0x100000", "access": "read"}, + ], + } + report = check(spec) + self.assertEqual(report["format"], CHECK_FORMAT) + self.assertEqual( + set(report), + {"format", "name", "firmware", "gmmu", "page_mode", "pdb_pa", + "memory_pages", "contract", "needed_pages", "complete", "limits"}) + self.assertEqual(report["memory_pages"], 0) + self.assertFalse(report["complete"]) + for row in report["contract"]: + self.assertEqual(row["status"], "missing") + self.assertEqual(row["first_missing"]["pa"], "0x12f16e000") + self.assertEqual(row["first_missing"]["table_pa"], "0x12f16e000") + self.assertEqual(row["first_missing"]["level"], "root") + self.assertEqual(row["first_missing"]["depth"], 0) + self.assertFalse(row["first_missing"]["present"]) + self.assertEqual(row["path"][-1]["pa"], "0x12f16e000") + self.assertFalse(row["path"][-1]["present"]) + needs = report["needed_pages"] + self.assertEqual([page["pa"] for page in needs], ["0x12f16e000"]) + self.assertEqual(needs[0]["levels"], ["root"]) + self.assertEqual(needs[0]["legs"], ["qmd", "ring"]) + + def test_check_partial_memory_reports_first_missing_with_path(self): + va = 0x204608000 + root = 0x60000000 + pde1 = 0x50000000 + 0x2000 + full = _tree_for_va(root, va, 0x12345000) + partial = _drop(full, {pde1, 0x50000000 + 0x4000}) + report = check(_spec(root, partial, [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ])) + row = report["contract"][0] + self.assertEqual(row["status"], "missing") + missing = row["first_missing"] + index = (va >> 29) & 0x1FF + self.assertEqual(missing["pa"], f"{pde1:#x}") + self.assertEqual(missing["table_pa"], f"{pde1:#x}") + self.assertEqual(missing["level"], "pde1") + self.assertEqual(missing["depth"], 2) + self.assertEqual(missing["index"], index) + self.assertEqual(missing["entry_pa"], f"{pde1 + index * 8:#x}") + self.assertEqual([hop["level"] for hop in row["path"]], + ["root", "pde2", "pde1"]) + self.assertEqual(row["path"][0]["table_pa"], f"{root:#x}") + self.assertTrue(row["path"][0]["present"]) + self.assertFalse(row["path"][-1]["present"]) + self.assertEqual(report["needed_pages"][0]["pa"], f"{pde1:#x}") + self.assertEqual(report["needed_pages"][0]["legs"], ["qmd"]) + + def test_check_partial_memory_marks_complete_and_missing_legs(self): + root = 0x60000000 + base = 0x51000000 + pd2, pd1 = base + 0x1000, base + 0x2000 + pd0_done, pt_done = base + 0x3000, base + 0x4000 + pd0_missing = base + 0x5000 + target = 0x12345000 + va_done, va_miss = 0x100000, 0x204608000 + + def idx(va): + return ((va >> 47) & 3, (va >> 38) & 0x1FF, (va >> 29) & 0x1FF, + (va >> 21) & 0xFF, (va >> 12) & 0x1FF) + + i0d, i1d, i2d, i3d, i4d = idx(va_done) + i0m, i1m, i2m, _i3m, _i4m = idx(va_miss) + self.assertEqual((i0d, i1d), (i0m, i1m)) # share root[i0] and pd2[i1] + self.assertNotEqual(i2d, i2m) + + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0d * 8, encode_pde_pointer(pd2)) + mem.add(root, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1d * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2d * 8, encode_pde_pointer(pd0_done)) + struct.pack_into("<Q", page, i2m * 8, encode_pde_pointer(pd0_missing)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3d * 16, encode_dual_slot(None, pt_done)) + mem.add(pd0_done, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i4d * 8, encode_pte(target)) + mem.add(pt_done, bytes(page)) + + report = check(_spec(root, mem, [ + {"label": "done", "va": f"{va_done:#x}", "access": "read"}, + {"label": "miss", "va": f"{va_miss:#x}", "access": "write"}, + ])) + rows = {row["label"]: row for row in report["contract"]} + self.assertEqual(rows["done"]["status"], "complete") + self.assertEqual(rows["done"]["resolved_pa"], f"{target:#x}") + self.assertEqual(rows["miss"]["status"], "missing") + self.assertEqual(rows["miss"]["first_missing"]["pa"], f"{pd0_missing:#x}") + self.assertEqual(rows["miss"]["first_missing"]["level"], "pde0") + self.assertEqual(rows["miss"]["first_missing"]["index"], idx(va_miss)[3]) + self.assertEqual([page["pa"] for page in report["needed_pages"]], + [f"{pd0_missing:#x}"]) + self.assertFalse(report["complete"]) + + def test_check_needed_pages_dedup_and_ordering(self): + root = 0x60000000 + pd2_a, pd2_b = 0x70001000, 0x70000000 + va_a1, va_a2 = 0x204608000, 0x100000 + va_b = 0x800000000000 + self.assertEqual((va_a1 >> 47) & 3, 0) + self.assertEqual((va_a2 >> 47) & 3, 0) + self.assertEqual((va_b >> 47) & 3, 1) + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, 0, encode_pde_pointer(pd2_a)) + struct.pack_into("<Q", page, 8, encode_pde_pointer(pd2_b)) + mem.add(root, bytes(page)) + report = check(_spec(root, mem, [ + {"label": "b-leg", "va": f"{va_b:#x}", "access": "read"}, + {"label": "a1-leg", "va": f"{va_a1:#x}", "access": "write"}, + {"label": "a2-leg", "va": f"{va_a2:#x}", "access": "read"}, + ])) + pages = report["needed_pages"] + self.assertEqual([page["pa"] for page in pages], + [f"{pd2_b:#x}", f"{pd2_a:#x}"]) + self.assertEqual(pages[0]["levels"], ["pde2"]) + self.assertEqual(pages[0]["legs"], ["b-leg"]) + self.assertEqual(pages[1]["levels"], ["pde2"]) + self.assertEqual(pages[1]["legs"], ["a1-leg", "a2-leg"]) + self.assertFalse(report["complete"]) + + def test_check_fault_leg_needs_no_pages(self): + va = 0x204608000 + root = 0x60000000 + mem = _tree_for_va(root, va, None) # zero leaf entry -> pte_invalid + report = check(_spec(root, mem, [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ])) + row = report["contract"][0] + self.assertEqual(row["status"], "fault") + self.assertEqual(row["fault"], "pte_invalid") + self.assertIsNone(row["first_missing"]) + self.assertEqual(report["needed_pages"], []) + self.assertTrue(report["complete"]) + + def test_check_64k_dual_missing_big_table(self): + va = 0x204600000 + root, pd2, pd1, pd0 = 0x60000000, 0x50001000, 0x50002000, 0x50003000 + big_pt, small_pt = 0x80000000, 0x80001000 + i0, i1, i2 = (va >> 47) & 3, (va >> 38) & 0x1FF, (va >> 29) & 0x1FF + i3 = (va >> 21) & 0xFF + mem = MemoryView() + page = bytearray(0x1000) + struct.pack_into("<Q", page, i0 * 8, encode_pde_pointer(pd2)) + mem.add(root, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i1 * 8, encode_pde_pointer(pd1)) + mem.add(pd2, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<Q", page, i2 * 8, encode_pde_pointer(pd0)) + mem.add(pd1, bytes(page)) + page = bytearray(0x1000) + struct.pack_into("<16s", page, i3 * 16, + encode_dual_slot(big_pt, small_pt)) + mem.add(pd0, bytes(page)) + report = check(_spec(root, mem, [ + {"label": "prog", "va": f"{va:#x}", "access": "read"}, + ], page_mode="64k")) + row = report["contract"][0] + self.assertEqual(row["status"], "missing") + missing = row["first_missing"] + self.assertEqual(missing["pa"], f"{big_pt:#x}") + self.assertEqual(missing["level"], "pte") + self.assertEqual(missing["depth"], 4) + self.assertEqual(missing["index"], (va >> 16) & 0x1FF) + + def test_check_is_deterministic(self): + va = 0x204608000 + root = 0x60000000 + full = _tree_for_va(root, va, 0x12345000) + partial = _drop(full, {0x50002000}) + spec = _spec(root, partial, [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ]) + self.assertEqual(json.dumps(check(spec), sort_keys=True), + json.dumps(check(spec), sort_keys=True)) + + def test_check_spec_errors(self): + with self.assertRaises(AuditError): + check({"format": SPEC_FORMAT}) + with self.assertRaises(AuditError): + check({"format": "nope", "pdb_pa": "0x1000", "contract": []}) + with self.assertRaises(AuditError): + check(_spec(0x60000000, MemoryView(), [], gmmu="ver9")) + with self.assertRaises(AuditError): + check(_spec(0x60000000, MemoryView(), [{"label": "x"}])) + + def test_cli_check_exit_codes_and_artifact(self): + va = 0x204608000 + root = 0x60000000 + full = _tree_for_va(root, va, 0x12345000) + partial = _drop(full, {0x50002000}) + spec = _spec(root, partial, [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ]) + with tempfile.TemporaryDirectory() as tmp: + spec_path = os.path.join(tmp, "spec.json") + with open(spec_path, "w", encoding="utf-8") as fh: + json.dump(spec, fh) + rc, out, _ = _run_main(["check", "--spec", spec_path, "--out", tmp]) + self.assertEqual(rc, 3) + self.assertIn("need 0x50002000", out) + self.assertIn("dump next:", out) + report_path = os.path.join(tmp, "launch-audit-check.json") + self.assertTrue(os.path.isfile(report_path)) + with open(report_path, encoding="utf-8") as fh: + report = json.load(fh) + self.assertEqual(report["format"], CHECK_FORMAT) + self.assertEqual(report["needed_pages"][0]["pa"], "0x50002000") + # a complete view -> exit 0 + spec["memory"] = [{"pa": f"{pa:#x}", "hex": b.hex()} + for pa, b in _blobs(full)] + with open(spec_path, "w", encoding="utf-8") as fh: + json.dump(spec, fh) + rc, out, _ = _run_main(["check", "--spec", spec_path]) + self.assertEqual(rc, 0) + self.assertIn("complete", out) + + +class DemoAndCliTests(unittest.TestCase): + def test_demo_is_green_and_deterministic(self): + first = run_demo() + second = run_demo() + self.assertEqual(first["format"], "greeneye-launch-audit-demo/1") + self.assertTrue(first["ok"]) + self.assertEqual(json.dumps(first, sort_keys=True), + json.dumps(second, sort_keys=True)) + + def test_audit_is_deterministic(self): + va, shadow = 0x204608000, 0x15E8FA000 + root = 0x60000000 + spec = _spec(root, _tree_for_va(root, va, shadow), [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ]) + self.assertEqual(json.dumps(audit(spec), sort_keys=True), + json.dumps(audit(spec), sort_keys=True)) + + def test_cli_demo_and_hints(self): + rc, out, _ = _run_main(["demo"]) + self.assertEqual(rc, 0) + self.assertIn("captured channel contract: ok=True", out) + self.assertIn("engine-write-shadow-host-checks-guest", out) + rc, out, _ = _run_main(["capture-hints"]) + self.assertEqual(rc, 0) + self.assertIn("CAPTURE HINTS", out) + + def test_engine_write_shadow_demo_verdicts(self): + demo = run_demo() + scenarios = {s["name"]: s for s in demo["engine_write_shadow"]} + mismatch = scenarios["engine-write-shadow-host-checks-guest"] + aligned = scenarios["engine-write-shadow-bookkeeping-aligned"] + self.assertEqual(mismatch["expected_verdict"], VERDICT_PA_MISMATCH) + self.assertEqual( + mismatch["report"]["contract"][0]["verdict"], VERDICT_PA_MISMATCH) + self.assertEqual(aligned["expected_verdict"], VERDICT_OK) + self.assertEqual( + aligned["report"]["contract"][0]["verdict"], VERDICT_OK) + self.assertTrue(demo["ok"]) + # the original demo sections are still present and unchanged in shape + self.assertTrue(demo["captured_channel"]["ok"]) + self.assertEqual(len(demo["qmd_hypotheses"]), 3) + + def test_cli_run_writes_report_and_exit_codes(self): + va, shadow = 0x204608000, 0x15E8FA000 + root = 0x60000000 + spec = _spec(root, _tree_for_va(root, va, shadow), [ + {"label": "qmd", "va": f"{va:#x}", "access": "write"}, + ]) + with tempfile.TemporaryDirectory() as tmp: + spec_path = os.path.join(tmp, "spec.json") + with open(spec_path, "w", encoding="utf-8") as fh: + json.dump(spec, fh) + rc, out, _ = _run_main(["run", "--spec", spec_path, "--out", tmp]) + self.assertEqual(rc, 0) + self.assertTrue(os.path.isfile(os.path.join(tmp, "launch-audit.json"))) + # mismatch -> exit 3 + spec["contract"][0]["expect_pa"] = f"{va:#x}" + with open(spec_path, "w", encoding="utf-8") as fh: + json.dump(spec, fh) + rc, _, _ = _run_main(["run", "--spec", spec_path]) + self.assertEqual(rc, 3) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_libos_port.py b/debug/risc-v-gsp-rm-vm/tests/test_libos_port.py new file mode 100644 index 0000000..0f01c01 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_libos_port.py @@ -0,0 +1,291 @@ +"""libos port receive-path model tests (sim/libos_port.py, docs/51 section 7). + +Hermetic: a fake machine (register file only) and a fake base syscall model; +no image, no boot, no machine. The tests pin the port-syscall ABI observed on +the real 570.144 image (send args a0-a3, receive args a4-a7, wait id t1, +timeout t2, return a0/a4/a5/a6), the async receive post, the boot handshake +send, the host doorbell wake's external-interrupt handoff, and the strict +"never complete an uncompleted receive" rule. + +Offline only: no hardware, kernel module, mdev, guest, network or QEMU. + +Run:: + + cd open-iov/debug/risc-v-gsp-rm-vm + python3 -m unittest tests.test_libos_port -v +""" +from __future__ import annotations + +import json +import os +import sys +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.libos import INTERRUPT_ARM_EXTERNAL, TASK_PORT_SEND_RECV_WAIT # noqa: E402 +from sim.libos_port import ( # noqa: E402 + EXTINTR_WAKE_CONTRACT, + HANDSHAKE_CONTRACT, + REPORT_FORMAT, + SHUTTLE_ANY, + TIMEOUT_INFINITE, + LibosPortModel, + role_for, +) + + +class FakeMachine: + """Register file only: the port model never touches memory.""" + + def __init__(self): + self.x = [0] * 32 + self.insns = 0 + self.pc = 0 + + def set(self, index: int, value: int) -> None: + self.x[index] = value & 0xFFFFFFFFFFFFFFFF + + +class FakeBase: + """Minimal base syscall model: handles arm, records and stops the rest.""" + + name = "fake-libos" + + def __init__(self): + self.calls: list[int] = [] + self.armed_mask = 0 + self.stopped: dict | None = None + + def handle(self, machine, pc: int = 0): + number = int(machine.x[5]) + if number == INTERRUPT_ARM_EXTERNAL: + self.armed_mask |= int(machine.x[10]) + return True, 0 + self.calls.append(number) + self.stopped = {"num": number, "name": f"sys_{number}", "pc": int(pc)} + return False, None + + +def _port_call(machine, *, send=None, recv=None, wait=None, + timeout=TIMEOUT_INFINITE) -> None: + """Write one ``TASK_PORT_SEND_RECV_WAIT`` argument set (explicit fields).""" + machine.set(5, TASK_PORT_SEND_RECV_WAIT) # t0 + for index, value in enumerate(send or (0, 0, 0, 0), start=10): + machine.set(index, value) + for index, value in enumerate(recv or (0, 0, 0, 0), start=14): + machine.set(index, value) + machine.set(6, 0 if wait is None else wait) # t1 + machine.set(7, timeout) # t2 + + +class PortSurfaceTests(unittest.TestCase): + """Role table and argument contract.""" + + def test_observed_roles(self): + self.assertEqual(role_for(6), "extintr") + self.assertEqual(role_for(7), "notification") + self.assertEqual(role_for(8), "request") + self.assertEqual(role_for(12), "handshake") + self.assertEqual(role_for(13), "event") + self.assertEqual(role_for(99), "data") + + def test_async_post_keeps_receive_args_and_returns_ok(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + _port_call(machine, recv=(10, 6, 0, 0)) + handled, result = model.handle(machine, pc=0x4018038) + self.assertTrue(handled) + self.assertEqual(result, 0) + self.assertEqual(len(model.posts), 1) + self.assertEqual(model.posts[0]["shuttle"], 10) + self.assertEqual(model.posts[0]["role"], "extintr") + # The kernel returns LIBOS_OK through a0; the wrapper masks a0 only. + self.assertEqual(machine.x[14], 10) + self.assertEqual(machine.x[15], 6) + + def test_send_completes_with_the_transcribe_contract(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + _port_call(machine, send=(1, 12, 0, 0, ), wait=1) + handled, result = model.handle(machine, pc=0x4019CD4) + self.assertTrue(handled) + self.assertEqual(result, 0) + self.assertEqual(len(model.sends), 1) + self.assertEqual(model.sends[0]["contract"], HANDSHAKE_CONTRACT) + self.assertEqual(model.sends[0]["role"], "handshake") + self.assertEqual(machine.x[14], 0) # completed size + self.assertEqual(machine.x[15], 0) # originating task + self.assertEqual(machine.x[16], 1) # completed shuttle + self.assertEqual(machine.x[17], 0) + + def test_blocking_wait_stops_without_completion(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + _port_call(machine, recv=(9, 8, 0x4015070, 0x808)) + model.handle(machine, pc=0x4018038) + _port_call(machine, wait=SHUTTLE_ANY) + handled, result = model.handle(machine, pc=0x4019DF4) + self.assertFalse(handled) + self.assertIsNone(result) + self.assertIsNotNone(model.stopped) + self.assertEqual(model.stopped["reason"], "port-wait-blocks") + self.assertEqual(model.stopped["num"], TASK_PORT_SEND_RECV_WAIT) + self.assertEqual(model.waits[-1]["reason"], "port-wait-blocks") + + def test_unmodelled_syscall_is_recorded_and_stopped(self): + base = FakeBase() + model = LibosPortModel(base=base) + machine = FakeMachine() + machine.set(5, 6) # task_state_query + machine.set(10, 0) + handled, result = model.handle(machine, pc=0x401A46C) + self.assertFalse(handled) + self.assertIsNone(result) + self.assertEqual(model.stopped["num"], 6) + self.assertEqual(model.stopped["reason"], "unmodelled-syscall") + self.assertEqual(base.calls, [6]) + + +class DoorbellWakeTests(unittest.TestCase): + """The host wake's external-interrupt handoff (RTOS report contract).""" + + def test_kick_before_post_and_arm_is_delivered_at_post(self): + base = FakeBase() + model = LibosPortModel(base=base) + machine = FakeMachine() + event = model.host_kick(queue=0, value=0) + self.assertIsNone(event["delivered"]) + self.assertTrue(event["pending"]) + # Arm first, then post the extintr receive: the pending wake fills it. + machine.set(5, INTERRUPT_ARM_EXTERNAL) + machine.set(10, 0xFF) + model.handle(machine, pc=0x4018054) + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + self.assertEqual(model.kicks_pending, 0) + self.assertEqual([entry["shuttle"] for entry in model.completed], [10]) + self.assertEqual(model.completed[0]["payload_size"], 0) + self.assertEqual(model.completed[0]["origin_task"], 0) + self.assertEqual(model.completed[0]["contract"], EXTINTR_WAKE_CONTRACT) + + def test_wake_requires_an_arm(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + event = model.host_kick() + self.assertIsNone(event["delivered"]) + self.assertEqual(model.kicks_pending, 1) + self.assertEqual(model.completed, []) + # Without interrupts armed the wait still blocks (no fabricated wake). + _port_call(machine, wait=SHUTTLE_ANY) + handled, _ = model.handle(machine, pc=0x4019DF4) + self.assertFalse(handled) + + def test_arm_seeds_from_the_attached_base_model(self): + base = FakeBase() + base.armed_mask = 0xFF + model = LibosPortModel(base=base) + self.assertEqual(model.armed_mask, 0xFF) + machine = FakeMachine() + model.host_kick() + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + self.assertEqual(model.kicks_pending, 0) + self.assertEqual([entry["shuttle"] for entry in model.completed], [10]) + + def test_wait_returns_the_wake_with_the_argument_contract(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + model.host_kick() + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + machine.set(5, INTERRUPT_ARM_EXTERNAL) + machine.set(10, 0xFF) + model.handle(machine, pc=0x4018054) + _port_call(machine, wait=SHUTTLE_ANY) + handled, result = model.handle(machine, pc=0x4019DF4) + self.assertTrue(handled) + self.assertEqual(result, 0) + self.assertEqual(machine.x[14], 0) # completed size + self.assertEqual(machine.x[15], 0) # originating task + self.assertEqual(machine.x[16], 10) # completed shuttle + self.assertEqual(model.waits[-1]["completed"], 10) + self.assertEqual(model.waits[-1]["role"], "extintr") + + def test_one_wake_completes_one_receive_only(self): + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + model.host_kick() + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + machine.set(5, INTERRUPT_ARM_EXTERNAL) + machine.set(10, 0xFF) + model.handle(machine, pc=0x4018054) + self.assertEqual(len(model.completed), 1) + # Consume the wake, re-post and wait again: nothing completes. + _port_call(machine, wait=SHUTTLE_ANY) + handled, _ = model.handle(machine, pc=0x4019DF4) + self.assertTrue(handled) + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018C8) + _port_call(machine, wait=SHUTTLE_ANY) + handled, _ = model.handle(machine, pc=0x4019DF4) + self.assertFalse(handled) + self.assertEqual(model.completed, []) + + +class StateTests(unittest.TestCase): + """Deterministic, serialisable report state.""" + + def _run(self) -> dict: + model = LibosPortModel(base=FakeBase()) + machine = FakeMachine() + model.host_kick(queue=0, value=0) + _port_call(machine, recv=(10, 6, 0, 0)) + model.handle(machine, pc=0x4018038) + machine.set(5, INTERRUPT_ARM_EXTERNAL) + machine.set(10, 0xFF) + model.handle(machine, pc=0x4018054) + _port_call(machine, recv=(9, 8, 0x4015070, 0x808)) + model.handle(machine, pc=0x4018038) + _port_call(machine, wait=SHUTTLE_ANY) + model.handle(machine, pc=0x4019DF4) + return model.state() + + def test_state_is_byte_identical_across_runs(self): + first = self._run() + second = self._run() + self.assertEqual(first["format"], REPORT_FORMAT) + self.assertEqual(json.dumps(first, sort_keys=True), + json.dumps(second, sort_keys=True)) + self.assertEqual(first["armed_mask"], 0xFF) + self.assertEqual(first["host_kicks"], 1) + self.assertEqual(first["kicks_pending"], 0) + self.assertEqual([post["role"] for post in first["posts"]], + ["extintr", "request"]) + self.assertEqual(first["completed"], []) + self.assertEqual(first["waits"][-1]["completed"], 10) + + def test_state_has_no_host_timestamps(self): + state = self._run() + + def keys(value): + if isinstance(value, dict): + for key, item in value.items(): + yield key + yield from keys(item) + elif isinstance(value, list): + for item in value: + yield from keys(item) + + for key in keys(state): + self.assertNotIn(str(key), ("t", "ts", "timestamp", "time")) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_log_decode.py b/debug/risc-v-gsp-rm-vm/tests/test_log_decode.py new file mode 100644 index 0000000..b43dc00 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_log_decode.py @@ -0,0 +1,335 @@ +"""Tests for the offline GSP-RM log decoder (harness/log_decode.py). + +Crafted inputs cover both real dump forms: the raw ring hexdump +(``LOGINIT-raw:`` + rows) and the dmesg/evidence record lines +(``rec seq=.. recType=.. errorCode=.. source=..`` plus ``assertCallStack`` +frames, ``entry[k]:`` aggregates and the liblogdecode ``rec NNN ts=..`` +text form). A real-file test runs only when the campaign dump is present. + +Run:: + + cd TOOL_ROOT && python3 -m unittest tests.test_log_decode -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.log_decode import ( # noqa: E402 + SCHEMA, + LogDecodeError, + apply_filters, + decode_path, + decode_text, + load_symbols, + main, + run_demo, +) + +REAL_LOGDUMP = ( + "/home/rmcore/Git/integdev_gpu_drv/open-iov/backups/" + "evidence-20261002-gr/21-logdump-grctx4.txt" +) + +RAW_TEXT = """\ +vgsp-boot: libos task log buffers (read-only) +LOGINIT-raw: 64 bytes (allocated 64) VA=0xffff0000 dma=0x1000 +00000000: 00000005 00000000 0000000a deadbeef +00000010: 01020304 05060708 090a0b0c 0d0e0f10 +""" + +RECORDS_TEXT = """\ +[Tue Oct 6 12:40:52 2026] vgsp-nocat: rec seq=1 recType=5 errorCode=0x57f1344 subsystem=0x0 bugcheck=0x0 payload=1212 source="ASSERT" faultingEngine="" +[Tue Oct 6 12:40:52 2026] vgsp-nocat: rec seq=1 assertCallStack count=1 [0x401000,0x402000,0x0] (offsets from load base 0x4000000) +[Tue Oct 6 12:40:53 2026] vgsp-nocat: entry[0]: count=1 recType=5 errorCode=0x57f1344 firstSeq=1 lastSeq=1 source="ASSERT" +an unmatched filler line +rec 7 ts=0x18dbb29118ec6200 meta=0x0000000005bb9c48 args=0x401000,0x402000 +rec 9 ts=0x18dbb29118ec7280 meta=0x0000000005bb9bd0 args=[0x10, 0x20] +""" + +SYMBOLS = { + "symbols": [ + {"name": "gr_init", "addr": "0x401000"}, + {"name": "gr_done", "addr": 0x403000}, + ] +} + + +def _write(tmp: str, name: str, text: str) -> str: + path = os.path.join(tmp, name) + with open(path, "w", encoding="utf-8") as fh: + fh.write(text) + return path + + +def _records(report: dict, kind: str = "record") -> list[dict]: + return [r for r in report["records"] if r["kind"] == kind] + + +class RawHexTests(unittest.TestCase): + def test_header_and_rows_decode_to_words(self): + report = decode_text(RAW_TEXT, source="raw.txt") + self.assertEqual(report["schema"], SCHEMA) + self.assertEqual(report["source"], "raw.txt") + self.assertEqual(report["count"], 1) + self.assertEqual(report["unparsed"], 1) # the intro line is not a record + raw = report["records"][0] + self.assertEqual(raw["kind"], "raw-hex") + self.assertEqual(raw["label"], "LOGINIT") + self.assertEqual(raw["source_name"], "LOGINIT") + self.assertEqual(len(raw["frames"]), 8) + self.assertEqual(raw["frames"][0], "0x5") + self.assertEqual(raw["frames"][3], "0xdeadbeef") + self.assertEqual(raw["frames"][7], "0xd0e0f10") + # raw is the decoded little-endian byte buffer of the rows + self.assertEqual(len(raw["raw"]), 2 * 8 * 8) + self.assertEqual(bytes.fromhex(raw["raw"])[:8].hex(), "0500000000000000") + self.assertEqual(raw["raw"].endswith("100f0e0d00000000"), True) + + def test_two_sections_stay_separate(self): + text = RAW_TEXT + ( + "LOGRM-raw: 16 bytes (allocated 16) VA=0x1 dma=0x2\n" + "00000000: 11111111 22222222 33333333 44444444\n" + ) + report = decode_text(text) + raw = _records(report, "raw-hex") + self.assertEqual(len(raw), 2) + self.assertEqual(raw[1]["label"], "LOGRM") + self.assertEqual(raw[1]["frames"], ["0x11111111", "0x22222222", + "0x33333333", "0x44444444"]) + + def test_blank_and_comment_lines_are_structure(self): + report = decode_text("\n# LOGINIT\n" + RAW_TEXT) + self.assertEqual(report["unparsed"], 1) # only the intro line + + +class RecordLineTests(unittest.TestCase): + def setUp(self): + self.report = decode_text(RECORDS_TEXT) + self.rec = self.report["records"][0] + + def test_rec_line_fields(self): + self.assertEqual(self.rec["kind"], "record") + self.assertEqual(self.rec["label"], "vgsp-nocat") + self.assertEqual(self.rec["seq"], 1) + self.assertEqual(self.rec["rec_type"], 5) + self.assertEqual(self.rec["error_code"], "0x57f1344") + self.assertEqual(self.rec["source_name"], "ASSERT") + + def test_assert_callstack_merges_frames(self): + self.assertEqual(self.rec["frames"], ["0x401000", "0x402000", "0x0"]) + # no duplicate standalone continuation record was emitted + self.assertEqual(len(_records(self.report)), 4) + + def test_entry_aggregate_line(self): + entry = self.report["records"][1] + self.assertEqual(entry["rec_type"], 5) + self.assertEqual(entry["error_code"], "0x57f1344") + self.assertEqual(entry["source_name"], "ASSERT") + self.assertIn("entry[0]:", entry["raw"]) + self.assertIn("vgsp-nocat:", entry["raw"]) + + def test_decoded_rec_shapes_with_and_without_brackets(self): + records = _records(self.report) + plain, bracketed = records[2], records[3] + self.assertEqual(plain["seq"], 7) + self.assertEqual(plain["frames"], ["0x401000", "0x402000"]) + self.assertEqual(bracketed["seq"], 9) + self.assertEqual(bracketed["frames"], ["0x10", "0x20"]) + + def test_unparsed_accounting(self): + self.assertEqual(self.report["unparsed"], 1) + self.assertEqual(len(self.report["records"]), 4) + + def test_histogram(self): + hist = self.report["histogram"] + self.assertEqual(hist["rec_type"], {"5": 2}) + self.assertEqual(hist["error_code"], {"0x57f1344": 2}) + self.assertEqual(hist["source"], {"ASSERT": 2}) + + def test_standalone_continuation_becomes_record(self): + text = ("vgsp-nocat: rec seq=4 assertCallStack count=1 " + "[0x1000,0x2000] (offsets from load base 0x4000000)\n") + report = decode_text(text) + self.assertEqual(report["count"], 1) + self.assertEqual(report["records"][0]["seq"], 4) + self.assertEqual(report["records"][0]["frames"], ["0x1000", "0x2000"]) + + +class SymbolTests(unittest.TestCase): + def test_nearest_below_with_offset(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(tmp, "symbols.json", json.dumps(SYMBOLS)) + table = load_symbols(path) + self.assertEqual(table.resolve(0x401000), ("gr_init", 0x0)) + self.assertEqual(table.resolve(0x402fff), ("gr_init", 0x1FFF)) + self.assertEqual(table.resolve(0x403000), ("gr_done", 0x0)) + self.assertIsNone(table.resolve(0x400fff)) # below the first symbol + self.assertIsNone(table.resolve(0x0)) + + def test_top_level_list_shape_accepted(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(tmp, "symbols.json", + json.dumps([{"name": "a", "addr": "0x10"}])) + table = load_symbols(path) + self.assertEqual(table.resolve(0x1f), ("a", 0xF)) + + def test_resolved_entries_align_with_frames(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(tmp, "symbols.json", json.dumps(SYMBOLS)) + report = decode_text(RECORDS_TEXT, symbols=load_symbols(path)) + resolved = report["records"][0]["resolved"] + self.assertEqual(resolved[0], + {"frame": "0x401000", "symbol": "gr_init", + "offset": "0x0"}) + self.assertEqual(resolved[1], + {"frame": "0x402000", "symbol": "gr_init", + "offset": "0x1000"}) + self.assertEqual(resolved[2], + {"frame": "0x0", "symbol": None, "offset": None}) + + def test_malformed_symbols_fail_cleanly(self): + cases = [ + "{not json", + json.dumps([{"name": "no-addr"}]), + json.dumps({"symbols": "nope"}), + json.dumps({"nope": []}), + json.dumps([42]), + json.dumps([{"name": "x", "addr": "zzz"}]), + ] + with tempfile.TemporaryDirectory() as tmp: + for index, payload in enumerate(cases): + path = _write(tmp, f"bad{index}.json", payload) + with self.assertRaises(LogDecodeError): + load_symbols(path) + self.assertRaises(LogDecodeError, load_symbols, + os.path.join(tmp, "missing.json")) + # the CLI exits non-zero with a clear message, not a traceback + good = _write(tmp, "good.txt", RECORDS_TEXT) + bad = _write(tmp, "bad.json", cases[0]) + stderr = io.StringIO() + with contextlib.redirect_stderr(stderr): + code = main(["decode", good, "--symbols", bad]) + self.assertEqual(code, 2) + self.assertIn("symbols JSON", stderr.getvalue()) + + +class FilterTests(unittest.TestCase): + def setUp(self): + self.report = decode_text(RECORDS_TEXT) + self.raw_report = decode_text(RAW_TEXT) + + def test_kind_filter(self): + self.assertEqual(apply_filters(self.report, kind="record")["count"], 4) + self.assertEqual(apply_filters(self.report, kind="raw-hex")["count"], 0) + self.assertEqual(apply_filters(self.raw_report, kind="raw-hex")["count"], 1) + + def test_rec_type_filter_accepts_hex_and_decimal(self): + for value in ("5", "0x5"): + filtered = apply_filters(self.report, rec_type=value) + self.assertEqual(filtered["count"], 2) + self.assertTrue(all(r["rec_type"] == 5 for r in filtered["records"])) + + def test_error_code_filter(self): + filtered = apply_filters(self.report, error_code="0x57f1344") + self.assertEqual(filtered["count"], 2) + zeros = apply_filters(self.report, error_code="0x1") + self.assertEqual(zeros["count"], 0) + + def test_source_filter_is_case_insensitive_substring(self): + self.assertEqual(apply_filters(self.report, source="assert")["count"], 2) + self.assertEqual(apply_filters(self.report, source="Display")["count"], 0) + + def test_limit_and_combined_filters(self): + self.assertEqual(apply_filters(self.report, limit=1)["count"], 1) + combined = apply_filters(self.report, kind="record", source="ASSERT", + limit=1) + self.assertEqual(combined["count"], 1) + self.assertEqual(combined["records"][0]["seq"], 1) + self.assertEqual(apply_filters(self.report, limit=0)["count"], 0) + + def test_bad_filter_value_raises(self): + with self.assertRaises(LogDecodeError): + apply_filters(self.report, rec_type="zzz") + with self.assertRaises(LogDecodeError): + apply_filters(self.report, kind="bogus") + with self.assertRaises(LogDecodeError): + apply_filters(self.report, limit=-1) + + def test_histogram_follows_filter(self): + filtered = apply_filters(self.report, kind="record", limit=1) + self.assertEqual(filtered["histogram"]["rec_type"], {"5": 1}) + self.assertEqual(filtered["unparsed"], self.report["unparsed"]) + + +class CliTests(unittest.TestCase): + def test_decode_writes_json_report(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(tmp, "records.txt", RECORDS_TEXT) + out = os.path.join(tmp, "out", "report.json") + stdout = io.StringIO() + with contextlib.redirect_stdout(stdout): + code = main(["decode", path, "--json", out, + "--source", "ASSERT", "--limit", "1"]) + self.assertEqual(code, 0) + with open(out, "r", encoding="utf-8") as fh: + report = json.load(fh) + self.assertEqual(report["schema"], SCHEMA) + self.assertEqual(report["source"], "records.txt") + self.assertEqual(report["count"], 1) + self.assertIn("report_json", stdout.getvalue()) + + def test_missing_input_fails_cleanly(self): + stderr = io.StringIO() + with contextlib.redirect_stderr(stderr): + code = main(["decode", "/nonexistent/log.txt"]) + self.assertEqual(code, 2) + self.assertIn("input not found", stderr.getvalue()) + + def test_demo_runs_deterministic(self): + first = run_demo() + second = run_demo() + self.assertTrue(first["ok"], first) + self.assertEqual(first["raw"]["count"], second["raw"]["count"]) + self.assertEqual(first["records"]["records"][0]["frames"], + second["records"]["records"][0]["frames"]) + stdout = io.StringIO() + with contextlib.redirect_stdout(stdout): + code = main(["demo"]) + self.assertEqual(code, 0) + self.assertIn('"ok": true', stdout.getvalue()) + + +@unittest.skipUnless(os.path.isfile(REAL_LOGDUMP), + f"real logdump not present: {REAL_LOGDUMP}") +class RealLogdumpTests(unittest.TestCase): + """Decode the campaign dump when it is present; skipped otherwise.""" + + def test_real_logdump_decodes(self): + report = decode_path(REAL_LOGDUMP) + self.assertEqual(report["schema"], SCHEMA) + self.assertEqual(report["source"], os.path.basename(REAL_LOGDUMP)) + self.assertGreater(report["count"], 0) + self.assertTrue(any(report["histogram"].values()), + report["histogram"]) + raw_labels = {r["label"] for r in _records(report, "raw-hex")} + self.assertEqual(raw_labels, {"LOGINIT", "LOGRM"}) + framed = [r for r in _records(report) + if r.get("seq") is not None and r.get("frames")] + self.assertTrue(framed) + self.assertEqual(report["count"], + len(report["records"])) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_msgq_attach.py b/debug/risc-v-gsp-rm-vm/tests/test_msgq_attach.py new file mode 100644 index 0000000..749157e --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_msgq_attach.py @@ -0,0 +1,342 @@ +"""Message-queue attach tests (sim/msgq_attach.py, docs/51 section 7). + +Hermetic tests drive the pure classification/report functions with crafted +inputs -- no image, no boot, no machine. One image test runs the real 570.144 +firmware with a small bounded budget when the image is present and asserts +honest, deterministic outcomes only: it never requires the message loop to be +reached, because the tool must report a first divergence rather than fake one. + +Offline only: no hardware, kernel module, mdev, guest, network or QEMU. + +Run:: + + cd open-iov/debug/risc-v-gsp-rm-vm + python3 -m unittest tests.test_msgq_attach -v +""" +from __future__ import annotations + +import os +import sys +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.host_traffic import ( # noqa: E402 + DOORBELL_BASE, + DOORBELL_SIZE, + GSP_MSG_QUEUE_ELEMENT_HDR_SIZE, + GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + MSGQ_ENTRY_OFF, + MSGQ_RX_HDR_OFF, + MSGQ_TX_HEADER_BYTES, + SHARED_MEM_BASE, + HostTraffic, +) +from sim.msgq_attach import ( # noqa: E402 + DEFAULT_IMAGE, + MAX_RUN_CYCLES, + REPORT_FORMAT, + MsgqAttachError, + RingGeometry, + assemble_report, + classify_ring_access, + missing_semantic_after_handoff, + ring_geometry_from_host, + run_attach, +) + +#: Small total budget: boot-to-B3 needs ~474k instructions; the rest is a +#: bounded continuation tail. Two runs stay well under the test time cap. +SMALL_BUDGET = 600_000 + + +def _geometry() -> RingGeometry: + return RingGeometry( + shared_base=SHARED_MEM_BASE, + cmd_base=SHARED_MEM_BASE, + stat_base=SHARED_MEM_BASE + 0x40000, + cmd_size=0x40000, + stat_size=0x40000, + msg_size=GSP_MSG_QUEUE_ELEMENT_SIZE_MIN, + entry_off=MSGQ_ENTRY_OFF, + rx_hdr_off=MSGQ_RX_HDR_OFF, + doorbell_base=DOORBELL_BASE, + doorbell_size=DOORBELL_SIZE, + ) + + +def _image() -> dict: + return {"path": "/nonexistent/image.bin", "sha256": "0" * 64, "size": 1} + + +def _profile() -> dict: + return {"name": "crafted", "path": "/nonexistent/profile.json"} + + +def _boot(reached: bool) -> dict: + return { + "ran": True, + "skipped": False, + "rung": "B3" if reached else "B1", + "message_loop_wait_reached": reached, + "instructions_executed": 474043 if reached else 1000, + "stop": {"reason": "ecall" if reached else "budget", + "pc": 0x4018038 if reached else 0x4009000}, + } + + +def _assemble(reached: bool, polls, divergence) -> dict: + return assemble_report( + image=_image(), + profile=_profile(), + boot=_boot(reached), + attach={"cmd_base": SHARED_MEM_BASE}, + doorbell={"requested": True}, + polls=polls, + divergence=divergence, + missing_mmio={"count": 0, "first": None}, + determinism={"instructions": 474043 if reached else 1000}, + ) + + +class ClassifierTests(unittest.TestCase): + """Pure ring-access classification (no machine, no boot).""" + + def test_geometry_from_host_matches_the_attached_layout(self): + host = HostTraffic() + try: + geometry = ring_geometry_from_host(host) + finally: + host.close() + self.assertEqual(geometry.cmd_base, SHARED_MEM_BASE) + self.assertEqual(geometry.stat_base, SHARED_MEM_BASE + 0x40000) + self.assertEqual(geometry.msg_size, GSP_MSG_QUEUE_ELEMENT_SIZE_MIN) + self.assertEqual(geometry.entry_off, MSGQ_ENTRY_OFF) + self.assertEqual(geometry.rx_hdr_off, MSGQ_RX_HDR_OFF) + self.assertEqual(geometry.doorbell_base, DOORBELL_BASE) + self.assertEqual(geometry.doorbell_size, DOORBELL_SIZE) + + def test_queue_head_classification(self): + geometry = _geometry() + poll = classify_ring_access(geometry.doorbell_base + 8, 4, 0x11, geometry) + self.assertTrue(poll["in_ring"]) + self.assertEqual(poll["kind"], "queue-head") + self.assertEqual(poll["queue"], 1) + self.assertEqual(poll["register"], "NV_PGSP_QUEUE_HEAD(1)") + self.assertEqual(poll["window"], "PRIV+0x110c00") + self.assertEqual(poll["value"], 0x11) + + def test_tx_header_classification(self): + geometry = _geometry() + poll = classify_ring_access(geometry.cmd_base + 16, 4, 0, geometry) + self.assertTrue(poll["in_ring"]) + self.assertEqual(poll["kind"], "tx-header") + self.assertEqual(poll["buffer"], "command") + self.assertEqual(poll["field"], "writePtr") + self.assertEqual(poll["field_offset"], 16) + self.assertEqual(MSGQ_TX_HEADER_BYTES, 32) + + def test_rx_header_classification_carries_the_cross_link(self): + geometry = _geometry() + poll = classify_ring_access(geometry.stat_base + MSGQ_RX_HDR_OFF, 4, 0, + geometry) + self.assertEqual(poll["kind"], "rx-header") + self.assertEqual(poll["buffer"], "status") + self.assertEqual(poll["cross_linked_owner"], "command") + self.assertEqual(poll["semantics"], "readPtr") + + def test_entry_classification_and_slot(self): + geometry = _geometry() + addr = (geometry.cmd_base + MSGQ_ENTRY_OFF + + 3 * GSP_MSG_QUEUE_ELEMENT_SIZE_MIN + 0x40) + poll = classify_ring_access(addr, 4, 0xAB, geometry) + self.assertEqual(poll["kind"], "entry") + self.assertEqual(poll["buffer"], "command") + self.assertEqual(poll["entry_index"], 3) + self.assertEqual(poll["slot_offset"], 0x40) + self.assertEqual(poll["region"], "entry-payload") + header = classify_ring_access( + geometry.cmd_base + MSGQ_ENTRY_OFF + 0x28, 4, 0, geometry) + self.assertEqual(header["region"], "entry-header") + self.assertEqual(GSP_MSG_QUEUE_ELEMENT_HDR_SIZE, 0x30) + + def test_non_ring_access_is_other(self): + geometry = _geometry() + poll = classify_ring_access(0xDEAD, 4, 0, geometry) + self.assertFalse(poll["in_ring"]) + self.assertEqual(poll["kind"], "other") + + +class ReportTests(unittest.TestCase): + """Pure report assembly: schema, outcomes and honest divergence.""" + + def test_not_reached_keeps_divergence_and_no_poll(self): + geometry = _geometry() + divergence = {"kind": "budget", + "reason": "message-loop port wait not reached within budget"} + report = _assemble(False, [], divergence) + self.assertEqual(report["format"], REPORT_FORMAT) + self.assertEqual(report["outcome"], "not-reached") + self.assertFalse(report["ok"]) + self.assertIsNone(report["first_poll"]) + self.assertEqual(report["subsequent_polls"], []) + self.assertEqual(report["poll_count"], 0) + self.assertTrue(report["divergence"]["reason"]) + self.assertEqual(report["determinism"]["report_sha256"], + _assemble(False, [], divergence)["determinism"]["report_sha256"]) + self.assertEqual(geometry.doorbell_base, DOORBELL_BASE) + + def test_reached_with_poll_is_classified(self): + geometry = _geometry() + poll = classify_ring_access(geometry.doorbell_base, 4, 0, geometry) + poll["pc"] = 0x4019000 + report = _assemble(True, [poll], None) + self.assertEqual(report["outcome"], "wait-reached-poll-classified") + self.assertTrue(report["ok"]) + self.assertEqual(report["first_poll"]["kind"], "queue-head") + self.assertEqual(report["first_poll"]["pc"], 0x4019000) + self.assertEqual(report["subsequent_polls"], []) + self.assertIsNone(report["divergence"]) + + def test_reached_without_poll_is_honest(self): + report = _assemble(True, [], None) + self.assertEqual(report["outcome"], "wait-reached-no-poll") + self.assertTrue(report["ok"]) + self.assertIsNone(report["first_poll"]) + + def test_subsequent_polls_are_bounded_to_sixteen(self): + geometry = _geometry() + polls = [] + for index in range(20): + poll = classify_ring_access(geometry.cmd_base + 8, 4, index, geometry) + poll["pc"] = 0x4018000 + index + polls.append(poll) + report = _assemble(True, polls, None) + self.assertEqual(report["poll_count"], 20) + self.assertEqual(report["first_poll"]["value"], 0) + self.assertEqual(len(report["subsequent_polls"]), 16) + self.assertEqual(report["subsequent_polls"][-1]["value"], 16) + + def test_absent_image_is_a_clean_skip(self): + report = run_attach(image="/nonexistent/gsp-image.bin", + max_cycles=SMALL_BUDGET) + self.assertEqual(report["format"], REPORT_FORMAT) + self.assertEqual(report["outcome"], "skipped") + self.assertFalse(report["ok"]) + self.assertTrue(report["boot"]["reason"]) + + def test_bad_budget_is_refused(self): + with self.assertRaises(MsgqAttachError): + run_attach(image="/nonexistent/gsp-image.bin", max_cycles=0) + with self.assertRaises(MsgqAttachError): + run_attach(image="/nonexistent/gsp-image.bin", + max_cycles=MAX_RUN_CYCLES + 1) + with self.assertRaises(MsgqAttachError): + run_attach(image="/nonexistent/gsp-image.bin", max_cycles=True) + + +class PortModelReportTests(unittest.TestCase): + """The additive port_model / missing_semantic report fields.""" + + def test_defaults_are_additive_and_honest(self): + report = _assemble(True, [], None) + self.assertEqual(report["port_model"], {"enabled": False}) + self.assertIsNone(report["missing_semantic"]) + self.assertEqual(report["format"], REPORT_FORMAT) + + def test_port_model_state_is_kept_verbatim(self): + state = {"enabled": True, "format": "greeneye-msgq-port/1", + "armed_mask": 0xFF, "host_kicks": 1} + report = assemble_report( + image=_image(), + profile=_profile(), + boot=_boot(True), + attach={}, + doorbell={"requested": True}, + polls=[], + divergence=None, + missing_mmio={"count": 0, "first": None}, + determinism={"instructions": 1}, + port_model=state, + missing_semantic={"stage": "after-extintr-handoff"}, + ) + self.assertTrue(report["port_model"]["enabled"]) + self.assertEqual(report["port_model"]["armed_mask"], 0xFF) + self.assertEqual(report["missing_semantic"]["stage"], + "after-extintr-handoff") + + def test_missing_semantic_after_handoff_is_precise(self): + stopped = {"num": 6, "name": "task_state_query", "pc": 0x401A46C, + "reason": "unmodelled-syscall"} + missing = missing_semantic_after_handoff(stopped=stopped, polls=[]) + self.assertEqual(missing["stage"], "after-extintr-handoff") + self.assertEqual(missing["stop"]["num"], 6) + self.assertEqual(missing["polls"], 0) + self.assertGreaterEqual(len(missing["missing"]), 3) + self.assertTrue(missing["next_step"]) + self.assertTrue(all(isinstance(item, str) for item in missing["observed"])) + + +@unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "real GSP firmware absent") +class ImageTests(unittest.TestCase): + """Bounded real-firmware run: deterministic, honest outcome only. + + Limitation: the budget is deliberately small (600k instructions; boot-to-B3 + needs ~474k). If the firmware needs more, the report is expected to take + the honest ``not-reached`` path with a non-empty divergence reason -- this + test accepts either outcome and asserts only determinism, never success. + With the default port model the run advances past the host wake to the + external-interrupt dispatch and reports the exact unmodelled semantic + (``task-state-query``); a fabricated poll is never accepted. + """ + + def test_bounded_run_is_deterministic_and_honest(self): + first = run_attach(image=DEFAULT_IMAGE, max_cycles=SMALL_BUDGET, + doorbell=True) + second = run_attach(image=DEFAULT_IMAGE, max_cycles=SMALL_BUDGET, + doorbell=True) + allowed = ("not-reached", "wait-reached-no-poll", + "wait-reached-poll-classified") + for report in (first, second): + self.assertEqual(report["format"], REPORT_FORMAT) + self.assertIn(report["outcome"], allowed) + self.assertGreater(report["image"]["size"], 0) + self.assertEqual(len(report["image"]["sha256"]), 64) + self.assertTrue(report["port_model"]["enabled"]) + if not report["boot"]["message_loop_wait_reached"]: + self.assertIsNotNone(report["divergence"]) + self.assertTrue(report["divergence"]["reason"]) + if report["first_poll"] is not None: + self.assertTrue(report["first_poll"]["in_ring"]) + self.assertTrue(report["first_poll"]["kind"] != "other") + divergence = report["divergence"] + if divergence and divergence.get("kind") == "task-state-query": + # The port model delivered the doorbell wake (extintr shuttle + # 10) and stopped at the libos trap/report path; the missing + # semantic is named, and no poll is claimed. + self.assertIsNone(report["first_poll"]) + self.assertIsNotNone(report["missing_semantic"]) + self.assertEqual(report["missing_semantic"]["stage"], + "after-extintr-handoff") + completed = [wait.get("completed") + for wait in report["port_model"].get("waits", [])] + self.assertIn(10, completed) + self.assertEqual(first["boot"]["message_loop_wait_reached"], + second["boot"]["message_loop_wait_reached"]) + self.assertEqual(first["boot"]["instructions_executed"], + second["boot"]["instructions_executed"]) + self.assertEqual(first["outcome"], second["outcome"]) + self.assertEqual(first["first_poll"], second["first_poll"]) + self.assertEqual(first["divergence"], second["divergence"]) + self.assertEqual(first["continuation"], second["continuation"]) + self.assertEqual(first["port_model"], second["port_model"]) + self.assertEqual(first["missing_semantic"], second["missing_semantic"]) + self.assertEqual(first["determinism"]["report_sha256"], + second["determinism"]["report_sha256"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_rehost.py b/debug/risc-v-gsp-rm-vm/tests/test_rehost.py new file mode 100644 index 0000000..3c88355 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_rehost.py @@ -0,0 +1,135 @@ +"""Tests for the surgical-rehost harness (harness/rehost.py, docs/46). + +Fixtures: the demo firmware (built by ``make demo``) is rehosted offline. The +real-image probes are skipped when the firmware is absent, and are bounded to +one short call each. No hardware, no module, no guest, no network. + +Run:: + + python3 -m unittest tests.test_rehost -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import sys +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.rehost import ( # noqa: E402 + DEFAULT_IMAGE, + DEMO_ELF, + DEMO_FORMAT, + REHOST_FORMAT, + RehostError, + main, + parse_hex_bytes, + parse_mem_spec, + parse_snap_spec, + rehost, + run_demo, +) + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class SpecTests(unittest.TestCase): + def test_parse_hex_bytes_and_mem_spec(self): + self.assertEqual(parse_hex_bytes("aa bb cc"), b"\xaa\xbb\xcc") + addr, data = parse_mem_spec("0x1000=0102") + self.assertEqual((addr, data), (0x1000, b"\x01\x02")) + + def test_bad_specs_are_clean_errors(self): + with self.assertRaises(RehostError): + parse_hex_bytes("abc") + with self.assertRaises(RehostError): + parse_mem_spec("0x1000") + with self.assertRaises(RehostError): + parse_snap_spec("0x1000") + + def test_missing_image_is_clean_error(self): + with self.assertRaises(RehostError): + rehost(image="/nonexistent/fw.bin", entry=0x1000) + + +@unittest.skipUnless(os.path.isfile(DEMO_ELF), "demo firmware not built") +class DemoRehostTests(unittest.TestCase): + def test_demo_rehost_is_deterministic_and_correct(self): + payload = run_demo() + self.assertEqual(payload["format"], DEMO_FORMAT) + self.assertTrue(payload["ok"]) + self.assertEqual(payload["first"]["status"], "return") + self.assertEqual(payload["first"]["retval"], payload["expected"]) + self.assertEqual(payload["first"]["retval"], payload["second"]["retval"]) + + def test_rehost_writes_report_and_session(self): + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + report = rehost( + image=DEMO_ELF, + entry="rpc_sum", + args=(0x80012000, 4), + mem=((0x80012000, b"\x01\x02\x03\x04"),), + snap=((0x80012000, 4),), + budget=50_000, + timeout=20.0, + out=tmp, + record=True, + name="rehost-test", + ) + self.assertEqual(report["format"], REHOST_FORMAT) + self.assertEqual(report["status"], "return") + self.assertEqual(report["retval"], 10) + self.assertTrue(os.path.isfile(report["report_json"])) + self.assertEqual(len(report["snapshots"]), 1) + self.assertEqual(report["snapshots"][0]["changed_bytes"], 0) + self.assertTrue(report["snapshots"][0]["before_sha256"]) + self.assertIn("fidelity", " ".join(report["limits"]).lower()) + + def test_cli_demo_returns_zero(self): + rc, out, _ = _run_main(["demo"]) + self.assertEqual(rc, 0) + self.assertTrue(json.loads(out)["ok"]) + + def test_cli_missing_image_returns_two(self): + rc, _, err = _run_main(["run", "--image", "/nonexistent/fw.bin", "--entry", "0x1000"]) + self.assertEqual(rc, 2) + self.assertIn("error:", err) + + +class RealImageProbeTests(unittest.TestCase): + """Bounded probes on the shipped 570.144 image; skipped without it.""" + + @unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "570.144 firmware absent") + def test_gate_probe_reproduces_invalid_state_without_state(self): + report = rehost( + image=DEFAULT_IMAGE, + profile_path=os.path.join(TOOL_ROOT, "sim", "profiles", "tu104-570.144.json"), + entry=0x5B01210, # per-buffer commit gate (from live analysis; numeric only) + args=(0,), + budget=50_000, + timeout=20.0, + record=False, + name="probe-gate-test", + ) + self.assertEqual(report["format"], REHOST_FORMAT) + self.assertIn(report["status"], ("return", "fault", "budget", "ecall")) + # With no engine-context state the gate family returns NV_ERR_INVALID_STATE. + self.assertEqual(report["retval"], 0x40) + self.assertIn("Not a boot", report["limits"][0]) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_rm_capture.py b/debug/risc-v-gsp-rm-vm/tests/test_rm_capture.py new file mode 100644 index 0000000..cb62bea --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_rm_capture.py @@ -0,0 +1,409 @@ +"""Tests for the GSP RPC capture ingestion harness (harness/rm_capture.py, docs/45). + +All fixtures are synthetic text written in-process or to temporary +directories: no capture, no hardware, no driver, no simulator run, no QEMU, +no network. The decode/compare/stimulus loop is offline file-in/file-out. +Tests that would need a real capture assert the clean error path instead. + +Run:: + + python3 -m unittest tests.test_rm_capture -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.rm_capture import ( # noqa: E402 + CAPTURE_FORMAT, + COMPARE_FORMAT, + DEMO_FORMAT, + GSP_RM_ALLOC, + GSP_RM_CONTROL, + MANUAL_DECODE_BASELINE_SECONDS, + RmCaptureError, + STIMULUS_FORMAT, + WORKFLOW_THRESHOLD_FRACTION, + canonical_json, + compare, + decode_file, + decode_text, + load_capture, + main, + print_capture_hints, + run_compare, + run_decode, + run_demo, + run_stimulus, + stimulus_from_capture, +) + + +def _hex_lines(tag: str, payload: bytes) -> list[str]: + return [f"GE-{tag} 000:" + "".join(f" {b:02x}" for b in payload)] + + +def _sample_log() -> str: + """A small log exercising every tag; params are contiguous single chunks.""" + return "\n".join( + [ + "[ 100.000000] NVRM: GE-GOLDEN-TRIGGER gspclient=1 mig=0", + "[ 100.000010] NVRM: GE-GOLDEN-BEGIN", + "[ 100.000020] NVRM: GE-ALLOC cls=0x90f1 hClient=0xbaba0001 hParent=0x0 " + "hObject=0xbaba1001 size=0x10 paramp=0xffffffffc0a00000 ret=0xffffffffc0b00000", + "[ 100.000020] NVRM: " + _hex_lines("ALLOC", bytes(range(16)))[0], + "[ 100.000030] NVRM: GE-CTRL cmd=0x2080012b hClient=0xc2000006 " + "hObject=0xabcd0080 size=0x8 paramp=0xffffffffc0a10000 ret=0xffffffffc0b00000", + "[ 100.000030] NVRM: " + _hex_lines("CTRL", bytes([1, 2, 3, 4, 5, 6, 7, 8]))[0], + "[ 100.000040] NVRM: GE-SEND fn=0x0000004c len=0x00000028 ret=0xffffffffc0b00000", + "[ 100.000040] NVRM: " + + _hex_lines("SEND", (0xC2000006).to_bytes(4, "little") + + (0xABCD0080).to_bytes(4, "little") + + (0x2080012B).to_bytes(4, "little") + + bytes(4))[0], + "[ 100.000050] NVRM: GE-CTRL-DONE cmd=0x2080012b rpc_status=0x0 params_status=0x0", + "[ 100.000060] NVRM: GE-GOLDEN-END status=0x0", + ] + ) + "\n" + + +def _follow(text: str, header_fragment: str) -> str: + """Return the hex block emitted immediately after a header line.""" + lines = text.splitlines() + for index, line in enumerate(lines): + if header_fragment in line: + return lines[index + 1] + raise AssertionError(f"fragment not found: {header_fragment}") + + +def _chunked_control_log(size: int) -> str: + payload = bytes((i * 5) & 0xFF for i in range(size)) + lines = ["[ 1.0] NVRM: GE-GOLDEN-BEGIN"] + lines.append( + f"[ 1.1] NVRM: GE-CTRL cmd=0x2080012b hClient=0xc2000006 " + f"hObject=0xabcd0080 size={size:#x} paramp=0x0 ret=0x0" + ) + for start in range(0, len(payload), 192): + piece = payload[start : start + 192] + lines.append( + f"[ 1.1] NVRM: GE-CTRL {start:03x}:" + + "".join(f" {b:02x}" for b in piece) + ) + lines.append("[ 1.2] NVRM: GE-GOLDEN-END status=0x0") + return "\n".join(lines) + "\n" + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class DecodeTests(unittest.TestCase): + def test_decodes_every_record_kind_in_order(self): + capture = decode_text(_sample_log(), source="sample") + kinds = [record.kind for record in capture.records] + self.assertEqual( + kinds, + ["golden", "golden", "alloc", "control", "send", "ctrl-done", "golden"], + ) + trigger, begin = capture.records[0], capture.records[1] + self.assertEqual(trigger.marker, "trigger") + self.assertEqual(trigger.gspclient, 1) + self.assertEqual(trigger.mig, 0) + self.assertEqual(begin.marker, "begin") + alloc = capture.records[2] + self.assertEqual(alloc.hclass, 0x90F1) + self.assertEqual(alloc.hclient, 0xBABA0001) + self.assertEqual(alloc.hobject, 0xBABA1001) + self.assertEqual(alloc.size, 0x10) + self.assertEqual(alloc.caller, 0xFFFFFFFFC0B00000) + control = capture.records[3] + self.assertEqual(control.cmd, 0x2080012B) + self.assertEqual(control.params, bytes([1, 2, 3, 4, 5, 6, 7, 8])) + send = capture.records[4] + self.assertEqual(send.fn, 0x4C) + self.assertEqual(send.length, 0x28) + self.assertEqual(send.expected_params_size, 0x28 - 32) + self.assertEqual(capture.records[5].rpc_status, 0) + self.assertEqual(capture.records[6].status, 0) + self.assertEqual(capture.golden_status(), 0) + self.assertFalse(capture.warnings) + + def test_dmesg_timestamp_and_prefix_are_stripped(self): + capture = decode_text(_sample_log(), source="sample") + self.assertEqual(capture.records[0].ts, " 100.000000") + + def test_multichunk_payload_is_assembled(self): + capture = decode_text(_chunked_control_log(560), source="chunked") + control = capture.records[1] + self.assertEqual(len(control.params), 560) + self.assertFalse(control.truncated) + self.assertEqual(control.chunks[0].__len__(), 192) + self.assertIn(0xC0, control.chunks) + self.assertIn(0x180, control.chunks) + self.assertEqual(control.params[192], (192 * 5) & 0xFF) + + def test_truncated_payload_is_flagged(self): + # The runbook helper caps at 192 bytes: declared 560, captured 192. + capture = decode_text(_chunked_control_log(192), source="truncated") + control = capture.records[1] + control.size = 560 + self.assertTrue(control.truncated) + self.assertEqual(control.expected_params_size, 560) + self.assertEqual(len(control.params), 192) + + def test_orphan_hex_warns_and_is_not_attached(self): + capture = decode_text("[ 1.0] NVRM: GE-CTRL 000: 01 02 03 04\n", source="orphan") + self.assertEqual(capture.records, []) + self.assertEqual(len(capture.warnings), 1) + self.assertIn("orphan CTRL hex", capture.warnings[0]) + + def test_wire_id_derived_from_send_payload(self): + control_send = decode_text(_sample_log(), source="s").records[4] + self.assertEqual(control_send.wire_id, 0x2080012B) + alloc_payload = ( + (0xBABA0001).to_bytes(4, "little") + + (0x0).to_bytes(4, "little") + + (0xBABA1002).to_bytes(4, "little") + + (0x0040).to_bytes(4, "little") + ) + log = ( + "[ 1.0] NVRM: GE-SEND fn=0x00000067 len=0x30 ret=0x0\n" + "[ 1.0] NVRM: GE-SEND 000:" + "".join(f" {b:02x}" for b in alloc_payload) + "\n" + ) + record = decode_text(log, source="a").records[0] + self.assertEqual(record.fn, GSP_RM_ALLOC) + self.assertEqual(record.wire_id, 0x0040) + self.assertNotEqual(GSP_RM_CONTROL, GSP_RM_ALLOC) + + def test_overrun_is_an_anomaly(self): + log = ( + "[ 1.0] NVRM: GE-CTRL cmd=0x1 hClient=0x0 hObject=0x0 size=0x4 " + "paramp=0x0 ret=0x0\n" + "[ 1.0] NVRM: GE-CTRL 000: 01 02 03 04 05 06 07 08\n" + ) + record = decode_text(log, source="overrun").records[0] + self.assertTrue(record.overrun) + self.assertTrue(any("> declared" in item for item in record.anomalies)) + + def test_golden_status_none_without_end_record(self): + log = "[ 1.0] NVRM: GE-GOLDEN-BEGIN\n" + self.assertIsNone(decode_text(log, source="x").golden_status()) + + +class CaptureIOTests(unittest.TestCase): + def test_decode_write_load_roundtrip(self): + with tempfile.TemporaryDirectory() as tmp: + log_path = os.path.join(tmp, "capture.log") + with open(log_path, "w", encoding="utf-8") as fh: + fh.write(_sample_log()) + out_dir = os.path.join(tmp, "out") + payload = run_decode(log_path, out_dir) + self.assertEqual(payload["format"], CAPTURE_FORMAT) + self.assertTrue(os.path.isfile(payload["artifacts"]["capture_json"])) + self.assertTrue(os.path.isfile(payload["artifacts"]["capture_md"])) + loaded = load_capture(payload["artifacts"]["capture_json"]) + self.assertEqual(len(loaded.records), len(decode_file(log_path).records)) + self.assertEqual(loaded.records[3].cmd, 0x2080012B) + + def test_load_capture_autodetects_raw_log(self): + with tempfile.TemporaryDirectory() as tmp: + log_path = os.path.join(tmp, "raw.log") + with open(log_path, "w", encoding="utf-8") as fh: + fh.write(_sample_log()) + loaded = load_capture(log_path) + self.assertEqual(loaded.records[0].kind, "golden") + + def test_load_capture_rejects_unknown_format(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "bad.json") + with open(path, "w", encoding="utf-8") as fh: + json.dump({"format": "something-else", "records": []}, fh) + with self.assertRaises(RmCaptureError): + load_capture(path) + + def test_missing_file_is_clean_error(self): + with self.assertRaises(RmCaptureError): + decode_file("/nonexistent/capture.log") + + +class CompareTests(unittest.TestCase): + def _captures(self): + capture_a = decode_text(_sample_log(), source="a") + capture_b = decode_text(_sample_log(), source="b") + return capture_a, capture_b + + def test_identical_captures_fully_agree(self): + capture_a, capture_b = self._captures() + report = compare(capture_a, capture_b) + self.assertEqual(report["format"], COMPARE_FORMAT) + summary = report["summary"] + self.assertEqual(summary["mismatch"], 0) + self.assertEqual(summary["a_only"], 0) + self.assertEqual(summary["b_only"], 0) + self.assertEqual(summary["agree"], len(capture_a.records)) + self.assertEqual(summary["agreement_pct"], 100.0) + self.assertEqual(summary["coverage_pct"], 100.0) + + def test_dropped_record_is_a_only(self): + capture_a, capture_b = self._captures() + capture_b.records = [ + record for record in capture_b.records if record.kind != "alloc" + ] + report = compare(capture_a, capture_b) + self.assertEqual(report["summary"]["a_only"], 1) + self.assertEqual(report["a_only"][0]["label"].startswith("ALLOC"), True) + + def test_param_byte_flip_is_shape_agree(self): + capture_a, capture_b = self._captures() + control = next(r for r in capture_b.records if r.kind == "control") + control.chunks[0] = bytes([9]) + control.chunks[0][1:] + report = compare(capture_a, capture_b) + self.assertEqual(report["summary"]["shape_agree"], 1) + row = next(r for r in report["rows"] if r["status"] == "shape-agree") + self.assertEqual(row["detail"]["first_diff_offset"], 0) + self.assertLess(row["detail"]["byte_agreement_pct"], 100.0) + + def test_truncated_pair_is_partial(self): + capture_a = decode_text(_chunked_control_log(192), source="a") + capture_b = decode_text(_chunked_control_log(192), source="b") + for capture in (capture_a, capture_b): + capture.records[1].size = 560 + report = compare(capture_a, capture_b) + self.assertEqual(report["summary"]["partial"], 1) + self.assertTrue(report["rows"][1]["a_truncated"]) + + def test_compare_is_deterministic_and_self_describing(self): + capture_a, capture_b = self._captures() + report_one = compare(capture_a, capture_b) + report_two = compare(capture_a, capture_b) + self.assertEqual(canonical_json(report_one), canonical_json(report_two)) + self.assertEqual(report_one["format"], COMPARE_FORMAT) + self.assertIn("L2", report_one["fidelity"]["levels"]) + self.assertTrue(report_one["limits"]) + + def test_run_compare_writes_json_and_markdown(self): + with tempfile.TemporaryDirectory() as tmp: + log_path = os.path.join(tmp, "capture.log") + with open(log_path, "w", encoding="utf-8") as fh: + fh.write(_sample_log()) + report = run_compare(log_path, log_path, os.path.join(tmp, "out")) + self.assertTrue(os.path.isfile(report["artifacts"]["report_json"])) + self.assertTrue(os.path.isfile(report["artifacts"]["report_md"])) + with open(report["artifacts"]["report_md"], "r", encoding="utf-8") as fh: + self.assertIn("# GSP RPC capture comparison", fh.read()) + + +class StimulusTests(unittest.TestCase): + def test_stimulus_lists_rpcs_in_order_without_handles(self): + capture = decode_text(_sample_log(), source="s") + stimulus = stimulus_from_capture(capture) + self.assertEqual(stimulus["format"], STIMULUS_FORMAT) + kinds = [message["kind"] for message in stimulus["messages"]] + self.assertEqual(kinds, ["alloc", "control", "send"]) + self.assertEqual(stimulus["messages"][1]["cmd"], 0x2080012B) + self.assertEqual(stimulus["messages"][2]["fn"], GSP_RM_CONTROL) + self.assertEqual(stimulus["messages"][2]["wire_id"], 0x2080012B) + self.assertNotIn("hclient", stimulus["messages"][1]) + self.assertEqual(len(stimulus["markers"]), 3) + self.assertTrue(stimulus["limits"]) + + def test_stimulus_roundtrips_params_and_flags_truncation(self): + capture = decode_text(_chunked_control_log(192), source="s") + capture.records[1].size = 560 + stimulus = stimulus_from_capture(capture) + message = stimulus["messages"][0] + self.assertTrue(message["truncated"]) + self.assertEqual(message["params_size_expected"], 560) + self.assertEqual(bytes.fromhex(message["params_hex"]), capture.records[1].params) + + def test_run_stimulus_writes_file(self): + with tempfile.TemporaryDirectory() as tmp: + out_path = os.path.join(tmp, "nested", "stimulus.json") + log_path = os.path.join(tmp, "capture.log") + with open(log_path, "w", encoding="utf-8") as fh: + fh.write(_sample_log()) + payload = run_stimulus(log_path, out_path) + self.assertTrue(os.path.isfile(out_path)) + self.assertEqual(payload["format"], STIMULUS_FORMAT) + with open(out_path, "r", encoding="utf-8") as fh: + written = json.load(fh) + self.assertEqual(written["format"], STIMULUS_FORMAT) + + +class DemoAndCliTests(unittest.TestCase): + def test_demo_is_ok_and_within_threshold(self): + payload = run_demo() + self.assertEqual(payload["format"], DEMO_FORMAT) + self.assertTrue(payload["ok"]) + self.assertLessEqual( + payload["wall_seconds"], + MANUAL_DECODE_BASELINE_SECONDS * WORKFLOW_THRESHOLD_FRACTION, + ) + self.assertGreaterEqual(payload["summary"]["agree"], 1) + self.assertEqual(payload["summary"]["shape_agree"], 1) + self.assertEqual(payload["summary"]["a_only"], 1) + + def test_demo_is_deterministic_apart_from_timing(self): + first = run_demo() + second = run_demo() + for payload in (first, second): + payload.pop("wall_seconds", None) + payload.pop("artifacts", None) + self.assertEqual(canonical_json(first), canonical_json(second)) + + def test_demo_writes_artifacts(self): + with tempfile.TemporaryDirectory() as tmp: + payload = run_demo(tmp) + for key in ("capture_a", "replay_b", "compare", "stimulus"): + self.assertTrue(os.path.isfile(payload["artifacts"][key])) + + def test_cli_decode_compare_stimulus_return_zero(self): + with tempfile.TemporaryDirectory() as tmp: + log_path = os.path.join(tmp, "capture.log") + with open(log_path, "w", encoding="utf-8") as fh: + fh.write(_sample_log()) + rc, out, _ = _run_main(["decode", "--log", log_path, "--out", tmp]) + self.assertEqual(rc, 0) + self.assertEqual(json.loads(out)["format"], CAPTURE_FORMAT) + rc, out, _ = _run_main( + ["compare", "--a", log_path, "--b", log_path, "--out", tmp] + ) + self.assertEqual(rc, 0) + self.assertEqual(json.loads(out)["format"], COMPARE_FORMAT) + rc, out, _ = _run_main( + ["stimulus", "--capture", log_path, "--out", os.path.join(tmp, "s.json")] + ) + self.assertEqual(rc, 0) + self.assertEqual(json.loads(out)["format"], STIMULUS_FORMAT) + + def test_cli_missing_input_returns_two(self): + rc, _, err = _run_main(["decode", "--log", "/nonexistent/capture.log"]) + self.assertEqual(rc, 2) + self.assertIn("error:", err) + + def test_hints_execute_nothing_and_name_the_rules(self): + out = io.StringIO() + with contextlib.redirect_stdout(out): + print_capture_hints() + text = out.getvalue() + self.assertIn("executes", text) + self.assertIn("NOTHING", text) + self.assertIn("GE-SEND", text) + self.assertIn("192 bytes", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_server_snapshot.py b/debug/risc-v-gsp-rm-vm/tests/test_server_snapshot.py new file mode 100644 index 0000000..b82d09c --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_server_snapshot.py @@ -0,0 +1,364 @@ +"""Tests for the server-state snapshot collector (harness/server_snapshot.py). + +Crafted evidence covers every supported shape (``LABEL[offset]=0xVALUE``, +labelled hexdump blocks, ``LOGINIT-raw`` headers, record lines, duplicate +labels and unparseable dump-looking lines). Optional tests run against the +real 2026-10-02 campaign backups when they are present; they are skipped +otherwise. All tests are offline, hermetic and touch no hardware. + +Run:: + + python3 -m unittest tests.test_server_snapshot -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import sys +import tempfile +import unittest +from unittest import mock + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.server_snapshot import ( # noqa: E402 + DIFF_FORMAT, + SNAPSHOT_FORMAT, + SnapshotError, + collect, + diff_snapshots, + main, +) + +EVIDENCE_VICTORY = ("/home/rmcore/Git/integdev_gpu_drv/open-iov/backups/" + "evidence-20261002-victory") +LADDER_M12 = os.path.join(EVIDENCE_VICTORY, "06-ladder-m12.txt") +PHYS_STATE_M26 = os.path.join(EVIDENCE_VICTORY, "20-gsp-phys-state-m26.txt") +EVIDENCE_GR = ("/home/rmcore/Git/integdev_gpu_drv/open-iov/backups/" + "evidence-20261002-gr") +LOGDUMP_GRCTX4 = os.path.join(EVIDENCE_GR, "21-logdump-grctx4.txt") + +FIXED_CREATED = "2026-01-01T00:00:00Z" + + +def _run_main(argv): + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class ServerSnapshotTests(unittest.TestCase): + """Collector normalization, collision, diff, CLI and determinism.""" + + def setUp(self): + holder = tempfile.TemporaryDirectory(prefix="server-snapshot-test-") + self.addCleanup(holder.cleanup) + self.tmp = holder.name + + def _write(self, name, text): + path = os.path.join(self.tmp, name) + with open(path, "w", encoding="utf-8") as handle: + handle.write(text) + return path + + def _collect_text(self, name, text, **kwargs): + path = self._write(name, text) + kwargs.setdefault("source", f"test:{name}") + kwargs.setdefault("created", FIXED_CREATED) + snapshot, stats = collect(files=[path], **kwargs) + return snapshot, stats, path + + @staticmethod + def _chunks(snapshot): + return {chunk["label"]: chunk for chunk in snapshot["chunks"]} + + # -- crafted shapes ---------------------------------------------------- + + def test_labeled_value_lines(self): + snapshot, stats, _ = self._collect_text( + "values.txt", + "alpha[0]=0x00000011\n" + "alpha[4]=0x00000022\n" + "beta[8]=0x0000000112345678\n", + ) + chunks = self._chunks(snapshot) + self.assertEqual(chunks["alpha"]["base"], "0x0") + self.assertEqual(chunks["alpha"]["words"], + ["0x00000011", "0x00000022"]) + self.assertEqual(chunks["beta"]["base"], "0x8") + self.assertEqual(chunks["beta"]["words"], + ["0x12345678", "0x00000001"]) + self.assertEqual(stats["unparsed"], 0) + self.assertEqual(stats["conflicts"], 0) + + def test_labeled_hexdump_block(self): + snapshot, stats, _ = self._collect_text( + "block.txt", + "preamble text that is not a header\n" + "SOME_BUF: 64 bytes of server state\n" + "00000000: 00000001 00000002 00000003 00000004\n" + "00000010: 00000005 00000006 00000007 00000008\n" + "RAMFC: a second block with a nonzero base\n" + "00000020: 0000000a 0000000b\n", + ) + chunks = self._chunks(snapshot) + self.assertEqual(chunks["SOME_BUF"]["base"], "0x0") + self.assertEqual( + chunks["SOME_BUF"]["words"], + ["0x00000001", "0x00000002", "0x00000003", "0x00000004", + "0x00000005", "0x00000006", "0x00000007", "0x00000008"]) + self.assertEqual(chunks["RAMFC"]["base"], "0x20") + self.assertEqual(chunks["RAMFC"]["words"], + ["0x0000000a", "0x0000000b"]) + self.assertEqual(stats["unparsed"], 0) + self.assertEqual(stats["chunks"], 2) + + def test_logininit_raw_block(self): + snapshot, stats, _ = self._collect_text( + "loginit.txt", + "vgsp-boot: libos task log buffers (read-only)\n" + "LOGINIT-raw: 65536 bytes (allocated 65536) " + "VA=0xffff88a673af0000 dma=0x4f3af0000\n" + "00000000: 000002d2 00000000 05bb9e10 00000000\n" + "00000010: f7660780 18dbb28f fe2c0000 61800001\n", + ) + chunk = self._chunks(snapshot)["LOGINIT-raw"] + self.assertEqual(chunk["base"], "0x0") + self.assertEqual( + chunk["words"], + ["0x000002d2", "0x00000000", "0x05bb9e10", "0x00000000", + "0xf7660780", "0x18dbb28f", "0xfe2c0000", "0x61800001"]) + self.assertEqual(stats["unparsed"], 0) + + def test_inline_labeled_rows_group_into_one_chunk(self): + snapshot, _, _ = self._collect_text( + "inline.txt", + "[dmesg] vgsp-boot: rm-call: INSTANCE[0000]: " + "00000000 00000000 0a545003 00000001\n" + "[dmesg] vgsp-boot: rm-call: INSTANCE[0020]: " + "0000face 00000000 00000000 00000000\n", + ) + chunk = self._chunks(snapshot)["INSTANCE"] + self.assertEqual(chunk["base"], "0x0") + self.assertEqual(chunk["words"], + ["0x00000000", "0x00000000", "0x0a545003", + "0x00000001", "0x0000face", "0x00000000", + "0x00000000", "0x00000000"]) + + def test_record_lines(self): + snapshot, stats, _ = self._collect_text( + "records.txt", + 'vgsp-nocat: rec seq=1 recType=5 errorCode=0x57f1344 ' + 'subsystem=0x0 bugcheck=0x0 payload=1212 source="ASSERT" ' + 'faultingEngine=""\n', + ) + self.assertEqual(len(snapshot["records"]), 1) + record = snapshot["records"][0] + self.assertEqual(record["kind"], "record") + self.assertEqual(record["rec_type"], 5) + self.assertEqual(record["error_code"], 0x57F1344) + self.assertEqual(record["source_name"], "ASSERT") + self.assertEqual(stats["records"], 1) + self.assertEqual(snapshot["chunks"], []) + + # -- duplicates, collisions, unparsed --------------------------------- + + def test_identical_duplicate_merged(self): + snapshot, stats, _ = self._collect_text( + "merged.txt", + "INSTANCE[0000]: 00000001 00000002\n" + "plain prose closes the first block\n" + "INSTANCE[0000]: 00000001 00000002\n", + ) + self.assertEqual(list(self._chunks(snapshot)), ["INSTANCE"]) + self.assertEqual(stats["merged"], 1) + self.assertEqual(stats["collisions"], 0) + + def test_conflicting_duplicate_renamed(self): + snapshot, stats, _ = self._collect_text( + "collision.txt", + "INSTANCE[0000]: 00000001 00000002\n" + "plain prose closes the first block\n" + "INSTANCE[0000]: 00000003 00000004\n", + ) + self.assertEqual(list(self._chunks(snapshot)), + ["INSTANCE", "INSTANCE#2"]) + self.assertEqual(stats["collisions"], 1) + self.assertEqual(stats["collision_list"], + [{"label": "INSTANCE", "renamed": "INSTANCE#2", + "base": "0x0", "words": 2}]) + + def test_unparsed_candidates_counted(self): + snapshot, stats, _ = self._collect_text( + "unparsed.txt", + "BROKEN[0x10]: zzzz\n" + "00000000: zz00\n" + "ordinary prose is not a candidate\n", + ) + self.assertEqual(snapshot["chunks"], []) + self.assertEqual(stats["unparsed"], 2) + + # -- diff -------------------------------------------------------------- + + def test_diff_changed_added_removed(self): + snap_a, _, _ = self._collect_text( + "run-a.txt", + "MOCK[0000]: 00000001 00000002\n" + "shared[0]=0x00000007\n" + "only_a[0]=0x00000009\n", + ) + snap_b, _, _ = self._collect_text( + "run-b.txt", + "MOCK[0000]: 00000001 00000003 00000004\n" + "shared[0]=0x00000007\n" + "only_b[0]=0x0000000a\n", + ) + report = diff_snapshots(snap_a, snap_b) + self.assertEqual(report["schema"], DIFF_FORMAT) + rows = {row["label"]: row for row in report["chunks"]} + self.assertEqual(rows["MOCK"]["status"], "changed") + self.assertEqual(rows["MOCK"]["changed"], + [{"index": 1, "a": "0x00000002", "b": "0x00000003"}]) + self.assertEqual(rows["MOCK"]["added"], + [{"index": 2, "value": "0x00000004"}]) + self.assertEqual(rows["MOCK"]["removed"], []) + self.assertEqual(rows["shared"]["status"], "unchanged") + self.assertEqual(rows["only_a"]["status"], "removed") + self.assertEqual(rows["only_a"]["removed"], + [{"index": 0, "value": "0x00000009"}]) + self.assertEqual(rows["only_b"]["status"], "added") + self.assertEqual(rows["only_b"]["added"], + [{"index": 0, "value": "0x0000000a"}]) + summary = report["summary"] + self.assertFalse(summary["identical"]) + self.assertEqual(summary["chunks_added"], 1) + self.assertEqual(summary["chunks_removed"], 1) + self.assertEqual(summary["chunks_changed"], 1) + self.assertEqual(summary["chunks_unchanged"], 1) + self.assertEqual(summary["words_added"], 2) + self.assertEqual(summary["words_removed"], 1) + self.assertEqual(summary["words_changed"], 1) + + def test_diff_identical(self): + snapshot, _, _ = self._collect_text( + "same.txt", "INSTANCE[0000]: 00000001 00000002\n") + report = diff_snapshots(snapshot, snapshot) + self.assertTrue(report["summary"]["identical"]) + self.assertEqual(report["summary"]["words_changed"], 0) + self.assertEqual([row["status"] for row in report["chunks"]], + ["unchanged"]) + + def test_diff_rejects_wrong_schema(self): + with self.assertRaises(SnapshotError): + diff_snapshots({"schema": "other"}, {"schema": SNAPSHOT_FORMAT}) + + # -- determinism and CLI ---------------------------------------------- + + def test_determinism_with_source_date_epoch(self): + path = self._write( + "det.txt", + "INSTANCE[0000]: 00000001 00000002\n" + 'vgsp-nocat: rec seq=1 recType=5 errorCode=0x57f1344 ' + 'source="ASSERT"\n', + ) + with mock.patch.dict(os.environ, {"SOURCE_DATE_EPOCH": "1700000000"}): + first, _ = collect(files=[path], source="det") + second, _ = collect(files=[path], source="det") + self.assertEqual(first["created"], "2023-11-14T22:13:20Z") + self.assertEqual(first, second) + self.assertEqual(json.dumps(first, sort_keys=True), + json.dumps(second, sort_keys=True)) + + def test_cli_collect_stdout_and_json(self): + path = self._write("cli.txt", "INSTANCE[0000]: 00000001 00000002\n") + rc, out, err = _run_main( + ["collect", "--file", path, "--source", "cli", + "--created", FIXED_CREATED]) + self.assertEqual(rc, 0) + snapshot = json.loads(out) + self.assertEqual(set(snapshot), + {"schema", "source", "created", "chunks", "records"}) + self.assertEqual(snapshot["schema"], SNAPSHOT_FORMAT) + self.assertEqual(snapshot["source"], "cli") + self.assertIn("1 chunk(s)", err) + + json_path = os.path.join(self.tmp, "snapshot.json") + rc, out, _ = _run_main( + ["collect", "--file", path, "--json", json_path, + "--created", FIXED_CREATED]) + self.assertEqual(rc, 0) + with open(json_path, "r", encoding="utf-8") as handle: + written = json.load(handle) + self.assertEqual(written["schema"], SNAPSHOT_FORMAT) + self.assertEqual(json.loads(out)["chunks"], len(written["chunks"])) + + def test_cli_errors(self): + rc, _, err = _run_main(["collect"]) + self.assertEqual(rc, 2) + self.assertIn("error:", err) + missing = os.path.join(self.tmp, "missing.txt") + rc, _, _ = _run_main(["collect", "--file", missing]) + self.assertEqual(rc, 2) + + def test_discover_include_globs(self): + self._write("keep.txt", "AAA[0]=0x00000001\n") + self._write("skip.bin", "BBB[0]=0x00000002\n") + snapshot, _ = collect(dirs=[self.tmp], source="glob", + created=FIXED_CREATED) + self.assertEqual(list(self._chunks(snapshot)), ["AAA"]) + snapshot, _ = collect(dirs=[self.tmp], includes=["*.bin"], + source="glob", created=FIXED_CREATED) + self.assertEqual(list(self._chunks(snapshot)), ["AAA", "BBB"]) + + def test_demo_cli(self): + rc, out, _ = _run_main(["demo"]) + self.assertEqual(rc, 0) + self.assertIn("MOCK", out) + summary = json.loads(out[out.index("{"):]) + self.assertFalse(summary["identical"]) + self.assertEqual(summary["words_changed"], 2) + + # -- real campaign evidence (optional) -------------------------------- + + @unittest.skipUnless(os.path.isdir(EVIDENCE_VICTORY), + "victory evidence backup not present") + def test_real_victory_evidence(self): + files = [path for path in (LADDER_M12, PHYS_STATE_M26) + if os.path.isfile(path)] + if not files: + self.skipTest("expected victory evidence files missing") + snapshot, stats = collect(files=files, source="evidence-victory", + created=FIXED_CREATED) + self.assertTrue(snapshot["chunks"]) + chunks = self._chunks(snapshot) + self.assertIn("INSTANCE", chunks) + self.assertEqual(chunks["INSTANCE"]["base"], "0x0") + self.assertEqual(len(chunks["INSTANCE"]["words"]), 1024) + self.assertIn("pt", chunks) + self.assertEqual(len(chunks["pt"]["words"]), 5) + self.assertGreaterEqual(stats["records"], 1) + self.assertEqual(stats["files"], len(files)) + + @unittest.skipUnless(os.path.isfile(LOGDUMP_GRCTX4), + "gr logdump evidence not present") + def test_real_loginit_raw_block(self): + snapshot, stats = collect(files=[LOGDUMP_GRCTX4], + source="evidence-gr", + created=FIXED_CREATED) + chunks = self._chunks(snapshot) + self.assertIn("LOGINIT-raw", chunks) + self.assertIn("LOGRM-raw", chunks) + self.assertEqual(chunks["LOGINIT-raw"]["base"], "0x0") + self.assertEqual(len(chunks["LOGINIT-raw"]["words"]), 65536 // 4) + self.assertEqual(len(chunks["LOGRM-raw"]["words"]), 65536 // 4) + self.assertEqual(stats["unparsed"], 0) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_state_decode.py b/debug/risc-v-gsp-rm-vm/tests/test_state_decode.py new file mode 100644 index 0000000..5c05ccd --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_state_decode.py @@ -0,0 +1,518 @@ +"""Tests for the engine-visible state decoder (harness/state_decode.py). + +Offline only: text/hexdump/binary/snapshot-JSON parsing, the bundled +INSTANCE/RAMFC field table, map override/extension, decode/diff reports and +the CLI. The real-evidence test parses a frozen live ladder trace and skips +when the backup is absent. No hardware, no module, no guest, no network. + +Run:: + + python3 -m unittest tests.test_state_decode -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import struct +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from harness.state_decode import ( # noqa: E402 + DECODE_FORMAT, + DEMO_FORMAT, + DIFF_FORMAT, + MAP_FORMAT, + SNAPSHOT_FORMAT, + StateDecodeError, + decode_chunk, + decode_specs, + diff_reports, + load_input, + load_map, + main, + run_demo, +) + +EVIDENCE = os.environ.get("GREENEYE_STATE_DECODE_EVIDENCE") or os.path.normpath( + os.path.join(TOOL_ROOT, "..", "..", "backups", + "evidence-20261002-victory", "06-ladder-m12.txt")) + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +def _write(path: str, text: str) -> str: + with open(path, "w", encoding="utf-8") as handle: + handle.write(text) + return path + + +def _write_snapshot(tmp: str, name: str, chunks: list[dict]) -> str: + path = os.path.join(tmp, name) + with open(path, "w", encoding="utf-8") as handle: + json.dump({"schema": SNAPSHOT_FORMAT, "source": name, + "chunks": chunks}, handle) + return path + + +def _write_map(tmp: str, name: str, doc) -> str: + path = os.path.join(tmp, name) + with open(path, "w", encoding="utf-8") as handle: + json.dump(doc, handle) + return path + + +def _report_from_chunks(chunks: list[dict], map_path: str | None = None) -> dict: + entries, source = load_map(map_path) + return { + "format": DECODE_FORMAT, + "map_source": source, + "sources": [], + "chunks": [decode_chunk(chunk, entries) for chunk in chunks], + "limits": [], + } + + +def _chunk_field(report: dict, label: str, name: str) -> dict: + for chunk in report["chunks"]: + if chunk["label"] != label: + continue + for field in chunk["fields"]: + if field["name"] == name: + return field + raise AssertionError(f"field {name} not found in chunk {label}") + + +def _spec(path: str, label: str | None = None, base=None, + binary: bool = False) -> dict: + return {"path": path, "label": label, "base": base, "binary": binary} + + +class ParsingTests(unittest.TestCase): + def test_text_lines_parse_labels_offsets_and_values(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(os.path.join(tmp, "lines.txt"), "\n".join([ + "banner line that must be ignored", + "INSTANCE[0x10]=0xface", + "mid-text noise", + "Instance[00ac]=0x00030000", + "INSTANCE[72]=0x1", # decimal offset 72 == 0x48 + "INSTANCE[0x4c]=64206", # decimal value 64206 == 0xface + "", + ])) + document = load_input(_spec(path)) + self.assertEqual(document["format"], "text") + self.assertEqual(len(document["chunks"]), 1) + chunk = document["chunks"][0] + self.assertEqual(chunk["label"], "INSTANCE") # first spelling wins + self.assertEqual(chunk["words"][0x10], 0xFACE) + self.assertEqual(chunk["words"][0xAC], 0x00030000) + self.assertEqual(chunk["words"][0x48], 0x1) + self.assertEqual(chunk["words"][0x4C], 64206) + + def test_hexdump_rows_parse_with_label_and_base(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(os.path.join(tmp, "dump.txt"), "\n".join([ + "00000000: 000002d2 00000000 00000000 00000000", + "00000010: 0000face 00000000 00000000 00000000", + ])) + document = load_input(_spec(path, label="INSTANCE", + base="0x1f4000000")) + chunk = document["chunks"][0] + self.assertEqual(chunk["label"], "INSTANCE") + self.assertEqual(chunk["base"], 0x1F4000000) + self.assertEqual(chunk["words"][0x0], 0x2D2) + self.assertEqual(chunk["words"][0x4], 0x0) + self.assertEqual(chunk["words"][0x10], 0xFACE) + + def test_raw_binary_parses_little_endian_words(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "ramfc.bin") + with open(path, "wb") as handle: + handle.write(struct.pack("<4I", 0x2D2, 0, 0x30000, 0)) + document = load_input(_spec(path, label="RAMFC", binary=True)) + self.assertEqual(document["format"], "binary") + chunk = document["chunks"][0] + self.assertEqual(chunk["label"], "RAMFC") + self.assertEqual(chunk["words"][0x0], 0x2D2) + self.assertEqual(chunk["words"][0x8], 0x30000) + + def test_raw_binary_is_auto_detected(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "auto.bin") + with open(path, "wb") as handle: + handle.write(struct.pack("<8I", 1, 2, 3, 4, 5, 6, 7, 8)) + document = load_input(_spec(path, label="WINDOW")) + self.assertEqual(document["format"], "binary") + chunk = document["chunks"][0] + self.assertEqual(chunk["words"][0x1C], 8) + + def test_snapshot_json_parses_exactly(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write_snapshot(tmp, "snap.json", [{ + "label": "RAMFC", "base": "0x1000", + "words": ["0x2d2", 0, "0x30000", "0x0"], + }]) + document = load_input(_spec(path)) + self.assertEqual(document["format"], "snapshot-json") + chunk = document["chunks"][0] + self.assertEqual(chunk["label"], "RAMFC") + self.assertEqual(chunk["base"], 0x1000) + self.assertEqual(chunk["words"][0x0], 0x2D2) + self.assertEqual(chunk["words"][0x8], 0x30000) + + def test_snapshot_schema_rejected(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "other.json") + with open(path, "w", encoding="utf-8") as handle: + json.dump({"schema": "something-else/1", "chunks": []}, handle) + with self.assertRaises(StateDecodeError): + load_input(_spec(path)) + + def test_unparseable_text_rejected(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(os.path.join(tmp, "nothing.txt"), + "just prose\nwith no fields at all\n") + with self.assertRaises(StateDecodeError): + load_input(_spec(path)) + + def test_bad_binary_size_rejected(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "odd.bin") + with open(path, "wb") as handle: + handle.write(b"\x01\x02\x03") + with self.assertRaises(StateDecodeError): + load_input(_spec(path, binary=True)) + + +class DecodeTests(unittest.TestCase): + def _ramfc_target(self, word48: int) -> dict: + report = _report_from_chunks([ + {"label": "RAMFC", "base": 0, "words": {0x48: word48}}, + ]) + return _chunk_field(report, "RAMFC", "RAMFC_TARGET") + + def test_ramfc_target_both_bits_set(self): + field = self._ramfc_target(0x00030000) + self.assertEqual(field["offset"], "0x48") + self.assertEqual(field["raw"], "0x00030000") + self.assertEqual(field["kind"], "bits") + bits = {bit["name"]: bit for bit in field["bitfields"]} + self.assertEqual(bits["ENG_CTX_VALID"]["value"], 1) + self.assertEqual(bits["ENG_CTX_VALID"]["state"], "set") + self.assertEqual(bits["CE_CTX_VALID"]["value"], 1) + self.assertEqual(bits["CE_CTX_VALID"]["state"], "set") + + def test_ramfc_target_only_eng_bit16(self): + field = self._ramfc_target(0x00010000) + bits = {bit["name"]: bit for bit in field["bitfields"]} + self.assertEqual(bits["ENG_CTX_VALID"]["state"], "set") + self.assertEqual(bits["CE_CTX_VALID"]["state"], "clear") + + def test_ramfc_target_only_ce_bit17(self): + field = self._ramfc_target(0x00020000) + bits = {bit["name"]: bit for bit in field["bitfields"]} + self.assertEqual(bits["ENG_CTX_VALID"]["state"], "clear") + self.assertEqual(bits["CE_CTX_VALID"]["state"], "set") + + def test_instance_fields_and_unmapped_words(self): + report = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, + "words": {0x0: 0, 0x10: 0xFACE, 0x48: 0x100000, 0x4C: 0x30000}, + }]) + chunk = report["chunks"][0] + names = {field["name"] for field in chunk["fields"]} + self.assertEqual(names, {"MAGIC_0FACE", "GPFIFO_VA"}) + self.assertEqual(chunk["word_count"], 4) + self.assertEqual(chunk["unmapped_count"], 2) + self.assertEqual({word["offset"] for word in chunk["unmapped"]}, + {"0x0", "0x4c"}) + self.assertEqual(chunk["words"]["0x48"], "0x00100000") + + def test_decode_report_format_and_determinism(self): + report = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x48: 0x100000}, + }]) + self.assertEqual(report["format"], DECODE_FORMAT) + again = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x48: 0x100000}, + }]) + self.assertEqual(json.dumps(report, sort_keys=True), + json.dumps(again, sort_keys=True)) + + +class MapTests(unittest.TestCase): + def test_override_replaces_bundled_and_extends(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write_map(tmp, "map.json", { + "format": MAP_FORMAT, + "entries": [ + {"chunk": "INSTANCE", "label": "GPFIFO_VA_ALT", + "offset": "0x48", "width": 4, "note": "override"}, + {"chunk": "INSTANCE", "label": "CTX_MARK", + "offset": "0x4c", "width": 4, + "bits": [ + {"name": "ENG", "lo": 16, "hi": 16}, + {"name": "CE", "lo": 17, "hi": 17}, + {"name": "MODE", "lo": 0, "hi": 7, + "values": {"0": "zero", "1": "one"}}, + ]}, + ], + }) + report = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, + "words": {0x10: 0xFACE, 0x48: 0x100000, 0x4C: 0x00030001}, + }], map_path=path) + chunk = report["chunks"][0] + names = [field["name"] for field in chunk["fields"]] + self.assertIn("GPFIFO_VA_ALT", names) + self.assertNotIn("GPFIFO_VA", names) + self.assertIn("MAGIC_0FACE", names) # bundled table still extends + mark = _chunk_field(report, "INSTANCE", "CTX_MARK") + bits = {bit["name"]: bit for bit in mark["bitfields"]} + self.assertEqual(bits["MODE"]["state"], "one") + self.assertEqual(bits["ENG"]["state"], "set") + self.assertEqual(bits["CE"]["state"], "set") + self.assertEqual(report["map_source"], os.path.abspath(path)) + + def test_replace_mode_drops_bundled_table(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write_map(tmp, "replace.json", { + "format": MAP_FORMAT, + "mode": "replace", + "entries": [ + {"chunk": "RAMFC", "label": "TARGET_ALT", + "offset": "0x48", "width": 4, + "bits": [{"name": "VALID", "lo": 16, "hi": 16}]}, + ], + }) + report = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x10: 0xFACE}, + }, { + "label": "RAMFC", "base": 0, "words": {0x48: 0x10000}, + }], map_path=path) + self.assertEqual(report["chunks"][0]["fields"], []) + field = _chunk_field(report, "RAMFC", "TARGET_ALT") + self.assertEqual(field["bitfields"][0]["state"], "set") + + def test_bad_map_rejected(self): + cases = [ + {"entries": [{"chunk": "X", "label": "Y", "offset": "0x48", + "width": 3}]}, + {"entries": [{"chunk": "X", "label": "Y", "offset": "0x48", + "width": 4, + "bits": [{"name": "B", "lo": 0, "hi": 99}]}]}, + {"entries": [{"chunk": "X", "label": "Y", "offset": "0x0", + "width": 4}], "mode": "sideways"}, + ] + for doc in cases: + with self.subTest(doc=doc): + with tempfile.TemporaryDirectory() as tmp: + path = _write_map(tmp, "bad.json", doc) + with self.assertRaises(StateDecodeError): + load_map(path) + + +class DiffTests(unittest.TestCase): + def test_diff_detects_changed_word_and_named_field(self): + left = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, + "words": {0x10: 0xFACE, 0x48: 0x100000}, + }]) + right = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, + "words": {0x10: 0xFACE, 0x48: 0x200000}, + }]) + report = diff_reports(left, right) + self.assertEqual(report["format"], DIFF_FORMAT) + self.assertFalse(report["identical"]) + chunk = report["chunks"][0] + self.assertEqual(chunk["status"], "changed") + self.assertEqual(chunk["changed"], + [{"offset": "0x48", "old": "0x00100000", + "new": "0x00200000"}]) + self.assertEqual(len(chunk["changed_fields"]), 1) + field = chunk["changed_fields"][0] + self.assertEqual(field["name"], "GPFIFO_VA") + self.assertEqual(field["old"], "0x00100000") + self.assertEqual(field["new"], "0x00200000") + self.assertEqual(report["summary"]["changed_words"], 1) + self.assertEqual(report["summary"]["changed_fields"], 1) + + def test_diff_reports_bitfield_state_change(self): + left = _report_from_chunks([ + {"label": "RAMFC", "base": 0, "words": {0x48: 0x00030000}}]) + right = _report_from_chunks([ + {"label": "RAMFC", "base": 0, "words": {0x48: 0x00010000}}]) + report = diff_reports(left, right) + field = report["chunks"][0]["changed_fields"][0] + self.assertEqual(field["name"], "RAMFC_TARGET") + bits = {bit["name"]: bit for bit in field["bits"]} + self.assertEqual(bits["ENG_CTX_VALID"]["old"], "set") + self.assertEqual(bits["ENG_CTX_VALID"]["new"], "set") + self.assertEqual(bits["CE_CTX_VALID"]["old"], "set") + self.assertEqual(bits["CE_CTX_VALID"]["new"], "clear") + + def test_diff_identical_is_empty_and_deterministic(self): + left = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x10: 0xFACE}, + }]) + right = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x10: 0xFACE}, + }]) + report = diff_reports(left, right) + self.assertTrue(report["identical"]) + self.assertEqual(report["summary"]["changed_words"], 0) + again = diff_reports(left, right) + self.assertEqual(json.dumps(report, sort_keys=True), + json.dumps(again, sort_keys=True)) + + def test_diff_added_and_removed_words(self): + left = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x0: 1, 0x4: 2}, + }]) + right = _report_from_chunks([{ + "label": "INSTANCE", "base": 0, "words": {0x0: 1, 0x8: 3}, + }]) + report = diff_reports(left, right) + chunk = report["chunks"][0] + self.assertEqual([row["offset"] for row in chunk["removed"]], ["0x4"]) + self.assertEqual([row["offset"] for row in chunk["added"]], ["0x8"]) + self.assertEqual(report["summary"]["added"], 1) + self.assertEqual(report["summary"]["removed"], 1) + + +class CliAndDemoTests(unittest.TestCase): + def test_cli_decode_prints_table_and_writes_json(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(os.path.join(tmp, "dump.txt"), "\n".join([ + "00000000: 00000000 00000000 00000000 00000000", + "00000010: 0000face 00000000 00000000 00000000", + "00000020: 00000000 00000000 00000000 00000000", + "00000030: 00000000 00000000 00000000 00000000", + "00000040: 00000000 00000000 00100000 00030000", + ])) + out_path = os.path.join(tmp, "decoded.json") + rc, out, err = _run_main(["decode", "--input", path, + "--label", "INSTANCE", + "--json", out_path]) + self.assertEqual(rc, 0, err) + self.assertIn("GPFIFO_VA", out) + self.assertIn("0x00100000", out) + with open(out_path, encoding="utf-8") as handle: + report = json.load(handle) + self.assertEqual(report["format"], DECODE_FORMAT) + self.assertEqual(len(report["chunks"]), 1) + self.assertEqual(_chunk_field(report, "INSTANCE", + "GPFIFO_VA")["raw"], "0x00100000") + + def test_cli_multiple_inputs_with_per_input_labels(self): + with tempfile.TemporaryDirectory() as tmp: + left = _write(os.path.join(tmp, "left.txt"), + "00000000: 00000001 00000000 00000000 00000000\n") + right = _write(os.path.join(tmp, "right.txt"), + "00000000: 00000002 00000000 00000000 00000000\n") + rc, out, err = _run_main([ + "decode", "--input", left, "--label", "LEFT", + "--input", right, "--label", "RIGHT"]) + self.assertEqual(rc, 0, err) + self.assertIn("chunk LEFT", out) + self.assertIn("chunk RIGHT", out) + + def test_cli_decode_text_lines_reports_canonical_words(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write(os.path.join(tmp, "lines.txt"), + "INSTANCE[0x48]=0x100000\n") + out_path = os.path.join(tmp, "decoded.json") + rc, _, err = _run_main(["decode", "--input", path, + "--json", out_path]) + self.assertEqual(rc, 0, err) + with open(out_path, encoding="utf-8") as handle: + report = json.load(handle) + chunk = report["chunks"][0] + self.assertEqual(chunk["label"], "INSTANCE") + self.assertEqual(chunk["words"]["0x48"], "0x00100000") + self.assertEqual(chunk["unmapped_count"], 0) + + def test_cli_diff_exit_codes(self): + with tempfile.TemporaryDirectory() as tmp: + left = _write_snapshot(tmp, "left.json", [{ + "label": "RAMFC", "base": "0x0", + "words": ["0x0", "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", + "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", + "0x0", "0x0", "0x0", "0x0", "0x30000", "0x0"], + }]) + right = _write_snapshot(tmp, "right.json", [{ + "label": "RAMFC", "base": "0x0", + "words": ["0x0", "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", + "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", "0x0", + "0x0", "0x0", "0x0", "0x0", "0x10000", "0x0"], + }]) + rc, out, err = _run_main(["diff", left, right]) + self.assertEqual(rc, 3, err) + self.assertIn("CE_CTX_VALID: set -> clear", out) + rc, out, err = _run_main(["diff", left, left]) + self.assertEqual(rc, 0, err) + self.assertIn("identical=True", out) + + def test_cli_diff_accepts_decoded_reports(self): + with tempfile.TemporaryDirectory() as tmp: + path = _write_snapshot(tmp, "snap.json", [{ + "label": "INSTANCE", "base": "0x0", + "words": ["0xface", "0x0", "0x0", "0x0"], + }]) + report_path = os.path.join(tmp, "decoded.json") + rc, _, err = _run_main(["decode", "--input", path, + "--json", report_path]) + self.assertEqual(rc, 0, err) + rc, out, err = _run_main(["diff", report_path, report_path]) + self.assertEqual(rc, 0, err) + self.assertIn("identical=True", out) + + def test_cli_demo_green(self): + rc, out, err = _run_main(["demo"]) + self.assertEqual(rc, 0, err) + self.assertIn("ok=True", out) + self.assertIn("RAMFC_TARGET", out) + self.assertIn("CE_CTX_VALID: set -> clear", out) + + def test_demo_api_is_self_contained_and_deterministic(self): + first = run_demo() + second = run_demo() + self.assertEqual(first["format"], DEMO_FORMAT) + self.assertTrue(first["ok"]) + self.assertEqual(first["diff"]["summary"]["changed_words"], 2) + self.assertEqual(first["diff"]["summary"]["changed_fields"], 2) + self.assertEqual(json.dumps(first["diff"], sort_keys=True), + json.dumps(second["diff"], sort_keys=True)) + + +@unittest.skipUnless(os.path.isfile(EVIDENCE), "ladder evidence absent") +class EvidenceTests(unittest.TestCase): + def test_ladder_trace_decodes_instance_fields(self): + entries, source = load_map(None) + report = decode_specs([_spec(EVIDENCE)], entries, source) + self.assertGreaterEqual(len(report["chunks"]), 1) + chunks = {chunk["label"]: chunk for chunk in report["chunks"]} + self.assertIn("INSTANCE", chunks) + instance = chunks["INSTANCE"] + self.assertGreater(instance["word_count"], 0) + fields = {field["name"]: field for field in instance["fields"]} + self.assertEqual(fields["GPFIFO_VA"]["raw"], "0x00100000") + self.assertEqual(fields["MAGIC_0FACE"]["raw"], "0x0000face") + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_state_seed.py b/debug/risc-v-gsp-rm-vm/tests/test_state_seed.py new file mode 100644 index 0000000..439f36d --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_state_seed.py @@ -0,0 +1,311 @@ +"""Tests for the RM-object state-seeding harness (harness/state_seed.py, docs/52). + +Loader tests run against the native machine directly (no firmware image); the +FECS mechanism proof and the construct-gate attempt run on the shipped 570.144 +image and skip when it is absent. No hardware, no module, no guest, no +network. + +Run:: + + python3 -m unittest tests.test_state_seed -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.machine import Machine # noqa: E402 + +from harness.rehost import DEFAULT_IMAGE # noqa: E402 +from harness.state_seed import ( # noqa: E402 + DEMO_FORMAT, + REPORT_FORMAT, + SEED_FORMAT, + StateSeedError, + apply_seed, + fecs_seed, + load_seed, + main, + run_seed, + target_seed, +) +from harness.valprobe import encode_return_stub # noqa: E402 + +PROFILE = os.path.join(TOOL_ROOT, "sim", "profiles", "tu104-570.144.json") + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +def _machine() -> Machine: + return Machine(memory_mb=64, ram_base=0, unknown_mmio="lenient") + + +def _two_object_seed() -> dict: + return { + "format": SEED_FORMAT, + "name": "loader-test", + "entry": "0x5290dd4", + "allocator": {"base": "0x100000", "align": 0x100}, + "objects": [ + {"name": "client", "type": "client", "size": "0x2000", "fields": { + "0x1be0": "@ctx", + "0x1dd8": {"value": 0, "size": 8}, + "0x1aa8": "unknown", + }}, + {"name": "ctx", "type": "graphics_context", "size": "0x1000", "fields": { + "0x190": "@table", + "0x161": {"value": 0, "size": 1}, + }}, + {"name": "table", "type": "rm_object", "size": "0x100", "fields": { + "0x30": {"value": 0xCAFE, "size": 8}, + }}, + ], + "registers": {"a0": "@client", "a1": "@ctx", "t1": 0xC}, + "stack": "0x300000", + "memory": [{"pa": "0x200000", "u32": 7}], + "stubs": [{"name": "getter", "addr": "0x3f0000", "return": "0x280000"}], + "provenance": {"author": "test"}, + } + + +class SchemaTests(unittest.TestCase): + def test_layout_allocates_objects_in_order(self): + plan = load_seed(_two_object_seed()) + by_name = {item["name"]: item for item in plan["layout"]} + self.assertEqual(by_name["client"]["pa"], 0x100000) + self.assertEqual(by_name["ctx"]["pa"], 0x102000) + self.assertEqual(by_name["table"]["pa"], 0x103000) + + def test_pointer_and_value_fields_are_encoded(self): + plan = load_seed(_two_object_seed()) + writes = {item["label"]: item for item in plan["writes"]} + self.assertEqual(bytes.fromhex(writes["client+0x1be0"]["hex"]), + (0x102000).to_bytes(8, "little")) + self.assertEqual(bytes.fromhex(writes["ctx+0x190"]["hex"]), + (0x103000).to_bytes(8, "little")) + self.assertEqual(bytes.fromhex(writes["table+0x30"]["hex"]), + (0xCAFE).to_bytes(8, "little")) + self.assertEqual(bytes.fromhex(writes["mem+0x200000"]["hex"]), + (7).to_bytes(4, "little")) + + def test_unknown_marker_is_reported_and_not_written(self): + plan = load_seed(_two_object_seed()) + unknown = {(item["object"], item["off"]) for item in plan["unknown_fields"]} + self.assertIn(("client", "0x1aa8"), unknown) + labels = {item["label"] for item in plan["writes"]} + self.assertNotIn("client+0x1aa8", labels) + + def test_pc_register_becomes_entry(self): + seed = _two_object_seed() + del seed["entry"] + seed["registers"]["pc"] = "0x5290dd4" + plan = load_seed(seed) + self.assertEqual(plan["entry"], 0x5290DD4) + self.assertNotIn("pc", plan["registers"]) + + def test_plan_is_deterministic(self): + first = load_seed(_two_object_seed()) + second = load_seed(_two_object_seed()) + self.assertEqual(first["writes"], second["writes"]) + self.assertEqual(first["layout"], second["layout"]) + + def test_bad_format_rejected(self): + seed = _two_object_seed() + seed["format"] = "greeneye-state-seed/2" + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_missing_entry_rejected(self): + seed = _two_object_seed() + del seed["entry"] + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_unknown_object_type_rejected(self): + seed = _two_object_seed() + seed["objects"].append({"name": "x", "type": "gpu_of_doom"}) + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_duplicate_symbol_rejected(self): + seed = _two_object_seed() + seed["objects"].append({"name": "client", "type": "client"}) + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_unknown_pointer_rejected(self): + seed = _two_object_seed() + seed["objects"][0]["fields"]["0x1de8"] = "@does-not-exist" + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_bad_memory_item_rejected(self): + seed = _two_object_seed() + seed["memory"] = [{"pa": "0x200000"}] + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_overlapping_objects_rejected(self): + seed = _two_object_seed() + seed["objects"][1]["pa"] = "0x100800" + with self.assertRaises(StateSeedError): + load_seed(seed) + + def test_bad_register_rejected(self): + seed = _two_object_seed() + seed["registers"]["r99"] = 1 + with self.assertRaises(StateSeedError): + load_seed(seed) + + +class LoaderTests(unittest.TestCase): + def test_apply_seed_places_bytes_and_registers(self): + plan = load_seed(_two_object_seed()) + machine = _machine() + apply_seed(machine, plan) + client = 0x100000 + self.assertEqual(machine.read_mem(client + 0x1be0, 8), + (0x102000).to_bytes(8, "little")) + self.assertEqual(machine.read_mem(client + 0x1dd8, 8), b"\x00" * 8) + self.assertEqual(machine.read_mem(client + 0x1aa8, 8), b"\x00" * 8) + self.assertEqual(machine.read_mem(0x200000, 4), (7).to_bytes(4, "little")) + self.assertEqual(machine.x[10], client) + self.assertEqual(machine.x[11], 0x102000) + self.assertEqual(machine.x[6], 0xC) + self.assertEqual(machine.read_mem(0x3F0000, 12), encode_return_stub(0x280000)) + + def test_run_seed_missing_image_is_clean_error(self): + with self.assertRaises(StateSeedError): + run_seed(_two_object_seed(), image="/nonexistent/fw.bin") + + +@unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "570.144 firmware absent") +class FecsMechanismTests(unittest.TestCase): + """Prove the seeder on a path that needs no RM objects (docs/47 predicate).""" + + def test_seeded_slot_value_drives_the_verdict(self): + nonnull = run_seed(fecs_seed(slot=1), profile_path=PROFILE, record=False, + name="state-seed-test") + null = run_seed(fecs_seed(slot=0), profile_path=PROFILE, record=False, + name="state-seed-test") + self.assertEqual(nonnull["format"], REPORT_FORMAT) + self.assertEqual(nonnull["status"], "outcome") + self.assertEqual(nonnull["outcome"], "refuse-fecs-slot-exists-0x1f") + self.assertEqual(null["status"], "outcome") + self.assertEqual(null["outcome"], "accept-commit-path") + + def test_fecs_run_is_deterministic(self): + first = run_seed(fecs_seed(slot=1), profile_path=PROFILE, record=False, + name="state-seed-test") + second = run_seed(fecs_seed(slot=1), profile_path=PROFILE, record=False, + name="state-seed-test") + self.assertEqual(first["outcome"], second["outcome"]) + self.assertEqual(first["instructions"], second["instructions"]) + self.assertEqual(first["state_reads"], second["state_reads"]) + + def test_report_records_session_and_json(self): + with tempfile.TemporaryDirectory() as tmp: + report = run_seed(fecs_seed(slot=1), profile_path=PROFILE, out=tmp, + record=True, name="state-seed-test") + self.assertTrue(os.path.isfile(report["report_json"])) + with open(report["report_json"], "r", encoding="utf-8") as fh: + doc = json.load(fh) + self.assertEqual(doc["format"], REPORT_FORMAT) + self.assertTrue(report["session"]) + self.assertTrue(os.path.isdir(report["session"])) + + +@unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "570.144 firmware absent") +class TargetGateTests(unittest.TestCase): + """Attempt FUN_ram_052a70ec with a crafted client/channel/context seed.""" + + def test_minimal_seed_reports_first_gate(self): + report = run_seed(target_seed(advance=False), profile_path=PROFILE, + record=False, name="state-seed-test") + self.assertEqual(report["status"], "fault") + self.assertIsNotNone(report["gate"]) + self.assertEqual(report["gate"]["kind"], "null_indirect_call") + self.assertEqual(report["gate"]["label"], "engine_node+0x600") + self.assertEqual(report["gate"]["value"], 0) + labels = [item["label"] for item in report["state_reads"]] + self.assertIn("client+0x1de8", labels) + self.assertLess(labels.index("client+0x1de8"), + labels.index("engine_node+0x600")) + self.assertTrue(report["unknown_fields"]) + + def test_stub_advances_to_the_next_gate(self): + report = run_seed(target_seed(advance=True), profile_path=PROFILE, + record=False, name="state-seed-test") + self.assertEqual(report["status"], "fault") + self.assertEqual(report["gate"]["kind"], "null_indirect_call") + self.assertEqual(report["gate"]["label"], "channel+0x898") + + def test_target_seed_is_deterministic(self): + first = run_seed(target_seed(advance=False), profile_path=PROFILE, + record=False, name="state-seed-test") + second = run_seed(target_seed(advance=False), profile_path=PROFILE, + record=False, name="state-seed-test") + self.assertEqual(first["gate"], second["gate"]) + self.assertEqual(first["state_reads"], second["state_reads"]) + + +@unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "570.144 firmware absent") +class CliTests(unittest.TestCase): + def test_emit_and_validate_round_trip(self): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "fecs.json") + rc, out, _ = _run_main(["emit", "--kind", "fecs", "--out", path]) + self.assertEqual(rc, 0) + self.assertTrue(os.path.isfile(path)) + rc, out, _ = _run_main(["validate", "--seed", path]) + self.assertEqual(rc, 0) + doc = json.loads(out) + self.assertEqual(doc["format"], SEED_FORMAT) + self.assertEqual(doc["entry"], "0x5290dd4") + + def test_run_emitted_seed_with_out(self): + with tempfile.TemporaryDirectory() as tmp: + seed_path = os.path.join(tmp, "fecs.json") + rc, _, _ = _run_main(["emit", "--kind", "fecs", "--out", seed_path]) + self.assertEqual(rc, 0) + out_dir = os.path.join(tmp, "out") + rc, out, _ = _run_main(["run", "--seed", seed_path, "--out", out_dir, + "--no-record"]) + self.assertEqual(rc, 0) + doc = json.loads(out) + self.assertEqual(doc["status"], "outcome") + self.assertEqual(doc["outcome"], "refuse-fecs-slot-exists-0x1f") + self.assertTrue(os.path.isfile( + os.path.join(out_dir, "state-seed-report.json"))) + + def test_run_with_bad_seed_returns_two(self): + rc, _, err = _run_main(["run", "--seed", "/nonexistent/seed.json"]) + self.assertEqual(rc, 2) + self.assertIn("error:", err) + + def test_demo_returns_zero(self): + rc, out, _ = _run_main(["demo"]) + self.assertEqual(rc, 0) + doc = json.loads(out) + self.assertEqual(doc["format"], DEMO_FORMAT) + self.assertTrue(doc["ok"]) + self.assertIsNotNone(doc["target"]["gate"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_valprobe.py b/debug/risc-v-gsp-rm-vm/tests/test_valprobe.py new file mode 100644 index 0000000..cbf530a --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_valprobe.py @@ -0,0 +1,200 @@ +"""Tests for the validation-predicate probe (harness/valprobe.py, docs/47). + +The bundled FECS-promote matrix runs offline on the shipped 570.144 image +(skipped when the image is absent). No hardware, no module, no guest, no +network. + +Run:: + + python3 -m unittest tests.test_valprobe -v +""" +from __future__ import annotations + +import contextlib +import io +import json +import os +import struct +import sys +import tempfile +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.abspath(os.path.join(HERE, "..")) +if TOOL_ROOT not in sys.path: + sys.path.insert(0, TOOL_ROOT) + +from sim.decode import K_JALR, K_LUI, K_OPI, decode32 # noqa: E402 + +from harness.rehost import DEFAULT_IMAGE, _parse_u64 # noqa: E402 +from harness.valprobe import ( # noqa: E402 + MATRIX_FORMAT, + PROBE_FORMAT, + PROMOTE_VALIDATION, + SPEC_FORMAT, + ValprobeError, + decode_memory_spec, + encode_return_stub, + fecs_spec, + main, + reg_index, + run_spec, +) + +PROFILE = os.path.join(TOOL_ROOT, "sim", "profiles", "tu104-570.144.json") + + +def _run_main(argv: list[str]) -> tuple[int, str, str]: + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + rc = main(argv) + return rc, out.getvalue(), err.getvalue() + + +class EncodingTests(unittest.TestCase): + def test_stub_decodes_as_lui_addi_ret(self): + code = encode_return_stub(0x2FFFF9800) + self.assertEqual(len(code), 12) + words = struct.unpack("<III", code) + lui = decode32(0, words[0]) + addi = decode32(4, words[1]) + ret = decode32(8, words[2]) + self.assertEqual(lui[0], K_LUI) + self.assertEqual(lui[2], 10) + self.assertEqual(addi[0], K_OPI) + self.assertEqual(addi[2], 10) + self.assertEqual(ret[0], K_JALR) + self.assertEqual(ret[1], 0) # jalr zero, 0(ra) + + def test_stub_value_roundtrip(self): + # The lui+addi pair must reconstruct every value the probe uses + # (scratch addresses below 2^31, offset by the 12-bit signed addi). + for value in (0, 0x800, 0xFFF, 0x1234, 0x10000000, 0x1FFFFFFF, 0x7FFF_F000): + code = encode_return_stub(value) + hi = struct.unpack("<I", code[0:4])[0] >> 12 + lo = struct.unpack("<I", code[4:8])[0] >> 20 + if lo >= 0x800: + lo -= 0x1000 + self.assertEqual((hi << 12) + lo, value) + + def test_memory_spec_widths(self): + self.assertEqual(decode_memory_spec({"addr": "0x10", "hex": "aabb"}), + (0x10, b"\xaa\xbb")) + self.assertEqual(decode_memory_spec({"addr": "0x10", "u64": 1}), + (0x10, b"\x01" + b"\x00" * 7)) + self.assertEqual(decode_memory_spec({"addr": "0x10", "u32": 2}), + (0x10, b"\x02\x00\x00\x00")) + self.assertEqual(decode_memory_spec({"addr": "0x10", "u16": 3}), + (0x10, b"\x03\x00")) + self.assertEqual(decode_memory_spec({"addr": "0x10", "u8": 4}), + (0x10, b"\x04")) + with self.assertRaises(ValprobeError): + decode_memory_spec({"addr": "0x10"}) + + def test_register_aliases(self): + for name, index in (("s2", 18), ("x18", 18), ("a3", 13), ("t1", 6), ("fp", 8)): + self.assertEqual(reg_index(name), index) + with self.assertRaises(ValprobeError): + reg_index("r99") + + +class SpecTests(unittest.TestCase): + def test_fecs_spec_shape(self): + spec = fecs_spec() + self.assertEqual(spec["format"], SPEC_FORMAT) + self.assertEqual(_parse_u64(spec["target"]), PROMOTE_VALIDATION) + self.assertGreaterEqual(len(spec["variants"]), 15) + for variant in spec["variants"]: + self.assertTrue(variant["label"]) + self.assertIn(variant["expect"], set(spec["outcomes"].values())) + self.assertTrue(variant["stubs"]) + + def test_fecs_spec_layout_stays_inside_ram(self): + spec = fecs_spec() + profile_ram_mb = 12288 + ram_hi = profile_ram_mb * 1024 * 1024 + for variant in spec["variants"]: + for item in variant["memory"]: + self.assertLess(_parse_u64(item["addr"]) + 8, ram_hi) + + def test_missing_spec_is_clean_error(self): + rc, _, err = _run_main(["run", "--spec", "/nonexistent/spec.json"]) + self.assertEqual(rc, 2) + self.assertIn("error:", err) + + +@unittest.skipUnless(os.path.isfile(DEFAULT_IMAGE), "570.144 firmware absent") +class RealImageMatrixTests(unittest.TestCase): + def _reduced_spec(self) -> dict: + spec = fecs_spec() + keep = { + "FECS init vidmem, slot non-NULL", + "FECS init vidmem, slot NULL", + "FECS init sysmem attr 0x6, slot non-NULL", + "FECS no-init mapped, slot non-NULL", + "PM init", + } + spec["variants"] = [v for v in spec["variants"] if v["label"] in keep] + return spec + + def test_matrix_reproduces_live_verdicts_deterministically(self): + spec = self._reduced_spec() + first = run_spec(spec=spec, profile_path=PROFILE, record=False, name="valprobe-test") + second = run_spec(spec=spec, profile_path=PROFILE, record=False, name="valprobe-test") + self.assertEqual(first["format"], MATRIX_FORMAT) + self.assertEqual(first["evaluated"], len(spec["variants"])) + self.assertEqual(first["failed"], 0) + verdicts_a = [(r["label"], r["outcome"]) for r in first["variants"]] + verdicts_b = [(r["label"], r["outcome"]) for r in second["variants"]] + self.assertEqual(verdicts_a, verdicts_b) + by_label = dict(verdicts_a) + self.assertEqual(by_label["FECS init vidmem, slot non-NULL"], + "refuse-fecs-slot-exists-0x1f") + self.assertEqual(by_label["FECS init vidmem, slot NULL"], "accept-commit-path") + self.assertEqual(by_label["FECS no-init mapped, slot non-NULL"], "accept-skip-path") + + def test_probe_reports_firmware_slot_index_and_stub(self): + spec = self._reduced_spec() + spec["variants"] = [v for v in spec["variants"] + if v["label"] == "FECS init vidmem, slot non-NULL"] + matrix = run_spec(spec=spec, profile_path=PROFILE, record=False, name="valprobe-test") + row = matrix["variants"][0] + self.assertEqual(row["format"], PROBE_FORMAT) + self.assertEqual(row["status"], "outcome") + self.assertEqual(row["instructions"], row["instructions"]) + self.assertTrue(row["stubs"]) + slot_index = [r for r in row["reads"] if r.get("label") == "firmware-mapped-slot-index"] + self.assertEqual(slot_index[0]["u32"], 6) # firmware maps FECS -> slot 6 + + def test_boot_to_b3_then_probe(self): + spec = self._reduced_spec() + spec["variants"] = [v for v in spec["variants"] + if v["label"] == "FECS init vidmem, slot non-NULL"] + spec["boot_to"] = "B3" + matrix = run_spec(spec=spec, profile_path=PROFILE, record=False, name="valprobe-test") + row = matrix["variants"][0] + self.assertIsNotNone(row["boot"]) + self.assertEqual(row["boot"]["rung"], "B3") + self.assertTrue(row["boot"]["reached"]) + self.assertEqual(row["outcome"], "refuse-fecs-slot-exists-0x1f") + + def test_matrix_writes_report_json(self): + spec = self._reduced_spec() + spec["variants"] = spec["variants"][:2] + with tempfile.TemporaryDirectory() as tmp: + matrix = run_spec(spec=spec, profile_path=PROFILE, out=tmp, + record=False, name="valprobe-test") + self.assertTrue(os.path.isfile(os.path.join(tmp, "valprobe-matrix.json"))) + with open(matrix["report_json"], "r", encoding="utf-8") as fh: + doc = json.load(fh) + self.assertEqual(doc["format"], MATRIX_FORMAT) + self.assertEqual(len(doc["variants"]), 2) + + def test_cli_sweep_returns_zero(self): + rc, out, _ = _run_main(["sweep", "--no-record"]) + self.assertEqual(rc, 0) + self.assertIn("PASS", out) + + +if __name__ == "__main__": + unittest.main() |
