diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-03 14:05:38 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-03 14:05:38 -0400 |
| commit | cc79b478843f0230fe6aa8ecd822de7ba71c891e (patch) | |
| tree | 5de760ddec4e45cc23d39ce7502635f5c0711217 /debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md | |
| parent | 40351167dd209012ab485e6164ce94f0b8965422 (diff) | |
| download | open-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.tar.gz open-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.zip | |
vgsp: host CUDA bridge + sema release recovery; greeneye simulator wave
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean
- tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests
- tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode
- debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding
- STATE-CURRENT/docs updates
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md | 249 |
1 files changed, 249 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md b/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md new file mode 100644 index 0000000..42c76cc --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md @@ -0,0 +1,249 @@ +# 37 — Simulator vs the QEMU line: evaluation harness + real RE proof + +> **Status: in development (2026-10-03).** This document is the S4 (value +> validation) record for the standalone GSP simulator workstream. It defines +> the evaluation harness, freezes the thresholds used, reports the first +> measured A/B run on the real TU104 570.144 image, and records the U51 +> real-reverse-engineering demonstration. Nothing here is a product claim: +> simulator evidence is L0/L1 (docs/06). +> +> **Update (2026-10-03, later):** S1 (device model + profiles + boot shim) +> landed and the harness was wired to it; §4.3 records the re-measured A/B +> where the native side is the S1 `sim.gsp_boot` path (verdict `above` on the +> frozen checks). §4.1/§4.2 remain the pre-S1 measured record, and §7's +> "S1 not landed" limit applies only to those sections. +> +> **Artifacts:** `harness/sim_qemu_line.py` (A/B harness), +> `tests/test_sim_qemu_line.py` (shape + determinism + QEMU regression), +> `harness/re_demo.py` + `harness/re-demo-report.md` (real RE proof). +> **Line definition source:** docs/35-GREENEYE-SIMULATOR-DESIGN.md §1.1 and +> the boot ladder in §3.3. + +--- + +## 1. The line being evaluated + +The QEMU line (docs/35 §1.1) is the v0.1/v0.2 tool behaviour on the real +image: hooks/watchpoints, memory/struct/snapshot tooling, sessions, symbols +and analyze/compare, probes/plugins, Ghidra bridges, trace, injection SDK and +`Session.call`/`run_until`, plus the documented first fault at firmware vaddr +`0x4009014` (4-byte read of `0x2000000000000A00`). + +The boot ladder (docs/35 §3.3) is `pre-B0` (image loads, first MMIO observed +*as a fault*) < `B0` (first MMIO observed; lenient) < `B1` (first fault past +the QEMU point, explained by device semantics) < `B2` (boot-stage progress) < +`B3` (first mailbox poll) < `B4` (message loop) < `B5` (one scripted message +served). + +**"Above the QEMU line"** requires *all* of the frozen checks in §3, +including the docs/35 §1.1 requirement that the milestone be reached "with +the device model and shims documented". Lenient MMIO is an observation +mechanism, not a device model, so it cannot satisfy the contract by itself. + +## 2. Method + +Everything is stdlib-only, bounded (`<= 2,000,000` native instructions, +`<= 10 s` QEMU), and touches no hardware, GPU, mdev, module, guest or network. + +```bash +cd open-iov/debug/risc-v-gsp-rm-vm +python3 harness/sim_qemu_line.py # A/B harness -> harness/qemu_line/ +python3 -m unittest tests.test_sim_qemu_line -v +python3 harness/re_demo.py # U51 proof -> harness/re-demo-report.md +``` + +* **QEMU side** — `harness.common.open_flat_session` (the same relocation and + `guest_errors,unimp,int` logging as `greeneye probe`), bounded run, first + exception parsed from the bounded head of the log (trap loops flood it), + first-touch device map harvested, session recorded with the log blob. +* **Native side** — a discovered S1 profile runs `sim.gsp_boot` (device model + + recorded boot shim) twice and reports the observed rung; without one, the + lenient `sim.gsp_profile.run_profile` baseline (deterministic RV64IMAC, + modeled cycles) runs twice with `greeneye.device_map` for MMIO. S1's + profile is discovered defensively (`sim.devices` Python entry points, then + `sim/profiles/*.json` matched by firmware sha / `sim/devices/*.json`); when + absent the run is the lenient `B0` baseline and the report says so. +* **Feature matrix** — exercised on the bundled demo firmware + (`firmware/uproc_demo`): hooks, breakpoints, watchpoints, memory, struct + decode, sessions, analyze/compare, exec trace, injection SDK. MCP is + recorded `not_run` here to keep this harness independent of the MCP surface + (the native MCP tools are tested in `tests/test_mcp.py`; docs/20). + +## 3. Frozen thresholds (2026-10-03, before the first measurement) + +Stored in `harness/sim_qemu_line.py:FROZEN_THRESHOLDS` and +`harness/re_demo.py:FROZEN_THRESHOLDS`; they are never moved silently. + +| check | threshold | why | +|---|---|---| +| `qemu_first_fault_reproduced` | QEMU reproduces fw `0x4009014` / `0x2000000000000A00` | otherwise the two sides are not comparable | +| `native_above_qemu` | native rung >= QEMU rung + 1 | the line's rung rule | +| `identical_runs` | 2 identical native runs (key fields) | determinism is a docs/35 §1.1 requirement | +| `native_pair_wall_s` | <= 20 s for the two native runs | keeps the harness usable | +| `device_model_documented` | a discovered S1 device map/profile is used | docs/35 §1.1 "with the device model and shims documented" | + +`re_demo.py` additionally freezes: emulator wall <= 120 s, 5 answers present, +2 identical lenient runs, milestone past the QEMU point; the manual static +baseline is a documented **floor estimate** of 600 s (method note in the +script: ~90 instructions across three sections read by hand and correlated +with the QEMU fault report; not measured). + +## 4. Measured results (2026-10-03) + +### 4.1 A/B harness on the real image + +Firmware `gsp-570.144.fwimage.bin`, sha256 `2d1bb5c60b208cfd…`; S1 absent at +run time (`sim/devices/` and `sim/profiles/` not present), so the native side +ran lenient. + +| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence | +|---|---|---|---:|---:|---|---:|---| +| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.95 | `harness/qemu_line/sessions/qemu-probe` | +| native run 1 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.21 | `harness/qemu_line/sessions/native-1` | +| native run 2 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.22 | `harness/qemu_line/sessions/native-2` | + +Wall figures are from one measured run (host-load dependent); re-run the +command for current numbers. Session hashes for the record are in +`harness/qemu_line/sim-qemu-line.json`. + +* first-touch match with the documented reference: **true** (same address, + same 4-byte read, same fw PC); +* identical across the two runs: **true**; session compare ran (breakpoint, + watchpoint and written-page deltas); +* verdict: **`above_lenient_only`** — rung progress exists (`B0` is one past + `pre-B0`) but the device-model check fails, so `above_qemu_line = false`. + This is the honest underclaim: the native run observes and continues past + the QEMU fault, but lenient MMIO is not device semantics. +* native feature matrix (demo firmware): `hooks`, `breakpoints`, `watch`, + `memory`, `struct`, `sessions`, `analyze_compare`, `trace`, `injection` all + **pass**; `mcp` **not_run** (S3-owned). +* session hashes for every side/session are in + `harness/qemu_line/sim-qemu-line.json` and rendered in `.md`. + +Expected delta when S1 landed (realised; see §4.3): `device_model_used = true`, +rung `B2` because the model serves the QEMU first touch and the shim reaches +the relocation stage, and `verdict = above` because the run stays deterministic +and within the wall bound. S1's profile is picked up automatically from +`sim/profiles/*.json` when its `firmware.sha256` matches (or pass +`--device-map FILE` for an observational map). + +### 4.2 Real RE proof (U51) + +Scenario: *what happens on the real GSP image between entry and the first +fault/milestone, and what does the state tell us about boot?* + +| side | method | wall s | answers | +|---|---|---:|---| +| baseline | QEMU probe (measured, 0.5 s bound) | 0.99 | first fault only | +| baseline | manual static reading (floor estimate) | 600 | entry->fault path by hand | +| baseline | **total** | **600.99** | full question | +| emulator | 3 recorded native sessions (`harness/re_demo.py`) | **0.13** | full question + evidence | + +Frozen threshold: emulator wall <= 120 s, 5/5 answers present, two identical +lenient runs, milestone past the QEMU point. Result: **PASS** (all checks +true). Session hashes and the JSON answers are in `harness/re_demo/result.json` +and `harness/re-demo-report.md`. + +Answer (evidence-backed; simulator L0/L1, no silicon claim): + +1. **First touch:** a 4-byte read of PA `0x2000000000000A00` at fw PC + `0x4009014`; it is exactly the QEMU-documented first fault. +2. **Path:** the entry state is all-zero (no bootloader handoff is modelled); + `.bootstrap` materialises the device PA with five arithmetic instructions, + reads it, then runs a chip-ID comparison chain — 25 executed instructions + from entry to the halt (exec trace). +3. **Field:** the read value's bits `[28:20]` are extracted (`srli 20`, + `andi 0x1ff`) and compared against `354..360` (Turing IDs) and `368` + (Ampere ID). +4. **Dispatch:** `354..360` jumps to `.tu10x_text` at `0x4007000`; `368` + jumps to `.ga100_text` at `0x4005000`; any other value (including the + lenient zero) falls into the fail-closed `j .` loop at `0x4009070`. +5. **Next blocker (B1 candidate):** with a Turing ID served from a one-entry + device map, `.tu10x_text` programs CSRs `0x7c8–0x7cc` (region + descriptors), enables `mstatus` FP/SIMD state, builds the **absolute** + link-time address `0x40070f8` and jumps to it. Because the loader + relocates segments but not absolute immediates, that jump lands outside + RAM and faults (first fault past the QEMU point, 74 instructions). The + next rung therefore needs a chip-ID device model **plus** load-at-link + address (or alias/translation of absolute immediates); the modeled ID + value itself is an assumption (the QEMU baseline faults before reading + it), while the field position and the compared IDs are firmware evidence. + +### 4.3 Re-measurement after S1 landed (2026-10-03, later) + +The harness now discovers the S1 `greeneye-sim-profile/1` document +(`sim/profiles/*.json`, matched by `firmware.sha256`) and runs the native side +through `sim.gsp_boot` (device model + recorded boot shim) instead of lenient +MMIO. Frozen thresholds unchanged. Measured run on the same image +(sha256 `2d1bb5c60b208cfd…`): + +| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence | +|---|---|---|---:|---:|---|---:|---| +| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.98 | `harness/qemu_line/sessions/qemu-probe` | +| native run 1 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` (DMEM runspace unknown) | 0.16 | `harness/qemu_line/sessions/native-1` | +| native run 2 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` | 0.17 | `harness/qemu_line/sessions/native-2` | + +* `device_model_used = true`, `device_model_documented = true`, + `identical_runs = true` (device-state sha256 + `3fa264b9f81081e8…` on both runs), `native_pair_wall_s = 0.33`, + `native_above_qemu = true`; **verdict `above`** on all five frozen checks. +* milestones observed: `bootargs_consumer`, `relocation_copy`; + `unknown MMIO = 0`, `device_unhandled = 0`. +* What this does **not** claim: B3/B4 are not reached (the run stops at the + documented DMEM-runspace unknown); the shim values are declared assumptions; + cycles are modeled (docs/32); no timing/device-environment claim. +* Command: `python3 harness/sim_qemu_line.py` (regenerates + `harness/qemu_line/sim-qemu-line.json|.md`). Wall figures are host-load + dependent. + +## 5. Failure signals (when to disable, per U48/U56) + +* Any frozen check false **with QEMU available and the firmware present**: + keep `above_qemu_line = false`, do not claim the line. +* `device_model_documented` still false at the Phase S pause review: the S1 + work package has not met its acceptance; the harness keeps reporting + `above_lenient_only`. +* A feature matrix entry `fail`: disable that feature flag and record it as a + failed experiment; an entry silently stuck at `not_run` blocks the S4 exit. +* `re_demo.py` pass false: U51 is not satisfied; fix the tool chain before + any Phase G design work. +* Thresholds change only with a dated entry in this document; a moved + threshold without a measurement is a review failure. + +## 6. Re-running / reproducing + +```bash +cd open-iov/debug/risc-v-gsp-rm-vm +python3 harness/sim_qemu_line.py # full A/B + feature matrix +python3 harness/sim_qemu_line.py --no-qemu # native only, no baseline claim +python3 -m unittest tests.test_sim_qemu_line -v # shape/determinism/QEMU gate +python3 harness/re_demo.py # real RE proof +``` + +Without the 570.144 image the commands skip cleanly (exit 2 for the CLIs, +skipped tests), because the tool tree is used on machines without the +firmware. The QEMU side also needs `qemu-system-riscv64`; without it the +report is emitted with `unverified_qemu_absent`. + +## 7. Honest limits + +* **S1 partial** — the device model reaches B2 (relocation copy) with the + recorded boot shim; B3/B4 (CSB poll, message loop) are not reached and the + DMEM-runspace unknown is the documented ceiling (docs/36 §6). The §4.1 + lenient record predates S1; §4.3 is the post-S1 measurement. +* **L2/L3 partial** — cycles are modeled (`rv64imac-modeled/1`, docs/32), not + measured; the device model is partial and carries per-register evidence + (docs/36). No timing or full device-environment claim is made. +* **MCP native surface not exercised here** — the A/B feature matrix records + it `not_run`; the native MCP tools have their own suite (docs/20 §9). +* **Modeled chip-ID value is an assumption** — only the field extraction and + the compared ID set come from the image; in the S1 profile the `0x164` + value is declared by the profile, and the boot shim values are declared + assumptions recorded in the session. +* **Relocation artifact** — the B1-candidate fault at `0x40070f8` is a + property of the bias-relocated harness, not of silicon; it is labelled as + such and not counted as a device milestone. (The S1 path loads at the link + addresses and does not use that relocation.) +* **Manual baseline is an estimate** — labelled a floor with its method; the + measured parts of the baseline (QEMU probe) are reproducible with the + harness. |
