summaryrefslogtreecommitdiffstats
path: root/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
commitcc79b478843f0230fe6aa8ecd822de7ba71c891e (patch)
tree5de760ddec4e45cc23d39ce7502635f5c0711217 /debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md
parent40351167dd209012ab485e6164ce94f0b8965422 (diff)
downloadopen-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.tar.gz
open-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.zip
vgsp: host CUDA bridge + sema release recovery; greeneye simulator wave
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean - tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests - tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode - debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding - STATE-CURRENT/docs updates
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md')
-rw-r--r--debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md249
1 files changed, 249 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md b/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md
new file mode 100644
index 0000000..42c76cc
--- /dev/null
+++ b/debug/risc-v-gsp-rm-vm/docs/37-SIM-QEMU-LINE.md
@@ -0,0 +1,249 @@
+# 37 — Simulator vs the QEMU line: evaluation harness + real RE proof
+
+> **Status: in development (2026-10-03).** This document is the S4 (value
+> validation) record for the standalone GSP simulator workstream. It defines
+> the evaluation harness, freezes the thresholds used, reports the first
+> measured A/B run on the real TU104 570.144 image, and records the U51
+> real-reverse-engineering demonstration. Nothing here is a product claim:
+> simulator evidence is L0/L1 (docs/06).
+>
+> **Update (2026-10-03, later):** S1 (device model + profiles + boot shim)
+> landed and the harness was wired to it; §4.3 records the re-measured A/B
+> where the native side is the S1 `sim.gsp_boot` path (verdict `above` on the
+> frozen checks). §4.1/§4.2 remain the pre-S1 measured record, and §7's
+> "S1 not landed" limit applies only to those sections.
+>
+> **Artifacts:** `harness/sim_qemu_line.py` (A/B harness),
+> `tests/test_sim_qemu_line.py` (shape + determinism + QEMU regression),
+> `harness/re_demo.py` + `harness/re-demo-report.md` (real RE proof).
+> **Line definition source:** docs/35-GREENEYE-SIMULATOR-DESIGN.md §1.1 and
+> the boot ladder in §3.3.
+
+---
+
+## 1. The line being evaluated
+
+The QEMU line (docs/35 §1.1) is the v0.1/v0.2 tool behaviour on the real
+image: hooks/watchpoints, memory/struct/snapshot tooling, sessions, symbols
+and analyze/compare, probes/plugins, Ghidra bridges, trace, injection SDK and
+`Session.call`/`run_until`, plus the documented first fault at firmware vaddr
+`0x4009014` (4-byte read of `0x2000000000000A00`).
+
+The boot ladder (docs/35 §3.3) is `pre-B0` (image loads, first MMIO observed
+*as a fault*) < `B0` (first MMIO observed; lenient) < `B1` (first fault past
+the QEMU point, explained by device semantics) < `B2` (boot-stage progress) <
+`B3` (first mailbox poll) < `B4` (message loop) < `B5` (one scripted message
+served).
+
+**"Above the QEMU line"** requires *all* of the frozen checks in §3,
+including the docs/35 §1.1 requirement that the milestone be reached "with
+the device model and shims documented". Lenient MMIO is an observation
+mechanism, not a device model, so it cannot satisfy the contract by itself.
+
+## 2. Method
+
+Everything is stdlib-only, bounded (`<= 2,000,000` native instructions,
+`<= 10 s` QEMU), and touches no hardware, GPU, mdev, module, guest or network.
+
+```bash
+cd open-iov/debug/risc-v-gsp-rm-vm
+python3 harness/sim_qemu_line.py # A/B harness -> harness/qemu_line/
+python3 -m unittest tests.test_sim_qemu_line -v
+python3 harness/re_demo.py # U51 proof -> harness/re-demo-report.md
+```
+
+* **QEMU side** — `harness.common.open_flat_session` (the same relocation and
+ `guest_errors,unimp,int` logging as `greeneye probe`), bounded run, first
+ exception parsed from the bounded head of the log (trap loops flood it),
+ first-touch device map harvested, session recorded with the log blob.
+* **Native side** — a discovered S1 profile runs `sim.gsp_boot` (device model
+ + recorded boot shim) twice and reports the observed rung; without one, the
+ lenient `sim.gsp_profile.run_profile` baseline (deterministic RV64IMAC,
+ modeled cycles) runs twice with `greeneye.device_map` for MMIO. S1's
+ profile is discovered defensively (`sim.devices` Python entry points, then
+ `sim/profiles/*.json` matched by firmware sha / `sim/devices/*.json`); when
+ absent the run is the lenient `B0` baseline and the report says so.
+* **Feature matrix** — exercised on the bundled demo firmware
+ (`firmware/uproc_demo`): hooks, breakpoints, watchpoints, memory, struct
+ decode, sessions, analyze/compare, exec trace, injection SDK. MCP is
+ recorded `not_run` here to keep this harness independent of the MCP surface
+ (the native MCP tools are tested in `tests/test_mcp.py`; docs/20).
+
+## 3. Frozen thresholds (2026-10-03, before the first measurement)
+
+Stored in `harness/sim_qemu_line.py:FROZEN_THRESHOLDS` and
+`harness/re_demo.py:FROZEN_THRESHOLDS`; they are never moved silently.
+
+| check | threshold | why |
+|---|---|---|
+| `qemu_first_fault_reproduced` | QEMU reproduces fw `0x4009014` / `0x2000000000000A00` | otherwise the two sides are not comparable |
+| `native_above_qemu` | native rung >= QEMU rung + 1 | the line's rung rule |
+| `identical_runs` | 2 identical native runs (key fields) | determinism is a docs/35 §1.1 requirement |
+| `native_pair_wall_s` | <= 20 s for the two native runs | keeps the harness usable |
+| `device_model_documented` | a discovered S1 device map/profile is used | docs/35 §1.1 "with the device model and shims documented" |
+
+`re_demo.py` additionally freezes: emulator wall <= 120 s, 5 answers present,
+2 identical lenient runs, milestone past the QEMU point; the manual static
+baseline is a documented **floor estimate** of 600 s (method note in the
+script: ~90 instructions across three sections read by hand and correlated
+with the QEMU fault report; not measured).
+
+## 4. Measured results (2026-10-03)
+
+### 4.1 A/B harness on the real image
+
+Firmware `gsp-570.144.fwimage.bin`, sha256 `2d1bb5c60b208cfd…`; S1 absent at
+run time (`sim/devices/` and `sim/profiles/` not present), so the native side
+ran lenient.
+
+| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence |
+|---|---|---|---:|---:|---|---:|---|
+| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.95 | `harness/qemu_line/sessions/qemu-probe` |
+| native run 1 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.21 | `harness/qemu_line/sessions/native-1` |
+| native run 2 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.22 | `harness/qemu_line/sessions/native-2` |
+
+Wall figures are from one measured run (host-load dependent); re-run the
+command for current numbers. Session hashes for the record are in
+`harness/qemu_line/sim-qemu-line.json`.
+
+* first-touch match with the documented reference: **true** (same address,
+ same 4-byte read, same fw PC);
+* identical across the two runs: **true**; session compare ran (breakpoint,
+ watchpoint and written-page deltas);
+* verdict: **`above_lenient_only`** — rung progress exists (`B0` is one past
+ `pre-B0`) but the device-model check fails, so `above_qemu_line = false`.
+ This is the honest underclaim: the native run observes and continues past
+ the QEMU fault, but lenient MMIO is not device semantics.
+* native feature matrix (demo firmware): `hooks`, `breakpoints`, `watch`,
+ `memory`, `struct`, `sessions`, `analyze_compare`, `trace`, `injection` all
+ **pass**; `mcp` **not_run** (S3-owned).
+* session hashes for every side/session are in
+ `harness/qemu_line/sim-qemu-line.json` and rendered in `.md`.
+
+Expected delta when S1 landed (realised; see §4.3): `device_model_used = true`,
+rung `B2` because the model serves the QEMU first touch and the shim reaches
+the relocation stage, and `verdict = above` because the run stays deterministic
+and within the wall bound. S1's profile is picked up automatically from
+`sim/profiles/*.json` when its `firmware.sha256` matches (or pass
+`--device-map FILE` for an observational map).
+
+### 4.2 Real RE proof (U51)
+
+Scenario: *what happens on the real GSP image between entry and the first
+fault/milestone, and what does the state tell us about boot?*
+
+| side | method | wall s | answers |
+|---|---|---:|---|
+| baseline | QEMU probe (measured, 0.5 s bound) | 0.99 | first fault only |
+| baseline | manual static reading (floor estimate) | 600 | entry->fault path by hand |
+| baseline | **total** | **600.99** | full question |
+| emulator | 3 recorded native sessions (`harness/re_demo.py`) | **0.13** | full question + evidence |
+
+Frozen threshold: emulator wall <= 120 s, 5/5 answers present, two identical
+lenient runs, milestone past the QEMU point. Result: **PASS** (all checks
+true). Session hashes and the JSON answers are in `harness/re_demo/result.json`
+and `harness/re-demo-report.md`.
+
+Answer (evidence-backed; simulator L0/L1, no silicon claim):
+
+1. **First touch:** a 4-byte read of PA `0x2000000000000A00` at fw PC
+ `0x4009014`; it is exactly the QEMU-documented first fault.
+2. **Path:** the entry state is all-zero (no bootloader handoff is modelled);
+ `.bootstrap` materialises the device PA with five arithmetic instructions,
+ reads it, then runs a chip-ID comparison chain — 25 executed instructions
+ from entry to the halt (exec trace).
+3. **Field:** the read value's bits `[28:20]` are extracted (`srli 20`,
+ `andi 0x1ff`) and compared against `354..360` (Turing IDs) and `368`
+ (Ampere ID).
+4. **Dispatch:** `354..360` jumps to `.tu10x_text` at `0x4007000`; `368`
+ jumps to `.ga100_text` at `0x4005000`; any other value (including the
+ lenient zero) falls into the fail-closed `j .` loop at `0x4009070`.
+5. **Next blocker (B1 candidate):** with a Turing ID served from a one-entry
+ device map, `.tu10x_text` programs CSRs `0x7c8–0x7cc` (region
+ descriptors), enables `mstatus` FP/SIMD state, builds the **absolute**
+ link-time address `0x40070f8` and jumps to it. Because the loader
+ relocates segments but not absolute immediates, that jump lands outside
+ RAM and faults (first fault past the QEMU point, 74 instructions). The
+ next rung therefore needs a chip-ID device model **plus** load-at-link
+ address (or alias/translation of absolute immediates); the modeled ID
+ value itself is an assumption (the QEMU baseline faults before reading
+ it), while the field position and the compared IDs are firmware evidence.
+
+### 4.3 Re-measurement after S1 landed (2026-10-03, later)
+
+The harness now discovers the S1 `greeneye-sim-profile/1` document
+(`sim/profiles/*.json`, matched by `firmware.sha256`) and runs the native side
+through `sim.gsp_boot` (device model + recorded boot shim) instead of lenient
+MMIO. Frozen thresholds unchanged. Measured run on the same image
+(sha256 `2d1bb5c60b208cfd…`):
+
+| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence |
+|---|---|---|---:|---:|---|---:|---|
+| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.98 | `harness/qemu_line/sessions/qemu-probe` |
+| native run 1 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` (DMEM runspace unknown) | 0.16 | `harness/qemu_line/sessions/native-1` |
+| native run 2 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` | 0.17 | `harness/qemu_line/sessions/native-2` |
+
+* `device_model_used = true`, `device_model_documented = true`,
+ `identical_runs = true` (device-state sha256
+ `3fa264b9f81081e8…` on both runs), `native_pair_wall_s = 0.33`,
+ `native_above_qemu = true`; **verdict `above`** on all five frozen checks.
+* milestones observed: `bootargs_consumer`, `relocation_copy`;
+ `unknown MMIO = 0`, `device_unhandled = 0`.
+* What this does **not** claim: B3/B4 are not reached (the run stops at the
+ documented DMEM-runspace unknown); the shim values are declared assumptions;
+ cycles are modeled (docs/32); no timing/device-environment claim.
+* Command: `python3 harness/sim_qemu_line.py` (regenerates
+ `harness/qemu_line/sim-qemu-line.json|.md`). Wall figures are host-load
+ dependent.
+
+## 5. Failure signals (when to disable, per U48/U56)
+
+* Any frozen check false **with QEMU available and the firmware present**:
+ keep `above_qemu_line = false`, do not claim the line.
+* `device_model_documented` still false at the Phase S pause review: the S1
+ work package has not met its acceptance; the harness keeps reporting
+ `above_lenient_only`.
+* A feature matrix entry `fail`: disable that feature flag and record it as a
+ failed experiment; an entry silently stuck at `not_run` blocks the S4 exit.
+* `re_demo.py` pass false: U51 is not satisfied; fix the tool chain before
+ any Phase G design work.
+* Thresholds change only with a dated entry in this document; a moved
+ threshold without a measurement is a review failure.
+
+## 6. Re-running / reproducing
+
+```bash
+cd open-iov/debug/risc-v-gsp-rm-vm
+python3 harness/sim_qemu_line.py # full A/B + feature matrix
+python3 harness/sim_qemu_line.py --no-qemu # native only, no baseline claim
+python3 -m unittest tests.test_sim_qemu_line -v # shape/determinism/QEMU gate
+python3 harness/re_demo.py # real RE proof
+```
+
+Without the 570.144 image the commands skip cleanly (exit 2 for the CLIs,
+skipped tests), because the tool tree is used on machines without the
+firmware. The QEMU side also needs `qemu-system-riscv64`; without it the
+report is emitted with `unverified_qemu_absent`.
+
+## 7. Honest limits
+
+* **S1 partial** — the device model reaches B2 (relocation copy) with the
+ recorded boot shim; B3/B4 (CSB poll, message loop) are not reached and the
+ DMEM-runspace unknown is the documented ceiling (docs/36 §6). The §4.1
+ lenient record predates S1; §4.3 is the post-S1 measurement.
+* **L2/L3 partial** — cycles are modeled (`rv64imac-modeled/1`, docs/32), not
+ measured; the device model is partial and carries per-register evidence
+ (docs/36). No timing or full device-environment claim is made.
+* **MCP native surface not exercised here** — the A/B feature matrix records
+ it `not_run`; the native MCP tools have their own suite (docs/20 §9).
+* **Modeled chip-ID value is an assumption** — only the field extraction and
+ the compared ID set come from the image; in the S1 profile the `0x164`
+ value is declared by the profile, and the boot shim values are declared
+ assumptions recorded in the session.
+* **Relocation artifact** — the B1-candidate fault at `0x40070f8` is a
+ property of the bias-relocated harness, not of silicon; it is labelled as
+ such and not counted as a device milestone. (The S1 path loads at the link
+ addresses and does not use that relocation.)
+* **Manual baseline is an estimate** — labelled a floor with its method; the
+ measured parts of the baseline (QEMU probe) are reproducible with the
+ harness.