diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/35-GREENEYE-SIMULATOR-DESIGN.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/35-GREENEYE-SIMULATOR-DESIGN.md | 291 |
1 files changed, 291 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/35-GREENEYE-SIMULATOR-DESIGN.md b/debug/risc-v-gsp-rm-vm/docs/35-GREENEYE-SIMULATOR-DESIGN.md new file mode 100644 index 0000000..83954d4 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/35-GREENEYE-SIMULATOR-DESIGN.md @@ -0,0 +1,291 @@ +# 35 — GreenEye simulator design (single design document) + +> **Status:** in development (2026-10-03). Phase S (standalone GSP simulator) +> is the only implementation scope until its pause gate; Phase G (GPGPU-Sim +> board model) is designed and gated, not started. +> +> **Audience:** agents and maintainers of risc-v-gsp-rm-debugger. +> + +--- + +## 0. What this document is + +One consolidated design for the simulator program requested for +risc-v-gsp-rm-debugger. It covers, in order of execution: + +1. **Phase S — standalone GSP simulator** (now): execution of the real + firmware under the modelled, evidence-labelled device environment (L0/L1; + silicon accuracy is **not** claimed), full tool/workflow integration, and + the above-the-QEMU-line contract on the frozen checks (docs/37 §3). + **The program pauses at Phase S exit** before any GPGPU-Sim work. +2. **Phase G — GPGPU-Sim full-board model** (later, gated): TU104 board + devices (GMMU, FBIF, FALCON→GMMU DMA, vidmem heap, VBIOS), GSP↔core + interactions, PCIe/VFIO bridge for a vGSP VF, QEMU-gdb time sync, MCP + extension, multi-GPU/fabric backlog. +3. **Plan B** — the narrowly scoped reference plan for "a simulator simply + better than the QEMU baseline for the open-iov workflow" + (`../Handoff-Simulator-Plan-B.md`, private), with an explicit Plan A/Plan B + comparison and decision procedure. + +Owner directives are numbered U1–U45 in the private handoff (§1 and §13); +this document restates the operative ones and never contradicts them. + +## 1. Definitions + +### 1.1 The QEMU line (the bar to beat) + +The **QEMU baseline** is the v0.1/v0.2 tool behaviour measured on the real +GSP image: RSP transport, hooks/breakpoints/watchpoints, memory/struct/ +snapshot tooling, sessions, symbols/analyze/compare, probes/plugins, Ghidra +symbol + control bridges, TCG trace, injection SDK, `Session.call`/ +`run_until`; and on the real `gsp-570.144.fwimage.bin`: relocation-free flat +load, first exception at firmware vaddr `0x4009014` (load access fault +reading `0x2000000000000A00`), then `mtvec=0` fetch loop. Planned QEMU +features (device model H2.2, boot shims H2.3, deterministic multi-core H2.4, +RM service injection H1.3, trace→session folding R4, debugger parity H1.5) +are **part of the bar** even though QEMU has not implemented them. + +**"Above the QEMU line"** means, at minimum: + +* executes the real 570.144 image deterministically to at least one recorded + milestone **past** the QEMU first fault (ladder in §3.3), with the device + model and shims documented; +* every QEMU feature available against the native backend, plus the planned + QEMU features above; +* every run produces `greeneye-session/1` evidence with simulator build hash, + firmware hash, profile/hash, shim assumptions and fidelity labels; +* run-to-run determinism proven; cycle counts emitted and labelled *modeled*; +* a real-workload demonstration whose eng-ops baseline/threshold were frozen + before the run. + +### 1.2 Fidelity labels + +`L0` instruction semantics, `L1` memory behaviour, `L2` time/concurrency, +`L3` device environment (extended from docs/06). The standalone GSP +simulator targets **L0–L2 for the GSP core and its device model**, with L2 +explicitly labelled *modeled* (table/source-derived) until calibrated. The +GPGPU-Sim board model adds board-level L3. No silicon-accuracy claim without +calibration evidence and a declared uncertainty band. + +### 1.3 Accuracy metrics + +* `M1–M6` — the existing metric contract (inventory, replies, ordering, state, + error paths, memory side effects; docs/06). +* `C1–C6` — cycle calibration (instruction latency agreement, memory latency, + bandwidth, kernel correlation, determinism, clock-domain ratio); bands are + frozen in the private handoff §9.2 and must be recorded before first use. + +## 2. The workflow this exists to serve + +The simulator must move one of the project's OODA loops faster with evidence +(eng-ops §3): **dynamic RE** (what does the firmware do?), **portability** +(new firmware/device triage), **adversarial** (unexpected behaviour), and +**correctness** (does our emulation match?). Time-to-value is the currency: +every capability below names the baseline it replaces and the number that +decides keep/redesign/retire. + +## 3. Phase S — standalone GSP simulator (NOW) + +### 3.1 Architecture + +``` +greeneye control plane (Python; unchanged public API) + Session · hooks · bp/watch · probes · plugins · recorder · analyze/compare + Ghidra bridge · MCP · injection SDK + │ Backend ABI (typed; QEMU and native are peers) + ▼ +native backend (sim/) + machine.py RV64IMAC core, deterministic virtual time, cycle model + devices/ NEW: GSP device model (falcon/SEC2 windows, CSB/TRANSCOM, + WPR2/FWSEC state, PTIMER, IRQ routing, DMA/IOMMU, PMU stub) + profiles/ NEW: versioned device profiles (JSON) + boot shims + trace.py exec/access/cycle events → session blob + moments (S2) snapshot/restore + deterministic replay +``` + +*The QEMU backend stays untouched and remains the differential reference.* +Backend selection is explicit (`--backend qemu|native`); no shared mutable +state; the session format evolves additively (`greeneye-session/1` + new +event classes `cycle`, `device`, `mcu`). + +### 3.2 Work packages + +| WP | Name | Deliverable | Agent | +|---|---|---|---| +| S0 | baseline (done) | native core, session parity, M1.5 trace, modeled cycles, real-image bounded run | — | +| S1 | GSP device model + boot ladder | `sim/devices/` + `sim/profiles/` + milestone sessions + tests | S1 | +| S2 | debugger parity | catchpoints, core/thread selection, memory-region metadata, moments/replay, parity matrix | S2 | +| S3 | MCP + scripting + injection | MCP tools against native backend; injection against the simulated GSP; docs | S3 | +| S4 | real-workload validation | boot-ladder A/B vs QEMU probe; scenario harness; value demo + report | S4 | +| S5 | stability/packaging | long-run soak, error taxonomy, session size budgets, quickstart, release notes | successor | + +### 3.3 Boot ladder (S1 acceptance) + +Each rung is a recorded session with a session hash; each rung must be +reproducible twice identically and have its device requirements listed. + +| Rung | Milestone | Current status | +|---|---|---| +| B0 | flat image loads; first MMIO observed at `0x2000000000000A00` | done (native, lenient) | +| B1 | device model serves the whole boot path: 0 unknown MMIO, 0 unhandled in windows; deterministic 2M-instruction run stopping at the boot-halt/report loop (`0x4010928`) | done (S1 device model; sessions `dumps/gsp-boot-b1-run1/2`) | +| B2 | boot stage progress: vector table/TCM/MPU/trap handling exercised with the shim; passes the bootargs consumer and reaches the relocation copy, then stops on the known DMEM-runspace unknown (`fault @ 0x400EA0C`; docs/36 §4) | done (shim) | +| B3 | first CSB/TRANSCOM mailbox poll with correct read-back semantics | target | +| B4 | message loop reached (`appMain`/equivalent); waits on host message deterministically | target | +| B5 | one scripted host-side RM message served end-to-end with a recorded reply | target (gated by S3 injection) | + +The ladder is deliberately versioned-data driven: a new firmware version is a +new profile + shim record, not a code fork. + +### 3.4 What is explicitly not in Phase S + +GPGPU-Sim changes, board devices beyond the GSP complex, guest/PCIe models, +VBIOS, VF bridge, Ampere/Blackwell, fabric — all Phase G or backlog (§10). + +### 3.5 Pause gate + +Phase S ends when: S1–S4 exits met; suite green incl. new tests; a real +workload demo exists with a frozen baseline and measured time-to-value; +`docs/30–40` updated; adversarial review (§9) signed off. **Pause and report +before any Phase G work.** + +## 4. Phase G — GPGPU-Sim board model (later, gated) + +This is the previously requested GPGPU-Sim program, now explicitly sequenced +after Phase S. It is *designed* now so Phase S artifacts (device map, +profiles, session/event classes, metrics, MCP surface) fit it without rework. + +### 4.1 Board device inventory (TU104 family) + +Expanded from owner requirements (GMMU, FBIF, FALCON→GMMU DMA, vidmem heap) +and the porting dossier. + +| Device | Model target | Evidence class | +|---|---|---| +| GMMU (+ PTEs, page tables, ATS/IOMMU seam) | page-table walk, TLB, fault reporting, VA→PA translation for firmware descriptors | register/field maps + formats; A/B vs probe | +| FBIF (FB interface, compression, ECC) | FB access path, alignment/partition hashing, vidmem heap model | register maps; documented assumptions | +| FALCON→GMMU DMA line | descriptor-driven DMA with cycle/latency accounting | boot/mainline firmware usage; source traces | +| vidmem heap stack | allocator map, reserved regions, WPR2 carve-outs | boot state + shims | +| GSP core + CSB/TRANSCOM | message queues, doorbells, interrupts | Phase S artifacts | +| PMU / SEC2 / booter / ACR/SOE | interfaces, mailboxes, state machines, latencies | Phase S MCU models | +| VBIOS/PROM | image layout, init tables, device declaration for guests | public parsing + profile | +| PCIe endpoint | config space, BARs/MMIO windows, MSI-X, TLPs, D2D | descriptor + QEMU/GDB observations | +| VF bridge | vGSP VF traffic → simulator BAR/MMIO instead of real GPU | design-only until human approval | + +### 4.2 Light-touch VF bridge (design) + +* Userspace gpgpu-sim hosts the board model; a future (separately approved) + kernel module presents a PCIe/VFIO device FD. +* Protocol: shared-memory queue + eventfd doorbells; simulator services + MMIO/BAR reads/writes, MSI-X injection and DMA windows as messages; no + changes to `vgsp` primitives or data paths. +* **Hard rule:** no `vgsp` kernel-module edits without explicit human + approval. Phase S/G implement everything up to that boundary; the module is + design + tests on stubs only. +* A conformance harness replays recorded QEMU/VFIO traffic against the + simulator model and compares replies (accuracy loop), so the bridge is + proven before any kernel work is proposed. + +### 4.3 QEMU-gdb time sync (design) + +Guest QEMU and the simulator exchange a clock-sync protocol (periodic +time-base beacons + event ordering barriers). Deterministic mode: simulator +virtual time is the master; QEMU is granted step/sync credits. Never a +security boundary; no confidentiality claims. + +### 4.4 MCP extension (design) + +The existing 31-tool MCP surface grows simulator-native tools: run/step with +budget, cycle queries, device-state dumps, MCU state, boot-ladder report, +board topology, DMA/TLP traces. Read/observe default; writes/step gated by +flags; every call attributable to a session. + +## 5. Feature parity matrix + +| Capability | QEMU (current) | QEMU (planned) | Native GSP (now) | Native GSP target | +|---|---|---|---|---| +| RSP-free transports/launch | yes | — | launch/attach | yes | +| Hooks/watchpoints/breakpoints | yes | catchpoints/core select | hooks/watch/bp | full parity (S2) | +| Memory/struct/snapshots | yes | region metadata | memory/struct | snapshots/moments (S2) | +| Sessions/analyze/compare | yes | trace folding | yes + cycles | trace folded, coverage (S2) | +| Probe/record/plugins | yes | — | backend peer | yes | +| Ghidra symbol/control bridges | yes | watch/heat-map/nav | protocol-compatible | full (S2/S3) | +| Injection SDK | demo | RM-core interception H1.3 | demo backend | services vs GSP model (S3) | +| Device model / real boot | no (faults) | H2.2/H2.3/H2.4 planned | ladder B0–B2 (S1, device model + shim) | B4/B5 (S1/S3) | +| Determinism/replay | wall-clock | H2.4 budgets | instruction-count | event replay (S2) | +| MCP | yes (QEMU) | — | native sim tools + backend selection (20) | wider GSP-model tools (S3) | +| Tiering | — | — | modeled cycles | calibrated bands (S4) | + +## 6. Boundaries and invariants + +1. All changes live under `debug/risc-v-gsp-rm-vm/` (private handoff copies at + workspace root excepted). Never edit project directories outside it. +2. **No `vgsp` kernel-module edits without explicit human approval** (U33). +3. No physical GPU, mdev, guest, module loads, `/dev` or reboots from agents; + RM 610 is a no-fly zone (docs/23); no unattended cycles (RUNBOOK P1–P9). +4. The simulator is a study environment, never a product dependency, never in + a guest, no security claims (RUNBOOK P5/P6). +5. Upstream GPGPU-Sim stays a pinned read-only checkout; our changes are + tracked patches + overlay (Phase G), never a silent fork. +6. The QEMU path must remain green while native features land (differential + suite is the regression net). + +## 7. eng-ops admissions + +`gpgpu-sim/ADMISSION.md` holds draft records TOOL-2026-SIM-01..05 (build/run, +TU104 calibration, GSP-in-sim, MCU model, debugger backend) with the five +required fields; Phase S adds one admission record per S work package (S1 +device model, S2 parity, S3 MCP/injection, S4 validation). Thresholds are +frozen before the first run and never moved silently; every result carries a +session directory and hashes. + +## 8. Real-workload proof (S4) + +The "real workload" for Phase S is the open-iov workflow on the real image: +boot the 570.144 GSP to the highest ladder rung, expose device/MCU state +through the standard tooling (CLI/Python/MCP/Ghidra), serve one scripted +message, and record sessions. Baselines: (a) QEMU `probe` first fault; +(b) static analysis time estimates from the project handoff. The demo is +accepted only if it beats the frozen threshold; otherwise the failure signal +triggers redesign per eng-ops §4. + +## 9. Adversarial review + +| Risk | Failure signal | Mitigation | +|---|---|---| +| "Accuracy" theatre (lenient MMIO masquerades as a model) | rung claimed with lenient policy | each rung requires explicit device semantics; lenient runs labelled | +| Device model guesses without evidence | unexplained delta in A/B | every register backed by source or observed firmware use; record assumption | +| Parity breaks the QEMU path | differential suite red | QEMU untouched; feature-flag additions | +| Scope creep into Phase G | files outside `sim/`+docs | gate checks; private handoff scope contract | +| Non-determinism | replay delta non-empty | virtual-time only; double-run tests | +| False "above QEMU" claim | missing A/B session | acceptance requires session pair + hashes | +| Kernel-module temptation | any `open-iov/kernel/` diff | hard rule U33; reviews | +| Time-to-value negative | measured baseline worse | per-WP admission thresholds | + +## 10. Plan B comparison plan + +`Handoff-Simulator-Plan-B.md` (private) defines the narrow fallback: the +smallest simulator that is simply better than QEMU for the open-iov workflow. +At Phase S exit (and earlier if a work package fails twice), compare Plan A +output vs the Plan B design on: rung reached, feature coverage, determinism, +cycle evidence, integration effort, maintenance cost, and measured +time-to-value. Decision: **keep Plan A**, **apply Plan B lessons in place**, +or **start the Plan B implementation** (with a migration note and re-pilot). +The comparison and decision are recorded in the private handoff. + +## 11. Status and wave log + +| Wave | Date | Result | +|---|---|---| +| Recon + master handoff | 2026-10-03 | `handoff-simulator.md`, local sources register | +| W1 GPGPU-Sim build | 2026-10-03 | CUDA 12 port patches; `vectorAdd` functional+perf green | +| W1 TU104 config/overlay | 2026-10-03 | config scaffold + sanity 26/26 + admissions draft | +| W1 GSP cycle-0 | 2026-10-03 | modeled cycles; real image 200k instr deterministic; 298 tests green (2026-10-03 suite run; count varies with prerequisites) | +| W1 dossier | 2026-10-03 | Turing/Ampere/Blackwell parameter docs + cross-generation survey | +| W2 Phase S (S1 + S4) | 2026-10-03 | `sim/devices` + `sim/profiles` device model: B0–B2 on the real image (0 unknown MMIO), 2-run determinism; A/B harness discovers the profile and records verdict `above` on the frozen checks (docs/36, docs/37 §4.3, docs/40) | +| W2 Phase S | launched | S2 parity, S3 MCP/injection, S4 validation | + +Backlog (recorded, not started): Ampere/Blackwell models; fabric (PCIe, +NVLink/C2C, RDMA/NIC/switch/cable/transceiver, NVMe TLPs, bounce buffers, +DRAM tiers, rendezvous hashing); 8-GPU chassis; H100 SuperPOD RA; QEMU-gdb +time sync; VBIOS; VF bridge module; Verilog/FPGA; multi-version sweeps. |