diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md | 156 |
1 files changed, 148 insertions, 8 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md b/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md index 2b77ca3..2f99f7b 100644 --- a/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md +++ b/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md @@ -9,10 +9,116 @@ backend: a small deterministic RISC-V machine (`libgreenmachine`) implementing the same `Session` contract as today's QEMU backend. It expands `GREEN-EYE-ROADMAP.md` §H3 (H3.1–H3.4) and the §H6 soft-processor annex. -**Nothing here exists today.** The only supported backend is `QemuTarget` + -`RspClient` (`greeneye/target.py`, `greeneye/rsp.py`); native is gated behind -H3 entry — H2.2, the QEMU device model that makes the real GSP image reach its -message loop — and must not be started ad hoc. This document only scopes it. +**Beyond M1.5 (§1.1), nothing here exists yet.** The only production backend is +`QemuTarget` + `RspClient` (`greeneye/target.py`, `greeneye/rsp.py`); M2+ of +the native backend is gated behind H3 entry — H2.2, the QEMU device model that +makes the real GSP image reach its message loop — and must not be started ad +hoc. This document scopes it. + +### 1.1 M1 status (implemented in this tree) + +M1 was explicitly commissioned and is implemented behind `--backend native`; +the status marker above still applies to the remaining milestones (M2–M6) and +to any use on real GSP images. + +* **What exists.** A pure-Python, stdlib-only RV64IMAC interpreter under + `sim/` (`sim/decode.py` decoder, `sim/machine.py` 256 MiB RAM + NS16550A + UART + execution loop) with an `RspClient`-shaped adapter in + `greeneye/sim_backend.py` (`SimClient`, `SimTarget`, `launch_native`). + `greeneye.session.Session` runs unchanged on it: registers by name, memory, + SW/HW breakpoints, watchpoints, step/continue/interrupt, `run_until`/`call`. +* **Firmware parity.** The bundled `uproc_demo` boots unmodified; a scripted + hook/watch/leave scenario produces an event-for-event identical profile on + QEMU and native (`tests/test_sim_backend.py::DifferentialTests`), including + handler arguments and return values. Watchpoints stop *before* the accessing + instruction, which reproduces QEMU's `watch` stop and the Session + clear/step/re-arm dance exactly. +* **Deviations from this design.** M1 here is the *reference* interpreter, not + the Rust product core of §12; no MMU, FPU, GC, snapshots/moments, map DSL, + scheduler, event bus or injection ABI (those are M2–M5). There is no + self-modifying code: the per-PC decode cache is invalidated by debugger + writes only. +* **Time model.** Virtual time is instruction count: `continue_(timeout)` + executes at most `timeout * 1_000_000` instructions (SIM_IPS), then reports + a budget timeout. `Session.run(duration=...)` therefore bounds native runs + by virtual time, deterministically; QEMU's wall-clock execution naturally + reaches different heartbeat counts in the same duration (see §14). +* **Measured performance** (one host thread, CPython 3.12, this machine): + ~2.3 MIPS tight RAM loop, ~1.5 MIPS sustained demo workload + (`examples/scripts/sim_bench.py`), ~1.25 MIPS with a hook + watchpoint + active. The ≥10 MIPS M1 target in §14 is a compiled-core target and is not + met by the Python reference; the demo self-test completes in ~2k + instructions, so the parity suite still runs in seconds. +* **Stop semantics.** `ebreak` (the demo's `rpc_fault`) stops *at* the trap as + a SIGTRAP like the gdbstub, and `step` executes over it; `wfi`, `fence` and + `fence.i` are no-ops (single hart, no ordering model); unmapped accesses and + illegal instructions stop as SIGSEGV/SIGILL stops instead of raising. + +### 1.1a M1.5 status - native exec/access trace (implemented in this tree) + +The roadmap R4 full-trace path now exists natively, without the QEMU TCG +plugin and without leaving the process: + +* **Engine.** `sim/trace.py` (`Trace`, `make_trace`) collects compact JSONL + lines from the execution loop in `sim/machine.py`: + `{"t":"exec","vcpu":0,"pc":"0x...","vt":N}` and + `{"t":"read"|"write","vcpu":0,"addr":"0x...","size":S,"pc":"0x...","vt":N}`, + ending in the plugin's `{"t":"summary",...}` line. `vt` is the machine's + deterministic instruction count; nothing reads the host clock. +* **Knobs (same semantics as `qemu/greeneye_trace.c`).** `exec=on|off` + (default on in the CLI), `start=`/`end=` inclusive PC filters applied to the + executing instruction - so memory accesses are filtered by *instruction* PC + exactly as the plugin filters at translation time - and a ring budget + `max_events` after which lines are counted as `dropped` (default 1,000,000, + `0` = unlimited). +* **Delivery.** The trace is written at Session close as the session blob + `blobs/trace.jsonl` (referenced from `session.json` as `trace_blob`, with + `trace` config/counts in the metadata), so it never swamps `events.jsonl`; + folded `exec`/`access`/`read`/`write` events in `events.jsonl` are also + accepted by the analyzer. +* **Analyzer.** `greeneye analyze` renders an `## Execution trace` section: + exec/read/write totals, unique PCs grouped by symbol, and accessed 4 KiB + pages with read/write counts. `greeneye compare` adds a trace-stream totals + delta and includes trace writes in the written-page delta. +* **API.** `ge.launch(..., backend="native", trace=...)` and + `launch_native(trace=...)` take `True`, a dict + `{"exec", "start", "end", "max_events"}`, or a `sim.trace.Trace`. +* **Measured cost.** On the tight one-instruction loop microbenchmark the + always-present `if trace` guard costs ~7% versus an unguarded copy; on the + demo workload (best-of runs) untraced throughput is unchanged within noise + (1.53 MIPS before, 1.55 MIPS after). Tracing itself costs ~2.4x on the demo + workload and ~3x on the microbenchmark. `examples/scripts/sim_bench.py + --trace` reports both. + +### 1.2 Usage + +```bash +./greeneye.sh demo --backend native # the canned demo, native backend +./greeneye.sh run examples/scripts/trace_rpc.py --backend native +printf 'bp rpc_sum\nc 0.2\nquit\n' | ./greeneye.sh attach --backend native +./greeneye.sh record --backend native --trace --duration 0.5 +./greeneye.sh record --backend native --trace --trace-exec off \ + --trace-start 0x80000000 --trace-end 0x80010000 +./greeneye.sh analyze dumps/<stamp>-record # renders the trace coverage section +python3 examples/scripts/sim_bench.py --trace # throughput + trace overhead +python3 examples/scripts/sim_diff.py # QEMU/native profile diff +``` + +```python +import greeneye as ge +s = ge.launch(kernel="firmware/uproc_demo/build/uproc_demo.elf", backend="native", + record="dumps/traced", trace={"exec": True, "start": 0x80000000}) +s.hook("rpc_sum", on_enter=lambda hit: print(hit.arg(0), hit.arg(1))) +s.watch(s.symbols.lookup("gsp_ctrl") + 0x10, size=8, kind="w") +s.run(duration=1.0) +s.close() # writes blobs/trace.jsonl +``` + +`--backend` is accepted by `demo`, `run`, `attach`, `record`, `inject` and +`ghidra`; the default stays `qemu` and the QEMU path is behaviourally +unchanged. `attach --port` stays QEMU-only (there is no server to attach to), +and `--trace` is native-only (QEMU uses the TCG plugin). Everything below +remains the design for M2+. ## 2. Why a custom simulator @@ -147,6 +253,13 @@ class Backend(Protocol): Compatibility decisions: +* **M1 implementation note.** Instead of the `Backend` protocol and typed + `Session(backend=...)` sketched below, M1 implements the narrower, already + exercised `RspClient` method surface (`greeneye/sim_backend.py`) and leaves + `Session` untouched. The `Backend` refactor is deferred until a second + non-RSP backend (e.g. Verilator, §16) actually needs it; §5's table is + still the M2+ target. + * `Session(client=...)` keeps working; `Session(backend=...)` is added and `client=` becomes an alias for `RspBackend(client)`. `Memory` is re-typed against `Backend`; its public methods do not change. @@ -291,16 +404,17 @@ single producer for stop-based and full-access events: |---|---|---| | `session_start`, `image_loaded`, `hook`, `watch`, `note`, `exit`, `session_end` | existing | unchanged | | `bp_hit`, `leave`, `watch_hit`, `stop`, `step`, `mem_read`, `mem_write`, `snapshot*`, `probe*`, `plugin*` | existing | unchanged; `mem_*` may now come from the bus, not only tooling | -| `exec` | new (M1, optional) | `pc`, `insn`, `vt`, `core`, `symbol` | -| `access` | new (M2) | `pc`, `addr`, `size`, `rw`, `region`, `vt`, `core` | +| `exec` | native M1.5 (blob) | `pc`, `vt` (+ `symbol` when folded into an event) | +| `access` | native M1.5 (`read`/`write` lines or folded events) | `pc`, `addr`, `size`, `vt`; `rw` only for the folded `access` form | | `device` | new (M2) | `device`, `name`, `rw`, `value`, `vt` | | `irq` / `schedule` | new (M3) | `line`, `state`, `core`, `vt`, `reason` | | `inject_*` | existing shape (docs/12) | `service_injected`, `call_enter`, `call_return` unchanged | `t` stays the recorder's monotonic seconds **filled from virtual time**, so the same run yields the same file. `exec`/`access` volume is controlled by -profile logging and a ring budget; the analyzer gains coverage/heatmap -rendering (roadmap R4). `greeneye compare` treats backend identity, `t` and +profile logging and a ring budget; for the native backend that is implemented +(M1.5, §1.1a): the stream is a session blob and the analyzer renders coverage +and accessed pages. `greeneye compare` treats backend identity, `t` and host paths as non-semantic, so cross-backend comparison is first-class. ## 11. Injection ABI @@ -352,12 +466,20 @@ familiar with the codebase; milestones are sequential. | # | Scope | Exit criteria | Effort | |---|---|---|---| | M1 | RV64IMAC interpreter, RAM, NS16550A-compatible UART, loader; `Session` adapter with `Backend` | bundled demo uproc boots unmodified; the canned instrumentation script (`03 §12`) produces the same hook/watch/stop timeline and memory snapshots as QEMU (canonical comparison); determinism test green | 4–6 wk | +| M1.5 | Native exec/access trace (R4 equivalent): `--trace`, filters, blob + analyzer coverage | trace determinism test green; analyzer renders unique PCs/pages; non-trace path unregressed (`sim_bench.py --trace`) | 1–2 wk | | M2 | Memory-map DSL + `falcon`/`csb`/`ptimer` devices + profile validation; native region snapshots; moments v1 | a re-hosted uproc build boots from a profile with no core changes; moment save/restore round-trips; restoring reproduces the event-stream hash | 4–6 wk | | M3 | Deterministic scheduler (budgets, timers, IRQ events) + event log/replay | multi-task test profile replays identically from the log on two hosts; tie-break and replay tests in CI | 3–4 wk | | M4 | Device-model parity with H2.2 + `dma`/`pmu_stub` + boot shims | real GSP image reaches the same stable state as the QEMU H2 exit under the same profile; differential suite passes for probe and boot fixtures; L3 parity documented, L2 still not claimed | 6–10 wk | | M5 | Injection ABI + service registry + `log`/`stub`/`forward` interception | an SDK v1 blob loads and serves a control end-to-end; each interception mode has a recorded acceptance session; fuzzing seed/restore demonstrated | 4–6 wk | | M6 | Performance/ergonomics: profiling, fast paths, `--backend native`, docs + flag-removal criteria | §14 targets met; `greeneye demo --backend native` in the differential CI job; user docs updated | 3–4 wk | +> **M1 status: done (pure-Python reference implementation).** Ported without +> the `Backend` refactor (see §5 note); differential parity and a determinism +> test are in `tests/test_sim_backend.py`; performance is §1.1. **M1.5 (native +> trace) is done** in §1.1a: `sim/trace.py`, `tests/test_sim_trace.py`, +> analyzer coverage rendering and `examples/scripts/sim_diff.py`. M2–M6 above +> are unchanged. + Total: roughly 24–36 engineering-weeks, consistent with the roadmap's "months; explicitly later" classification; none of it starts before the H3 entry gate, and if H2 slips this plan waits. @@ -389,6 +511,24 @@ harness, trended in CI): | MMIO access vs RAM access | ≤5× | M2 | | Event bus throughput | ≥1 M events/s buffered, lossless under budget | M3 | +> **M1 measured (pure-Python reference).** 2.3 MIPS tight RAM loop, 1.5 MIPS +> sustained demo workload, 1.25 MIPS with a hook + watchpoint active +> (`examples/scripts/sim_bench.py`, CPython 3.12). The ≥10 MIPS M1 row is a +> compiled-core gate and remains open for the Rust core; the Python reference +> exists to pin semantics and differentially test the firmware, which it does +> at parity. QEMU executes comparatively more instructions per wall second on +> this host, so equal wall-clock `run` durations produce different heartbeat +> counts across backends — the hook/watch/leave event profile is identical. +> +> **M1.5 trace overhead (same host).** The tracing guard is a single local +> boolean test per instruction plus one per memory access; A/B against an +> otherwise identical loop copy: ~2.4 MIPS vs ~2.6 MIPS on a pathological +> one-instruction loop (~7%), while the demo workload is unchanged within +> run-to-run noise (1.53 vs 1.55 MIPS best-of). With `exec=on` the demo +> workload runs at ~0.54 MIPS (~2.4x slower); an access-only trace is +> proportionally cheaper. Use `--trace-start/--trace-end` to bound the logged +> region, and the default 1M-event ring budget to bound memory. + ## 15. CI strategy * The Python package and QEMU path remain the default, always-on suite; no |