path: root/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md

# 16 — Custom simulator: design and implementation plan
> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status; this is a
> far-later design, do not implement ahead of the H3 gate.

## 1. Purpose and status

This document is the design and delivery plan for a risc-v-gsp-rm-debugger-native execution
backend: a small deterministic RISC-V machine (`libgreenmachine`) implementing
the same `Session` contract as today's QEMU backend. It expands
`RISC-V-GSP-RM-DEBUGGER-ROADMAP.md` §H3 (H3.1–H3.4) and the §H6 soft-processor annex.

**Beyond M1.5 (§1.1), nothing here exists yet.** The only production backend is
`QemuTarget` + `RspClient` (`greeneye/target.py`, `greeneye/rsp.py`); M2+ of
the native backend is gated behind H3 entry — H2.2, the QEMU device model that
makes the real GSP image reach its message loop — and must not be started ad
hoc. This document scopes it.

### 1.1 M1 status (implemented in this tree)

M1 was explicitly commissioned and is implemented behind `--backend native`;
the status marker above still applies to the remaining milestones (M2–M6) and
to any use on real GSP images.

* **What exists.** A pure-Python, stdlib-only RV64IMAC interpreter under
  `sim/` (`sim/decode.py` decoder, `sim/machine.py` 256 MiB RAM + NS16550A
  UART + execution loop) with an `RspClient`-shaped adapter in
  `greeneye/sim_backend.py` (`SimClient`, `SimTarget`, `launch_native`).
  `greeneye.session.Session` runs unchanged on it: registers by name, memory,
  SW/HW breakpoints, watchpoints, step/continue/interrupt, `run_until`/`call`.
* **Firmware parity.** The bundled `uproc_demo` boots unmodified; a scripted
  hook/watch/leave scenario produces an event-for-event identical profile on
  QEMU and native (`tests/test_sim_backend.py::DifferentialTests`), including
  handler arguments and return values. Watchpoints stop *before* the accessing
  instruction, which reproduces QEMU's `watch` stop and the Session
  clear/step/re-arm dance exactly.
* **Deviations from this design.** M1 here is the *reference* interpreter, not
  the Rust product core of §12; no MMU, FPU, GC, snapshots/moments, map DSL,
  scheduler, event bus or injection ABI (those are M2–M5). There is no
  self-modifying code: the per-PC decode cache is invalidated by debugger
  writes only.
* **Time model.** Virtual time is instruction count: `continue_(timeout)`
  executes at most `timeout * 1_000_000` instructions (SIM_IPS), then reports
  a budget timeout. `Session.run(duration=...)` therefore bounds native runs
  by virtual time, deterministically; QEMU's wall-clock execution naturally
  reaches different heartbeat counts in the same duration (see §14).
* **Measured performance** (one host thread, CPython 3.12, this machine):
  ~2.3 MIPS tight RAM loop, ~1.5 MIPS sustained demo workload
  (`examples/scripts/sim_bench.py`), ~1.25 MIPS with a hook + watchpoint
  active. The ≥10 MIPS M1 target in §14 is a compiled-core target and is not
  met by the Python reference; the demo self-test completes in ~2k
  instructions, so the parity suite still runs in seconds.
* **Stop semantics.** `ebreak` (the demo's `rpc_fault`) stops *at* the trap as
  a SIGTRAP like the gdbstub, and `step` executes over it; `wfi`, `fence` and
  `fence.i` are no-ops (single hart, no ordering model); unmapped accesses and
  illegal instructions stop as SIGSEGV/SIGILL stops instead of raising.

### 1.1a M1.5 status - native exec/access trace (implemented in this tree)

The roadmap R4 full-trace path now exists natively, without the QEMU TCG
plugin and without leaving the process:

* **Engine.** `sim/trace.py` (`Trace`, `make_trace`) collects compact JSONL
  lines from the execution loop in `sim/machine.py`:
  `{"t":"exec","vcpu":0,"pc":"0x...","vt":N}` and
  `{"t":"read"|"write","vcpu":0,"addr":"0x...","size":S,"pc":"0x...","vt":N}`,
  ending in the plugin's `{"t":"summary",...}` line. `vt` is the machine's
  deterministic instruction count; nothing reads the host clock.
* **Knobs (same semantics as `qemu/greeneye_trace.c`).** `exec=on|off`
  (default on in the CLI), `start=`/`end=` inclusive PC filters applied to the
  executing instruction - so memory accesses are filtered by *instruction* PC
  exactly as the plugin filters at translation time - and a ring budget
  `max_events` after which lines are counted as `dropped` (default 1,000,000,
  `0` = unlimited).
* **Delivery.** The trace is written at Session close as the session blob
  `blobs/trace.jsonl` (referenced from `session.json` as `trace_blob`, with
  `trace` config/counts in the metadata), so it never swamps `events.jsonl`;
  folded `exec`/`access`/`read`/`write` events in `events.jsonl` are also
  accepted by the analyzer.
* **Analyzer.** `greeneye analyze` renders an `## Execution trace` section:
  exec/read/write totals, unique PCs grouped by symbol, and accessed 4 KiB
  pages with read/write counts. `greeneye compare` adds a trace-stream totals
  delta and includes trace writes in the written-page delta.
* **API.** `ge.launch(..., backend="native", trace=...)` and
  `launch_native(trace=...)` take `True`, a dict
  `{"exec", "start", "end", "max_events"}`, or a `sim.trace.Trace`.
* **Measured cost.** On the tight one-instruction loop microbenchmark the
  always-present `if trace` guard costs ~7% versus an unguarded copy; on the
  demo workload (best-of runs) untraced throughput is unchanged within noise
  (1.53 MIPS before, 1.55 MIPS after). Tracing itself costs ~2.4x on the demo
  workload and ~3x on the microbenchmark. `examples/scripts/sim_bench.py
  --trace` reports both.

### 1.2 Usage

```bash
./risc-v-gsp-rm-debugger.sh demo --backend native          # the canned demo, native backend
./risc-v-gsp-rm-debugger.sh run examples/scripts/trace_rpc.py --backend native
printf 'bp rpc_sum\nc 0.2\nquit\n' | ./risc-v-gsp-rm-debugger.sh attach --backend native
./risc-v-gsp-rm-debugger.sh record --backend native --trace --duration 0.5
./risc-v-gsp-rm-debugger.sh record --backend native --trace --trace-exec off \
    --trace-start 0x80000000 --trace-end 0x80010000
./risc-v-gsp-rm-debugger.sh analyze dumps/<stamp>-record   # renders the trace coverage section
python3 examples/scripts/sim_bench.py --trace    # throughput + trace overhead
python3 examples/scripts/sim_diff.py             # QEMU/native profile diff
```

```python
import greeneye as ge
s = ge.launch(kernel="firmware/uproc_demo/build/uproc_demo.elf", backend="native",
              record="dumps/traced", trace={"exec": True, "start": 0x80000000})
s.hook("rpc_sum", on_enter=lambda hit: print(hit.arg(0), hit.arg(1)))
s.watch(s.symbols.lookup("gsp_ctrl") + 0x10, size=8, kind="w")
s.run(duration=1.0)
s.close()   # writes blobs/trace.jsonl
```

`--backend` is accepted by `demo`, `run`, `attach`, `record`, `inject` and
`ghidra`; the default stays `qemu` and the QEMU path is behaviourally
unchanged. `attach --port` stays QEMU-only (there is no server to attach to),
and `--trace` is native-only (QEMU uses the TCG plugin). Everything below
remains the design for M2+.

## 2. Why a custom simulator

QEMU is an excellent execution engine and a poor *workflow* engine here. The
limits below are consequences of QEMU being a whole-board emulator with a GDB
stub as its scriptable surface; each maps to a §H3.1 requirement.

| QEMU property | Consequence for risc-v-gsp-rm-debugger | Requirement it forces |
|---|---|---|
| RSP serialises every access (hex frame, TCP round-trip, `PacketSize` chunking) | register/memory polling and watchpoint step-over are transport-bound; full-access tracing is impractical | in-process backend ABI with direct memory/register calls |
| No cheap machine snapshots; `savevm` is board-level and not exposed over the stub | region snapshots are copied over RSP; no instant checkpoint exists | copy-on-write full-machine snapshot/restore |
| No in-process plugin ABI: TCG plugins are C objects against QEMU internals; `qRcmd` is stringly | per-access callback *objects*, Python-side devices and deterministic fault injection have no home | native plugin/callback ABI for regions and devices |
| Whole-board device model (`virt` models boards, not a TU104-class firmware environment) | device behaviour lives in QEMU code and cannot be versioned as data with a firmware profile | memory-map DSL + small device modules, versioned as profiles |
| Host clock and iothread drive timers and wakeups | two runs of the same firmware interleave differently; replay is not reproducible | deterministic scheduler, virtual time, event-log replay |
| GDB stub stop model | instructions between stops are invisible unless a trace plugin is built and loaded | event bus emitting `exec`/`access` natively |
| Injection is debugger writes plus `Session.call` | RM-core call interception (stub/log/forward) is impossible at the ABI boundary | injection ABI with service registry and interceptors |

The §H3.1 contract: RV64IMAC (+GC when needed) interpreter at ≥10–50 MIPS on
one host thread (not realtime); memory map DSL; deterministic scheduler;
snapshot/restore; the existing `events.jsonl` schema; a backend ABI identical
to `Session`'s expectations; an injection ABI; Python-first embedding. §H3.4
adds: differential tests against QEMU, gate behind H2, minimal `Backend`.

## 3. Goals and non-goals

**Goals**

1. Byte-for-byte compatible `Session` behaviour for every script, probe,
   plugin, recording and comparison that works today.
2. Determinism: same image + profile + event log ⇒ identical event stream
   (modulo host-only metadata) on every run and host.
3. Cheap snapshot/restore and time-travel from snapshot + replay.
4. Interceptable device and RM-core boundaries: accesses and calls can be
   logged, stubbed, forwarded or (later, H4) mutated.
5. Firmware/device knowledge as versioned data (profiles), not code edits.

**Non-goals**

* Realtime or wall-clock guarantees; virtual time only.
* Confidentiality/isolation/security claims: a study environment only (§06 §6).
* Replacing QEMU before M4; native is opt-in (`--backend native`, M6).
* Graphics/board emulation, undeclared fidelity claims, or shipping: nothing
  here becomes a dependency of `gsp-rm-vfio` or a guest.

### 3.1 Fidelity claims

Levels are exactly those of `docs/06-ACCURACY.md`; the native backend does
not get to invent new ones.

| Level | Property | QEMU backend today | Native target (gated) |
|---|---|---|---|
| L0 | instruction semantics | high (TCG, same ISA family) | parity with QEMU on differential tests (M1) |
| L1 | memory behaviour | high for RAM-resident code | parity per profile; map is versioned data (M2) |
| L2 | time/concurrency | not modelled | deterministic virtual time and event order, **not** cycle accuracy (M3); cycle claims only via H6 |
| L3 | device environment | absent | parity with the H2.2 device model (M4) |

No L2/L3 conclusion may be published before its gate; timing is out of scope
entirely — the L2 row means reproducibly *ordered* events, not a timing model.

## 4. Architecture

The control plane is unchanged: `Session`, hooks, watchpoints, `Memory`,
`SymbolDB`, `Recorder`, `analyze`/`compare`, probes, plugins and the Ghidra
bridge sit above a backend interface. The §H3.2 sketch becomes:

```
+-------------------- risc-v-gsp-rm-debugger Python control plane --------------------+
| Session API, hooks/watchpoints, recorder, plugins, Ghidra/agent bridge |
+---------------------------- Backend interface -------------------------+
        | QEMU-RSP backend (today)            | native backend (planned)
        v                                     v
  qemu-system-riscv64                libgreenmachine
  + greeneye/rsp.py                  + cpu/    decoder + execute
  + greeneye/target.py               + mem/    map DSL, COW snapshots
                                     + dev/    falcon, csb, ptimer, dma, pmu
                                     + sched/  cores, timers, event queue
                                     + trace/  event bus, exec/access
                                     + inject/ blob loader, interceptors
```

Design rules: (1) the backend implements transport primitives only —
Frida-like `run_until`/`call` stay in `Session` (`greeneye/session.py:373`,
`:403`) and work on every backend for free; (2) device knowledge is data
(profile) plus a small module, never a core special case; (3) every backend
is optional — the Python package stays stdlib-only, and the native artifact
is loaded at session construction with a version check.

## 5. Backend interface

`Session` today assumes an RSP-like client. The planned interface is the same
small set of operations; `run_until` and `call` are implemented above it.

```python
class Backend(Protocol):
    def capabilities(self) -> set[str]: ...   # machine_snapshot | injection | haystack
    def close(self) -> None: ...
    def read_registers(self) -> dict[str, int]: ...
    def read_reg(self, name: str) -> int: ...
    def write_reg(self, name: str, value: int) -> None: ...
    @property
    def pc(self) -> int: ...
    def read_mem(self, addr: int, size: int) -> bytes: ...
    def write_mem(self, addr: int, data: bytes) -> None: ...
    def step(self) -> StopReply: ...
    def continue_(self, budget: Budget | None = None) -> StopReply: ...
    def interrupt(self) -> StopReply: ...
    def set_sw_break(self, addr: int) -> None: ...
    def clear_sw_break(self, addr: int) -> None: ...
    def set_hw_break(self, addr: int) -> None: ...
    def clear_hw_break(self, addr: int) -> None: ...
    def set_watch(self, addr: int, size: int, kind: str) -> None: ...
    def clear_watch(self, addr: int, size: int, kind: str) -> None: ...
    def save_moment(self, label: str) -> str: ...
    def restore_moment(self, moment_id: str) -> None: ...
    def inject(self, blob_elf: str, slot: str) -> "InjectionHandle": ...
```

| Session need | Backend primitive | QEMU-RSP backend (today) | Native backend (planned) |
|---|---|---|---|
| registers | `read_registers`/`read_reg`/`write_reg` | `g`/`p`/`P` packets | register file + CSRs, direct |
| memory | `read_mem`/`write_mem` | `m`/`M`/`X`, chunked to `PacketSize` | region lookup; RAM direct, MMIO callbacks |
| software break | `set_sw_break`/`clear_sw_break` | `Z0`/`z0`; `Session._step_over` | breakpoint table in the dispatch loop |
| hardware break | `set_hw_break`/`clear_hw_break` | `Z1`/`z1` | PC match |
| watchpoint | `set_watch`/`clear_watch` | `Z2/Z3/Z4`; stops *at* the accessing instruction | access check; contract below |
| step | `step` | `s` | one decoded instruction |
| continue | `continue_(budget)` | `c` + wall timeout + interrupt | run to breakpoint/budget/event |
| stop | `interrupt` | `0x03` | out-of-band flag checked per instruction |
| region snapshot | `read_mem` via `Memory` | copy over RSP | direct map copy |
| machine snapshot | `save_moment`/`restore_moment` | unsupported (raise, never lie) | COW pages + `moment.json` |
| injection | `inject` | docs/12 path: writes + `Session.call` | loader + service registry + interceptors |
| virtual time | `budget`, `StopReply.vt` | absent (host wall clock) | instruction count + timer ticks |

Compatibility decisions:

* **M1 implementation note.** Instead of the `Backend` protocol and typed
  `Session(backend=...)` sketched below, M1 implements the narrower, already
  exercised `RspClient` method surface (`greeneye/sim_backend.py`) and leaves
  `Session` untouched. The `Backend` refactor is deferred until a second
  non-RSP backend (e.g. Verilator, §16) actually needs it; §5's table is
  still the M2+ target.

* `Session(client=...)` keeps working; `Session(backend=...)` is added and
  `client=` becomes an alias for `RspBackend(client)`. `Memory` is re-typed
  against `Backend`; its public methods do not change.
* **Watchpoint contract.** QEMU stops after the watched access and requires
  the clear/step/re-arm dance (`greeneye/session.py:538`). Native advertises
  `watch_after_access` so the Session skips the dance on that path only; a
  `watch_before_access` mode exists for injection experiments but is not the
  default, which must reproduce today's event profiles.
* `Budget` is `{instructions, until_event}`; the wall-clock `duration=`
  argument is translated via the profile's nominal clock, so native `run` is
  deterministic. `StopReply` gains optional `vt` and `access`, defaulting for
  old parsers.

## 6. Core choice: interpreter first, JIT later

| Option | Determinism | Speed | Debuggability | Cost / risk |
|---|---|---|---|---|
| Pure-Python interpreter | highest | very low (≪1 MIPS) | highest | prototype only; excellent oracle |
| Decode/execute interpreter (recommended M1–M5) | highest | 10–50 MIPS achievable in Rust/C++ | high; per-instruction hooks and single-step are trivial | low-moderate |
| JIT (copy-and-patch / TCG-like) | high if codegen is fixed | ≫100 MIPS | hard: self-modifying code, translation-cache invalidation | deferred to M6, only if profiling demands |
| Existing core (e.g. a small open RISC-V interpreter/JIT) | depends | varies | depends | evaluation only; hooks/snapshots must be retrofit |

**Decision.** Build a decode/execute interpreter with the ISA behind a
`CpuCore` interface: a decoder mapping instruction words to typed operations,
an execute loop dispatching into the memory map. `MISA` is a profile property
(`rv64imac` default, `rv64gc` selectable, per 07 §P3); compressed
instructions are canonicalised first so breakpoints and single-step work
uniformly. No FPU/vector until a GC profile is needed. An embedded existing
core (public options such as `libriscv`/`rv8`) is an **evaluation note only**:
revisit at M4 as a `CpuCore` if its memory callbacks, watchpoint granularity,
snapshot hooks and licence/provenance fit — it is not the M1 substrate,
because those hooks *are* the product.

## 7. Memory map DSL and device plugins

The map is versioned data interpreted by `mem/`. Region kinds are `ram`,
`mmio`, `dma` and `host-window`; each has attributes, logging policy and
either a backing store or a device. Windows resolve to device instances.

```yaml
schema: greenmachine-profile/1
profile: tu10x-gsp
isa: rv64imac
devices:
  falcon0: { module: falcon, instance: gsp }
  csb0:    { module: csb, mailbox_depth: 8 }
  ptimer0: { module: ptimer, hz: 1000000 }
  pmu0:    { module: pmu_stub, revision: r1 }
  dma0:    { module: dma, iommu: pass-through }
regions:
  - { name: tcm, base: 0x4000000, size: 0x40000, kind: ram, attrs: [r,w,x] }
  - { name: gsp_regs, base: 0x2000000000000000, size: 0x100000, kind: mmio,
      log: changes,   # off | changes | access
      windows: [{ offset: 0x000000, size: 0x1000, device: falcon0 },
                { offset: 0x001000, size: 0x1000, device: csb0, writable: true },
                { offset: 0x002000, size: 0x1000, device: ptimer0 },
                { offset: 0x200a00, size: 0x0200, device: pmu0 }] }
  - { name: host_dma, base: 0x100000000, size: 0x10000000, kind: dma,
      device: dma0 }
```

Device modules are small in-process plugins; the same interface is what makes
`stub`/`log`/`forward` interception possible later:

```python
class Device(Protocol):
    def reset(self, ctx) -> None: ...
    def read(self, ctx, offset: int, size: int) -> int: ...
    def write(self, ctx, offset: int, size: int, value: int) -> None: ...
    def tick(self, ctx, now: int) -> list[Event]: ...
    def save_state(self) -> dict: ...
    def load_state(self, state: dict) -> None: ...
```

Planned modules (all stubs until the H2.2 model defines semantics):

| Module | Role | Notes |
|---|---|---|
| `falcon` | FALCON/SEC2 register windows, imem/dmem, bootvec | read-back semantics for boot polling |
| `csb` | CSB/TRANSCOM mailbox windows | message queues, doorbells as events |
| `ptimer` | programmable timer | feeds the scheduler; deterministic ticks |
| `dma` | descriptor engine into host/FB windows | completions are scheduled events, not threads |
| `pmu_stub` | identity/clock provisioning reads | canned but recorded; never a timing source |
| `boot_shim` | FWSEC/booter/SEC2 hand-off state | documented shim; assumptions in session metadata |

Profiles are keyed by `(device, firmware-family, profile-version)` and carry
the firmware hash, provenance, shim assumptions and the validated map. A new
pairing is a new profile, never a core change; profiles are JSON-Schema
validated in CI.

## 8. Deterministic scheduler

Virtual time advances only by executed instructions and scheduled events; no
host clock, thread or entropy source may influence guest-visible state.

* **Cores.** One or more vCPUs with instruction budgets; the default is a
  strict, seeded round-robin so interleavings are reproducible.
* **Event queue.** Timers, IRQ lines, DMA completions and external stimuli
  are ordered `(vt, kind, source, payload, seq)`; ties break by sequence
  number, never host timing. `ptimer` ticks enqueue IRQ events; delivery
  defaults to instruction boundaries, and injected interrupts are explicit
  and recorded.
* **Replay.** A recording captures a seed plus the ordered external-event
  log; re-running the same image, profile and log must reproduce the event
  stream and final memory hashes. The determinism test runs every suite twice
  and compares a canonical hash.
* **Hygiene (tested).** No `dict` iteration order in device dumps, no
  unordered set serialisation, no host paths in semantic fields, fixed page
  allocation order, fixed hash seeds where a container affects order.

## 9. Snapshot/restore and time-travel

A machine snapshot ("moment") is a new artifact type, not a change to region
snapshots (`snapshots/*.bin.z`, `01-DESIGN.md §6`):

```
moments/<id>/
  moment.json   # greenmachine-moment/1, profile id+hash, vt, event-log offset,
                # register file, CSR/MPU state, scheduler cursor, device state
  pages/        # COW page list: base page id + XOR/delta or full page
  events.tail   # optional copy of events since the previous moment
```

* **Memory** is page-granular copy-on-write from a base moment; a 256 MiB
  image that changes little is a page list, not a copy. **Device state**
  comes from `Device.save_state()`; devices without it are marked "reset
  required" and the moment is partial. **Event-log position** indexes
  `events.jsonl`; restore replays external events from that index.
* **Use cases (M2/M3):** checkpoint before a fault and explore both branches;
  flip one reply byte and compare; restore and re-run a suffix to delta-debug
  an injected service; seed a fuzzing harness from a moment. **Limits:**
  in-flight DMA and out-of-band input must be in the event log, host/FB
  windows backed by real files are excluded, and the profile hash must match.

## 10. Event bus

The recorder schema (`greeneye-session/1`, one JSON object per line with `t`
and `type`, `greeneye/record.py`) is preserved exactly. The native bus is the
single producer for stop-based and full-access events:

| Event type | Status | Fields |
|---|---|---|
| `session_start`, `image_loaded`, `hook`, `watch`, `note`, `exit`, `session_end` | existing | unchanged |
| `bp_hit`, `leave`, `watch_hit`, `stop`, `step`, `mem_read`, `mem_write`, `snapshot*`, `probe*`, `plugin*` | existing | unchanged; `mem_*` may now come from the bus, not only tooling |
| `exec` | native M1.5 (blob) | `pc`, `vt` (+ `symbol` when folded into an event) |
| `access` | native M1.5 (`read`/`write` lines or folded events) | `pc`, `addr`, `size`, `vt`; `rw` only for the folded `access` form |
| `device` | new (M2) | `device`, `name`, `rw`, `value`, `vt` |
| `irq` / `schedule` | new (M3) | `line`, `state`, `core`, `vt`, `reason` |
| `inject_*` | existing shape (docs/12) | `service_injected`, `call_enter`, `call_return` unchanged |

`t` stays the recorder's monotonic seconds **filled from virtual time**, so
the same run yields the same file. `exec`/`access` volume is controlled by
profile logging and a ring budget; for the native backend that is implemented
(M1.5, §1.1a): the stream is a session blob and the analyzer renders coverage
and accessed pages. `greeneye compare` treats backend identity, `t` and
host paths as non-semantic, so cross-backend comparison is first-class.

## 11. Injection ABI

The docs/12 blob contract is reused unchanged: ELF64 `EM_RISCV`, RV64IMAC,
soft-float, PIC, one dispatch entry, optional `ge_service_t` metadata. The
native backend changes *where the blob is registered* and *what can be
intercepted* (H1.3/H3.1):

* **Loader.** `Backend.inject()` verifies PIC/no-relocations, maps PT_LOAD
  segments through the memory map, and records `service_injected`.
* **Service registry.** The demo path registers into the firmware handler
  slot as today; the RM path (M5) registers into a service table presented to
  the simulated RM core, so the firmware dispatches to the service itself.
* **Interception points** are declared in the profile and matched at runtime:

| Mode | Behaviour | Intended use |
|---|---|---|
| `log` | real call executes; target, args and return recorded | observation, M5 evidence |
| `stub` | simulator returns a canned reply; real code skipped | isolate the RM front-end from an engine |
| `forward` | call routed to a nominated implementation or injected service | prototype a replacement engine |
| `mutate` (gated H4) | argument or reply deterministically corrupted; mutation id recorded | fuzzing harness |

**ABI v2 sketch (M5).** `ge_service_fn(cmd, a0, a1)` becomes
`ge_service_v2(ctx, cmd, a0, a1)` where `ctx` carries a call table; blobs
declare their ABI version so loaders can refuse mismatches. Interception
applies to call-table calls, MMIO accesses and the RM dispatch boundary.

## 12. Language and embedding options

| Option | Speed | Embedding | Safety/ergonomics | Build & CI cost | Assessment |
|---|---|---|---|---|---|
| C/C++ core | high | CPython C API or pybind11 | manual memory; best if a core is embedded later | compiler matrix; sanitizers essential | viable fallback; no build advantage |
| Rust core + PyO3 | high | first-class via PyO3/maturin | memory-safe; enums/decoders fit the ISA well | cargo + maturin; easy wheel caching | **recommended** |
| Pure Python | lowest | native | fastest iteration; great reference model | none | M0/M1 oracle, not the product core |

**Recommendation.** Build a pure-Python reference interpreter first as an
executable specification and differential oracle (small, CI-friendly, forces
the memory-map semantics to be testable), then the production core in Rust
with PyO3 behind an identical C-visible ABI. The C ABI lets a C/C++ core or
an existing core slot into `CpuCore` without touching the control plane.

## 13. Implementation plan (expands H3.3)

Entry gate for all milestones: **H2.2 exit** and a stable H2.2 device
semantics baseline. Effort is rough engineering-weeks for one developer
familiar with the codebase; milestones are sequential.

| # | Scope | Exit criteria | Effort |
|---|---|---|---|
| M1 | RV64IMAC interpreter, RAM, NS16550A-compatible UART, loader; `Session` adapter with `Backend` | bundled demo uproc boots unmodified; the canned instrumentation script (`03 §12`) produces the same hook/watch/stop timeline and memory snapshots as QEMU (canonical comparison); determinism test green | 4–6 wk |
| M1.5 | Native exec/access trace (R4 equivalent): `--trace`, filters, blob + analyzer coverage | trace determinism test green; analyzer renders unique PCs/pages; non-trace path unregressed (`sim_bench.py --trace`) | 1–2 wk |
| M2 | Memory-map DSL + `falcon`/`csb`/`ptimer` devices + profile validation; native region snapshots; moments v1 | a re-hosted uproc build boots from a profile with no core changes; moment save/restore round-trips; restoring reproduces the event-stream hash | 4–6 wk |
| M3 | Deterministic scheduler (budgets, timers, IRQ events) + event log/replay | multi-task test profile replays identically from the log on two hosts; tie-break and replay tests in CI | 3–4 wk |
| M4 | Device-model parity with H2.2 + `dma`/`pmu_stub` + boot shims | real GSP image reaches the same stable state as the QEMU H2 exit under the same profile; differential suite passes for probe and boot fixtures; L3 parity documented, L2 still not claimed | 6–10 wk |
| M5 | Injection ABI + service registry + `log`/`stub`/`forward` interception | an SDK v1 blob loads and serves a control end-to-end; each interception mode has a recorded acceptance session; fuzzing seed/restore demonstrated | 4–6 wk |
| M6 | Performance/ergonomics: profiling, fast paths, `--backend native`, docs + flag-removal criteria | §14 targets met; `greeneye demo --backend native` in the differential CI job; user docs updated | 3–4 wk |

> **M1 status: done (pure-Python reference implementation).** Ported without
> the `Backend` refactor (see §5 note); differential parity and a determinism
> test are in `tests/test_sim_backend.py`; performance is §1.1. **M1.5 (native
> trace) is done** in §1.1a: `sim/trace.py`, `tests/test_sim_trace.py`,
> analyzer coverage rendering and `examples/scripts/sim_diff.py`. M2–M6 above
> are unchanged.

Total: roughly 24–36 engineering-weeks, consistent with the roadmap's
"months; explicitly later" classification; none of it starts before the H3
entry gate, and if H2 slips this plan waits.

## 14. Differential testing and performance targets

**Differential testing (the fidelity gate for every milestone).** Fixture:
the bundled demo firmware plus a re-hosted uproc build once M2 exists. The
same script runs on both backends; recordings are canonicalised (drop `t`,
host paths, backend identity, QEMU-log-only fields) and compared: identical
event-type sequence/counts; identical hook/watch hits and `pc` per stop;
byte-identical region snapshots at matched points; `exec`/`access` streams
equal in order, multiplicity and virtual timestamps at equal log levels.

Divergence blocks the milestone and is filed with both session directories as
evidence (`06 §5`/`§7`); a declared fidelity gap is recorded, not waived.

**Performance targets** (one host thread; a checked `greeneye-sim bench`
harness, trended in CI):

| Metric | Target | Gate |
|---|---|---|
| RAM-bound tight loop (no logging) | ≥10 MIPS | M1 |
| Interpreter optimized | ≥50 MIPS | M6 |
| JIT (only if built) | ≥200 MIPS | post-M6, gated |
| Moment save, 256 MiB RAM, sparse changes | ≤100 ms | M3 |
| Moment restore | ≤50 ms | M3 |
| Watchpoint-enabled slowdown | ≤2× | M4 |
| MMIO access vs RAM access | ≤5× | M2 |
| Event bus throughput | ≥1 M events/s buffered, lossless under budget | M3 |

> **M1 measured (pure-Python reference).** 2.3 MIPS tight RAM loop, 1.5 MIPS
> sustained demo workload, 1.25 MIPS with a hook + watchpoint active
> (`examples/scripts/sim_bench.py`, CPython 3.12). The ≥10 MIPS M1 row is a
> compiled-core gate and remains open for the Rust core; the Python reference
> exists to pin semantics and differentially test the firmware, which it does
> at parity. QEMU executes comparatively more instructions per wall second on
> this host, so equal wall-clock `run` durations produce different heartbeat
> counts across backends — the hook/watch/leave event profile is identical.
>
> **M1.5 trace overhead (same host).** The tracing guard is a single local
> boolean test per instruction plus one per memory access; A/B against an
> otherwise identical loop copy: ~2.4 MIPS vs ~2.6 MIPS on a pathological
> one-instruction loop (~7%), while the demo workload is unchanged within
> run-to-run noise (1.53 vs 1.55 MIPS best-of). With `exec=on` the demo
> workload runs at ~0.54 MIPS (~2.4x slower); an access-only trace is
> proportionally cheaper. Use `--trace-start/--trace-end` to bound the logged
> region, and the default 1M-event ring budget to bound memory.

## 15. CI strategy

* The Python package and QEMU path remain the default, always-on suite; no
  native artifact is required for it.
* Native tests carry a `native` marker and `pytest.importorskip`-style
  detection: without the built backend they skip, never fail. A dedicated job
  builds the core (cargo/maturin), runs native + differential tests, the
  determinism double-run and profile-schema validation; nightly while M1 is
  in flight, required at M1 exit.
* Real-image fixtures are opt-in (`GREENEYE_FW`); the default job uses the
  bundled demo. Golden sessions are per backend and profile-hash checked.

## 16. Annex — Verilog soft processor (roadmap H6, far later)

**Scope (H6.1).** An RTL soft implementation of the GSP core environment:
RV64 core with TCM, MPU/PMP, vectored interrupts and timers; a device fabric
with falcon windows, CSB/TRANSCOM, DMA to host and WPR2/FWSEC state. RTL is
separate from `libgreenmachine`; it only shares profile data.

**Run modes and integration (H6.2).** Verilator for determinism and
coverage, FPGA for speed, optionally both behind one co-simulation bridge. A
`Backend` talks to the simulation's clock/step and memory API: `step` becomes
clock edges, breakpoints become traps, watchpoints become signal/page
monitors, snapshots use saved simulation state. A new `cycle` event class
makes L2 timing *measurable* — only under docs/06 caveats, and physical
measurement is still required for any silicon claim.

**Gates (H6.3).** Requires H2/H3 proven, a validated device map, and a funded
reason to care about cycle accuracy; not on the critical path and not before
the baseline project-roadmap goals. **Security posture:** a study
environment, never a confidentiality boundary, never shipped, never in a
guest; no isolation or side-channel claim may be based on it (`06 §6`).

## 17. Risks and mitigations

| Risk | Impact | Mitigation |
|---|---|---|
| Fidelity divergence from QEMU/hardware | wrong conclusions | differential suite from M1; milestones gated on canonical event agreement; declared gaps recorded per docs/06 |
| Two backends to maintain | effort/tool decay | minimal `Backend` interface (H3.4); shared `Session` primitives; QEMU stays the reference backend in CI |
| Effort creep before the gate | roadmap damage | explicit H2.2 entry gate; this document is design-only until then |
| Determinism leaks (clock, hashing, dict order, host input) | replay broken, trust lost | double-run determinism test; sequence-number tie-breaks; no host time in semantic fields |
| DSL insufficient for device semantics | profile churn, core edits | `Device` modules own whole windows; escape hatch is a module, never a core special case |
| Snapshot size/I/O blowup | snapshots unusable on real images | page-level COW + sparse encoding; region-scoped logging; moment retention policy |
| Injection misuse (real hardware/guest) | safety/security incident | local-only, dev-only tooling; no guest presence; interception only in the simulator; docs/12 policy checks |

## 18. Open questions

1. Which GC extensions must the first `rv64gc` profile cover, and can FPU
   instructions be trapped and recorded rather than executed until then?
2. Are before-access watchpoints needed for injection, and how do their
   recordings stay distinct from the QEMU-compatible profile?
3. What page size/encoding do moments need, what restore budget, and how much
   RTOS semantics must be modelled for M3 (docs/09) without a timing claim?
4. Is Rust + Python sufficient, or does the profile DSL need a native loader,
   and does an embedded public core's licence fit this repository (M4)?
5. How are multi-core interleavings represented so "same seed + same log"
   stays the only replay contract, and which events must be byte-identical
   across backends for the evidence rules?