# 37 — Simulator vs the QEMU line: evaluation harness + real RE proof

> **Status: in development (2026-10-03).** This document is the S4 (value
> validation) record for the standalone GSP simulator workstream. It defines
> the evaluation harness, freezes the thresholds used, reports the first
> measured A/B run on the real TU104 570.144 image, and records the U51
> real-reverse-engineering demonstration. Nothing here is a product claim:
> simulator evidence is L0/L1 (docs/06).
>
> **Update (2026-10-03, later):** S1 (device model + profiles + boot shim)
> landed and the harness was wired to it; §4.3 records the re-measured A/B
> where the native side is the S1 `sim.gsp_boot` path (verdict `above` on the
> frozen checks). §4.1/§4.2 remain the pre-S1 measured record, and §7's
> "S1 not landed" limit applies only to those sections.
>
> **Artifacts:** `harness/sim_qemu_line.py` (A/B harness),
> `tests/test_sim_qemu_line.py` (shape + determinism + QEMU regression),
> `harness/re_demo.py` + `harness/re-demo-report.md` (real RE proof).
> **Line definition source:** docs/35-GREENEYE-SIMULATOR-DESIGN.md §1.1 and
> the boot ladder in §3.3.

---

## 1. The line being evaluated

The QEMU line (docs/35 §1.1) is the v0.1/v0.2 tool behaviour on the real
image: hooks/watchpoints, memory/struct/snapshot tooling, sessions, symbols
and analyze/compare, probes/plugins, Ghidra bridges, trace, injection SDK and
`Session.call`/`run_until`, plus the documented first fault at firmware vaddr
`0x4009014` (4-byte read of `0x2000000000000A00`).

The boot ladder (docs/35 §3.3) is `pre-B0` (image loads, first MMIO observed
*as a fault*) < `B0` (first MMIO observed; lenient) < `B1` (first fault past
the QEMU point, explained by device semantics) < `B2` (boot-stage progress) <
`B3` (first mailbox poll) < `B4` (message loop) < `B5` (one scripted message
served).

**"Above the QEMU line"** requires *all* of the frozen checks in §3,
including the docs/35 §1.1 requirement that the milestone be reached "with
the device model and shims documented". Lenient MMIO is an observation
mechanism, not a device model, so it cannot satisfy the contract by itself.

## 2. Method

Everything is stdlib-only, bounded (`<= 2,000,000` native instructions,
`<= 10 s` QEMU), and touches no hardware, GPU, mdev, module, guest or network.

```bash
cd open-iov/debug/risc-v-gsp-rm-vm
python3 harness/sim_qemu_line.py            # A/B harness -> harness/qemu_line/
python3 -m unittest tests.test_sim_qemu_line -v
python3 harness/re_demo.py                  # U51 proof -> harness/re-demo-report.md
```

* **QEMU side** — `harness.common.open_flat_session` (the same relocation and
  `guest_errors,unimp,int` logging as `greeneye probe`), bounded run, first
  exception parsed from the bounded head of the log (trap loops flood it),
  first-touch device map harvested, session recorded with the log blob.
* **Native side** — a discovered S1 profile runs `sim.gsp_boot` (device model
  + recorded boot shim) twice and reports the observed rung; without one, the
  lenient `sim.gsp_profile.run_profile` baseline (deterministic RV64IMAC,
  modeled cycles) runs twice with `greeneye.device_map` for MMIO. S1's
  profile is discovered defensively (`sim.devices` Python entry points, then
  `sim/profiles/*.json` matched by firmware sha / `sim/devices/*.json`); when
  absent the run is the lenient `B0` baseline and the report says so.
* **Feature matrix** — exercised on the bundled demo firmware
  (`firmware/uproc_demo`): hooks, breakpoints, watchpoints, memory, struct
  decode, sessions, analyze/compare, exec trace, injection SDK. MCP is
  recorded `not_run` here to keep this harness independent of the MCP surface
  (the native MCP tools are tested in `tests/test_mcp.py`; docs/20).

## 3. Frozen thresholds (2026-10-03, before the first measurement)

Stored in `harness/sim_qemu_line.py:FROZEN_THRESHOLDS` and
`harness/re_demo.py:FROZEN_THRESHOLDS`; they are never moved silently.

| check | threshold | why |
|---|---|---|
| `qemu_first_fault_reproduced` | QEMU reproduces fw `0x4009014` / `0x2000000000000A00` | otherwise the two sides are not comparable |
| `native_above_qemu` | native rung >= QEMU rung + 1 | the line's rung rule |
| `identical_runs` | 2 identical native runs (key fields) | determinism is a docs/35 §1.1 requirement |
| `native_pair_wall_s` | <= 20 s for the two native runs | keeps the harness usable |
| `device_model_documented` | a discovered S1 device map/profile is used | docs/35 §1.1 "with the device model and shims documented" |

`re_demo.py` additionally freezes: emulator wall <= 120 s, 5 answers present,
2 identical lenient runs, milestone past the QEMU point; the manual static
baseline is a documented **floor estimate** of 600 s (method note in the
script: ~90 instructions across three sections read by hand and correlated
with the QEMU fault report; not measured).

## 4. Measured results (2026-10-03)

### 4.1 A/B harness on the real image

Firmware `gsp-570.144.fwimage.bin`, sha256 `2d1bb5c60b208cfd…`; S1 absent at
run time (`sim/devices/` and `sim/profiles/` not present), so the native side
ran lenient.

| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence |
|---|---|---|---:|---:|---|---:|---|
| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.95 | `harness/qemu_line/sessions/qemu-probe` |
| native run 1 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.21 | `harness/qemu_line/sessions/native-1` |
| native run 2 | yes | `B0` | 200,000 | 200,012 | `budget@0x4009070` | 0.22 | `harness/qemu_line/sessions/native-2` |

Wall figures are from one measured run (host-load dependent); re-run the
command for current numbers. Session hashes for the record are in
`harness/qemu_line/sim-qemu-line.json`.

* first-touch match with the documented reference: **true** (same address,
  same 4-byte read, same fw PC);
* identical across the two runs: **true**; session compare ran (breakpoint,
  watchpoint and written-page deltas);
* verdict: **`above_lenient_only`** — rung progress exists (`B0` is one past
  `pre-B0`) but the device-model check fails, so `above_qemu_line = false`.
  This is the honest underclaim: the native run observes and continues past
  the QEMU fault, but lenient MMIO is not device semantics.
* native feature matrix (demo firmware): `hooks`, `breakpoints`, `watch`,
  `memory`, `struct`, `sessions`, `analyze_compare`, `trace`, `injection` all
  **pass**; `mcp` **not_run** (S3-owned).
* session hashes for every side/session are in
  `harness/qemu_line/sim-qemu-line.json` and rendered in `.md`.

Expected delta when S1 landed (realised; see §4.3): `device_model_used = true`,
rung `B2` because the model serves the QEMU first touch and the shim reaches
the relocation stage, and `verdict = above` because the run stays deterministic
and within the wall bound. S1's profile is picked up automatically from
`sim/profiles/*.json` when its `firmware.sha256` matches (or pass
`--device-map FILE` for an observational map).

### 4.2 Real RE proof (U51)

Scenario: *what happens on the real GSP image between entry and the first
fault/milestone, and what does the state tell us about boot?*

| side | method | wall s | answers |
|---|---|---:|---|
| baseline | QEMU probe (measured, 0.5 s bound) | 0.99 | first fault only |
| baseline | manual static reading (floor estimate) | 600 | entry->fault path by hand |
| baseline | **total** | **600.99** | full question |
| emulator | 3 recorded native sessions (`harness/re_demo.py`) | **0.13** | full question + evidence |

Frozen threshold: emulator wall <= 120 s, 5/5 answers present, two identical
lenient runs, milestone past the QEMU point. Result: **PASS** (all checks
true). Session hashes and the JSON answers are in `harness/re_demo/result.json`
and `harness/re-demo-report.md`.

Answer (evidence-backed; simulator L0/L1, no silicon claim):

1. **First touch:** a 4-byte read of PA `0x2000000000000A00` at fw PC
   `0x4009014`; it is exactly the QEMU-documented first fault.
2. **Path:** the entry state is all-zero (no bootloader handoff is modelled);
   `.bootstrap` materialises the device PA with five arithmetic instructions,
   reads it, then runs a chip-ID comparison chain — 25 executed instructions
   from entry to the halt (exec trace).
3. **Field:** the read value's bits `[28:20]` are extracted (`srli 20`,
   `andi 0x1ff`) and compared against `354..360` (Turing IDs) and `368`
   (Ampere ID).
4. **Dispatch:** `354..360` jumps to `.tu10x_text` at `0x4007000`; `368`
   jumps to `.ga100_text` at `0x4005000`; any other value (including the
   lenient zero) falls into the fail-closed `j .` loop at `0x4009070`.
5. **Next blocker (B1 candidate):** with a Turing ID served from a one-entry
   device map, `.tu10x_text` programs CSRs `0x7c8–0x7cc` (region
   descriptors), enables `mstatus` FP/SIMD state, builds the **absolute**
   link-time address `0x40070f8` and jumps to it. Because the loader
   relocates segments but not absolute immediates, that jump lands outside
   RAM and faults (first fault past the QEMU point, 74 instructions). The
   next rung therefore needs a chip-ID device model **plus** load-at-link
   address (or alias/translation of absolute immediates); the modeled ID
   value itself is an assumption (the QEMU baseline faults before reading
   it), while the field position and the compared IDs are firmware evidence.

### 4.3 Re-measurement after S1 landed (2026-10-03, later)

The harness now discovers the S1 `greeneye-sim-profile/1` document
(`sim/profiles/*.json`, matched by `firmware.sha256`) and runs the native side
through `sim.gsp_boot` (device model + recorded boot shim) instead of lenient
MMIO. Frozen thresholds unchanged. Measured run on the same image
(sha256 `2d1bb5c60b208cfd…`):

| side | ran | rung | instructions | cycles (modeled) | stop | wall s | evidence |
|---|---|---|---:|---:|---|---:|---|
| QEMU probe | yes | `pre-B0` | – | – | `fault_load` fw `0x4009014` -> `0x2000000000000A00` | 0.98 | `harness/qemu_line/sessions/qemu-probe` |
| native run 1 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` (DMEM runspace unknown) | 0.16 | `harness/qemu_line/sessions/native-1` |
| native run 2 | yes | `B2` | 14,682 | 27,294 | `fault@0x400EA0C` | 0.17 | `harness/qemu_line/sessions/native-2` |

* `device_model_used = true`, `device_model_documented = true`,
  `identical_runs = true` (device-state sha256
  `3fa264b9f81081e8…` on both runs), `native_pair_wall_s = 0.33`,
  `native_above_qemu = true`; **verdict `above`** on all five frozen checks.
* milestones observed: `bootargs_consumer`, `relocation_copy`;
  `unknown MMIO = 0`, `device_unhandled = 0`.
* What this does **not** claim: B3/B4 are not reached (the run stops at the
  documented DMEM-runspace unknown); the shim values are declared assumptions;
  cycles are modeled (docs/32); no timing/device-environment claim.
* Command: `python3 harness/sim_qemu_line.py` (regenerates
  `harness/qemu_line/sim-qemu-line.json|.md`). Wall figures are host-load
  dependent.

## 5. Failure signals (when to disable, per U48/U56)

* Any frozen check false **with QEMU available and the firmware present**:
  keep `above_qemu_line = false`, do not claim the line.
* `device_model_documented` still false at the Phase S pause review: the S1
  work package has not met its acceptance; the harness keeps reporting
  `above_lenient_only`.
* A feature matrix entry `fail`: disable that feature flag and record it as a
  failed experiment; an entry silently stuck at `not_run` blocks the S4 exit.
* `re_demo.py` pass false: U51 is not satisfied; fix the tool chain before
  any Phase G design work.
* Thresholds change only with a dated entry in this document; a moved
  threshold without a measurement is a review failure.

## 6. Re-running / reproducing

```bash
cd open-iov/debug/risc-v-gsp-rm-vm
python3 harness/sim_qemu_line.py                 # full A/B + feature matrix
python3 harness/sim_qemu_line.py --no-qemu       # native only, no baseline claim
python3 -m unittest tests.test_sim_qemu_line -v  # shape/determinism/QEMU gate
python3 harness/re_demo.py                       # real RE proof
```

Without the 570.144 image the commands skip cleanly (exit 2 for the CLIs,
skipped tests), because the tool tree is used on machines without the
firmware. The QEMU side also needs `qemu-system-riscv64`; without it the
report is emitted with `unverified_qemu_absent`.

## 7. Honest limits

* **S1 partial** — the device model reaches B2 (relocation copy) with the
  recorded boot shim; B3/B4 (CSB poll, message loop) are not reached and the
  DMEM-runspace unknown is the documented ceiling (docs/36 §6). The §4.1
  lenient record predates S1; §4.3 is the post-S1 measurement.
* **L2/L3 partial** — cycles are modeled (`rv64imac-modeled/1`, docs/32), not
  measured; the device model is partial and carries per-register evidence
  (docs/36). No timing or full device-environment claim is made.
* **MCP native surface not exercised here** — the A/B feature matrix records
  it `not_run`; the native MCP tools have their own suite (docs/20 §9).
* **Modeled chip-ID value is an assumption** — only the field extraction and
  the compared ID set come from the image; in the S1 profile the `0x164`
  value is declared by the profile, and the boot shim values are declared
  assumptions recorded in the session.
* **Relocation artifact** — the B1-candidate fault at `0x40070f8` is a
  property of the bias-relocated harness, not of silicon; it is labelled as
  such and not counted as a device milestone. (The S1 path loads at the link
  addresses and does not use that relocation.)
* **Manual baseline is an estimate** — labelled a floor with its method; the
  measured parts of the baseline (QEMU probe) are reproducible with the
  harness.