# 51 — Host-traffic message loop: doorbell + msgq transport + replay (in development)

> **Status: in development (2026-10-06, B4/B5 transport wave).** This page
> documents `sim/host_traffic.py`, the transport half of boot-ladder rungs
> B4/B5 (docs/35-GREENEYE-SIMULATOR-DESIGN.md §3.3): a modelled host
> doorbell, a command/status message-queue pair laid out with the real queue
> arithmetic, queue-element framing with the real checksum, a stub dispatcher
> and deterministic replay of an `rm_capture` stimulus (docs/45). The **real
> 570.144 firmware is not run by this module**: there is no real message loop,
> no RM semantics and no firmware-behaviour claim.
>
> Development-only, offline: no GPU, kernel module, mdev, guest, network,
> QEMU, `/dev` access or reboot. Nothing here is a product dependency and
> nothing is injected into a guest.
>
> **See also:** `35-GREENEYE-SIMULATOR-DESIGN.md` §3.3 (ladder), `36` (device
> model; the falcon window this doorbell overlays), `45` (capture decode and
> stimulus), `49` (surface map; B4/B5 is queued build #1), `06-ACCURACY.md`
> (L0/L1 only), `open-iov/docs/eng-ops.md` §2 (admission rule).

## 1. What this is and why

The campaign keeps hitting one wall class: *what does the server do with the
traffic we send it, and what does it read/write/wait on for this control*.
The strategic tool for that class is B4/B5: replay a captured real RPC
sequence into the emulated GSP through an emulated transport and record the
replies and memory writes as evidence (handoff-simulator §27, §30, §31 item
B1; docs/49 §6 build order #1).

This wave lands the **transport and framing** so the replay plumbing exists
and is tested before the firmware loop is connected:

* a doorbell register model at the real hardware offset;
* command/status msgq rings in simulated memory, laid out with the arithmetic
  the real host queue code uses (including the `MSGQ_FLAGS_SWAP_RX`
  cross-link of the two rx headers);
* queue-element and RPC framing with the real header layouts and checksum;
* a 2-handler stub dispatcher plus `NV_ERR_NOT_SUPPORTED` for real RM
  functions, so a message can travel host → command ring → dispatch → status
  ring → host and be recorded;
* replay of a `greeneye-rm-rpc-stimulus/1` document (or a capture document
  converted by `harness/rm_capture.py`), message by message, in order.

The dispatcher is deliberately a **stub**: answers are labelled
`stub:echo`, `stub:status_query`, `stub:unsupported` in the transcript. A
stub verdict is never presented as firmware behaviour.

## 2. Admission record (draft — pending maintainer filing)

Copied from the queued execution item (private live contract §31 item B1),
expanded to the `open-iov/docs/eng-ops.md` §2 five-field format; a maintainer
files the final record. One scenario, one axis (axis 4, bug-analysis time).

```text
TOOL ADMISSION RECORD                                    id: TOOL-2026-XX (draft)
tool:           sim/host_traffic.py (new; B4/B5 transport + replay wave)
loop:           dynamic-RE (campaign traffic-interaction walls)
axis:           4 — bug-analysis time (supervised cycles avoided / TTFA)
owner/decider:  <maintainer>; executor: agent
scenario:       a campaign traffic-interaction wall (e.g. the V6 launch
                silence) asks what the server does with a given control or
                reply traffic; replay the captured stimulus into the modelled
                transport and read the server-side evidence offline.
measurable:     wall resolved from emulator evidence alone or with < 1
                supervised cycle; per wall: a session/transcript pair with
                every submitted message, dispatch handler and reply recorded.
baseline:       today = static decompiler dig + one or more supervised
                hardware cycles per hypothesis (unrecorded as a single
                number; docs/49 §4).
threshold:      keep if 2 of the next 3 wall-class instances are answered
                offline; N >= 3 wall-class instances; environment: this tool
                checkout, simulator only.
failure signal: replies diverge from captured server behaviour, or the tool
                cannot represent the control/queue shape -> record the
                fidelity gap and stop (no emulation edit from simulator-only
                evidence).
evidence:       tests/test_host_traffic.py (36 tests) + `demo`/`replay`
                session directories under the `--out` directory.
decision:       admit-pilot
review-by:      one quarter / five sessions      trials-dir: dumps/
```

Queued-item exit criterion (handoff §31 B1): *one real 512-byte RPC control
replayed end-to-end with the reply captured in a session*. This wave's demo
replays a **real-shaped** 512-byte `GSP_RM_CONTROL` (built to the documented
control shape, from no capture) end-to-end through the transport; its reply
is the stub dispatcher's framed `NV_ERR_NOT_SUPPORTED`, captured in a session.
`replay` consumes a real stimulus document the moment a supervised capture
provides one — the capture itself is still pending, so no captured-sequence
claim is made yet.

## 3. What the transport is based on (real offsets and structures)

All facts below come from public OpenRM register/queue header material (the
same class of evidence the tracked device model already uses). The capture
tool exposes the overlapping facts as constants (`RPC_HEADER_BYTES`, the
`GSP_RM_*` ids, the payload offsets), not as struct types, so this module
mirrors them and `test_rpc_constants_match_capture_tool` fails if they drift.
The models are L0/L1 framing and ordering only; no timing claim.

| Fact | Value | Use in the model |
|---|---|---|
| GSP queue-head register | `NV_PGSP_QUEUE_HEAD(i) = 0x110c00 + i*8`, 32-bit RW, i < 8 | doorbell window `PRIV+0x110c00`, 0x40 bytes; reached by the firmware through the PRIV aperture |
| Host kick | the 570.144 submit path writes the queue head after queueing (observed value 0) | `HostTraffic.submit` rings queue 0 with 0; the recorded value is the firmware-visible read-back |
| Falcon mailbox | `NV_PGSP_FALCON_MAILBOX0/1` 0x110040/0x110044 | already modelled by `sim/devices/falcon.py`; not duplicated here |
| `msgqTxHeader` | 8 x u32: version, size, msgSize, msgCount, writePtr, flags, rxHdrOff, entryOff | `MsgqTxHeader` / `MsgqQueue` |
| `msgqRxHeader` | 1 x u32: readPtr | `MsgqQueue.set_read` / `.read_ptr` |
| msgq arithmetic | `rxHdrOff = align32(16) = 32`, `entryOff = align36(16) = 48`, `msgCount = (size - entryOff) / msgSize` | `MsgqQueue.format` reproduces it exactly |
| Queue flag | `MSGQ_FLAGS_SWAP_RX = 1`; the real command queue sets it and `msgqRxLink` cross-links the rx headers between the command and status buffers | `SharedQueues` places the command rx header in the status buffer and vice versa |
| `GSP_MSG_QUEUE_ELEMENT` | 16 B authTag + 16 B aad + checksum + seqNum + elemCount + 4 B reserved + 32 B `rpc_message_header_v` (payload at 0x50) | `build_queue_element` / `parse_queue_element` |
| Element/queue sizes | element min 0x1000, max 0x10000; default command and status queue 0x40000 each; header size 0x1000; header align 4, element align 12 | defaults in the module; a message stages contiguously and splits across ring slots exactly as the real producer copies it |
| Multi-element | elemCount in the first element; the contiguous staged stream is split into 0x1000-byte slots (tail zero-padded); checksum covers the staged stream | `split_element_stream` + `SharedQueues._place`/`_take` (wrap-safe); max 16 elements |
| Checksum | XOR of 64-bit words over `0x30 + rpc.length` bytes, then hi32 ^ lo32; the stored field makes the receiver's recomputation zero | `compute_message_checksum` / `build_queue_element` (verified) |
| RPC header | 32 bytes: header_version (0x03000000), signature ("VRPC" 0x43505256), length (= 32 + payload), function, rpc_result, rpc_result_private, sequence, spare | `build_rpc_message` / `parse_rpc_message` |
| RM ids | `GSP_RM_CONTROL = 76`, `GSP_RM_ALLOC = 103`, `GSP_GET_GSP_STATIC_INFO = 65` (same as the capture tool) | replay mapping for `control`/`alloc` records |
| NV status | `NV_OK = 0`, `NV_ERR_NOT_SUPPORTED = 0x56` | stub replies |

Modelled rather than proven, and therefore labelled:

* the **shared memory address and queue sizes** are fixture defaults; the real
  driver learns them from the `GSP_ARGUMENTS_CACHED` / message-queue init
  arguments. Nothing here reads a real `MESSAGE_QUEUE_INIT_ARGUMENTS`.
* the **doorbell read-back** is storage of the last written value (the header
  says RW); the firmware's read use of this register has **not** been observed.
* the **command/status direction** is the documented one (host writes the
  command queue, GSP writes the status queue), but no real GSP has yet been
  observed consuming or producing through this model.
* the **MSI-X/event path** by which the real GSP signals a status-queue write
  is not modelled; replies are read by polling (`read_reply`).

## 4. Files and API

| File | Contents |
|---|---|
| `sim/host_traffic.py` | framing (`build_rpc_message`, `parse_rpc_message`, `build_queue_element`, `parse_queue_element`, `compute_message_checksum`), `MsgqQueue`, `SharedQueues`, `DoorbellModel`, `DoorbellBlock` (a `sim.devices` overlay), `HostTraffic`, `load_stimulus`, `replay_stimulus`, `run_demo`, CLI |
| `tests/test_host_traffic.py` | 36 offline tests: framing/checksum, queue arithmetic + cross-link, doorbell model + guest instruction access, submit/dispatch/reply, determinism, replay (stimulus + capture doc), recorder events, demo + CLI |
| `docs/51-HOST-TRAFFIC-LOOP.md` | this page |

`HostTraffic` surface:

* `submit(message)` / `submit_rpc(function, params, sequence=None)` — frame,
  write one command element, advance `writePtr`, ring the doorbell (recorded).
* `poll()` — GSP-side drain of the command ring through the handler table,
  writing framed replies to the status ring; returns the dispatched count.
  Bounded by construction (each iteration consumes one element).
* `read_reply()` — host-side consumption of one status element.
* `run_cycles(budget)` — run the attached `sim.machine.Machine` for a bounded
  budget (cap 2M) and then `poll`; without a machine it only polls.
* `transcript()` / `state()` / `close()` — deterministic transcript, queue and
  doorbell state; `close()` writes `blobs/host-traffic.json` when a recorder
  is attached.
* `doorbell_block()` — a `DoorbellBlock` sharing the same `DoorbellModel`, so
  the guest-visible register and the host model are one state. Attach it with
  `machine.devices.add(host.doorbell_block())`; the narrow window overrides
  the wider falcon window only for `0x110c00..0x110c3f`.

Records: `greeneye-host-traffic-{demo,replay,log,reply}/1`. The transcript has
no host timestamps and is byte-identical across runs; the optional
`greeneye-session/1` recorder events (`host_traffic_config`, `doorbell`,
`host_submit`, `host_dispatch`, `host_reply_read`, `host_replay_skip`,
`host_transcript`) follow the standard event shape, where `t` is host-timed
and non-semantic.

## 5. How to run it

```bash
cd open-iov/debug/risc-v-gsp-rm-vm

# Deterministic end-to-end demo (echo + status query + real-shaped 512-byte
# GSP_RM_CONTROL; stub replies), with a recorded session:
python3 -m sim.host_traffic demo --out dumps/host-traffic-demo

# Replay a stimulus document (or a capture document / raw capture log):
python3 -m sim.host_traffic replay --stimulus dumps/rm-stimulus.json --out dumps/host-traffic-replay
python3 -m sim.host_traffic replay --stimulus capture.json --allow-truncated

# Tests:
python3 -m unittest tests.test_host_traffic -v
```

`replay` maps stimulus records as follows: `send` carries the wire function
and payload directly; `control` and `alloc` records carry a named id and are
mapped with the capture tool's constants (76 / 103) using the captured
parameter block as the payload. That identity is an **assumption** until a
capture with full send-side payloads exists (the current runbook helper also
truncates payloads above 192 bytes; truncated messages are refused by default
and skipped with `--allow-truncated`).

## 6. Real vs stubbed — the honest table

| Piece | State |
|---|---|
| RPC header + queue element framing, checksum | **real** (mirrored; drift-tested against `harness/rm_capture.py` constants) |
| msgq header arithmetic + `SWAP_RX` cross-link | **real** (mirrored from the public queue header and `msgqTxCreate`/`msgqRxLink`) |
| Doorbell register offset/width/access | **real** (public register header); read-back is storage of the last write, firmware read use unobserved |
| Guest access to the doorbell window through `Machine` | **real** (device window + recorded store PC); exercised by instruction-level tests |
| Command/status rings in simulated memory | **real layout**, fixture addresses and sizes |
| Multi-element messages | **real split** (contiguous staged stream across ring slots, wrap-safe, checksum over the stream) |
| Dispatcher (`echo`, `status_query`) | **stub** (ids 0x40/0x41 are not RM functions) |
| Reply to real RM functions | **stub** `NV_ERR_NOT_SUPPORTED` — framed and transported exactly; not firmware behaviour |
| Real firmware message loop | **not run** (B5 remains open) |
| RM semantics (controls, allocs, objects, events) | **absent** |
| MSI-X / GSP→host interrupt | **not modelled** (polling) |
| Timing/cycles | **not claimed** (L2 absent; `run_cycles` is an instruction budget) |

## 7. What this does NOT do yet (and the next step to B5)

Not done: booting the real image and serving its message-loop poll; reading
the real `GSP_ARGUMENTS_CACHED`/queue-init parameters; event/MSI-X delivery;
any RM handler.

Next step for B5, concretely (each point is bounded and session-recorded):

1. Boot the real 570.144 image to B3 (`sim.gsp_boot`, 474k instructions,
   stop at the message-loop port wait, `ecall` at `0x4018038`).
2. Attach `host.doorbell_block()` to the boot registry before the run, so the
   firmware's first queue-head access is served by the model and recorded,
   and place the command/status rings at addresses the firmware is told
   (stub: the fixture block; real: the addresses from a captured
   `GSP_ARGUMENTS_CACHED`).
3. Ring the doorbell from the host model and continue the machine with
   `run_cycles` until it polls the rings; classify the first poll: which
   address does it read (queue header? Tx/Rx header? entry?) and what
   read-back semantics does it need.
4. Compare the firmware's observed ring arithmetic against this module's
   (field-by-field); where the real init used `MSGQ_FLAGS_SWAP_RX`, the
   cross-link must already match; fix only with a recorded session pair
   (docs/06 accuracy loop), never from a decompiler reading alone.
5. Then replay one captured stimulus message with the real image and record
   the reply/writes as the B5 session. Until a supervised capture exists,
   this stops at step 3–4 with the stub dispatcher.

The seam for step 3 already exists: `HostTraffic(machine=..., shared_base=...)`
uses the machine's RAM for the rings and `run_cycles` steps the core; the
doorbell block overlays the falcon window without touching
`sim/devices/falcon.py` or the profile.

## 8. Limits, deferrals and verification

Limits: L0/L1 only; no capture has been consumed yet (no supervised runbook
execution); control/alloc payload identity is an assumption; truncated
payloads are refused or skipped, never padded; a stub reply is never a
firmware claim; no real queue-init parameters.

Deferred: real-firmware B4/B5 wiring (above), event/MSI-X delivery, RM-object
state seeding (docs/49 build #2), golden matrices (#3).

Verification of this wave: `python3 -m unittest tests.test_host_traffic -v`
→ 36 tests green; full suite `make test` green; `demo` `ok: true` with all
checks (`echo` payload round-trip, status magic, 512-byte control replay,
framed stub verdict, one doorbell ring per submit, all replies read);
`ruff check` clean on the new files. Two-run determinism is asserted by
tests (transcript and replies byte-equal).