docs: record the first real GSP boot and the live GSP-RM session

STATE-CURRENT and the architecture docs now describe the observed order (FWSEC -> Booter Load -> RISC-V active -> event 0x1002 run-sequencer -> 0x1001 init-done), the four-handoff-bit clean-release constraint, and the --boot/--keep harness contract.

3 files changed, 123 insertions, 30 deletions

diff --git a/STATE-CURRENT.md b/STATE-CURRENT.md
index 6771f53..c86311b 100644
--- a/STATE-CURRENT.md
+++ b/STATE-CURRENT.md
@@ -1,6 +1,6 @@
 # Open-IOV Current State
 
-> **Snapshot: 2026-10-01 09:12.** This is a dated public status page, not live
+> **Snapshot: 2026-10-01 09:59.** This is a dated public status page, not live
 > state; it is refreshed on every progress check. Live working notes are
 > maintained outside this repository; the public known-knowns/unknowns document
 > is authoritative here:
@@ -31,7 +31,7 @@ meant to execute on the physical GPU.
 | `cuCtxCreate` returns | ✅ (the whole smoke test now completes) |
 | `cuMemAlloc` + `cuMemcpy` roundtrip (`ALL OK` in the smoke test) | ✅ payload roundtrip verified 64/64 in the guest |
 | Full `nvidia-smi` output / kernel launch | 🟡 the static runtime now initializes end-to-end with **real releases** (`cudaGetDeviceCount`/`cudaSetDevice`/3×`cudaMalloc`/2×`cudaMemcpyHtoD`/`kernel launch`/`cudaGetLastError` all `rc=0`) and queues a kernel launch; execution awaits the backend |
-| CUDA kernel execution (`vectorAdd`) | 🟡 execution backend: physical ownership + firmware staging verified; the unbind/rebind freeze hazard is mitigated by the supervised D0 power-domain lock (3/3 green claim cycles), and the real FB size (8 GiB) and boot layout are verified live. The supervised live secure-verification run raised the protected region (WPR2 up) for the first time, then proved a secure-verification-only carve is not host-recoverable on this fused part, so the fail-safe held the device and an approved reboot restored it (distribution driver rebound, GPU healthy). That one-way stage is now retired: the only device-write path is a single full boot lifecycle (secure verification -> bootloader -> RISC-V start -> resource-manager initialization), which is being implemented. |
+| CUDA kernel execution (`vectorAdd`) | 🟡 execution backend: physical ownership + firmware staging verified; the unbind/rebind freeze hazard is mitigated by the supervised D0 power-domain lock (3/3 green claim cycles), and the real FB size (8 GiB) and boot layout are verified live. The supervised live secure-verification run raised the protected region (WPR2 up) for the first time, then proved a secure-verification-only carve is not host-recoverable on this fused part, so the fail-safe held the device and an approved reboot restored it (distribution driver rebound, GPU healthy). That one-way stage is now retired: the only device-write path is a single full boot lifecycle (secure verification -> bootloader -> RISC-V start -> resource-manager initialization), whose code is implemented in the module sources and has now run live: the first real boot raised WPR2, passed Booter Load and went RISC-V active, and the module received the first real GSP-RM message (event 0x1002); init-done (0x1001) and a clean release are the next steps (see the 2026-10-01 section). |
 | Graphics (3D), video | ❌ |
 | Faults, multi-version, Windows, migration, multi-tenancy, performance | ❌ |
 
@@ -58,8 +58,10 @@ chain succeeds.
   host healthy, no crash record). The one-way stage is now retired: the only
   device-write path is a single supervised full boot lifecycle (secure
   verification -> bootloader -> RISC-V start -> resource-manager
-  initialization), whose next stage is being implemented so the normal boot
-  lifecycle makes teardown valid.
+  initialization). That lifecycle is now implemented in the module sources
+  (bootloader runner + generated verification/boot image data) and has run
+  live under the supervised harness: the first real boot succeeded (see the
+  section below).
 - **The full CUDA driver-API smoke test passes with correct data**: the client
   completes `cuCtxCreate -> cuMemAlloc -> cuMemcpyHtoD -> cuMemcpyDtoH ->
   cuMemFree -> cuCtxDestroy` and prints `ALL OK`, with the 64-byte payload
@@ -153,7 +155,12 @@ chain succeeds.
   (distribution driver unbound, power-domain locks held) rather than hand it
   back; a rescue path is being debugged (or, failing that, an approved
   reboot). A distribution-driver bus-master quirk on the claim path was also
-  found and fixed in this run. The booter and RISC-V start follow.
+  found and fixed in this run. The bootloader/RISC-V/resource-manager-init
+  stages are now implemented in the module sources (bootloader runner plus
+  generated boot and verification image data) and have now run live: the
+  boot completed through an active RISC-V core and the module received the
+  first real GSP-RM message (see the section below); a clean release is
+  pending the server shutdown RPC.
 - Host stability: the host hang class observed on 2026-09-28 (kernel stack
   overflow in a channel snapshot path) is fixed and frame-audited; repeated
   VM load/attach/teardown cycles run without panics or pstore records. A
@@ -180,6 +187,46 @@ chain succeeds.
   work does not touch the driver binding and is unaffected. See
   [`docs/03-architecture/08-physical-gpu-integration.md`](docs/03-architecture/08-physical-gpu-integration.md).
 
+## First real GSP boot + live GSP-RM session (2026-10-01)
+
+Cycle 09:53:27–09:54:59 on the supervised D0 harness (`--boot --keep`),
+`open-iov-vfio.ko` sha256 `7815dacc…`, srcversion `2993AF5C…`. The boot order
+observed on the physical TU104:
+
+1. **FWSEC** — WPR2 came up; FRTS error code 0; `WPR2_LO=0x01ffe000`,
+   `WPR2_HI=0x01ffee00`.
+2. **Booter Load** on SEC2 — halted with `M0=0x0` after 146 ms.
+3. **RISC-V active** — `RISCV_STATUS=0x00000035` (active bit 0),
+   `FALCON_OS=0`; secure scratch14 raw `0x11000000` (booter-load handoff bit
+   24 = 1, booter binary version 1).
+4. **First real GSP-RM message** — the staged status queue linked and the
+   server's first event was `function=0x1002`
+   (`NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER`, `rpc_result=0`), carrying a
+   command buffer the host must execute; `GSP_INIT_DONE` (0x1001) follows only
+   after that sequencer runs and was not observed in the 60 s poll.
+
+This is the first host↔real-GSP-RM exchange on the physical path. A clean
+release is not yet possible: the Booter Unload gate requires all four secure
+scratch14 handoff bits (24–27); a host-driven boot sets only bit 24, and bit
+27 is set by the server's shutdown path (host RPC `UNLOADING_GUEST_DRIVER`,
+not yet implemented). The host-side seed/unlock alternative is fuse-locked on
+this part. The session is therefore intentionally left claimed (`--keep`): the
+GPU is bound to `open-iov-gsp-phys` and runtime-active; `nvidia` stays unbound
+(host headless; `nvidia-smi` fails by design); host healthy, pstore empty. A
+fresh live cycle needs a power cycle.
+
+**Next steps:** (a) execute the 0x1002 sequencer command buffer in the module
+and log 0x1001 as `stage 3e complete`; (b) implement the command-queue TX path
+and send `UNLOADING_GUEST_DRIVER` on release so teardown passes the handoff
+gate and the GPU can be handed back without a reboot; (c) reboot (with
+approval, handoff first) when ready to test the next live cycle.
+
+Ending the claimed session (harness notice): `sudo rmmod open_iov_vfio`, then
+`echo auto | sudo tee /sys/bus/pci/devices/0000:01:00.0/power/control` and the
+same for `0000:00:01.0`, then `sudo systemctl start nvidia-persistenced`.
+Because the shutdown RPC is missing, a plain release cannot pass the handoff
+gate, so the reliable route back to the distribution driver is a power cycle.
+
 ## Initialization, work and teardown path
 
 Status tags: `[done]`, `[frontier]` (actively worked), `[partial]`,
diff --git a/docs/03-architecture/03-gsp-and-gsp-rm.md b/docs/03-architecture/03-gsp-and-gsp-rm.md
index fe51838..7f206ba 100644
--- a/docs/03-architecture/03-gsp-and-gsp-rm.md
+++ b/docs/03-architecture/03-gsp-and-gsp-rm.md
@@ -54,6 +54,16 @@ Our emulation implements stage completion rather than the stages themselves —
 which is legitimate because the *only observable* to the driver is the state
 machine it polls.
 
+On the physical path (2026-10-01), the same handshake ran against real
+firmware: FWSEC raised WPR2 with a zero FRTS error code, the SEC2 booter halted
+with mailbox0 = 0, the RISC-V core reported active (`RISCV_STATUS=0x00000035`,
+active bit 0) and `FALCON_OS` read 0. The server's first message was then a
+status-queue event, function `0x1002` (`NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER`,
+`rpc_result=0`), carrying a command buffer the host must execute;
+`GSP_INIT_DONE` (`0x1001`) follows only after that sequencer runs. The
+host-side queue pair behind the exchange is one shared page table plus a 256 KB
+command queue and a 256 KB status queue of 4 KB elements.
+
 ```mermaid
 sequenceDiagram
   autonumber
diff --git a/docs/03-architecture/08-physical-gpu-integration.md b/docs/03-architecture/08-physical-gpu-integration.md
index 5c9e385..0c18e84 100644
--- a/docs/03-architecture/08-physical-gpu-integration.md
+++ b/docs/03-architecture/08-physical-gpu-integration.md
@@ -1,15 +1,16 @@
 # Physical GPU Integration: How Work Reaches Real Hardware
 
 > **Status: mechanism chosen; ownership + firmware staging implemented;
-> power-domain lock implemented, pending supervised verification; real GSP
-> boot not started.** The control plane works today: the guest's driver
-> initializes and the CUDA driver-API baseline passes end to end (context
-> creation, allocation, copies, 64/64 payload round-trip), and the guest's
-> decoded work is executed in software as a development bridge. This document
-> records the interfaces that actually exist, the mechanism options, the
-> chosen path, the platform power-domain hazard the ownership path exposes on
-> the development host (§4.2), and what has to be built. It fills the gap
-> left open by
+> power-domain lock verified; first real GSP boot and first live GSP-RM
+> exchange complete (2026-10-01); host-side initialization follow-up and a
+> clean release are in progress.** The control plane works today: the guest's
+> driver initializes and the CUDA driver-API baseline passes end to end
+> (context creation, allocation, copies, 64/64 payload round-trip), and the
+> guest's decoded work is executed in software as a development bridge. This
+> document records the interfaces that actually exist, the mechanism options,
+> the chosen path, the platform power-domain hazard the ownership path
+> exposes on the development host (§4.2), and what has to be built. It fills
+> the gap left open by
 > [`06-host-module-and-hybrid-model.md`](06-host-module-and-hybrid-model.md) §5
 > and [`../04-implementation/02-practical-challenges-and-adversarial-interop.md`](../04-implementation/02-practical-challenges-and-adversarial-interop.md) §3,
 > which named "the host driver's RM API" as an option without a concrete call
@@ -150,12 +151,9 @@ that boot: an out-of-band platform power cut bypasses the lock and would
 produce the identical signature, and retrying would be a second dice roll on a
 host that has already lost the board.
 
-The lock is implemented in the ownership path and awaits supervised
-verification (§6, §8); a claim is refused unless both the claim parameter and
-the hazard acknowledgement are set (§8). The exact resource names, controller
-commands, captured timelines and the mitigation's code-level design live in
-the adjacent local analysis notes (`kernel/LOCAL-GSP-BOOT.md`); the
-adjacent document is part of the local analysis.
+The lock is implemented in the ownership path and verified across three
+supervised claim/release cycles on 2026-10-01 (§6, §8); a claim is refused
+unless both the claim parameter and the hazard acknowledgement are set (§8).
 
 ### 4.3 Device bring-up
 
@@ -189,8 +187,28 @@ The bring-up is staged so each step is reversible and independently verified:
 
 Stages 1 and 2 are implemented in the module and verified live (a claim/release
 cycle leaves the host driver working on the clean cycles; the staged image
-parses as expected). The power-domain lock of §4.2 guards the claim window;
-stage 3 — the real boot — has not been started.
+parses as expected). The power-domain lock of §4.2 guards the claim window and
+is verified across three supervised cycles. Stage 3 has now run live
+(2026-10-01), in the observed order **FWSEC → Booter Load → RISC-V active →
+first server event**: FWSEC raised WPR2 with a zero FRTS error code
+(`WPR2_LO=0x01ffe000`, `WPR2_HI=0x01ffee00`), the SEC2 Booter Load halted with
+`M0=0` after 146 ms, the RISC-V core reported active
+(`RISCV_STATUS=0x00000035`) with `FALCON_OS=0`, and a host-driven boot sets
+only the booter-load handoff bit (bit 24) in secure scratch14. Stage 4 has
+begun: the staged command/status queues linked and the first real GSP-RM
+message arrived (event 0x1002, run-CPU-sequencer, `rpc_result=0`). The server
+waits for the host to execute the sequencer command buffer carried in that
+event before it raises `GSP_INIT_DONE` (0x1001). **A plain release is not yet
+possible:** the Booter Unload gate requires all four secure scratch14 handoff
+bits (24–27); a
+host-driven boot sets only bit 24, and bit 27 is set by the server's shutdown
+path (host RPC `UNLOADING_GUEST_DRIVER`), which is not implemented, while the
+host-side seed/unlock alternative is fuse-locked on this part. Until that RPC
+lands, a green session is left claimed with the harness `--keep` flag and
+returning the GPU to the distribution driver takes a power cycle. The
+supervised harness modes are `--boot` (the only write mode: full boot
+lifecycle), `--boot-dryrun` (zero-write staging rehearsal) and `--keep`
+(leave a verified claimed session running); `--fwsec` is retired.
 
 ### 4.4 Host-side resource-manager client
 
@@ -245,9 +263,9 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
 | Driver-API baseline (init, devices, context, allocation, copies, free, destroy) | done — re-verified 64/64 payload round-trip on the current build |
 | Software-executed bridge for decoded memory/state operations | in progress (memsets, copies, releases) |
 | Device ownership + firmware staging | verified — claim/release and staged-image parse both verified live; the claim path is hazard-gated (§4.2) |
-| Power-domain lock across the claim window | implemented — pending supervised verification (§4.2, §8) |
-| Real GSP boot by the module | not started |
-| Host-side resource-manager client for real objects/channels | not started |
+| Power-domain lock across the claim window | verified — 3/3 supervised claim/release cycles (2026-10-01; §4.2, §8) |
+| Real GSP boot by the module | first live boot 2026-10-01 — FWSEC WPR2 up, Booter Load `M0=0`, RISC-V active; clean release pending the server shutdown RPC |
+| Host-side resource-manager client for real objects/channels | transport alive 2026-10-01 — first real GSP-RM event (0x1002) received; init-done (0x1001) awaits host sequencer execution |
 | Work translation and real submission | not started |
 | Completion relay through real hardware | not started |
 | CUDA kernel execution (`vectorAdd`) | not started |
@@ -265,11 +283,11 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
   controller action that does not go through the root port's D3 path — would
   produce the identical power-loss signature; the claim must abort on that
   signature and must not be retried in the same boot.
-* **Supervised verification of the lock:** the lock is implemented but not yet
-  empirically verified. The first claim/release with it active must be
-  supervised: explicit human go-ahead, other host work warned, a written
-  recovery plan in place, a held-open window long enough to prove the board
-  stayed powered, and pstore armed.
+* **Supervised operation of the lock:** the lock is implemented and verified
+  across three supervised claim/release cycles (2026-10-01). Every claim
+  remains supervised: explicit human go-ahead, other host work warned, a
+  written recovery plan in place, a held-open window long enough to prove the
+  board stayed powered, and pstore armed.
 
 ## 8. Decision record
 
@@ -289,6 +307,24 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
   plan. The claim path refuses without the acknowledgement. At the first
   power-loss signature: abort and do not retry in that boot; a host reboot
   requires explicit human approval and a written recovery plan updated first.
+* **Update (2026-10-01) — FWSEC-only retired:** stopping after the
+  secure-verification carve alone is a one-way door on this fused part (the
+  protected region is left up and is not host-recoverable). The supervised
+  claim harness therefore refuses `--fwsec` before any module load. The only
+  device-write mode is `--boot` (`gsp_phys_boot=1`, the full boot lifecycle
+  through resource-manager init); the zero-write rehearsal is `--boot-dryrun`.
+  Never stop after FWSEC alone on this host.
+* **Update (2026-10-01) — first live boot; clean release gated:** the
+  supervised `--boot --keep` cycle completed the first real boot (FWSEC →
+  Booter Load → RISC-V active) and received the first real GSP-RM event
+  (0x1002, run-CPU-sequencer); `GSP_INIT_DONE` (0x1001) awaits the host
+  executing the sequencer buffer carried in that event. Clean release is not
+  yet possible: the Booter Unload gate requires all four secure scratch14
+  handoff bits (24–27), a host-driven boot sets only bit 24, bit 27 is set by
+  the server's shutdown path (host RPC `UNLOADING_GUEST_DRIVER`, not
+  implemented), and host-side seed/unlock is fuse-locked on this part. The
+  live session is therefore left claimed (`--keep`) and a fresh cycle needs a
+  power cycle. See §4.3.
 
 ## 9. Interaction sequence (end state)