STATE-CURRENT: cycles 2-3 results - clean release without reboot; RM init failure 0x59 under diagnosis

1 files changed, 54 insertions, 40 deletions

diff --git a/STATE-CURRENT.md b/STATE-CURRENT.md
index 85fb904..156f35a 100644
--- a/STATE-CURRENT.md
+++ b/STATE-CURRENT.md
@@ -1,6 +1,6 @@
 # Open-IOV Current State
 
-> **Snapshot: 2026-10-01 11:30.** This is a dated public status page, not live
+> **Snapshot: 2026-10-01 12:00.** This is a dated public status page, not live
 > state; it is refreshed on every progress check. Live working notes are
 > maintained outside this repository; the public known-knowns/unknowns document
 > is authoritative here:
@@ -64,9 +64,21 @@ chain succeeds.
   section below), and a second live cycle on the rung-4 build repeated the
   boot and reached the server's run-sequencer request; the executor's size
   cap rejected the real request value (16354), which was diagnosed from a
-  raw host-memory read of the message, fixed, and rebuilt; the claimed
-  session from that cycle is awaiting a supervised power cycle to run the
-  corrected boot+shutdown flow.
+  raw host-memory read of the message, fixed, and rebuilt. The post-reboot
+  recovery is now verified and the zero-timeout REG_POLL default fix
+  (commit `98b861d`, build `87309e3a…`) was confirmed live. Cycles 2-3
+  (2026-10-01 11:40 and 11:46) then reached **stage 3e on real hardware**:
+  the real GSP-RM executed the full 165-command/490-dword sequencer and
+  reported RM init failure `0x59` (`NV_ERR_OPERATING_SYSTEM`) on both
+  firmware pairings (the default 570.211.01 image and a verified matching
+  570.144 image - the image-pairing hypothesis is disproven). Each failed
+  cycle now **releases cleanly back to the distribution driver without a
+  reboot**: the server's own shutdown sets the remaining secure-scratch
+  handoff bit, so all four handoff bits are set and a single `rmmod` clears
+  the protected region, rebinds the distribution driver and releases the
+  power-domain locks. The next evidence is the **GSP-RM NOCAT journal** (the
+  matching-image cycle streamed ~851 journal records), whose capture is
+  being built.
 - **The full CUDA driver-API smoke test passes with correct data**: the client
   completes `cuCtxCreate -> cuMemAlloc -> cuMemcpyHtoD -> cuMemcpyDtoH ->
   cuMemFree -> cuCtxDestroy` and prints `ALL OK`, with the 64-byte payload
@@ -210,47 +222,49 @@ observed on the physical TU104:
    command buffer the host must execute; `GSP_INIT_DONE` (0x1001) follows only
    after that sequencer runs and was not observed in the 60 s poll.
 
-This is the first host↔real-GSP-RM exchange on the physical path. A clean
-release is not yet possible: the Booter Unload gate requires all four secure
-scratch14 handoff bits (24–27); a host-driven boot sets only bit 24, and bit
-27 is set by the server's shutdown path (host RPC `UNLOADING_GUEST_DRIVER`,
-not yet implemented). The host-side seed/unlock alternative is fuse-locked on
-this part. The session is therefore intentionally left claimed (`--keep`): the
-GPU is bound to `open-iov-gsp-phys` and runtime-active; `nvidia` stays unbound
-(host headless; `nvidia-smi` fails by design); host healthy, pstore empty. A
-fresh live cycle needs a power cycle.
+This was the first host↔real-GSP-RM exchange on the physical path. At that
+point a clean release was not yet possible: the Booter Unload gate requires
+all four secure scratch14 handoff bits (24–27); a host-driven boot set only
+bit 24, and bit 27 was expected from the server's shutdown path (host RPC
+`UNLOADING_GUEST_DRIVER`, not yet implemented then). The host-side seed/unlock
+alternative is fuse-locked on this part. The session was therefore
+intentionally left claimed (`--keep`): the GPU was bound to
+`open-iov-gsp-phys` and runtime-active; `nvidia` stayed unbound (host
+headless; `nvidia-smi` fails by design); host healthy, pstore empty. A fresh
+live cycle needed a power cycle at that point. **Status update (cycles 2-3,
+same day):** this limitation is retired - the server's shutdown set the
+remaining handoff bit and a single `rmmod` released the device without a
+reboot (see the frontier section).
 
 **Next steps:** (a) execute the 0x1002 sequencer command buffer in the module
 and log 0x1001 as `stage 3e complete`; (b) send `UNLOADING_GUEST_DRIVER` on
 the command queue so the server shuts down, sets the remaining handoff bit and
 teardown can hand the GPU back without a reboot; (c) prove a request/reply
-round trip with the server. All three are now implemented but not yet loaded:
-the module sources contain the sequencer executor (BAR0 register-write/modify/
-poll, delay, store and core-resume opcodes), a generalized msgq producer
-(multi-element records, checksum/sequence/free-space rules, doorbell) with the
-unload RPC, a reply-matching status-queue reader, and a debugfs probe that
-requests the server's static configuration after init-done. The rebuilt module
-awaits the next supervised live cycle (which requires a power cycle, with
-approval, because the current claimed session cannot be released); the rung
-4a/4b/4c code passed an adversarial wire-format review plus its fixes.
-
-**Prepared for the next cycle (build, not loaded):** `open-iov-vfio.ko` sha256
-`f61308bd…`; the cycle should run `phys_claim_cycle.sh --yes --hold 60
---cycles 1 --boot --keep`, then exercise `gsp-phys-rm-probe` (static-info
-round trip) and `gsp-phys-unload-rpc` (server shutdown) through the module's
-read-only/debug state files before a clean `rmmod`. That cycle has now run:
-the boot repeated green, and the server's run-sequencer request was received
-but rejected by an over-strict host-side cap on the request's own buffer size.
-The message was read raw from host memory, the cap corrected and the module
-rebuilt; because the claimed session cannot be released without the shutdown
-RPC, the next supervised power cycle will run the corrected boot-and-shutdown
-flow.
-
-Ending the claimed session (harness notice): `sudo rmmod open_iov_vfio`, then
-`echo auto | sudo tee /sys/bus/pci/devices/0000:01:00.0/power/control` and the
-same for `0000:00:01.0`, then `sudo systemctl start nvidia-persistenced`.
-Because the shutdown RPC is missing, a plain release cannot pass the handoff
-gate, so the reliable route back to the distribution driver is a power cycle.
+round trip with the server. All three were implemented, built and have now
+run live: cycles 2-3 executed the sequencer and reached init-done, and the
+server's own shutdown supplied the missing handoff bit, so each failed cycle
+released without a reboot; a request/reply round trip remains to be
+exercised. The module sources contain the sequencer executor (BAR0
+register-write/modify/poll, delay, store and core-resume opcodes), a
+generalized msgq producer (multi-element records,
+checksum/sequence/free-space rules, doorbell) with the unload RPC, a
+reply-matching status-queue reader, and a debugfs probe that requests the
+server's static configuration after init-done; the rung 4a/4b/4c code passed
+an adversarial wire-format review plus its fixes.
+
+**Prepared for the next cycle (now history):** an early build (`f61308bd…`)
+ran the boot green but the server's run-sequencer request was rejected by an
+over-strict host-side cap on the request's own buffer size. The message was
+read raw from host memory, the cap corrected (build `87309e3a…`), and the
+sequencer then ran live in cycles 2-3 (see the frontier section).
+
+Ending a claimed session: `sudo rmmod open_iov_vfio`, then `echo auto | sudo
+tee /sys/bus/pci/devices/0000:01:00.0/power/control` and the same for
+`0000:00:01.0`, then `sudo systemctl start nvidia-persistenced`. Since cycles
+2-3 this passes the handoff gate (the server's shutdown sets the last handoff
+bit) and completes without a power cycle; if the protected region survives
+teardown the fail-safe keeps the locks and a power cycle is still required,
+with approval.
 
 ## Initialization, work and teardown path