summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-01 10:07:15 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-01 10:07:15 -0400
commit809e3fa09779b1a9082ba379dd7f346f07b9cee6 (patch)
tree8127ac6b9908be5e6266497dc65abac5726a4932 /docs
parent10e17e08487182342a4e13cbe50b73f250089db3 (diff)
downloadopen-iov-809e3fa09779b1a9082ba379dd7f346f07b9cee6.tar.gz
open-iov-809e3fa09779b1a9082ba379dd7f346f07b9cee6.zip
docs: record the first real GSP boot and the live GSP-RM session
STATE-CURRENT and the architecture docs now describe the observed order (FWSEC -> Booter Load -> RISC-V active -> event 0x1002 run-sequencer -> 0x1001 init-done), the four-handoff-bit clean-release constraint, and the --boot/--keep harness contract.
Diffstat (limited to 'docs')
-rw-r--r--docs/03-architecture/03-gsp-and-gsp-rm.md10
-rw-r--r--docs/03-architecture/08-physical-gpu-integration.md86
2 files changed, 71 insertions, 25 deletions
diff --git a/docs/03-architecture/03-gsp-and-gsp-rm.md b/docs/03-architecture/03-gsp-and-gsp-rm.md
index fe51838..7f206ba 100644
--- a/docs/03-architecture/03-gsp-and-gsp-rm.md
+++ b/docs/03-architecture/03-gsp-and-gsp-rm.md
@@ -54,6 +54,16 @@ Our emulation implements stage completion rather than the stages themselves —
which is legitimate because the *only observable* to the driver is the state
machine it polls.
+On the physical path (2026-10-01), the same handshake ran against real
+firmware: FWSEC raised WPR2 with a zero FRTS error code, the SEC2 booter halted
+with mailbox0 = 0, the RISC-V core reported active (`RISCV_STATUS=0x00000035`,
+active bit 0) and `FALCON_OS` read 0. The server's first message was then a
+status-queue event, function `0x1002` (`NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER`,
+`rpc_result=0`), carrying a command buffer the host must execute;
+`GSP_INIT_DONE` (`0x1001`) follows only after that sequencer runs. The
+host-side queue pair behind the exchange is one shared page table plus a 256 KB
+command queue and a 256 KB status queue of 4 KB elements.
+
```mermaid
sequenceDiagram
autonumber
diff --git a/docs/03-architecture/08-physical-gpu-integration.md b/docs/03-architecture/08-physical-gpu-integration.md
index 5c9e385..0c18e84 100644
--- a/docs/03-architecture/08-physical-gpu-integration.md
+++ b/docs/03-architecture/08-physical-gpu-integration.md
@@ -1,15 +1,16 @@
# Physical GPU Integration: How Work Reaches Real Hardware
> **Status: mechanism chosen; ownership + firmware staging implemented;
-> power-domain lock implemented, pending supervised verification; real GSP
-> boot not started.** The control plane works today: the guest's driver
-> initializes and the CUDA driver-API baseline passes end to end (context
-> creation, allocation, copies, 64/64 payload round-trip), and the guest's
-> decoded work is executed in software as a development bridge. This document
-> records the interfaces that actually exist, the mechanism options, the
-> chosen path, the platform power-domain hazard the ownership path exposes on
-> the development host (§4.2), and what has to be built. It fills the gap
-> left open by
+> power-domain lock verified; first real GSP boot and first live GSP-RM
+> exchange complete (2026-10-01); host-side initialization follow-up and a
+> clean release are in progress.** The control plane works today: the guest's
+> driver initializes and the CUDA driver-API baseline passes end to end
+> (context creation, allocation, copies, 64/64 payload round-trip), and the
+> guest's decoded work is executed in software as a development bridge. This
+> document records the interfaces that actually exist, the mechanism options,
+> the chosen path, the platform power-domain hazard the ownership path
+> exposes on the development host (§4.2), and what has to be built. It fills
+> the gap left open by
> [`06-host-module-and-hybrid-model.md`](06-host-module-and-hybrid-model.md) §5
> and [`../04-implementation/02-practical-challenges-and-adversarial-interop.md`](../04-implementation/02-practical-challenges-and-adversarial-interop.md) §3,
> which named "the host driver's RM API" as an option without a concrete call
@@ -150,12 +151,9 @@ that boot: an out-of-band platform power cut bypasses the lock and would
produce the identical signature, and retrying would be a second dice roll on a
host that has already lost the board.
-The lock is implemented in the ownership path and awaits supervised
-verification (§6, §8); a claim is refused unless both the claim parameter and
-the hazard acknowledgement are set (§8). The exact resource names, controller
-commands, captured timelines and the mitigation's code-level design live in
-the adjacent local analysis notes (`kernel/LOCAL-GSP-BOOT.md`); the
-adjacent document is part of the local analysis.
+The lock is implemented in the ownership path and verified across three
+supervised claim/release cycles on 2026-10-01 (§6, §8); a claim is refused
+unless both the claim parameter and the hazard acknowledgement are set (§8).
### 4.3 Device bring-up
@@ -189,8 +187,28 @@ The bring-up is staged so each step is reversible and independently verified:
Stages 1 and 2 are implemented in the module and verified live (a claim/release
cycle leaves the host driver working on the clean cycles; the staged image
-parses as expected). The power-domain lock of §4.2 guards the claim window;
-stage 3 — the real boot — has not been started.
+parses as expected). The power-domain lock of §4.2 guards the claim window and
+is verified across three supervised cycles. Stage 3 has now run live
+(2026-10-01), in the observed order **FWSEC → Booter Load → RISC-V active →
+first server event**: FWSEC raised WPR2 with a zero FRTS error code
+(`WPR2_LO=0x01ffe000`, `WPR2_HI=0x01ffee00`), the SEC2 Booter Load halted with
+`M0=0` after 146 ms, the RISC-V core reported active
+(`RISCV_STATUS=0x00000035`) with `FALCON_OS=0`, and a host-driven boot sets
+only the booter-load handoff bit (bit 24) in secure scratch14. Stage 4 has
+begun: the staged command/status queues linked and the first real GSP-RM
+message arrived (event 0x1002, run-CPU-sequencer, `rpc_result=0`). The server
+waits for the host to execute the sequencer command buffer carried in that
+event before it raises `GSP_INIT_DONE` (0x1001). **A plain release is not yet
+possible:** the Booter Unload gate requires all four secure scratch14 handoff
+bits (24–27); a
+host-driven boot sets only bit 24, and bit 27 is set by the server's shutdown
+path (host RPC `UNLOADING_GUEST_DRIVER`), which is not implemented, while the
+host-side seed/unlock alternative is fuse-locked on this part. Until that RPC
+lands, a green session is left claimed with the harness `--keep` flag and
+returning the GPU to the distribution driver takes a power cycle. The
+supervised harness modes are `--boot` (the only write mode: full boot
+lifecycle), `--boot-dryrun` (zero-write staging rehearsal) and `--keep`
+(leave a verified claimed session running); `--fwsec` is retired.
### 4.4 Host-side resource-manager client
@@ -245,9 +263,9 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
| Driver-API baseline (init, devices, context, allocation, copies, free, destroy) | done — re-verified 64/64 payload round-trip on the current build |
| Software-executed bridge for decoded memory/state operations | in progress (memsets, copies, releases) |
| Device ownership + firmware staging | verified — claim/release and staged-image parse both verified live; the claim path is hazard-gated (§4.2) |
-| Power-domain lock across the claim window | implemented — pending supervised verification (§4.2, §8) |
-| Real GSP boot by the module | not started |
-| Host-side resource-manager client for real objects/channels | not started |
+| Power-domain lock across the claim window | verified — 3/3 supervised claim/release cycles (2026-10-01; §4.2, §8) |
+| Real GSP boot by the module | first live boot 2026-10-01 — FWSEC WPR2 up, Booter Load `M0=0`, RISC-V active; clean release pending the server shutdown RPC |
+| Host-side resource-manager client for real objects/channels | transport alive 2026-10-01 — first real GSP-RM event (0x1002) received; init-done (0x1001) awaits host sequencer execution |
| Work translation and real submission | not started |
| Completion relay through real hardware | not started |
| CUDA kernel execution (`vectorAdd`) | not started |
@@ -265,11 +283,11 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
controller action that does not go through the root port's D3 path — would
produce the identical power-loss signature; the claim must abort on that
signature and must not be retried in the same boot.
-* **Supervised verification of the lock:** the lock is implemented but not yet
- empirically verified. The first claim/release with it active must be
- supervised: explicit human go-ahead, other host work warned, a written
- recovery plan in place, a held-open window long enough to prove the board
- stayed powered, and pstore armed.
+* **Supervised operation of the lock:** the lock is implemented and verified
+ across three supervised claim/release cycles (2026-10-01). Every claim
+ remains supervised: explicit human go-ahead, other host work warned, a
+ written recovery plan in place, a held-open window long enough to prove the
+ board stayed powered, and pstore armed.
## 8. Decision record
@@ -289,6 +307,24 @@ per-context scheduling. Tenant policy and fairness are a later layer on top.
plan. The claim path refuses without the acknowledgement. At the first
power-loss signature: abort and do not retry in that boot; a host reboot
requires explicit human approval and a written recovery plan updated first.
+* **Update (2026-10-01) — FWSEC-only retired:** stopping after the
+ secure-verification carve alone is a one-way door on this fused part (the
+ protected region is left up and is not host-recoverable). The supervised
+ claim harness therefore refuses `--fwsec` before any module load. The only
+ device-write mode is `--boot` (`gsp_phys_boot=1`, the full boot lifecycle
+ through resource-manager init); the zero-write rehearsal is `--boot-dryrun`.
+ Never stop after FWSEC alone on this host.
+* **Update (2026-10-01) — first live boot; clean release gated:** the
+ supervised `--boot --keep` cycle completed the first real boot (FWSEC →
+ Booter Load → RISC-V active) and received the first real GSP-RM event
+ (0x1002, run-CPU-sequencer); `GSP_INIT_DONE` (0x1001) awaits the host
+ executing the sequencer buffer carried in that event. Clean release is not
+ yet possible: the Booter Unload gate requires all four secure scratch14
+ handoff bits (24–27), a host-driven boot sets only bit 24, bit 27 is set by
+ the server's shutdown path (host RPC `UNLOADING_GUEST_DRIVER`, not
+ implemented), and host-side seed/unlock is fuse-locked on this part. The
+ live session is therefore left claimed (`--keep`) and a fresh cycle needs a
+ power cycle. See §4.3.
## 9. Interaction sequence (end state)