From 97a389906065337e2dbccef658965441067df104 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Thu, 1 Oct 2026 00:06:19 -0400 Subject: docs: refresh status wording after the 64/64 baseline and the physical-backend lock --- .../06-host-module-and-hybrid-model.md | 2 +- docs/03-architecture/09-full-system-picture.md | 8 +++++++- docs/README.md | 21 ++++++++++++--------- userspace/cuda/READ-ME-FIRST.md | 10 +++++++--- 4 files changed, 27 insertions(+), 14 deletions(-) diff --git a/docs/03-architecture/06-host-module-and-hybrid-model.md b/docs/03-architecture/06-host-module-and-hybrid-model.md index e29951c..d6b96e7 100644 --- a/docs/03-architecture/06-host-module-and-hybrid-model.md +++ b/docs/03-architecture/06-host-module-and-hybrid-model.md @@ -314,6 +314,6 @@ in `kernel-open` first). | Push decoding + semaphore completion | demonstrated live; being productized | | Page-table population for server-owned ranges | designed, not implemented | | GPU work forwarding (GR/CE execution) | designed, not implemented | -| Execution backend (module-owned GPU + direct real-GSP session) | chosen, not started ([`08-physical-gpu-integration.md`](08-physical-gpu-integration.md)) | +| Execution backend (module-owned GPU + direct real-GSP session) | mechanism chosen; ownership + firmware staging verified live; power-domain lock implemented, pending supervised verification; real GSP boot not started ([`08-physical-gpu-integration.md`](08-physical-gpu-integration.md)) | | Real VRAM backing | designed, not implemented | | Windows guest, Xen transport, aarch64 hosts | designed, not implemented | diff --git a/docs/03-architecture/09-full-system-picture.md b/docs/03-architecture/09-full-system-picture.md index 0df0dd2..e03a68e 100644 --- a/docs/03-architecture/09-full-system-picture.md +++ b/docs/03-architecture/09-full-system-picture.md @@ -91,7 +91,13 @@ that device and the physical engines is either emulated (control plane) or translated (work plane) by one kernel module, with no daemon, no host userspace and no modified driver on either side. The distribution `nvidia.ko` remains installed and unmodified on the host; it is simply unbound from the GPU while -the service owns it, and the host's display runs on another GPU. +the service owns it, and the host's display runs on another GPU. Unbinding is +not a neutral operation on every platform โ€” the board's power domain can +power-gate the GPU and hard-freeze the host โ€” so ownership must hold a D0 lock +over the GPU, its board companions and the root port across the whole +unbind/rebind window; the hazard and the lock are described in +[`08-physical-gpu-integration.md`](08-physical-gpu-integration.md) ยง4.2, and the +driver must never be unbound without the lock. ## 2. The control plane: GSP emulation diff --git a/docs/README.md b/docs/README.md index 284ca76..5f9a32f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -98,7 +98,7 @@ and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai- ## 3. Where is the project today? -**Honest status (2026-09-29):** +**Honest status (2026-10-01):** | Area | State | |------|-------| @@ -107,8 +107,8 @@ and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai- | GSP-RM message queue + RPC transport | โœ… Working | | GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | ๐ŸŸก Mostly โ€” the engine inventory now includes the async copy engine and the CE capabilities match the device; reply fidelity for the remaining consumed fields is still being tightened | | Guest `nvidia.ko` probe, `nvidia-smi` | โœ… Working (TU104) | -| Guest CUDA driver (`libcuda`) startup | ๐ŸŸก Partially โ€” `cuInit` and device enumeration work; the client reaches `cuCtxCreate` and currently ends in a forced `999` on a completion wait while the completion/translation plumbing is made reliable. See [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) and [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) | -| CUDA kernel execution | ๐Ÿ”ด Not yet โ€” context creation must complete first, then real work forwarding; the execution backend is chosen but not started ([`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md)); today only decoded memsets/copies/semaphore releases execute, in the host | +| Guest CUDA driver (`libcuda`) startup | โœ… Driver-API baseline โ€” `cuInit`, device enumeration, `cuCtxCreate`, `cuMemAlloc`, copies, free and destroy complete end to end; the smoke test prints `ALL OK` and the 64-byte payload round-trips exactly (64/64, re-verified 2026-10-01 on the current build). See [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) and [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) | +| CUDA kernel execution | ๐Ÿ”ด Not yet โ€” the runtime client initializes and the kernel launch is queued, but execution awaits the backend; the mechanism is chosen (module-owned GPU + real GSP session) with ownership + firmware staging verified and the power-domain lock implemented pending supervised verification ([`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md)); the real GSP boot has not started, and today only decoded memsets/copies/semaphore releases execute, in the host | | OpenGL / Vulkan | ๐Ÿ”ด Not yet | | Windows guest | ๐Ÿ”ด Not yet | | Multi-tenant isolation (VFs, mdev-per-VM) | ๐ŸŸก Designed, partially implemented | @@ -116,14 +116,17 @@ and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai- We also know exactly where the CUDA bring-up stands and what the next step is: the userspace `libcuda` segfault was fixed (an RM allocation reply now echoes the request parameters instead of returning zeros), the engine inventory is -complete, and the client reaches context creation, which currently ends on a -completion wait while decoded work is made to execute reliably. The full +complete, and the driver-API smoke test now completes end to end โ€” `cuCtxCreate` +returns and the 64-byte payload round-trips exactly (64/64, re-verified +2026-10-01 on the current build). The full mechanism, with offsets and the decompiled call path, is in [`../userspace/cuda/docs/05-570.144-decompilation.md`](../userspace/cuda/docs/05-570.144-decompilation.md). -Beyond that, the missing piece for kernel execution is forwarding the guest's -*work* (push buffers, page trees, execution) to the physical GPU under our -control; the chosen backend โ€” a module-owned GPU running its real GSP firmware -as the resource-manager server โ€” is specified in +The remaining frontier is real execution on the physical GPU: the runtime client +initializes and queues a kernel launch, but does not execute it. The chosen +backend โ€” a module-owned GPU running its real GSP firmware as the +resource-manager server โ€” has verified ownership + firmware staging and an +implemented (pending supervised verification) power-domain lock; the real GSP +boot has not started. It is specified in [`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md). For the visual end-state map and the cross-component call paths, see diff --git a/userspace/cuda/READ-ME-FIRST.md b/userspace/cuda/READ-ME-FIRST.md index 9adcf75..8d912dd 100644 --- a/userspace/cuda/READ-ME-FIRST.md +++ b/userspace/cuda/READ-ME-FIRST.md @@ -30,9 +30,13 @@ tooling. **echo the request's params**, because the driver copies 16 bytes of class params back over the caller's 8-byte local (the hook's saved `rbp`) โ€” a zeroed reply is what made `rbp=0` (ยง3.5). The engine inventory is now - complete (the async copy engine is included); context creation reaches a - completion wait that currently ends in `rc=999` (ยง3.6 carries the - historical analysis and a status note). ยง3.3 gives the engine-record layout + complete (the async copy engine is included); context creation now returns + and the whole driver-API smoke test completes `ALL OK`, with the 64-byte + payload round-tripping exactly (64/64, re-verified on the current build). + The frontier is real execution on the physical GPU โ€” the runtime client + initializes and queues a kernel launch, but execution awaits the backend. + ยง3.6 carries the historical analysis of the topology blocker. ยง3.3 gives the + engine-record layout (`base+0x2fec`, `+0x300c+(idx-1)*0x14`, `+0x350c`, `+0x351c+(idx-0x16)*0x14`; fields +0/+8/+0xc) and ยง3.7 lists exactly what the emulation should check first. -- cgit v1.2.3