diff options
Diffstat (limited to 'wiki/Physical-GPU-Backend.md')
| -rw-r--r-- | wiki/Physical-GPU-Backend.md | 37 |
1 files changed, 23 insertions, 14 deletions
diff --git a/wiki/Physical-GPU-Backend.md b/wiki/Physical-GPU-Backend.md index 1b83203..bbca084 100644 --- a/wiki/Physical-GPU-Backend.md +++ b/wiki/Physical-GPU-Backend.md @@ -5,7 +5,7 @@ > guest-facing emulation. Canonical sources: [`../docs/03-architecture/08-physical-gpu-integration.md`](../docs/03-architecture/08-physical-gpu-integration.md) > and [`../docs/03-architecture/11-physical-backend-guest-wiring.md`](../docs/03-architecture/11-physical-backend-guest-wiring.md); the docs tree wins on disagreement. -Status: 2026-10-12, tree `master @ 2f066727`. Tags: `[live]` proven with the +Status: 2026-10-20, tree `master @ 16291c72`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented default-off; `[wip]` partially built; `[wall]` current blocker; `[target]` planned end state. @@ -64,13 +64,15 @@ independently verified: 5. **Guest mediation** — the guest-facing emulation stays unchanged; real-resource objects translate to GSP-RM requests, and server-owned ranges populate the real MMU. `[wip]` 6. **Work path** — decoded guest pushes become real push streams; resolved host pages - map through module-built scratch page tables; the firmware schedules the work. `[gated]` + map through module-built scratch page tables; the firmware schedules the work. + `[live]` observed end to end for the verified guest kernel clients. 7. **Completion relay** — hardware-completed semaphores land in guest-polled memory; - events and faults relay through the existing MSI-X injection path. `[gated]` + events and faults relay through the existing MSI-X injection path. + `[live]` engine-authored completions relay to the guest and its waits resolve. ## 5. The two planes and the wiring -The **emulation plane** serves the guest: BAR shadows, the sysmem msgq protocol, a hand-built TU104 static-configuration model, and a software work pipeline that executes decoded memsets, copies and semaphore releases. The **physical backend plane** owns the device, boots the real GSP-RM and runs the RM client ladder. The wiring joins them at the lifecycle seam (mdev create/remove), a static-config identity overlay, a read-only control whitelist, and — for eligible copy-engine work — the scratch-mapped forwarding path, all default-off and fail-open to software. +The **emulation plane** serves the guest: BAR shadows, the sysmem msgq protocol, a hand-built TU104 static-configuration model, and a software work pipeline that executes decoded memsets, copies and semaphore releases. The **physical backend plane** owns the device, boots the real GSP-RM and runs the RM client ladder. The wiring joins them at the lifecycle seam (mdev create/remove), a static-config identity overlay, a read-only control whitelist, and — for eligible work — the scratch-mapped forwarding path, parameterised and fail-open to software. ```text GUEST (QEMU) HOST: vgsp.ko (module-owned GPU) PHYSICAL GPU @@ -85,8 +87,9 @@ INIT preflight: distribution driver green; no vgsp/mdev/QEMU [live] RM battery, UVM registration [live] WORK guest CUDA app -> libcuda -> nvidia.ko -> GPFIFO -> doorbell host pump decodes; eligible CE copies forwarded via module-built scratch - page tables; GR: launch reaches the front-end, descriptor not seen [wall] -DONE engine authors the release in guest memory; host relays via MSI-X [gated] + page tables; GR: launches execute on the engine, completions relay + to the guest [live] +DONE engine authors the release in guest memory; host relays via MSI-X [live] TEARDOWN mdev remove -> QEMU stop -> module unload -> clean GSP-RM shutdown -> protected region down -> distribution driver rebound [live] FAIL a failed mutating operation parks the device in a locked fail-safe; @@ -104,17 +107,20 @@ FAIL a failed mutating operation parks the device in a locked fail-safe; | Power domain held active across the ownership window | verified across supervised cycles `[live]` | | Real GSP boot by the module, through init-done and clean release | done, no reboot on clean cycles `[live]` | | Host RM client and first real engine work (static config, real objects, 4 KiB CE copy) | done `[live]` | -| Guest-copy forwarding, deferred release, hardware-primary copies, engine-authored relay, fault relay | default-off, verified live; one relayed-copy mismatch under triage `[gated]` | -| CUDA kernel execution (`vectorAdd`) | open `[wall]` — the launch-descriptor install path; correct results are the `[target]` | +| Guest-copy forwarding, deferred release, hardware-primary copies, engine-authored relay, fault relay | verified live (seams parameterised and fail-open); the large-buffer staged-transfer class is under repair `[live]` | +| CUDA kernel execution (`vectorAdd` and further clients) | done `[live]` — stock guest CUDA clients (vector-add, matrix-multiply, transformer) execute on the physical engine with verified results and engine-authored completions relayed to the guest | ## 7. The execution frontier The RM client ladder reaches the compute engine (a GR0 channel plus a -compute-class object). A forwarded guest CUDA launch now -reaches the front-end (ring entry fetched, methods executed) but the inline -launch-descriptor install has not been observed and no grid has dispatched; -context validity at dispatch is ruled out. The wall is the form of the -forwarded launch method stream — see [`Execution-Frontier.md`](Execution-Frontier.md). +compute-class object). Forwarded guest CUDA launches now execute on the +engine: the inline launch descriptor installs and is read, the grid +dispatches, the engine authors the completion, the host relays it, and the +guest's synchronisation wait resolves — verified on fresh boots with the +vector-add, matrix-multiply and transformer clients. The leading edge is the +large-buffer staged-transfer class (under repair), reproducibility across +fresh boots, and per-guest execution isolation — see +[`Execution-Frontier.md`](Execution-Frontier.md). A failed mutating operation on the backend parks the device in a locked fail-safe that is never auto-retried; recovery is a supervised step. @@ -147,6 +153,9 @@ fail-safe that is never auto-retried; recovery is a supervised step. access to guest memory stays behind the hypervisor's per-device DMA interface, and tenant policy is a later layer on top ([`GMMU-Multi-Tenant-Protection.md`](GMMU-Multi-Tenant-Protection.md)). +* Multiple mediated devices with different emulated VRAM sizes and multiple + guests already coexist at the device level; per-guest execution isolation + for simultaneous CUDA guests is in progress. ## 10. Open questions @@ -159,7 +168,7 @@ fail-safe that is never auto-retried; recovery is a supervised step. * [`../docs/03-architecture/08-physical-gpu-integration.md`](../docs/03-architecture/08-physical-gpu-integration.md) — canonical: mechanism choice and device bring-up. * [`../docs/03-architecture/11-physical-backend-guest-wiring.md`](../docs/03-architecture/11-physical-backend-guest-wiring.md) — canonical: the guest-wiring design. -* [`Execution-Frontier.md`](Execution-Frontier.md), [`Execution-Plane.md`](Execution-Plane.md) — the GR wall and the software execution plane. +* [`Execution-Frontier.md`](Execution-Frontier.md), [`Execution-Plane.md`](Execution-Plane.md) — the execution frontier and the software execution plane. * [`Host-Module.md`](Host-Module.md) — the module that owns both planes. * [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — page-table seams and scratch mappings. * [`Hardware-Software-Matrix.md`](Hardware-Software-Matrix.md), [`Project-Status.md`](Project-Status.md) — support scope and the honest status picture. |