diff options
Diffstat (limited to 'wiki/Execution-Plane.md')
| -rw-r--r-- | wiki/Execution-Plane.md | 90 |
1 files changed, 50 insertions, 40 deletions
diff --git a/wiki/Execution-Plane.md b/wiki/Execution-Plane.md index 5d1052f..d88a1f3 100644 --- a/wiki/Execution-Plane.md +++ b/wiki/Execution-Plane.md @@ -8,7 +8,7 @@ > when the two disagree, the docs tree wins. Front door: > [`../docs/README.md`](../docs/README.md). -Status: 2026-10-08, tree `master @ 1f571dda`. Tags: `[live]` proven with the +Status: 2026-11-11, tree `master @ 4acac093`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented, default-off or parameter-gated; `[wip]` partial; `[wall]` current blocker; `[target]` named success latch; `[planned]` designed, not built. @@ -37,7 +37,7 @@ would collide with existing project vocabulary. | Scheduler | which channel's work runs, at what pace | doorbell steering, the per-channel pump, discovery cadence, one ordered workqueue, one in-flight request | | Allocator | where rings, USERDs, counters, notifiers and scratch windows live | ring/USERD autopin and pins, content-keyed pairing, counter derive, ladder scratch/PT pools, the BAR1 vidmem shadow | | Control | which RM objects/controls shape execution | channel and event registration, `SET_PAGE_DIRECTORY`, `PROMOTE_CTX`, engine topology, the mode matrix | -| Share | how clients/mdevs share the one GPU | mdev backend refcount, one real session under a single lock, one workqueue; per-mdev clients and fairness are planned | +| Share | how clients/mdevs share the one GPU | mdev backend refcount, one real session under a single lock, one workqueue; two guests with different emulated VRAM sizes coexist at the device level, per-guest execution isolation is in progress | ## 2. The shape of the plane @@ -93,18 +93,19 @@ guest: push + QMD + GPFIFO entry + doorbell (BAR0 0x88704) pages into module shadow windows whose PTEs were planted before the channel commit (the engine never sees a late PTE write — it caches 128-byte fetch lines with no invalidate), builds the launch method block, publishes it to - the ladder channel's GPFIFO and rings the real doorbell. `[gated]` - default-off; latched on a proven real GR submission. -5. **Engine.** The front-end fetches the entry and executes methods; then the - **inline-QMD install** must materialise the descriptor for the engine to - read, currently **`[wall]`** — not observed. Context validity, destination - mapping and fetch/alignment are exonerated; the recovered-method-stream - form is the investigation. + the ladder channel's GPFIFO and rings the real doorbell. `[live]` — + exercised on every verified guest launch (default-off while the seams are + validated). +5. **Engine.** The front-end fetches the entry and executes methods; the + inline launch descriptor installs and the engine reads it, the grid + dispatches to the SMs, and the engine authors the release. `[live]` — + context validity, destination mapping and fetch/alignment were exonerated + on the way. 6. **Completion.** The engine authors the QMD release word and the channel release; the host applies the deferred release exactly once, publishes the completion counter, clears the notifier and posts the channel event; MSI-X - wakes the guest. CE work is live in this shape `[gated]`; for GR the - completion-counter publication is the current second face of the wall. + wakes the guest. `[live]` for CE and GR: engine-authored completions relay + to the guest and its synchronisation waits resolve. ## 4. How work is found: discovery, truth, completion @@ -129,27 +130,34 @@ guest: push + QMD + GPFIFO entry + doorbell (BAR0 0x88704) ## 5. Challenges, honestly -* **GR dispatch wall.** The forward path runs front-end methods, the context - is valid and the destination window is accepted — and the descriptor - install is still not observed. The mode matrix carries dozens of - single-variable falsifiers (promote shapes, prelude/PCAS form, local-memory - block, lane controls, shadow windows, pointer relocation). -* **Completion publication.** A boot where GR rings were discovered and - releases applied still saw the guest re-queue the same push and give up. A - later `sema_dbg=Y` run settled it: the release was written to the exact - slot, the notifier cleared and the event posted, while the completion - counter publication never wrote because the fixed-total autoreg mapping - makes it a structural no-op. The leading reading is the counter-slot - publication; the GP_GET write-back is the secondary, still-open candidate. - Both are debug-gated today. -* **Reproducibility (ISSUE-0001).** The green/stall split is per-boot. The - first fix wave landed and the earlier CE freeze is gone; the stall moved to - GR ring discovery, which the latest discovery work addresses. Acceptance - is 10 cold boots green, pending. -* **Multiple clients (ISSUE-0002).** A second CUDA process wedges in guest - UVM because host per-client sema/GR state is not reset when the guest - re-runs its per-client boot handshake. Per-client state containers and - reset are the designed path. +* **The staged-transfer class (compute).** Some large multi-chunk CUDA clients + still mis-verify; the failure modes are proven (a release applied before + the op it owes, a ledger-vs-writer page divergence, a readback racing a + writeback) and the fixes are in flight. +* **Every-boot reproducibility.** The boot-poisoning defect (wrong + status-queue consumption cursor under the driver's swap mode) and the + post-reboot ring-block ownership defect were root-caused and fixed; green + clients reproduce across fresh boots, and the formal every-boot acceptance + campaign is the remaining work. +* **Multiple clients and per-guest isolation.** Two guests with different + emulated VRAM sizes coexist at the device level; per-guest execution + isolation for simultaneous CUDA guests is in progress. +* **The graphics pixel frontier.** Graphics work rides this plane in three + shapes (verbatim 3D method-stream replay, synthesized launches, and the + direct GL client's own GR/CE channels, which now submit and retire). The + display/KMS path is served by the emulated display engine (modeset push + buffers plus the notifier/`UPDATE` completion, class-aware arm), so NvKms + registers, X settles and GL 4.6 is live, the X DDX waits are fixed, and + the guest's real GL draws now forward and execute on the physical GPU + (`backend_ok=5-7`, `backend_timeout=0`); the DDX 3D replay executes, the 2D + path is green, the first non-black GPU-rendered frames are proven (500/500 + off-screen frames plus a 45.4 s video), and the direct-GL **GR ring + `0xbeef0100` resolves with advancing releases**. No non-black **window/ + screen** frame has been produced through the NVIDIA path yet: the direct-GL + **CE ring `0xbeef0101` never resolves** (the guest wedges ~90 s after + `V_Init`), glxgears' window is black, and X window creation wedges on some + boots, so those are the named blockers. The lanes and the ladder are on + [`Graphics-Execution.md`](Graphics-Execution.md). ## 6. Portability @@ -164,15 +172,17 @@ human approval gate. ## 7. Future -The planned next steps are reproducibility acceptance, the descriptor-install -and completion-publication investigations (offline-first with the RISC-V GSP-RM debugger -where possible), the arbitrary-kernel acceptance suite, then the near-native -performance program. The unsolved future classes — multi-tenant fairness, -fault replay/cancel, context switching between tenants, hostile-guest +The planned next steps are the staged-transfer fixes, the every-boot +acceptance campaign, per-guest isolation, and the graphics pixel frontier +(offline-first with the RISC-V GSP-RM debugger where possible), then the +near-native performance program. The unsolved future classes — multi-tenant +fairness, fault replay/cancel, context switching between tenants, hostile-guest hardening, and version drift — each have a proactive design direction in the -canonical document. 3D graphics reuses this pipeline once compute dispatch is -proven; it adds engine breadth (video/image), more channels and synchronisation -volume, and a much larger method vocabulary, but not a new plane. +canonical document. 3D graphics reuses this pipeline and now executes across +three shapes, including the direct GL client's own GR/CE channels +([`Graphics-Execution.md`](Graphics-Execution.md)); it adds engine +breadth (video/image), more channels and synchronisation volume, and a much +larger method vocabulary, but not a new plane. --- |