diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-11-09 19:08:20 -0500 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-11-09 19:08:20 -0500 |
| commit | 4a2cf8292ad37767de22a26e7e35d0a8a06edad3 (patch) | |
| tree | 10f953a510175edd01b08ad14a8f9728abac34b3 /wiki | |
| parent | 21e0d611b5efbdcf6a14520c3294d6a78bb32220 (diff) | |
| download | open-iov-4a2cf8292ad37767de22a26e7e35d0a8a06edad3.tar.gz open-iov-4a2cf8292ad37767de22a26e7e35d0a8a06edad3.zip | |
wiki: 2026-11-09 evening refresh 2 - forced GLX reaches glcore; windowed GL waits on kernel kick completion
Refresh STATE-CURRENT, the Main Page and the status/frontier pages: with
SDL_VIDEO_X11_FORCE_EGL=0 the windowed GL path selects libnvidia-glcore,
but the minimal glxgears hangs in the glcore kick wait (poll on the
per-slot /dev/nvidia-modeset fd) while the host completes the beef
notifier memory. The X DDX get-word/notifier completion, the DDX 4 GB
sysmem footprint and physical dispatch of the GL draws remain. Published
to open-iov.org.
Diffstat (limited to 'wiki')
| -rw-r--r-- | wiki/.publish-manifest.json | 120 | ||||
| -rw-r--r-- | wiki/Execution-Frontier.md | 25 | ||||
| -rw-r--r-- | wiki/Execution-Plane.md | 9 | ||||
| -rw-r--r-- | wiki/FAQ.md | 21 | ||||
| -rw-r--r-- | wiki/Graphics-Execution.md | 46 | ||||
| -rw-r--r-- | wiki/Host-Module.md | 7 | ||||
| -rw-r--r-- | wiki/OpenGL.md | 17 | ||||
| -rw-r--r-- | wiki/Physical-GPU-Backend.md | 10 | ||||
| -rw-r--r-- | wiki/Project-Goals.md | 4 | ||||
| -rw-r--r-- | wiki/Project-Plan.md | 23 | ||||
| -rw-r--r-- | wiki/Project-Status.md | 13 | ||||
| -rw-r--r-- | wiki/System-Overview.md | 14 |
12 files changed, 169 insertions, 140 deletions
diff --git a/wiki/.publish-manifest.json b/wiki/.publish-manifest.json index 60f24dc..27c52de 100644 --- a/wiki/.publish-manifest.json +++ b/wiki/.publish-manifest.json @@ -64,11 +64,11 @@ }, "FAQ": { "file": "FAQ.wiki", - "new_revid": 24049, - "old_revid": 24035, - "source_sha": "7cc1ea9e3ec9494f", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:21Z", + "new_revid": 24061, + "old_revid": 24049, + "source_sha": "17fa28555cd7eb71", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:45Z", "title": "FAQ" }, "GMMU Multi-Tenant Protection": { @@ -127,11 +127,11 @@ }, "Graphics Execution": { "file": "Graphics-Execution.wiki", - "new_revid": 24050, - "old_revid": 24036, - "source_sha": "aa8f827e5273e941", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:21Z", + "new_revid": 24062, + "old_revid": 24050, + "source_sha": "11e8fa4f83fed247", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:46Z", "title": "Graphics Execution" }, "Guest Driver Lifecycle: A Field Guide": { @@ -190,11 +190,11 @@ }, "Main Page": { "file": "Main-Page.wiki", - "new_revid": 24058, - "old_revid": 24043, - "source_sha": "9383c4a57251d7bd", - "summary": "2026-11-09 evening status refresh: GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the graphics frontier", - "timestamp": "2026-11-10T00:03:29Z", + "new_revid": 24070, + "old_revid": 24058, + "source_sha": "d79ac879f1798d05", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait", + "timestamp": "2026-11-10T00:06:53Z", "title": "Main Page" }, "MediaWiki:Sidebar": { @@ -226,11 +226,11 @@ }, "OpenGL": { "file": "OpenGL.wiki", - "new_revid": 24052, - "old_revid": 24038, - "source_sha": "28cb3bf34fda32b9", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:23Z", + "new_revid": 24064, + "old_revid": 24052, + "source_sha": "cd8779694a581bf5", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:47Z", "title": "OpenGL" }, "Performance": { @@ -244,38 +244,38 @@ }, "Physical GPU Backend and Guest Wiring": { "file": "Physical-GPU-Backend.wiki", - "new_revid": 24053, - "old_revid": 24046, - "source_sha": "55491ec835b14c78", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:23Z", + "new_revid": 24065, + "old_revid": 24053, + "source_sha": "cce4eba81a924f9f", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:48Z", "title": "Physical GPU Backend and Guest Wiring" }, "Project Goals": { "file": "Project-Goals.wiki", - "new_revid": 24054, - "old_revid": 24039, - "source_sha": "593731fbdc44a7a3", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:24Z", + "new_revid": 24066, + "old_revid": 24054, + "source_sha": "f717cd829cb16ff3", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:49Z", "title": "Project Goals" }, "Project Plan": { "file": "Project-Plan.wiki", - "new_revid": 24055, - "old_revid": 24040, - "source_sha": "5841dc6b6f506c43", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:24Z", + "new_revid": 24067, + "old_revid": 24055, + "source_sha": "57c421e0af6c3d69", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:49Z", "title": "Project Plan" }, "Project Status": { "file": "Project-Status.wiki", - "new_revid": 24056, - "old_revid": 24041, - "source_sha": "8dcde28c41c033d1", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:25Z", + "new_revid": 24068, + "old_revid": 24056, + "source_sha": "91905880addcf2e7", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:50Z", "title": "Project Status" }, "RPC Message Format": { @@ -289,11 +289,11 @@ }, "System Overview": { "file": "System-Overview.wiki", - "new_revid": 24057, - "old_revid": 24042, - "source_sha": "9ac0f7844680d16d", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:25Z", + "new_revid": 24069, + "old_revid": 24057, + "source_sha": "49cc4d259bf1e8d5", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:50Z", "title": "System Overview" }, "Template:About message": { @@ -325,11 +325,11 @@ }, "The Execution Frontier": { "file": "Execution-Frontier.wiki", - "new_revid": 24047, - "old_revid": 24044, - "source_sha": "bf2a9ec21fe7befd", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:20Z", + "new_revid": 24059, + "old_revid": 24047, + "source_sha": "8bdff873c9b0f66b", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:44Z", "title": "The Execution Frontier" }, "The Guest Stack and CUDASW": { @@ -343,20 +343,20 @@ }, "The Host Module (vgsp.ko)": { "file": "Host-Module.wiki", - "new_revid": 24051, - "old_revid": 24037, - "source_sha": "38f4ef91cc0cdc1b", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:22Z", + "new_revid": 24063, + "old_revid": 24051, + "source_sha": "4631ad02e69f5e8f", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:47Z", "title": "The Host Module (vgsp.ko)" }, "The vGSP Execution Plane (vEP)": { "file": "Execution-Plane.wiki", - "new_revid": 24048, - "old_revid": 24045, - "source_sha": "86fd3830f44d7903", - "summary": "2026-11-09 evening refresh: X DDX waits fixed; GZDoom reaches MAP01 on llvmpipe; NVIDIA renderer, DDX completion and GL dispatch are the pixel walls", - "timestamp": "2026-11-10T00:03:20Z", + "new_revid": 24060, + "old_revid": 24048, + "source_sha": "578bf257456d5b8b", + "summary": "2026-11-09 evening refresh 2: forced GLX selects libnvidia-glcore; windowed glxgears hangs in the glcore kick wait; kernel-visible present completion is the wall", + "timestamp": "2026-11-10T00:06:45Z", "title": "The vGSP Execution Plane (vEP)" }, "UVM in the Guest": { diff --git a/wiki/Execution-Frontier.md b/wiki/Execution-Frontier.md index 57ede35..658f222 100644 --- a/wiki/Execution-Frontier.md +++ b/wiki/Execution-Frontier.md @@ -14,7 +14,7 @@ > [`../docs/04-implementation/03-execution-frontier.md`](../docs/04-implementation/03-execution-frontier.md); > the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` observed; `[wip]` partial; `[wall]` current blocker; theories are labelled as such. @@ -103,10 +103,13 @@ OpenGL 4.6.0 with direct rendering, and the X DDX waits are fixed release applied); the direct GL client's GR and CE channels submit and retire, and the X DDX kick channel is decoded. GZDoom reached `MAP01` once the DDX push-buffer loop was unblocked, but on Mesa llvmpipe — not the emulated vGPU. -What is missing is **rendered pixels through the NVIDIA path**: the renderer -backend (Vulkan ICD device creation or forced GLX), the DDX get-word/notifier -completion and physical dispatch of the GL draws are the named walls, and the -guest Vulkan pipeline's inline-to-memory upload arm remains its gap. The full +What is missing is **rendered pixels through the NVIDIA path**: forcing GLX +now selects `libnvidia-glcore`, but the minimal windowed `glxgears` hangs in +the glcore kick wait (the client polls its per-slot `/dev/nvidia-modeset` fd +and the guest kernel never signals the event), the DDX push-buffer +get-word/notifier completion is not implemented, and the direct-GL draws still +report no physical dispatch; the guest Vulkan pipeline's inline-to-memory +upload arm remains its gap. The full picture lives on [`Graphics-Execution.md`](Graphics-Execution.md). No breadth beyond these is claimed: Windows guests, migration and performance @@ -165,11 +168,13 @@ CUDA guests is in progress. **Issue D — the graphics pixel frontier.** The display/KMS stack and the GL client now run end to end (NvKms registers, X settles, GL 4.6, direct-GL -channels submit and retire, the X DDX waits are fixed), but no rendered -(non-black) frame has been produced yet through the NVIDIA path; GZDoom's -`MAP01` is a Mesa llvmpipe datapoint. The renderer backend (Vulkan ICD or -forced GLX), the DDX get-word/notifier completion and physical dispatch of -the GL draws (`forwarded=0`) are the named blockers +channels submit and retire, the X DDX waits are fixed), and forcing GLX +selects `libnvidia-glcore`; but the minimal windowed `glxgears` hangs in the +glcore kick wait (the client polls its per-slot `/dev/nvidia-modeset` fd and +the guest kernel never signals the event), and no rendered (non-black) frame +has been produced yet. The kernel-visible present completion, the DDX +get-word/notifier completion and physical dispatch of the GL draws +(`forwarded=0`) are the named blockers ([`Graphics-Execution.md`](Graphics-Execution.md)). ## 6. Lines of investigation (live) diff --git a/wiki/Execution-Plane.md b/wiki/Execution-Plane.md index f2b7bbf..0cb7e7c 100644 --- a/wiki/Execution-Plane.md +++ b/wiki/Execution-Plane.md @@ -8,7 +8,7 @@ > when the two disagree, the docs tree wins. Front door: > [`../docs/README.md`](../docs/README.md). -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` proven with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented, default-off or parameter-gated; `[wip]` partial; `[wall]` current blocker; `[target]` named success latch; `[planned]` designed, not built. @@ -147,9 +147,10 @@ guest: push + QMD + GPFIFO entry + doorbell (BAR0 0x88704) direct GL client's own GR/CE channels, which now submit and retire). The display/KMS path is served by the emulated display engine (modeset push buffers plus the notifier/`UPDATE` completion, class-aware arm), so NvKms - registers, X settles and GL 4.6 is live, and the X DDX waits are fixed. No - rendered (non-black) frame has been produced through the NVIDIA path yet: - the renderer backend (Vulkan ICD or forced GLX), the DDX push-buffer + registers, X settles and GL 4.6 is live, the X DDX waits are fixed, and + forcing GLX selects `libnvidia-glcore`. No rendered (non-black) frame has + been produced through the NVIDIA path yet: the kernel-visible GL present/kick + completion (the minimal `glxgears` hangs there), the DDX push-buffer get-word/notifier completion and physical dispatch of the GL draws (`forwarded=0`) are the named blockers; GZDoom's `MAP01` is a Mesa llvmpipe datapoint. The lanes and the ladder are on diff --git a/wiki/FAQ.md b/wiki/FAQ.md index 57f8218..5025175 100644 --- a/wiki/FAQ.md +++ b/wiki/FAQ.md @@ -5,7 +5,7 @@ > [`../docs/09-reference/glossary-and-faq.md`](../docs/09-reference/glossary-and-faq.md); > the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. +Status: 2026-11-09, tree `master @ 21e0d611`. ## Project and positioning @@ -40,14 +40,17 @@ progress). This is the project's known status, not a misconfiguration — see Partially, and it is an active frontier. The display/KMS stack and the X server now run on the emulated vGPU with the stock driver: nvidia-modeset/NvKms registers (`/dev/dri`), `glxinfo` reports OpenGL 4.6.0 with direct rendering, -the direct GL client's GR and CE channels submit and retire, and GZDoom -reaches renderer startup and `MAP01` — but currently on Mesa llvmpipe. The -guest Vulkan pipeline runs end to end (device → command buffer → fence → -readback, frames black). Rendered (non-black) frames through the NVIDIA path -are the remaining wall: the NVIDIA Vulkan ICD/forced-GLX renderer path, the X -DDX get-word/notifier completion, the DDX's 4 GB sysmem allocation (12 GB -guest workaround) and physical dispatch of the GL draws are the current gates -— see [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and +the direct GL client's GR and CE channels submit and retire, and forcing GLX +(`SDL_VIDEO_X11_FORCE_EGL=0`) selects `libnvidia-glcore`. The minimal +windowed `glxgears` then hangs in the glcore kick wait: the client polls its +per-slot `/dev/nvidia-modeset` fd and the guest kernel never signals the +event, even though the host completes the beef notifier memory. The guest +Vulkan pipeline runs end to end (device → command buffer → fence → readback, +frames black). Rendered (non-black) frames through the NVIDIA path are the +remaining wall: kernel-visible present completion, the X DDX +get-word/notifier completion, the DDX's 4 GB sysmem allocation (12 GB guest +workaround) and physical dispatch of the GL draws are the current gates — +see [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and [`Graphics-Execution.md`](Graphics-Execution.md). **How fast is it?** diff --git a/wiki/Graphics-Execution.md b/wiki/Graphics-Execution.md index cba9732..ffbcfaf 100644 --- a/wiki/Graphics-Execution.md +++ b/wiki/Graphics-Execution.md @@ -12,7 +12,7 @@ > [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md) > (the measured walls); the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` observed end to +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` observed end to end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined success state; `[planned]` designed, not built. @@ -89,10 +89,10 @@ fidelity: kick channel (`0xbfef0002`) is decoded from its ring head with its class-2D notification authored. The X DDX waits are fixed (class-aware display arm, `0x8000 -> 0` notification completion, decoded GL release - applied), and GZDoom reached `MAP01` once the DDX push-buffer loop was - unblocked. What remains is the renderer backend path (NVIDIA Vulkan ICD or - forced GLX), the DDX get-word/notifier completion, and physical dispatch - of the client's draws (§6). + applied), and with GLX forced (`SDL_VIDEO_X11_FORCE_EGL=0`) the windowed + GL path selects `libnvidia-glcore`. What remains is the kernel-visible GL + present/kick completion, the DDX get-word/notifier recycle, and physical + dispatch of the client's draws (§6). ## 4. The channel and object picture @@ -139,7 +139,10 @@ until it posts, so the X server's mode set finishes. For the X DDX's own 2D push buffers, completion also requires the GPU-written engine get word to advance and the notifier to be recycled to `0xffff` after a kick; both are host-side levers now, and a diagnostic write of the get word released the -DDX's reserve loop. +DDX's reserve loop. For a windowed GL client the present is a kick whose +completion the guest kernel observes by polling its per-slot +`/dev/nvidia-modeset` fd; the host must complete the exact notifier page the +kernel watches and signal that event, not only write the notifier memory. During bring-up the module also carries a **diagnostics-only mirror** for apps that spin on a shared progress word; it exists so a run can proceed while the @@ -148,11 +151,14 @@ requires it switched off, along with any external poke. ## 6. Where the frontier is (ranked walls) -1. **The renderer backend path** `[wall]`. GZDoom's default Vulkan backend - fails at `vkCreateDevice` in the NVIDIA Vulkan ICD, and the default - OpenGL/EGL path falls back to Mesa (`llvmpipe`). The campaign is forcing - GLX (`SDL_VIDEO_X11_FORCE_EGL=0`) so the client uses the NVIDIA GL stack; - the first NVIDIA-path frame is gated here. +1. **Kernel-visible GL present completion** `[wall]`. With GLX forced the + windowed client uses `libnvidia-glcore`; the minimal `glxgears` then hangs + in the glcore kick wait: the client polls its per-slot + `/dev/nvidia-modeset` fd (`poll(POLLIN, 1 s)`). The host's sweep completes + the beef notifier memory, but the guest kernel never signals the event the + poll waits on. The exact notifier page the kernel watches must be + completed and the guest event signalled (the kick-notify executor), not + just the memory written. 2. **The X DDX push-buffer completion** `[wall]`. The DDX's 2D reserve loop exits only when the GPU-written engine get word advances (`get <= put`) and the kick requires the notifier to be recycled to `0xffff` after @@ -168,9 +174,9 @@ requires it switched off, along with any external poke. dispatch to the physical GR engine (`forwarded=0`); the draws are decoded and host-completed, not executed on hardware. 5. **Non-black frames and sustained rate** `[wall]`/`[target]`. Once the - NVIDIA renderer path and the DDX get-word/notifier completion land, the - capture must show motion, then 60 fps in one VM with no forcing, then two - VMs on two mediated devices with non-black frames in both. + kernel-visible present completion and the DDX get-word/notifier completion + land, the capture must show motion, then 60 fps in one VM with no forcing, + then two VMs on two mediated devices with non-black frames in both. ## 7. What is proven today @@ -182,6 +188,8 @@ requires it switched off, along with any external poke. | `glxinfo` reports the emulated GPU with the NVIDIA renderer (OpenGL 4.6.0, direct rendering) | `[live]` | | GZDoom obtains a real GL context; windowed reaches renderer startup | `[live]` (pixels frontier) | | GZDoom reaches `MAP01` on Mesa llvmpipe (not the emulated vGPU path) | `[wip]` | +| Forced GLX (`SDL_VIDEO_X11_FORCE_EGL=0`) selects `libnvidia-glcore` | `[live]` | +| Windowed GL present/kick completion (minimal `glxgears`) | `[wall]` | | X server survives the GZDoom run on a 12 GB guest (the DDX's 4 GB sysmem allocation forces the size) | `[live]` | | GL section capture: the X server's `0xC597` stream decoded | `[live]` | | Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` | @@ -189,7 +197,8 @@ requires it switched off, along with any external poke. | 2D screen fills and copy-engine present ops execute | `[live]` | | Direct GL client: device-init gate handled, channels allocate, readback path returns clean | `[live]` | | Direct GL client channels submit and retire (GR `0xbeef0100`, CE `0xbeef0101`/`0xbeef0102`); DDX kick channel decoded | `[live]` | -| NVIDIA Vulkan ICD creates a device, or forced GLX reaches glcore | `[wall]` | +| Guest kernel observes the kick completion and signals the modeset event | `[wall]` | +| NVIDIA Vulkan ICD creates a device | `[wall]` | | DDX push-buffer get-word publish and notifier recycle | `[wall]` | | Physical dispatch of the direct-GL draws (`forwarded=0`) | `[wall]` | | Non-black pixels from the emulated vGPU's 3D pipeline | `[wall]` | @@ -200,9 +209,10 @@ requires it switched off, along with any external poke. * No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet. * Display/KMS bring-up (NvKms, `/dev/dri`, X, a real GL context) is not a rendered-frame claim; GZDoom reaching `MAP01` on Mesa llvmpipe is not the - emulated-vGPU path either. The NVIDIA renderer path, the DDX - get-word/notifier completion and the physical dispatch of the GL draws are - the current walls. + emulated-vGPU path, and the later forced-GLX run selects `libnvidia-glcore` + but hangs in the present/kick wait. The kernel-visible present completion, + the DDX get-word/notifier completion and the physical dispatch of the GL + draws are the current walls. * No frame-rate or latency numbers: the scheduled measurement is defined (60 fps over 60 s, no forcing) but not yet met. * The `virgl`-based demo videos are a different, labelled path — not this one. diff --git a/wiki/Host-Module.md b/wiki/Host-Module.md index e6b7ee9..35bf3c8 100644 --- a/wiki/Host-Module.md +++ b/wiki/Host-Module.md @@ -6,7 +6,7 @@ > source: [`../docs/03-architecture/06-host-module-and-hybrid-model.md`](../docs/03-architecture/06-host-module-and-hybrid-model.md); > the docs tree wins on disagreement with this page. > -> Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` observed with +> Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` observed with > the stock guest driver; `[wip]` demonstrated live, being productized; > `[gated]` implemented default-off behind supervised gates; `[wall]` known > blocking limit; `[target]` designed, not built; `[planned]` roadmap. @@ -122,7 +122,10 @@ nvidia-modeset/NvKms register (`/dev/dri`), X settle and GLX clients obtain a real GL context. The display-capability seam is parameterised. For the DDX's own 2D push buffers the remaining host levers are publishing/advancing the GPU-written engine get word and recycling the notifier to `0xffff` after -completion. +completion. For a windowed GL client the host must also make the kick +completion visible to the guest kernel and signal its modeset event: writing +the beef notifier memory alone leaves `poll()` on the per-slot +`/dev/nvidia-modeset` fd waiting. ## 6. Debug parameters diff --git a/wiki/OpenGL.md b/wiki/OpenGL.md index 08f26fa..45af4c8 100644 --- a/wiki/OpenGL.md +++ b/wiki/OpenGL.md @@ -12,7 +12,7 @@ > [`../docs/03-architecture/02-guest-stack-and-cudasw.md`](../docs/03-architecture/02-guest-stack-and-cudasw.md) > (the guest stack). The docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` observed with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` observed with the stock guest driver; `[wip]` partial; `[wall]` current blocker; `[target]` defined success state. @@ -77,13 +77,16 @@ Rated honestly, against the milestone ladder in the plan: | The X driver's RC-watchdog channel registers and its notifier is authored | `[live]` | | GZDoom obtains a real GL context; windowed GZDoom reaches renderer startup | `[live]` (menu/frames black) | | GZDoom reaches `MAP01` on Mesa llvmpipe (not the emulated vGPU path) | `[wip]` | +| Forced GLX (`SDL_VIDEO_X11_FORCE_EGL=0`) selects `libnvidia-glcore` | `[live]` | +| Windowed GL present/kick completion (minimal `glxgears`) | `[wall]` | | X server survives the GZDoom run on a 12 GB guest (the DDX's 4 GB sysmem allocation forces the size) | `[live]` | | X server's `0xC597` graphics section captured and decoded | `[live]` | | Engine executes a launch built from the captured section (real `sm_75` SASS retired) | `[live]` | | Verbatim 3D method-stream replay retired by the engine, no MMU fault | `[live]` | | Direct GL client: device-init gate handled; client channels allocate; readback path returns with no GL error | `[live]` | | Direct GL client submits and retires its per-frame draws (GR `0xbeef0100`, CE `0xbeef0101`/`0xbeef0102`); DDX kick channel decoded | `[live]` | -| NVIDIA Vulkan ICD creates a device, or forced GLX (`SDL_VIDEO_X11_FORCE_EGL=0`) reaches glcore | `[wall]` | +| Guest kernel observes the kick completion and signals the modeset event | `[wall]` | +| NVIDIA Vulkan ICD creates a device | `[wall]` | | DDX push-buffer get-word publish and notifier recycle | `[wall]` | | Physical dispatch of the direct-GL draws (`forwarded=0`) | `[wall]` | | Non-black rendered frames through the emulated vGPU | `[wall]` | @@ -108,15 +111,17 @@ non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two mdevs), M-GL6 (split-screen video). The project sits **between M-GL1 and M-GL2**: the interface and execution work is done — the display/KMS stack registers, X settles, the client submits and retires its GR/CE channels, and -the engine authors stores — and the remaining gates are the NVIDIA renderer -path (Vulkan ICD or forced GLX), the X DDX get-word/notifier completion, and -physical dispatch of the client's draws before the first rendered frame. +the engine authors stores — and the remaining gates are the kernel-visible GL +present/kick completion, the X DDX get-word/notifier completion, and physical +dispatch of the client's draws before the first rendered frame. ## 6. What is not claimed * No OpenGL frame from the emulated vGPU has produced real rendered pixels yet; every capture so far is black. GZDoom reaching `MAP01` on Mesa - llvmpipe is not a rendered frame from the emulated vGPU. + llvmpipe is not a rendered frame from the emulated vGPU, and the forced-GLX + run selects `libnvidia-glcore` but hangs in the present/kick wait + (`glxgears` reproduces it). * No frame-rate or latency numbers. * The published two-VM GZDoom demo videos used the labelled `virtio-gpu`/ `virgl` path, not the emulated vGPU. diff --git a/wiki/Physical-GPU-Backend.md b/wiki/Physical-GPU-Backend.md index 4c86dc4..21ad279 100644 --- a/wiki/Physical-GPU-Backend.md +++ b/wiki/Physical-GPU-Backend.md @@ -5,7 +5,7 @@ > guest-facing emulation. Canonical sources: [`../docs/03-architecture/08-physical-gpu-integration.md`](../docs/03-architecture/08-physical-gpu-integration.md) > and [`../docs/03-architecture/11-physical-backend-guest-wiring.md`](../docs/03-architecture/11-physical-backend-guest-wiring.md); the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` proven with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented default-off; `[wip]` partially built; `[wall]` current blocker; `[target]` planned end state. @@ -123,10 +123,10 @@ same machinery in three shapes (a verbatim 3D method-stream replay, synthesized launches, and the direct GL client's own GR/CE channels); the captured and synthesized streams execute on the engine, and the direct GL client's channels submit. The display/KMS stack registers, the X DDX waits are -fixed and X settles, so the remaining graphics gates are the NVIDIA renderer -path (Vulkan ICD or forced GLX), the DDX push-buffer get-word/notifier -completion, and physical dispatch of the client's draws (`forwarded=0`) -before rendered pixels. The remaining compute work is the staged-transfer +fixed, X settles and forcing GLX selects `libnvidia-glcore`, so the remaining +graphics gates are the kernel-visible GL present completion, the DDX +push-buffer get-word/notifier completion, and physical dispatch of the +client's draws (`forwarded=0`) before rendered pixels. The remaining compute work is the staged-transfer class (under repair), the every-boot acceptance campaign, and per-guest execution isolation — see [`Execution-Frontier.md`](Execution-Frontier.md) and diff --git a/wiki/Project-Goals.md b/wiki/Project-Goals.md index ba557da..ff35ff7 100644 --- a/wiki/Project-Goals.md +++ b/wiki/Project-Goals.md @@ -10,7 +10,7 @@ > [`Execution-Frontier.md`](Execution-Frontier.md) and > [`Graphics-Execution.md`](Graphics-Execution.md). -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` working with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` working with the stock guest driver; `[wip]` in progress; `[planned]` designed, not built; `[target]` bring-up ahead; `[non-goal]` explicitly out of scope. @@ -51,7 +51,7 @@ drivers, no license.** | Stock guest driver initialisation | The unmodified driver probes, initialises and enumerates as on bare metal (`nvidia-smi`) | `[live]` — TU104 reference platform | | Full CUDA driver-API surface | Context creation, allocation, copies, free, destroy complete with correct results | `[live]` — the driver-API smoke prints `ALL OK`; the 64-byte payload round-trips | | CUDA kernel execution on real hardware | Verified guest kernels execute on the physical GPU, with engine-authored completions waking the guest | `[live]` — vector-add, matrix-multiply and transformer clients green on fresh boots; the staged-transfer class, the every-boot acceptance campaign and per-guest isolation are in progress ([`Execution-Frontier.md`](Execution-Frontier.md)) | -| OpenGL (3D graphics) | Stock guest GL applications render real pixels through the emulated vGPU | `[wip]` — the display/KMS stack and X run (NvKms registers, `glxinfo` OpenGL 4.6.0 direct), and the direct GL client's GR/CE channels submit and retire; GZDoom reaches `MAP01` but on Mesa llvmpipe. The NVIDIA renderer path, the DDX get-word/notifier completion and physical dispatch of the GL draws are the frontier ([`OpenGL.md`](OpenGL.md)) | +| OpenGL (3D graphics) | Stock guest GL applications render real pixels through the emulated vGPU | `[wip]` — the display/KMS stack and X run (NvKms registers, `glxinfo` OpenGL 4.6.0 direct), the direct GL client's GR/CE channels submit and retire, and forcing GLX selects `libnvidia-glcore`; the minimal windowed `glxgears` hangs in the glcore kick wait. Kernel-visible present completion, the DDX get-word/notifier completion and physical dispatch of the GL draws are the frontier ([`OpenGL.md`](OpenGL.md)) | | Vulkan | Stock guest Vulkan applications render real pixels | `[wip]` — the pipeline runs end to end (device → command buffer → fence → readback); frames are black — program delivery and program-VA mapping are the frontier ([`Vulkan.md`](Vulkan.md)) | | Windows guests (DirectX, CUDA) | A stock WDDM driver and stock D3D/CUDA applications run in Windows guests | `[target]` — not started; same emulation target ([`DirectX.md`](DirectX.md)) | | Video (NVENC/NVDEC) | Encode/decode through the emulated device | `[planned]` | diff --git a/wiki/Project-Plan.md b/wiki/Project-Plan.md index 583264a..459dc59 100644 --- a/wiki/Project-Plan.md +++ b/wiki/Project-Plan.md @@ -15,7 +15,7 @@ > [`../docs/06-portability/03-device-validation-and-porting-sequence.md`](../docs/06-portability/03-device-validation-and-porting-sequence.md); > the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[done]`, `[active]`, +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[done]`, `[active]`, `[ahead]` — the plan's own state vocabulary, distinct from the goal tags. ## 1. How the plan works @@ -44,7 +44,7 @@ itself: |---|---|---|---| | **P0 — Platform** | Emulated vGPU device, GSP boot and GSP-RM emulation, stock guest driver initialisation | Driver-API smoke green (`ALL OK`) = evaluation rung 2 | `[done]` | | **P1 — First execution (compute)** | Guest CUDA kernels executing on the physical GPU, with engine-authored completions; then hardening | Verified clients + engine-authored completion = rung 3 core; hardening: staged-transfer class, every-boot acceptance, per-guest isolation | `[active]` — execution green; hardening in progress | -| **P2 — Graphics execution** | OpenGL/Vulkan draw commands executed by the physical 3D pipeline and delivered as readable pixels | The graphics ladder: M-GL1 client context-init executes; M-GL2 `glxgears` non-black; M-GL3 GZDoom GL non-black through the mdev | `[active]` — between M-GL1 and M-GL2; the display/KMS and direct-GL channel work is live; the NVIDIA renderer path, the DDX get-word/notifier completion and physical dispatch of the GL draws are the walls | +| **P2 — Graphics execution** | OpenGL/Vulkan draw commands executed by the physical 3D pipeline and delivered as readable pixels | The graphics ladder: M-GL1 client context-init executes; M-GL2 `glxgears` non-black; M-GL3 GZDoom GL non-black through the mdev | `[active]` — between M-GL1 and M-GL2; the display/KMS and direct-GL channel work is live; kernel-visible GL present completion, the DDX get-word/notifier completion and physical dispatch of the GL draws are the walls | | **P3 — Sustained and multi-tenant** | Frame rates without assistance; multiple VMs sharing the GPU with measured fairness; soak-stable operation | Graphics M-GL4 (60 fps, no forcing) and M-GL5 (two VMs, two mdevs); CUDA battery + soak; per-guest execution isolation = rungs 4/5 | `[ahead]` | | **P4 — Breadth** | Video (NVENC/NVDEC), Windows/DirectX guests, live migration, non-KVM hypervisor transports | Per-case bring-up gates plus the matrix's evidence expectations | `[ahead]` | | **P5 — Scale and portability** | New GPU generations and driver families carried forward; published performance programme | A profile + measured delta per version, validated on hardware; A/B numbers published per the measurement contract | `[ahead]` | @@ -57,11 +57,12 @@ available (P5 runs across the whole plan). Ranked, in dependency order: -1. **Graphics pixels (P2).** The client and display stacks execute; the - named walls are the NVIDIA renderer path (Vulkan ICD device creation or - forced GLX), the X DDX push-buffer get-word/notifier completion, the DDX's - 4 GB sysmem allocation (12 GB guest workaround), physical dispatch of the - GL draws (`forwarded=0`), then non-black frames. Full detail: +1. **Graphics pixels (P2).** The client and display stacks execute, and + forcing GLX selects `libnvidia-glcore`; the named walls are the + kernel-visible GL present/kick completion (minimal `glxgears` hangs in it), + the X DDX push-buffer get-word/notifier completion, the DDX's 4 GB sysmem + allocation (12 GB guest workaround), physical dispatch of the GL draws + (`forwarded=0`), then non-black frames. Full detail: [`Graphics-Execution.md`](Graphics-Execution.md) §6. 2. **Compute hardening (P1).** The staged-transfer class (failure modes proven, fixes in flight); the every-boot acceptance campaign; per-guest execution @@ -78,10 +79,10 @@ post-reboot client-stall defect (compute reproducibility); the display/KMS bring-up (NvKms registers, X settles, GL 4.6); the UVM channel-pool tracker; the direct-GL GR release and CE ring discovery; the X DDX kick-channel decode and its three wait fixes (class-aware display arm, `0x8000 -> 0` notification -completion, decoded GL release applied); GZDoom reaching renderer startup and -`MAP01` once the DDX push-buffer loop was unblocked (currently on Mesa -llvmpipe, not the emulated vGPU); and the guest Vulkan pipeline running end to -end. +completion, decoded GL release applied); forced GLX selecting `libnvidia-glcore` +(the EGL/Mesa fallback is gone); GZDoom reaching renderer startup and `MAP01` +once the DDX push-buffer loop was unblocked (a Mesa llvmpipe datapoint); and +the guest Vulkan pipeline running end to end. ## 4. The gates diff --git a/wiki/Project-Status.md b/wiki/Project-Status.md index 4ee4b50..e61d307 100644 --- a/wiki/Project-Status.md +++ b/wiki/Project-Status.md @@ -10,7 +10,7 @@ > (the execution frontier). When this page and the docs disagree, the docs tree > wins. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` proven with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented, default-off or behind supervised gates; `[wip]` partially implemented; `[wall]` current blocker; `[target]` named success state; `[planned]` designed, not built. @@ -28,7 +28,7 @@ named success state; `[planned]` designed, not built. | Physical backend on real hardware (module-owned GPU running its real GSP-RM) | `[live]` boot, RPC session, object creation, copy-engine work and GR execution proven — the verified guest CUDA kernel clients execute through this backend, and graphics method streams execute on the engine ([`Graphics-Execution.md`](Graphics-Execution.md)) | | Display and KMS on the emulated vGPU (nvidia-modeset/NvKms, `/dev/dri`, X) | `[live]` in the campaign configuration — the emulated display engine serves the guest's display push-buffer channels (notifier plus `UPDATE -> FINISHED`), nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`), `nvidia-smi -L` enumerates and X settles; the display-capability seam is parameterised ([`Graphics-Execution.md`](Graphics-Execution.md)) | | Multi-tenant isolation (mdev per VM, per-context page directories) | `[wip]` device-level coexistence works — multiple mediated devices with different emulated VRAM sizes and multiple guests coexist (two guests demonstrated); per-guest execution isolation for simultaneous CUDA guests is in progress | -| OpenGL (3D graphics) | `[wip]` — the display/KMS stack registers and X settles; `glxinfo` reports OpenGL 4.6.0 with direct rendering; the direct GL client's GR and CE channels submit and their releases land; GZDoom reaches renderer startup and `MAP01`, currently on Mesa llvmpipe. **No rendered pixels through the emulated vGPU yet:** the walls are the NVIDIA Vulkan ICD/forced-GLX renderer path, the X DDX push-buffer get-word/notifier completion, the DDX's 4 GB sysmem allocation (12 GB guest workaround), and the direct-GL draws' physical dispatch (`forwarded=0`) ([`OpenGL.md`](OpenGL.md)) | +| OpenGL (3D graphics) | `[wip]` — the display/KMS stack registers and X settles; `glxinfo` reports OpenGL 4.6.0 with direct rendering; with GLX forced (`SDL_VIDEO_X11_FORCE_EGL=0`) the windowed GL path uses `libnvidia-glcore`, and the direct GL client's GR and CE channels submit. **No rendered pixels through the emulated vGPU yet:** the minimal `glxgears` hangs in the glcore kick wait (the client polls the per-slot `/dev/nvidia-modeset` fd; the host completes the beef notifier memory but the guest kernel never signals the event), the X DDX get-word/notifier recycle is not implemented, the DDX's 4 GB sysmem allocation forces a 12 GB guest, and the direct-GL draws still report `forwarded=0` ([`OpenGL.md`](OpenGL.md)) | | Vulkan (3D/compute) | `[wip]` — the guest Vulkan pipeline runs end to end on the emulated vGPU (device, allocation, pipeline, command buffer, fence, queue submit, readback) and the engine executes translated launches; frames are still black — program delivery and program-VA mapping are the frontier ([`Vulkan.md`](Vulkan.md)) | | DirectX / Windows guests | `[planned]` (target) — no Windows guest booted yet; the architecture is the same emulation target ([`DirectX.md`](DirectX.md)) | | Video (NVENC/NVDEC) | `[planned]` | @@ -66,7 +66,7 @@ The project's definition of done for CUDA requires all of the following Until every gate holds, the honest formulation is: **initialization, the driver-API path and kernel execution on the physical GPU work and verify; the staged-transfer class, per-guest isolation and the graphics -renderer/DDX-completion/first-frame frontier remain.** +present-completion/DDX/first-frame frontier remain.** ## 3. How to see it yourself @@ -76,9 +76,10 @@ outcome is exactly the ladder above — driver probe, `nvidia-smi`, the full driver-API smoke, and guest CUDA kernels executing and verifying on the physical GPU. On the graphics side the same setup brings up the display/KMS stack and the X server on the emulated vGPU (`glxinfo` reports OpenGL 4.6.0 -with direct rendering); GZDoom reaches renderer startup but currently falls -back to Mesa llvmpipe, so the NVIDIA renderer path and the DDX completion are -the frontier ([`OpenGL.md`](OpenGL.md)). The +with direct rendering); forcing GLX (`SDL_VIDEO_X11_FORCE_EGL=0`) selects +`libnvidia-glcore`, but the minimal windowed `glxgears` hangs in the glcore +kick wait and the DDX completion is not yet authored, so kernel-visible +completion is the frontier ([`OpenGL.md`](OpenGL.md)). The risc-v-gsp-rm-debugger workbench ([`Riscv-GSP-RM-Debugger.md`](Riscv-GSP-RM-Debugger.md)) is the host-side tooling used to study the firmware side of that frontier. diff --git a/wiki/System-Overview.md b/wiki/System-Overview.md index 333d8c3..de6f632 100644 --- a/wiki/System-Overview.md +++ b/wiki/System-Overview.md @@ -7,7 +7,7 @@ > and [`../docs/03-architecture/10-interaction-and-scheduling.md`](../docs/03-architecture/10-interaction-and-scheduling.md) > for the full picture); the docs tree wins on disagreement. -Status: 2026-11-09, tree `master @ e342bd0f`. Tags: `[live]` proven with the +Status: 2026-11-09, tree `master @ 21e0d611`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented default-off; `[wip]` partial; `[wall]` current blocker; `[target]` designed end state. @@ -142,12 +142,12 @@ RPC from the guest to "firmware": buffers and the display notifier/completion (class-aware arm), so nvidia-modeset/NvKms registers (`/dev/dri`), X settles and `glxinfo` reports OpenGL 4.6.0 with direct rendering; the direct GL client's - channels submit, the X DDX waits are fixed, and the guest Vulkan pipeline - runs end to end. GZDoom reaches renderer startup and `MAP01`, currently on - Mesa llvmpipe. The remaining gate is the NVIDIA renderer path, the DDX - get-word/notifier completion and physical dispatch of the GL draws before - the first rendered (non-black) frame ([`OpenGL.md`](OpenGL.md), - [`Vulkan.md`](Vulkan.md)). + channels submit, the X DDX waits are fixed, forcing GLX selects + `libnvidia-glcore`, and the guest Vulkan pipeline runs end to end. The + minimal windowed `glxgears` hangs in the glcore kick wait, so the remaining + gate is kernel-visible present completion, the DDX get-word/notifier + completion and physical dispatch of the GL draws before the first rendered + (non-black) frame ([`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md)). ## 6. Where the detail lives |
