diff options
Diffstat (limited to 'docs/04-implementation')
| -rw-r--r-- | docs/04-implementation/04-real-render-walls.md | 110 |
1 files changed, 109 insertions, 1 deletions
diff --git a/docs/04-implementation/04-real-render-walls.md b/docs/04-implementation/04-real-render-walls.md index 3570769..4298772 100644 --- a/docs/04-implementation/04-real-render-walls.md +++ b/docs/04-implementation/04-real-render-walls.md @@ -110,4 +110,112 @@ a guest publisher writes frames into the kvmfr ivshmem BAR as the proven CUDA `frame_producer`), the host reader decodes them (`tools/lg/frame_reader.py`) and ffmpeg assembles video. The missing piece is a *working* graphics API on the guest GPU - the two subsystems -above.
\ No newline at end of file +above. +## 6. UPDATE 2026-10-25 (RESUME #56): the wall is now fully bisected + +Three new instruments settled the remaining questions: + +* **An `LD_PRELOAD` nvKms ioctl logger** (`tools/guest-diag/nvkmstrace.c`) + decodes the failing `NVKMS_IOCTL_ALLOC_DEVICE` reply: status **5 = + `NVKMS_ALLOC_DEVICE_STATUS_CORE_CHANNEL_ALLOC_FAILED`**. +* **The QEMU gdb stub (`-gdb tcp::1234`) as a guest-kernel microscope** + (the guest kernel's `nvidia_modeset` symbols are in `/proc/kallsyms`): + breaking inside `nvAllocCoreChannelEvo` (the blob's offsets were mapped + from a disassembly of the shipped `nv-modeset-kernel.o_binary`) dumped + the driver's class list (42 classes; all `NVC57x` display classes are + present; **`NV927C_BASE_CHANNEL_DMA` (0x927c) is not** - it exists in no + HAL class list, so the "base channel" fallback path can never succeed). + Breaking at the capability parse showed the **capability notifier buffer + is all zeros** (nothing emulates the display engine that would fill it). +* **The caps-fill experiment**: with gdb writing `SYS_CAP = HEAD0_EXISTS` + and `SYS_CAPB = WINDOW0..7_EXISTS` into the caps buffer right before the + parse, the driver flips from the dead base path to the **window path** + (`nvRMAllocateWindowChannels`) - the hypothesis that `numWindows > 0` + unblocks the flow is confirmed. + +The window path then fails one step later: `nvRmApiAlloc(display, +class=0xc57e NVC57E_WINDOW_CHANNEL_DMA)` returns **`NV_ERR_NOT_SUPPORTED` +(0x56) locally in the guest kernel** (it never reaches the emulated +GSP - no ALLOC RPC is logged). This matches the shipped driver's vGPU +architecture: the guest runs in vGPU mode (`IS_VIRTUAL`), where the +display/modeset channel classes are rejected (the reference source's +`_rpcAllocObjectPrologue` documents "Display classes are added for DCE, +vGPU Guest does not support those") - a vGPU guest's display is meant to +be emulated by the **host-side vGPU display service** (the vGPU RPC +display path), which the emulation does not implement. + +**Consequence for the roadmap.** Guest Xorg/GL/Vulkan on this emulated +vGPU needs (a) the display capability fill on the host side (mechanics +proven by the gdb experiment) and (b) a vGPU display service: the +display-engine MMIO/RPC backend the guest expects from its host (the +equivalent of NVIDIA's vGPU display plugin), or the specific vGPU +configuration that makes the kernel accept the display channel class. +Until (b) exists the guest's NVIDIA display path is closed by design, and +GZDOOM-class workloads cannot render on the guest GPU. +## 7. UPDATE 2026-10-25 #2 (SESSION 66): the 0x56 pinned to the RS alloc path + +A new instrument closed the gap: the guest's `nvidia.ko` (DKMS build) is +**not stripped** - copy it out (`/lib/modules/<kver>/updates/dkms/nvidia.ko`), +`objdump -t`/`-dr` it, compute the module's `.text` runtime base from any +`/proc/kallsyms` symbol, and every guest-kernel function/offset becomes +addressable for live gdb breakpoints on the QEMU stub. With it: + +* `rpcAllocObject_v29_06`/`_rpcAllocObjectPrologue` are **exonerated** for + 0xc57e (their reject list is 0x73/0xc67*/0xc770/0xc77d only; the register + mapping `hClass = r8d` was verified live). +* The failing chain is pinned exactly: `rmapiAllocWithSecInfo` **enters** + with class 0xc57e, calls `serverAllocResourceUnderLock`, and that + function returns **0x56** - *before* the GSP-RM alloc RPC + (`rpcRmApiAlloc_GSP`, called from under-lock+0x447; the emulation never + sees an ALLOC for 0xc57e). The same probe shows 0xc57d/0xc570/0xc573 + return 0. +* So the last blocker for the window path is a **local RS-layer validation + inside `serverAllocResourceUnderLock`** (candidates: the inlined + privilege/class-support checks, `rmapiFixupAllocParams`' per-class fixup, + or `clientAllocResource`'s construct dispatch). The 0x56-producing + instruction is not yet pinned - a `stepi` loop over the QEMU stub is far + too slow and wedged the guest (recovered by a fresh cycle); the next + session should use conditional breakpoints on the function's call-return + sites (the approach in `gdbdiag19` works; the earlier run had truncated + addresses and needs the `0xffffffff00000000` base restored) or a + watchpoint on the status word. + +**gdb-stub field notes**: the stub is single-client; killing an attached +gdb leaves the VM stopped (recover with a detach-only connect); +`stepi` loops over the stub freeze the guest's userspace for minutes - +prefer breakpoints; every probe run with the caps-fill breakpoint changes +the driver's path, so keep the fill in each experiment. + +## 8. UPDATE 2026-10-25 #3 (SESSION 126): the fence fold landed - the service authors the Vulkan fence releases from the driver's own push bytes; the compute-dispatch shape decoded (direct C5C0 method stream, no inline QMD) + +**The fence wall is closed.** The client's per-frame fence wait polls a +16 KB BO word that a submit-push INC-5 semaphore group targets +(`[2005c017][0x45ba090][0][payload 1][0][0x02100001]`). The group sits in a +GPFIFO entry beyond the driver's kick boundary (GP_PUT=11 while the ring +already carries the entries it pre-built for the next submits, idx12..46), +so the GP_PUT-bounded pump walk never decoded it and a host-side assist had +to write the word. The service now sweeps that pre-built continuation +itself: `sema_gr_ahead` (default 64) walks entries beyond GP_PUT through the +ring's validated translation, accepts only pushes that pass the strict +method-stream gate, extracts the addr-0x5c RELEASE groups and applies them +forward-only through the standard pending ledger. Live evidence: the sweep +finds the groups and the ledger applies the fence word once per frame; +`VKFRAME seq=1..3/DONE` completes with no assist. The frames are still +black - the shader work itself is unexecuted. + +**The compute-dispatch shape (why nothing executes).** The workload is a +single compute dispatch (640x480 storage image). Its push is a DIRECT +method-stream programming of the Turing compute class: `SET_OBJECT 0xC5C0` +on subchannel 1, then the context windows (`0x7b4` local-memory window B, +`0x2a4` shared-memory window B, `0x270`), an immediate-config block, and +the code/const data runs (`0x3800`/`0x39C8` pairs plus a subchannel-0 data +port stream of `0x11c`/`0x114` runs in a separate 17 KB push). There is NO +inline-QMD write (no `0x318/0x31c/0x320..0x41c` run), so the launch capture +that feeds the physical-engine forward path never fires - and the CE-shaped +ops path is not even consulted for GR-class channels by construction. Real +pixels therefore require a translator that recognizes the C5C0 section, +builds the vendor launch the physical channel expects (the same prelude/QMD +shape the CUDA path already executes) from the section's fields and data +runs, and forwards it with the existing operand/relocation machinery. The +method vocabulary overlaps the physical prelude's existing knowledge +(`0x7b0/0x7b4`, `0x2a4`), so the landmarks for that translator are known. |