diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/03-architecture/13-vexecution-plane.md | 33 | ||||
| -rw-r--r-- | docs/05-operations/06-guest-frame-path-and-looking-glass.md | 102 |
2 files changed, 135 insertions, 0 deletions
diff --git a/docs/03-architecture/13-vexecution-plane.md b/docs/03-architecture/13-vexecution-plane.md index caaad01..3d97d07 100644 --- a/docs/03-architecture/13-vexecution-plane.md +++ b/docs/03-architecture/13-vexecution-plane.md @@ -568,6 +568,39 @@ forwards (local evidence and the prepared harness: forwarding-service cost, not engine time — the number to drive down is the 7.9 ms and the per-window round trips. +### 3.5a Copy-out change detection: fingerprint rule + +A forwarded launch stages every referenced data page into the data-bounce +chunk, and after the retire wait the copy-out ships back only the pages the +engine actually modified. That decision used to be a 32-bit additive +checksum of the page's words, compared against a baseline captured at +copy-in. The additive form has a structural blind spot: doubling every word +of a 4 KB page raises each float exponent by one (+2^23 per word), and 1024 +words contribute `1024 * 2^23 = 2^33`, which is exactly `0 (mod 2^32)` — an +in-place power-of-two scale of a whole page (`c[i] *= 2.0f`) left the +checksum bit-identical, the page was classified "engine did not touch", +the write-back was skipped, and the client read the pre-scale values. The +same blindness applies to `*= 0.5f`, `* 4`, `* 8`, etc. Since `r452` +(commit `19399f4f`) every change decision in the copy path — the retire +wait's settle scan, the tracked and gap copy-outs, the window witness, the +late-store witness and the idempotent-launch fingerprint — uses a 64-bit +FNV-1a hash over the page instead; the additive sum survives only as the +all-zero filter. Policy rule for the plane: **a change-detection +fingerprint must not be an algebraic invariant of any plausible kernel +operation** — use a hash, never a sum. + +The reader side of the same decision is the landing table: a D2H copy of a +device VA resolves through the last recorded write of that VA. Since `r453` +the forwarded QMD's release0 is redirected to the module semaphore by +default (`gsp_phys_rmcall_gr_rel_mod=1`): the guest-visible release word is +authored by the host only on the completion edge, after the copy-out has +shipped the engine's stores, so a client's stream-ordered readback can no +longer run ahead of the write-back (the historical shape: the engine's own +release satisfied the guest while the copy-out was still pending). The +engine-completion observable remains the module semaphore, and the +per-outcome authoring path guarantees the guest word is applied on every +outcome (timeout, wedge, no-write). + ### 3.6 Citation map for this section | Claim | Authority | diff --git a/docs/05-operations/06-guest-frame-path-and-looking-glass.md b/docs/05-operations/06-guest-frame-path-and-looking-glass.md new file mode 100644 index 0000000..bc5447a --- /dev/null +++ b/docs/05-operations/06-guest-frame-path-and-looking-glass.md @@ -0,0 +1,102 @@ +# Guest frame path and Looking Glass status (2026-10-21) + +This page records the state of the "show the guest's GPU output on the host" +goal: what is proven, what is built, and exactly which pieces remain. It is +written from live evidence on the vGPU-IO stack (guest CUDA green, ivshmem +devices present, kvmfr module loaded on the host). + +## 1. What is proven + +* **Guest GPU rendering works through CUDA.** `tools/cuda-ab/frame_producer.cu` + renders an animated Julia set on the guest GPU (800x600 RGBA, ~10 fps), + copies the frame to host memory and publishes it into the ivshmem BAR + (`/sys/bus/pci/devices/0000:00:03.0/resource2`, the 128 MiB kvmfr device + QEMU backs with `/dev/shm/kvmfr-<uuid>`), with a small header + (`magic/version/width/height/stride/format/seq/timestamp`) at offset 1 MiB. +* **The host can read those frames.** `tools/lg/frame_reader.py` maps the same + shm file and writes each new sequence number out as a PPM, i.e. a real + GPU-rendered guest frame lands on the host. This is the full transport path + (guest GPU -> guest sysmem -> ivshmem -> host), independent of any display + protocol. +* **The ivshmem devices are visible in the guest** (`lspci`: two + "Inter-VM shared memory" functions; `resource2` of `0000:00:03.0` is the + 128 MiB window, `0000:00:04.0` the 2 MiB cursor window), and the guest can + mmap them (root required for the sysfs resource files). +* **EGL enumerates the NVIDIA device in the guest.** With + `EGL_EXT_platform_device`/`EGL_EXT_device_enumeration` present, + `eglQueryDevicesEXT` returns 3 devices and `eglInitialize` on the NVIDIA + device succeeds (`vendor=NVIDIA version=1.5`). + +## 2. What is blocked, and why + +* **Guest GL contexts do not allocate.** `eglMakeCurrent` on an offscreen + pbuffer fails with `EGL_BAD_ALLOC` (0x3003) on every platform tried + (device, surfaceless, default). Xorg's own failure is the same class: + `(EE) NVIDIA(GPU-0): Failed to allocate VASpace object` then + `Failing initialization of X screen`. The driver cannot allocate GPU virtual + address space for a GL context/surface through the vGPU, so both the X + display path and the EGL offscreen path stop at the same wall. This is the + single remaining blocker for guest OpenGL (and therefore for GZDOOM / + doomsday rendering in the guest). +* **The Looking Glass client (B4) cannot consume our frames directly.** + `looking-glass-client` (installed at `/usr/local/bin/looking-glass-client`, + runs under Xvfb with the `libbfd` shim in `/tmp/opencode/lgcompat`) supports + two transports, LGMP and SPICE. Its `app:shmFile` option still expects an + **LGMP session** (the `KVMFR` session header plus the shared-memory message + queues), not a raw framebuffer. Upstream's frame *producer* (`host/`) is the + Windows DXGI/IDD path. So a Linux guest-side producer must either implement + the LGMP session protocol or we keep the frame reader as the host-side + substitute (which is what `frame_reader.py` is). +* **GZDOOM is not packaged for Ubuntu noble** (`apt-cache policy gzdoom` has + no candidate); `doomsday` + `freedoom` are installed as the fallback, but + doomsday cannot start a renderer without a working GL context (blocker + above). + +## 3. Exact remaining pieces (in order of leverage) + +1. **Fix the guest GL VASpace allocation.** This is an RM/vGPU-side gap: the + driver's `nv_vgpu` VASpace allocation for a GL context fails. Fixing it + unlocks Xorg, EGL offscreen GL, doomsday/GZDOOM, and the standard Looking + Glass guest path (the guest's GL capture driver would then work as + upstream expects). +2. **Implement the LGMP session producer** (or a small Linux capture client) + that writes frames into the ivshmem region with the KVMFR session header + and the frame queue protocol the client expects. The transport half is + already proven; this is protocol work in the guest (or a host-side + substitute that speaks LGMP). +3. **Packaging**: GZDOOM would need a source build or a flatpak/snap; the + doomsday fallback is installed and only blocked by (1). + +## 4. Reproduce + +```bash +# guest (root, because the sysfs resource is root-only): +sudo /tmp/frame_producer 60 800 600 10 # 60 frames, 800x600, 10 fps + +# host: +python3 tools/lg/frame_reader.py --frames 5 --out /tmp/opencode/frames +# -> SAVED seq=N 800x600 fmt=0 -> /tmp/opencode/frames/frame-NNNN.ppm + +# guest EGL probe (shows the enumeration success and the MakeCurrent wall): +gcc -O2 -o egl_frame tools/guest-diag/egl_frame.c -lEGL -lGLESv2 +./egl_frame +``` + +## 5. Session-13 update (2026-10-21, r432→r441) + +* The single-frame path is unchanged and still proven: guest `frame_producer` + (CUDA Julia) -> kvmfr ivshmem BAR -> host `frame_reader.py` -> + `/tmp/opencode/frames/frame-0001.png` (320x240, 1431 colours). +* `tools/lg/frame_sequence.sh` captures an **animated** sequence end to end + (reader first, then N frames from the guest). It is blocked by the + GR no-execute wedge: when the lane is wedged the render launches write zero + pages and the producer republishes stale identical frames (observed: 6/6 + identical MD5s on r438; r439 published 5 identical plus one frame left in + the BAR by the previous boot). A multi-frame animation needs a healthy + lane; the frame pipeline itself is not the blocker. +* The GL wall is untouched: `Xorg :0` -> `(EE) NVIDIA(GPU-0): Failed to + allocate VASpace object`, guest dmesg `NVRM: vaspaceapiConstruct_IMPL: + Could not construct VA space. Status ffff`, i.e. `vmmCreateVaspace` inside + the guest's own RM. Next step: reproduce with the guest RM's + `ResmanDebugLevel` raised (a guest module reload, the service's m177 path) + to capture which inner call returns the generic error. |