summaryrefslogtreecommitdiffstats
path: root/docs/05-operations
diff options
context:
space:
mode:
Diffstat (limited to 'docs/05-operations')
-rw-r--r--docs/05-operations/01-performance-correctness-security.md42
-rw-r--r--docs/05-operations/06-guest-frame-path-and-looking-glass.md102
2 files changed, 126 insertions, 18 deletions
diff --git a/docs/05-operations/01-performance-correctness-security.md b/docs/05-operations/01-performance-correctness-security.md
index 2f10b35..1391426 100644
--- a/docs/05-operations/01-performance-correctness-security.md
+++ b/docs/05-operations/01-performance-correctness-security.md
@@ -17,26 +17,32 @@ Rung 4 SUSTAINED workloads run for hours; faults/errors recoverable
Rung 5 MULTI-TENANT N VMs, isolation + fairness + no cross-talk
```
-**Current rung (2026-10-02):** Rungs 1–2 are green: the stock driver probes,
-`nvidia-smi` enumerates, and the full CUDA driver-API smoke passes
-(`ALL OK` 64/64, copies matched) — `cuInit`, device queries, context creation,
-allocation, copies, free and destroy complete, and the copies are verified
-byte-for-byte. **Rung 3 is partial:** CE-class copies execute on the physical
-copy engine (default off) with engine-authored releases and a fault relay; for
-GR kernel execution a forwarded guest launch now reaches the front-end but the
-inline launch-descriptor install has not been observed (no grid dispatched;
-context validity at dispatch is ruled out —
-[`../04-implementation/03-execution-frontier.md`](../04-implementation/03-execution-frontier.md)).
-Before Rung 3
-is claimed, the
-**reply-fidelity gate** applies: a control whose reply fields the driver
-consumes is not implemented until the reply is coherent (status-only →
-coherent zeroed params → coherent values for consumed fields → exact values) —
-see
-[`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md).
-New to CUDA analysis here: start at
+**Current rung (2026-10-30):** Rungs 1–3 are green for the verified CUDA
+clients: the stock driver probes, `nvidia-smi` enumerates, the full CUDA
+driver-API smoke passes (`ALL OK` 64/64), and guest kernels **execute on the
+physical engine** — vector-add, matrix-multiply and transformer clients verify
+on fresh boots with engine-authored completions relayed to the guest. What
+remains before Rung 3 is claimed outright: the staged-transfer class (some
+large multi-chunk clients still mis-verify; failure modes proven) and the
+every-boot acceptance campaign — plus the **reply-fidelity gate** below. Rung 4
+(sustained) has partial evidence (a 100-launch loop, long soak pending); rung 5
+(multi-tenant) is partial: two mediated devices and two guests coexist, and
+per-guest execution isolation plus fairness measurement are in progress. On the
+graphics side, the X server/GLX run on the emulated vGPU, the engine executes
+GL method streams, and the guest Vulkan pipeline runs end to end; rendered
+(non-black) frames are the frontier
+([`../04-implementation/05-graphics-execution-plan.md`](../04-implementation/05-graphics-execution-plan.md)).
+**No performance numbers are published yet**; the A/B programme in §3 is the
+next measurement step, and the first scheduled public measurement is the
+graphics 60 fps target (no forcing). New to CUDA analysis here: start at
[`../../userspace/cuda/READ-ME-FIRST.md`](../../userspace/cuda/READ-ME-FIRST.md).
+The **reply-fidelity gate** applies to every emulated control: a control whose
+reply fields the driver consumes is not implemented until the reply is
+coherent (status-only → coherent zeroed params → coherent values for consumed
+fields → exact values) — see
+[`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md).
+
## 2. Correctness
### 2.1 Test corpus
diff --git a/docs/05-operations/06-guest-frame-path-and-looking-glass.md b/docs/05-operations/06-guest-frame-path-and-looking-glass.md
new file mode 100644
index 0000000..bc5447a
--- /dev/null
+++ b/docs/05-operations/06-guest-frame-path-and-looking-glass.md
@@ -0,0 +1,102 @@
+# Guest frame path and Looking Glass status (2026-10-21)
+
+This page records the state of the "show the guest's GPU output on the host"
+goal: what is proven, what is built, and exactly which pieces remain. It is
+written from live evidence on the vGPU-IO stack (guest CUDA green, ivshmem
+devices present, kvmfr module loaded on the host).
+
+## 1. What is proven
+
+* **Guest GPU rendering works through CUDA.** `tools/cuda-ab/frame_producer.cu`
+ renders an animated Julia set on the guest GPU (800x600 RGBA, ~10 fps),
+ copies the frame to host memory and publishes it into the ivshmem BAR
+ (`/sys/bus/pci/devices/0000:00:03.0/resource2`, the 128 MiB kvmfr device
+ QEMU backs with `/dev/shm/kvmfr-<uuid>`), with a small header
+ (`magic/version/width/height/stride/format/seq/timestamp`) at offset 1 MiB.
+* **The host can read those frames.** `tools/lg/frame_reader.py` maps the same
+ shm file and writes each new sequence number out as a PPM, i.e. a real
+ GPU-rendered guest frame lands on the host. This is the full transport path
+ (guest GPU -> guest sysmem -> ivshmem -> host), independent of any display
+ protocol.
+* **The ivshmem devices are visible in the guest** (`lspci`: two
+ "Inter-VM shared memory" functions; `resource2` of `0000:00:03.0` is the
+ 128 MiB window, `0000:00:04.0` the 2 MiB cursor window), and the guest can
+ mmap them (root required for the sysfs resource files).
+* **EGL enumerates the NVIDIA device in the guest.** With
+ `EGL_EXT_platform_device`/`EGL_EXT_device_enumeration` present,
+ `eglQueryDevicesEXT` returns 3 devices and `eglInitialize` on the NVIDIA
+ device succeeds (`vendor=NVIDIA version=1.5`).
+
+## 2. What is blocked, and why
+
+* **Guest GL contexts do not allocate.** `eglMakeCurrent` on an offscreen
+ pbuffer fails with `EGL_BAD_ALLOC` (0x3003) on every platform tried
+ (device, surfaceless, default). Xorg's own failure is the same class:
+ `(EE) NVIDIA(GPU-0): Failed to allocate VASpace object` then
+ `Failing initialization of X screen`. The driver cannot allocate GPU virtual
+ address space for a GL context/surface through the vGPU, so both the X
+ display path and the EGL offscreen path stop at the same wall. This is the
+ single remaining blocker for guest OpenGL (and therefore for GZDOOM /
+ doomsday rendering in the guest).
+* **The Looking Glass client (B4) cannot consume our frames directly.**
+ `looking-glass-client` (installed at `/usr/local/bin/looking-glass-client`,
+ runs under Xvfb with the `libbfd` shim in `/tmp/opencode/lgcompat`) supports
+ two transports, LGMP and SPICE. Its `app:shmFile` option still expects an
+ **LGMP session** (the `KVMFR` session header plus the shared-memory message
+ queues), not a raw framebuffer. Upstream's frame *producer* (`host/`) is the
+ Windows DXGI/IDD path. So a Linux guest-side producer must either implement
+ the LGMP session protocol or we keep the frame reader as the host-side
+ substitute (which is what `frame_reader.py` is).
+* **GZDOOM is not packaged for Ubuntu noble** (`apt-cache policy gzdoom` has
+ no candidate); `doomsday` + `freedoom` are installed as the fallback, but
+ doomsday cannot start a renderer without a working GL context (blocker
+ above).
+
+## 3. Exact remaining pieces (in order of leverage)
+
+1. **Fix the guest GL VASpace allocation.** This is an RM/vGPU-side gap: the
+ driver's `nv_vgpu` VASpace allocation for a GL context fails. Fixing it
+ unlocks Xorg, EGL offscreen GL, doomsday/GZDOOM, and the standard Looking
+ Glass guest path (the guest's GL capture driver would then work as
+ upstream expects).
+2. **Implement the LGMP session producer** (or a small Linux capture client)
+ that writes frames into the ivshmem region with the KVMFR session header
+ and the frame queue protocol the client expects. The transport half is
+ already proven; this is protocol work in the guest (or a host-side
+ substitute that speaks LGMP).
+3. **Packaging**: GZDOOM would need a source build or a flatpak/snap; the
+ doomsday fallback is installed and only blocked by (1).
+
+## 4. Reproduce
+
+```bash
+# guest (root, because the sysfs resource is root-only):
+sudo /tmp/frame_producer 60 800 600 10 # 60 frames, 800x600, 10 fps
+
+# host:
+python3 tools/lg/frame_reader.py --frames 5 --out /tmp/opencode/frames
+# -> SAVED seq=N 800x600 fmt=0 -> /tmp/opencode/frames/frame-NNNN.ppm
+
+# guest EGL probe (shows the enumeration success and the MakeCurrent wall):
+gcc -O2 -o egl_frame tools/guest-diag/egl_frame.c -lEGL -lGLESv2
+./egl_frame
+```
+
+## 5. Session-13 update (2026-10-21, r432→r441)
+
+* The single-frame path is unchanged and still proven: guest `frame_producer`
+ (CUDA Julia) -> kvmfr ivshmem BAR -> host `frame_reader.py` ->
+ `/tmp/opencode/frames/frame-0001.png` (320x240, 1431 colours).
+* `tools/lg/frame_sequence.sh` captures an **animated** sequence end to end
+ (reader first, then N frames from the guest). It is blocked by the
+ GR no-execute wedge: when the lane is wedged the render launches write zero
+ pages and the producer republishes stale identical frames (observed: 6/6
+ identical MD5s on r438; r439 published 5 identical plus one frame left in
+ the BAR by the previous boot). A multi-frame animation needs a healthy
+ lane; the frame pipeline itself is not the blocker.
+* The GL wall is untouched: `Xorg :0` -> `(EE) NVIDIA(GPU-0): Failed to
+ allocate VASpace object`, guest dmesg `NVRM: vaspaceapiConstruct_IMPL:
+ Could not construct VA space. Status ffff`, i.e. `vmmCreateVaspace` inside
+ the guest's own RM. Next step: reproduce with the guest RM's
+ `ResmanDebugLevel` raised (a guest module reload, the service's m177 path)
+ to capture which inner call returns the generic error.