# Real-render walls: Vulkan and Xorg on the emulated vGPU

Status: OPEN (2026-10-29, SESSION 64). This page records the measured state
of the guest's *display/graphics* consumers - the Vulkan ICD and the Xorg
server - on the emulated vGPU, with the exact evidence and the next
subsystem to implement. It complements
[05-operations/06-guest-frame-path-and-looking-glass.md](../05-operations/06-guest-frame-path-and-looking-glass.md)
(which covers the CUDA -> ivshmem -> host frame transport that is already
proven).

## 1. What works today

* The guest's shipped 570.144 Vulkan userspace **loads and enumerates**:
  `vkCreateInstance` succeeds and two physical devices are reported -
  `NVIDIA GeForce RTX 2080 SUPER` (discrete, Vulkan 1.4.303) and the
  software `llvmpipe` - from `/etc/vulkan/icd.d/nvidia_icd.json`. Probes:
  `tools/vk/vk_probe.py`, `tools/vk/vk_probe2.py`.
* `eglInitialize` succeeds with `vendor=NVIDIA` on the guest EGL device
  platform (the EGL probe `tools/guest-diag/egl_retry.c` and the older
  `egl_frame`).
* CUDA is unaffected (its clients run through the same emulated GSP; see
  the battery evidence in the handoff).

## 2. Wall 1 - `vkCreateDevice` fails in nvidia-modeset's ALLOC_DEVICE

`vkCreateDevice` returns `VK_ERROR_INITIALIZATION_FAILED`. An
`LD_PRELOAD` ioctl logger (`tools/guest-diag/nvkmstrace.c`) shows the
ICD's first nvKms ioctl failing:

```
NVKMS_IOCTL cmd=0 size=1488 addr=0x... rc=-1   # NVKMS_IOCTL_ALLOC_DEVICE
```

i.e. the kernel's `nvKmsAllocDevice` (in `nvidia-modeset`) returned a
non-success status; the userspace frees everything and reports the
generic initialization error. Reading the driver's own modeset source
(`nvkms-rm.c`) gives the alloc order and the two gates we hit and fixed:

1. `ProbeHeadCountAndWindowAssignment()` -> `NV0073_CTRL_CMD_SYSTEM_GET_NUM_HEADS`
   (`0x730102`): a zero reply logs `No heads found on board!` and fails the
   alloc. The emulated GSP RSP answered all NV0073 controls through the
   generic zero-reply path.
2. After the heads: `NoValidDisplays()` -> `NV0073_CTRL_CMD_SYSTEM_GET_SUPPORTED`
   (`0x730107`) must report at least one display id, and the connector
   path (`SPECIFIC_OR_GET_INFO` `0x73028b`, `SPECIFIC_GET_CONNECTOR_DATA`
   `0x730250`, `DFP_GET_INFO` `0x731140`) must describe a connector.

The host GSP-RM emulation now implements a minimal headless display:
one head (`0x730102`/`0x730287`), flexible window assignment (`0x7302ad`),
one static HDMI connector on display id 0 (SOR + single-link TMDS, which
avoids the DP-AUX library), plus DDC support. With this shim the alloc
gets past heads and `NoValidDisplays` and proceeds through the connector
queries and the display-object allocations (classes `0xc570`, `0xc57d`,
`0xc372`, controls `0x50700709`, `0x73010c`, `0xc3700201`) before failing
later in the display-core/modeset-channel path. That remaining subsystem
is the wall: the modeset core channel and the per-display dpy objects
must be emulated (a coupled object + channel + memory set whose exact
wire forms should be captured from a real-GPU session).

A related experiment ruled out one shortcut: booting the guest with
`nvidia-modeset`/`nvidia-drm` blacklisted (no `/dev/nvidia-modeset`) does
not avoid the wall - the ICD's nvKms client still issues ALLOC_DEVICE and
fails, so the display path cannot be sidestepped by disabling KMS.

## 3. Wall 2 - Xorg ScreenInit fails in nvidia-push's channel test

Xorg starts (the GPU is detected, `Virtual screen 1280x1024`) and then:

```
(EE) NVIDIA(GPU-0): Failed to initialize DMA.
(EE) NVIDIA(0): Failed to allocate push buffer
```

`nvidia-push` logs `Failed to initialize DMA.` from
`__nvPushTestPushBuffer()`: right after allocating a push buffer the
driver writes a NOP and waits (10 s) for the channel to consume it. Host
evidence shows why it never does:

* the X driver's channel registers (`class 0xc46f` GPFIFO channel,
  `gpfifo=0x120090000`, `userd=0x350000` sysmem) and lives with
  **`submissions=0`** - its published GP_PUT never advances in the host's
  view, because the host needs the channel's VA->PA translations to read
  its USERD/GPFIFO and that resolution comes from the guest-side scanner
  that currently follows *CUDA clients only* (`targeted resolve
  va=0x20400f000: full RAM scan found nothing`);
* on a busy boot (Xorg started while a CUDA battery ran) the channel's
  completion-counter slot is rejected: `counter auto h=0x1000013 rank=0
  pa=0x11c827ff0 rejected (slot owned by another channel)` - two
  concurrent clients both derive rank 0's slot from the per-device
  structural formula.

Fixing Xorg therefore needs (1) a platform-wide (non-CUDA) VA-resolution
path for the scanner or the host, and (2) counter-slot ownership keyed by
client identity (or a per-client slot domain), not just rank.

## 4. Wall 3 - the EGL/glsi engine-init gate (unchanged)

`eglMakeCurrent` returns `EGL_BAD_ALLOC (0x3003)`; the glsi engine-init
entry returns a "busy" status once, sets a state bit, and the eglcore
caller does not retry. Ten in-process retries (`egl_retry`) all fail, so
the state machine fails on a real input - the return-value trace of its
callees remains the way in (see
[userspace/cuda/570.144/docs/ANALYSIS-egl-engine-selection.md](../../userspace/cuda/570.144/docs/ANALYSIS-egl-engine-selection.md)).

## 5. Why this matters for the frame path

Everything downstream of a working GL/Vulkan context is already in place:
a guest publisher writes frames into the kvmfr ivshmem BAR
(`tools/vk/vk_frame.c`, `tools/guest-diag/xshm_pub.c` use the same layout
as the proven CUDA `frame_producer`), the host reader decodes them
(`tools/lg/frame_reader.py`) and ffmpeg assembles video. The missing
piece is a *working* graphics API on the guest GPU - the two subsystems
above.
## 6. UPDATE 2026-10-25 (RESUME #56): the wall is now fully bisected

Three new instruments settled the remaining questions:

* **An `LD_PRELOAD` nvKms ioctl logger** (`tools/guest-diag/nvkmstrace.c`)
  decodes the failing `NVKMS_IOCTL_ALLOC_DEVICE` reply: status **5 =
  `NVKMS_ALLOC_DEVICE_STATUS_CORE_CHANNEL_ALLOC_FAILED`**.
* **The QEMU gdb stub (`-gdb tcp::1234`) as a guest-kernel microscope**
  (the guest kernel's `nvidia_modeset` symbols are in `/proc/kallsyms`):
  breaking inside `nvAllocCoreChannelEvo` (the blob's offsets were mapped
  from a disassembly of the shipped `nv-modeset-kernel.o_binary`) dumped
  the driver's class list (42 classes; all `NVC57x` display classes are
  present; **`NV927C_BASE_CHANNEL_DMA` (0x927c) is not** - it exists in no
  HAL class list, so the "base channel" fallback path can never succeed).
  Breaking at the capability parse showed the **capability notifier buffer
  is all zeros** (nothing emulates the display engine that would fill it).
* **The caps-fill experiment**: with gdb writing `SYS_CAP = HEAD0_EXISTS`
  and `SYS_CAPB = WINDOW0..7_EXISTS` into the caps buffer right before the
  parse, the driver flips from the dead base path to the **window path**
  (`nvRMAllocateWindowChannels`) - the hypothesis that `numWindows > 0`
  unblocks the flow is confirmed.

The window path then fails one step later: `nvRmApiAlloc(display,
class=0xc57e NVC57E_WINDOW_CHANNEL_DMA)` returns **`NV_ERR_NOT_SUPPORTED`
(0x56) locally in the guest kernel** (it never reaches the emulated
GSP - no ALLOC RPC is logged). This matches the shipped driver's vGPU
architecture: the guest runs in vGPU mode (`IS_VIRTUAL`), where the
display/modeset channel classes are rejected (the reference source's
`_rpcAllocObjectPrologue` documents "Display classes are added for DCE,
vGPU Guest does not support those") - a vGPU guest's display is meant to
be emulated by the **host-side vGPU display service** (the vGPU RPC
display path), which the emulation does not implement.

**Consequence for the roadmap.** Guest Xorg/GL/Vulkan on this emulated
vGPU needs (a) the display capability fill on the host side (mechanics
proven by the gdb experiment) and (b) a vGPU display service: the
display-engine MMIO/RPC backend the guest expects from its host (the
equivalent of NVIDIA's vGPU display plugin), or the specific vGPU
configuration that makes the kernel accept the display channel class.
Until (b) exists the guest's NVIDIA display path is closed by design, and
GZDOOM-class workloads cannot render on the guest GPU.