path: root/wiki/Graphics-Execution.md

# Graphics Execution

> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and
> Vulkan command streams a stock guest submits — on the physical GPU, and what
> is proven about it. This is the shared machinery behind
> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX
> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and
> completion path for a different engine workload
> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources:
> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
> (the plan and milestone ladder) and
> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
> (the measured walls); the docs tree wins on disagreement.

Status: 2026-11-14, tree `master @ 339da842`. Tags: `[live]` observed end to
end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined
success state; `[planned]` designed, not built.

## 1. What "real pixels" means

The acceptance target is strict: a **stock** guest GL or Vulkan application's
draw commands are executed by the physical GPU's fixed-function 3D pipeline
(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the
render target is readable by the guest, and that image reaches the capture
path through the emulated mdev. The pixels must not come from another GPU, a
software rasteriser, or a host display round-trip.

## 2. The path a graphics command takes

```mermaid
graph TD
  APP["Guest applications<br/>OpenGL · Vulkan · CUDA"] --> LIB["Guest user space<br/>GL: GLX client + the driver's GL core<br/>Vulkan: the NVIDIA ICD<br/>CUDA: libcuda"]
  APP --> X["X server (Linux GL)<br/>GLX module + the NVIDIA DDX"]
  LIB --> KMD["Guest kernel: nvidia.ko<br/>RM core, GSP client, channels, modeset/drm"]
  X --> KMD
  KMD -- "GPFIFO rings + doorbell;<br/>method streams in push buffers" --> VG["Host module vgsp.ko<br/>device model + emulation core"]
  VG --> L1["Lane 1: verbatim 3D<br/>method-stream replay"]
  VG --> L2["Lane 2: synthesized launch<br/>from a captured section"]
  VG --> L3["Lane 3: forwarded CUDA<br/>launch (compute)"]
  L1 --> FE["Physical GPU<br/>front-end fetch -> GR engine"]
  L2 --> FE
  L3 --> FE
  FE -- "engine-authored releases<br/>(semaphores in guest memory)" --> VG
  VG -- "completion relay: MSI-X +<br/>poller/event wake the guest" --> KMD
```

```text
guest apps (GL / Vulkan / CUDA)
  -> guest user space (GLX client + GL core, Vulkan ICD, libcuda)
  -> guest X server (GLX module + NVIDIA DDX)           [OpenGL]
  -> nvidia.ko: channels, GPFIFO rings, doorbells       [all APIs]
  -> vgsp.ko: capture sections, decode pushes, forward
  -> physical GPU front-end -> GR engine (3D + compute) / copy engines
  <- engine-authored releases written into guest memory
  <- completion relay (MSI-X + wakes) back to the guest
ASCII fallback above; the diagram shows the three execution lanes.
```

## 3. The three execution lanes

Graphics work reaches the engines today in one of three shapes, in increasing
fidelity:

1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the
   X server's graphics-class (`0xC597`, `TURING_A`) section from its channel,
   relocates the operand addresses into host-built shadow pages the engine can
   reach, and submits the stream unchanged. The engine retires it: the uploads
   (`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and
   the release all execute with no MMU fault. This proves the *mechanics* of
   3D execution — object bind, window placement, operand relocation,
   completion — on real hardware.
2. **Synthesized launch from a captured section** `[live]`. A compute-shaped
   launch built from the captured section (operand list, program, a 64-word
   launch descriptor) reaches the engine and retires real `sm_75` SASS; the
   Turing shader-program-header form (`V3`) is accepted with zero header
   faults across repeated launches. This proved engine-side dispatch before
   the verbatim lane existed. The program itself is still a development
   stand-in at this rung — execution proof, not pixels.
3. **Forwarding the client's own pushes** `[wip]`. The X DDX channel
   `0x1000014` executes its 3D replay (12 launches per cycle), the 2D path is
   green, and a guest pbuffer `glClear(red)+glFinish+glReadPixels` probe
   returns a real pixel (`pixel=ff0000ff`): the replay and readback machinery
   works. The off-screen frame path then produced **500/500 non-black 640x480
   animated frames** per run and captured videos (45.4 s on 2026-11-11; a
   50 s 640x480 h264 capture on 2026-11-12). X and the stock NVIDIA GLX path
   are fully green (`glxinfo -B` returns 0, OpenGL 4.6.0 direct, `glxgears`
   ~520 FPS), and GZDoom reaches `V_Init: allocate screen` with its GL 4.6.0
   context whenever the shared nvClip `modeNum` is even. The direct-GL
   ring/VAS anchor and USERD machinery are substantially hardened (verified
   page-directory discovery, per-rank content scans, eager push pins,
   client-allocated USERD resolution, foreign-USERD skip, aperture-aware
   fallback, wrapped-PUT acceptance), and the 11-12/11-13 wave closed the X
   side of the path: the DDX idle-DMA/ScreenInit wall, the USERD-latch
   lifetime, the same-pass re-probe thrash, the post-ScreenInit Xorg
   kernel-mode spin (a display-engine GET-past-PUT on a driver ring wrap) and
   the flip-event timeout are all fixed, and captured GL sections now
   execute on the engine (`GR LAUNCH submitted` / `GUEST GR LAUNCH OK`) with
   the data window populated (`n_data` 0 -> 1024). The direct-GL client's
   submission is now **fully decoded**: no client push carries a
   method-grammar draw header — its nine NV906F GPFIFO entries tile one
   contiguous buffer and its per-frame work is carried as rewritten
   data-block blobs plus small real state pushes (324/476/728 B, head
   `90110087 80000800`) that invoke MME macros (the QMD-carrier hypothesis was
   tested and refuted: the entry format has no QMD bit, the blob is not a QMD,
   and no QMD magic exists in the 256 MiB BAR1). Delivery into the lane then
   landed: a content-gated FB-shadow rescue (`e1eac368`) and the LOW-band
   rescue (`b7f09616`) recover the recycled client pushes, the 324-B
   draw-state push opens a `C597` section, is captured and marked for the
   verbatim 3D lane (the first direct-GL stream3d marks), and with the golden
   MME overlay satisfying the prelude gate it **reaches the engine**
   (`GR LAUNCH submitted` + OK, MME-PRELUDE pack 1701 IR words, MME-GOLDEN
   slot 0 accepted). The current wall is **MME macro completion**: the beef
   lane's 2799-word packed program cannot complete inside its <=2048-word
   block together with the 156-400-word draw window, so the load-only carrier
   gate drops every beef window and `live_gen` never promotes; a multi-part
   keep-window mechanism is committed default-off (`b44694b5`) and enabled in
   the campaign parameters (`339da842`), with the A/B pending. RT routing was
   tested and refuted as the store cause (RT0 bind verified 16x, still
   `wrote=0`), and the `CODESRC` fault-park class is prevented (a suspect
   program page is relocated to the engine-mapped window-0 shadow;
   `17c7ed7c`). The window stays black; the window/screen present is still the
   frontier (§6).

## 4. The channel and object picture

* **Every submission lives on a channel**: a GPFIFO ring plus engine objects,
  with a USERD the driver advances its put pointer in and a doorbell the host
  module traps. Channels are typed by engine: a graphics channel carries 3D
  state and draws; copy-class channels carry memory movements.
* **The graphics object class** the guest's 3D stream binds is the Turing
  graphics class, `0xC597`. Streams now come from both the **X server's
  channel** and the **direct GL client's own channels** (`0xbeef0100` GR plus
  the `0xbeef0101`/`0xbeef0102` CE siblings); the 2D present/readback work
  rides lighter 2D-class and copy-engine shapes.
* **Display and modeset** `[live]`. The guest's modeset path is served by an
  emulated display engine: the display push-buffer channels (core `0xc57d`,
  window `0xc57e`, immediate/window `0xc57b`) are registered, their push
  buffers parsed, and the notifier/`UPDATE` completion authored. The arm is
  class-aware (the window class has no NOTIFY bit; a bound notifier is the
  arm), and the X DDX's `0x8000 -> 0` notification completion is authored.
  This is what lets nvidia-modeset/NvKms register (`/dev/dri`), X settle and
  GLX clients obtain a real GL context. The display-capability seam is
  parameterised in the campaign configuration.
* **Cells of state**: 3D streams are long—hundreds of method words per draw —
  and mix state binds (viewport, render target, ZS, blend/ROP), program and
  constant binds, vertex/index streams and draw-issuing methods. The capture
  machinery recognises the init-shaped operands today; the per-draw state
  vocabulary is being extended ([G-B in the plan](
  ../docs/04-implementation/05-graphics-execution-plan.md)).
* **Synchronisation**: tracking semaphores, notifications and fence values
  written by the engine; the guest's swap/present waits sit on top of those.

## 5. Completion, fences and presents

The acceptance signal is the **engine-authored release**: a semaphore the
engine writes when the work completes. For GL, completion flows through the
channel's semaphores/notifications to the X server and the app's swap wait;
for Vulkan, the client's fence value is released; for CUDA, the launch's
release group wakes the client. The emulation relays engine completions into
guest-observable memory (MSI-X plus a poller), and the guest resumes from
there.

For the display path, completion is the notifier write and `UPDATE ->
FINISHED`; a completion event deferred by the reply path is force-retried
until it posts, so the X server's mode set finishes. For the X DDX's own 2D
push buffers, completion also requires the GPU-written engine get word to
advance and the notifier to be recycled to `0xffff` after a kick; both are
host-side levers now, and a diagnostic write of the get word released the
DDX's reserve loop. For a windowed GL client the present is a kick; the GSP
event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches
the CPU-tree leaf, so the guest services the event and glxgears completes
swaps, and the GR backend no longer times out (the pair-close/pair-safe
   replay fixes are default ON; draws retire in 3-2400 ms). The stock GLX
   path is fully green (`glxinfo -B` returns 0 with OpenGL 4.6.0 direct
   rendering; `glxgears` renders ~520 FPS; the Xorg channel-idle WAIT count is
   zero), the DDX 3D replay renders and reads back a pbuffer pixel
   (`pixel=ff0000ff`), the off-screen frame path produced non-black animated
   frames and captured videos (45.4 s and 50 s), the RM write-lock holder
   (Xorg in the DDX vidheap scrub wait) is fixed (`df6e71bb`), the X
   DDX/ScreenInit stability walls are closed, and the direct-GL ring/USERD
   machinery is hardened. The next step is completing the direct-GL MME
   macro program (multi-part keep-window A/B), then the window/screen present
   path.

During bring-up the module also carries a **diagnostics-only mirror** for apps
that spin on a shared progress word; it exists so a run can proceed while the
natural completion path is being finished. The 60-fps milestone explicitly
requires it switched off, along with any external poke.

## 6. Where the frontier is (ranked walls)

1. **Direct-GL MME macro completion** `[wall]`. Delivery to the lane is no
   longer the gap: the submission is fully decoded (nine NV906F entries tile
   one contiguous buffer; no method-grammar draw header exists; the per-frame
   work is data-block blobs plus real state pushes that invoke MME macros),
   the recycled pushes are recovered by the content-gated FB-shadow and
   LOW-band rescues, the client's 324-B draw-state push opens a `C597`
   section, is captured and marked for the verbatim 3D lane, and reaches the
   engine with the golden MME overlay (`GR LAUNCH submitted` + OK,
   MME-PRELUDE pack 1701 IR words, MME-GOLDEN slot 0 accepted). The wall is
   the multi-part program gate: the beef lane's 2799-word packed MME program
   cannot complete inside a <=2048-word block together with its 156-400-word
   draw window, so the load-only carrier gate drops each beef window
   (`MME-LOAD-ONLY carrier ... window=156 ... dropped`) and `live_gen` never
   promotes. A multi-part keep-window mechanism is committed default-off
   (`b44694b5`) and enabled in the campaign parameters (`339da842`); the A/B
   is pending. Two hypotheses were closed on the way: the QMD-carrier theory
   (refuted: the entry format has no QMD bit, the blob is not a QMD, no QMD
   magic in 256 MiB BAR1) and RT routing (the clear targets RT0 while the
   window binds RT1; binding RT0 is verified 16x with correct target/geometry
   but the store still misses, 61/61 `wrote=0`) — the store wall is macro/
   front-end execution, not reachability. Related reliability result: the
   `CODESRC` fault-park class is prevented by relocating a suspect program
   page to the engine-mapped window-0 shadow (`17c7ed7c`; one validated boot:
   31 relocations, 51 GR launches, 79 GP_GET advances, 0 `FAULT_PDE`, 0
   `WEDGE-REARM`, `backend_ok=35`).
2. **Window/screen present** `[wall]`. The pbuffer and off-screen frame paths
   are proven (`pixel=ff0000ff`, 500/500 non-black frames, captured videos),
   but a windowed present does not land: glxgears' window is black, the DDX
   shared-surface `gpuRefCount`/`lastKickoffRefCount` poll (W1) must be
   satisfied through a verified surface VA, and a synchronous window draw
   wedges X.
3. **X screen-init and stability** `[wip]` (was `[wall]`). The idle-DMA/
   ScreenInit class, the USERD-latch lifetime, the same-pass re-probe thrash,
   the post-ScreenInit Xorg kernel-mode spin (display-engine GET-past-PUT on a
   driver ring wrap) and the flip-event timeout are fixed, and X has stayed
   green for >10 minutes in a validated window. What remains is a per-boot
   **first-generation lottery** that became the dominant throughput limiter
   for the graphics A/B runs: only an X started immediately after boot passes
   the DDX DMA init (3/7 windows in one runoff; long dead-start stretches
   later, with one green first-start boot, c382), later DMA instances can
   still fail, and the shared nvClip `modeNum` must be even for the GL
   client's state init (an odd value is guest userspace X-driver state,
   cleared by one X restart — not a host bug). Wrapped PUTs and the kick-idle
   marker/GET publication are handled; a guest kernel panic after a wedge is
   recovered by an offline
   filesystem check.
4. **Non-black window frames and sustained rate** `[wall]`/`[target]`. Once
   the command-stream delivery, the window/screen present and X screen-init
   land, the capture must show motion in a window, then 60 fps in one VM with
   no forcing, then two VMs on two mediated devices with non-black frames in
   both.
4. **Non-black window frames and sustained rate** `[wall]`/`[target]`. Once
   the command-stream delivery, the window/screen present and X screen-init
   land, the capture must show motion in a window, then 60 fps in one VM with
   no forcing, then two VMs on two mediated devices with non-black frames in
   both.

## 7. What is proven today

| Capability | State |
|---|---|
| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` |
| Display engine serves the modeset push buffers; notifier plus `UPDATE -> FINISHED` | `[live]` (campaign configuration) |
| nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`); `nvidia-smi -L` enumerates | `[live]` |
| `glxinfo` reports the emulated GPU with the NVIDIA renderer (OpenGL 4.6.0, direct rendering) | `[live]` |
| X green on attempt 1 in three consecutive cycles (`ebf3130f`); X + stock GLX fully green on the current stack (`glxinfo -B` rc 0, OpenGL 4.6.0 direct, `glxgears` ~520 FPS, Xorg WAIT 0) | `[live]` |
| GSP event-IRQ fix wakes the guest poll; glxgears completes swaps | `[live]` |
| 2D path end-to-end: accelerated X fills read back via `xwd` | `[live]` |
| Guest's real GL draws forward and execute on the physical GPU (`work_gr backend_ok=5-7`) | `[live]` |
| GR backend timeouts fixed (`backend_timeout=0`; draws retire in 3-2400 ms) | `[live]` |
| DDX 3D replay executes (`section captured` + `GR LAUNCH` on `0x1000014`, 12 launches/cycle) | `[live]` |
| First real GPU readback: pbuffer `glClear(red)+glReadPixels` -> `pixel=ff0000ff` | `[live]` |
| First non-black GPU-rendered frames: 500/500 off-screen 640x480 frames per run; 45.4 s video (`r1769`) and a 50 s capture (2026-11-12) | `[live]` |
| Direct-GL GR ring `0xbeef0100` resolves; releases advance; ring/VAS anchor hardened (verified page-directory discovery, eager push pins, per-rank scans) | `[live]` |
| Client-allocated USERD resolved from the channel's memory objects (`hUserdMemory`/`userdOffset`); foreign-USERD adoption skipped; aperture-aware wire fallback | `[live]` |
| Guest adapter init with the all-sysmem instance configuration (`RMInstLoc`) — the BAR2-verify trap | `[live]` |
| X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe thrash, Xorg kernel-mode spin (GET-past-PUT) and flip-event walls closed; X green for >10 minutes (`13a0c5d6`, `acb51d53`, `6ebc7cd4`, `770d6b4a`, `b9213c57`, `29690635`) | `[live]` |
| Captured GL sections execute on the engine (`GR LAUNCH submitted` / `GUEST GR LAUNCH OK`); data window populated (`n_data` 0 -> 1024) | `[live]` |
| Client push bodies decode as real GR method streams above 10 GiB through the VAS walk (live-PA bound raised to the registered guest-RAM top, `436cbf9d`; lookalike gate `7d27df77`) | `[live]` |
| Direct-GL submission fully decoded: nine NV906F entries tile one contiguous buffer; no method-grammar draw header in any push; per-frame work is data-block blobs + real state pushes (324/476/728 B) invoking MME macros; QMD-carrier hypothesis refuted | `[live]` |
| Recycled direct-GL pushes recovered (content-gated FB-shadow rescue `e1eac368`, LOW-band rescue `b7f09616`); client draw-state push opens a `C597` section, captures and is marked for the verbatim 3D lane | `[live]` |
| Direct-GL state push reaches the engine with the golden MME overlay (`GR LAUNCH submitted` + OK; MME-PRELUDE pack 1701 IR words; MME-GOLDEN slot 0 accepted) | `[live]` |
| `CODESRC` fault-park class prevented: suspect program page relocated to the engine-mapped window-0 shadow (`17c7ed7c`; validated boot 31 relocations, 51 GR launches, 79 GP_GET advances, 0 `FAULT_PDE`, 0 `WEDGE-REARM`, `backend_ok=35`) | `[live]` |
| RT routing tested and refuted as the store cause: RT0 bind verified 16x with correct target/geometry, still 61/61 `wrote=0` | `[live]` (negative result) |
| GZDoom passes `V_Init: allocate screen` with GL 4.6.0 when shared nvClip `modeNum` is even | `[live]` |
| RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (`df6e71bb`) | `[live]` |
| GL section capture: the X channel's `0xC597` stream decoded | `[live]` |
| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` |
| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` |
| Emulated FB/GART sizing traps fixed (FB info replies in KB; GART default 256 MiB), removing the 4 GiB guest-allocation class | `[live]` |
| Direct-GL MME macro completion (multi-part keep-window: the 2799-word packed program + 156-400-word draw window cannot fit the <=2048-word block; `live_gen` never promotes) | `[wall]` — mechanism committed `b44694b5` (default off) and enabled in params (`339da842`); A/B pending |
| Window/screen present produces non-black window pixels | `[wall]` |
| X screen-init first-generation lottery (`modeNum` parity; only an immediately-post-boot X passes DDX DMA init; dominant throughput limiter) | `[wip]` |
| Non-black screen pixels from the emulated vGPU's 3D pipeline | `[wall]` |
| Windows/DirectX guests | `[planned]` |

## 8. What is not claimed

* No **window/screen** pixel has been produced by the emulated vGPU's 3D
  pipeline yet; every screen capture so far is black. The proven non-black
  frames and the videos (45.4 s and 50 s) come from the off-screen pbuffer
  path (pbuffer -> sysmem -> kvmfr ivshmem -> host reader), not from a
  windowed X screen capture.
* Display/KMS bring-up (NvKms, `/dev/dri`, X, a real GL context) is not a
  rendered-frame claim. GZDoom reaching `V_Init: allocate screen` proves the
  client's GL init and the direct-GL GR lane, not the game loop or a
  presented frame; the client's draw-state push reaches the engine, but the
  MME macro program does not complete, so its draws do not store and the
  window stays black.
* The engine-zero-write (NOWRITE) campaign is not a rendering result: it and
  the draw census together proved the submission is macro-driven (no
  method-grammar draw header exists), that RT routing is not the cause, and
  that the store wall is macro/front-end execution; they are measurements of
  the wall, not a 3D-pipeline defect or a pixels claim.
* No frame-rate or latency numbers: the scheduled measurement is defined
  (60 fps over 60 s, no forcing) but not yet met.

## 9. Related pages

* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status.
* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status.
* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target.
* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain,
  proven end to end.
* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
  is forwarded to.
* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO,
  engines, and page tables.
* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding
  live.
* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
  the canonical plan: gap list, three approaches, and the M-GL1..M-GL6
  milestone ladder with gates.