# Graphics Execution

> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and
> Vulkan command streams a stock guest submits — on the physical GPU, and what
> is proven about it. This is the shared machinery behind
> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX
> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and
> completion path for a different engine workload
> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources:
> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
> (the plan and milestone ladder) and
> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
> (the measured walls); the docs tree wins on disagreement.

Status: 2026-11-09, tree `master @ bb802909`. Tags: `[live]` observed end to
end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined
success state; `[planned]` designed, not built.

## 1. What "real pixels" means

The acceptance target is strict: a **stock** guest GL or Vulkan application's
draw commands are executed by the physical GPU's fixed-function 3D pipeline
(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the
render target is readable by the guest, and that image reaches the capture
path through the emulated mdev. The pixels must not come from another GPU, a
software rasteriser, or a host display round-trip.

(The project has also published two deliberately-labelled two-VM demo videos
that use the standard `virtio-gpu`/`virgl` virtual GPU on a host integrated
GPU, and a CUDA-animation video that *does* run on the emulated vGPU through
the compute path. The `virgl` videos are honest demonstrations of the guest
experience; they are not this execution path, and their captions say so.)

## 2. The path a graphics command takes

```mermaid
graph TD
  APP["Guest applications<br/>OpenGL · Vulkan · CUDA"] --> LIB["Guest user space<br/>GL: GLX client + the driver's GL core<br/>Vulkan: the NVIDIA ICD<br/>CUDA: libcuda"]
  APP --> X["X server (Linux GL)<br/>GLX module + the NVIDIA DDX"]
  LIB --> KMD["Guest kernel: nvidia.ko<br/>RM core, GSP client, channels, modeset/drm"]
  X --> KMD
  KMD -- "GPFIFO rings + doorbell;<br/>method streams in push buffers" --> VG["Host module vgsp.ko<br/>device model + emulation core"]
  VG --> L1["Lane 1: verbatim 3D<br/>method-stream replay"]
  VG --> L2["Lane 2: synthesized launch<br/>from a captured section"]
  VG --> L3["Lane 3: forwarded CUDA<br/>launch (compute)"]
  L1 --> FE["Physical GPU<br/>front-end fetch -> GR engine"]
  L2 --> FE
  L3 --> FE
  FE -- "engine-authored releases<br/>(semaphores in guest memory)" --> VG
  VG -- "completion relay: MSI-X +<br/>poller/event wake the guest" --> KMD
```

```text
guest apps (GL / Vulkan / CUDA)
  -> guest user space (GLX client + GL core, Vulkan ICD, libcuda)
  -> guest X server (GLX module + NVIDIA DDX)           [OpenGL]
  -> nvidia.ko: channels, GPFIFO rings, doorbells       [all APIs]
  -> vgsp.ko: capture sections, decode pushes, forward
  -> physical GPU front-end -> GR engine (3D + compute) / copy engines
  <- engine-authored releases written into guest memory
  <- completion relay (MSI-X + wakes) back to the guest
ASCII fallback above; the diagram shows the three execution lanes.
```

## 3. The three execution lanes

Graphics work reaches the engines today in one of three shapes, in increasing
fidelity:

1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the
   X server's graphics-class (`0xC597`, `TURING_A`) section from its channel,
   relocates the operand addresses into host-built shadow pages the engine can
   reach, and submits the stream unchanged. The engine retires it: the uploads
   (`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and
   the release all execute with no MMU fault. This proves the *mechanics* of
   3D execution — object bind, window placement, operand relocation,
   completion — on real hardware.
2. **Synthesized launch from a captured section** `[live]`. A compute-shaped
   launch built from the captured section (operand list, program, a 64-word
   launch descriptor) reaches the engine and retires real `sm_75` SASS; the
   Turing shader-program-header form (`V3`) is accepted with zero header
   faults across repeated launches. This proved engine-side dispatch before
   the verbatim lane existed. The program itself is still a development
   stand-in at this rung — execution proof, not pixels.
3. **Forwarding the client's own pushes** `[wip]`. The GL client's own
   channels now register, submit and retire: the direct-GL GR channel
   (`0xbeef0100`) and its CE siblings (`0xbeef0101`/`0xbeef0102`) are
   discovered, their rings and releases are decoded, and the X server's DDX
   kick channel (`0xbfef0002`) is decoded from its ring head with its
   class-2D notification authored. The X DDX waits are fixed (class-aware
   display arm, `0x8000 -> 0` notification completion, decoded GL release
   applied), forcing GLX selects `libnvidia-glcore`, and the event-IRQ fix
   (`a478f0b4`) lets the guest service the completion, so glxgears completes
   swaps and the guest's real GL draws now forward and execute on the
   physical GPU (`work_gr backend_ok=5-7`, `backend_timeout=0`) and X is
   reliable, but the draw still writes 0 pages. What remains is the beef GR
   ring resolution race, the fixed-CB image's zero MVP block, and the
   render-target bind (§6).

## 4. The channel and object picture

* **Every submission lives on a channel**: a GPFIFO ring plus engine objects,
  with a USERD the driver advances its put pointer in and a doorbell the host
  module traps. Channels are typed by engine: a graphics channel carries 3D
  state and draws; copy-class channels carry memory movements.
* **The graphics object class** the guest's 3D stream binds is the Turing
  graphics class, `0xC597`. Streams now come from both the **X server's
  channel** and the **direct GL client's own channels** (`0xbeef0100` GR plus
  the `0xbeef0101`/`0xbeef0102` CE siblings); the 2D present/readback work
  rides lighter 2D-class and copy-engine shapes.
* **Display and modeset** `[live]`. The guest's modeset path is served by an
  emulated display engine: the display push-buffer channels (core `0xc57d`,
  window `0xc57e`, immediate/window `0xc57b`) are registered, their push
  buffers parsed, and the notifier/`UPDATE` completion authored. The arm is
  class-aware (the window class has no NOTIFY bit; a bound notifier is the
  arm), and the X DDX's `0x8000 -> 0` notification completion is authored.
  This is what lets nvidia-modeset/NvKms register (`/dev/dri`), X settle and
  GLX clients obtain a real GL context. The display-capability seam is
  parameterised in the campaign configuration.
* **Cells of state**: 3D streams are long—hundreds of method words per draw —
  and mix state binds (viewport, render target, ZS, blend/ROP), program and
  constant binds, vertex/index streams and draw-issuing methods. The capture
  machinery recognises the init-shaped operands today; the per-draw state
  vocabulary is being extended ([G-B in the plan](
  ../docs/04-implementation/05-graphics-execution-plan.md)).
* **Synchronisation**: tracking semaphores, notifications and fence values
  written by the engine; the guest's swap/present waits sit on top of those.

## 5. Completion, fences and presents

The acceptance signal is the **engine-authored release**: a semaphore the
engine writes when the work completes. For GL, completion flows through the
channel's semaphores/notifications to the X server and the app's swap wait;
for Vulkan, the client's fence value is released; for CUDA, the launch's
release group wakes the client. The emulation relays engine completions into
guest-observable memory (MSI-X plus a poller), and the guest resumes from
there.

For the display path, completion is the notifier write and `UPDATE ->
FINISHED`; a completion event deferred by the reply path is force-retried
until it posts, so the X server's mode set finishes. For the X DDX's own 2D
push buffers, completion also requires the GPU-written engine get word to
advance and the notifier to be recycled to `0xffff` after a kick; both are
host-side levers now, and a diagnostic write of the get word released the
DDX's reserve loop. For a windowed GL client the present is a kick; the GSP
event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches
the CPU-tree leaf, so the guest services the event and glxgears completes
swaps, and the GR backend no longer times out (the pair-close/pair-safe
replay fixes are default ON; draws retire in 3-2400 ms). The next completion
step is the beef GR ring (so the fixed-CB scan runs on every cycle) and the
CB image (its `0x80` MVP block is zero even when the scan runs), then the
render-target bind.

During bring-up the module also carries a **diagnostics-only mirror** for apps
that spin on a shared progress word; it exists so a run can proceed while the
natural completion path is being finished. The 60-fps milestone explicitly
requires it switched off, along with any external poke.

## 6. Where the frontier is (ranked walls)

1. **Beef GR ring resolution** `[wall]`. `0xbeef0100` (the GL client GR
   channel) registers with `vas=0` before its CE siblings publish
   `0xbeef0300`; the shared VAS is then registered UNVERIFIED by the 1 ms
   age-out, so the ring at `0x4020000` stays `ring_pa=0` and the fixed-CB scan
   (gated to the `0xbeef0000` family) never runs. The next fix is to gate the
   candidate promote on *verified* rather than *known* and retry every pump
   turn.
2. **The fixed-CB image MVP block** `[wall]`. Even on a cycle where the scan
   runs (`r1687`), it decodes 1492 `0x3800` offsets plus 1497 `0x39c8` data
   runs (1092 words) but the image's `0x80` MVP block is zero. Identified
   parser bugs (subchannel bits 15:13, `0x238c` ONE_INC inline data, selector
   B/C swap) are not yet applied; the MVP may live at an offset >= `0x1000` or
   a selector B/C binding. With a zero MVP every vertex collapses to the
   origin, so there are no fragments.
3. **The render-target bind** `[wall]`. Confirm that the replayed draw's
   render target is the page `bounce copy-out` compares
   (`svc_surf_prelude_capture` currently picks the last trusted bind
   globally).
4. **The X DDX footprint** `[wall]`. The DDX allocates 4 GB of system pages at
   startup; an 8 GB guest has no headroom once GZDoom's own footprint lands
   and both are OOM-killed. Current runs use a 12 GB guest; the allocation is
   a suspected emulation gap (a 512 MiB emulated VRAM falling back to a 4 GB
   sysmem allocation) to fix.
5. **Non-black frames and sustained rate** `[wall]`/`[target]`. Once the beef
   ring, the CB image and the render-target bind land, the capture must show
   motion, then 60 fps in one VM with no forcing, then two VMs on two mediated
   devices with non-black frames in both.

## 7. What is proven today

| Capability | State |
|---|---|
| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` |
| Display engine serves the modeset push buffers; notifier plus `UPDATE -> FINISHED` | `[live]` (campaign configuration) |
| nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`); `nvidia-smi -L` enumerates | `[live]` |
| `glxinfo` reports the emulated GPU with the NVIDIA renderer (OpenGL 4.6.0, direct rendering) | `[live]` |
| X green on attempt 1 in three consecutive cycles (`ebf3130f`) | `[live]` |
| GSP event-IRQ fix wakes the guest poll; glxgears completes swaps | `[live]` |
| 2D path end-to-end: accelerated X fills read back via `xwd` | `[live]` |
| Guest's real GL draws forward and execute on the physical GPU (`work_gr backend_ok=5-7`) | `[live]` |
| GR backend timeouts fixed (`backend_timeout=0`; draws retire in 3-2400 ms) | `[live]` |
| CB-DIAG decodes the draw's fixed-CB stream (1492 `0x3800` offsets, 1092 words) | `[live]` |
| Beef GR channel `0xbeef0100` captures real GL draw sections | `[live]` |
| GL section capture: the X channel's `0xC597` stream decoded | `[live]` |
| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` |
| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` |
| X server survives the run on a 12 GB guest (the DDX's 4 GB sysmem allocation forces the size) | `[live]` |
| Beef GR ring resolves deterministically; fixed-CB scan runs every cycle | `[wall]` |
| Fixed-CB image's `0x80` MVP block becomes non-zero | `[wall]` |
| Render-target bind matches the copy-out page | `[wall]` |
| NVIDIA Vulkan ICD creates a device | `[wall]` |
| Non-black pixels from the emulated vGPU's 3D pipeline | `[wall]` |
| Windows/DirectX guests | `[planned]` |

## 8. What is not claimed

* No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet.
* Display/KMS bring-up (NvKms, `/dev/dri`, X, a real GL context) is not a
  rendered-frame claim; the guest's GL draws now execute on the physical GPU,
  X is reliable and glxgears completes swaps, but the frames are black. The
  beef GR ring resolution, the fixed-CB image's zero MVP block and the
  render-target bind are the current walls.
* No frame-rate or latency numbers: the scheduled measurement is defined
  (60 fps over 60 s, no forcing) but not yet met.
* The `virgl`-based demo videos are a different, labelled path — not this one.

## 9. Related pages

* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status.
* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status.
* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target.
* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain,
  proven end to end.
* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
  is forwarded to.
* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO,
  engines, and page tables.
* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding
  live.
* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
  the canonical plan: gap list, three approaches, and the M-GL1..M-GL6
  milestone ladder with gates.