path: root/wiki/OpenGL.md

# OpenGL

> **Scope.** How a stock guest OpenGL application's commands travel through the
> unmodified NVIDIA stack on the emulated vGPU, and the honest state of running
> OpenGL on it. The host-side execution machinery shared by all graphics APIs
> is on [`Graphics-Execution.md`](Graphics-Execution.md); CUDA compute is on
> [`Execution-Frontier.md`](Execution-Frontier.md). Canonical sources:
> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
> (the plan and milestones),
> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
> (the measured walls) and
> [`../docs/03-architecture/02-guest-stack-and-cudasw.md`](../docs/03-architecture/02-guest-stack-and-cudasw.md)
> (the guest stack). The docs tree wins on disagreement.

Status: 2026-11-14, tree `master @ 339da842`. Tags: `[live]` observed with the
stock guest driver; `[wip]` partial; `[wall]` current blocker; `[target]`
defined success state.

## 1. The guest OpenGL stack

On Linux, an OpenGL application does not talk to the kernel driver directly:
it talks to the GL implementation, and — when a display is involved — through
the X server's GLX extension:

```text
application (toolkit / SDL / GLFW / raw GLX)
  -> libGL (dispatch)                    -> libGLX_nvidia (NVIDIA GLX client)
  -> libnvidia-glcore (the vendor GL engine: state, programs, shaders, draws)
  -> ioctls + mappings to nvidia.ko      -> channels, buffers, synchronization
X server (Xorg)
  -> GLX extension module (glxserver_nvidia) and the NVIDIA DDX (X driver)
  -> owns the screen, drawables and the present path
```

Two forms of operation exist, and both are visible in the guest:

* **Direct rendering** — the normal desktop case. The GL client builds GPU
  commands in its own process and the X server coordinates buffers and
  presents. The client's channel is the source of truth for per-draw state.
* **Indirect GLX** — the client's GL protocol is serialised to the X server,
  which executes it against its own GL context. Here the X server's channel
  carries the graphics stream.

On the emulated vGPU the readable command sources today are the **direct GL
client's own channels** — the GR channel `0xbeef0100` and its CE siblings
`0xbeef0101`/`0xbeef0102` — and the **X server's channels**, whose
graphics-class (`0xC597`, `TURING_A`) sections the emulation captures and
whose DDX kick channel (`0xbfef0002`) is decoded from its ring head
([`Graphics-Execution.md`](Graphics-Execution.md) §3).

## 2. What an NVIDIA GL frame involves (observed shapes)

* **Context and state setup** — shader programs (SASS) uploaded to device
  memory, constant/descriptor banks, render targets and depth surfaces
  allocated and mapped.
* **Per-draw work** — a graphics-class method stream: state binds (viewport,
  render target, ZS, blend/ROP), program and constant binds, vertex/index
  data, and draw-issuing methods that drive the fixed-function pipeline
  (vertex fetch, primitive assembly, rasterisation, fragment work, ROP).
* **Program/descriptor delivery** — the driver uploads shader code and
  descriptors through its copy/utility-channel inline-data path. The GL draw
  path decodes these uploads and the engine executes the draw; Vulkan's
  inline-to-memory arm is the remaining mirror-image gap
  ([`Vulkan.md`](Vulkan.md)).
* **Present** — the X server's present/readback path runs as light 2D-class
  and copy-engine operations; the emulator decodes and executes those shapes.

## 3. Status on the emulated vGPU

Rated honestly, against the milestone ladder in the plan:

| Item | State |
|---|---|
| X server initialises on the emulated vGPU (driver push-DMA sanity test, ScreenInit) | `[live]` |
| nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`); `nvidia-smi -L` enumerates | `[live]` (campaign configuration) |
| `glxinfo -B` reports the NVIDIA renderer (OpenGL 4.6.0, driver 570.144, direct rendering) | `[live]` |
| The stock GLX path is fully green: `glxinfo -B` returns 0, `glxgears` renders ~520 FPS, Xorg WAIT count 0 | `[live]` |
| The X driver's RC-watchdog channel registers and its notifier is authored | `[live]` |
| X green on attempt 1 in three consecutive cycles (`ebf3130f`) | `[live]` |
| GSP event-IRQ fix wakes the guest poll; glxgears completes swaps | `[live]` |
| 2D path end-to-end: accelerated X fills read back via `xwd` | `[live]` |
| Guest's real GL draws forward and execute on the physical GPU (`work_gr backend_ok=5-7`) | `[live]` |
| GR backend timeouts fixed (`backend_timeout=0`; draws retire in 3-2400 ms) | `[live]` |
| DDX 3D replay executes (`section captured` + `GR LAUNCH` on `0x1000014`, 12 launches/cycle) | `[live]` |
| First real GPU readback: pbuffer `glClear(red)+glReadPixels` -> `pixel=ff0000ff` | `[live]` |
| First non-black GPU-rendered frames: 500/500 off-screen 640x480 frames per run; 45.4 s video (`r1769`) and a 50 s capture (2026-11-12) | `[live]` |
| Direct-GL GR ring `0xbeef0100` resolves; releases advance; ring/VAS anchor hardened (verified page-directory discovery, eager push pins, per-rank scans) | `[live]` |
| Client-allocated USERD resolved from the channel's memory objects (`hUserdMemory`/`userdOffset`); foreign-USERD adoption skipped; aperture-aware wire fallback | `[live]` |
| X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe thrash, Xorg kernel-mode spin (GET-past-PUT) and flip-event walls closed; X green >10 minutes (`13a0c5d6`, `acb51d53`, `6ebc7cd4`, `770d6b4a`, `b9213c57`, `29690635`) | `[live]` |
| Captured GL sections execute on the engine (`GR LAUNCH submitted` / `GUEST GR LAUNCH OK`); data window populated (`n_data` 0 -> 1024) | `[live]` |
| Client push bodies decode as real GR method streams above 10 GiB via the VAS walk (live-PA bound raised to the registered guest-RAM top, `436cbf9d`) | `[live]` |
| Direct-GL submission fully decoded: nine NV906F entries tile one contiguous buffer; no method-grammar draw header; per-frame work is data-block blobs + real state pushes (324/476/728 B) invoking MME macros; QMD-carrier hypothesis refuted | `[live]` |
| Recycled direct-GL pushes recovered (content-gated FB-shadow rescue `e1eac368`, LOW-band rescue `b7f09616`); the 324-B draw-state push opens a `C597` section, captures and is marked for the verbatim 3D lane | `[live]` |
| Direct-GL state push reaches the engine with the golden MME overlay (`GR LAUNCH submitted` + OK; MME-PRELUDE pack 1701 IR words; MME-GOLDEN slot 0 accepted) | `[live]` |
| `CODESRC` fault-park class prevented (program page relocated to the engine-mapped window-0 shadow, `17c7ed7c`; validated boot 31 relocations, 51 GR launches, 79 GP_GET advances, 0 `FAULT_PDE`, 0 `WEDGE-REARM`, `backend_ok=35`) | `[live]` |
| RT routing tested and refuted as the store cause: RT0 bind verified 16x with correct target/geometry, still 61/61 `wrote=0` | `[live]` (negative result) |
| GZDoom passes `V_Init: allocate screen` with GL 4.6.0 when shared nvClip `modeNum` is even | `[live]` |
| RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (`df6e71bb`) | `[live]` |
| Engine executes a launch built from the captured section (real `sm_75` SASS retired) | `[live]` |
| Verbatim 3D method-stream replay retired by the engine, no MMU fault | `[live]` |
| Emulated FB/GART sizing traps fixed (FB info replies in KB; GART default 256 MiB), removing the 4 GiB guest-allocation class | `[live]` |
| Direct-GL MME macro completion (multi-part keep-window: 2799-word packed program + 156-400-word draw window cannot fit the <=2048-word block; `live_gen` never promotes) | `[wall]` — mechanism committed `b44694b5` (default off) and enabled in params (`339da842`); A/B pending |
| Window/screen present produces non-black screen pixels | `[wall]` |
| X screen-init first-generation lottery (`modeNum` parity; only an immediately-post-boot X passes DDX DMA init; dominant throughput limiter) | `[wip]` |
| 60 fps sustained, no forcing (M-GL4) | `[target]` |

## 4. How OpenGL runs are verified

Instruments that matter, in order of usefulness: the X server log and
`xdpyinfo` (did the server initialise?), `glxinfo -B` (which renderer is
real), the application's own progress output, and frame capture at the GL
source (a swap-hook capture is the ground truth for pixels — the X screen
itself is not, on some stacks). Captures are decoded offline and scored:
fraction of pixels above a luminance threshold, colour count, and
frame-to-frame motion. A run counts only when the capture shows motion with
the NVIDIA renderer in use.

## 5. Milestones on this path

The graphics plan defines M-GL1 (client context-init decodes and executes; the
client leaves its retry loop), M-GL2 (`glxgears` non-black), M-GL3 (GZDoom GL
non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two
mdevs), M-GL6 (split-screen video). The project sits **between M-GL1 and
M-GL2, with M-GL2's off-screen half proven**: the display/KMS stack registers,
X and the stock GLX path are fully green (`glxinfo -B` rc 0, ~520 FPS
`glxgears`, Xorg WAIT 0), the DDX/X-stability walls were closed (X green for
more than 10 minutes), the direct-GL ring/USERD machinery is hardened, and the
GPU-rendered pbuffer path produced non-black animated frames and captured
videos (45.4 s and 50 s). Captured GL sections now execute on the engine, and
the direct-GL client's submission is fully decoded: it carries no classic
draw header — the work is macro-driven (data-block blobs plus small state
pushes invoking MME macros), the recycled pushes are recovered, and the
client's draw-state push opens, captures and reaches the engine. The
remaining gates are **MME macro completion** (the 2799-word packed program
cannot complete inside its <=2048-word block with the 156-400-word draw
window, so `live_gen` never promotes; the multi-part keep-window mechanism is
committed default-off and enabled for the pending A/B), the window/screen
present path (glxgears' window is black and a window draw can wedge X), and
the X first-generation screen-init lottery — then the first rendered screen
frame.

## 6. What is not claimed

* No **window/screen** frame has rendered real pixels through the emulated
  vGPU yet; every screen capture so far is black. The non-black frames and the
  videos (45.4 s and 50 s) come from the off-screen pbuffer path (pbuffer ->
  sysmem -> kvmfr ivshmem -> host reader), not from a windowed X screen
  capture. The client's draw-state push now reaches the engine, but the MME
  macro program does not complete so the draw never stores, a synchronous
  window draw wedges X, and X start retains a per-boot first-generation
  lottery. Those are the current walls.
* GZDoom reaching `V_Init: allocate screen` is not a rendered-frame claim; it
  proves the client's GL init and the direct-GL GR lane, not the game loop or
  a presented frame.
* No frame-rate or latency numbers.

## 7. Related pages

* [`Graphics-Execution.md`](Graphics-Execution.md) — the shared execution
  machinery and the ranked walls.
* [`Vulkan.md`](Vulkan.md) — the sibling API path, further along in the
  pipeline lifecycle.
* [`DirectX.md`](DirectX.md) — the Windows/DirectX target.
* [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — how the NVIDIA
  userspace stack is analysed.
* [`Install-and-First-Run.md`](Install-and-First-Run.md) — bringing the stack
  up.
* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
  the canonical plan, gap list and gates.