path: root/wiki/OpenGL.md

# OpenGL

> **Scope.** How a stock guest OpenGL application's commands travel through the
> unmodified NVIDIA stack on the emulated vGPU, and the honest state of running
> OpenGL on it. The host-side execution machinery shared by all graphics APIs
> is on [`Graphics-Execution.md`](Graphics-Execution.md); CUDA compute is on
> [`Execution-Frontier.md`](Execution-Frontier.md). Canonical sources:
> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
> (the plan and milestones),
> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
> (the measured walls) and
> [`../docs/03-architecture/02-guest-stack-and-cudasw.md`](../docs/03-architecture/02-guest-stack-and-cudasw.md)
> (the guest stack). The docs tree wins on disagreement.

Status: 2026-11-15, tree `master @ 6aeb20c7`. Tags: `[live]` observed with the
stock guest driver; `[wip]` partial; `[wall]` current blocker; `[target]`
defined success state.

## 1. The guest OpenGL stack

On Linux, an OpenGL application does not talk to the kernel driver directly:
it talks to the GL implementation, and — when a display is involved — through
the X server's GLX extension:

```text
application (toolkit / SDL / GLFW / raw GLX)
  -> libGL (dispatch)                    -> libGLX_nvidia (NVIDIA GLX client)
  -> libnvidia-glcore (the vendor GL engine: state, programs, shaders, draws)
  -> ioctls + mappings to nvidia.ko      -> channels, buffers, synchronization
X server (Xorg)
  -> GLX extension module (glxserver_nvidia) and the NVIDIA DDX (X driver)
  -> owns the screen, drawables and the present path
```

Two forms of operation exist, and both are visible in the guest:

* **Direct rendering** — the normal desktop case. The GL client builds GPU
  commands in its own process and the X server coordinates buffers and
  presents. The client's channel is the source of truth for per-draw state.
* **Indirect GLX** — the client's GL protocol is serialised to the X server,
  which executes it against its own GL context. Here the X server's channel
  carries the graphics stream.

On the emulated vGPU the readable command sources today are the **direct GL
client's own channels** — the GR channel `0xbeef0100` and its CE siblings
`0xbeef0101`/`0xbeef0102` — and the **X server's channels**, whose
graphics-class (`0xC597`, `TURING_A`) sections the emulation captures and
whose DDX kick channel (`0xbfef0002`) is decoded from its ring head
([`Graphics-Execution.md`](Graphics-Execution.md) §3).

## 2. What an NVIDIA GL frame involves (observed shapes)

* **Context and state setup** — shader programs (SASS) uploaded to device
  memory, constant/descriptor banks, render targets and depth surfaces
  allocated and mapped.
* **Per-draw work** — a graphics-class method stream: state binds (viewport,
  render target, ZS, blend/ROP), program and constant binds, vertex/index
  data, and draw-issuing methods that drive the fixed-function pipeline
  (vertex fetch, primitive assembly, rasterisation, fragment work, ROP).
* **Program/descriptor delivery** — the driver uploads shader code and
  descriptors through its copy/utility-channel inline-data path. The GL draw
  path decodes these uploads and the engine executes the draw; Vulkan's
  inline-to-memory arm is the remaining mirror-image gap
  ([`Vulkan.md`](Vulkan.md)).
* **Present** — the X server's present/readback path runs as light 2D-class
  and copy-engine operations; the emulator decodes and executes those shapes.

## 3. Status on the emulated vGPU

Rated honestly, against the milestone ladder in the plan:

| Item | State |
|---|---|
| X server initialises on the emulated vGPU (driver push-DMA sanity test, ScreenInit) | `[live]` |
| nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`); `nvidia-smi -L` enumerates | `[live]` (campaign configuration) |
| `glxinfo -B` reports the NVIDIA renderer (OpenGL 4.6.0, driver 570.144, direct rendering) | `[live]` |
| The stock GLX path is fully green: `glxinfo -B` returns 0, `glxgears` renders ~520 FPS, Xorg WAIT count 0 | `[live]` |
| The X driver's RC-watchdog channel registers and its notifier is authored | `[live]` |
| X green on attempt 1 in three consecutive cycles (`ebf3130f`) | `[live]` |
| GSP event-IRQ fix wakes the guest poll; glxgears completes swaps | `[live]` |
| 2D path end-to-end: accelerated X fills read back via `xwd` | `[live]` |
| Guest's real GL draws forward and execute on the physical GPU (`work_gr backend_ok=5-7`) | `[live]` |
| GR backend timeouts fixed (`backend_timeout=0`; draws retire in 3-2400 ms) | `[live]` |
| DDX 3D replay executes (`section captured` + `GR LAUNCH` on `0x1000014`, 12 launches/cycle) | `[live]` |
| First real GPU readback: pbuffer `glClear(red)+glReadPixels` -> `pixel=ff0000ff` | `[live]` |
| First non-black GPU-rendered frames: 500/500 off-screen 640x480 frames per run; 45.4 s video (`r1769`) and a 50 s capture (2026-11-12) | `[live]` |
| Direct-GL GR ring `0xbeef0100` resolves; releases advance; ring/VAS anchor hardened (verified page-directory discovery, eager push pins, per-rank scans) | `[live]` |
| Client-allocated USERD resolved from the channel's memory objects (`hUserdMemory`/`userdOffset`); foreign-USERD adoption skipped; aperture-aware wire fallback | `[live]` |
| X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe thrash, Xorg kernel-mode spin (GET-past-PUT) and flip-event walls closed; X green >10 minutes (`13a0c5d6`, `acb51d53`, `6ebc7cd4`, `770d6b4a`, `b9213c57`, `29690635`) | `[live]` |
| Captured GL sections execute on the engine (`GR LAUNCH submitted` / `GUEST GR LAUNCH OK`); data window populated (`n_data` 0 -> 1024) | `[live]` |
| Client push bodies decode as real GR method streams above 10 GiB via the VAS walk (live-PA bound raised to the registered guest-RAM top, `436cbf9d`) | `[live]` |
| Direct-GL submission fully decoded: nine NV906F entries tile one contiguous buffer; no method-grammar draw header; per-frame work is data-block blobs + real state pushes (324/476/728 B) invoking MME macros; QMD-carrier hypothesis refuted | `[live]` |
| Recycled direct-GL pushes recovered (content-gated FB-shadow rescue `e1eac368`, LOW-band rescue `b7f09616`); the 324-B draw-state push opens a `C597` section, captures and is marked for the verbatim 3D lane | `[live]` |
| Direct-GL state push reaches the engine with the golden MME overlay (`GR LAUNCH submitted` + OK; MME-PRELUDE pack 1701 IR words; MME-GOLDEN slot 0 accepted) | `[live]` |
| `CODESRC` fault-park class prevented (program page relocated to the engine-mapped window-0 shadow, `17c7ed7c`; validated boot 31 relocations, 51 GR launches, 79 GP_GET advances, 0 `FAULT_PDE`, 0 `WEDGE-REARM`, `backend_ok=35`) | `[live]` |
| RT routing tested and refuted as the store cause: RT0 bind verified 16x with correct target/geometry, still 61/61 `wrote=0` | `[live]` (negative result) |
| GZDoom passes `V_Init: allocate screen` with GL 4.6.0 when shared nvClip `modeNum` is even | `[live]` |
| RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (`df6e71bb`) | `[live]` |
| Engine executes a launch built from the captured section (real `sm_75` SASS retired) | `[live]` |
| Verbatim 3D method-stream replay retired by the engine, no MMU fault | `[live]` |
| Emulated FB/GART sizing traps fixed (FB info replies in KB; GART default 256 MiB), removing the 4 GiB guest-allocation class | `[live]` |
| Multi-part MME keep-window: multi-frame windows admitted; the 2799-word pack completes (2799/2799) | `[live]` |
| Part-launch poison eliminated: client `SET_REPORT_SEMAPHORE`/`SET_NOTIFY` targets redirected to a mapped scratch slot (49 cycles, 0 poison, 0 MMU faults) | `[live]` |
| First `MME-PACK-REARM` captured (engine-global completion plus keep-done-across-recycle); fixed-CB region aliased; released calls run macros | `[live]` |
| `DRAW_IAB` (`mac=0x22`) executes on the 324-B window class after a cross-lane `MME-PACK-REARM`; SPH-BIND prelude emits 84 words; DS stage VAs populate | `[live]` |
| DS stage-0/1 image: a VTG SPH is authored at `family+0x900` and the window's own slot-1 bind rewritten to it; the DS latch reads `st0=st1=020a0481` (`0x86040626 -> 0x84000420`, two boots) | `[live]` |
| Saved-window release carrier: the held window is snapshotted at `MME-CALL-HOLD` and re-submitted with calls released once the pack is resident (c772: save 240/68 -> fire -> synthetic `GR LAUNCH 0x6d6d6500` -> REARM -> DS `0x84000420`) | `[live]` |
| Held-CALL census + SA[34] golden fill: the slot-34 `DRAW_IAB` call is present in the saved 728-B window; `client=golden=used=0x265` (`MME-GOLDEN sa-fill slot=34 entry=0x265`) | `[live]` |
| Call order: the 728-B window places slot-34 at `off=0x340` after the `0x33/0x39` setup, so the `0x39` DS latch pre-empts CALL(34) (c780: `mac=0x33/0x39/0x0`); the reorder `a03a68ea` is landed but unexercised | `[wall]` |
| Pack-completion lottery (poisoned pack-part; no pump pass after the carry arm; carry-armed-never-fired) gates the deterministic release | `[wall]` |
| DS `0x84000420` (SPH5/SPH10/SPH18 stage type/context) after the draw-issue call; stores `wrote=0` | `[wall]` |
| Window/screen present produces non-black screen pixels | `[wall]` |
| X screen-init first-generation lottery (`modeNum` parity; only an immediately-post-boot X passes DDX DMA init; dominant throughput limiter) | `[wip]` |
| 60 fps sustained, no forcing (M-GL4) | `[target]` |

## 4. How OpenGL runs are verified

Instruments that matter, in order of usefulness: the X server log and
`xdpyinfo` (did the server initialise?), `glxinfo -B` (which renderer is
real), the application's own progress output, and frame capture at the GL
source (a swap-hook capture is the ground truth for pixels — the X screen
itself is not, on some stacks). Captures are decoded offline and scored:
fraction of pixels above a luminance threshold, colour count, and
frame-to-frame motion. A run counts only when the capture shows motion with
the NVIDIA renderer in use.

## 5. Milestones on this path

The graphics plan defines M-GL1 (client context-init decodes and executes; the
client leaves its retry loop), M-GL2 (`glxgears` non-black), M-GL3 (GZDoom GL
non-black through the mdev), M-GL4 (60 fps, no forcing), M-GL5 (two VMs, two
mdevs), M-GL6 (split-screen video). The project sits **between M-GL1 and
M-GL2, with M-GL2's off-screen half proven**: the display/KMS stack registers,
X and the stock GLX path are fully green (`glxinfo -B` rc 0, ~520 FPS
`glxgears`, Xorg WAIT 0), the DDX/X-stability walls were closed (X green for
more than 10 minutes), the direct-GL ring/USERD machinery is hardened, and the
GPU-rendered pbuffer path produced non-black animated frames and captured
videos (45.4 s and 50 s). Captured GL sections now execute on the engine, and
the direct-GL client's submission is fully decoded: it carries no classic
draw header — the work is macro-driven (data-block blobs plus small state
pushes invoking MME macros), the recycled pushes are recovered, and the
client's draw-state push opens, captures and reaches the engine. The MME chain
now completes and executes the draw macro: multi-frame keep-windows land, the
2799-word pack completes (2799/2799), the first `MME-PACK-REARM` fires, the
fixed-CB region is aliased, and `DRAW_IAB` (`mac=0x22`) runs with the SPH-BIND
prelude intact (84 words) and the DS stage VAs populated. The **DS stage-0/1
image is now authored and live** (a VTG SPH at `family+0x900`, first word
`020a0481`, with the window's own slot-1 bind rewritten to it): the DS latch
reads `st0=st1=020a0481 code=ff007321` and the DS value falls
`0x86040626 -> 0x84000420`, reproduced on two boots. The release path is now
host-deterministic when the pack completes: the saved-window release carrier
snapshots the held window and re-submits it with the calls released once the
pack is resident (c772 end to end), and the held-CALL census plus the SA[34]
golden fill prove the slot-34 `DRAW_IAB` call and its target are present
(`MME-CALL-TARGET slot=34 method=0x3910 client=0x265 golden=0x265
used=0x265`). The remaining blocker is **call order**: the saved 728-B state
window places the slot-34 run after the `0x33/0x39` setup, so the last `0x39`
latches DS `0x84000420` and the MFE never reaches CALL(34) (c780); the 324-B
DRAW window places it first and ran `mac=0x22` (c716). A release-window
reorder (`a03a68ea`) is landed but not yet exercised, and pack completion is
still a per-boot lottery (`6aeb20c7`, eng64, chains the release behind a
successful pack carrier and names the stalled gate; validation pending).
After that come the remaining DS stage type/context
bits (SPH5/SPH10/SPH18), the stores, the window/screen present path
(glxgears' window is black and a window draw can wedge X), and the per-boot
binding/X first-generation start lottery — then the first rendered screen
frame.

## 6. What is not claimed

* No **window/screen** frame has rendered real pixels through the emulated
  vGPU yet; every screen capture so far is black. The non-black frames and the
  videos (45.4 s and 50 s) come from the off-screen pbuffer path (pbuffer ->
  sysmem -> kvmfr ivshmem -> host reader), not from a windowed X screen
  capture. The client's draw-state push reaches the engine, the MME program
  completes and the draw macro executes, and the authored DS stage image is
  fetched by both vertex slots; the release carrier can now replay the held
  window deterministically when the pack completes, but the 728-B window
  places the slot-34 draw-issue call after the setup calls (so the DS latch
  pre-empts it) and the stores are still missing, a synchronous window draw
  can wedge X, and X start retains a per-boot first-generation lottery. Those
  are the current walls.
* GZDoom reaching `V_Init: allocate screen` is not a rendered-frame claim; it
  proves the client's GL init and the direct-GL GR lane, not the game loop or
  a presented frame.
* No frame-rate or latency numbers.

## 7. Related pages

* [`Graphics-Execution.md`](Graphics-Execution.md) — the shared execution
  machinery and the ranked walls.
* [`Vulkan.md`](Vulkan.md) — the sibling API path, further along in the
  pipeline lifecycle.
* [`DirectX.md`](DirectX.md) — the Windows/DirectX target.
* [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — how the NVIDIA
  userspace stack is analysed.
* [`Install-and-First-Run.md`](Install-and-First-Run.md) — bringing the stack
  up.
* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
  the canonical plan, gap list and gates.