From 5d72e017ceb99b7d65ff0aafbed351d59497f8c2 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Fri, 30 Oct 2026 13:08:30 -0400 Subject: site: 2026-10-30 wiki refresh - new Project Goals, Performance and Interfaces and Stability pages; GSP explainer expanded (bare-metal behaviour + the two emulation modes); graphics pages (Graphics Execution, OpenGL, Vulkan, DirectX) with two new diagrams; status refresh across 15 pages; canonical docs updated (README status table, purpose-and-scope, performance contract); published to open-iov.org (67 pages, 0 problems) --- wiki/Graphics-Execution.md | 182 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 wiki/Graphics-Execution.md (limited to 'wiki/Graphics-Execution.md') diff --git a/wiki/Graphics-Execution.md b/wiki/Graphics-Execution.md new file mode 100644 index 0000000..48423f8 --- /dev/null +++ b/wiki/Graphics-Execution.md @@ -0,0 +1,182 @@ +# Graphics Execution + +> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and +> Vulkan command streams a stock guest submits — on the physical GPU, and what +> is proven about it. This is the shared machinery behind +> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX +> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and +> completion path for a different engine workload +> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources: +> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) +> (the plan and milestone ladder) and +> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md) +> (the measured walls); the docs tree wins on disagreement. + +Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` observed end to +end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined +success state; `[planned]` designed, not built. + +## 1. What "real pixels" means + +The acceptance target is strict: a **stock** guest GL or Vulkan application's +draw commands are executed by the physical GPU's fixed-function 3D pipeline +(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the +render target is readable by the guest, and that image reaches the capture +path through the emulated mdev. The pixels must not come from another GPU, a +software rasteriser, or a host display round-trip. + +(The project has also published two deliberately-labelled two-VM demo videos +that use the standard `virtio-gpu`/`virgl` virtual GPU on a host integrated +GPU, and a CUDA-animation video that *does* run on the emulated vGPU through +the compute path. The `virgl` videos are honest demonstrations of the guest +experience; they are not this execution path, and their captions say so.) + +## 2. The path a graphics command takes + +```mermaid +graph TD + APP["Guest applications
OpenGL · Vulkan · CUDA"] --> LIB["Guest user space
GL: GLX client + the driver's GL core
Vulkan: the NVIDIA ICD
CUDA: libcuda"] + APP --> X["X server (Linux GL)
GLX module + the NVIDIA DDX"] + LIB --> KMD["Guest kernel: nvidia.ko
RM core, GSP client, channels, modeset/drm"] + X --> KMD + KMD -- "GPFIFO rings + doorbell;
method streams in push buffers" --> VG["Host module vgsp.ko
device model + emulation core"] + VG --> L1["Lane 1: verbatim 3D
method-stream replay"] + VG --> L2["Lane 2: synthesized launch
from a captured section"] + VG --> L3["Lane 3: forwarded CUDA
launch (compute)"] + L1 --> FE["Physical GPU
front-end fetch -> GR engine"] + L2 --> FE + L3 --> FE + FE -- "engine-authored releases
(semaphores in guest memory)" --> VG + VG -- "completion relay: MSI-X +
poller/event wake the guest" --> KMD +``` + +```text +guest apps (GL / Vulkan / CUDA) + -> guest user space (GLX client + GL core, Vulkan ICD, libcuda) + -> guest X server (GLX module + NVIDIA DDX) [OpenGL] + -> nvidia.ko: channels, GPFIFO rings, doorbells [all APIs] + -> vgsp.ko: capture sections, decode pushes, forward + -> physical GPU front-end -> GR engine (3D + compute) / copy engines + <- engine-authored releases written into guest memory + <- completion relay (MSI-X + wakes) back to the guest +ASCII fallback above; the diagram shows the three execution lanes. +``` + +## 3. The three execution lanes + +Graphics work reaches the engines today in one of three shapes, in increasing +fidelity: + +1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the + X server's graphics-class (`0xC597`, `TURING_A`) section from its channel, + relocates the operand addresses into host-built shadow pages the engine can + reach, and submits the stream unchanged. The engine retires it: the uploads + (`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and + the release all execute with no MMU fault. This proves the *mechanics* of + 3D execution — object bind, window placement, operand relocation, + completion — on real hardware. +2. **Synthesized launch from a captured section** `[live]`. A compute-shaped + launch built from the captured section (operand list, program, a 64-word + launch descriptor) reaches the engine and retires real `sm_75` SASS; the + Turing shader-program-header form (`V3`) is accepted with zero header + faults across repeated launches. This proved engine-side dispatch before + the verbatim lane existed. The program itself is still a development + stand-in at this rung — execution proof, not pixels. +3. **Forwarding the client's own pushes** `[wall]`. The goal state: the GL + client's own GPFIFO ring and method streams are read, decoded and executed + with the client's real per-draw state. The GL client's channel is created + today, but it never submits — its pushes have not been made readable — and + this lane is gated on the ring/push-readability work (§6). + +## 4. The channel and object picture + +* **Every submission lives on a channel**: a GPFIFO ring plus engine objects, + with a USERD the driver advances its put pointer in and a doorbell the host + module traps. Channels are typed by engine: a graphics channel carries 3D + state and draws; copy-class channels carry memory movements. +* **The graphics object class** the guest's 3D stream binds is the Turing + graphics class, `0xC597`. The stream the emulation can capture today comes + from the **X server's channel** (`OpenGL`); the 2D present/readback work + rides lighter 2D-class and copy-engine shapes. +* **Cells of state**: 3D streams are long—hundreds of method words per draw — + and mix state binds (viewport, render target, ZS, blend/ROP), program and + constant binds, vertex/index streams and draw-issuing methods. The capture + machinery recognises the init-shaped operands today; the per-draw state + vocabulary is being extended ([G-B in the plan]( + ../docs/04-implementation/05-graphics-execution-plan.md)). +* **Synchronisation**: tracking semaphores, notifications and fence values + written by the engine; the guest's swap/present waits sit on top of those. + +## 5. Completion, fences and presents + +The acceptance signal is the **engine-authored release**: a semaphore the +engine writes when the work completes. For GL, completion flows through the +channel's semaphores/notifications to the X server and the app's swap wait; +for Vulkan, the client's fence value is released; for CUDA, the launch's +release group wakes the client. The emulation relays engine completions into +guest-observable memory (MSI-X plus a poller), and the guest resumes from +there. + +During bring-up the module also carries a **diagnostics-only mirror** for apps +that spin on a shared progress word; it exists so a run can proceed while the +natural completion path is being finished. The 60-fps milestone explicitly +requires it switched off, along with any external poke. + +## 6. Where the frontier is (ranked walls) + +1. **The GL client's submissions** `[wall]`. The client's channel exists but + never submits: direct measurement shows the client process writes no put + pointer in any mapping the service can see, and its pushes are not yet + reachable. Everything downstream waits on this. +2. **Program (SASS) delivery** `[wall]`. Graphics programs reach the device + through the driver's copy/utility-channel uploads; those inline-data + writes are not yet materialised for the GL/Vulkan clients, and the program + page the engine fetches is empty. With the data-set gate bypassed, the + engine fetches the launch and MMU-faults (`FAULT_PDE`) at the program VA — + proof the channel and context are alive, and that mapping + content are + the missing pair. +3. **Non-black frames** `[wall]`. Once draws execute with real state and real + programs, the capture must show motion; every frame captured so far is + black. +4. **Sustained frame rate, and two VMs** `[target]`. The defined end state: + 60 fps in one VM with no forcing, then two VMs on two mediated devices + with non-black frames in both. + +## 7. What is proven today + +| Capability | State | +|---|---| +| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` | +| `glxinfo` enumerates the emulated GPU with the NVIDIA renderer | `[live]` | +| GZDoom launches and runs on the emulated vGPU | `[live]` (pixels frontier) | +| GL section capture: the X server's `0xC597` stream decoded | `[live]` | +| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` | +| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` | +| 2D screen fills and copy-engine present ops execute | `[live]` | +| Direct GL client: device-init gate handled, channels allocate, readback path returns clean | `[wip]` (client submissions not yet) | +| Non-black pixels from the emulated vGPU's 3D pipeline | `[wall]` | +| Windows/DirectX guests | `[planned]` | + +## 8. What is not claimed + +* No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet. +* No frame-rate or latency numbers: the scheduled measurement is defined + (60 fps over 60 s, no forcing) but not yet met. +* The `virgl`-based demo videos are a different, labelled path — not this one. + +## 9. Related pages + +* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status. +* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status. +* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target. +* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain, + proven end to end. +* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work + is forwarded to. +* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO, + engines, and page tables. +* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding + live. +* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) — + the canonical plan: gap list, three approaches, and the M-GL1..M-GL6 + milestone ladder with gates. -- cgit v1.2.3