summaryrefslogtreecommitdiffstats
path: root/wiki/Graphics-Execution.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-30 13:08:30 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-30 13:08:30 -0400
commit5d72e017ceb99b7d65ff0aafbed351d59497f8c2 (patch)
tree7300b046c1a40156c142a1aba1957d25d4e3ea3e /wiki/Graphics-Execution.md
parent0b0b86d88cd1ef499db6817e605fff888bc7a16c (diff)
downloadopen-iov-5d72e017ceb99b7d65ff0aafbed351d59497f8c2.tar.gz
open-iov-5d72e017ceb99b7d65ff0aafbed351d59497f8c2.zip
site: 2026-10-30 wiki refresh - new Project Goals, Performance and Interfaces and Stability pages; GSP explainer expanded (bare-metal behaviour + the two emulation modes); graphics pages (Graphics Execution, OpenGL, Vulkan, DirectX) with two new diagrams; status refresh across 15 pages; canonical docs updated (README status table, purpose-and-scope, performance contract); published to open-iov.org (67 pages, 0 problems)
Diffstat (limited to 'wiki/Graphics-Execution.md')
-rw-r--r--wiki/Graphics-Execution.md182
1 files changed, 182 insertions, 0 deletions
diff --git a/wiki/Graphics-Execution.md b/wiki/Graphics-Execution.md
new file mode 100644
index 0000000..48423f8
--- /dev/null
+++ b/wiki/Graphics-Execution.md
@@ -0,0 +1,182 @@
+# Graphics Execution
+
+> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and
+> Vulkan command streams a stock guest submits — on the physical GPU, and what
+> is proven about it. This is the shared machinery behind
+> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX
+> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and
+> completion path for a different engine workload
+> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources:
+> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
+> (the plan and milestone ladder) and
+> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
+> (the measured walls); the docs tree wins on disagreement.
+
+Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` observed end to
+end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined
+success state; `[planned]` designed, not built.
+
+## 1. What "real pixels" means
+
+The acceptance target is strict: a **stock** guest GL or Vulkan application's
+draw commands are executed by the physical GPU's fixed-function 3D pipeline
+(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the
+render target is readable by the guest, and that image reaches the capture
+path through the emulated mdev. The pixels must not come from another GPU, a
+software rasteriser, or a host display round-trip.
+
+(The project has also published two deliberately-labelled two-VM demo videos
+that use the standard `virtio-gpu`/`virgl` virtual GPU on a host integrated
+GPU, and a CUDA-animation video that *does* run on the emulated vGPU through
+the compute path. The `virgl` videos are honest demonstrations of the guest
+experience; they are not this execution path, and their captions say so.)
+
+## 2. The path a graphics command takes
+
+```mermaid
+graph TD
+ APP["Guest applications<br/>OpenGL · Vulkan · CUDA"] --> LIB["Guest user space<br/>GL: GLX client + the driver's GL core<br/>Vulkan: the NVIDIA ICD<br/>CUDA: libcuda"]
+ APP --> X["X server (Linux GL)<br/>GLX module + the NVIDIA DDX"]
+ LIB --> KMD["Guest kernel: nvidia.ko<br/>RM core, GSP client, channels, modeset/drm"]
+ X --> KMD
+ KMD -- "GPFIFO rings + doorbell;<br/>method streams in push buffers" --> VG["Host module vgsp.ko<br/>device model + emulation core"]
+ VG --> L1["Lane 1: verbatim 3D<br/>method-stream replay"]
+ VG --> L2["Lane 2: synthesized launch<br/>from a captured section"]
+ VG --> L3["Lane 3: forwarded CUDA<br/>launch (compute)"]
+ L1 --> FE["Physical GPU<br/>front-end fetch -> GR engine"]
+ L2 --> FE
+ L3 --> FE
+ FE -- "engine-authored releases<br/>(semaphores in guest memory)" --> VG
+ VG -- "completion relay: MSI-X +<br/>poller/event wake the guest" --> KMD
+```
+
+```text
+guest apps (GL / Vulkan / CUDA)
+ -> guest user space (GLX client + GL core, Vulkan ICD, libcuda)
+ -> guest X server (GLX module + NVIDIA DDX) [OpenGL]
+ -> nvidia.ko: channels, GPFIFO rings, doorbells [all APIs]
+ -> vgsp.ko: capture sections, decode pushes, forward
+ -> physical GPU front-end -> GR engine (3D + compute) / copy engines
+ <- engine-authored releases written into guest memory
+ <- completion relay (MSI-X + wakes) back to the guest
+ASCII fallback above; the diagram shows the three execution lanes.
+```
+
+## 3. The three execution lanes
+
+Graphics work reaches the engines today in one of three shapes, in increasing
+fidelity:
+
+1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the
+ X server's graphics-class (`0xC597`, `TURING_A`) section from its channel,
+ relocates the operand addresses into host-built shadow pages the engine can
+ reach, and submits the stream unchanged. The engine retires it: the uploads
+ (`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and
+ the release all execute with no MMU fault. This proves the *mechanics* of
+ 3D execution — object bind, window placement, operand relocation,
+ completion — on real hardware.
+2. **Synthesized launch from a captured section** `[live]`. A compute-shaped
+ launch built from the captured section (operand list, program, a 64-word
+ launch descriptor) reaches the engine and retires real `sm_75` SASS; the
+ Turing shader-program-header form (`V3`) is accepted with zero header
+ faults across repeated launches. This proved engine-side dispatch before
+ the verbatim lane existed. The program itself is still a development
+ stand-in at this rung — execution proof, not pixels.
+3. **Forwarding the client's own pushes** `[wall]`. The goal state: the GL
+ client's own GPFIFO ring and method streams are read, decoded and executed
+ with the client's real per-draw state. The GL client's channel is created
+ today, but it never submits — its pushes have not been made readable — and
+ this lane is gated on the ring/push-readability work (§6).
+
+## 4. The channel and object picture
+
+* **Every submission lives on a channel**: a GPFIFO ring plus engine objects,
+ with a USERD the driver advances its put pointer in and a doorbell the host
+ module traps. Channels are typed by engine: a graphics channel carries 3D
+ state and draws; copy-class channels carry memory movements.
+* **The graphics object class** the guest's 3D stream binds is the Turing
+ graphics class, `0xC597`. The stream the emulation can capture today comes
+ from the **X server's channel** (`OpenGL`); the 2D present/readback work
+ rides lighter 2D-class and copy-engine shapes.
+* **Cells of state**: 3D streams are long—hundreds of method words per draw —
+ and mix state binds (viewport, render target, ZS, blend/ROP), program and
+ constant binds, vertex/index streams and draw-issuing methods. The capture
+ machinery recognises the init-shaped operands today; the per-draw state
+ vocabulary is being extended ([G-B in the plan](
+ ../docs/04-implementation/05-graphics-execution-plan.md)).
+* **Synchronisation**: tracking semaphores, notifications and fence values
+ written by the engine; the guest's swap/present waits sit on top of those.
+
+## 5. Completion, fences and presents
+
+The acceptance signal is the **engine-authored release**: a semaphore the
+engine writes when the work completes. For GL, completion flows through the
+channel's semaphores/notifications to the X server and the app's swap wait;
+for Vulkan, the client's fence value is released; for CUDA, the launch's
+release group wakes the client. The emulation relays engine completions into
+guest-observable memory (MSI-X plus a poller), and the guest resumes from
+there.
+
+During bring-up the module also carries a **diagnostics-only mirror** for apps
+that spin on a shared progress word; it exists so a run can proceed while the
+natural completion path is being finished. The 60-fps milestone explicitly
+requires it switched off, along with any external poke.
+
+## 6. Where the frontier is (ranked walls)
+
+1. **The GL client's submissions** `[wall]`. The client's channel exists but
+ never submits: direct measurement shows the client process writes no put
+ pointer in any mapping the service can see, and its pushes are not yet
+ reachable. Everything downstream waits on this.
+2. **Program (SASS) delivery** `[wall]`. Graphics programs reach the device
+ through the driver's copy/utility-channel uploads; those inline-data
+ writes are not yet materialised for the GL/Vulkan clients, and the program
+ page the engine fetches is empty. With the data-set gate bypassed, the
+ engine fetches the launch and MMU-faults (`FAULT_PDE`) at the program VA —
+ proof the channel and context are alive, and that mapping + content are
+ the missing pair.
+3. **Non-black frames** `[wall]`. Once draws execute with real state and real
+ programs, the capture must show motion; every frame captured so far is
+ black.
+4. **Sustained frame rate, and two VMs** `[target]`. The defined end state:
+ 60 fps in one VM with no forcing, then two VMs on two mediated devices
+ with non-black frames in both.
+
+## 7. What is proven today
+
+| Capability | State |
+|---|---|
+| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` |
+| `glxinfo` enumerates the emulated GPU with the NVIDIA renderer | `[live]` |
+| GZDoom launches and runs on the emulated vGPU | `[live]` (pixels frontier) |
+| GL section capture: the X server's `0xC597` stream decoded | `[live]` |
+| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` |
+| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` |
+| 2D screen fills and copy-engine present ops execute | `[live]` |
+| Direct GL client: device-init gate handled, channels allocate, readback path returns clean | `[wip]` (client submissions not yet) |
+| Non-black pixels from the emulated vGPU's 3D pipeline | `[wall]` |
+| Windows/DirectX guests | `[planned]` |
+
+## 8. What is not claimed
+
+* No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet.
+* No frame-rate or latency numbers: the scheduled measurement is defined
+ (60 fps over 60 s, no forcing) but not yet met.
+* The `virgl`-based demo videos are a different, labelled path — not this one.
+
+## 9. Related pages
+
+* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status.
+* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status.
+* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target.
+* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain,
+ proven end to end.
+* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
+ is forwarded to.
+* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO,
+ engines, and page tables.
+* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding
+ live.
+* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
+ the canonical plan: gap list, three approaches, and the M-GL1..M-GL6
+ milestone ladder with gates.