1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
# Graphics Execution
> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and
> Vulkan command streams a stock guest submits — on the physical GPU, and what
> is proven about it. This is the shared machinery behind
> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX
> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and
> completion path for a different engine workload
> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources:
> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md)
> (the plan and milestone ladder) and
> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md)
> (the measured walls); the docs tree wins on disagreement.
Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` observed end to
end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined
success state; `[planned]` designed, not built.
## 1. What "real pixels" means
The acceptance target is strict: a **stock** guest GL or Vulkan application's
draw commands are executed by the physical GPU's fixed-function 3D pipeline
(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the
render target is readable by the guest, and that image reaches the capture
path through the emulated mdev. The pixels must not come from another GPU, a
software rasteriser, or a host display round-trip.
(The project has also published two deliberately-labelled two-VM demo videos
that use the standard `virtio-gpu`/`virgl` virtual GPU on a host integrated
GPU, and a CUDA-animation video that *does* run on the emulated vGPU through
the compute path. The `virgl` videos are honest demonstrations of the guest
experience; they are not this execution path, and their captions say so.)
## 2. The path a graphics command takes
```mermaid
graph TD
APP["Guest applications<br/>OpenGL · Vulkan · CUDA"] --> LIB["Guest user space<br/>GL: GLX client + the driver's GL core<br/>Vulkan: the NVIDIA ICD<br/>CUDA: libcuda"]
APP --> X["X server (Linux GL)<br/>GLX module + the NVIDIA DDX"]
LIB --> KMD["Guest kernel: nvidia.ko<br/>RM core, GSP client, channels, modeset/drm"]
X --> KMD
KMD -- "GPFIFO rings + doorbell;<br/>method streams in push buffers" --> VG["Host module vgsp.ko<br/>device model + emulation core"]
VG --> L1["Lane 1: verbatim 3D<br/>method-stream replay"]
VG --> L2["Lane 2: synthesized launch<br/>from a captured section"]
VG --> L3["Lane 3: forwarded CUDA<br/>launch (compute)"]
L1 --> FE["Physical GPU<br/>front-end fetch -> GR engine"]
L2 --> FE
L3 --> FE
FE -- "engine-authored releases<br/>(semaphores in guest memory)" --> VG
VG -- "completion relay: MSI-X +<br/>poller/event wake the guest" --> KMD
```
```text
guest apps (GL / Vulkan / CUDA)
-> guest user space (GLX client + GL core, Vulkan ICD, libcuda)
-> guest X server (GLX module + NVIDIA DDX) [OpenGL]
-> nvidia.ko: channels, GPFIFO rings, doorbells [all APIs]
-> vgsp.ko: capture sections, decode pushes, forward
-> physical GPU front-end -> GR engine (3D + compute) / copy engines
<- engine-authored releases written into guest memory
<- completion relay (MSI-X + wakes) back to the guest
ASCII fallback above; the diagram shows the three execution lanes.
```
## 3. The three execution lanes
Graphics work reaches the engines today in one of three shapes, in increasing
fidelity:
1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the
X server's graphics-class (`0xC597`, `TURING_A`) section from its channel,
relocates the operand addresses into host-built shadow pages the engine can
reach, and submits the stream unchanged. The engine retires it: the uploads
(`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and
the release all execute with no MMU fault. This proves the *mechanics* of
3D execution — object bind, window placement, operand relocation,
completion — on real hardware.
2. **Synthesized launch from a captured section** `[live]`. A compute-shaped
launch built from the captured section (operand list, program, a 64-word
launch descriptor) reaches the engine and retires real `sm_75` SASS; the
Turing shader-program-header form (`V3`) is accepted with zero header
faults across repeated launches. This proved engine-side dispatch before
the verbatim lane existed. The program itself is still a development
stand-in at this rung — execution proof, not pixels.
3. **Forwarding the client's own pushes** `[wall]`. The goal state: the GL
client's own GPFIFO ring and method streams are read, decoded and executed
with the client's real per-draw state. The GL client's channel is created
today, but it never submits — its pushes have not been made readable — and
this lane is gated on the ring/push-readability work (§6).
## 4. The channel and object picture
* **Every submission lives on a channel**: a GPFIFO ring plus engine objects,
with a USERD the driver advances its put pointer in and a doorbell the host
module traps. Channels are typed by engine: a graphics channel carries 3D
state and draws; copy-class channels carry memory movements.
* **The graphics object class** the guest's 3D stream binds is the Turing
graphics class, `0xC597`. The stream the emulation can capture today comes
from the **X server's channel** (`OpenGL`); the 2D present/readback work
rides lighter 2D-class and copy-engine shapes.
* **Cells of state**: 3D streams are long—hundreds of method words per draw —
and mix state binds (viewport, render target, ZS, blend/ROP), program and
constant binds, vertex/index streams and draw-issuing methods. The capture
machinery recognises the init-shaped operands today; the per-draw state
vocabulary is being extended ([G-B in the plan](
../docs/04-implementation/05-graphics-execution-plan.md)).
* **Synchronisation**: tracking semaphores, notifications and fence values
written by the engine; the guest's swap/present waits sit on top of those.
## 5. Completion, fences and presents
The acceptance signal is the **engine-authored release**: a semaphore the
engine writes when the work completes. For GL, completion flows through the
channel's semaphores/notifications to the X server and the app's swap wait;
for Vulkan, the client's fence value is released; for CUDA, the launch's
release group wakes the client. The emulation relays engine completions into
guest-observable memory (MSI-X plus a poller), and the guest resumes from
there.
During bring-up the module also carries a **diagnostics-only mirror** for apps
that spin on a shared progress word; it exists so a run can proceed while the
natural completion path is being finished. The 60-fps milestone explicitly
requires it switched off, along with any external poke.
## 6. Where the frontier is (ranked walls)
1. **The GL client's submissions** `[wall]`. The client's channel exists but
never submits: direct measurement shows the client process writes no put
pointer in any mapping the service can see, and its pushes are not yet
reachable. Everything downstream waits on this.
2. **Program (SASS) delivery** `[wall]`. Graphics programs reach the device
through the driver's copy/utility-channel uploads; those inline-data
writes are not yet materialised for the GL/Vulkan clients, and the program
page the engine fetches is empty. With the data-set gate bypassed, the
engine fetches the launch and MMU-faults (`FAULT_PDE`) at the program VA —
proof the channel and context are alive, and that mapping + content are
the missing pair.
3. **Non-black frames** `[wall]`. Once draws execute with real state and real
programs, the capture must show motion; every frame captured so far is
black.
4. **Sustained frame rate, and two VMs** `[target]`. The defined end state:
60 fps in one VM with no forcing, then two VMs on two mediated devices
with non-black frames in both.
## 7. What is proven today
| Capability | State |
|---|---|
| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` |
| `glxinfo` enumerates the emulated GPU with the NVIDIA renderer | `[live]` |
| GZDoom launches and runs on the emulated vGPU | `[live]` (pixels frontier) |
| GL section capture: the X server's `0xC597` stream decoded | `[live]` |
| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` |
| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` |
| 2D screen fills and copy-engine present ops execute | `[live]` |
| Direct GL client: device-init gate handled, channels allocate, readback path returns clean | `[wip]` (client submissions not yet) |
| Non-black pixels from the emulated vGPU's 3D pipeline | `[wall]` |
| Windows/DirectX guests | `[planned]` |
## 8. What is not claimed
* No non-black pixel has been produced by the emulated vGPU's 3D pipeline yet.
* No frame-rate or latency numbers: the scheduled measurement is defined
(60 fps over 60 s, no forcing) but not yet met.
* The `virgl`-based demo videos are a different, labelled path — not this one.
## 9. Related pages
* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status.
* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status.
* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target.
* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain,
proven end to end.
* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
is forwarded to.
* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO,
engines, and page tables.
* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding
live.
* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) —
the canonical plan: gap list, three approaches, and the M-GL1..M-GL6
milestone ladder with gates.
|
