diff options
Diffstat (limited to 'wiki/Graphics-Execution.md')
| -rw-r--r-- | wiki/Graphics-Execution.md | 245 |
1 files changed, 245 insertions, 0 deletions
diff --git a/wiki/Graphics-Execution.md b/wiki/Graphics-Execution.md new file mode 100644 index 0000000..eea0574 --- /dev/null +++ b/wiki/Graphics-Execution.md @@ -0,0 +1,245 @@ +# Graphics Execution + +> **Scope.** How the emulated vGPU runs **3D graphics** work — the OpenGL and +> Vulkan command streams a stock guest submits — on the physical GPU, and what +> is proven about it. This is the shared machinery behind +> [`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md) and the Windows/DirectX +> target ([`DirectX.md`](DirectX.md)); CUDA compute uses the same submission and +> completion path for a different engine workload +> ([`Execution-Frontier.md`](Execution-Frontier.md)). Canonical sources: +> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) +> (the plan and milestone ladder) and +> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md) +> (the measured walls); the docs tree wins on disagreement. + +Status: 2026-11-11, tree `master @ 4acac093`. Tags: `[live]` observed end to +end; `[wip]` partially built; `[wall]` current blocker; `[target]` defined +success state; `[planned]` designed, not built. + +## 1. What "real pixels" means + +The acceptance target is strict: a **stock** guest GL or Vulkan application's +draw commands are executed by the physical GPU's fixed-function 3D pipeline +(vertex fetch, primitive assembly, rasterisation, fragment work, ROP), the +render target is readable by the guest, and that image reaches the capture +path through the emulated mdev. The pixels must not come from another GPU, a +software rasteriser, or a host display round-trip. + +## 2. The path a graphics command takes + +```mermaid +graph TD + APP["Guest applications<br/>OpenGL · Vulkan · CUDA"] --> LIB["Guest user space<br/>GL: GLX client + the driver's GL core<br/>Vulkan: the NVIDIA ICD<br/>CUDA: libcuda"] + APP --> X["X server (Linux GL)<br/>GLX module + the NVIDIA DDX"] + LIB --> KMD["Guest kernel: nvidia.ko<br/>RM core, GSP client, channels, modeset/drm"] + X --> KMD + KMD -- "GPFIFO rings + doorbell;<br/>method streams in push buffers" --> VG["Host module vgsp.ko<br/>device model + emulation core"] + VG --> L1["Lane 1: verbatim 3D<br/>method-stream replay"] + VG --> L2["Lane 2: synthesized launch<br/>from a captured section"] + VG --> L3["Lane 3: forwarded CUDA<br/>launch (compute)"] + L1 --> FE["Physical GPU<br/>front-end fetch -> GR engine"] + L2 --> FE + L3 --> FE + FE -- "engine-authored releases<br/>(semaphores in guest memory)" --> VG + VG -- "completion relay: MSI-X +<br/>poller/event wake the guest" --> KMD +``` + +```text +guest apps (GL / Vulkan / CUDA) + -> guest user space (GLX client + GL core, Vulkan ICD, libcuda) + -> guest X server (GLX module + NVIDIA DDX) [OpenGL] + -> nvidia.ko: channels, GPFIFO rings, doorbells [all APIs] + -> vgsp.ko: capture sections, decode pushes, forward + -> physical GPU front-end -> GR engine (3D + compute) / copy engines + <- engine-authored releases written into guest memory + <- completion relay (MSI-X + wakes) back to the guest +ASCII fallback above; the diagram shows the three execution lanes. +``` + +## 3. The three execution lanes + +Graphics work reaches the engines today in one of three shapes, in increasing +fidelity: + +1. **Verbatim 3D method-stream replay** `[live]`. The emulation captures the + X server's graphics-class (`0xC597`, `TURING_A`) section from its channel, + relocates the operand addresses into host-built shadow pages the engine can + reach, and submits the stream unchanged. The engine retires it: the uploads + (`LINE_LENGTH_IN`/`LAUNCH_DMA`), the state, program and texture binds, and + the release all execute with no MMU fault. This proves the *mechanics* of + 3D execution — object bind, window placement, operand relocation, + completion — on real hardware. +2. **Synthesized launch from a captured section** `[live]`. A compute-shaped + launch built from the captured section (operand list, program, a 64-word + launch descriptor) reaches the engine and retires real `sm_75` SASS; the + Turing shader-program-header form (`V3`) is accepted with zero header + faults across repeated launches. This proved engine-side dispatch before + the verbatim lane existed. The program itself is still a development + stand-in at this rung — execution proof, not pixels. +3. **Forwarding the client's own pushes** `[wip]`. The X DDX channel + `0x1000014` executes its 3D replay (12 launches per cycle), the 2D path is + green, and a guest pbuffer `glClear(red)+glFinish+glReadPixels` probe + returns a real pixel (`pixel=ff0000ff`): the replay and readback machinery + works. The off-screen frame path then produced **500/500 non-black 640x480 + animated frames** and a captured 45.4 s video. The direct-GL **GR ring + `0xbeef0100` now resolves** (`4acac093`: beef-family always-probe, + per-channel promote backoff, rank-collision fallback, parked-candidate + eviction); its releases advance frame by frame and GZDoom reaches + `V_Init: allocate screen` with its GL 4.6.0 context. The direct-GL **CE + ring `0xbeef0101` never resolves**, so the client's CE work does not + execute and the guest wedges ~90 s after `V_Init`; window/screen present + is still the frontier (§6). + +## 4. The channel and object picture + +* **Every submission lives on a channel**: a GPFIFO ring plus engine objects, + with a USERD the driver advances its put pointer in and a doorbell the host + module traps. Channels are typed by engine: a graphics channel carries 3D + state and draws; copy-class channels carry memory movements. +* **The graphics object class** the guest's 3D stream binds is the Turing + graphics class, `0xC597`. Streams now come from both the **X server's + channel** and the **direct GL client's own channels** (`0xbeef0100` GR plus + the `0xbeef0101`/`0xbeef0102` CE siblings); the 2D present/readback work + rides lighter 2D-class and copy-engine shapes. +* **Display and modeset** `[live]`. The guest's modeset path is served by an + emulated display engine: the display push-buffer channels (core `0xc57d`, + window `0xc57e`, immediate/window `0xc57b`) are registered, their push + buffers parsed, and the notifier/`UPDATE` completion authored. The arm is + class-aware (the window class has no NOTIFY bit; a bound notifier is the + arm), and the X DDX's `0x8000 -> 0` notification completion is authored. + This is what lets nvidia-modeset/NvKms register (`/dev/dri`), X settle and + GLX clients obtain a real GL context. The display-capability seam is + parameterised in the campaign configuration. +* **Cells of state**: 3D streams are long—hundreds of method words per draw — + and mix state binds (viewport, render target, ZS, blend/ROP), program and + constant binds, vertex/index streams and draw-issuing methods. The capture + machinery recognises the init-shaped operands today; the per-draw state + vocabulary is being extended ([G-B in the plan]( + ../docs/04-implementation/05-graphics-execution-plan.md)). +* **Synchronisation**: tracking semaphores, notifications and fence values + written by the engine; the guest's swap/present waits sit on top of those. + +## 5. Completion, fences and presents + +The acceptance signal is the **engine-authored release**: a semaphore the +engine writes when the work completes. For GL, completion flows through the +channel's semaphores/notifications to the X server and the app's swap wait; +for Vulkan, the client's fence value is released; for CUDA, the launch's +release group wakes the client. The emulation relays engine completions into +guest-observable memory (MSI-X plus a poller), and the guest resumes from +there. + +For the display path, completion is the notifier write and `UPDATE -> +FINISHED`; a completion event deferred by the reply path is force-retried +until it posts, so the X server's mode set finishes. For the X DDX's own 2D +push buffers, completion also requires the GPU-written engine get word to +advance and the notifier to be recycled to `0xffff` after a kick; both are +host-side levers now, and a diagnostic write of the get word released the +DDX's reserve loop. For a windowed GL client the present is a kick; the GSP +event-IRQ fix now sets the RISC-V SWGEN0 IRQSTAT/IRQMASK/IRQDEST and latches +the CPU-tree leaf, so the guest services the event and glxgears completes +swaps, and the GR backend no longer times out (the pair-close/pair-safe +replay fixes are default ON; draws retire in 3-2400 ms). The DDX 3D replay +renders and reads back a pbuffer pixel (`pixel=ff0000ff`), the off-screen +frame path produced non-black animated frames and a 45.4 s video, the +RM write-lock holder (Xorg in the DDX vidheap scrub wait) is fixed +(`df6e71bb`), and the direct-GL GR ring now resolves with advancing releases. +The next step is the direct-GL CE ring `0xbeef0101` and the window/screen +present path. + +During bring-up the module also carries a **diagnostics-only mirror** for apps +that spin on a shared progress word; it exists so a run can proceed while the +natural completion path is being finished. The 60-fps milestone explicitly +requires it switched off, along with any external poke. + +## 6. Where the frontier is (ranked walls) + +1. **Direct-GL CE ring resolution** `[wall]`. The GR ring `0xbeef0100` + resolves and its releases advance, but the CE ring `0xbeef0101` + (`gpfifo=0x4100000`) never maps: `VAS-PROMOTE` walks a parked candidate and + returns `rc=-2`, then `NOT mapped by any registered VAS` every ~7 s. The + client's CE work never executes and GZDoom wedges ~90 s after `V_Init`. + Next: walk all parked candidates regardless of owner VAS and/or keep at + least two fresh same-VAS parks. +2. **Window/screen present and the GLX lottery** `[wall]`. The pbuffer and + off-screen frame paths are proven (`pixel=ff0000ff`, 500/500 non-black + frames, 45.4 s video), but glxgears' window is black and the GLX + probe/frames path is a per-window lottery (hang/black/BadValue). A + synchronous window draw wedges X; screen capture must use the ivshmem + frame path and the present must be made to land. +3. **X startup / stability** `[wall]`. X startup is a per-boot lottery: on + 3/6 boots window creation wedges Xorg in an unkillable kernel RM spin + (`_nv052600rm+0x7b`); `Failed to initialize DMA`, push-buffer `0x51`, + `Wait for channel idle timed out` and occasional vmalloc failures also + appear. A fresh cycle clears it. The shared nvClip `modeNum` must be even + for the GL client's state init to succeed; the odd value is the guest + kernel's nvKms deferred close, not module-induced. +4. **The X DDX footprint** `[wall]`. The DDX allocates 4 GB of system pages at + startup; an 8 GB guest has no headroom once GZDoom's own footprint lands + and both are OOM-killed. Current runs use a 12 GB guest; the allocation is + a suspected emulation gap (a 512 MiB emulated VRAM falling back to a 4 GB + sysmem allocation) to fix. +5. **Non-black window frames and sustained rate** `[wall]`/`[target]`. Once + the CE ring, the window/screen present and X stability land, the capture + must show motion in a window, then 60 fps in one VM with no forcing, then + two VMs on two mediated devices with non-black frames in both. + +## 7. What is proven today + +| Capability | State | +|---|---| +| X server initialises on the emulated vGPU (driver DMA sanity push, ScreenInit) | `[live]` | +| Display engine serves the modeset push buffers; notifier plus `UPDATE -> FINISHED` | `[live]` (campaign configuration) | +| nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`); `nvidia-smi -L` enumerates | `[live]` | +| `glxinfo` reports the emulated GPU with the NVIDIA renderer (OpenGL 4.6.0, direct rendering) | `[live]` | +| X green on attempt 1 in three consecutive cycles (`ebf3130f`) | `[live]` | +| GSP event-IRQ fix wakes the guest poll; glxgears completes swaps | `[live]` | +| 2D path end-to-end: accelerated X fills read back via `xwd` | `[live]` | +| Guest's real GL draws forward and execute on the physical GPU (`work_gr backend_ok=5-7`) | `[live]` | +| GR backend timeouts fixed (`backend_timeout=0`; draws retire in 3-2400 ms) | `[live]` | +| DDX 3D replay executes (`section captured` + `GR LAUNCH` on `0x1000014`, 12 launches/cycle) | `[live]` | +| First real GPU readback: pbuffer `glClear(red)+glReadPixels` -> `pixel=ff0000ff` | `[live]` | +| First non-black GPU-rendered frames: 500/500 off-screen frames; 45.4 s video captured (`r1769`) | `[live]` | +| Direct-GL GR ring `0xbeef0100` resolves; releases advance (`4acac093`) | `[live]` | +| GZDoom passes `V_Init: allocate screen` with GL 4.6.0 when shared nvClip `modeNum` is even | `[live]` | +| RM write-lock holder identified (Xorg in the DDX vidheap scrub wait) and fixed (`df6e71bb`) | `[live]` | +| GL section capture: the X channel's `0xC597` stream decoded | `[live]` | +| Verbatim 3D method-stream replay retired by the engine (no MMU fault) | `[live]` | +| Synthesized launch executes real `sm_75` SASS; Turing `V3` program header accepted | `[live]` | +| X server survives the run on a 12 GB guest (the DDX's 4 GB sysmem allocation forces the size) | `[live]` | +| Direct-GL CE ring `0xbeef0101` resolves and the client's CE work executes | `[wall]` | +| Window/screen present produces non-black window pixels | `[wall]` | +| X startup/stability (`_nv052600rm` soft-lock, `0x51`, vmalloc) | `[wall]` | +| Non-black screen pixels from the emulated vGPU's 3D pipeline | `[wall]` | +| Windows/DirectX guests | `[planned]` | + +## 8. What is not claimed + +* No **window/screen** pixel has been produced by the emulated vGPU's 3D + pipeline yet; every screen capture so far is black. The proven non-black + frames and the 45.4 s video come from the off-screen pbuffer path + (pbuffer -> sysmem -> kvmfr ivshmem -> host reader), not from a windowed X + screen capture. +* Display/KMS bring-up (NvKms, `/dev/dri`, X, a real GL context) is not a + rendered-frame claim. GZDoom reaching `V_Init: allocate screen` proves the + client's GL init and the direct-GL GR lane, not the game loop or a + presented frame; the direct-GL CE ring still does not resolve. +* No frame-rate or latency numbers: the scheduled measurement is defined + (60 fps over 60 s, no forcing) but not yet met. + +## 9. Related pages + +* [`OpenGL.md`](OpenGL.md) — the guest OpenGL path and its status. +* [`Vulkan.md`](Vulkan.md) — the guest Vulkan path and its status. +* [`DirectX.md`](DirectX.md) — Windows guests and the DirectX target. +* [`Execution-Frontier.md`](Execution-Frontier.md) — the CUDA compute chain, + proven end to end. +* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work + is forwarded to. +* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — channels, GPFIFO, + engines, and page tables. +* [`Host-Module.md`](Host-Module.md) — where capture, decode and forwarding + live. +* [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md) — + the canonical plan: gap list, three approaches, and the M-GL1..M-GL6 + milestone ladder with gates. |