diff options
Diffstat (limited to 'wiki/Vulkan.md')
| -rw-r--r-- | wiki/Vulkan.md | 127 |
1 files changed, 127 insertions, 0 deletions
diff --git a/wiki/Vulkan.md b/wiki/Vulkan.md new file mode 100644 index 0000000..d213136 --- /dev/null +++ b/wiki/Vulkan.md @@ -0,0 +1,127 @@ +# Vulkan + +> **Scope.** How a stock guest Vulkan application runs on the emulated vGPU: +> the guest-side stack, the submit/completion path, and the honest state of +> the work — including the one thing every Vulkan run on the emulated vGPU +> currently has in common: **the pipeline completes, and the frames are +> black**. The shared host machinery is on +> [`Graphics-Execution.md`](Graphics-Execution.md); the CUDA compute chain +> (further along) is on [`Execution-Frontier.md`](Execution-Frontier.md). +> Canonical sources: +> [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md) +> and +> [`../docs/04-implementation/05-graphics-execution-plan.md`](../docs/04-implementation/05-graphics-execution-plan.md); +> the docs tree wins on disagreement. + +Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` observed with the +stock guest driver; `[wip]` partial; `[wall]` current blocker; `[target]` +defined success state. + +## 1. The guest Vulkan stack + +A Vulkan application loads the **loader** (`libvulkan.so`), which dispatches to +an **installable client driver (ICD)** — for NVIDIA GPUs, the NVIDIA Vulkan +driver. On Linux, Vulkan compute does not require an X server; the display +path does (through `nvidia-modeset`/DRM for presentation). The kernel-side +requirements are the same as CUDA's: `nvidia.ko`, the GSP client, channels, +memory objects and fences. + +At device creation the driver also walks the **display/modeset device** +allocation path (the `ALLOC_DEVICE` class family) — on a vGPU-style device the +driver expects the host to provide those display classes, which the emulation +must answer for `vkCreateDevice` to succeed. Past that gate, the driver +behaves much like the compute stack: memory types through its VidHeap classes, +queues as channels, and work as method streams. + +## 2. The submit and completion path (observed) + +```mermaid +sequenceDiagram + participant App as Guest app + participant ICD as NVIDIA Vulkan driver + participant KMD as nvidia.ko + GSP client + participant VG as vgsp.ko emulation + participant GPU as Physical GPU (GR/CE) + App->>ICD: record command buffer, then vkQueueSubmit(fence) + ICD->>KMD: push buffer + GPFIFO entry + doorbell + KMD-->>VG: method stream (compute C5C0 shape + copy shapes) + VG->>VG: capture the section, translate to a vendor launch,<br/>relocate operands into reachable shadow pages + VG->>GPU: submit on a real channel (launch descriptor install) + GPU-->>VG: engine-authored release + VG-->>KMD: completion relay (MSI-X + poller) + KMD-->>App: fence released, vkWaitForFences returns + App->>App: reuse buffers, read back the frame +``` + +```text +app: vkQueueSubmit(fence) + -> NVIDIA driver: push buffer + GPFIFO entry + doorbell + -> vgsp.ko: capture section, translate to a vendor launch, + relocate operands into shadow pages, submit on a real channel + -> physical GPU: front-end fetch, engine executes, authors release + -> vgsp.ko: completion relay (MSI-X + poller) back to the guest + -> app: fence releases; frame readback proceeds +ASCII fallback above; the sequence shows one queue submit end to end. +``` + +Two mechanics make this work today: the emulation decodes the driver's +compute-dispatch shape — a **direct method stream** (object bind, window runs, +code runs, a subchannel data-port stream) with **no inline launch descriptor** +— and *translates* it into a launch the real engine accepts, and it derives +the submit's release groups from the driver's own push words so the fence +value is released when the engine completes (the "fence fold": the pre-built +GPFIFO continuation beyond the put pointer is swept and the submit's release +groups are applied forward-only). + +## 3. Status on the emulated vGPU + +| Item | State | +|---|---| +| `vkCreateDevice` on the emulated vGPU (display/modeset allocation path answered) | `[live]` | +| Full pipeline: instance, device, allocation, image/view/buffer/shader/descriptor, pipeline, command buffer, fence | `[live]` | +| Queue submit, engine-side execution of the translated launch, fence release, frame readback | `[live]` | +| The client saves frames end to end (`VKFRAME` evidence) | `[live]` | +| Non-black frames | `[wall]` | +| The driver's program (SASS) upload materialised by the emulation | `[wall]` | +| The engine's page tables mapping the program VA | `[wall]` | + +## 4. Where the wall is, precisely + +The client *believes* it uploaded its shader and dispatches with a valid +program address, but the bytes never appear in any source the emulation can +see: the driver delivers programmed SASS and descriptors through an **MME +program + inline-to-memory stream** on its copy/utility channel, and the +inline-data writes are not yet materialised for the graphics clients. Two +coupled facts define the remaining work: + +1. **Delivery** — decode and execute the driver's inline upload so the + program page becomes non-zero (the same machinery already works for CUDA's + uploads; the graphics clients' shapes are the extension). +2. **Mapping** — the engine's page tables must map the program VA. With the + data-set gate bypassed, the engine fetches the launch and raises a real + MMU fault (`FAULT_PDE`) at the program VA — proof the channel, context and + fetch path are alive, and that mapping plus content is the missing pair. + +When both land, the emulated vGPU should carry its first non-black Vulkan +frame — the milestone the whole graphics effort is converging on +([`Graphics-Execution.md`](Graphics-Execution.md) §6). + +## 5. What this page is not claiming + +* No rendered pixels yet — every frame saved so far is black. +* No performance numbers. +* `vkCreateDevice` success does not imply rendering; it means the device + model answers the driver's device-view correctly. + +## 6. Related pages + +* [`Graphics-Execution.md`](Graphics-Execution.md) — the shared lanes, the + `0xC597` graphics stream and the ranked walls. +* [`OpenGL.md`](OpenGL.md) — the GL path through the X server. +* [`DirectX.md`](DirectX.md) — the Windows/DirectX target. +* [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — the userspace + driver model the Vulkan ICD mirrors in structure. +* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md) — translation and the + engines. +* [`../docs/04-implementation/04-real-render-walls.md`](../docs/04-implementation/04-real-render-walls.md) — + the measured wall ledger, with every experiment and its verdict. |