# Project Status > **What this page is.** The honest, dated state of the project: what works > today, what is in progress, and what is not started. It is refreshed > periodically and each claim maps to a maintained document in the repository. > Canonical sources: [`../docs/README.md`](../docs/README.md) (project front > door), [`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md) > (known-knowns and known-unknowns) and > [`../docs/04-implementation/03-execution-frontier.md`](../docs/04-implementation/03-execution-frontier.md) > (the execution frontier). When this page and the docs disagree, the docs tree > wins. Status: 2026-11-17, tree `master @ 1db3831e`. Tags: `[live]` proven with the stock guest driver; `[gated]` implemented, default-off or behind supervised gates; `[wip]` partially implemented; `[wall]` current blocker; `[target]` named success state; `[planned]` designed, not built. ## 1. Milestone ladder | Milestone | State | |---|---| | Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | `[live]` working | | GSP boot emulation (FWSEC / booter / RISCV handshake) | `[live]` working | | GSP-RM message queues and RPC transport | `[live]` working | | Stock guest driver probe and `nvidia-smi` (TU104) | `[live]` working | | CUDA userspace init and the driver-API smoke (`cuInit`, device enumeration, `cuCtxCreate`, allocation, copies, free, destroy) | `[live]` working — the smoke prints `ALL OK` and the 64-byte payload round-trips exactly | | Guest CUDA kernel execution on real hardware | `[live]` working — verified green on fresh boots: the vector-add (`VADD n=32768 bad=0`), matrix-multiply (`MATMUL n=256 bad=0`) and transformer (`dh=256 seq=64 bad=0`, `FFN4 layers=4 bad=0`) clients print `STATUS OK`, engine-authored completions relay to the guest and its synchronisation waits resolve. A ring-block ownership defect that could stall the first client after a reboot was root-caused and fixed. Remaining: the staged-transfer class (some large multi-chunk clients still mis-verify; the failure modes are proven and under repair) and per-guest execution isolation for simultaneous CUDA guests | | Physical backend on real hardware (module-owned GPU running its real GSP-RM) | `[live]` boot, RPC session, object creation, copy-engine work and GR execution proven — the verified guest CUDA kernel clients execute through this backend, and graphics method streams execute on the engine ([`Graphics-Execution.md`](Graphics-Execution.md)) | | Display and KMS on the emulated vGPU (nvidia-modeset/NvKms, `/dev/dri`, X) | `[live]` in the campaign configuration — the emulated display engine serves the guest's display push-buffer channels (notifier plus `UPDATE -> FINISHED`), nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`), `nvidia-smi -L` enumerates and X settles. The X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe-thrash, Xorg kernel-mode-spin and flip-event walls were closed (X green for >10 minutes in a validated window); what remains is the per-boot **first-generation X start lottery**, which by 11-13/11-14 was the dominant throughput limiter for the graphics A/B runs (only an X started immediately after boot passes DDX DMA init; long stretches of dead starts, one green first-start boot). The display-capability seam is parameterised ([`Graphics-Execution.md`](Graphics-Execution.md)) | | Multi-tenant isolation (mdev per VM, per-context page directories) | `[wip]` device-level coexistence works — multiple mediated devices with different emulated VRAM sizes and multiple guests coexist (two guests demonstrated); per-guest execution isolation for simultaneous CUDA guests is in progress | | OpenGL (3D graphics) | `[wip]` — the display/KMS stack registers, X and the stock NVIDIA GLX path are fully green (`glxinfo -B` returns 0 with OpenGL 4.6.0 and direct rendering; `glxgears` renders ~520 FPS with the emulated GPU's NVIDIA renderer), and the **first non-black GPU-rendered output** is proven: the pbuffer frame path produced 500/500 non-black 640x480 frames per run and captured videos (45.4 s on 2026-11-11; a 50 s 640x480 h264 capture on 2026-11-12). GZDoom reaches `V_Init: allocate screen` with its GL 4.6.0 context whenever the shared nvClip `modeNum` is even. **The direct-GL submission is now fully decoded:** no client push carries a method-grammar draw header — the channel's 9 NV906F entries tile one contiguous buffer and its per-frame work is carried as rewritten data-block blobs plus small state pushes (324/476/728 B) that invoke MME macros (the QMD-carrier hypothesis was tested and refuted). A content-gated FB-shadow rescue and a LOW-band rescue recover the recycled pushes: the client's draw-state push now opens a C597 section, is captured and marked for the verbatim 3D lane (the first direct-GL stream3d marks), and reaches the engine (`GR LAUNCH submitted`/`OK`, golden MME overlay accepted). **The MME chain now completes and executes the draw macro.** The multi-part keep-window mechanism won multi-frame windows; the part-launch poison was eliminated (the real fault was the client's own `SET_REPORT_SEMAPHORE`/`SET_NOTIFY` writes to client VAs that the host scratch page-table pool remaps — redirected to a mapped scratch slot; 49 cycles with zero poison and zero MMU faults); the 2799-word pack completes (2799/2799); engine-global completion plus keep-done-across-recycle produced the first `MME-PACK-REARM`; the fixed-CB region is aliased; and the released calls run macros through to **`DRAW_IAB` (`mac=0x22`)**. Two synthesized-prelude defects were then fixed: the SPH-BIND prelude self-destructed because an auto-dropped run broke its kept/final count (`444ae2c1`), and the stage block was silently skipped by a double-counted budget; with both fixed the prelude emits 84 words and the DS stage VAs populate (one validated boot carried the full post-REARM chain, `mac=0x22` and `DS 86040624`). **The DS stage-0/1 image is now authored and live**: a VTG SPH is written at `family+0x900` (`SPH-AUTHOR`, first word `020a0481`), the window's own later `0x2054` slot-1 bind is rewritten to the authored page (`SPH-SLOT1-REWRITE`), and the DS latch reads the authored image for both vertex slots (`DS-IMAGE st0=st1=020a0481 code=ff007321`; the DS value fell `0x86040626 -> 0x84000420`), reproduced on two boots. The 11-16 wave then proved the **host-side chain end to end**: the real frame pushes are admitted (a persisted colour-target bind carried from an earlier trusted push, draw markers accepted inside counted method runs, the 40-chunk full-window replay, a data-led window-start snap), the beef draw lane latches a **clean DS** (`DS-FAST2 ... ds_after=80000000`, c963, at the live-derived GS map), capture-less boots get the **golden 2799-word MME pack** (`MME-PRELUDE`, `MME-CARRY-FIRE`, `MME-PACK-REARM`, `MME-SA-REARM`), the prelude latch survives the per-fire channel recycle, and every admitted-window chunk is forwarded in order. At that point the decisive finding was guest-side: across 36 boots (c990-c1019) the client never submitted a non-null `CALL(34)` draw macro — every `0x3910/0x3914` hit is an interior method write with `data=0x0`, no CALL header exists in the admitted windows, and the same cycles logged guest EGL/dri2 failures and OOM kills, so the guest GL path was not producing a real draw submission. A GZDoom video (4 files, 22 s flight + 18 s orbit, 640x480/1280x960, 30 fps, byte-verified) was delivered through the guest's **software GL** path (Mesa llvmpipe) as a guaranteed-quality deliverable; it is not a vGPU rendering claim. The **late-11-16 wave then crossed the guest submission gap**: an offscreen SDL EGL/GBM shim gives GZDoom a real NVIDIA GL context without Xorg/DDX (`GL_RENDERER: NVIDIA GeForce RTX 2080 SUPER/PCIe/SSE2`, `GL_VERSION: 4.6.0 NVIDIA 570.144`; no llvmpipe), and the real 12068-B draw window (op=4 `CALL(0)` method `0x3800`, imm `0xd48`, plus slots 44/57/58) is admitted, chunked and replayed verbatim. `CALL(0)` no longer wedges: eng89t resolves the immediate through client SA[0] (then golden) and elides a not-resident call to a NOP; c1026/c1027 entered the captured macro without trapping and c1027 produced the first DS with real stage data. The remaining gate is precise: the client's **per-draw MME macro body was never captured** (the prelude carries only the 12 setup macros), and the surviving wedge is the synthetic golden body (`badf1201`); stores on the vGPU path still miss (`wrote=0`, no `px_nonblack`). Next: capture/verify the per-draw macro body, then the vertex/store and the window/screen present, with the X first-generation start lottery as the throughput limiter ([`OpenGL.md`](OpenGL.md)) | | Vulkan (3D/compute) | `[wip]` — the guest Vulkan pipeline runs end to end on the emulated vGPU (device, allocation, pipeline, command buffer, fence, queue submit, readback) and the engine executes translated launches; frames are still black — program delivery and program-VA mapping are the frontier ([`Vulkan.md`](Vulkan.md)) | | DirectX / Windows guests | `[planned]` (target) — no Windows guest booted yet; the architecture is the same emulation target ([`DirectX.md`](DirectX.md)) | | Video (NVENC/NVDEC) | `[planned]` | | Xen / Argo / OpenXT / Qubes Air transports | `[planned]` (the emulation core is hypervisor-independent by construction) | | Live migration | `[planned]` | | Performance numbers | `[planned]` — no published numbers yet; any claim will ship with its measurement methodology | ## 2. What "CUDA works" means here A green `nvidia-smi` or even a passing driver-API smoke is not "CUDA works". The project's definition of done for CUDA requires all of the following ([`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md)): 1. Stock guest driver probe, `nvidia-smi`, device enumeration — done. 2. Context creation and the full driver-API surface (`cuMemAlloc`, copies, free, destroy) — done. 3. A guest kernel launch that **executes on real hardware** and is proven by an engine-authored completion — demonstrated: the verified clients execute on the physical engine and the engine-authored completions relay to the guest. 4. Memory the guest's work refers to resolving correctly end to end (the VA↔PA model) — partially, for the paths exercised so far. 5. Correct completion, semaphore and fault semantics back to the guest — the GR completion path now relays (engine-authored releases wake the guest's synchronisation waits); fault semantics remain partial. 6. Reproducibility: every boot, not a lucky one — two root causes landed: the boot-poisoning defect (the wrong status-queue consumption cursor could desynchronise the reply ring under load) and the ring-block ownership defect that could stall the first client after a reboot. Green clients are reproduced across fresh boots; a formal every-boot acceptance campaign is the remaining work. 7. No ad-hoc default-off parameters required for the canonical path — still open (the forwarding and graphics seams remain parameterised while they are validated). Until every gate holds, the honest formulation is: **initialization, the driver-API path and kernel execution on the physical GPU work and verify; the staged-transfer class, per-guest isolation and the graphics direct-GL/present/first-frame frontier remain.** ## 3. How to see it yourself The reference setup takes about half an hour from a fresh host ([`Install-and-First-Run.md`](Install-and-First-Run.md)); the current expected outcome is exactly the ladder above — driver probe, `nvidia-smi`, the full driver-API smoke, and guest CUDA kernels executing and verifying on the physical GPU. On the graphics side the same setup brings up the display/KMS stack and the X server on the emulated vGPU (`glxinfo` reports OpenGL 4.6.0 with direct rendering, `glxgears` renders ~520 FPS); the first non-black GPU-rendered frames are proven off-screen (the pbuffer path produced a 500-frame 45.4 s video on 2026-11-11 and a 50 s capture on 2026-11-12), and GZDoom reaches its screen-allocation stage with GL 4.6.0 on an even modeNum. The host-side draw chain is now proven end to end — admission of the real frame pushes, a clean DS (`DS-FAST2 ds_after=80000000` on the draw lane), the golden 2799-word MME pack, the CALL hold, the release carrier and the SA re-arm all fire — and the late-11-16 wave crossed the guest submission gap: an offscreen EGL/GBM shim gives GZDoom a real NVIDIA GL context without Xorg/DDX, and the real `CALL(0)` `0x3800` draw is admitted, chunked and replayed, executes its captured macro without trapping, and reaches a DS with real stage data. The remaining gate is the client's per-draw MME macro body (never captured; the prelude carries only the setup macros), with the synthetic golden body the surviving wedge and no surface store yet. A GZDoom video was delivered through the guest's *software* GL path (Mesa llvmpipe) as a guaranteed-quality deliverable; it is not a vGPU rendering claim. Windowed/screen pixels remain the frontier (the per-draw macro body, then the vertex/store and the window/screen present, with X start keeping a first-generation lottery) — see [`OpenGL.md`](OpenGL.md). The risc-v-gsp-rm-debugger workbench ([`Riscv-GSP-RM-Debugger.md`](Riscv-GSP-RM-Debugger.md)) is the host-side tooling used to study the firmware side of that frontier. ## 4. How this page is maintained Every claim above carries a date and a tag, maps to a maintained document in the repository, and is refreshed as milestones land. The execution story, with its evidence, its resolved theories and its open work, is on [`Execution-Frontier.md`](Execution-Frontier.md). ## 5. Related pages * [`Install-and-First-Run.md`](Install-and-First-Run.md) * [`System-Overview.md`](System-Overview.md) * [`Execution-Frontier.md`](Execution-Frontier.md) * [`Hardware-Software-Matrix.md`](Hardware-Software-Matrix.md) — which hardware and software combinations are Reference, Target or Experimental. * [`FAQ.md`](FAQ.md)