1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
|
# Project Status
> **What this page is.** The honest, dated state of the project: what works
> today, what is in progress, and what is not started. It is refreshed
> periodically and each claim maps to a maintained document in the repository.
> Canonical sources: [`../docs/README.md`](../docs/README.md) (project front
> door), [`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md)
> (known-knowns and known-unknowns) and
> [`../docs/04-implementation/03-execution-frontier.md`](../docs/04-implementation/03-execution-frontier.md)
> (the execution frontier). When this page and the docs disagree, the docs tree
> wins.
Status: 2026-11-15, tree `master @ 6aeb20c7`. Tags: `[live]` proven with the
stock guest driver; `[gated]` implemented, default-off or behind supervised
gates; `[wip]` partially implemented; `[wall]` current blocker; `[target]`
named success state; `[planned]` designed, not built.
## 1. Milestone ladder
| Milestone | State |
|---|---|
| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | `[live]` working |
| GSP boot emulation (FWSEC / booter / RISCV handshake) | `[live]` working |
| GSP-RM message queues and RPC transport | `[live]` working |
| Stock guest driver probe and `nvidia-smi` (TU104) | `[live]` working |
| CUDA userspace init and the driver-API smoke (`cuInit`, device enumeration, `cuCtxCreate`, allocation, copies, free, destroy) | `[live]` working — the smoke prints `ALL OK` and the 64-byte payload round-trips exactly |
| Guest CUDA kernel execution on real hardware | `[live]` working — verified green on fresh boots: the vector-add (`VADD n=32768 bad=0`), matrix-multiply (`MATMUL n=256 bad=0`) and transformer (`dh=256 seq=64 bad=0`, `FFN4 layers=4 bad=0`) clients print `STATUS OK`, engine-authored completions relay to the guest and its synchronisation waits resolve. A ring-block ownership defect that could stall the first client after a reboot was root-caused and fixed. Remaining: the staged-transfer class (some large multi-chunk clients still mis-verify; the failure modes are proven and under repair) and per-guest execution isolation for simultaneous CUDA guests |
| Physical backend on real hardware (module-owned GPU running its real GSP-RM) | `[live]` boot, RPC session, object creation, copy-engine work and GR execution proven — the verified guest CUDA kernel clients execute through this backend, and graphics method streams execute on the engine ([`Graphics-Execution.md`](Graphics-Execution.md)) |
| Display and KMS on the emulated vGPU (nvidia-modeset/NvKms, `/dev/dri`, X) | `[live]` in the campaign configuration — the emulated display engine serves the guest's display push-buffer channels (notifier plus `UPDATE -> FINISHED`), nvidia-modeset/NvKms registers (`/dev/dri/card0`, `renderD128`, `fb0`), `nvidia-smi -L` enumerates and X settles. The X DDX idle-DMA/ScreenInit, USERD-latch lifetime, re-probe-thrash, Xorg kernel-mode-spin and flip-event walls were closed (X green for >10 minutes in a validated window); what remains is the per-boot **first-generation X start lottery**, which by 11-13/11-14 was the dominant throughput limiter for the graphics A/B runs (only an X started immediately after boot passes DDX DMA init; long stretches of dead starts, one green first-start boot). The display-capability seam is parameterised ([`Graphics-Execution.md`](Graphics-Execution.md)) |
| Multi-tenant isolation (mdev per VM, per-context page directories) | `[wip]` device-level coexistence works — multiple mediated devices with different emulated VRAM sizes and multiple guests coexist (two guests demonstrated); per-guest execution isolation for simultaneous CUDA guests is in progress |
| OpenGL (3D graphics) | `[wip]` — the display/KMS stack registers, X and the stock NVIDIA GLX path are fully green (`glxinfo -B` returns 0 with OpenGL 4.6.0 and direct rendering; `glxgears` renders ~520 FPS with the emulated GPU's NVIDIA renderer), and the **first non-black GPU-rendered output** is proven: the pbuffer frame path produced 500/500 non-black 640x480 frames per run and captured videos (45.4 s on 2026-11-11; a 50 s 640x480 h264 capture on 2026-11-12). GZDoom reaches `V_Init: allocate screen` with its GL 4.6.0 context whenever the shared nvClip `modeNum` is even. **The direct-GL submission is now fully decoded:** no client push carries a method-grammar draw header — the channel's 9 NV906F entries tile one contiguous buffer and its per-frame work is carried as rewritten data-block blobs plus small state pushes (324/476/728 B) that invoke MME macros (the QMD-carrier hypothesis was tested and refuted). A content-gated FB-shadow rescue and a LOW-band rescue recover the recycled pushes: the client's draw-state push now opens a C597 section, is captured and marked for the verbatim 3D lane (the first direct-GL stream3d marks), and reaches the engine (`GR LAUNCH submitted`/`OK`, golden MME overlay accepted). **The MME chain now completes and executes the draw macro.** The multi-part keep-window mechanism won multi-frame windows; the part-launch poison was eliminated (the real fault was the client's own `SET_REPORT_SEMAPHORE`/`SET_NOTIFY` writes to client VAs that the host scratch page-table pool remaps — redirected to a mapped scratch slot; 49 cycles with zero poison and zero MMU faults); the 2799-word pack completes (2799/2799); engine-global completion plus keep-done-across-recycle produced the first `MME-PACK-REARM`; the fixed-CB region is aliased; and the released calls run macros through to **`DRAW_IAB` (`mac=0x22`)**. Two synthesized-prelude defects were then fixed: the SPH-BIND prelude self-destructed because an auto-dropped run broke its kept/final count (`444ae2c1`), and the stage block was silently skipped by a double-counted budget; with both fixed the prelude emits 84 words and the DS stage VAs populate (one validated boot carried the full post-REARM chain, `mac=0x22` and `DS 86040624`). **The DS stage-0/1 image is now authored and live**: a VTG SPH is written at `family+0x900` (`SPH-AUTHOR`, first word `020a0481`), the window's own later `0x2054` slot-1 bind is rewritten to the authored page (`SPH-SLOT1-REWRITE`), and the DS latch reads the authored image for both vertex slots (`DS-IMAGE st0=st1=020a0481 code=ff007321`; the DS value fell `0x86040626 -> 0x84000420`), reproduced on two boots. The release path is now host-deterministic when the pack completes (the saved-window release carrier replays the held window with the calls released; the slot-34 census and SA[34] golden fill prove `client=golden=used=0x265`). The blocker is **call order**: the 728-B window places the slot-34 `DRAW_IAB` run after the `0x33/0x39` setup, so the `0x39` DS latch (`0x84000420`) pre-empts it; the 324-B window places it first and ran `mac=0x22`. A release-window reorder (`a03a68ea`) is landed but unexercised, pack completion is still a per-boot lottery (`6aeb20c7`, eng64, chains the release behind a successful pack carrier and names the stalled gate; validation pending), and the remaining DS stage type/context bits (SPH5/SPH10/SPH18) leave the stores missing (`wrote=0`, `px0=0`); the window/screen present path is unproven; and the per-boot binding lottery (the GL client usually rides the X channel in indirect mode, and X start keeps a first-generation lottery) is the throughput limiter ([`OpenGL.md`](OpenGL.md)) |
| Vulkan (3D/compute) | `[wip]` — the guest Vulkan pipeline runs end to end on the emulated vGPU (device, allocation, pipeline, command buffer, fence, queue submit, readback) and the engine executes translated launches; frames are still black — program delivery and program-VA mapping are the frontier ([`Vulkan.md`](Vulkan.md)) |
| DirectX / Windows guests | `[planned]` (target) — no Windows guest booted yet; the architecture is the same emulation target ([`DirectX.md`](DirectX.md)) |
| Video (NVENC/NVDEC) | `[planned]` |
| Xen / Argo / OpenXT / Qubes Air transports | `[planned]` (the emulation core is hypervisor-independent by construction) |
| Live migration | `[planned]` |
| Performance numbers | `[planned]` — no published numbers yet; any claim will ship with its measurement methodology |
## 2. What "CUDA works" means here
A green `nvidia-smi` or even a passing driver-API smoke is not "CUDA works".
The project's definition of done for CUDA requires all of the following
([`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md)):
1. Stock guest driver probe, `nvidia-smi`, device enumeration — done.
2. Context creation and the full driver-API surface (`cuMemAlloc`, copies,
free, destroy) — done.
3. A guest kernel launch that **executes on real hardware** and is proven by an
engine-authored completion — demonstrated: the verified clients execute on
the physical engine and the engine-authored completions relay to the guest.
4. Memory the guest's work refers to resolving correctly end to end (the VA↔PA
model) — partially, for the paths exercised so far.
5. Correct completion, semaphore and fault semantics back to the guest — the
GR completion path now relays (engine-authored releases wake the guest's
synchronisation waits); fault semantics remain partial.
6. Reproducibility: every boot, not a lucky one — two root causes landed: the
boot-poisoning defect (the wrong status-queue consumption cursor could
desynchronise the reply ring under load) and the ring-block ownership
defect that could stall the first client after a reboot. Green clients are
reproduced across fresh boots; a formal every-boot acceptance campaign is
the remaining work.
7. No ad-hoc default-off parameters required for the canonical path — still
open (the forwarding and graphics seams remain parameterised while they
are validated).
Until every gate holds, the honest formulation is: **initialization, the
driver-API path and kernel execution on the physical GPU work and verify;
the staged-transfer class, per-guest isolation and the graphics
direct-GL/present/first-frame frontier remain.**
## 3. How to see it yourself
The reference setup takes about half an hour from a fresh host
([`Install-and-First-Run.md`](Install-and-First-Run.md)); the current expected
outcome is exactly the ladder above — driver probe, `nvidia-smi`, the full
driver-API smoke, and guest CUDA kernels executing and verifying on the
physical GPU. On the graphics side the same setup brings up the display/KMS
stack and the X server on the emulated vGPU (`glxinfo` reports OpenGL 4.6.0
with direct rendering, `glxgears` renders ~520 FPS); the first non-black
GPU-rendered frames are proven off-screen (the pbuffer path produced a
500-frame 45.4 s video on 2026-11-11 and a 50 s capture on 2026-11-12), and
GZDoom reaches its screen-allocation stage with GL 4.6.0 on an even modeNum.
The direct-GL submission is fully decoded, the MME multi-part program now
completes and the draw macro executes, and the DS stage image is authored and
live for both vertex slots; the saved-window release carrier now replays the
held window deterministically when the pack completes, but windowed/screen
pixels remain the frontier (the release call order, the pack-completion
lottery and the stores are the next named gates, and X start keeps a
first-generation lottery) — see [`OpenGL.md`](OpenGL.md). The
risc-v-gsp-rm-debugger workbench
([`Riscv-GSP-RM-Debugger.md`](Riscv-GSP-RM-Debugger.md)) is the host-side
tooling used to study the firmware side of that frontier.
## 4. How this page is maintained
Every claim above carries a date and a tag, maps to a maintained document in
the repository, and is refreshed as milestones land. The execution story, with
its evidence, its resolved theories and its open work, is on
[`Execution-Frontier.md`](Execution-Frontier.md).
## 5. Related pages
* [`Install-and-First-Run.md`](Install-and-First-Run.md)
* [`System-Overview.md`](System-Overview.md)
* [`Execution-Frontier.md`](Execution-Frontier.md)
* [`Hardware-Software-Matrix.md`](Hardware-Software-Matrix.md) — which
hardware and software combinations are Reference, Target or Experimental.
* [`FAQ.md`](FAQ.md)
|
