1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
# Practical Challenges and Adversarial Interoperability
> How we intend to implement compute/CUDA support *fast and correctly*, and the
> discipline that keeps the whole stack unmodified — ours included.
## 1. The central engineering bet
> Emulate the interface. Pass through the work. Never patch anyone.
Everything below follows from taking that seriously. It forces hard choices
(no "just add a flag to the guest driver"), and it pays for itself by making
the project portable, maintainable and legally clean.
## 2. Fast and correct: the tension
### 2.1 What "correct" means here
Correctness is not "no crash". It is:
* the guest observes the same **values** (capabilities, handles, table
contents) it would on real hardware — a control whose reply fields the
driver consumes is only "implemented" once its reply is coherent, not just
`SUCCESS`-shaped (the reply-fidelity ladder: status-only → coherent zeroed
params → coherent values for consumed fields → exact values; see
[`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md)),
* the guest observes the same **ordering** (completions happen after the work
they signal; faults arrive before replays),
* the guest observes the same **failure semantics** (errors are surfaced, not
swallowed),
* and the work produces the same **results** (a CUDA kernel's output matches
native execution).
### 2.2 What "fast" means here
Two budgets matter, and they are different:
* **Initialization latency** — how long from VM start to CUDA ready. Dominated
by RPC round-trips and guest driver's own work. Target: within 2× of physical
device initialization in the same guest.
* **Steady-state throughput/latency** — kernel launch overhead, copy
bandwidth, and completion latency. Target: published, stable, and within an
agreed factor of native for each supported operation class.
We will publish the methodology and the numbers together; a performance claim
without a methodology is not accepted in this project.
### 2.3 The fast paths
* **Pure state operations** (semaphores, membar, fences, some memops) are
satisfied in the emulation directly — they are microseconds in hardware and
would otherwise cost a round trip to the physical GPU and back.
* **Management operations** (object creation, capability queries, mappings)
are answered from cached/derived state where the hardware has no side effects
the guest can observe.
* **Real work** (grid launches, copies) is meant to be forwarded to the
physical GPU (see §3). CE-class copies and their semaphore releases are now
forwarded to the physical copy engine (default off) with a host-side fault
relay, and memsets/copies/releases the host decoder understands still run in
software as a fail-closed correctness bridge. GR (kernel) execution is the
open frontier: a forwarded guest launch reaches the front-end but the inline
launch-descriptor install has not been observed and no grid has dispatched
(context validity at dispatch is ruled out — see
[`03-execution-frontier.md`](03-execution-frontier.md)).
## 3. Forwarding: the options and the chosen mechanism
```
(A) Proxy through the host driver
guest push → translate → host driver's RM → real channel → GPU
Not implementable with an unmodified distribution driver: no callable
RM entry point from kernel context (see 03-architecture/08).
(B) Own the GPU; own scheduling (no GSP)
guest push → translate → our own channels/runlists/page tables → GPU
Reproduces the scheduler and MMU ownership; large, capability-limiting.
(C) Hybrid
Only meaningful once (A) and (B) both exist.
(D) Own the GPU; use its real GSP firmware as the server ← chosen
guest push → translate → real GSP-RM session → real channel → GPU
The module is the host-side resource-manager client; the firmware
schedules.
```
The full analysis — which interfaces exist, why the proxy is unavailable, and
the ownership model — is in
[`../03-architecture/08-physical-gpu-integration.md`](../03-architecture/08-physical-gpu-integration.md).
The work path stays abstracted behind one interface so the backend can be
swapped. Software execution of decoded state/memory operations remains a
correctness bridge for bring-up, not the data plane.
## 4. Hard problems we must solve, and our stance on each
| Problem | Stance |
|---|---|
| **Page tables on both sides** | We populate server-owned ranges using the layout the client hands us and the entry encodings from AMAP/HAL references; a validator compares our tables against what the guest expects before enabling a channel. |
| **Memory coherence** | Guest sysmem is coherent by construction (same physical pages); VRAM objects must be migrated to real VRAM or DMA-mapped. Phase 2 picks per-object policy with correctness first. |
| **Scheduling between tenants** | One real GPU, N guest channels — schedule explicitly (time-slice/partition), measure fairness, audit isolation. Never let the guest's own scheduler influence another tenant. |
| **Fault handling** | Reproduce faults under forwarding, implement replay/cancel on the guest's terms, and make fault latency measurable. |
| **State explosion** | Drivers create enormous state. We do not model all of it — we model what the guest *observes*, and forward what it *uses*. Anything unobserved stays unmodeled. |
| **Version drift** | Version-keyed handlers with a configuration-driven profile per (driver, firmware) family; new versions added by measurement, never by guessing. |
| **Security of the debug tooling** | Debug parameters are development-grade; the productized build keeps only read-only diagnostics and gates destructive ones behind explicit opt-in. |
## 5. Adversarial interoperability
### 5.1 The rule
**No modification is permitted in any component we do not own**: guest driver,
host driver, GPU firmware, hypervisor, device model, guest OS.
Why so absolute: every permitted exception becomes a maintenance liability that
scales with the number of versions in the field. A patch to the guest driver
means maintaining forks for every guest OS and driver version. A patch to the
hypervisor means every hypervisor project must carry it. A firmware change is
impossible in practice.
In this repository, patch files exist only as build glue for the *optional*
OpenRM submodule build (`legacy-patches/nvidia/`, applied by `install.sh`).
The runtime and the guest stack are never patched, and the deployment path uses
the distribution driver — so nothing patched is required to run the stack.
### 5.2 How compliance is verified
* **Contract tests** (goal): CI builds run against *stock* drivers obtained
from distribution repositories (or vendor runfiles), and against
*unmodified* hypervisor packages. Today CI is push/PR-triggered only — builds,
kernel/unit/KUnit/mock tests, static analysis and docs checks; there is no
scheduled guest suite or stock-driver contract test yet, so the guest ladder
is still run by hand (see the operations docs).
* **No-patch checks**: our build must not depend on any patch file for
third-party components. Where patches exist in the tree for convenience
(e.g. build glue for the optional OpenRM submodule), they are development
scaffolding only: the contract tests must pass without them, and the
deployment path uses the unpatched distribution driver; anything that fails
this is a bug.
* **Behavioral equivalence**: where we answer from emulation, we validate the
values against traces from real hardware (or from our decompiled study notes) —
not against our own previous answers.
### 5.3 What we *can* change
* Our own module, userspace library, tools, scripts and docs.
* Configuration: device profiles, version profiles, capability tables.
* Tests and harnesses.
### 5.4 Where the line gets tested
* **Firmware signatures**: the driver verifies firmware. We satisfy the
handshake by *emulating the interface*, not by patching the image or the
driver.
* **PCI IDs and VBIOS**: the guest driver decides how to treat the device from
what it reads. We present a coherent device; we never rely on the driver
skipping checks.
* **Licensing checks**: our device is our device; we do not impersonate a
licensed product. Interoperability is the goal, not circumvention of a
vendor's own licensing for their own products.
## 6. Why this approach is durable
Because our artifacts are:
* **Interface definitions** (protocol tables, structures) — survive versions.
* **Measured contracts** (what the guest observes) — survive reimplementation.
* **Configuration** (profiles per version/hardware) — survive hardware.
* **Tests** (contract ladders) — catch drift immediately.
A new driver version or GPU is therefore *data + verification*, not a rewrite —
the difference between a hobby project and infrastructure.
## 7. The "thrilling" part (for developers)
If you are reading this to decide whether to work on it: this project is one of
the few places where you can:
* reverse engineer a 28 MB RISC-V firmware and turn it into maintainable C,
* design a memory model that makes two different address-space owners agree,
* build a virtualization path for the most commercially important accelerator
on Earth, under a permissive license,
* and have your work outlive the hardware generation you started on.
Start with [`../07-developers/onboarding-and-standards.md`](../07-developers/onboarding-and-standards.md).
|
