1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
|
# Practical Challenges and Adversarial Interoperability
> How we intend to implement compute/CUDA support *fast and correctly*, and the
> discipline that keeps the whole stack unmodified — ours included.
## 1. The central engineering bet
> Emulate the interface. Pass through the work. Never patch anyone.
Everything below follows from taking that seriously. It forces hard choices
(no "just add a flag to the guest driver"), and it pays for itself by making
the project portable, maintainable and legally clean.
## 2. Fast and correct: the tension
### 2.1 What "correct" means here
Correctness is not "no crash". It is:
* the guest observes the same **values** (capabilities, handles, table
contents) it would on real hardware,
* the guest observes the same **ordering** (completions happen after the work
they signal; faults arrive before replays),
* the guest observes the same **failure semantics** (errors are surfaced, not
swallowed),
* and the work produces the same **results** (a CUDA kernel's output matches
native execution).
### 2.2 What "fast" means here
Two budgets matter, and they are different:
* **Initialization latency** — how long from VM start to CUDA ready. Dominated
by RPC round-trips and guest driver's own work. Target: within 2× of physical
device initialization in the same guest.
* **Steady-state throughput/latency** — kernel launch overhead, copy
bandwidth, and completion latency. Target: published, stable, and within an
agreed factor of native for each supported operation class.
We will publish the methodology and the numbers together; a performance claim
without a methodology is not accepted in this project.
### 2.3 The fast paths
* **Pure state operations** (semaphores, membar, fences, some memops) are
satisfied in the emulation directly — they are microseconds in hardware and
would otherwise cost a round trip to the physical GPU and back.
* **Management operations** (object creation, capability queries, mappings)
are answered from cached/derived state where the hardware has no side effects
the guest can observe.
* **Real work** (grid launches, copies) is forwarded. Nothing else is allowed
on the hot path.
## 3. Forwarding: the three options and how we choose
```
(A) Host RM/GSP proxy
guest push → translate → host driver's RM API → real channel → GPU
+ reuses a mature driver's knowledge
− extra hop, depends on host driver interfaces
(B) Direct engine submission
guest push → translate → our own channel/push construction → GPU
+ lowest latency, no host driver dependency
− we must own page tables, scheduling, invalidation, completion
(C) Hybrid
management via (A), hot path via (B)
```
**Decision procedure:** prototype (A) first to get *correct* execution, with
measurement harnesses around it. Then measure the overhead. If the overhead
violates the budgets in §2.2 for the target workloads, implement (B) for the
hot path and keep (A) for management. The architecture must not preclude
either, so the work path is abstracted behind an interface from day one.
## 4. Hard problems we must solve, and our stance on each
| Problem | Stance |
|---|---|
| **Page tables on both sides** | We populate server-owned ranges using the layout the client hands us and the entry encodings from AMAP/HAL references; a validator compares our tables against what the guest expects before enabling a channel. |
| **Memory coherence** | Guest sysmem is coherent by construction (same physical pages); VRAM objects must be migrated to real VRAM or DMA-mapped. Phase 2 picks per-object policy with correctness first. |
| **Scheduling between tenants** | One real GPU, N guest channels — schedule explicitly (time-slice/partition), measure fairness, audit isolation. Never let the guest's own scheduler influence another tenant. |
| **Fault handling** | Reproduce faults under forwarding, implement replay/cancel on the guest's terms, and make fault latency measurable. |
| **State explosion** | Drivers create enormous state. We do not model all of it — we model what the guest *observes*, and forward what it *uses*. Anything unobserved stays unmodeled. |
| **Version drift** | Version-keyed handlers with a configuration-driven profile per (driver, firmware) family; new versions added by measurement, never by guessing. |
| **Security of the debug tooling** | Debug parameters are development-grade; the productized build keeps only read-only diagnostics and gates destructive ones behind explicit opt-in. |
## 5. Adversarial interoperability
### 5.1 The rule
**No modification is permitted in any component we do not own**: guest driver,
host driver, GPU firmware, hypervisor, device model, guest OS.
Why so absolute: every permitted exception becomes a maintenance liability that
scales with the number of versions in the field. A patch to the guest driver
means maintaining forks for every guest OS and driver version. A patch to the
hypervisor means every hypervisor project must carry it. A firmware change is
impossible in practice.
### 5.2 How compliance is verified
* **Contract tests**: CI builds run against *stock* drivers obtained from
distribution repositories (or vendor runfiles), and against *unmodified*
hypervisor packages.
* **No-patch checks**: our build must not depend on any patch file for
third-party components. Where patches exist in the tree for convenience
(e.g. build glue), the contract tests must pass without them; anything that
fails this is a bug.
* **Behavioral equivalence**: where we answer from emulation, we validate the
values against traces from real hardware (or from our decompiled study notes) —
not against our own previous answers.
### 5.3 What we *can* change
* Our own module, userspace library, tools, scripts and docs.
* Configuration: device profiles, version profiles, capability tables.
* Tests and harnesses.
### 5.4 Where the line gets tested
* **Firmware signatures**: the driver verifies firmware. We satisfy the
handshake by *emulating the interface*, not by patching the image or the
driver.
* **PCI IDs and VBIOS**: the guest driver decides how to treat the device from
what it reads. We present a coherent device; we never rely on the driver
skipping checks.
* **Licensing checks**: our device is our device; we do not impersonate a
licensed product. Interoperability is the goal, not circumvention of a
vendor's own licensing for their own products.
## 6. Why this approach is durable
Because our artifacts are:
* **Interface definitions** (protocol tables, structures) — survive versions.
* **Measured contracts** (what the guest observes) — survive reimplementation.
* **Configuration** (profiles per version/hardware) — survive hardware.
* **Tests** (contract ladders) — catch drift immediately.
A new driver version or GPU is therefore *data + verification*, not a rewrite —
the difference between a hobby project and infrastructure.
## 7. The "thrilling" part (for developers)
If you are reading this to decide whether to work on it: this project is one of
the few places where you can:
* reverse engineer a 28 MB RISC-V firmware and turn it into maintainable C,
* design a memory model that makes two different address-space owners agree,
* build a virtualization path for the most commercially important accelerator
on Earth, under a permissive license,
* and have your work outlive the hardware generation you started on.
Start with [`../07-developers/onboarding-and-standards.md`](../07-developers/onboarding-and-standards.md).
|
