1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
# Purpose and Scope
> Read this first. It defines what Open-IOV is for, what it is not for, and how
> we will judge whether we are succeeding.
## 1. Mission
**Make NVIDIA GPUs fully virtualizable, by anyone, for any hypervisor, without
modifying NVIDIA software, firmware or hardware.**
Concretely, Open-IOV builds an independent, maintained IP block that:
* presents a virtual NVIDIA GPU to a guest operating system,
* lets the guest run the **stock NVIDIA driver** (Linux and Windows),
* supports **compute (CUDA)** first, then **3D graphics** and **video**,
* runs on ordinary NVIDIA GPUs already deployed in the field,
* isolates guests from each other and from the host,
* and does not require a license key, a proprietary host service, a patched
guest driver, or a specific hypervisor.
## 2. What problem are we solving?
Today there are four ways to use an NVIDIA GPU with virtual machines, and all
of them have a hard limitation:
| Approach | Limitation |
|---|---|
| Discrete device assignment (VFIO passthrough) | One GPU per VM. Complex, fragile for laptops and consumer PCs. No sharing. |
| NVIDIA proprietary vGPU | Requires a license. Requires a vendor host stack. Driver-version locked. Windows/Linux only where the vendor ships builds. |
| Software emulation of a GPU (e.g. QEMU's virtual GPUs, virgl) | No real CUDA. No real 3D performance. Not usable for AI. |
| Cloud-only inference | Needs an external service. Unacceptable for secure edge, offline, or private data. |
Open-IOV occupies the gap: **real GPU acceleration, multiple VMs, stock
drivers, no license.**
## 3. Scope
### In scope
* **Host-side virtualization of NVIDIA GPUs** for Linux hosts via VFIO/mdev
(KVM now; Xen and others later), as a self-contained kernel module plus a
small userspace library.
* **Full guest-visible device emulation**: PCI configuration space, BARs
(MMIO register space, VRAM aperture, instance memory), MSI-X, doorbells,
power management stubs.
* **GSP and GSP-RM interface emulation**: firmware boot handshake, message
queues, RPC transport, resource-manager controls, object model, event
channels, and the pieces of the hosted resource manager the guest depends on.
* **A memory model** that connects guest GPU address spaces to real GPU memory
and to guest system memory, including page tables and fault handling.
* **Compute forwarding**: taking the work the guest submits (channels, push
buffers, runlists, compute commands) and executing it on the physical GPU in
a way that is correct, isolated and fast.
* **Graphics and video forwarding** (later phase, same architecture).
* **Guest support**: Linux (x86_64 and aarch64) and Windows (x86_64).
* **Host support**: Linux (x86_64 now, aarch64 later), KVM today, Xen/Argo/
OpenXT/Qubes OS Air on the roadmap.
* **Hardware portability**: Turing (TU104) today; Ampere, Hopper,
Grace-Hopper, Grace-Blackwell and GB10 targets; a configuration-driven
adaptation layer for new GPUs.
### Out of scope (non-goals)
* **Replacing bare-metal inference serving.** vLLM's continuous batching on
bare-metal hosts is how the industry scales multi-tenant inference in
datacenters, and it is better at that than any virtualization layer will be.
Open-IOV is for the cases where compute must live *inside* a guest.
* **Forking or patching the NVIDIA driver**, in guest or host.
* **Forking or patching GPU firmware.**
* **Forking or patching hypervisors** (QEMU, KVM, Xen, Argo, OpenXT, ...) to
make the device work.
* **Anti-cheat / DRM circumvention**, cloud-gaming-specifically defeating
vendor licensing checks for their own products. We interoperate; we do not
counterfeit. (We do not need a license for *our* device, because it is our
device.)
* **Emulating GPU instruction execution in software.** Software emulation of
CUDA kernels is 100–1000× too slow; work must run on real silicon.
## 4. Value
For **users**: GPU acceleration in VMs on hardware you own, with no per-VM fee,
on Windows and Linux, across generations of GPUs.
For **integrators and hypervisor projects**: a GPU component that behaves like
any other virtual device, enabling security models that were previously
impossible (see HAAEEP in
[`../08-ecosystem/haaeep-and-collaboration.md`](../08-ecosystem/haaeep-and-collaboration.md)).
For **the wider ecosystem**: an independent, permissively licensed
implementation of a previously single-vendor interface, with public
documentation of the protocol, the firmware and the architecture — a public
good that outlives any one vendor's product decisions.
## 5. Model of multi-tenancy
Open-IOV is built for **multiple independent tenants on one GPU**:
* Each VM gets its own mediated device instance (mdev) and its own
guest-visible GPU.
* Memory is isolated by the platform IOMMU: a tenant's device can only reach
*that tenant's* memory.
* Interrupts are delivered per instance; no shared vectors.
* The host module holds no tenant-secret state; everything is per-VM.
* Scheduling between tenants is explicit policy (time-sliced now; MIG-backed
partitioning later where hardware supports it).
* The security posture is *fail closed*: if the isolation assumptions cannot be
verified at run time, the device refuses to start.
How this is enforced and audited is documented in
[`../05-operations/01-performance-correctness-security.md`](../05-operations/01-performance-correctness-security.md).
## 6. How we judge success
We evaluate the project on five axes; each has a dedicated methodology
document:
1. **Initialization** — the guest driver loads, probes, and initializes to the
same point it does on bare metal: `nvidia-smi`, CUDA runtime, `cuInit`,
context creation, module load.
2. **Runtime** — the guest *runs* real work: CUDA kernels with correct results,
OpenGL/Vulkan rendering, encode/decode; repeated, sustained, resilient.
3. **Interoperability** — no changes required anywhere: guest driver builds,
host driver builds, firmware, hypervisor, device model software. Both host
and guest firmware interfaces are honored.
4. **Stability** — crash-free under stress: start/stop cycles, VM kill, guest
driver reload, host driver reload, suspend/resume, migration.
5. **Performance and security** — within an agreed factor of native for the
supported cases, with isolation that survives adversarial review.
## 7. Where we are, and where we need improvement
* **Working today**: the virtual device, the GSP boot handshake, the RPC
transport, the resource-manager emulation surface the guest driver needs to
probe and enumerate, guest `nvidia.ko` and `nvidia-smi`, and the full CUDA
driver-API path through context creation, allocation and verified copies
(the driver-API smoke passes).
* **Needs improvement**: engine execution first — the backend is live and
CE-class copies and their releases are forwarded to the physical copy engine
(default off) with a fault relay, but for CUDA kernel execution the guest's
launch now reaches the front-end and the inline launch-descriptor install has
not been observed (context validity at dispatch is ruled out; see
[`../04-implementation/03-execution-frontier.md`](../04-implementation/03-execution-frontier.md))
— then the remaining reply fidelity (some consumed
fields are still generic fallbacks), the memory/page-table model, Windows
guest support, non-KVM hypervisors, and performance instrumentation. Each of
these is tracked with explicit "known-unknowns" in
[`../04-implementation/01-known-knowns-and-unknowns.md`](../04-implementation/01-known-knowns-and-unknowns.md);
for the CUDA analysis area, start at
[`../../userspace/cuda/READ-ME-FIRST.md`](../../userspace/cuda/READ-ME-FIRST.md).
## 8. Guiding principles
1. **Own the interface, not a snapshot of it.** Every emulated interface is
described by a version-agnostic model plus per-version configuration; new
versions are added by measurement, not by rewriting.
2. **Emulate the interface, pass through the work.** Anything that must be fast
runs on real hardware; everything the guest can observe is virtual.
3. **No patches anywhere.** If a change is needed in someone else's software,
firmware or hardware, our design is wrong.
4. **Make the invisible visible.** Instrument everything; keep
`dmesg`-readable telemetry for every decision the emulation makes.
5. **Fail closed, fail loudly.** Isolation failures stop the device; protocol
surprises are logged with enough context to reproduce them.
6. **Document for the next person.** A feature is not done until a newcomer can
understand and maintain it from these docs alone.
7. **Permissive and portable by design** (BSD-3-Clause): adoption is a feature.
|
