1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
|
# Open-IOV Documentation
> **One independent IP block for NVIDIA GPU virtualization.**
> Unmodified host drivers. Unmodified guest drivers. No firmware patches.
> No hypervisor forks. No licenses to buy.
Welcome. This is the front door to the Open-IOV project. If you read only one
page, read this one — it tells you what Open-IOV is, why it exists, where it is
today, and exactly which document to read next for whatever you care about.
---
## 1. What is Open-IOV?
Open-IOV is an open source virtualization layer for NVIDIA GPUs. It runs
**stock, unmodified NVIDIA drivers inside virtual machines** — Linux and
Windows — with GPU compute (CUDA), and eventually 3D graphics and video, on
hardware you already own, without a proprietary vGPU license, without patching
the guest driver, without patching the host driver, and without patching the
GPU firmware.
It does this by implementing, in a host kernel module, the *interface* the
guest driver expects to talk to — the GPU System Processor (GSP) and its
resource manager firmware, GSP-RM — and by arranging for the *work* the guest
submits to run on real GPU hardware.
Think of Open-IOV as a **virtualization IP block**: a self-contained component
that can be attached to many ecosystems (KVM, Xen, Argo, OpenXT, Qubes OS
Air, ...), many guest operating systems (Linux, Windows, ...), many hardware
generations (Turing, Ampere, Hopper, Grace-Hopper, Grace-Blackwell, GB10, ...)
and many driver versions — without any of those ecosystems, operating systems,
drivers or devices having to change.
```
┌─────────────────────────────────────────────┐
│ GUEST VM │
│ stock nvidia.ko + CUDA / OpenGL / Vulkan │
└──────────────┬──────────────────────────────┘
│ PCIe (emulated device)
┌───────────────────────────────┴───────────────────────────────┐
│ HYPERVISOR (KVM/Xen) │
│ VFIO / mdev transport, IOMMU isolation │
└───────────────────────────────┬───────────────────────────────┘
│
┌───────────────────────────────┴───────────────────────────────┐
│ HOST: open-iov-vfio.ko (this project) │
│ • virtual GPU device (BARs, config space, MSI-X) │
│ • GSP / GSP-RM interface emulation (RPC, msgq, controls) │
│ • memory model + GPU work forwarding to the physical GPU │
└───────────────────────────────┬───────────────────────────────┘
│
┌──────────────┴──────────────┐
│ UNMODIFIED host nvidia.ko │
│ (or GPU passthrough path) │
└──────────────┬──────────────┘
│
┌───────┴────────┐
│ NVIDIA GPU │
└────────────────┘
```
## 2. Why does this need to exist?
Three real needs converge:
1. **Compute in guests, on hardware people have.** Multi-tenant AI systems are
moving toward *bare-metal* vLLM continuous batching inside NVIDIA
SuperPOD-style clusters — and that is the right answer for centralized
datacenter scale. But there is a large class of systems that still need
*compute runtimes inside guests*: Windows and Linux desktops and servers that
need CUDA, 3D, and video; edge devices that must run inference locally; and
fleets where several VMs must share the GPU without discrete device
assignment. Today, the only way to do that is a proprietary, licensed,
driver-version-locked vGPU stack.
2. **Secure AI agents at the edge.** An agent runtime (Claude Code, OpenCode,
...) should never share a trust boundary with the model server, with the
hypervisor's other tenants, or with your data. GPUs are the one resource
that makes this hard: they are big, stateful and not designed for
partitioning. Virtualizing them *without* a license and *without* closed
host components is what makes "local inference, full isolation, no cloud
dependency" practical.
3. **Long-term durability.** Any GPU virtualization stack must survive new
driver versions, new firmware, new hardware and new hypervisors. The only
sustainable way to do that is to own the *interface*, not a snapshot of it —
and to make the project a durable, independent IP block with a permissive
license (BSD-3-Clause) so anyone can carry it forward.
See [`01-orientation/01-purpose-and-scope.md`](01-orientation/01-purpose-and-scope.md)
and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai-edge-and-ecosystem.md).
## 3. Where is the project today?
**Honest status (2026-09-23):**
| Area | State |
|------|-------|
| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | ✅ Working |
| GSP boot emulation (FWSEC/booter/RISCV handshake) | ✅ Working |
| GSP-RM message queue + RPC transport | ✅ Working |
| GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | ✅ Working |
| Guest `nvidia.ko` probe, `nvidia-smi` | ✅ Working (TU104) |
| Guest CUDA driver (`libcuda`) startup | 🟡 Partially — UVM advances through channel creation and page-tree init; see [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) |
| CUDA kernel execution | 🔴 Not yet — requires GPU work forwarding |
| OpenGL / Vulkan | 🔴 Not yet |
| Windows guest | 🔴 Not yet |
| Multi-tenant isolation (VFs, mdev-per-VM) | 🟡 Designed, partially implemented |
We also know exactly *why* CUDA kernels do not run yet, and what the path is:
see the known-knowns/known-unknowns document. The short version: the emulation
is now faithful enough that the guest stack reaches UVM initialization; the
missing piece is forwarding the guest's *work* (push buffers, page trees,
execution) to the physical GPU under our control.
## 4. How to read these docs (progressive disclosure)
Read in order if you are new. Jump directly if you know what you need.
| # | Document | Audience | Time |
|---|----------|----------|------|
| 1 | [Purpose and scope](01-orientation/01-purpose-and-scope.md) | everyone | 10 min |
| 2 | [The AI edge and the ecosystem](01-orientation/02-the-ai-edge-and-ecosystem.md) | everyone | 15 min |
| 3 | [Install and first run](02-getting-started/01-install-and-first-run.md) | users, evaluators | 20 min |
| 4 | [System overview](03-architecture/01-system-overview.md) | everyone technical | 20 min |
| 5 | [Guest stack and CUDASW](03-architecture/02-guest-stack-and-cudasw.md) | developers | 25 min |
| 6 | [GSP and GSP-RM](03-architecture/03-gsp-and-gsp-rm.md) | developers | 30 min |
| 7 | [UVM](03-architecture/04-uvm.md) | developers | 25 min |
| 8 | [GMMU, AMAPlib, HALs, runlist, GR](03-architecture/05-gmmu-amap-hal-gr.md) | developers | 25 min |
| 9 | [Host module and hybrid model](03-architecture/06-host-module-and-hybrid-model.md) | developers | 25 min |
| 10 | [Driver lifecycle field guide](03-architecture/07-driver-lifecycle-field-guide.md) | developers, debuggers | 30 min |
| 11 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min |
| 12 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min |
| 13 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min |
| 14 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min |
| 15 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min |
| 16 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min |
| 17 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min |
| 18 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference |
Existing deep-dive material (kept, still valid, being folded into this tree):
the `wiki/` directory (firmware analysis, address reference, wire protocol,
RPC message format) and the `docs/` reference documents
(`GSP_EMULATION_ARCHITECTURE.md`, `DEBUGGING.md`, `PERFORMANCE.md`,
`BUILD_NVIDIA_OPENRM.md`, `VERSION_UPGRADE.md`).
## 5. What makes Open-IOV different?
| | Proprietary vGPU | Discrete passthrough (VFIO) | Emulated GPU (QEMU) | **Open-IOV** |
|---|---|---|---|---|
| Guest driver | vendor vGPU driver | stock driver | stock driver (limited) | **stock driver** |
| Guest CUDA | yes | yes | no/emulated | **yes (goal: real execution)** |
| Multiple VMs per GPU | yes (licensed, capped) | no (1 VM = 1 GPU) | yes (no accel) | **yes** |
| License | paid, per-VM | free | free | **BSD-3-Clause** |
| Host driver changes | vendor stack | none | none | **none** |
| Guest driver changes | vendor build | none | none | **none** |
| Survives driver updates | vendor-controlled | yes | n/a | **designed for it** |
| Windows + Linux guests | yes | yes | limited | **yes (goal)** |
The distinguishing property is **adversarial interoperability**: we reimplement
the *behavior* of the NVIDIA interface — firmware, RPC protocol, memory model,
scheduling — from observation and public documentation, so that no component in
the stack has to be modified or licensed.
## 6. How to get involved
* **Use it.** Try the reference setup, file issues with logs (our debugging
docs tell you exactly what to capture).
* **Test it.** Hardware and driver matrix contributions are the single most
valuable thing you can send — especially non-Turing hardware, Windows
guests, and non-KVM hypervisors.
* **Extend it.** The architecture docs mark the interfaces that are expected to
grow (memory model, forwarding backends, config-driven portability).
* **Integrate it.** If you build a hypervisor, an edge platform or a security
stack, see [`08-ecosystem/haaeep-and-collaboration.md`](08-ecosystem/haaeep-and-collaboration.md).
* **Fund or co-develop it.** Commercial use is explicitly welcome under
BSD-3-Clause; see the same document for collaboration models.
## 7. License
Open-IOV is **BSD 3-Clause** ([`../LICENSE`](../LICENSE)) for everything
reusable — userspace libraries, tools, scripts, configuration and docs —
and **GPL-2.0** for kernel modules (`kernel/**`) that must legally link
against GPL-only kernel exports (VFIO, mdev, IOMMU). Per-file SPDX
identifiers govern. How we study vendor interfaces (reverse engineering and
decompilation) and what we publish is described in
[`09-reference/legal-and-licensing.md`](09-reference/legal-and-licensing.md).
---
*This documentation tree is the authoritative entry point for the project. If
you find a document that contradicts it, the contradiction is a bug: please
report it.*
|
