# Open-IOV Documentation > **One independent IP block for NVIDIA GPU virtualization.** > Unmodified host drivers. Unmodified guest drivers. No firmware patches. > No hypervisor forks. No licenses to buy. Welcome. This is the front door to the Open-IOV project. If you read only one page, read this one — it tells you what Open-IOV is, why it exists, where it is today, and exactly which document to read next for whatever you care about. --- ## 1. What is Open-IOV? Open-IOV is an open source virtualization layer for NVIDIA GPUs. It runs **stock, unmodified NVIDIA drivers inside virtual machines** — Linux and Windows — with GPU compute (CUDA), and eventually 3D graphics and video, on hardware you already own, without a proprietary vGPU license, without patching the guest driver, without patching the host driver, and without patching the GPU firmware. It does this by implementing, in a host kernel module, the *interface* the guest driver expects to talk to — the GPU System Processor (GSP) and its resource manager firmware, GSP-RM — and by arranging for the *work* the guest submits to run on real GPU hardware. Think of Open-IOV as a **virtualization IP block**: a self-contained component that can be attached to many ecosystems (KVM, Xen, Argo, OpenXT, Qubes OS Air, ...), many guest operating systems (Linux, Windows, ...), many hardware generations (Turing, Ampere, Hopper, Grace-Hopper, Grace-Blackwell, GB10, ...) and many driver versions — without any of those ecosystems, operating systems, drivers or devices having to change. ``` ┌─────────────────────────────────────────────┐ │ GUEST VM │ │ stock nvidia.ko + CUDA / OpenGL / Vulkan │ └──────────────┬──────────────────────────────┘ │ PCIe (emulated device) ┌───────────────────────────────┴───────────────────────────────┐ │ HYPERVISOR (KVM/Xen) │ │ VFIO / mdev transport, IOMMU isolation │ └───────────────────────────────┬───────────────────────────────┘ │ ┌───────────────────────────────┴───────────────────────────────┐ │ HOST: open-iov-vfio.ko (this project) │ │ • virtual GPU device (BARs, config space, MSI-X) │ │ • GSP / GSP-RM interface emulation (RPC, msgq, controls) │ │ • memory model + GPU work forwarding to the physical GPU │ └───────────────────────────────┬───────────────────────────────┘ │ ┌──────────────┴──────────────┐ │ UNMODIFIED host nvidia.ko │ │ (or GPU passthrough path) │ └──────────────┬──────────────┘ │ ┌───────┴────────┐ │ NVIDIA GPU │ └────────────────┘ ``` ## 2. Why does this need to exist? Three real needs converge: 1. **Compute in guests, on hardware people have.** Multi-tenant AI systems are moving toward *bare-metal* vLLM continuous batching inside NVIDIA SuperPOD-style clusters — and that is the right answer for centralized datacenter scale. But there is a large class of systems that still need *compute runtimes inside guests*: Windows and Linux desktops and servers that need CUDA, 3D, and video; edge devices that must run inference locally; and fleets where several VMs must share the GPU without discrete device assignment. Today, the only way to do that is a proprietary, licensed, driver-version-locked vGPU stack. 2. **Secure AI agents at the edge.** An agent runtime (Claude Code, OpenCode, ...) should never share a trust boundary with the model server, with the hypervisor's other tenants, or with your data. GPUs are the one resource that makes this hard: they are big, stateful and not designed for partitioning. Virtualizing them *without* a license and *without* closed host components is what makes "local inference, full isolation, no cloud dependency" practical. 3. **Long-term durability.** Any GPU virtualization stack must survive new driver versions, new firmware, new hardware and new hypervisors. The only sustainable way to do that is to own the *interface*, not a snapshot of it — and to make the project a durable, independent IP block with a permissive license (BSD-3-Clause) so anyone can carry it forward. See [`01-orientation/01-purpose-and-scope.md`](01-orientation/01-purpose-and-scope.md) and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai-edge-and-ecosystem.md). ## 3. Where is the project today? **Honest status (2026-09-29):** | Area | State | |------|-------| | Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | ✅ Working | | GSP boot emulation (FWSEC/booter/RISCV handshake) | ✅ Working | | GSP-RM message queue + RPC transport | ✅ Working | | GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | 🟡 Mostly — the engine inventory now includes the async copy engine and the CE capabilities match the device; reply fidelity for the remaining consumed fields is still being tightened | | Guest `nvidia.ko` probe, `nvidia-smi` | ✅ Working (TU104) | | Guest CUDA driver (`libcuda`) startup | 🟡 Partially — `cuInit` and device enumeration work; the client reaches `cuCtxCreate` and currently ends in a forced `999` on a completion wait while the completion/translation plumbing is made reliable. See [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) and [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) | | CUDA kernel execution | 🔴 Not yet — context creation must complete first, then real work forwarding; the execution backend is chosen but not started ([`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md)); today only decoded memsets/copies/semaphore releases execute, in the host | | OpenGL / Vulkan | 🔴 Not yet | | Windows guest | 🔴 Not yet | | Multi-tenant isolation (VFs, mdev-per-VM) | 🟡 Designed, partially implemented | We also know exactly where the CUDA bring-up stands and what the next step is: the userspace `libcuda` segfault was fixed (an RM allocation reply now echoes the request parameters instead of returning zeros), the engine inventory is complete, and the client reaches context creation, which currently ends on a completion wait while decoded work is made to execute reliably. The full mechanism, with offsets and the decompiled call path, is in [`../userspace/cuda/docs/05-570.144-decompilation.md`](../userspace/cuda/docs/05-570.144-decompilation.md). Beyond that, the missing piece for kernel execution is forwarding the guest's *work* (push buffers, page trees, execution) to the physical GPU under our control; the chosen backend — a module-owned GPU running its real GSP firmware as the resource-manager server — is specified in [`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md). ## 4. How to read these docs (progressive disclosure) Read in order if you are new. Jump directly if you know what you need. > **Debugging the CUDA bring-up?** Start with > [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) — > a five-minute guide to the userspace driver analysis, the corrected crash > path, the current completion frontier and the chosen execution backend, and > the lookup recipes for `libcuda+0x...` offsets. It links back into this tree > for the interface and emulation contract. | # | Document | Audience | Time | |---|----------|----------|------| | 1 | [Purpose and scope](01-orientation/01-purpose-and-scope.md) | everyone | 10 min | | 2 | [The AI edge and the ecosystem](01-orientation/02-the-ai-edge-and-ecosystem.md) | everyone | 15 min | | 3 | [Install and first run](02-getting-started/01-install-and-first-run.md) | users, evaluators | 20 min | | 4 | [System overview](03-architecture/01-system-overview.md) | everyone technical | 20 min | | 5 | [Guest stack and CUDASW](03-architecture/02-guest-stack-and-cudasw.md) | developers | 25 min | | 6 | [GSP and GSP-RM](03-architecture/03-gsp-and-gsp-rm.md) | developers | 30 min | | 7 | [UVM](03-architecture/04-uvm.md) | developers | 25 min | | 8 | [GMMU, AMAPlib, HALs, runlist, GR](03-architecture/05-gmmu-amap-hal-gr.md) | developers | 25 min | | 9 | [Host module and hybrid model](03-architecture/06-host-module-and-hybrid-model.md) | developers | 25 min | | 10 | [Driver lifecycle field guide](03-architecture/07-driver-lifecycle-field-guide.md) | developers, debuggers | 30 min | | 11 | [Physical GPU integration](03-architecture/08-physical-gpu-integration.md) | developers, architects | 20 min | | 12 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min | | 13 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min | | 14 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min | | 15 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min | | 16 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min | | 17 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min | | 18 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min | | 19 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference | Deep-dive material lives in [`../wiki/`](../wiki/README.md) (firmware decompilation record, address reference, historical protocol notes). Wiki pages that predate the current implementation carry a status banner naming the canonical document in this tree; use the banner, not the old body. The pre-reboot flat documents have been merged into this tree and removed (2026-09-23). Where their material went: * debugging: gdbstub/NMI workflows, the EPT-livelock case study and the failure taxonomy → [`05-operations/02-debugging-and-maintenance-workflows.md`](05-operations/02-debugging-and-maintenance-workflows.md) * performance: the microbenchmark list and the A/B methodology → [`05-operations/01-performance-correctness-security.md`](05-operations/01-performance-correctness-security.md) * version upgrades: the firmware/driver upgrade runbook (with the corrected host/guest framing) → [`05-operations/02-debugging-and-maintenance-workflows.md`](05-operations/02-debugging-and-maintenance-workflows.md) §4 * kernel-code pitfalls and the must/must-not rules → [`07-developers/onboarding-and-standards.md`](07-developers/onboarding-and-standards.md) * the OpenRM build/symbol/load order → [`03-architecture/06-host-module-and-hybrid-model.md`](03-architecture/06-host-module-and-hybrid-model.md) §7 * architecture, portability and implementation-plan material is now covered by [`03-architecture/01-system-overview.md`](03-architecture/01-system-overview.md), [`03-architecture/06-host-module-and-hybrid-model.md`](03-architecture/06-host-module-and-hybrid-model.md), [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md), [`04-implementation/02-practical-challenges-and-adversarial-interop.md`](04-implementation/02-practical-challenges-and-adversarial-interop.md) and [`06-portability/hardware-software-matrix.md`](06-portability/hardware-software-matrix.md). What remains historical is the pre-supersession wiki material, `old/ARCHITECTURE.md`, and [`../STATE.md`](../STATE.md) (a 2026-09-21 snapshot); their status banners name the canonical document. ## 5. What makes Open-IOV different? | | Proprietary vGPU | Discrete passthrough (VFIO) | Emulated GPU (QEMU) | **Open-IOV** | |---|---|---|---|---| | Guest driver | vendor vGPU driver | stock driver | stock driver (limited) | **stock driver** | | Guest CUDA | yes | yes | no/emulated | **yes (goal: real execution)** | | Multiple VMs per GPU | yes (licensed, capped) | no (1 VM = 1 GPU) | yes (no accel) | **yes** | | License | paid, per-VM | free | free | **BSD-3-Clause** | | Host driver changes | vendor stack | none | none | **none** | | Guest driver changes | vendor build | none | none | **none** | | Survives driver updates | vendor-controlled | yes | n/a | **designed for it** | | Windows + Linux guests | yes | yes | limited | **yes (goal)** | The distinguishing property is **adversarial interoperability**: we reimplement the *behavior* of the NVIDIA interface — firmware, RPC protocol, memory model, scheduling — from observation and public documentation, so that no component in the stack has to be modified or licensed. ## 6. How to get involved * **Use it.** Try the reference setup, file issues with logs (our debugging docs tell you exactly what to capture). * **Test it.** Hardware and driver matrix contributions are the single most valuable thing you can send — especially non-Turing hardware, Windows guests, and non-KVM hypervisors. * **Extend it.** The architecture docs mark the interfaces that are expected to grow (memory model, forwarding backends, profile-based portability — a design target today, see [`06-portability/hardware-software-matrix.md` §6](06-portability/hardware-software-matrix.md)). * **Integrate it.** If you build a hypervisor, an edge platform or a security stack, see [`08-ecosystem/haaeep-and-collaboration.md`](08-ecosystem/haaeep-and-collaboration.md). * **Fund or co-develop it.** Commercial use is explicitly welcome under BSD-3-Clause; see the same document for collaboration models. ## 7. License Open-IOV is **BSD 3-Clause** ([`../LICENSE`](../LICENSE)) for everything reusable — userspace libraries, tools, scripts, configuration and docs — and **GPL-2.0** for kernel modules (`kernel/**`) that must legally link against GPL-only kernel exports (VFIO, mdev, IOMMU). Per-file SPDX identifiers govern. How we study vendor interfaces (reverse engineering and decompilation) and what we publish is described in [`09-reference/legal-and-licensing.md`](09-reference/legal-and-licensing.md). --- *This documentation tree is the authoritative entry point for the project. If you find a document that contradicts it, the contradiction is a bug: please report it.*