path: root/docs/README.md

# Open-IOV Documentation

> **One independent IP block for NVIDIA GPU virtualization.**
> Unmodified host drivers. Unmodified guest drivers. No firmware patches.
> No hypervisor forks. No licenses to buy.

Welcome. This is the front door to the Open-IOV project. If you read only one
page, read this one — it tells you what Open-IOV is, why it exists, where it is
today, and exactly which document to read next for whatever you care about.

---

## 1. What is Open-IOV?

Open-IOV is an open source virtualization layer for NVIDIA GPUs. It runs
**stock, unmodified NVIDIA drivers inside virtual machines** — Linux and
Windows — with GPU compute (CUDA), and eventually 3D graphics and video, on
hardware you already own, without a proprietary vGPU license, without patching
the guest driver, without patching the host driver, and without patching the
GPU firmware.

It does this by implementing, in a host kernel module, the *interface* the
guest driver expects to talk to — the GPU System Processor (GSP) and its
resource manager firmware, GSP-RM — and by arranging for the *work* the guest
submits to run on real GPU hardware.

Think of Open-IOV as a **virtualization IP block**: a self-contained component
that can be attached to many ecosystems (KVM, Xen, Argo, OpenXT, Qubes OS
Air, ...), many guest operating systems (Linux, Windows, ...), many hardware
generations (Turing, Ampere, Hopper, Grace-Hopper, Grace-Blackwell, GB10, ...)
and many driver versions — without any of those ecosystems, operating systems,
drivers or devices having to change.

```mermaid
flowchart TB
  subgraph VM["Guest VM - stock drivers, stock applications"]
    A["CUDA / OpenGL / Vulkan applications"]
    B["Userspace stack - libcuda, libcudart, libGLX, libvulkan"]
    C["Stock nvidia.ko - RM core and UVM"]
    A --> B --> C
  end
  subgraph HYP["Hypervisor - KVM and QEMU"]
    D["VFIO mdev transport + IOMMU isolation"]
  end
  subgraph MOD["open-iov-vfio.ko - the only host component"]
    E["Virtual device - BARs, config space, MSI-X"]
    F["GSP boot + GSP-RM emulation"]
    G["Memory model + work path"]
    H["Execution backend - real GSP session"]
    E --> F --> G --> H
  end
  C -->|"emulated PCIe"| D --> E
  H --> I["Physical GPU - engines, GMMU, VRAM"]
  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  classDef hw fill:#e2e8f0,stroke:#4a5568
  class E,F done
  class G wip
  class H wip
  class I hw
```

The full visual map — every component, status tag and interaction — is in
[`03-architecture/09-full-system-picture.md`](03-architecture/09-full-system-picture.md)
(end-to-end) and
[`03-architecture/10-interaction-and-scheduling.md`](03-architecture/10-interaction-and-scheduling.md)
(call paths, queues, interrupts, scheduling).

## 2. Why does this need to exist?

Three real needs converge:

1. **Compute in guests, on hardware people have.** Multi-tenant AI systems are
   moving toward *bare-metal* vLLM continuous batching inside NVIDIA
   SuperPOD-style clusters — and that is the right answer for centralized
   datacenter scale. But there is a large class of systems that still need
   *compute runtimes inside guests*: Windows and Linux desktops and servers that
   need CUDA, 3D, and video; edge devices that must run inference locally; and
   fleets where several VMs must share the GPU without discrete device
   assignment. Today, the only way to do that is a proprietary, licensed,
   driver-version-locked vGPU stack.
2. **Secure AI agents at the edge.** An agent runtime (Claude Code, OpenCode,
   ...) should never share a trust boundary with the model server, with the
   hypervisor's other tenants, or with your data. GPUs are the one resource
   that makes this hard: they are big, stateful and not designed for
   partitioning. Virtualizing them *without* a license and *without* closed
   host components is what makes "local inference, full isolation, no cloud
   dependency" practical.
3. **Long-term durability.** Any GPU virtualization stack must survive new
   driver versions, new firmware, new hardware and new hypervisors. The only
   sustainable way to do that is to own the *interface*, not a snapshot of it —
   and to make the project a durable, independent IP block with a permissive
   license (BSD-3-Clause) so anyone can carry it forward.

See [`01-orientation/01-purpose-and-scope.md`](01-orientation/01-purpose-and-scope.md)
and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai-edge-and-ecosystem.md).

## 3. Where is the project today?

**Honest status (2026-10-02):**

A dated snapshot of this status, refreshed on every progress check, is kept at
[`../STATE-CURRENT.md`](../STATE-CURRENT.md).

| Area | State |
|------|-------|
| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | ✅ Working |
| GSP boot emulation (FWSEC/booter/RISCV handshake) | ✅ Working |
| GSP-RM message queue + RPC transport | ✅ Working |
| GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | 🟡 Mostly — the engine inventory now includes the async copy engine and the CE capabilities match the device; reply fidelity for the remaining consumed fields is still being tightened |
| Guest `nvidia.ko` probe, `nvidia-smi` | ✅ Working (TU104) |
| Guest CUDA driver (`libcuda`) startup | ✅ Driver-API baseline — `cuInit`, device enumeration, `cuCtxCreate`, `cuMemAlloc`, copies, free and destroy complete end to end; the smoke test prints `ALL OK` and the 64-byte payload round-trips exactly (64/64, re-verified 2026-10-02 on the current build). See [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) and [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) |
| CUDA kernel execution | 🟡 Copy-engine-class guest copies and their releases are forwarded to the real hardware with release and fault relay in place (parameters default off); GR (graphics/compute) execution is the open wall — the host GR ladder brings up a GR0 channel with a compute-class object, but its first push faults on a context that was never committed. The device is currently held in the designed locked fail-safe awaiting a supervised reboot. See [`03-architecture/11-physical-backend-guest-wiring.md`](03-architecture/11-physical-backend-guest-wiring.md), [`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md) and [`../STATE-CURRENT.md`](../STATE-CURRENT.md) |
| OpenGL / Vulkan | 🔴 Not yet |
| Windows guest | 🔴 Not yet |
| Multi-tenant isolation (VFs, mdev-per-VM) | 🟡 Designed, partially implemented |

We also know exactly where the CUDA bring-up stands and what the next step is:
the userspace `libcuda` segfault was fixed (an RM allocation reply now echoes
the request parameters instead of returning zeros), the engine inventory is
complete, and the driver-API smoke test now completes end to end — `cuCtxCreate`
returns and the 64-byte payload round-trips exactly (64/64, re-verified
2026-10-02 on the current build). The full
mechanism, with offsets and the decompiled call path, is in
[`../userspace/cuda/docs/05-570.144-decompilation.md`](../userspace/cuda/docs/05-570.144-decompilation.md).
The remaining frontier is guest-facing execution: copy-engine-class copies and
their releases are forwarded to real hardware with release relay and fault
relay in place (parameters default off); GR execution is the open wall — the
host GR ladder brings up a GR0 channel with a compute-class object, but its
first push faults on an engine context that was never committed. The device is
currently held in the designed locked fail-safe awaiting a supervised reboot.
The backend, a module-owned GPU running its real GSP firmware as the
resource-manager server, is specified in
[`03-architecture/08-physical-gpu-integration.md`](03-architecture/08-physical-gpu-integration.md);
the bridge from the live session to the guest-facing mdev/VFIO path is specified
in
[`03-architecture/11-physical-backend-guest-wiring.md`](03-architecture/11-physical-backend-guest-wiring.md).

For the visual end-state map and the cross-component call paths, see
[`03-architecture/09-full-system-picture.md`](03-architecture/09-full-system-picture.md)
and
[`03-architecture/10-interaction-and-scheduling.md`](03-architecture/10-interaction-and-scheduling.md).

## 4. How to read these docs (progressive disclosure)

Read in order if you are new. Jump directly if you know what you need.

> **Debugging the CUDA bring-up?** Start with
> [`../userspace/cuda/READ-ME-FIRST.md`](../userspace/cuda/READ-ME-FIRST.md) —
> a five-minute guide to the userspace driver analysis, the corrected crash
> path, the current completion frontier and the chosen execution backend, and
> the lookup recipes for `libcuda+0x...` offsets. It links back into this tree
> for the interface and emulation contract.

| # | Document | Audience | Time |
|---|----------|----------|------|
| 1 | [Purpose and scope](01-orientation/01-purpose-and-scope.md) | everyone | 10 min |
| 2 | [The AI edge and the ecosystem](01-orientation/02-the-ai-edge-and-ecosystem.md) | everyone | 15 min |
| 3 | [Install and first run](02-getting-started/01-install-and-first-run.md) | users, evaluators | 20 min |
| 4 | [System overview](03-architecture/01-system-overview.md) | everyone technical | 20 min |
| 5 | [The full system picture](03-architecture/09-full-system-picture.md) | everyone technical | 30 min |
| 6 | [Interaction and scheduling](03-architecture/10-interaction-and-scheduling.md) | developers | 30 min |
| 7 | [Guest stack and CUDASW](03-architecture/02-guest-stack-and-cudasw.md) | developers | 25 min |
| 8 | [GSP and GSP-RM](03-architecture/03-gsp-and-gsp-rm.md) | developers | 30 min |
| 9 | [UVM](03-architecture/04-uvm.md) | developers | 25 min |
| 10 | [GMMU, AMAPlib, HALs, runlist, GR](03-architecture/05-gmmu-amap-hal-gr.md) | developers | 25 min |
| 11 | [Host module and hybrid model](03-architecture/06-host-module-and-hybrid-model.md) | developers | 25 min |
| 12 | [Driver lifecycle field guide](03-architecture/07-driver-lifecycle-field-guide.md) | developers, debuggers | 30 min |
| 13 | [Physical GPU integration](03-architecture/08-physical-gpu-integration.md) | developers, architects | 20 min |
| 14 | [Physical backend guest wiring](03-architecture/11-physical-backend-guest-wiring.md) | developers, architects | 25 min |
| 15 | [Host VF acceleration — future goal (deferred)](03-architecture/12-host-vf-acceleration-future-goal.md) | architects, planners | 10 min |
| 16 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min |
| 17 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min |
| 18 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min |
| 19 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min |
| 20 | [GSP in a RISC-V VM (risc-v-gsp-rm-debugger, experimental)](05-operations/03-risc-v-gsp-rm-vm.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
| 21 | [risc-v-gsp-rm-debugger user guide (experimental)](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
| 22 | [Defensive Operations Runbook](RUNBOOK.md) | maintainers, operators | 20 min |
| 23 | [Engineering operations](eng-ops.md) — tool admission and workflows (Part I) plus live physical-backend operations (Part II) | maintainers, developers | 20 min |
| 24 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min |
| 25 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min |
| 26 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min |
| 27 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference |

<!-- change (2026-10-02): insert the risc-v-gsp-rm-debugger user guide and the Open-IOV
Defensive Operations Runbook into the reading list; renumber rows 21-24 to
23-26; additive only. -->

<!-- change (2026-10-02): insert the engineering-operations document into the
operations group; renumber rows 23-26 to 24-27; additive only. -->

Row 15 is a **deferred future goal**: host NVIDIA-driver acceleration over a
virtual function. The feasibility verdict and a first design sketch are
recorded there for planning; it is not on the active critical path (guest CUDA
execution, multi-VM, 3D, performance come first).

> **Tool documentation is under development.** The per-tool pages under
> [`tools/`](tools/README.md) (for example `rmtool.md` and `rmtop.md`) are
> being expanded; until they are complete, start with the
> [Defensive Operations Runbook](RUNBOOK.md) and the
> [risc-v-gsp-rm-debugger user guide](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md).

<!-- change (2026-10-02): state that the per-tool documentation section is
under development and point readers at the RUNBOOK and the risc-v-gsp-rm-debugger user
guide; additive only. -->

Deep-dive material lives in [`../wiki/`](../wiki/README.md) (firmware
decompilation record, address reference, historical protocol notes). Wiki
pages that predate the current implementation carry a status banner naming
the canonical document in this tree; use the banner, not the old body.

The pre-reboot flat documents have been merged into this tree and removed
(2026-09-23). Where their material went:

* debugging: gdbstub/NMI workflows, the EPT-livelock case study and the
  failure taxonomy →
  [`05-operations/02-debugging-and-maintenance-workflows.md`](05-operations/02-debugging-and-maintenance-workflows.md)
* defensive operations: incident scenarios, the OODA ordering, evidence
  bundles and the escalation tree →
  [`RUNBOOK.md`](RUNBOOK.md); the risc-v-gsp-rm-debugger quickstart, tool chooser and
  troubleshooting →
  [`05-operations/04-risc-v-gsp-rm-debugger-user-guide.md`](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md)

<!-- change (2026-10-02): point the post-merge document map at the new RUNBOOK
and risc-v-gsp-rm-debugger user guide; additive only. -->

* firmware runtime analysis (experimental): running GSP/uproc firmware in a
  RISC-V QEMU VM with hooks, watchpoints, memory dumps and `gsp-rm-vfio`
  accuracy measurement — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) →
  [`05-operations/03-risc-v-gsp-rm-vm.md`](05-operations/03-risc-v-gsp-rm-vm.md)
  (tooling in [`../debug/risc-v-gsp-rm-vm/`](../debug/risc-v-gsp-rm-vm/README.md))
* performance: the microbenchmark list and the A/B methodology →
  [`05-operations/01-performance-correctness-security.md`](05-operations/01-performance-correctness-security.md)
* version upgrades: the firmware/driver upgrade runbook (with the corrected
  host/guest framing) →
  [`05-operations/02-debugging-and-maintenance-workflows.md`](05-operations/02-debugging-and-maintenance-workflows.md) §4
* kernel-code pitfalls and the must/must-not rules →
  [`07-developers/onboarding-and-standards.md`](07-developers/onboarding-and-standards.md)
* the OpenRM build/symbol/load order →
  [`03-architecture/06-host-module-and-hybrid-model.md`](03-architecture/06-host-module-and-hybrid-model.md) §7
* architecture, portability and implementation-plan material is now covered by
  [`03-architecture/01-system-overview.md`](03-architecture/01-system-overview.md),
  [`03-architecture/06-host-module-and-hybrid-model.md`](03-architecture/06-host-module-and-hybrid-model.md),
  [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md),
  [`04-implementation/02-practical-challenges-and-adversarial-interop.md`](04-implementation/02-practical-challenges-and-adversarial-interop.md)
  and [`06-portability/hardware-software-matrix.md`](06-portability/hardware-software-matrix.md).

What remains historical is the pre-supersession wiki material and
`old/ARCHITECTURE.md`; the latter carries an explicit superseded banner
pointing at this page, and the older wiki pages carry status banners naming
the canonical document. (The
former 2026-09-21 `STATE.md` snapshot is retired; its content lives on in the
git history and its replacements above.)

## 5. What makes Open-IOV different?

| | Proprietary vGPU | Discrete passthrough (VFIO) | Emulated GPU (QEMU) | **Open-IOV** |
|---|---|---|---|---|
| Guest driver | vendor vGPU driver | stock driver | stock driver (limited) | **stock driver** |
| Guest CUDA | yes | yes | no/emulated | **yes (goal: real execution)** |
| Multiple VMs per GPU | yes (licensed, capped) | no (1 VM = 1 GPU) | yes (no accel) | **yes** |
| License | paid, per-VM | free | free | **BSD-3-Clause** |
| Host driver changes | vendor stack | none | none | **none** |
| Guest driver changes | vendor build | none | none | **none** |
| Survives driver updates | vendor-controlled | yes | n/a | **designed for it** |
| Windows + Linux guests | yes | yes | limited | **yes (goal)** |

The distinguishing property is **adversarial interoperability**: we reimplement
the *behavior* of the NVIDIA interface — firmware, RPC protocol, memory model,
scheduling — from observation and public documentation, so that no component in
the stack has to be modified or licensed.

## 6. How to get involved

* **Use it.** Try the reference setup, file issues with logs (our debugging
  docs tell you exactly what to capture).
* **Test it.** Hardware and driver matrix contributions are the single most
  valuable thing you can send — especially non-Turing hardware, Windows
  guests, and non-KVM hypervisors.
* **Extend it.** The architecture docs mark the interfaces that are expected to
  grow (memory model, forwarding backends, profile-based portability — a design
  target today, see [`06-portability/hardware-software-matrix.md` §6](06-portability/hardware-software-matrix.md)).
* **Integrate it.** If you build a hypervisor, an edge platform or a security
  stack, see [`08-ecosystem/haaeep-and-collaboration.md`](08-ecosystem/haaeep-and-collaboration.md).
* **Fund or co-develop it.** Commercial use is explicitly welcome under
  BSD-3-Clause; see the same document for collaboration models.

## 7. License

Open-IOV is **BSD 3-Clause** ([`../LICENSE`](../LICENSE)) for everything
reusable — userspace libraries, tools, scripts, configuration and docs —
and **GPL-2.0** for kernel modules (`kernel/**`) that must legally link
against GPL-only kernel exports (VFIO, mdev, IOMMU). Per-file SPDX
identifiers govern. How we study vendor interfaces (reverse engineering and
decompilation) and what we publish is described in
[`09-reference/legal-and-licensing.md`](09-reference/legal-and-licensing.md).

---

*This documentation tree is the authoritative entry point for the project. If
you find a document that contradicts it, the contradiction is a bug: please
report it.*