diff options
Diffstat (limited to 'docs/README.md')
| -rw-r--r-- | docs/README.md | 191 |
1 files changed, 191 insertions, 0 deletions
diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..fff4f82 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,191 @@ +# Open-IOV Documentation + +> **One independent IP block for NVIDIA GPU virtualization.** +> Unmodified host drivers. Unmodified guest drivers. No firmware patches. +> No hypervisor forks. No licenses to buy. + +Welcome. This is the front door to the Open-IOV project. If you read only one +page, read this one — it tells you what Open-IOV is, why it exists, where it is +today, and exactly which document to read next for whatever you care about. + +--- + +## 1. What is Open-IOV? + +Open-IOV is an open source virtualization layer for NVIDIA GPUs. It runs +**stock, unmodified NVIDIA drivers inside virtual machines** — Linux and +Windows — with GPU compute (CUDA), and eventually 3D graphics and video, on +hardware you already own, without a proprietary vGPU license, without patching +the guest driver, without patching the host driver, and without patching the +GPU firmware. + +It does this by implementing, in a host kernel module, the *interface* the +guest driver expects to talk to — the GPU System Processor (GSP) and its +resource manager firmware, GSP-RM — and by arranging for the *work* the guest +submits to run on real GPU hardware. + +Think of Open-IOV as a **virtualization IP block**: a self-contained component +that can be attached to many ecosystems (KVM, Xen, Argo, OpenXT, Qubes OS +Air, ...), many guest operating systems (Linux, Windows, ...), many hardware +generations (Turing, Ampere, Hopper, Grace-Hopper, Grace-Blackwell, GB10, ...) +and many driver versions — without any of those ecosystems, operating systems, +drivers or devices having to change. + +``` + ┌─────────────────────────────────────────────┐ + │ GUEST VM │ + │ stock nvidia.ko + CUDA / OpenGL / Vulkan │ + └──────────────┬──────────────────────────────┘ + │ PCIe (emulated device) + ┌───────────────────────────────┴───────────────────────────────┐ + │ HYPERVISOR (KVM/Xen) │ + │ VFIO / mdev transport, IOMMU isolation │ + └───────────────────────────────┬───────────────────────────────┘ + │ + ┌───────────────────────────────┴───────────────────────────────┐ + │ HOST: open-iov-vfio.ko (this project) │ + │ • virtual GPU device (BARs, config space, MSI-X) │ + │ • GSP / GSP-RM interface emulation (RPC, msgq, controls) │ + │ • memory model + GPU work forwarding to the physical GPU │ + └───────────────────────────────┬───────────────────────────────┘ + │ + ┌──────────────┴──────────────┐ + │ UNMODIFIED host nvidia.ko │ + │ (or GPU passthrough path) │ + └──────────────┬──────────────┘ + │ + ┌───────┴────────┐ + │ NVIDIA GPU │ + └────────────────┘ +``` + +## 2. Why does this need to exist? + +Three real needs converge: + +1. **Compute in guests, on hardware people have.** Multi-tenant AI systems are + moving toward *bare-metal* vLLM continuous batching inside NVIDIA + SuperPOD-style clusters — and that is the right answer for centralized + datacenter scale. But there is a large class of systems that still need + *compute runtimes inside guests*: Windows and Linux desktops and servers that + need CUDA, 3D, and video; edge devices that must run inference locally; and + fleets where several VMs must share the GPU without discrete device + assignment. Today, the only way to do that is a proprietary, licensed, + driver-version-locked vGPU stack. +2. **Secure AI agents at the edge.** An agent runtime (Claude Code, OpenCode, + ...) should never share a trust boundary with the model server, with the + hypervisor's other tenants, or with your data. GPUs are the one resource + that makes this hard: they are big, stateful and not designed for + partitioning. Virtualizing them *without* a license and *without* closed + host components is what makes "local inference, full isolation, no cloud + dependency" practical. +3. **Long-term durability.** Any GPU virtualization stack must survive new + driver versions, new firmware, new hardware and new hypervisors. The only + sustainable way to do that is to own the *interface*, not a snapshot of it — + and to make the project a durable, independent IP block with a permissive + license (BSD-3-Clause) so anyone can carry it forward. + +See [`01-orientation/01-purpose-and-scope.md`](01-orientation/01-purpose-and-scope.md) +and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai-edge-and-ecosystem.md). + +## 3. Where is the project today? + +**Honest status (2026-09-23):** + +| Area | State | +|------|-------| +| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | ✅ Working | +| GSP boot emulation (FWSEC/booter/RISCV handshake) | ✅ Working | +| GSP-RM message queue + RPC transport | ✅ Working | +| GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | ✅ Working | +| Guest `nvidia.ko` probe, `nvidia-smi` | ✅ Working (TU104) | +| Guest CUDA driver (`libcuda`) startup | 🟡 Partially — UVM advances through channel creation and page-tree init; see [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) | +| CUDA kernel execution | 🔴 Not yet — requires GPU work forwarding | +| OpenGL / Vulkan | 🔴 Not yet | +| Windows guest | 🔴 Not yet | +| Multi-tenant isolation (VFs, mdev-per-VM) | 🟡 Designed, partially implemented | + +We also know exactly *why* CUDA kernels do not run yet, and what the path is: +see the known-knowns/known-unknowns document. The short version: the emulation +is now faithful enough that the guest stack reaches UVM initialization; the +missing piece is forwarding the guest's *work* (push buffers, page trees, +execution) to the physical GPU under our control. + +## 4. How to read these docs (progressive disclosure) + +Read in order if you are new. Jump directly if you know what you need. + +| # | Document | Audience | Time | +|---|----------|----------|------| +| 1 | [Purpose and scope](01-orientation/01-purpose-and-scope.md) | everyone | 10 min | +| 2 | [The AI edge and the ecosystem](01-orientation/02-the-ai-edge-and-ecosystem.md) | everyone | 15 min | +| 3 | [Install and first run](02-getting-started/01-install-and-first-run.md) | users, evaluators | 20 min | +| 4 | [System overview](03-architecture/01-system-overview.md) | everyone technical | 20 min | +| 5 | [Guest stack and CUDASW](03-architecture/02-guest-stack-and-cudasw.md) | developers | 25 min | +| 6 | [GSP and GSP-RM](03-architecture/03-gsp-and-gsp-rm.md) | developers | 30 min | +| 7 | [UVM](03-architecture/04-uvm.md) | developers | 25 min | +| 8 | [GMMU, AMAPlib, HALs, runlist, GR](03-architecture/05-gmmu-amap-hal-gr.md) | developers | 25 min | +| 9 | [Host module and hybrid model](03-architecture/06-host-module-and-hybrid-model.md) | developers | 25 min | +| 10 | [Driver lifecycle field guide](03-architecture/07-driver-lifecycle-field-guide.md) | developers, debuggers | 30 min | +| 11 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min | +| 12 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min | +| 13 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min | +| 14 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min | +| 15 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min | +| 16 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min | +| 17 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min | +| 18 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference | + +Existing deep-dive material (kept, still valid, being folded into this tree): +the `wiki/` directory (firmware analysis, address reference, wire protocol, +RPC message format) and the `docs/` reference documents +(`GSP_EMULATION_ARCHITECTURE.md`, `DEBUGGING.md`, `PERFORMANCE.md`, +`BUILD_NVIDIA_OPENRM.md`, `VERSION_UPGRADE.md`). + +## 5. What makes Open-IOV different? + +| | Proprietary vGPU | Discrete passthrough (VFIO) | Emulated GPU (QEMU) | **Open-IOV** | +|---|---|---|---|---| +| Guest driver | vendor vGPU driver | stock driver | stock driver (limited) | **stock driver** | +| Guest CUDA | yes | yes | no/emulated | **yes (goal: real execution)** | +| Multiple VMs per GPU | yes (licensed, capped) | no (1 VM = 1 GPU) | yes (no accel) | **yes** | +| License | paid, per-VM | free | free | **BSD-3-Clause** | +| Host driver changes | vendor stack | none | none | **none** | +| Guest driver changes | vendor build | none | none | **none** | +| Survives driver updates | vendor-controlled | yes | n/a | **designed for it** | +| Windows + Linux guests | yes | yes | limited | **yes (goal)** | + +The distinguishing property is **adversarial interoperability**: we reimplement +the *behavior* of the NVIDIA interface — firmware, RPC protocol, memory model, +scheduling — from observation and public documentation, so that no component in +the stack has to be modified or licensed. + +## 6. How to get involved + +* **Use it.** Try the reference setup, file issues with logs (our debugging + docs tell you exactly what to capture). +* **Test it.** Hardware and driver matrix contributions are the single most + valuable thing you can send — especially non-Turing hardware, Windows + guests, and non-KVM hypervisors. +* **Extend it.** The architecture docs mark the interfaces that are expected to + grow (memory model, forwarding backends, config-driven portability). +* **Integrate it.** If you build a hypervisor, an edge platform or a security + stack, see [`08-ecosystem/haaeep-and-collaboration.md`](08-ecosystem/haaeep-and-collaboration.md). +* **Fund or co-develop it.** Commercial use is explicitly welcome under + BSD-3-Clause; see the same document for collaboration models. + +## 7. License + +Open-IOV is **BSD 3-Clause** ([`../LICENSE`](../LICENSE)) for everything +reusable — userspace libraries, tools, scripts, configuration and docs — +and **GPL-2.0** for kernel modules (`kernel/**`) that must legally link +against GPL-only kernel exports (VFIO, mdev, IOMMU). Per-file SPDX +identifiers govern. How we study vendor interfaces (reverse engineering and +decompilation) and what we publish is described in +[`09-reference/legal-and-licensing.md`](09-reference/legal-and-licensing.md). + +--- + +*This documentation tree is the authoritative entry point for the project. If +you find a document that contradicts it, the contradiction is a bug: please +report it.* |