Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 191 insertions, 0 deletions

diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..fff4f82
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,191 @@
+# Open-IOV Documentation
+
+> **One independent IP block for NVIDIA GPU virtualization.**
+> Unmodified host drivers. Unmodified guest drivers. No firmware patches.
+> No hypervisor forks. No licenses to buy.
+
+Welcome. This is the front door to the Open-IOV project. If you read only one
+page, read this one — it tells you what Open-IOV is, why it exists, where it is
+today, and exactly which document to read next for whatever you care about.
+
+---
+
+## 1. What is Open-IOV?
+
+Open-IOV is an open source virtualization layer for NVIDIA GPUs. It runs
+**stock, unmodified NVIDIA drivers inside virtual machines** — Linux and
+Windows — with GPU compute (CUDA), and eventually 3D graphics and video, on
+hardware you already own, without a proprietary vGPU license, without patching
+the guest driver, without patching the host driver, and without patching the
+GPU firmware.
+
+It does this by implementing, in a host kernel module, the *interface* the
+guest driver expects to talk to — the GPU System Processor (GSP) and its
+resource manager firmware, GSP-RM — and by arranging for the *work* the guest
+submits to run on real GPU hardware.
+
+Think of Open-IOV as a **virtualization IP block**: a self-contained component
+that can be attached to many ecosystems (KVM, Xen, Argo, OpenXT, Qubes OS
+Air, ...), many guest operating systems (Linux, Windows, ...), many hardware
+generations (Turing, Ampere, Hopper, Grace-Hopper, Grace-Blackwell, GB10, ...)
+and many driver versions — without any of those ecosystems, operating systems,
+drivers or devices having to change.
+
+```
+                    ┌─────────────────────────────────────────────┐
+                    │              GUEST VM                       │
+                    │  stock nvidia.ko  +  CUDA / OpenGL / Vulkan │
+                    └──────────────┬──────────────────────────────┘
+                                   │ PCIe (emulated device)
+   ┌───────────────────────────────┴───────────────────────────────┐
+   │                       HYPERVISOR (KVM/Xen)                    │
+   │              VFIO / mdev transport, IOMMU isolation           │
+   └───────────────────────────────┬───────────────────────────────┘
+                                   │
+   ┌───────────────────────────────┴───────────────────────────────┐
+   │              HOST: open-iov-vfio.ko  (this project)           │
+   │  • virtual GPU device (BARs, config space, MSI-X)             │
+   │  • GSP / GSP-RM interface emulation (RPC, msgq, controls)     │
+   │  • memory model + GPU work forwarding to the physical GPU     │
+   └───────────────────────────────┬───────────────────────────────┘
+                                   │
+                    ┌──────────────┴──────────────┐
+                    │  UNMODIFIED host nvidia.ko  │
+                    │   (or GPU passthrough path) │
+                    └──────────────┬──────────────┘
+                                   │
+                           ┌───────┴────────┐
+                           │  NVIDIA GPU    │
+                           └────────────────┘
+```
+
+## 2. Why does this need to exist?
+
+Three real needs converge:
+
+1. **Compute in guests, on hardware people have.** Multi-tenant AI systems are
+   moving toward *bare-metal* vLLM continuous batching inside NVIDIA
+   SuperPOD-style clusters — and that is the right answer for centralized
+   datacenter scale. But there is a large class of systems that still need
+   *compute runtimes inside guests*: Windows and Linux desktops and servers that
+   need CUDA, 3D, and video; edge devices that must run inference locally; and
+   fleets where several VMs must share the GPU without discrete device
+   assignment. Today, the only way to do that is a proprietary, licensed,
+   driver-version-locked vGPU stack.
+2. **Secure AI agents at the edge.** An agent runtime (Claude Code, OpenCode,
+   ...) should never share a trust boundary with the model server, with the
+   hypervisor's other tenants, or with your data. GPUs are the one resource
+   that makes this hard: they are big, stateful and not designed for
+   partitioning. Virtualizing them *without* a license and *without* closed
+   host components is what makes "local inference, full isolation, no cloud
+   dependency" practical.
+3. **Long-term durability.** Any GPU virtualization stack must survive new
+   driver versions, new firmware, new hardware and new hypervisors. The only
+   sustainable way to do that is to own the *interface*, not a snapshot of it —
+   and to make the project a durable, independent IP block with a permissive
+   license (BSD-3-Clause) so anyone can carry it forward.
+
+See [`01-orientation/01-purpose-and-scope.md`](01-orientation/01-purpose-and-scope.md)
+and [`01-orientation/02-the-ai-edge-and-ecosystem.md`](01-orientation/02-the-ai-edge-and-ecosystem.md).
+
+## 3. Where is the project today?
+
+**Honest status (2026-09-23):**
+
+| Area | State |
+|------|-------|
+| Emulated vGPU device (PCI, BARs, MSI-X, VFIO mdev) | ✅ Working |
+| GSP boot emulation (FWSEC/booter/RISCV handshake) | ✅ Working |
+| GSP-RM message queue + RPC transport | ✅ Working |
+| GSP-RM controls: device/fb/bus tables, CE caps, vaspace, GMMU static info | ✅ Working |
+| Guest `nvidia.ko` probe, `nvidia-smi` | ✅ Working (TU104) |
+| Guest CUDA driver (`libcuda`) startup | 🟡 Partially — UVM advances through channel creation and page-tree init; see [`04-implementation/01-known-knowns-and-unknowns.md`](04-implementation/01-known-knowns-and-unknowns.md) |
+| CUDA kernel execution | 🔴 Not yet — requires GPU work forwarding |
+| OpenGL / Vulkan | 🔴 Not yet |
+| Windows guest | 🔴 Not yet |
+| Multi-tenant isolation (VFs, mdev-per-VM) | 🟡 Designed, partially implemented |
+
+We also know exactly *why* CUDA kernels do not run yet, and what the path is:
+see the known-knowns/known-unknowns document. The short version: the emulation
+is now faithful enough that the guest stack reaches UVM initialization; the
+missing piece is forwarding the guest's *work* (push buffers, page trees,
+execution) to the physical GPU under our control.
+
+## 4. How to read these docs (progressive disclosure)
+
+Read in order if you are new. Jump directly if you know what you need.
+
+| # | Document | Audience | Time |
+|---|----------|----------|------|
+| 1 | [Purpose and scope](01-orientation/01-purpose-and-scope.md) | everyone | 10 min |
+| 2 | [The AI edge and the ecosystem](01-orientation/02-the-ai-edge-and-ecosystem.md) | everyone | 15 min |
+| 3 | [Install and first run](02-getting-started/01-install-and-first-run.md) | users, evaluators | 20 min |
+| 4 | [System overview](03-architecture/01-system-overview.md) | everyone technical | 20 min |
+| 5 | [Guest stack and CUDASW](03-architecture/02-guest-stack-and-cudasw.md) | developers | 25 min |
+| 6 | [GSP and GSP-RM](03-architecture/03-gsp-and-gsp-rm.md) | developers | 30 min |
+| 7 | [UVM](03-architecture/04-uvm.md) | developers | 25 min |
+| 8 | [GMMU, AMAPlib, HALs, runlist, GR](03-architecture/05-gmmu-amap-hal-gr.md) | developers | 25 min |
+| 9 | [Host module and hybrid model](03-architecture/06-host-module-and-hybrid-model.md) | developers | 25 min |
+| 10 | [Driver lifecycle field guide](03-architecture/07-driver-lifecycle-field-guide.md) | developers, debuggers | 30 min |
+| 11 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min |
+| 12 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min |
+| 13 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min |
+| 14 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min |
+| 15 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min |
+| 16 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min |
+| 17 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min |
+| 18 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference |
+
+Existing deep-dive material (kept, still valid, being folded into this tree):
+the `wiki/` directory (firmware analysis, address reference, wire protocol,
+RPC message format) and the `docs/` reference documents
+(`GSP_EMULATION_ARCHITECTURE.md`, `DEBUGGING.md`, `PERFORMANCE.md`,
+`BUILD_NVIDIA_OPENRM.md`, `VERSION_UPGRADE.md`).
+
+## 5. What makes Open-IOV different?
+
+| | Proprietary vGPU | Discrete passthrough (VFIO) | Emulated GPU (QEMU) | **Open-IOV** |
+|---|---|---|---|---|
+| Guest driver | vendor vGPU driver | stock driver | stock driver (limited) | **stock driver** |
+| Guest CUDA | yes | yes | no/emulated | **yes (goal: real execution)** |
+| Multiple VMs per GPU | yes (licensed, capped) | no (1 VM = 1 GPU) | yes (no accel) | **yes** |
+| License | paid, per-VM | free | free | **BSD-3-Clause** |
+| Host driver changes | vendor stack | none | none | **none** |
+| Guest driver changes | vendor build | none | none | **none** |
+| Survives driver updates | vendor-controlled | yes | n/a | **designed for it** |
+| Windows + Linux guests | yes | yes | limited | **yes (goal)** |
+
+The distinguishing property is **adversarial interoperability**: we reimplement
+the *behavior* of the NVIDIA interface — firmware, RPC protocol, memory model,
+scheduling — from observation and public documentation, so that no component in
+the stack has to be modified or licensed.
+
+## 6. How to get involved
+
+* **Use it.** Try the reference setup, file issues with logs (our debugging
+  docs tell you exactly what to capture).
+* **Test it.** Hardware and driver matrix contributions are the single most
+  valuable thing you can send — especially non-Turing hardware, Windows
+  guests, and non-KVM hypervisors.
+* **Extend it.** The architecture docs mark the interfaces that are expected to
+  grow (memory model, forwarding backends, config-driven portability).
+* **Integrate it.** If you build a hypervisor, an edge platform or a security
+  stack, see [`08-ecosystem/haaeep-and-collaboration.md`](08-ecosystem/haaeep-and-collaboration.md).
+* **Fund or co-develop it.** Commercial use is explicitly welcome under
+  BSD-3-Clause; see the same document for collaboration models.
+
+## 7. License
+
+Open-IOV is **BSD 3-Clause** ([`../LICENSE`](../LICENSE)) for everything
+reusable — userspace libraries, tools, scripts, configuration and docs —
+and **GPL-2.0** for kernel modules (`kernel/**`) that must legally link
+against GPL-only kernel exports (VFIO, mdev, IOMMU). Per-file SPDX
+identifiers govern. How we study vendor interfaces (reverse engineering and
+decompilation) and what we publish is described in
+[`09-reference/legal-and-licensing.md`](09-reference/legal-and-licensing.md).
+
+---
+
+*This documentation tree is the authoritative entry point for the project. If
+you find a document that contradicts it, the contradiction is a bug: please
+report it.*