Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 116 insertions, 0 deletions

diff --git a/docs/BUILD_NVIDIA_OPENRM.md b/docs/BUILD_NVIDIA_OPENRM.md
new file mode 100644
index 0000000..5699036
--- /dev/null
+++ b/docs/BUILD_NVIDIA_OPENRM.md
@@ -0,0 +1,116 @@
+# Building NVIDIA OpenRM with Open-IOV vGPU VFIO Symbols
+
+## Overview
+
+The NVIDIA OpenRM `nvidia.ko` kernel module requires the vGPU VFIO interface symbols exported by `open-iov-vfio.ko`:
+- `nvidia_vgpu_vfio_probe`
+- `nvidia_vgpu_vfio_remove`
+- `nvidia_vgpu_vfio_get_ops`
+- `nvidia_vgpu_vfio_set_ops`
+- `nv_vgpu_mdev_register_parent`
+- `nv_vgpu_mdev_unregister_parent`
+- And all GSP RPC integration symbols
+
+These symbols must be resolved at build time via `KBUILD_EXTRA_SYMBOLS` pointing to `open-iov-vfio.ko`'s `Module.symvers`.
+
+## Build Order (Critical)
+
+```
+1. Build open-iov-vfio.ko          → Generates kernel/Module.symvers
+2. Build NVIDIA OpenRM with KBUILD_EXTRA_SYMBOLS=kernel/Module.symvers
+3. Load nvidia.ko first
+4. Load open-iov-vfio.ko
+```
+
+## Automated Build (Recommended)
+
+Use the helper script:
+
+```bash
+cd open-iov
+./scripts/build_nvidia_openrm.sh
+```
+
+This script:
+1. Verifies `open-iov-vfio.ko` has been built (checks for `Module.symvers`)
+2. Cleans previous NVIDIA OpenRM build
+3. Builds with `KBUILD_EXTRA_SYMBOLS` correctly set
+4. Verifies vGPU symbols are resolved
+
+## Manual Build
+
+```bash
+# Step 1: Build Open-IOV kernel module
+cd $OPEN_IOV_ROOT
+make kernel
+
+# Step 2: Build NVIDIA OpenRM with extra symbols
+cd $OPEN_IOV_ROOT/nvidia-open-gpu-kernel-modules/kernel-open
+KBUILD_EXTRA_SYMBOLS=$OPEN_IOV_ROOT/kernel/Module.symvers make -j$(nproc) modules
+
+# Step 3: Verify symbols resolved
+grep -i vgpu Module.symvers
+```
+
+## Verification
+
+After building, verify the symbols are resolved:
+
+```bash
+# Check Module.symvers has vGPU symbols
+grep -i vgpu $OPEN_IOV_ROOT/nvidia-open-gpu-kernel-modules/kernel-open/Module.symvers
+
+# Check nvidia.ko modinfo
+modinfo nvidia-open-gpu-kernel-modules/kernel-open/nvidia.ko | grep -i vgpu
+```
+
+## Common Issues
+
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| `Unknown symbol nvidia_vgpu_vfio_probe` | `KBUILD_EXTRA_SYMBOLS` not passed or wrong path | Use helper script or verify path to `Module.symvers` |
+| `Module.symvers not found` | open-iov-vfio.ko not built yet | Run `make kernel` first |
+| Symbols in Module.symvers but not in nvidia.ko | Stale build cache | Run `make clean` in kernel-open before rebuild |
+| `mdev_set_iommu_device` conftest fails | Kernel config missing IOMMU support | Enable `CONFIG_IOMMU_API` in kernel config |
+
+## Module Loading Order
+
+```bash
+# 1. Load NVIDIA OpenRM (provides vGPU VFIO symbols)
+sudo insmod nvidia-open-gpu-kernel-modules/kernel-open/nvidia.ko
+
+# 2. Load Open-IOV VFIO driver (registers mdev parent with nvidia.ko)
+sudo insmod kernel/open-iov-vfio.ko
+
+# 3. Copy firmware
+sudo cp firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin /lib/firmware/
+
+# 4. Create mdev device
+./scripts/create_mdev.sh
+
+# 5. Launch QEMU with vGPU
+./scripts/launch_qemu_vgpu.sh
+```
+
+## Key Files
+
+| File | Purpose |
+|------|---------|
+| `scripts/build_nvidia_openrm.sh` | Automated build script |
+| `kernel/Module.symvers` | Exported symbols from open-iov-vfio.ko |
+| `nvidia-open-gpu-kernel-modules/kernel-open/Kbuild` | Has `KBUILD_EXTRA_SYMBOLS` assignment |
+| `nvidia-open-gpu-kernel-modules/kernel-open/Makefile` | Passes `KBUILD_EXTRA_SYMBOLS` to Kbuild |
+
+## Troubleshooting
+
+If symbols are still unresolved after rebuild:
+
+```bash
+# Full clean rebuild
+cd nvidia-open-gpu-kernel-modules/kernel-open
+make clean
+KBUILD_EXTRA_SYMBOLS=$OPEN_IOV_ROOT/kernel/Module.symvers make -j$(nproc) modules
+
+# Check modpost output for unresolved symbols
+KBUILD_EXTRA_SYMBOLS=$OPEN_IOV_ROOT/kernel/Module.symvers make -j$(nproc) modules V=1 2>&1 | grep -i "modpost\|nvidia_vgpu"
+```
\ No newline at end of file