Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 158 insertions, 0 deletions

diff --git a/tools/README.md b/tools/README.md
new file mode 100644
index 0000000..f9e0018
--- /dev/null
+++ b/tools/README.md
@@ -0,0 +1,158 @@
+# Open-IOV Development Tools
+
+Collection of scripts and utilities for vGPU development, debugging, and testing.
+
+## Directory Structure
+
+```
+tools/
+├── module_ops/           # Kernel module load/unload/reload utilities
+├── debug/                # Debugging and diagnostic tools
+├── test/                 # Test runners and validation scripts
+├── vm/                   # VM/QEMU management for vGPU testing
+├── firmware/             # GSP firmware management
+└── monitoring/           # Real-time monitoring utilities
+```
+
+## Quick Start
+
+```bash
+# Load modules in converged mode (host + vGPU)
+sudo ./module_ops/load_converged.sh
+
+# Unload all modules cleanly
+sudo ./module_ops/unload_all.sh
+
+# Run dry-run validation
+sudo ./module_ops/dry_run.sh
+
+# Monitor kernel logs in real-time
+./monitoring/watch_dmesg.sh
+
+# Verify vGPU sysfs interface
+./debug/verify_vgpu_sysfs.sh
+```
+
+## Module Operations
+
+### `module_ops/load_converged.sh`
+Loads the complete vGPU stack in converged mode:
+1. `mdev` core module
+2. `nvidia.ko` (patched open-gpu-kernel-modules with NV_VGPU_KVM_BUILD)
+3. `nvidia-modeset.ko`, `nvidia-uvm.ko`, `nvidia-drm.ko`
+4. `nvidia-vgpu-vfio.ko` (open-iov kernel module)
+
+### `module_ops/unload_all.sh`
+Cleanly unloads all modules in reverse order with proper cleanup.
+
+### `module_ops/dry_run.sh`
+Runs nvidia-vgpu-vfio.ko in dry-run mode to validate firmware, BAR layout, IRQ caps, IOMMU without hardware changes.
+
+### `module_ops/reload_vgpu.sh`
+Reloads only the vGPU kernel module (nvidia-vgpu-vfio.ko) while keeping host nvidia.ko loaded.
+
+## Debug Tools
+
+### `debug/verify_vgpu_sysfs.sh`
+Verifies mdev parent registration and vGPU types in sysfs.
+
+### `debug/check_module_state.sh`
+Checks module initstate, refcnt, and holder information.
+
+### `debug/dump_vgpu_regs.sh`
+Dumps vGPU device registers via debugfs (if available).
+
+### `debug/capture_crash_logs.sh`
+Captures kernel logs, dmesg, and module state after a crash.
+
+## Test Scripts
+
+### `test/run_integration_tests.sh`
+Runs the integration test suite (host_init, guest_init).
+
+### `test/test_mdev_create.sh`
+Creates an mdev device and verifies basic functionality.
+
+### `test/benchmark_vgpu.sh`
+Runs performance benchmarks on vGPU.
+
+## VM Management
+
+### `vm/create_test_vm.sh`
+Creates a test VM with vGPU passthrough using QEMU.
+
+### `vm/launch_guest_driver.sh`
+Launches guest with NVIDIA driver installed for validation.
+
+## Firmware Tools
+
+### `firmware/verify_gsp_firmware.sh`
+Verifies GSP firmware files exist and are correct version.
+
+### `firmware/install_tu104_firmware.sh`
+Installs TU104 GSP firmware to /lib/firmware/nvidia/tu104/gsp/
+
+## Monitoring
+
+### `monitoring/watch_dmesg.sh`
+Real-time kernel log monitoring with filtering for vGPU/NVIDIA messages.
+
+### `monitoring/watch_module_params.sh`
+Monitors module parameters and sysfs attributes.
+
+## Environment Variables
+
+```bash
+export OPEN_IOV_ROOT=$OPEN_IOV_ROOT
+export NVIDIA_OPEN_KERNEL=$OPEN_IOV_ROOT/nvidia-open-gpu-kernel-modules/kernel-open
+export VGPU_KERNEL=$OPEN_IOV_ROOT/kernel
+export VGPU_USERSPACE=$OPEN_IOV_ROOT/userspace
+```
+
+## Module Parameters for Debugging
+
+| Parameter | Description | Values |
+|-----------|-------------|--------|
+| `NVreg_RmMsg` | RM debug logging | `":"` (verbose), `"0"` (none) |
+| `NVreg_RegisterPCIDriver` | PCI driver registration | `0` (disable), `1` (enable) |
+| `NVreg_GpuInitOnProbe` | GPU init at probe | `0` (defer), `1` (immediate) |
+| `NVreg_PreserveVideoMemoryAllocations` | Video memory preservation | `0` (disable), `1` (enable), `2` (auto) |
+| `NVreg_UseKernelSuspendNotifiers` | Suspend notifiers | `0` (disable), `1` (enable) |
+| `dyndbg` | Dynamic debug | `'+p'` (all), `'+pf'` (probe+fail) |
+
+## Converged Mode Architecture
+
+```
+Host nvidia.ko (patched)          nvidia-vgpu-vfio.ko (open-iov)
+├── mdev_register_parent()  ←──  mdev_register_driver() only
+├── EXPORT nvidia_vgpu_vfio_set_ops
+├── EXPORT nvidia_vgpu_vfio_get_ops
+└── Implements RM vGPU ops       Calls RM via get_ops, provides VFIO ops via set_ops
+```
+
+## Troubleshooting
+
+### Module stuck in "going" state
+```bash
+cat /sys/module/nvidia-vgpu-vfio/initstate
+cat /sys/module/nvidia-vgpu-vfio/refcnt
+# Requires reboot to clear
+```
+
+### mdev parent not registered
+```bash
+ls /sys/bus/pci/devices/0000:01:00.0/mdev_supported_types/
+# Should show nvidia-vgpu-* directories with create/available_instances
+```
+
+### Symbol version mismatch
+```bash
+# Rebuild open-gpu-kernel-modules with VGX_KVM_BUILD=1
+cd $NVIDIA_OPEN_KERNEL && make clean && make -j$(nproc) VGX_KVM_BUILD=1
+```
+
+### IOMMU/VT-d not enabled
+```bash
+cat /proc/cmdline | grep -E "iommu|intel_iommu"
+# Add: intel_iommu=on iommu=pt to kernel cmdline
+```
\ No newline at end of file