Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 33 insertions, 0 deletions

diff --git a/tools/guest-diag/kret.sh b/tools/guest-diag/kret.sh
new file mode 100755
index 0000000..9ed1220
--- /dev/null
+++ b/tools/guest-diag/kret.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+# kret.sh - kretprobe the nvGpuOps* call chain in the guest's nvidia.ko and
+# capture return values while /tmp/cdt runs. Run with sudo.
+set -u
+
+TR=/sys/kernel/tracing
+SYMS="nvGpuOpsGetGpuInfo nvGpuOpsDeviceCreate nvGpuOpsRmDeviceCreate \
+nvGpuOpsRmSubDeviceCreate queryFbInfo findDeviceClasses queryVirtMode \
+getSysmemLinkInfo getSystemMemoryWindow getNvswitchInfo getEgmInfo getAtsInfo \
+nvGpuOpsQueryGpuConfidentialComputeCaps allocNvlinkStatus"
+
+echo > $TR/kprobe_events
+for f in $SYMS; do
+    if ! grep -q " [tT] $f\b" /proc/kallsyms; then
+        echo "skip $f (not in kallsyms)"
+        continue
+    fi
+    if ! echo "r:ret_$f $f \$retval" >> $TR/kprobe_events 2>/dev/null; then
+        echo "FAILED to register $f"
+    fi
+done
+echo "--- registered:"
+grep ret_ $TR/kprobe_events 2>/dev/null | head -20
+
+if [ -d $TR/events/kprobes ]; then
+    echo 1 > $TR/events/kprobes/enable
+    dmesg -C
+    /tmp/cdt >/dev/null 2>&1
+    echo "--- results (retval per probe):"
+    grep -E "ret_" $TR/trace | awk '{n=$4; sub(/^ret_/,"",n); printf "%s %s\n", n, $NF}' | sort | uniq -c | sort -rn | head -30
+    echo 0 > $TR/events/kprobes/enable
+    echo > $TR/trace
+fi