diff options
| author | Open-IOV <dev@open-iov.org> | 2026-09-23 05:44:08 -0400 |
|---|---|---|
| committer | Open-IOV <dev@open-iov.org> | 2026-09-23 05:46:43 -0400 |
| commit | 68cdd7728a7cc6715bbbff7decfe2b11c3bdf558 (patch) | |
| tree | 3dc8ce2e6af313a97ec6852f5c663a9a93665d2e /docs/05-operations/01-performance-correctness-security.md | |
| download | open-iov-68cdd7728a7cc6715bbbff7decfe2b11c3bdf558.tar.gz open-iov-68cdd7728a7cc6715bbbff7decfe2b11c3bdf558.zip | |
Open-IOV: initial public release
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
Diffstat (limited to 'docs/05-operations/01-performance-correctness-security.md')
| -rw-r--r-- | docs/05-operations/01-performance-correctness-security.md | 140 |
1 files changed, 140 insertions, 0 deletions
diff --git a/docs/05-operations/01-performance-correctness-security.md b/docs/05-operations/01-performance-correctness-security.md new file mode 100644 index 0000000..c4d8153 --- /dev/null +++ b/docs/05-operations/01-performance-correctness-security.md @@ -0,0 +1,140 @@ +# Performance, Correctness, and Security Evaluation + +> How we measure whether Open-IOV is any good — and how auditors can verify the +> claims. Numbers without methodology are not accepted here. + +## 1. The evaluation ladder + +Every platform (hardware + driver + hypervisor + guest) is evaluated through +the same five-rung ladder. Each rung is a gate: you do not claim the next one +until the previous one is green and reproducible. + +``` +Rung 1 INITIALIZATION guest driver probes; nvidia-smi; device enumeration +Rung 2 RUNTIME START cuInit; context; allocation; UVM registration +Rung 3 EXECUTION kernels produce correct results; copies verified +Rung 4 SUSTAINED workloads run for hours; faults/errors recoverable +Rung 5 MULTI-TENANT N VMs, isolation + fairness + no cross-talk +``` + +## 2. Correctness + +### 2.1 Test corpus + +| Layer | Tests | +|---|---| +| Device | PCI config validation, BAR sizing, MSI-X semantics, power-management stubs | +| Driver init | guest `nvidia.ko` probe, `nvidia-smi` full query, `deviceQuery` | +| CUDA | `vectorAdd`, reduction, shared-memory kernels, memcpy patterns, streams/events, error injection | +| UVM | oversubscribed memory (fault path), unified memory migration, concurrency | +| Graphics (later) | GLXInfo/VulkanInfo, rendering correctness screenshots | +| Stability | start/stop cycles ×100, VM kill during work, guest reboot in flight | + +### 2.2 Verification methods + +1. **Golden results**: kernel outputs compared against native execution on the + same GPU, byte-for-byte, for deterministic kernels. +2. **Guest self-checks**: the guest's own validation paths are the strongest + signal (UVM asserts, RM validation, CUDA runtime self-tests). Every guest + diagnostic that fires is a test failure for us, even if the workload + continues. +3. **Differential runs**: same workload on (a) native, (b) our device — diff + observables, not just outputs: semaphore value sequences, PUT/GET progress, + interrupt counts. +4. **Negative testing**: inject malformed device behaviour in a lab profile and + require the guest driver to recover exactly as it would on hardware with a + transient error. + +### 2.3 Regression gates + +`make test` runs the host suite. The CI ladder runs the guest suite on a +scheduled cadence and on every release. A rung regression is a release blocker; +the handoff documents carry the current rung. + +## 3. Performance + +### 3.1 Metrics + +| Metric | Definition | Why it matters | +|---|---|---| +| T_init | VM boot → `cuInit` returns | user-visible startup | +| RPC latency p50/p99 | request write → reply visible in status queue | init and management overhead | +| Push overhead | guest submission → engine start (forwarded) | steady-state latency | +| Semaphore completion latency | release written → tracker observes | per-operation sync cost | +| Copy bandwidth | H2D/D2H/D2D, large and small | data-movement quality | +| Kernel throughput | kernels/second for a fixed workload | launch overhead | +| Multi-tenant fairness | per-tenant progress under contention | isolation quality | + +### 3.2 Method + +* **Same-guest baselines**: measure native GPU performance *in the same VM with + a passthrough device* where possible; otherwise on the same hardware with the + same driver. Comparisons across hosts are noise. +* **Warm and cold**: report both; report variance; report the number of runs. +* **Instrument everything**: the emulation timestamps requests and replies + (tracepoints, `trace_printk`, or counters) so overhead is attributable to a + component, not to a mystery. +* **Publish the harness**: the benchmark scripts live in `tools/`, and a + result is not comparable unless produced by them (or an equivalent, + documented harness). + +### 3.3 Targets + +* Initialization: within 2× of native, same guest. +* Synchronization-heavy operations: within an agreed constant factor, published + per operation class. +* Bulk copies and sustained compute: as close to native as the forwarding path + allows; the forwarding design (proxy vs direct) is decided by measurement. + +## 4. Security and isolation + +### 4.1 Threat model + +| Adversary | Capability | Boundary that must hold | +|---|---|---| +| Guest VM (tenant) | full control of its own driver, MMIO, DMA | cannot touch host or other tenants' memory/state | +| Malicious guest driver | arbitrary device programming | emulation must validate everything, fail closed | +| Host user (unprivileged) | can access `/dev/*` permissions given | no privilege escalation via our interfaces | +| Adjacent tenant | shares the GPU | cannot observe timing/state/interrupts beyond policy | +| Supply chain | modified driver/firmware | measured boot (TrenchBoot) verifies the platform | + +### 4.2 Enforcement + +* **Memory isolation is the platform IOMMU's job, and we never weaken it.** + The device performs DMA only through the hypervisor-provided DMA interface + for its own VM. No global mappings, no "identity for convenience". +* **Per-instance state**: every mdev has its own BAR shadows, queues, + interrupt state and emulation context. No shared mutable state between + tenants in the host module. +* **Fail closed**: if a required isolation property cannot be established + (IOMMU absent, DMA mapping failure, inconsistent state), the device refuses + to start and says why in `dmesg`. +* **Interrupts**: per-instance vectors; masked interrupts are queued in the + emulated PBA, not dropped into another tenant's vector. +* **Debug surfaces**: destructive debug parameters are opt-in, documented, and + excluded from hardened builds. Reads are side-effect-free. +* **No secrets in the module**: nothing about one tenant's workload is stored + where another tenant could reach it. + +### 4.3 Auditing + +* The security-relevant code paths are small and enumerated in this document + tree: device registration, DMA access, interrupt injection, memory + translation, and the debug surface. Auditors should be able to enumerate + them mechanically (`grep` for the gate points). +* Each release produces a checklist: threat model reviewed, fail-closed paths + exercised by tests, debug surface verified gated, dependency licenses + unchanged. +* We explicitly invite adversarial review — including of the emulation's + response to malformed guest input. + +## 5. Reporting + +Each release publishes: + +1. **Rung achieved** per platform, with logs. +2. **Performance table** (metrics above) with methodology and variance. +3. **Known issues**, linked to the known-unknowns document. +4. **Security checklist** status. + +This is the only accepted form of a claim in this project. |
