diff options
Diffstat (limited to 'userspace/cuda/docs/08-virtualization-approaches.md')
| -rw-r--r-- | userspace/cuda/docs/08-virtualization-approaches.md | 67 |
1 files changed, 67 insertions, 0 deletions
diff --git a/userspace/cuda/docs/08-virtualization-approaches.md b/userspace/cuda/docs/08-virtualization-approaches.md new file mode 100644 index 0000000..3523ad1 --- /dev/null +++ b/userspace/cuda/docs/08-virtualization-approaches.md @@ -0,0 +1,67 @@ +# 08 — Virtualization approaches for CUDA: options, trade-offs, and where we are + +One-page decision summary for how a guest's CUDA work gets executed in +Open-IOV. Companion to `docs/04-virtualization-and-emulation.md` (design +principles) and to the project implementation docs (`docs/refactored-architecture.md`, +`docs/04-implementation/02-practical-challenges-and-adversarial-interop.md`). + +## 1. The options + +| # | Approach | What runs where | Benefits | Drawbacks / what we must own | +|---|---|---|---|---| +| **0** | **Full software emulation** — execute every command in the host emulator (memsets, copies, semaphores, and in principle grids) | Everything in the host module | No physical GPU needed at all; perfectly isolated; simplest correctness story; portable | Orders of magnitude too slow for compute; only viable for state/memory operations; never the whole answer | +| **A** | **Host RM/GSP proxy** — guest push → translate → host driver's RM API → real channel → GPU | Control and work both through the host driver | Reuses a mature driver's knowledge (page tables, scheduling, faults); fastest path to *correct*; no direct hardware ownership | Extra hop (latency/throughput cost); depends on host driver interfaces staying usable; sharing the host GPU with the host driver needs a policy | +| **B** | **Direct engine submission** — guest push → translate → our own channel/push construction → GPU | Control emulated, work submitted by us | Lowest latency, no host-driver dependency, full control of scheduling/isolation | We must own page tables, channel/runlist setup, cache invalidation, completion, fault handling — the largest engineering burden | +| **C** | **Hybrid** — management via A, hot path via B | Split by operation class | Pragmatic: correctness now, performance where it costs | Two data paths to keep consistent; the boundary must be an interface from day one | +| **D** | **Hardware partitioning (SR-IOV/VF, MIG)** | Guest on its own partition/VF | Strong isolation, near-native performance | Not available on consumer GPUs; still needs a device model for the VF; complementary, not a replacement for A/B/C | +| **E** | **Guest-assisted (patch driver / shim / API interception)** | Guest modified | Easy performance wins in the short term | Breaks the "stock guest" rule, the anti-fragility rules and the legal/clean-room stance — **rejected** | + +**Fast paths are orthogonal to the choice:** pure state operations (semaphores, +fences, some memops) should be satisfied directly in the emulation regardless +of A/B/C — they are microseconds in hardware and would lose by round-tripping. + +## 2. Design implications + +* Where the boundary sits decides what we must own: A keeps ownership in the + host driver, B moves it into Open-IOV. Either way the **interface** the + guest sees is identical — that is what makes the choice reversible. +* Isolation and correctness do not depend on the path, but **the failure + modes do**: A fails as host-driver errors and sharing conflicts, B fails as + hardware faults and page-table/scheduling bugs we introduced. +* Performance targets (launch overhead, copy bandwidth, completion latency) + can only be met by forwarding *real work*; software execution is for + state/memory operations and as a correctness bridge, not the data plane. +* The work path must therefore be abstracted behind one interface now, so A + can be swapped for B (or C) without touching the control plane. + +## 3. What Open-IOV is pursuing + +**Stated architecture:** *emulate the interface, pass through the work, never +patch anyone* — a mediated-passthrough vGPU where PCI config/BARs, the GSP boot +handshake, the GSP-RM protocol and interrupt routing are emulated in +`open-iov-vfio.ko`, while real work is meant to execute on the physical GPU. +The project's own decision procedure is explicit: **prototype A first for +correctness, measure, then move the hot path to B if the budgets demand it** +(option C in practice), with the work path abstracted from day one. + +**Current reality (2026-09, honest):** the *control plane* follows the +architecture — GSP-RM emulation, RPC handling, BAR/PCI/MSI-X emulation and +object/control replies are implemented, and the guest's stock driver + +`libcuda` now get through `cuInit`, `cuDeviceGetCount`/`cuDeviceGet` and into +`cuCtxCreate`. The old `cuCtxCreate` segfault is solved (the alloc reply must +echo the request's params — `docs/05` §3.5); context creation is blocked one +step later by engine topology: the class-filtered engine list is missing the +async copy engine, so the walker reads a zeroed secondary record and the call +returns `rc=999` (`docs/05` §3.6). The *data plane* is at the bridge stage: the +host module decodes the guest's push buffers as method streams and **executes +the small work it understands itself** (memsets, copies, semaphore releases) +instead of forwarding it to the physical GPU. No CUDA kernel has been +forwarded yet. So: yes, we are pursuing the intended architecture, but we are +in the "emulate + software-execute state/memory ops" phase of it, with the +A→B/C forwarding decision still ahead. + +**Risks to guard now:** (1) software execution silently becoming the data +plane instead of a bridge; (2) the work-path interface being skipped because +the direct path "works for now"; (3) tuning the emulation to one driver +version instead of the interface. The anti-fragility, performance and security +rules in `docs/04` exist to prevent exactly this drift. |