1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
|
# 08 — Virtualization approaches for CUDA: options, trade-offs, and where we are
One-page decision summary for how a guest's CUDA work gets executed in
Open-IOV. Companion to `docs/04-virtualization-and-emulation.md` (design
principles) and to the project implementation docs (`docs/03-architecture/06-host-module-and-hybrid-model.md`,
`docs/04-implementation/02-practical-challenges-and-adversarial-interop.md`).
## 1. The options
| # | Approach | What runs where | Benefits | Drawbacks / what we must own |
|---|---|---|---|---|
| **0** | **Full software emulation** — execute every command in the host emulator (memsets, copies, semaphores, and in principle grids) | Everything in the host module | No physical GPU needed at all; perfectly isolated; simplest correctness story; portable | Orders of magnitude too slow for compute; only viable for state/memory operations; never the whole answer |
| **A** | **Host RM/GSP proxy** — guest push → translate → host driver's RM API → real channel → GPU | Control and work both through the host driver | Reuses a mature driver's knowledge (page tables, scheduling, faults); fastest path to *correct*; no direct hardware ownership | Extra hop (latency/throughput cost); depends on host driver interfaces staying usable; sharing the host GPU with the host driver needs a policy |
| **B** | **Direct engine submission** — guest push → translate → our own channel/push construction → GPU | Control emulated, work submitted by us | Lowest latency, no host-driver dependency, full control of scheduling/isolation | We must own page tables, channel/runlist setup, cache invalidation, completion, fault handling — the largest engineering burden |
| **C** | **Hybrid** — management via A, hot path via B | Split by operation class | Pragmatic: correctness now, performance where it costs | Two data paths to keep consistent; the boundary must be an interface from day one |
| **D** | **Hardware partitioning (SR-IOV/VF, MIG)** | Guest on its own partition/VF | Strong isolation, near-native performance | Not available on consumer GPUs; still needs a device model for the VF; complementary, not a replacement for A/B/C |
| **E** | **Guest-assisted (patch driver / shim / API interception)** | Guest modified | Easy performance wins in the short term | Breaks the "stock guest" rule, the anti-fragility rules and the legal/clean-room stance — **rejected** |
**Fast paths are orthogonal to the choice:** pure state operations (semaphores,
fences, some memops) should be satisfied directly in the emulation regardless
of A/B/C — they are microseconds in hardware and would lose by round-tripping.
## 2. Design implications
* Where the boundary sits decides what we must own: A keeps ownership in the
host driver, B moves it into Open-IOV. Either way the **interface** the
guest sees is identical — that is what makes the choice reversible.
* Isolation and correctness do not depend on the path, but **the failure
modes do**: A fails as host-driver errors and sharing conflicts, B fails as
hardware faults and page-table/scheduling bugs we introduced.
* Performance targets (launch overhead, copy bandwidth, completion latency)
can only be met by forwarding *real work*; software execution is for
state/memory operations and as a correctness bridge, not the data plane.
* The work path must therefore be abstracted behind one interface now, so A
can be swapped for B (or C) without touching the control plane.
## 3. What Open-IOV is pursuing
**Stated architecture:** *emulate the interface, pass through the work, never
patch anyone* — a mediated-passthrough vGPU where PCI config/BARs, the GSP boot
handshake, the GSP-RM protocol and interrupt routing are emulated in
`open-iov-vfio.ko`, while real work is meant to execute on the physical GPU.
The project's own decision procedure is explicit: **prototype A first for
correctness, measure, then move the hot path to B if the budgets demand it**
(option C in practice), with the work path abstracted from day one.
**Current reality (2026-09, honest):** the *control plane* follows the
architecture — GSP-RM emulation, RPC handling, BAR/PCI/MSI-X emulation and
object/control replies are implemented, and the guest's stock driver +
`libcuda` now get through `cuInit`, `cuDeviceGetCount`/`cuDeviceGet` and into
`cuCtxCreate`. The old `cuCtxCreate` segfault is solved (the alloc reply must
echo the request's params — `docs/05` §3.5); context creation is blocked one
step later by engine topology: the class-filtered engine list is missing the
async copy engine, so the walker reads a zeroed secondary record and the call
returns `rc=999` (`docs/05` §3.6). The *data plane* is at the bridge stage: the
host module decodes the guest's push buffers as method streams and **executes
the small work it understands itself** (memsets, copies, semaphore releases)
instead of forwarding it to the physical GPU. No CUDA kernel has been
forwarded yet. So: yes, we are pursuing the intended architecture, but we are
in the "emulate + software-execute state/memory ops" phase of it, with the
A→B/C forwarding decision still ahead.
**Risks to guard now:** (1) software execution silently becoming the data
plane instead of a bridge; (2) the work-path interface being skipped because
the direct path "works for now"; (3) tuning the emulation to one driver
version instead of the interface. The anti-fragility, performance and security
rules in `docs/04` exist to prevent exactly this drift.
|
