# 08 — Virtualization approaches for CUDA: options, trade-offs, and where we are

One-page decision summary for how a guest's CUDA work gets executed in
Open-IOV. Companion to `docs/04-virtualization-and-emulation.md` (design
principles) and to the project implementation docs (`docs/03-architecture/06-host-module-and-hybrid-model.md`,
`docs/04-implementation/02-practical-challenges-and-adversarial-interop.md`).

## 1. The options

| # | Approach | What runs where | Benefits | Drawbacks / what we must own |
|---|---|---|---|---|
| **0** | **Full software emulation** — execute every command in the host emulator (memsets, copies, semaphores, and in principle grids) | Everything in the host module | No physical GPU needed at all; perfectly isolated; simplest correctness story; portable | Orders of magnitude too slow for compute; only viable for state/memory operations; never the whole answer |
| **A** | **Host RM/GSP proxy** — guest push → translate → host driver's RM API → real channel → GPU | Control and work both through the host driver | Reuses a mature driver's knowledge (page tables, scheduling, faults); fastest path to *correct*; no direct hardware ownership | Extra hop (latency/throughput cost); depends on host driver interfaces staying usable; sharing the host GPU with the host driver needs a policy |
| **B** | **Direct engine submission** — guest push → translate → our own channel/push construction → GPU | Control emulated, work submitted by us | Lowest latency, no host-driver dependency, full control of scheduling/isolation | We must own page tables, channel/runlist setup, cache invalidation, completion, fault handling — the largest engineering burden |
| **C** | **Hybrid** — management via A, hot path via B | Split by operation class | Pragmatic: correctness now, performance where it costs | Two data paths to keep consistent; the boundary must be an interface from day one |
| **D** | **Hardware partitioning (SR-IOV/VF, MIG)** | Guest on its own partition/VF | Strong isolation, near-native performance | Not available on consumer GPUs; still needs a device model for the VF; complementary, not a replacement for A/B/C |
| **E** | **Guest-assisted (patch driver / shim / API interception)** | Guest modified | Easy performance wins in the short term | Breaks the "stock guest" rule, the anti-fragility rules and the legal/clean-room stance — **rejected** |

**Fast paths are orthogonal to the choice:** pure state operations (semaphores,
fences, some memops) should be satisfied directly in the emulation regardless
of A/B/C — they are microseconds in hardware and would lose by round-tripping.

**Availability with an unmodified distribution host driver:** approach A has no
callable entry point from kernel context (see
[`../../../docs/03-architecture/08-physical-gpu-integration.md`](../../../docs/03-architecture/08-physical-gpu-integration.md));
approach B requires owning scheduling; the project's chosen mechanism is a
module-owned device that keeps the GPU's real GSP firmware as the scheduling
server. Approach E remains deployment-gated.

## 2. Design implications

* Where the boundary sits decides what we must own: A keeps ownership in the
  host driver, B moves it into Open-IOV. Either way the **interface** the
  guest sees is identical — that is what makes the choice reversible.
* Isolation and correctness do not depend on the path, but **the failure
  modes do**: A fails as host-driver errors and sharing conflicts, B fails as
  hardware faults and page-table/scheduling bugs we introduced.
* Performance targets (launch overhead, copy bandwidth, completion latency)
  can only be met by forwarding *real work*; software execution is for
  state/memory operations and as a correctness bridge, not the data plane.
* The work path must therefore be abstracted behind one interface now, so A
  can be swapped for B (or C) without touching the control plane.

## 3. What Open-IOV is pursuing

**Stated architecture:** *emulate the interface, pass through the work, never
patch anyone* — a mediated-passthrough vGPU where PCI config/BARs, the GSP boot
handshake, the GSP-RM protocol and interrupt routing are emulated in
`open-iov-vfio.ko`, while real work is meant to execute on the physical GPU.
The original decision procedure was to prototype A first for correctness; the
interface analysis
([`../../../docs/03-architecture/08-physical-gpu-integration.md`](../../../docs/03-architecture/08-physical-gpu-integration.md))
showed that proxying through an unmodified distribution host driver has no
callable entry point from kernel context, so the chosen mechanism is a
**module-owned GPU running its real GSP firmware as the resource-manager
server** (the module becomes the host-side RM client). The work path stays
abstracted from day one so the backend can be swapped.

**Current reality (2026-09-29, honest):** the *control plane* follows the
architecture — GSP-RM emulation, RPC handling, BAR/PCI/MSI-X emulation and
object/control replies are implemented, and the guest's stock driver +
`libcuda` now get through `cuInit`, `cuDeviceGetCount`/`cuDeviceGet` and into
`cuCtxCreate`. The old `cuCtxCreate` segfault is solved (the alloc reply must
echo the request's params — `docs/05` §3.5), and the engine inventory is now
complete (the async copy engine is included; the copy-engine capabilities
match the device). Context creation currently ends in a forced `rc=999` on a
completion wait while the completion/translation plumbing is being made
reliable. The *data plane* is at the bridge stage: the host module decodes the
guest's push buffers as method streams and executes the small work it
understands itself (memsets, copies, semaphore releases) instead of forwarding
it to the physical GPU. No CUDA kernel has been forwarded yet; the execution
backend is chosen but not started. So: we are still in the
"emulate + software-execute state/memory ops" phase, with the backend decision
now made (option D in the integration document).

**Risks to guard now:** (1) software execution silently becoming the data
plane instead of a bridge; (2) the work-path interface being skipped because
the direct path "works for now"; (3) tuning the emulation to one driver
version instead of the interface. The anti-fragility, performance and security
rules in `docs/04` exist to prevent exactly this drift.