docs: record host VF acceleration as a deferred future goal

1 files changed, 140 insertions, 0 deletions

diff --git a/docs/03-architecture/12-host-vf-acceleration-future-goal.md b/docs/03-architecture/12-host-vf-acceleration-future-goal.md
new file mode 100644
index 0000000..298b329
--- /dev/null
+++ b/docs/03-architecture/12-host-vf-acceleration-future-goal.md
@@ -0,0 +1,140 @@
+# Host VF Acceleration: A Deferred Future Goal
+
+> **Status: deferred design goal (2026-10-02). Research and design only — nothing
+> from this document is planned for the current roadmap, and no implementation
+> is in the tree.** It records the goal, the feasibility verdict on the current
+> platform, the options evaluated, and the conditions under which the work would
+> be worth starting. It ranks below the active critical path: guest CUDA
+> execution
+> ([`11-physical-backend-guest-wiring.md`](11-physical-backend-guest-wiring.md)),
+> multi-VM, 3D and performance. The verdict is grounded in a read-only study on
+> the development host (TU104 mobile, stock distribution driver, QEMU/KVM).
+
+## 1. The goal
+
+Give a process **running on the host OS** GPU acceleration through the open-iov
+device model: bind the host's stock NVIDIA kernel driver to a virtual function
+or virtual device presented by `open-iov-vfio.ko`, so that host userspace
+(`libcuda`, `nvidia-smi`, CUDA applications) consumes the mediated device the
+way a guest does, with no VM in between.
+
+This is deliberately different from what the project does today:
+
+* **Current path (sound, active):** a local VM or microVM is attached to the
+  mdev; the guest runs the stock driver and stock CUDA, and guest submissions
+  reach real hardware once the physical backend is wired in
+  ([`11-physical-backend-guest-wiring.md`](11-physical-backend-guest-wiring.md)).
+  The host orchestrates but does not own the CUDA context.
+* **This goal:** the CUDA context lives in the host kernel, with no VM
+  boundary, no guest-memory pinning and no virtualized interrupt path.
+
+## 2. Feasibility verdict on the current platform: not possible as-is
+
+Four independent blockers were verified. Each alone is sufficient.
+
+1. **Driver-model bus mismatch — decisive and structural.** A mediated device
+   is a device on the `mdev` bus. The stock NVIDIA driver registers only a PCI
+   driver, and the driver core offers a device only to drivers on that device's
+   own bus. An mdev has no PCI identity; the mdev bus has no auto-match at all
+   and binds only the parent's declared mdev driver at device-creation time.
+   Emulating PCI config space and BARs inside the mdev does not change the bus
+   the device lives on, so "bind the host NVIDIA driver to an mdev" is a
+   category error in the Linux driver model, not a missing feature.
+2. **No SR-IOV silicon on this part.** The development GPU exposes no SR-IOV
+   extended capability: no PF SR-IOV sysfs attributes and no VF BARs. A module
+   cannot add a capability the silicon does not have, and the physical
+   function's config space belongs to the real hardware and to whichever driver
+   owns the PF. On parts that do expose VFs, the stock open driver still does
+   not run its resource manager on a VF: a normal build refuses the VF probe,
+   and a vGPU-aware build returns early after an IOMMU check and delegates to a
+   proprietary mediated-device driver. VFs are a VM-passthrough mechanism, not a
+   host-compute mechanism.
+3. **A synthetic PCI shim is rejected.** Kernel PCI host-bridge APIs would in
+   principle let an out-of-tree module fabricate a root bus with custom config
+   space, but Linux has no generic virtual-PCI-function framework and no
+   supported way to trap the kernel's own MMIO accesses to a fabricated BAR.
+   The stock driver's probe also expects real BARs, DMA, interrupts and a full
+   GSP firmware boot. A shim would have to reimplement a hardware-equivalent GPU
+   in kernel context with no hypervisor trap layer and no isolation boundary:
+   any bug panics the host. It is strictly harder and less safe than the
+   existing VFIO/QEMU design, which already has both. Effort XL+, probability of
+   a maintainable, safe result very low.
+4. **No host-side mdev support in the stock driver.** The stock driver
+   registers no mdev driver and no mdev parent operations; it has no code path
+   that runs RM against a mediated device. The vendor's own architecture uses
+   the physical function for host CUDA and the mediated device only between
+   hypervisor and guest. Verified rather than assumed.
+
+**Today's sound stock-CUDA route therefore remains a local VM/microVM on the
+mdev.** Everything a host-resident CUDA process would need from the virtual
+device — a bindable host driver instance, host-memory DMA reachability, host
+MSI-X delivery — does not exist and cannot be created without one of the
+blockers above being overturned.
+
+## 3. Options evaluated
+
+| Option | Stock CUDA on host? | Effort | Probability | Verdict |
+|---|---|---|---|---|
+| Local VM / microVM on the mdev | In guest (stock) | S–M | High — control plane already works | The sound path today; current critical path |
+| Module-owned host API (own RM client exposed to host userspace) | No — Open-IOV API only | L–XL | Medium | Viable as an Open-IOV interface, not `libcuda`; requires owning the GPU, so host PF acceleration is lost on a single-GPU host |
+| SR-IOV VF model | No | XL | ~0 here; hardware/vendor-gated elsewhere | Blocked by silicon and vendor semantics |
+| Synthetic PCI shim + host NVIDIA driver | Hypothetically | XL+ | Very low | Rejected (structurally unsound) |
+| Status quo: host NVIDIA driver on the PF | Yes | None | Works | Not a mediated-device path; the baseline this goal would replace |
+
+## 4. Why it is deferred
+
+* The current critical path already delivers the user-visible capability — CUDA
+  running on the physical GPU through open-iov — via the guest path; this goal
+  adds a second, much harder path to the same capability.
+* No sound implementation exists. Every non-VM option is either blocked by
+  silicon/vendor semantics or requires months of low-probability work with a
+  host-panic failure mode.
+* On a single-GPU host, the project's real-execution model already owns the GPU
+  (unbinding the distribution driver), so host PF acceleration is unavailable
+  for the duration anyway; meaningful host acceleration from a mediated device
+  needs a multi-GPU or host-proxy design that does not exist yet.
+* Priority order remains: guest CUDA execution, multi-VM, 3D, performance, then
+  host-resident acceleration.
+
+## 5. Conditions that would make it worth starting
+
+* **SR-IOV-capable hardware** where the PF stack can create VFs and there is a
+  sanctioned way for the stock driver to bind a VF for host compute — the only
+  option that keeps the NVIDIA driver inside its supported driver model.
+* **Driver-model work**: a kernel-supported virtual-device framework, or a
+  sanctioned host-side mediated-device consumer in the stock driver. Without one
+  of these, no safe implementation exists regardless of effort.
+* **A hard deployment requirement** that a local VM/microVM cannot satisfy: a
+  host-resident workload that cannot tolerate VM memory or I/O overhead, or a
+  constraint that mandates `libcuda` on the host rather than an Open-IOV API.
+* **A multi-GPU or host-proxy execution design** so host acceleration and guest
+  execution can coexist on one machine.
+
+## 6. First steps if it is picked up
+
+1. **Re-check preconditions** (hardware and software) and write them down:
+   SR-IOV capability, PF provider support for VFs, stock driver VF handling,
+   host kernel device-model support. A negative answer anywhere ends the
+   project before code is written.
+2. **Choose the device model** — VF-backed (preferred where possible) or a
+   host-bindable virtual device — and write a driver-model conformance note
+   before writing any code.
+3. **Prototype only the identity and bind path**: present or configure the
+   device and confirm the stock driver probes it; stop before GSP boot, DMA and
+   interrupts, and record exactly what the probe requires.
+4. **Design the memory and interrupt model next**: how host-process memory
+   becomes DMA-reachable, how MSI-X is delivered, and how faults surface, with
+   an isolation story for a host that has no hypervisor trap layer.
+5. **Wire the RM path last**: serve the GSP-RM protocol for a host client,
+   starting from the module's live host-side GSP-RM session
+   ([`08-physical-gpu-integration.md`](08-physical-gpu-integration.md)), with
+   state independent of the guest-facing emulation.
+6. **Gate and supervise**: an explicit module parameter, a supervised hardware
+   harness, and a deployment model that states the single-GPU trade-off (host
+   acceleration is lost while the module owns the card).
+
+## See also
+
+* [`08-physical-gpu-integration.md`](08-physical-gpu-integration.md) — the chosen execution backend (module-owned GPU + real GSP-RM session)
+* [`11-physical-backend-guest-wiring.md`](11-physical-backend-guest-wiring.md) — the active critical path this goal ranks below
+* [`../04-implementation/02-practical-challenges-and-adversarial-interop.md`](../04-implementation/02-practical-challenges-and-adversarial-interop.md) — forwarding options and interoperability constraints