# System Overview

> **What this page is.** The one page to read if you want to understand *how
> Open-IOV works* before diving into any subsystem. Condensed from
> [`../docs/03-architecture/01-system-overview.md`](../docs/03-architecture/01-system-overview.md)
> (with [`../docs/03-architecture/09-full-system-picture.md`](../docs/03-architecture/09-full-system-picture.md)
> and [`../docs/03-architecture/10-interaction-and-scheduling.md`](../docs/03-architecture/10-interaction-and-scheduling.md)
> for the full picture); the docs tree wins on disagreement.

Status: 2026-10-12, tree `master @ 2f066727`. Tags: `[live]` proven with the
stock guest driver; `[gated]` implemented default-off; `[wip]` partial;
`[wall]` current blocker; `[target]` designed end state.

## 1. The core principle

**Emulate the interface. Pass through the work. Never patch anyone.**

A virtual GPU is a *mediated passthrough* device. The guest must believe it
owns a real NVIDIA GPU with a real GPU System Processor running firmware; the
host must arrange for the guest's work to execute on real hardware, because
emulating compute in software is orders of magnitude too slow to be useful.

```text
EMULATE  (software; the guest expects virtual hardware)
  PCI configuration space, capability structures, power management
  BAR0 register interface (shadow, not the physical GPU's BAR0)
  BAR1 VRAM aperture (guest-visible video memory)
  GSP firmware boot handshake (FWSEC -> booter -> RISCV -> boot complete)
  GSP-RM message queues, RPC transport, resource-manager controls
  event/notification channels; MSI-X injection (via eventfd)

PASS THROUGH  (real hardware; correctness and performance)
  GPU memory: guest VRAM objects backed by real VRAM       [target]
  page tables / GMMU state used by the GPU                 [wip]
  channel execution: guest push buffers on real channels   [gated/wall]
  compute (CUDA) and, later, 3D/video work                 [wip/planned]
  DMA, completion and fault interrupts relayed to the guest
```

The line between the two is the **interface boundary**, and it moves as the
project matures: early work emulates everything the guest can observe; later
work forwards everything that must be fast.

## 2. Layer map

```text
GUEST
  application ── CUDA runtime (libcudart) ─┐
  application ── CUDA driver  (libcuda)   ─┤  userspace: CUDASW stack
  OpenGL/Vulkan ── libGLX/libvulkan ──────┘
        │  ioctls / mmaps
  nvidia.ko (unmodified)  ── RM core, resource server, UVM, modeset/drm
        │  MMIO + DMA + MSI-X   (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════
HYPERVISOR (KVM/QEMU, later Xen/...)  VFIO mdev transport + IOMMU
        │
HOST MODULE  vgsp.ko
  ├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
  ├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
  ├─ memory model: guest RAM access, VA<->PA maps, VRAM backing
  └─ work path: channel/push forwarding to the physical GPU
        │ module-owned device + real GSP session
HOST BACKEND  the GPU's own hardware + GSP firmware, driven by this module
        │     (the distribution nvidia.ko stays unmodified and unbound in
        │      this mode)
        v
REAL HARDWARE (engines, memory, interrupts)
```

The guest never contains Open-IOV code; the host driver is unmodified; the
hypervisor is unmodified. The only new component is `vgsp.ko`.

## 3. How each side works

* **Guest side.** The stock driver performs the GSP boot handshake, sets up
  RPC message queues in guest memory, creates RM objects, programs channels
  and submits work — exactly as on bare metal
  ([`Guest-Stack-and-CUDASW.md`](Guest-Stack-and-CUDASW.md),
  [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md),
  [`UVM-in-the-Guest.md`](UVM-in-the-Guest.md)).
* **Host module.** `vgsp.ko` is a VFIO mediated-device driver: it registers a
  parent on the physical GPU, exposes virtual GPU types, and each mdev is one
  virtual GPU for one VM. Inside, it presents a plausible Turing-class device
  (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the
  RPC protocol with well-formed replies, and maintains the memory model that
  resolves guest physical addresses to host addresses
  ([`Host-Module.md`](Host-Module.md)).
* **The interface that makes this possible.** Since Turing, the driver's
  management path is not *silicon*, it is *firmware*: the GSP runs a version of
  NVIDIA's Resource Manager and the driver talks to it over shared-memory
  queues with a small RPC protocol. That protocol is small, stable and
  observable — so the project emulates **the firmware's interface** instead of
  emulating a GPU ([`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md),
  [`../docs/03-architecture/03-gsp-and-gsp-rm.md`](../docs/03-architecture/03-gsp-and-gsp-rm.md)).
* **Execution.** Guest work that must be fast (copies, kernels) is forwarded
  to a real pipeline driven through the module's own GSP-RM session
  ([`Execution-Plane.md`](Execution-Plane.md),
  [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md),
  [`Execution-Frontier.md`](Execution-Frontier.md)).

## 4. Data flows

```text
MMIO from the guest:
  guest nvidia.ko writes BAR0 offset X
    -> KVM exit -> QEMU -> VFIO region write into the mdev
    -> BAR0 shadow state machine (doorbells, VF registers, notify)
    -> may wake the emulation thread

RPC from the guest to "firmware":
  guest writes a command-queue element into a guest-memory msgq
    -> doorbell (BAR0)
    -> emulation thread reads the queue through the hypervisor's guest-RAM
       accessor, parses the element, dispatches the RPC
    -> handler produces a reply element in the status queue
    -> emulation injects an MSI-X interrupt to the guest
```

## 5. What runs today

1. Guest boots, enumerates the virtual PCIe device, binds `nvidia.ko`.
2. The GSP boot handshake completes against the emulation.
3. Message queues are set up, versions negotiated, RM initialization
   exchanges complete; object creation, capability queries, vaspaces,
   channels and the engine inventory are answered with well-formed replies
   (a small set of control IDs is special-cased with real values; reply
   fidelity for consumed fields keeps improving).
4. `nvidia-smi -L` enumerates the device; CUDA userspace starts; context
   creation completes; the full driver-API smoke passes (`ALL OK`, 64-byte
   payload round-trips).
5. CE-class copies are forwarded to the physical copy engine (default off)
   with engine-authored semaphore releases and a fault relay. For GR
   (graphics/compute), a forwarded guest launch reaches the front-end but the
   engine has not executed it yet — the current frontier
   ([`Execution-Frontier.md`](Execution-Frontier.md)).

## 6. Where the detail lives

* [`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md) — the firmware/RM architecture and
  the transport we emulate.
* [`Host-Module.md`](Host-Module.md) — the module's internals.
* [`Guest-Stack-and-CUDASW.md`](Guest-Stack-and-CUDASW.md),
  [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — the guest side.
* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md),
  [`UVM-in-the-Guest.md`](UVM-in-the-Guest.md) — memory and translation.
* [`Execution-Plane.md`](Execution-Plane.md),
  [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md),
  [`Execution-Frontier.md`](Execution-Frontier.md) — work and execution.
* [`Hardware-Software-Matrix.md`](Hardware-Software-Matrix.md) — supported
  combinations and portability.
* [`../docs/03-architecture/01-system-overview.md`](../docs/03-architecture/01-system-overview.md)
  and the rest of the architecture corpus in the repository.