# System Overview

> **What this page is.** The one page to read if you want to understand *how
> Open-IOV works* before diving into any subsystem. Condensed from
> [`../docs/03-architecture/01-system-overview.md`](../docs/03-architecture/01-system-overview.md)
> (with [`../docs/03-architecture/09-full-system-picture.md`](../docs/03-architecture/09-full-system-picture.md)
> and [`../docs/03-architecture/10-interaction-and-scheduling.md`](../docs/03-architecture/10-interaction-and-scheduling.md)
> for the full picture); the docs tree wins on disagreement.

Status: 2026-11-09, tree `master @ cb8e822b`. Tags: `[live]` proven with the
stock guest driver; `[gated]` implemented default-off; `[wip]` partial;
`[wall]` current blocker; `[target]` designed end state.

## 1. The core principle

**Emulate the interface. Pass through the work. Never patch anyone.**

A virtual GPU is a *mediated passthrough* device. The guest must believe it
owns a real NVIDIA GPU with a real GPU System Processor running firmware; the
host must arrange for the guest's work to execute on real hardware, because
emulating compute in software is orders of magnitude too slow to be useful.

```text
EMULATE  (software; the guest expects virtual hardware)
  PCI configuration space, capability structures, power management
  BAR0 register interface (shadow, not the physical GPU's BAR0)
  BAR1 VRAM aperture (guest-visible video memory)
  GSP firmware boot handshake (FWSEC -> booter -> RISCV -> boot complete)
  GSP-RM message queues, RPC transport, resource-manager controls
  event/notification channels; MSI-X injection (via eventfd)

PASS THROUGH  (real hardware; correctness and performance)
  GPU memory: guest VRAM objects backed by real VRAM       [target]
  page tables / GMMU state used by the GPU                 [wip]
  channel execution: guest push buffers on real channels   [wip]
  compute (CUDA) kernel launches on the real engine        [live]
  3D graphics method streams on the real engine            [wip]
  rendered 3D pixels / video work                          [wall/planned]
  DMA, completion and fault interrupts relayed to the guest
```

The line between the two is the **interface boundary**, and it moves as the
project matures: early work emulates everything the guest can observe; later
work forwards everything that must be fast.

## 2. Layer map

```text
GUEST
  application ── CUDA runtime (libcudart) ─┐
  application ── CUDA driver  (libcuda)   ─┤  userspace: CUDASW stack
  OpenGL/Vulkan ── libGLX/libvulkan ──────┘
        │  ioctls / mmaps
  nvidia.ko (unmodified)  ── RM core, resource server, UVM, modeset/drm
        │  MMIO + DMA + MSI-X   (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════
HYPERVISOR (KVM/QEMU, later Xen/...)  VFIO mdev transport + IOMMU
        │
HOST MODULE  vgsp.ko
  ├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
  ├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
  ├─ memory model: guest RAM access, VA<->PA maps, VRAM backing
  └─ work path: channel/push forwarding to the physical GPU
        │ module-owned device + real GSP session
HOST BACKEND  the GPU's own hardware + GSP firmware, driven by this module
        │     (the distribution nvidia.ko stays unmodified and unbound in
        │      this mode)
        v
REAL HARDWARE (engines, memory, interrupts)
```

The guest never contains Open-IOV code; the host driver is unmodified; the
hypervisor is unmodified. The only new component is `vgsp.ko`.

## 3. How each side works

* **Guest side.** The stock driver performs the GSP boot handshake, sets up
  RPC message queues in guest memory, creates RM objects, programs channels
  and submits work — exactly as on bare metal
  ([`Guest-Stack-and-CUDASW.md`](Guest-Stack-and-CUDASW.md),
  [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md),
  [`UVM-in-the-Guest.md`](UVM-in-the-Guest.md)).
* **Host module.** `vgsp.ko` is a VFIO mediated-device driver: it registers a
  parent on the physical GPU, exposes virtual GPU types, and each mdev is one
  virtual GPU for one VM. Inside, it presents a plausible Turing-class device
  (config space, BAR shadows, MSI-X), answers the GSP boot handshake and the
  RPC protocol with well-formed replies, and maintains the memory model that
  resolves guest physical addresses to host addresses
  ([`Host-Module.md`](Host-Module.md)).
* **The interface that makes this possible.** Since Turing, the driver's
  management path is not *silicon*, it is *firmware*: the GSP runs a version of
  NVIDIA's Resource Manager and the driver talks to it over shared-memory
  queues with a small RPC protocol. That protocol is small, stable and
  observable — so the project emulates **the firmware's interface** instead of
  emulating a GPU ([`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md),
  [`../docs/03-architecture/03-gsp-and-gsp-rm.md`](../docs/03-architecture/03-gsp-and-gsp-rm.md)).
* **Execution.** Guest work that must be fast (copies, kernels) is forwarded
  to a real pipeline driven through the module's own GSP-RM session
  ([`Execution-Plane.md`](Execution-Plane.md),
  [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md),
  [`Execution-Frontier.md`](Execution-Frontier.md)).

## 4. Data flows

```text
MMIO from the guest:
  guest nvidia.ko writes BAR0 offset X
    -> KVM exit -> QEMU -> VFIO region write into the mdev
    -> BAR0 shadow state machine (doorbells, VF registers, notify)
    -> may wake the emulation thread

RPC from the guest to "firmware":
  guest writes a command-queue element into a guest-memory msgq
    -> doorbell (BAR0)
    -> emulation thread reads the queue through the hypervisor's guest-RAM
       accessor, parses the element, dispatches the RPC
    -> handler produces a reply element in the status queue
    -> emulation injects an MSI-X interrupt to the guest
```

## 5. What runs today

1. Guest boots, enumerates the virtual PCIe device, binds `nvidia.ko`.
2. The GSP boot handshake completes against the emulation.
3. Message queues are set up, versions negotiated, RM initialization
   exchanges complete; object creation, capability queries, vaspaces,
   channels and the engine inventory are answered with well-formed replies
   (a small set of control IDs is special-cased with real values; reply
   fidelity for consumed fields keeps improving).
4. `nvidia-smi -L` enumerates the device; CUDA userspace starts; context
   creation completes; the full driver-API smoke passes (`ALL OK`, 64-byte
   payload round-trips).
5. CE-class copies are forwarded to the physical copy engine (default off)
   with engine-authored semaphore releases and a fault relay. For GR
   (graphics/compute), forwarded guest CUDA launches execute on the engine
   with engine-authored completions relayed to the guest; the direct GL
   client's GR and CE channels submit and retire, and graphics method streams
   execute in three shapes ([`Graphics-Execution.md`](Graphics-Execution.md));
   rendered 3D pixels are the graphics frontier
   ([`Execution-Frontier.md`](Execution-Frontier.md)).
6. On the graphics side, the emulated display engine serves the modeset push
   buffers and the display notifier/completion (class-aware arm), so
   nvidia-modeset/NvKms registers (`/dev/dri`), X settles and `glxinfo`
   reports OpenGL 4.6.0 with direct rendering; the direct GL client's
   channels submit, the X DDX waits are fixed, the guest's real GL draws
   forward and execute on the physical GPU, the 2D path is green, and the
   guest Vulkan pipeline runs end to end. The draw writes 0 pixels because its
   constant-buffer (MVP) page does not resolve, so the remaining gate is the
   constant-buffer content, MME/wedge stability and the DDX get-word/notifier
   completion before the first rendered (non-black) frame
   ([`OpenGL.md`](OpenGL.md), [`Vulkan.md`](Vulkan.md)).

## 6. Where the detail lives

* [`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md) — the firmware/RM architecture and
  the transport we emulate.
* [`Host-Module.md`](Host-Module.md) — the module's internals.
* [`Guest-Stack-and-CUDASW.md`](Guest-Stack-and-CUDASW.md),
  [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — the guest side.
* [`Introduction-to-GMMU.md`](Introduction-to-GMMU.md),
  [`UVM-in-the-Guest.md`](UVM-in-the-Guest.md) — memory and translation.
* [`Execution-Plane.md`](Execution-Plane.md),
  [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md),
  [`Execution-Frontier.md`](Execution-Frontier.md) — work and execution.
* [`Hardware-Software-Matrix.md`](Hardware-Software-Matrix.md) — supported
  combinations and portability.
* [`../docs/03-architecture/01-system-overview.md`](../docs/03-architecture/01-system-overview.md)
  and the rest of the architecture corpus in the repository.