# System Overview

> The one document to read if you want to understand *how Open-IOV works* before
> diving into any subsystem.

## 1. The core architectural principle

> **Emulate the interface. Pass through the work.**

A virtual GPU is a *mediated passthrough* device. The guest must believe it
owns a real NVIDIA GPU with a real GPU System Processor (GSP) running NVIDIA's
firmware. The host must arrange for the guest's work to execute on real
hardware, because emulating compute in software is 100–1000× too slow to be
useful.

Everything therefore falls into two buckets:

```
┌─────────────────────────────────────────────────────────────────────────┐
│  EMULATE  (software; guest expects virtual hardware)                    │
│    • PCI configuration space, capability structures, power management   │
│    • BAR0 register interface (shadow, not the physical GPU's BAR0)      │
│    • BAR1 VRAM aperture (guest-visible video memory)                    │
│    • Instance/PRAMIN window                                             │
│    • GSP firmware boot handshake (FWSEC → booter → RISCV → boot complete)│
│    • GSP-RM message queues, RPC transport, resource-manager controls    │
│    • Event/notification channels                                        │
│    • MSI-X injection (via eventfd, from the hypervisor's perspective)   │
└─────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────┐
│  PASS THROUGH  (real hardware; for correctness and performance)         │
│    • GPU memory: guest VRAM objects backed by real VRAM (goal)          │
│    • Page tables / GMMU state used by the GPU (goal)                    │
│    • Channel execution: guest push buffers on real GPU channels (goal;  │
│      today the host software-executes decoded work — a bridge)          │
│    • Compute (CUDA) and, later, 3D/video work                           │
│    • DMA, completion and fault interrupts relayed back to the guest     │
└─────────────────────────────────────────────────────────────────────────┘
```

The line between the two is the **interface boundary**, and it moves as the
project matures: early work emulates more (everything the guest can observe),
later work forwards more (everything that must be fast).

## 2. Layer map

```
GUEST
  application ── CUDA runtime (libcudart) ─┐
  application ── CUDA driver  (libcuda)   ─┤  userspace: CUDASW stack
  OpenGL/Vulkan ── libGLX/libvulkan ──────┘
        │  ioctls / mmaps
  nvidia.ko (unmodified)  ── RM core, resource server, UVM, modeset/drm
        │  MMIO + DMA + MSI-X   (a virtual PCIe device)
════════╪════════════════════════════════════════════════════════════════
HYPERVISOR (KVM/Xen/...)  VFIO mdev transport + IOMMU isolation
        │
HOST MODULE  open-iov-vfio.ko
  ├─ virtual device: config space, BAR0/BAR1/BAR3 windows, MSI-X
  ├─ emulation core: GSP boot + GSP-RM (RPC msgq, controls, objects, events)
  ├─ memory model: guest RAM access, VA↔PA maps, VRAM backing
  └─ work path: channel/push forwarding to the physical GPU   [goal; software bridge today]
        │
HOST DRIVER  nvidia.ko (unmodified) and/or direct GPU interfaces
        │
PHYSICAL GPU
```

## 3. Host module architecture

`open-iov-vfio.ko` is a **VFIO mediated-device (mdev) driver**. It registers a
parent on the physical GPU and exposes virtual GPU types; each `mdev` instance
is one virtual GPU for one VM.

| Subsystem | Files (illustrative) | Responsibility |
|---|---|---|
| mdev/VFIO registration | `kernel/main.c` | parent/type registration, VFIO device, region and IRQ setup |
| Config space | `kernel/main.c`, `kernel/bar_init.c` | identify/command/status, BARs, PCIe caps, MSI-X caps, VF windows |
| BAR0 shadow | `kernel/bar_emulation.c` | 16 MB emulated register space; register reads/writes the guest driver depends on |
| VRAM aperture (BAR1) | `kernel/main.c`, `memory.c` | 256 MB guest-visible video memory (host RAM today; real VRAM later) |
| Instance memory (BAR3) | `kernel/bar_emulation.c` | instance/PRAMIN window behaviour |
| GSP emulation thread | `kernel/gsp_emulation.c` | boot stage completion, doorbell polling, MSI-X injection |
| GSP-RM emulation | `kernel/gsp_rm_emulation.c` | message queues, RPC dispatch, resource-manager controls, objects |
| RPC handlers (firmware-derived) | `kernel/gsp_rpc_client.c` | behaviours decompiled from the GSP firmware |
| Firmware analysis (runtime) | `kernel/firmware_analyzer.c` | CRC/SHA signatures, pattern matching, version identification |
| Version dispatch | `kernel/dynamic_rpc_dispatcher.c`, `kernel/version/` | version-aware handler selection |
| Hardware abstraction | `kernel/architecture/` | per-architecture constants and behaviours |
| Userspace library | `userspace/lib/libvgpu.c` | management API (`vgpu_*`), diagnostics |

**Data flow for an MMIO access from the guest:**

```
guest nvidia.ko writes BAR0 offset X
   → KVM exit → QEMU → VFIO region write into the mdev
   → nv_vgpu_vfio_dev_write()  (or directly into the BAR0 shadow if mmap'd)
   → BAR0 shadow state machine (doorbells, VF registers, notify)
   → may wake the GSP emulation thread
```

**Data flow for an RPC from the guest to "firmware":**

```
guest writes a command-queue element into a guest-memory msgq and rings the
GSP doorbell (BAR0)
   → emulation thread reads the queue through the hypervisor's guest-RAM
     accessor (vfio_dma_rw), parses the element, dispatches the RPC
   → handler produces a reply element in the status queue
   → emulation injects an MSI-X interrupt to the guest
```

## 4. Guest stack summary

* `nvidia.ko` is the **unmodified** NVIDIA kernel driver. It contains the
  resource manager (RM) core, the resource server (`resserv`), the memory
  manager, the FIFO/runlist code, the GPU System Processor client path, and
  UVM (the unified memory/virtual-memory module, `nvidia_uvm.ko`).
* User space consists of the CUDA driver (`libcuda.so`), the CUDA runtime
  (`libcudart.so`), plus graphics/video libraries. The family of NVIDIA
  *software* components that mediate between applications and the driver is
  referred to in this project as **CUDASW**.
* The guest expects to program a real GPU: it writes registers, allocates
  objects, creates channels, submits push buffers and waits on semaphores —
  exactly as on bare metal.

Detailed subsystem documents:

* [`02-guest-stack-and-cudasw.md`](02-guest-stack-and-cudasw.md)
* [`03-gsp-and-gsp-rm.md`](03-gsp-and-gsp-rm.md)
* [`04-uvm.md`](04-uvm.md)
* [`05-gmmu-amap-hal-gr.md`](05-gmmu-amap-hal-gr.md)
* [`06-host-module-and-hybrid-model.md`](06-host-module-and-hybrid-model.md)
* [`07-driver-lifecycle-field-guide.md`](07-driver-lifecycle-field-guide.md)

## 5. Why a GSP emulation is possible at all

The guest driver never talks to GPU silicon directly for management: since the
Turing generation, NVIDIA GPUs boot a **RISC-V firmware (the GSP)** that hosts
the Resource Manager. The driver's normal path is to talk to that firmware over
shared memory queues using a small RPC protocol.

That protocol is:

* **small** (a few dozen message types at the levels we need),
* **stable** (message layouts are versioned and compiled into the driver), and
* **observable** (everything arrives in guest memory we can read).

So instead of emulating a GPU, we emulate **the firmware's interface**. The
guest driver is satisfied because the protocol is satisfied; the guest's
*work* is then given to real hardware.

The firmware itself we study directly: it is a 28 MB RV64IMAC ELF that we have
decompiled and mapped (see `wiki/GSP-570.144W-Firmware-Analysis.md` and
`wiki/Address-Reference.md`).

## 6. What runs today

1. Guest boots, enumerates the virtual PCIe device, binds `nvidia.ko`.
2. The driver performs the GSP boot handshake with our emulation and receives
   plausible firmware identity/configuration.
3. The driver sets up the RPC message queues (guest sysmem), negotiates
   versions, and performs the resource-manager initialization exchanges.
4. Object creation (clients, devices, subdevices), capability queries,
   vaspaces, channels, CE caps, GMMU static info — every control the driver
   probes — is answered with a well-formed reply. Only a small set (23 control
   IDs) is special-cased with real values; the rest fall back to generic
   `SUCCESS` with zeroed params, and `GSP_RM_ALLOC` echoes the request params
   without server-side object management. Where the driver consumes a reply
   value, that value is still being corrected (the reply-fidelity ladder).
5. `nvidia-smi` works; CUDA user-space starts (`cuInit` / `cuDeviceGet`
   succeed). Context creation is currently blocked: `cuCtxCreate` returns
   `rc=999` (`CUDA_ERROR_UNKNOWN`) because the emulated engine list lacks the
   async copy engine (`COPY14`), so UVM's channel-manager and page-tree
   initialization is as far as the guest gets.

The CUDA-side contract and the reply-fidelity rules are documented in
[`../../userspace/cuda/docs/03-driver-and-gpu-interface.md` §9](../../userspace/cuda/docs/03-driver-and-gpu-interface.md)
and
[`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md);
start at
[`../../userspace/cuda/READ-ME-FIRST.md`](../../userspace/cuda/READ-ME-FIRST.md).

## 7. What is next (and why)

CUDA kernel execution requires the guest's *work* to run on the physical GPU.
That means the host module must:

1. know the memory the guest's GPU work refers to (the VA↔PA model),
2. reproduce or forward the channel/page-table state that the guest expects the
   GSP to own, and
3. submit the guest's compute work to real engine(s) and relay completions,
   faults and semaphores back.

The design for this, including what is known and what remains to be learned, is
in:
[`../04-implementation/01-known-knowns-and-unknowns.md`](../04-implementation/01-known-knowns-and-unknowns.md)
and
[`../04-implementation/02-practical-challenges-and-adversarial-interop.md`](../04-implementation/02-practical-challenges-and-adversarial-interop.md).