path: root/docs/03-architecture/09-full-system-picture.md

# The Full System Picture

> **What this document is.** The visual, end-to-end picture of Open-IOV: every
> component in the finished system, how they interact, where the work actually
> executes, how tenancy is isolated and scheduled, how the design stays portable
> across driver and hardware versions, and how far each piece is from done.
> The other architecture documents explain *why*; this one shows *what* and
> *where*, all the way down. Diagrams are Mermaid and render on GitHub and in
> most Markdown viewers.

**Status legend used throughout** (the project is honest about maturity):

| Tag | Meaning |
|---|---|
| `[done]` | implemented and exercised with a stock guest driver |
| `[wip]` | implemented in part; actively being completed |
| `[planned]` | designed, not implemented |
| hardware/firmware | exists in the physical world; referenced, not built by us |
| vendor | NVIDIA components we interoperate with, unmodified |

## 1. The end-state system map

All components of the finished system, with the execution backend this project
has chosen (a module-owned GPU driven through its real GSP firmware; see
[`08-physical-gpu-integration.md`](08-physical-gpu-integration.md)).

```mermaid
flowchart TB
  subgraph VM["Guest VM - stock software, no Open-IOV code"]
    APP["Applications<br/>CUDA / OpenGL / Vulkan / video"]
    USR["Userspace stack<br/>libcuda, libcudart, libGLX, libvulkan"]
    GKO["Stock nvidia.ko<br/>RM core + UVM + modeset/DRM"]
    APP --> USR --> GKO
  end

  subgraph HV["Hypervisor - KVM and QEMU"]
    VFIO["VFIO mdev transport<br/>regions, mmap, eventfd / irqfd"]
    IOMMU["IOMMU / IOMMUFD<br/>per-VM DMA address spaces"]
  end

  subgraph MOD["open-iov-vfio.ko - the only host component"]
    VDEV["Virtual device<br/>config space, BAR0/1/3, MSI-X"]
    EMU["Emulation core<br/>GSP boot + GSP-RM RPC + objects"]
    MEMM["Memory model<br/>VA to PA, pins, VRAM shadow, page tables"]
    WORK["Work path<br/>pump, decode, translate, forward"]
    BE["Execution backend<br/>real GSP session + RM client"]
    VDEV --> EMU
    EMU --> MEMM
    EMU --> WORK
    MEMM --> WORK
    WORK --> BE
    BE --> MEMM
  end

  subgraph HW["Physical GPU"]
    GSP["GSP firmware - the real RM server"]
    ENG["Engines<br/>GR, CE pair, DEC, ENC"]
    GMMU["GMMU and page tables"]
    VRAM["VRAM + framebuffer"]
    DB["BAR0 / USERD doorbells, MSI-X"]
  end

  subgraph HOSTOS["Host OS"]
    ND["Distribution nvidia.ko<br/>unmodified; unbound while the service owns the GPU"]
    IGPU["Display on another GPU / integrated graphics"]
  end

  GKO -->|"MMIO + DMA + MSI-X"| VFIO
  VFIO --> VDEV
  BE -->|"msgq + doorbells"| GSP
  WORK -->|"real push streams on real channels"| ENG
  BE --> GMMU
  MEMM --> IOMMU

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  classDef hw fill:#e2e8f0,stroke:#4a5568
  classDef vendor fill:#e7e0f5,stroke:#5a4a8a

  class VDEV,EMU done
  class MEMM,WORK wip
  class BE plan
  class IOMMU,VFIO done
  class GSP,ENG,GMMU,VRAM,DB hw
  class ND,USR,GKO,APP vendor
```

**Reading the map.** The guest sees only a PCIe device. Everything between
that device and the physical engines is either emulated (control plane) or
translated (work plane) by one kernel module, with no daemon, no host userspace
and no modified driver on either side. The distribution `nvidia.ko` remains
installed and unmodified on the host; it is simply unbound from the GPU while
the service owns it, and the host's display runs on another GPU. Unbinding is
not a neutral operation on every platform — the board's power domain can
power-gate the GPU and hard-freeze the host — so ownership must hold a D0 lock
over the GPU, its board companions and the root port across the whole
unbind/rebind window; the hazard and the lock are described in
[`08-physical-gpu-integration.md`](08-physical-gpu-integration.md) §4.2, and the
driver must never be unbound without the lock.

## 2. The control plane: GSP emulation

### 2.1 The GSP boot handshake

The guest driver believes it boots a real GSP. The module completes the
observable boot stages itself and reports the status the driver polls. In the
end state, the *backend* additionally boots the physical GPU's real GSP for
work execution — the two sessions are independent.

```mermaid
stateDiagram-v2
  direction LR
  [*] --> PCIProbe
  PCIProbe --> FWSEC: driver kicks boot
  FWSEC --> Booter: FRTS status observed
  Booter --> RISCV: WPR2 programmed
  RISCV --> BootComplete: falcon reports running
  BootComplete --> MsgqInit: queue setup in guest sysmem
  MsgqInit --> RmInit: version, client, device, subdevice
  RmInit --> Runtime: objects, vaspaces, channels
  Runtime --> [*]: driver unload / reset

  note right of FWSEC
    Emulated today: the module drives the
    stage transitions and answers status
    reads from the BAR0 shadow. The real
    backend boots the physical GSP in
    parallel [planned].
  end note
```

### 2.2 Guest initialization, end to end

```mermaid
sequenceDiagram
  autonumber
  participant GD as Stock nvidia.ko, guest
  participant DEV as Virtual device, BAR0 shadow + MSI-X
  participant GE as GSP emulation thread
  participant GRM as GSP-RM emulation
  participant BE as Execution backend

  GD->>DEV: PCI probe, BAR setup, MSI-X enable
  DEV-->>GD: config space and capability replies
  GD->>DEV: boot kicks and WPR2 writes
  DEV->>GE: stage transitions
  GE-->>GD: falcon status - boot complete
  GD->>GRM: message queues + RPCs - version, client, device
  Note over GRM: every reply follows the fidelity ladder -<br/>status only, then coherent zeroed params,<br/>then coherent consumed values, then exact values
  GRM-->>GD: coherent replies for consumed fields
  GD->>GRM: object allocations, vaspaces, channels, CE caps
  GRM-->>GD: handles, engine and class topology
  GD->>GRM: UVM register GPU, page trees, context
  GRM->>BE: resource needs - real memory, channels [planned]
  BE-->>GRM: real objects and channels [planned]
```

### 2.3 RPC surface and object lifecycle

```mermaid
flowchart LR
  subgraph GUEST["Guest RM requests"]
    A1["version / capability queries"]
    A2["client, device, subdevice"]
    A3["memory alloc + mmap descriptors"]
    A4["vaspace + page-table contracts"]
    A5["channels, CE caps, events"]
  end
  subgraph EMU["GSP-RM emulation"]
    B1["dispatcher by function ID"]
    B2["special-cased handlers - real values"]
    B3["generic fallback - success + zeroed params"]
    B4["object table - handles, params echoed"]
    B5["reply fidelity checks"]
  end
  A1 --> B1
  A2 --> B1
  A3 --> B1
  A4 --> B1
  A5 --> B1
  B1 --> B2
  B1 --> B3
  B2 --> B4
  B3 --> B5
  B4 --> B5

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  class B1,B2,B4 done
  class B3,B5 wip
```

## 3. The memory model

Guest work refers to GPU virtual addresses. The module must know which physical
pages they denote — for semaphores, push buffers, page trees, and (in the end
state) for programming the real GPU's MMU so hardware walks the same pages.

```mermaid
flowchart LR
  subgraph G["Guest"]
    GVA["GPU virtual addresses<br/>client objects and mappings"]
    GPT["Guest page tables<br/>incl. server-owned ranges"]
    SYS["Guest system memory"]
    VRS["VRAM objects - BAR1 aperture view"]
  end

  subgraph MM["open-iov-vfio.ko memory model"]
    DISC["Discovery<br/>structure walk + content scan + pins"]
    PMAP["VA to PA map<br/>page-granular, merge rebuilds"]
    SHAD["BAR1 VRAM shadow"]
    PIN["Pin tables<br/>self-discovered + authoritative injected"]
  end

  subgraph H["Host / hardware"]
    DMA["VFIO DMA - read and write guest pages"]
    RGMMU["Real GMMU tables<br/>same pages, real hardware walks"]
  end

  GVA --> GPT --> SYS
  GVA --> VRS
  VRS --> SHAD
  SYS --> DMA
  GPT --> DISC
  DISC --> PMAP
  PIN --> PMAP
  PMAP --> DMA
  PMAP --> RGMMU
  SYS -.->|"pinned and mapped once"| RGMMU

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class DISC,PMAP,DMA done
  class PIN,SHAD wip
  class RGMMU plan
```

The two contracts that make this possible without guest cooperation:

* Channel memory descriptors the guest creates already carry **physical
  addresses** (instance memory, method buffers, USERD, RAMFC, notifiers).
* The server-owned page-table contract (`COPY_SERVER_RESERVED_PDES`) gives the
  page-table level physical addresses; the leaves are ours to populate.

The remaining work is to derive every mapping from these protocol objects as
they are created, so a clean boot needs no external tooling.

## 4. The work path

### 4.1 Today: software-executed bridge

```mermaid
flowchart LR
  A["Guest push appended<br/>GPFIFO entry + doorbell"] --> B["Channel pump reads entry"]
  B --> C{"Entry readable?"}
  C -->|yes| D["Decode method stream"]
  C -->|no| E["Rescue / reconstruct the page<br/>or resolve via pins"]
  E --> D
  D --> F["Execute understood work in software<br/>memsets, copies, semaphore releases"]
  F --> G["Write releases and completions<br/>into guest memory"]
  G --> H["Guest trackers advance"]

  D -.->|"planned"| I["Forward to a real engine"]

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class A,B,D done
  class E,F,G,H wip
  class I plan
```

This is a correctness bridge: it advances synchronization and memory
operations, and it is why the guest's initialization walks as far as it does
today. It cannot execute kernels.

### 4.2 End state: forwarded work

```mermaid
sequenceDiagram
  autonumber
  participant GU as Guest GPFIFO + doorbell
  participant P as Channel pump
  participant D as Decoder
  participant M as Memory model
  participant B as Execution backend
  participant E as GPU engines
  participant S as Guest trackers

  GU->>P: doorbell / put pointer advance
  P->>D: read push entry
  alt entry unreadable
    D->>M: resolve or reconstruct the page
  end
  D->>D: parse the method stream
  alt pure state or notification
    D->>M: apply release, membar, fence directly
  else real engine work
    D->>B: translated submission
    B->>E: real push stream on a real channel
    E-->>B: completion or fault from hardware
    B->>M: completion written into guest memory
  end
  M-->>S: tracker values advance
```

Fast paths (semaphores, fences, some memops) stay in the emulation in both
phases: they are microseconds in hardware and lose value by round-tripping
through the real GPU.

### 4.3 How the GPU reaches runtime and executes work

```mermaid
flowchart TB
  subgraph BOOT["Guest view - emulated"]
    A["Probe"] --> B["GSP boot stages"]
    B --> C["GSP-RM init"]
    C --> D["Objects, vaspaces, channels"]
    D --> E["UVM register + page trees"]
    E --> F["Context created, allocations mapped"]
    F --> G["Pushes submitted"]
  end
  subgraph REAL["Backend view - real"]
    R1["Module binds GPU, boots real GSP"] --> R2["Real RM client: device, vaspaces"]
    R2 --> R3["Real channels and page tables"]
    R3 --> R4["Guest VA mapped to same pages"]
    R4 --> R5["Real submissions scheduled by GSP"]
    R5 --> R6["Completion and faults relayed to guest"]
  end
  G -.->|"translation [planned]"| R4

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class A,B,C,D,E done
  class F,G wip
  class R1,R2,R3,R4,R5,R6 plan
```

## 5. Scheduling and multi-tenancy

There is no hardware partitioning (no SR-IOV) on this GPU generation, so
tenancy is a **logical VF model**: each VM holds one mdev instance, and the
module gives each instance its own GSP-RM session, channels, page tables and
scheduling policy. “Per-VF GSP emulation” therefore means per-mdev emulation
context plus a per-mdev real GSP client in the backend.

```mermaid
flowchart TB
  subgraph TEN["Tenants"]
    T1["VM A - mdev 1"]
    T2["VM B - mdev 2"]
    T3["VM C - mdev N"]
  end
  subgraph MOD["open-iov-vfio.ko"]
    Q1["Emulation context 1<br/>BARs, queues, counters"]
    Q2["Emulation context 2"]
    Q3["Emulation context N"]
    POL["Fairness policy [planned]<br/>time slices, quotas, priority, burst"]
    C1["Real GSP client 1 [planned]"]
    C2["Real GSP client 2 [planned]"]
    C3["Real GSP client N [planned]"]
  end
  subgraph GPU["Physical GPU"]
    SCHED["Real GSP scheduler<br/>engine contexts and runlists"]
    GRC["GR contexts"]
    CEC["CE contexts"]
  end
  T1 --> Q1 --> C1
  T2 --> Q2 --> C2
  T3 --> Q3 --> C3
  C1 --> POL
  C2 --> POL
  C3 --> POL
  POL --> SCHED
  SCHED --> GRC
  SCHED --> CEC

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class Q1,Q2,Q3 done
  class C1,C2,C3,POL,SCHED plan
```

**How multi-tenant performance works (and is intended to):**

* Each tenant's *state* operations complete in the module on a fast path
  (microseconds, no hardware round trip).
* Each tenant's *real work* runs on the physical engines at native speed; the
  virtualization overhead is the submission translation plus completion relay,
  which amortizes with work size.
* Fairness is a module-side policy on top of the firmware scheduler (time
  slices, quotas). It must be measured, not assumed; the measurement harness is
  planned with the performance ladder in
  [`../05-operations/01-performance-correctness-security.md`](../05-operations/01-performance-correctness-security.md).

## 6. Isolation: what is guaranteed, at which level

```mermaid
flowchart TB
  H1["Hardware IOMMU / SMMU<br/>per-VM DMA address spaces<br/>guarantee: memory isolation between VMs"]
  H2["GPU MMU contexts - per client page tables [planned]<br/>guarantee: one tenant's engine work cannot address another's memory"]
  V1["VFIO mdev boundary<br/>all device access mediated; config/BAR filtering<br/>guarantee: guest gets only its own device instance"]
  M1["Module state separation<br/>per-mdev emulation context, queues, pins<br/>guarantee: no cross-tenant state pointers"]
  P1["Policy and audit [planned]<br/>fairness, rate limits, side-channel review<br/>guarantee: performance isolation, adversarial robustness"]

  H1 --> V1 --> M1
  H2 --> P1
  M1 --> P1
```

| Layer | Isolation strength | Status |
|---|---|---|
| IOMMU / SMMU between VMs | hardware-enforced | available today (hypervisor) |
| KVM memory and vCPU boundary | hypervisor-enforced | available today |
| VFIO mdev mediation (one device per tenant) | hypervisor-enforced | implemented |
| Per-mdev module state and emulation context | software-enforced | implemented |
| Per-client real page tables and GSP clients | software plus GPU MMU | planned |
| Performance isolation and side channels | policy | planned, must be measured |

The honest statement: **memory isolation is guaranteed by hardware and the
hypervisor today**; *engine-level context separation and performance isolation*
are the parts this project must still build and audit.

## 7. How the host kernel schedules the module

```mermaid
flowchart LR
  subgraph CTX["Caller contexts"]
    QEMU["QEMU / VFIO ioctls and mmap"]
    VCPU["vCPU exits on trapped MMIO"]
    FW["Physical GSP messages - backend"]
    USR["Debug interfaces - dev only"]
  end
  subgraph KERN["Linux execution contexts in the module"]
    VF["VFIO callbacks"]
    IRQ["IRQ handlers - physical MSI-X, eventfd signals"]
    WQ["Workqueues - RPC, cleanup, bottom halves"]
    KT["kthreads - GSP emulation per mdev, channel service"]
    TM["Timers and poll loops - doorbells, housekeeping"]
  end
  QEMU --> VF --> WQ
  VCPU --> VF
  FW --> IRQ --> WQ
  WQ --> KT
  KT --> TM
  KT --> WQ

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class VF,IRQ,WQ,KT done
  class TM plan
```

Design rules that keep this healthy: bounded stack frames (audited), no long
scans while holding service locks, blocking work in kthreads not in VFIO
callbacks, and never tearing down the module while a VM holds the device.

## 8. Hypervisor integration: mdev, VFIO, KVM

```mermaid
sequenceDiagram
  autonumber
  participant ORCH as Orchestrator / admin
  participant SYS as Module + mdev sysfs
  participant QEMU as QEMU with vfio-pci
  participant KVM as KVM
  participant GUEST as Guest kernel

  ORCH->>SYS: create mdev - type nvidia-nvidia-vgpu-turing, UUID
  SYS-->>ORCH: /sys/bus/mdev/devices/uuid ready
  QEMU->>SYS: open VFIO device, query regions and IRQ info
  QEMU->>SYS: mmap BAR shadows - region index to vaddr
  QEMU->>SYS: VFIO_DEVICE_SET_IRQS - eventfd per MSI-X vector
  QEMU->>KVM: register irqfd for guest injection
  QEMU->>KVM: map guest RAM - IOMMU / IOMMUFD
  GUEST->>QEMU: MMIO access to the virtual BAR0
  QEMU->>SYS: region read or write, or direct shadow access
  SYS-->>GUEST: MSI-X delivered via eventfd to irqfd to vCPU
  ORCH->>SYS: destroy mdev after VM stops - reverse order
```

Key properties: the mdev is the *device instance*; VFIO is the *transport*
(region access, IRQ signalling, DMA bridging); KVM is the *delivery* (irqfd,
MMIO exits, vCPU scheduling). The module registers an emulated IOMMU device so
VFIO creates the group; guest memory access flows through the VFIO DMA
interface. Full detail in
[`06-host-module-and-hybrid-model.md`](06-host-module-and-hybrid-model.md).

## 9. Component detail: BARs, engines, channels, GMMU

```mermaid
flowchart LR
  subgraph GV["Guest-visible device"]
    CFG["PCI config space<br/>TU104 identity, BAR sizes, MSI-X caps"]
    B0["BAR0 - 16 MB registers<br/>doorbell, mailbox, PBA"]
    B1["BAR1 - 256 MB VRAM aperture"]
    B3["BAR3 - 32 MB instance / PRAMIN window"]
    MX["MSI-X - 32 vectors"]
  end
  subgraph MODS["Module shadow state"]
    C0["Filtered config writes"]
    S0["BAR0 shadow - boot state, doorbells"]
    S1["BAR1 shadow - VRAM backing [wip]"]
    S3["BAR3 shadow - instance memory"]
    IR["Per-vector eventfd injection"]
  end
  subgraph CH["Channel machinery"]
    GPF["GPFIFO - push entries"]
    USR["USERD + doorbell tokens"]
    RF["RAMFC + instance memory"]
    MB["Method buffers"]
    NOT["Notifiers and error buffers"]
  end
  subgraph EN["Engines"]
    GR["GR - compute, grid launches"]
    CE["CE pair - copies and UVM memops"]
    DEC["DEC - video decode [planned]"]
    ENC["ENC - video encode [planned]"]
    RL["Runlist / TSG scheduling"]
  end
  subgraph MM["GMMU"]
    HI["Page-directory hierarchy"]
    LEAF["Leaf PTEs"]
    SRV["Server-owned ranges"]
  end
  CFG --> C0
  B0 --> S0 --> GPF
  B1 --> S1
  B3 --> S3 --> RF
  MX --> IR
  GPF --> USR
  GPF --> RF
  GPF --> MB
  GPF --> NOT
  USR --> GR
  USR --> CE
  CE --> RL
  GR --> RL
  RL --> ENC
  RL --> DEC
  S1 --> HI
  S3 --> HI
  HI --> LEAF
  HI --> SRV

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class CFG,C0,B0,S0,B3,S3,MX,IR,GPF,USR,RF,MB,NOT,GR,CE,RL done
  class B1,S1,HI,LEAF,SRV wip
  class DEC,ENC plan
```

## 10. Portability: versions, hardware, hypervisors

```mermaid
flowchart LR
  A["Firmware / driver image"] -->|"CRC32 + SHA-256"| B["Firmware analyzer"]
  B --> C["Version tables<br/>and YAML profiles"]
  C --> D["Dynamic RPC dispatcher"]
  C --> E["Hardware abstraction profiles"]
  D --> F["Version-keyed handlers"]
  E --> G["Architecture tables<br/>engine inventory, GMMU formats"]
  F --> H["GSP-RM emulation"]
  G --> H
  H --> I["Same guest interface"]
  J["New driver version<br/>or new GPU"] --> A
  J --> K["Measurement + A/B against host oracle"]
  K --> C

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef wip fill:#fff3cd,stroke:#9a7b1a
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class B done
  class C,D,F done
  class E,G,H wip
  class K plan
```

The *interface* is the product: RPC/control inventories are measured, not
guessed; new versions enter by adding a profile and running the A/B workflow
(see [`../05-operations/02-debugging-and-maintenance-workflows.md`](../05-operations/02-debugging-and-maintenance-workflows.md)).
The execution backend is similarly version-keyed: the real GSP protocol is
spoken as a client, with the module carrying the small set of host-side calls
it needs.

## 11. Performance: distance to bare metal

```mermaid
flowchart LR
  subgraph FAST["Emulated fast path - state only"]
    A["Release, fence, membar"] --> B["Applied in the module"]
    B --> C["microseconds; no GPU round trip;<br/>faster than hardware would be"]
  end
  subgraph DATA["Forwarded work path"]
    D["Guest submission"] --> E["Decode + translate + submit"]
    E --> F["Engine executes at native speed"]
    F --> G["Completion relay"]
  end
  C -.-> H["Cost model: overhead is per submission,<br/>not per unit of work"]
  G -.-> H
  H -.-> I["Large copies and kernels amortize to near native;<br/>launch-bound workloads pay the translation"]

  classDef done fill:#d5f5d5,stroke:#1e7a1e
  classDef plan fill:#f8d7da,stroke:#9a1a1a
  class A,B,C done
  class D,E,F,G,H,I plan
```

| Aspect | Expectation | Status |
|---|---|---|
| State/synchronization latency | low microseconds, no round trip | measured in debug logs |
| Copy bandwidth (large) | near native; DMA is DMA | not measured yet |
| Kernel launch overhead | translation cost per submission; amortized | not measured yet |
| Multi-tenant fairness | policy-controlled, measurable | policy planned |
| Bare-metal parity claim | only after the measurement ladder is green | see operations doc |

We do not claim parity today, and the docs say so in plain words: the honest
claim is "initialization works; execution is in progress".

## 12. Implementation and test status

How far each component is from the finished system. "Tested" means exercised
against a stock guest driver with evidence in the repository.

| Component | Implemented | Tested | Next step |
|---|---|---|---|
| mdev + VFIO device, config space, BARs, MSI-X | yes | yes | keep regression-free |
| GSP boot emulation (all stages) | yes | yes | unchanged under backend |
| GSP-RM message queues + RPC + controls | yes | yes | reply fidelity for remaining consumed fields |
| Object/alloc model (handles, echoes) | yes | yes | server-side object tracking where consumed |
| Engine/class topology (incl. async copy engine) | yes | yes | version profiles |
| Guest memory access + VA-to-PA discovery | partial | yes | protocol-derived discovery, no external pins |
| Push decode + semaphore completion | partial | yes | complete GR/compute encodings |
| Software execution bridge (memsets, copies, releases) | partial | yes | remains bridge only |
| Page-table population for server-owned ranges | no | no | design + validator, then implement |
| Execution backend (own GPU + real GSP) | no | no | device ownership, real GSP bring-up, RM client |
| Real work forwarding (GR and CE) | no | no | after backend bring-up |
| Fault semantics (buffers, replay, cancel) | no | no | after forwarding |
| Multi-tenant fairness policy | no | no | after multi-tenant forwarding |
| Version matrix (2+ driver families) | partial | partial | per-version profiles + A/B |
| Windows guest | no | no | after Linux execution path |
| Non-KVM transports (Xen, Argo, OpenXT) | no | no | transport interface + backend |
| aarch64 hosts (Grace, GB10) | no | no | architecture-neutral memory model |
| Debug interfaces (dev only) | yes | yes | gate destructive ones; production build |

## 13. How the unmodified-driver guarantee is achieved

* **Emulate what the guest observes, forward what it uses.** Every interface
  the stock driver touches is answered by the module; every operation that must
  be fast goes to real hardware. The guest never learns it is virtualized for
  the CUDA/initialization path — it sees a TU104 whose GSP answers correctly.
* **Reply fidelity, not just status.** A reply whose fields the driver consumes
  is only considered done when the values are coherent (status-only, then
  coherent zeroed params, then coherent consumed fields, then exact values).
  This is the mechanism that makes arbitrary graphics and CUDA stacks work
  without per-application code.
* **No guest code, no paravirtual enlightenment, no patches.** The guest may
  install whatever software it likes; the module presents physical hardware.
  The planned execution backend keeps this property: the guest's work is
  translated, never the guest's driver.

## See also

- [`01-system-overview.md`](01-system-overview.md) - principles and layers in prose
- [`06-host-module-and-hybrid-model.md`](06-host-module-and-hybrid-model.md) - the module, memory model, build contracts
- [`08-physical-gpu-integration.md`](08-physical-gpu-integration.md) - the chosen execution backend and its decision record
- [`../04-implementation/01-known-knowns-and-unknowns.md`](../04-implementation/01-known-knowns-and-unknowns.md) - gaps and plans
- [`../05-operations/01-performance-correctness-security.md`](../05-operations/01-performance-correctness-security.md) - measurement ladder, threat model