# Performance

> **Scope.** How Open-IOV's performance is modelled, traced, and planned: the
> cost structure of the system, the metric set, what is observed today
> (honestly — there are no published numbers yet), the known gaps with their
> improvement plans, and the architecture questions the performance work will
> have to answer. Canonical sources:
> [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md)
> (the measurement contract), [`../STATE-CURRENT.md`](../STATE-CURRENT.md) (the
> dated snapshot) and [`RUNBOOK.md`](../docs/RUNBOOK.md); the docs tree wins on
> disagreement.

Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` measured with
working code; `[wip]` measurement framework partial; `[planned]` designed,
not built.

## 1. The performance model

The cost structure of a vGPU session splits into four layers, and each is
measured separately:

| Layer | What it costs | Representative metric |
|---|---|---|
| **Initialization** | VM boot → driver probe → `cuInit` | `T_init` |
| **Control plane** | Resource-manager request/reply round trips (objects, controls, memory) | RPC latency p50/p99 |
| **Steady-state data path** | Submission → decode → forward → engine → completion → guest wake-up | push overhead; completion latency; kernel throughput |
| **Data movement** | Copies between guest memory and device memory | copy bandwidth (H2D/D2H/D2D) |

Presentation (3D frames) sits on top of the data path, with its own budget
(frame pacing, not just frame rate).

Three project rules govern every number:

1. **Correctness before speed.** A change that makes the vGPU faster but
   breaks the acceptance corpus does not count.
2. **Proxy first, optimise by measurement.** The architecture deliberately
   prototyped the safe path (emulate the interface, forward the work through
   the module's own GSP client) before considering a hot path; whether to move
   the hot path closer to direct submission is a *measurement decision*, not a
   design preference.
3. **No number without methodology.** Numbers are only quoted with their
   harness, versions, clocks and warm/cold state, and ideally as a ratio
   against a same-guest, same-hardware baseline (the A/B method below).

## 2. The metric set

| Metric | Definition | Why it matters |
|---|---|---|
| `T_init` | VM boot → `cuInit` returns | user-visible startup |
| RPC latency p50/p99 | request write → reply visible in the status queue | init and management overhead |
| Push overhead | guest submission → engine start (forwarded) | steady-state latency |
| Semaphore completion latency | release written → tracker observes | per-operation sync cost |
| Copy bandwidth | H2D/D2H/D2D, large and small | data-movement quality |
| Kernel throughput | kernels/second for a fixed workload | launch overhead |
| Multi-tenant fairness | per-tenant progress under contention | isolation quality |
| Frame pacing | inter-frame deltas under load | graphics smoothness (p95 is the budget) |

## 3. How performance is traced today

* **Host-side telemetry.** The module timestamps its own work: per-run
  counters in the kernel log (accepted / failed / retried / refused work,
  decode outcomes, forward outcomes), `trace_printk`/tracepoints at dispatch
  entry/exit for distributions, and counter surfaces readable through the
  kernel's tracing and debug filesystems. Module parameters exist for targeted
  probes during bring-up — they are debug surfaces, not configuration
  ([`Interfaces-and-Stability.md`](Interfaces-and-Stability.md)).
* **Guest-side instruments.** The guest test clients print their own
  acceptance *and* timing lines (they are the ground truth for what the
  application observed); guest-side kernel probes and ptrace tools attribute
  driver-side waits when a client stalls.
* **The A/B harness.** Every measurement runs twice on the same machine and
  preferably the same boot: **A** = the workload on the physical GPU with the
  stock driver; **B** = the identical workload in the guest on the emulated
  vGPU. Results are reported as **B/A ratios**; warm and cold, with variance
  and run counts. One variable at a time; the harness is published so results
  are comparable.
* **Offline tools — with a hard boundary.** The risc-v-gsp-rm-debugger
  workbench models *instruction semantics and memory behaviour only*; it makes
  **no timing claims** (its fidelity levels explicitly exclude time and
  concurrency). Timing questions are answered on hardware, with
  instrumentation.
* **The evaluation ladder.** Performance is only meaningful from the
  *execution* rung upwards: initialization → runtime start → execution →
  sustained → multi-tenant. Claims are made per rung, never across it.

## 4. Where it stands today (honest)

**There are no published performance numbers yet.** The framework above is
the contract; the first formally recorded measurements are scheduled, not
done. What *is* observed today, clearly labelled as internal observation
rather than benchmark:

* **Forwarded CUDA launch latency is on the order of hundreds of
  milliseconds** end-to-end on the development setup (one ten-client arc
  measured p50 ≈ 650 ms). This is the dominant known cost, dominated by the
  submission trap → decode → forward chain and by polled completion waits —
  not by the engine itself.
* **Data movement is verified correct, not yet measured for throughput**:
  copies through the forwarded path round-trip byte-exact; the A/B bandwidth
  runs (the `nvbandwidth`-class suite) are pending.
* **Graphics frame delivery is far below interactive rates** in the recorded
  pipeline runs — the defined target is 60 fps *sustained with no forcing*
  ([`Graphics-Execution.md`](Graphics-Execution.md)); today's frame numbers
  come from development instrumentation and are explicitly not a benchmark.
* **Initialization is functional**: the backend's GSP boot reaches init-done
  in about a second on the reference hardware; `T_init` as a guest-visible
  measurement has not been taken.
* **The software path is already cheap where it is work-shaped**: decoded
  memory operations execute at host memory speed; control-plane replies are
  produced inline in the emulation thread; the expensive parts are the
  *control-heavy* structures (per-push passes, scanning, polling).

## 5. Observed gaps and the plan for each

| Gap | Observation | Plan |
|---|---|---|
| Forwarded-launch latency | hundreds of ms (internal); dominated by the multi-step forward chain and completion polling | instrument the distribution end to end; reduce per-push passes; move completion to event-driven paths; re-measure per step |
| Completion latency | releases are detected by polling the engine/guest memory | prefer engine-authored release paths already used by verified launches; doorbell/event wake-ups instead of periodic scans where possible |
| RPC dispatch cost | all management funnels through one emulation thread | build the RPC round-trip microbenchmark first; batch and prioritise; reduce wake-ups |
| Scanning overhead | per-pass discovery/scan work (guest RAM, shadow apertures) exists for correctness while registrations are incomplete | convert discovery to registered/learned state so per-pass cost scales with *work*, not with memory size |
| BAR0 shadow reads | uncached to the guest, as MMIO must be; bulk reads would be slow | if a consumer bulk-reads, introduce cache-safe sub-regions rather than a slower guest |
| Memory-model gap | guest VRAM is backed by host RAM today | real VRAM backing is a target; DMA-heavy workloads are not representative until it exists |
| Graphics frame path | far below 60 fps; forcing/diagnostic assistance excluded by the target definition | the graphics milestone ladder: natural completions with no mirror, then measure pacing |
| Multi-tenant fairness | two guests coexist; fairness under load is untested | define the fairness metric, run two-VM contention, then add scheduling policy |
| Initialization | functional but unmeasured | take `T_init` and the reply-fidelity tail as the next control-plane measurement |

## 6. The improvement programme

1. **Build the microbenchmarks first** — the four the measurement contract
   names: RPC round-trip, MSI-X delivery latency, BAR0 shadow read latency,
   and doorbell-polling cost. These attribute overhead to components before
   any optimisation.
2. **Run and publish the A/B corpus** — the canonical suite (vector-add,
   SGEMM, copy bandwidth, a GL frame-rate app, a framework op loop) with the
   contract's pass criteria (e.g. ≥95% B/A for compute, ≥90% for GL and
   framework loops) and full environment capture.
3. **Optimise the observed hot structures** — fewer per-push passes, batched
   decode, event-driven completion, shadow-window reuse — each as an A/B
   change with its numbers kept.
4. **Evaluate the hot-path placement decision** — keep the module as a GSP
   client proxy, or move submissions closer to direct channels — using the
   measured budgets as the criterion.
5. **Publish the first graphics measurement** — 60 fps sustained, no forcing,
   p95 inter-frame pacing — as the first end-to-end performance claim.
6. **Then multi-tenant**: define fairness, measure it, and only then claim it.

## 7. Architecture areas that may need to improve — open questions

These are the structural questions the performance programme exists to close:

* **One ordered workqueue, one in-flight request.** The current design
  serialises forwarded work for ordering safety. Is that a throughput ceiling
  for multi-channel/multi-launch workloads, and how much pipelining is
  possible without weakening the ordering guarantees the guest observes?
* **Per-pass scanning vs registered state.** Ring/USERD discovery, content
  scans and shadow diffs are correctness mechanisms; can they be replaced (or
  narrowed) by registrations learned on first sight, so steady-state cost does
  not scale with aperture size?
* **Completion publication machinery.** Which of poll, engine-authored
  release, notifier and event paths should carry the steady state — and what
  is the p99, not just the median?
* **MMIO trap frequency.** Doorbells and shadow writes pay a VM-exit cost;
  which guest patterns produce the most traps, and can they be coalesced or
  served with less work?
* **The transport itself.** Guest-system-memory message queues plus doorbells
  are deliberately conservative; would a shared-ring fast path change the
  budget materially?
* **Hot-path placement.** Proxy through the module's own GSP session vs
  moving hot submissions closer to the hardware — decided by the A/B numbers,
  not by preference.
* **Multi-tenant scheduling.** Time-slicing exists only as a default; the
  fairness policy (and MIG-backed partitioning where available) is a design
  area, with fairness measurements defined before the claim.
* **Fidelity boundary.** The offline simulator can never answer timing or
  concurrency questions; the performance programme therefore owns all timing
  claims, and no timing claim may cite a simulator session.

## 8. Related pages

* [`Interfaces-and-Stability.md`](Interfaces-and-Stability.md) — which
  surfaces are contracts and which are free to change.
* [`Execution-Frontier.md`](Execution-Frontier.md) — the compute path whose
  latency this page budgets.
* [`Graphics-Execution.md`](Graphics-Execution.md) — the 60 fps target and its
  milestone ladder.
* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
  is forwarded to.
* [`Host-Module.md`](Host-Module.md) — the module whose threads, queues and
  scans carry the costs above.
* [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md) —
  the measurement contract: ladder, metrics, A/B method, microbenchmarks,
  targets.