diff options
Diffstat (limited to 'wiki/Performance.md')
| -rw-r--r-- | wiki/Performance.md | 192 |
1 files changed, 192 insertions, 0 deletions
diff --git a/wiki/Performance.md b/wiki/Performance.md new file mode 100644 index 0000000..b62d6a7 --- /dev/null +++ b/wiki/Performance.md @@ -0,0 +1,192 @@ +# Performance + +> **Scope.** How Open-IOV's performance is modelled, traced, and planned: the +> cost structure of the system, the metric set, what is observed today +> (honestly — there are no published numbers yet), the known gaps with their +> improvement plans, and the architecture questions the performance work will +> have to answer. Canonical sources: +> [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md) +> (the measurement contract), [`../STATE-CURRENT.md`](../STATE-CURRENT.md) (the +> dated snapshot) and [`RUNBOOK.md`](../docs/RUNBOOK.md); the docs tree wins on +> disagreement. + +Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` measured with +working code; `[wip]` measurement framework partial; `[planned]` designed, +not built. + +## 1. The performance model + +The cost structure of a vGPU session splits into four layers, and each is +measured separately: + +| Layer | What it costs | Representative metric | +|---|---|---| +| **Initialization** | VM boot → driver probe → `cuInit` | `T_init` | +| **Control plane** | Resource-manager request/reply round trips (objects, controls, memory) | RPC latency p50/p99 | +| **Steady-state data path** | Submission → decode → forward → engine → completion → guest wake-up | push overhead; completion latency; kernel throughput | +| **Data movement** | Copies between guest memory and device memory | copy bandwidth (H2D/D2H/D2D) | + +Presentation (3D frames) sits on top of the data path, with its own budget +(frame pacing, not just frame rate). + +Three project rules govern every number: + +1. **Correctness before speed.** A change that makes the vGPU faster but + breaks the acceptance corpus does not count. +2. **Proxy first, optimise by measurement.** The architecture deliberately + prototyped the safe path (emulate the interface, forward the work through + the module's own GSP client) before considering a hot path; whether to move + the hot path closer to direct submission is a *measurement decision*, not a + design preference. +3. **No number without methodology.** Numbers are only quoted with their + harness, versions, clocks and warm/cold state, and ideally as a ratio + against a same-guest, same-hardware baseline (the A/B method below). + +## 2. The metric set + +| Metric | Definition | Why it matters | +|---|---|---| +| `T_init` | VM boot → `cuInit` returns | user-visible startup | +| RPC latency p50/p99 | request write → reply visible in the status queue | init and management overhead | +| Push overhead | guest submission → engine start (forwarded) | steady-state latency | +| Semaphore completion latency | release written → tracker observes | per-operation sync cost | +| Copy bandwidth | H2D/D2H/D2D, large and small | data-movement quality | +| Kernel throughput | kernels/second for a fixed workload | launch overhead | +| Multi-tenant fairness | per-tenant progress under contention | isolation quality | +| Frame pacing | inter-frame deltas under load | graphics smoothness (p95 is the budget) | + +## 3. How performance is traced today + +* **Host-side telemetry.** The module timestamps its own work: per-run + counters in the kernel log (accepted / failed / retried / refused work, + decode outcomes, forward outcomes), `trace_printk`/tracepoints at dispatch + entry/exit for distributions, and counter surfaces readable through the + kernel's tracing and debug filesystems. Module parameters exist for targeted + probes during bring-up — they are debug surfaces, not configuration + ([`Interfaces-and-Stability.md`](Interfaces-and-Stability.md)). +* **Guest-side instruments.** The guest test clients print their own + acceptance *and* timing lines (they are the ground truth for what the + application observed); guest-side kernel probes and ptrace tools attribute + driver-side waits when a client stalls. +* **The A/B harness.** Every measurement runs twice on the same machine and + preferably the same boot: **A** = the workload on the physical GPU with the + stock driver; **B** = the identical workload in the guest on the emulated + vGPU. Results are reported as **B/A ratios**; warm and cold, with variance + and run counts. One variable at a time; the harness is published so results + are comparable. +* **Offline tools — with a hard boundary.** The risc-v-gsp-rm-debugger + workbench models *instruction semantics and memory behaviour only*; it makes + **no timing claims** (its fidelity levels explicitly exclude time and + concurrency). Timing questions are answered on hardware, with + instrumentation. +* **The evaluation ladder.** Performance is only meaningful from the + *execution* rung upwards: initialization → runtime start → execution → + sustained → multi-tenant. Claims are made per rung, never across it. + +## 4. Where it stands today (honest) + +**There are no published performance numbers yet.** The framework above is +the contract; the first formally recorded measurements are scheduled, not +done. What *is* observed today, clearly labelled as internal observation +rather than benchmark: + +* **Forwarded CUDA launch latency is on the order of hundreds of + milliseconds** end-to-end on the development setup (one ten-client arc + measured p50 ≈ 650 ms). This is the dominant known cost, dominated by the + submission trap → decode → forward chain and by polled completion waits — + not by the engine itself. +* **Data movement is verified correct, not yet measured for throughput**: + copies through the forwarded path round-trip byte-exact; the A/B bandwidth + runs (the `nvbandwidth`-class suite) are pending. +* **Graphics frame delivery is far below interactive rates** in the recorded + pipeline runs — the defined target is 60 fps *sustained with no forcing* + ([`Graphics-Execution.md`](Graphics-Execution.md)); today's frame numbers + come from development instrumentation and are explicitly not a benchmark. +* **Initialization is functional**: the backend's GSP boot reaches init-done + in about a second on the reference hardware; `T_init` as a guest-visible + measurement has not been taken. +* **The software path is already cheap where it is work-shaped**: decoded + memory operations execute at host memory speed; control-plane replies are + produced inline in the emulation thread; the expensive parts are the + *control-heavy* structures (per-push passes, scanning, polling). + +## 5. Observed gaps and the plan for each + +| Gap | Observation | Plan | +|---|---|---| +| Forwarded-launch latency | hundreds of ms (internal); dominated by the multi-step forward chain and completion polling | instrument the distribution end to end; reduce per-push passes; move completion to event-driven paths; re-measure per step | +| Completion latency | releases are detected by polling the engine/guest memory | prefer engine-authored release paths already used by verified launches; doorbell/event wake-ups instead of periodic scans where possible | +| RPC dispatch cost | all management funnels through one emulation thread | build the RPC round-trip microbenchmark first; batch and prioritise; reduce wake-ups | +| Scanning overhead | per-pass discovery/scan work (guest RAM, shadow apertures) exists for correctness while registrations are incomplete | convert discovery to registered/learned state so per-pass cost scales with *work*, not with memory size | +| BAR0 shadow reads | uncached to the guest, as MMIO must be; bulk reads would be slow | if a consumer bulk-reads, introduce cache-safe sub-regions rather than a slower guest | +| Memory-model gap | guest VRAM is backed by host RAM today | real VRAM backing is a target; DMA-heavy workloads are not representative until it exists | +| Graphics frame path | far below 60 fps; forcing/diagnostic assistance excluded by the target definition | the graphics milestone ladder: natural completions with no mirror, then measure pacing | +| Multi-tenant fairness | two guests coexist; fairness under load is untested | define the fairness metric, run two-VM contention, then add scheduling policy | +| Initialization | functional but unmeasured | take `T_init` and the reply-fidelity tail as the next control-plane measurement | + +## 6. The improvement programme + +1. **Build the microbenchmarks first** — the four the measurement contract + names: RPC round-trip, MSI-X delivery latency, BAR0 shadow read latency, + and doorbell-polling cost. These attribute overhead to components before + any optimisation. +2. **Run and publish the A/B corpus** — the canonical suite (vector-add, + SGEMM, copy bandwidth, a GL frame-rate app, a framework op loop) with the + contract's pass criteria (e.g. ≥95% B/A for compute, ≥90% for GL and + framework loops) and full environment capture. +3. **Optimise the observed hot structures** — fewer per-push passes, batched + decode, event-driven completion, shadow-window reuse — each as an A/B + change with its numbers kept. +4. **Evaluate the hot-path placement decision** — keep the module as a GSP + client proxy, or move submissions closer to direct channels — using the + measured budgets as the criterion. +5. **Publish the first graphics measurement** — 60 fps sustained, no forcing, + p95 inter-frame pacing — as the first end-to-end performance claim. +6. **Then multi-tenant**: define fairness, measure it, and only then claim it. + +## 7. Architecture areas that may need to improve — open questions + +These are the structural questions the performance programme exists to close: + +* **One ordered workqueue, one in-flight request.** The current design + serialises forwarded work for ordering safety. Is that a throughput ceiling + for multi-channel/multi-launch workloads, and how much pipelining is + possible without weakening the ordering guarantees the guest observes? +* **Per-pass scanning vs registered state.** Ring/USERD discovery, content + scans and shadow diffs are correctness mechanisms; can they be replaced (or + narrowed) by registrations learned on first sight, so steady-state cost does + not scale with aperture size? +* **Completion publication machinery.** Which of poll, engine-authored + release, notifier and event paths should carry the steady state — and what + is the p99, not just the median? +* **MMIO trap frequency.** Doorbells and shadow writes pay a VM-exit cost; + which guest patterns produce the most traps, and can they be coalesced or + served with less work? +* **The transport itself.** Guest-system-memory message queues plus doorbells + are deliberately conservative; would a shared-ring fast path change the + budget materially? +* **Hot-path placement.** Proxy through the module's own GSP session vs + moving hot submissions closer to the hardware — decided by the A/B numbers, + not by preference. +* **Multi-tenant scheduling.** Time-slicing exists only as a default; the + fairness policy (and MIG-backed partitioning where available) is a design + area, with fairness measurements defined before the claim. +* **Fidelity boundary.** The offline simulator can never answer timing or + concurrency questions; the performance programme therefore owns all timing + claims, and no timing claim may cite a simulator session. + +## 8. Related pages + +* [`Interfaces-and-Stability.md`](Interfaces-and-Stability.md) — which + surfaces are contracts and which are free to change. +* [`Execution-Frontier.md`](Execution-Frontier.md) — the compute path whose + latency this page budgets. +* [`Graphics-Execution.md`](Graphics-Execution.md) — the 60 fps target and its + milestone ladder. +* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work + is forwarded to. +* [`Host-Module.md`](Host-Module.md) — the module whose threads, queues and + scans carry the costs above. +* [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md) — + the measurement contract: ladder, metrics, A/B method, microbenchmarks, + targets. |