diff options
Diffstat (limited to 'docs/05-operations/01-performance-correctness-security.md')
| -rw-r--r-- | docs/05-operations/01-performance-correctness-security.md | 71 |
1 files changed, 71 insertions, 0 deletions
diff --git a/docs/05-operations/01-performance-correctness-security.md b/docs/05-operations/01-performance-correctness-security.md index 0a27709..583d6c8 100644 --- a/docs/05-operations/01-performance-correctness-security.md +++ b/docs/05-operations/01-performance-correctness-security.md @@ -101,6 +101,77 @@ handoff documents carry the current rung. * Bulk copies and sustained compute: as close to native as the forwarding path allows; the forwarding design (proxy vs direct) is decided by measurement. +### 3.4 Microbenchmarks (build these first) + +1. **RPC round-trip** — from the guest, submit a DEBUG/NOOP-ish RPC in a loop; + time doorbell-write → response-visible. Host side: `gsp_emu_get_stats()` + gives `total_latency_ns` and counts — divide for per-RPC host cost. Add + `trace_printk` at dispatch entry/exit for a distribution; read via + `/sys/kernel/debug/tracing/trace`. +2. **MSI-X latency** — timestamp (guest TSC) at RPC submit → the guest + driver's interrupt handler. Compare with the host `eventfd_signal` + timestamp. +3. **BAR0 shadow read latency** — guest loops reading a shadow register; + expect memory speed (uncached, so roughly 100–300 ns). Anything ≥ 1 µs + means a trap where there should not be one. +4. **Doorbell polling cost** — host `top`/`perf` on the `gsp_emu` thread; the + adaptive poll (busy → 50–100 µs sleep) should sit at ~0% CPU when idle and + react within one sleep quantum under load. Tune the poll quantum only with + data. + +### 3.5 A/B methodology + +Every measurement runs **twice on the same machine, same boot if possible**: + +* **A (bare metal)**: the workload on the physical GPU via the host's stock + driver (or the OpenRM `nvidia.ko` directly on the hardware). +* **B (vGPU)**: the identical workload in the guest on the emulated vGPU. + +Same binary, same input sizes, same environment (governor, clocks, hugepages). +Report **B/A ratio %** — never absolute numbers alone. Change one variable at a +time; if B regresses, you changed the stack, the workload, or the machine, and +finding which comes before optimizing. + +The runner is `tools/benchmark_infrastructure.sh` (extend it; keep the A/B +pattern); in-guest execution goes through `scripts/vgpu_vm.py exec` or the +guest test agent. Workload suites and their pass criteria: + +| Suite | What | Pass criterion | +|---|---|---| +| `cuda-samples vectorAdd` | smoke + element-wise throughput | correct result; ≥ 95% B/A | +| `simpleCUBLAS` | SGEMM TFLOPS | ≥ 95% B/A | +| `nvbandwidth` (if available) | H2D/D2H/D2D copy GB/s | report; DMA-path validation | +| `glxgears -info` / a real GL app | frame rate | ≥ 90% B/A | +| `nvenc`/`nvdec` sample | transcode fps | functional (perf later) | +| PyTorch CUDA op (matmul loop) | end-to-end framework | ≥ 90% B/A | + +Always log: guest/host driver versions, GPU clocks (`nvidia-smi -q -d CLOCK` +on both sides — throttling skews everything), and the command lines. Record +results with the release evidence (§5) so regressions are visible across +commits. + +### 3.6 Where the overhead lives (profile in this order) + +1. **RPC dispatch** — the single biggest lever; everything the guest driver + does funnels through the emulation thread. +2. **Doorbell polling cadence** — too slow costs latency, too hot burns host + CPU. +3. **eventfd/MSI-X delivery** — measure, do not assume. +4. **BAR0 uncached semantics** — the shadow is uncached to the guest, which is + MMIO-correct but slow for large reads; if the driver bulk-reads shadows, + consider cache-safe sub-regions. +5. **Memory-model gap** — guest VRAM is host RAM today; DMA-heavy workloads + are not representative until the real backing path exists. + +### 3.7 Optimizing safely + +* Change one thing, re-run the A/B pair, keep the numbers with the change. +* Never optimize off bare-metal-only data: the goal is B/A, not A. +* A change that makes B faster but breaks the §2.1 corpus or the rung ladder + does not count. +* Record results (date, versions, B/A ratios) where the next release can see + them. + ## 4. Security and isolation ### 4.1 Threat model |