1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
|
# Performance
> **Scope.** How Open-IOV's performance is modelled, traced, and planned: the
> cost structure of the system, the metric set, what is observed today
> (honestly — there are no published numbers yet), the known gaps with their
> improvement plans, and the architecture questions the performance work will
> have to answer. Canonical sources:
> [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md)
> (the measurement contract), [`../STATE-CURRENT.md`](../STATE-CURRENT.md) (the
> dated snapshot) and [`RUNBOOK.md`](../docs/RUNBOOK.md); the docs tree wins on
> disagreement.
Status: 2026-10-30, tree `master @ b4930168`. Tags: `[live]` measured with
working code; `[wip]` measurement framework partial; `[planned]` designed,
not built.
## 1. The performance model
The cost structure of a vGPU session splits into four layers, and each is
measured separately:
| Layer | What it costs | Representative metric |
|---|---|---|
| **Initialization** | VM boot → driver probe → `cuInit` | `T_init` |
| **Control plane** | Resource-manager request/reply round trips (objects, controls, memory) | RPC latency p50/p99 |
| **Steady-state data path** | Submission → decode → forward → engine → completion → guest wake-up | push overhead; completion latency; kernel throughput |
| **Data movement** | Copies between guest memory and device memory | copy bandwidth (H2D/D2H/D2D) |
Presentation (3D frames) sits on top of the data path, with its own budget
(frame pacing, not just frame rate).
Three project rules govern every number:
1. **Correctness before speed.** A change that makes the vGPU faster but
breaks the acceptance corpus does not count.
2. **Proxy first, optimise by measurement.** The architecture deliberately
prototyped the safe path (emulate the interface, forward the work through
the module's own GSP client) before considering a hot path; whether to move
the hot path closer to direct submission is a *measurement decision*, not a
design preference.
3. **No number without methodology.** Numbers are only quoted with their
harness, versions, clocks and warm/cold state, and ideally as a ratio
against a same-guest, same-hardware baseline (the A/B method below).
## 2. The metric set
| Metric | Definition | Why it matters |
|---|---|---|
| `T_init` | VM boot → `cuInit` returns | user-visible startup |
| RPC latency p50/p99 | request write → reply visible in the status queue | init and management overhead |
| Push overhead | guest submission → engine start (forwarded) | steady-state latency |
| Semaphore completion latency | release written → tracker observes | per-operation sync cost |
| Copy bandwidth | H2D/D2H/D2D, large and small | data-movement quality |
| Kernel throughput | kernels/second for a fixed workload | launch overhead |
| Multi-tenant fairness | per-tenant progress under contention | isolation quality |
| Frame pacing | inter-frame deltas under load | graphics smoothness (p95 is the budget) |
## 3. How performance is traced today
* **Host-side telemetry.** The module timestamps its own work: per-run
counters in the kernel log (accepted / failed / retried / refused work,
decode outcomes, forward outcomes), `trace_printk`/tracepoints at dispatch
entry/exit for distributions, and counter surfaces readable through the
kernel's tracing and debug filesystems. Module parameters exist for targeted
probes during bring-up — they are debug surfaces, not configuration
([`Interfaces-and-Stability.md`](Interfaces-and-Stability.md)).
* **Guest-side instruments.** The guest test clients print their own
acceptance *and* timing lines (they are the ground truth for what the
application observed); guest-side kernel probes and ptrace tools attribute
driver-side waits when a client stalls.
* **The A/B harness.** Every measurement runs twice on the same machine and
preferably the same boot: **A** = the workload on the physical GPU with the
stock driver; **B** = the identical workload in the guest on the emulated
vGPU. Results are reported as **B/A ratios**; warm and cold, with variance
and run counts. One variable at a time; the harness is published so results
are comparable.
* **Offline tools — with a hard boundary.** The risc-v-gsp-rm-debugger
workbench models *instruction semantics and memory behaviour only*; it makes
**no timing claims** (its fidelity levels explicitly exclude time and
concurrency). Timing questions are answered on hardware, with
instrumentation.
* **The evaluation ladder.** Performance is only meaningful from the
*execution* rung upwards: initialization → runtime start → execution →
sustained → multi-tenant. Claims are made per rung, never across it.
## 4. Where it stands today (honest)
**There are no published performance numbers yet.** The framework above is
the contract; the first formally recorded measurements are scheduled, not
done. What *is* observed today, clearly labelled as internal observation
rather than benchmark:
* **Forwarded CUDA launch latency is on the order of hundreds of
milliseconds** end-to-end on the development setup (one ten-client arc
measured p50 ≈ 650 ms). This is the dominant known cost, dominated by the
submission trap → decode → forward chain and by polled completion waits —
not by the engine itself.
* **Data movement is verified correct, not yet measured for throughput**:
copies through the forwarded path round-trip byte-exact; the A/B bandwidth
runs (the `nvbandwidth`-class suite) are pending.
* **Graphics frame delivery is far below interactive rates** in the recorded
pipeline runs — the defined target is 60 fps *sustained with no forcing*
([`Graphics-Execution.md`](Graphics-Execution.md)); today's frame numbers
come from development instrumentation and are explicitly not a benchmark.
* **Initialization is functional**: the backend's GSP boot reaches init-done
in about a second on the reference hardware; `T_init` as a guest-visible
measurement has not been taken.
* **The software path is already cheap where it is work-shaped**: decoded
memory operations execute at host memory speed; control-plane replies are
produced inline in the emulation thread; the expensive parts are the
*control-heavy* structures (per-push passes, scanning, polling).
## 5. Observed gaps and the plan for each
| Gap | Observation | Plan |
|---|---|---|
| Forwarded-launch latency | hundreds of ms (internal); dominated by the multi-step forward chain and completion polling | instrument the distribution end to end; reduce per-push passes; move completion to event-driven paths; re-measure per step |
| Completion latency | releases are detected by polling the engine/guest memory | prefer engine-authored release paths already used by verified launches; doorbell/event wake-ups instead of periodic scans where possible |
| RPC dispatch cost | all management funnels through one emulation thread | build the RPC round-trip microbenchmark first; batch and prioritise; reduce wake-ups |
| Scanning overhead | per-pass discovery/scan work (guest RAM, shadow apertures) exists for correctness while registrations are incomplete | convert discovery to registered/learned state so per-pass cost scales with *work*, not with memory size |
| BAR0 shadow reads | uncached to the guest, as MMIO must be; bulk reads would be slow | if a consumer bulk-reads, introduce cache-safe sub-regions rather than a slower guest |
| Memory-model gap | guest VRAM is backed by host RAM today | real VRAM backing is a target; DMA-heavy workloads are not representative until it exists |
| Graphics frame path | far below 60 fps; forcing/diagnostic assistance excluded by the target definition | the graphics milestone ladder: natural completions with no mirror, then measure pacing |
| Multi-tenant fairness | two guests coexist; fairness under load is untested | define the fairness metric, run two-VM contention, then add scheduling policy |
| Initialization | functional but unmeasured | take `T_init` and the reply-fidelity tail as the next control-plane measurement |
## 6. The improvement programme
1. **Build the microbenchmarks first** — the four the measurement contract
names: RPC round-trip, MSI-X delivery latency, BAR0 shadow read latency,
and doorbell-polling cost. These attribute overhead to components before
any optimisation.
2. **Run and publish the A/B corpus** — the canonical suite (vector-add,
SGEMM, copy bandwidth, a GL frame-rate app, a framework op loop) with the
contract's pass criteria (e.g. ≥95% B/A for compute, ≥90% for GL and
framework loops) and full environment capture.
3. **Optimise the observed hot structures** — fewer per-push passes, batched
decode, event-driven completion, shadow-window reuse — each as an A/B
change with its numbers kept.
4. **Evaluate the hot-path placement decision** — keep the module as a GSP
client proxy, or move submissions closer to direct channels — using the
measured budgets as the criterion.
5. **Publish the first graphics measurement** — 60 fps sustained, no forcing,
p95 inter-frame pacing — as the first end-to-end performance claim.
6. **Then multi-tenant**: define fairness, measure it, and only then claim it.
## 7. Architecture areas that may need to improve — open questions
These are the structural questions the performance programme exists to close:
* **One ordered workqueue, one in-flight request.** The current design
serialises forwarded work for ordering safety. Is that a throughput ceiling
for multi-channel/multi-launch workloads, and how much pipelining is
possible without weakening the ordering guarantees the guest observes?
* **Per-pass scanning vs registered state.** Ring/USERD discovery, content
scans and shadow diffs are correctness mechanisms; can they be replaced (or
narrowed) by registrations learned on first sight, so steady-state cost does
not scale with aperture size?
* **Completion publication machinery.** Which of poll, engine-authored
release, notifier and event paths should carry the steady state — and what
is the p99, not just the median?
* **MMIO trap frequency.** Doorbells and shadow writes pay a VM-exit cost;
which guest patterns produce the most traps, and can they be coalesced or
served with less work?
* **The transport itself.** Guest-system-memory message queues plus doorbells
are deliberately conservative; would a shared-ring fast path change the
budget materially?
* **Hot-path placement.** Proxy through the module's own GSP session vs
moving hot submissions closer to the hardware — decided by the A/B numbers,
not by preference.
* **Multi-tenant scheduling.** Time-slicing exists only as a default; the
fairness policy (and MIG-backed partitioning where available) is a design
area, with fairness measurements defined before the claim.
* **Fidelity boundary.** The offline simulator can never answer timing or
concurrency questions; the performance programme therefore owns all timing
claims, and no timing claim may cite a simulator session.
## 8. Related pages
* [`Interfaces-and-Stability.md`](Interfaces-and-Stability.md) — which
surfaces are contracts and which are free to change.
* [`Execution-Frontier.md`](Execution-Frontier.md) — the compute path whose
latency this page budgets.
* [`Graphics-Execution.md`](Graphics-Execution.md) — the 60 fps target and its
milestone ladder.
* [`Physical-GPU-Backend.md`](Physical-GPU-Backend.md) — the backend the work
is forwarded to.
* [`Host-Module.md`](Host-Module.md) — the module whose threads, queues and
scans carry the costs above.
* [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md) —
the measurement contract: ladder, metrics, A/B method, microbenchmarks,
targets.
|
