diff options
Diffstat (limited to 'docs/03-architecture/11-physical-backend-guest-wiring.md')
| -rw-r--r-- | docs/03-architecture/11-physical-backend-guest-wiring.md | 76 |
1 files changed, 76 insertions, 0 deletions
diff --git a/docs/03-architecture/11-physical-backend-guest-wiring.md b/docs/03-architecture/11-physical-backend-guest-wiring.md index 2229365..943dd44 100644 --- a/docs/03-architecture/11-physical-backend-guest-wiring.md +++ b/docs/03-architecture/11-physical-backend-guest-wiring.md @@ -145,6 +145,82 @@ emulated. The remaining bridge must be added at the reply point `gsp_rm_send()`, the inbound dispatch point `gsp_rm_handle_element()`, and the work-plane seams (see §2). +### 0.4 The lifecycle at a glance (init -> work -> completion -> teardown) + +One page for the whole physical-backed path, with the module's function names +for grepping. Status tags: `[live]` proven with the stock driver; `[gated]` +implemented but default-off, awaiting the ladder gate; `[wall]` the current +blocker; `[target]` the success latch that arms the guest work path. + +```text +GUEST (QEMU) HOST: vgsp.ko (module-owned GPU) PHYSICAL TU104 +──────────────────────────────────────────────────────────────────────────────────── + +PHASE I - INIT + I0 preflight nvidia bound + green | no vgsp/mdev/QEMU | pstore empty [live] + I1 claim insmod: D0 lock (board + root port) -> unbind distribution + driver -> bind open-iov-gsp-phys -> board check [live] + I2 boot FWSEC -> Booter Load -> RISC-V active / WPR2 up (stage 3d) + -> GSP-RM init event 0x1001 rpc_result=0 (stage 3e, ~950 ms) [live] + I3 mdev create -> nv_vgpu_device + gsp_emu_init(); backend start: + "joined the live module-owned claim (READY)" [live] + I4 ladder debugfs gsp-phys-rm-call triggers 5/3/4: + stage 1 root/device/subdevice + stage 5 adopt [live] + stage 3 GR ctx prep -> channel -> GPU_PROMOTE_CTX [wall] + stage 4 BIND -> GPFIFO_SCHEDULE -> TSG schedule -> token + -> push; latch "REAL GR SEM OK" => rmcall_gr_exec_ok = 1 [target] + I5 guest QEMU with mdev -> nvidia.ko probes; emulation plane serves + boot handshake, static-config overlay, RM battery, UVM + registration -> CUDA ready [live] + I6 M1 prep guest RAM registered (gsp-phys-guest-ram); watchers + (pin_gr_stream, semfix_uvmt) [gated] + +PHASE II - RUNNING WORK (native GR path; gsp_phys_gr_forward=1, + gsp_phys_guest_mem=1, cuda_bridge=0) + guest CUDA app -> libcuda -> nvidia.ko -> GR channel GPFIFO push -> BAR doorbell + host gsp_sema_service_doorbell() -> svc_pump_channels() -> svc_decode_stream() + -> svc_qmd_accum() -> svc_gr_qmd_decode() / svc_gr_cand_capture() + -> svc_gr_request() / svc_gr_req_fill() -> svc_fwd_gr_work() + -> gate gsp_phys_backend_gr_ready() [needs rmcall_gr_exec_ok] + -> gsp_phys_backend_guest_gr_submit() + Tier 1: guest VA -> PA -> HPA, fresh 128 B PT at the guest VA, + DMA_UPDATE_PDE_2 per 2 MiB window + Tier 2: captured inline bytes -> shadow page (else refused) + -> gsp_phys_rmcall_build_gr_launch() + -> gsp_phys_rmcall_publish_locked() -> ladder channel GPFIFO + doorbell + GPU real GSP scheduler -> GR engine executes the QMD + (scaffolding, not the destination: cuda_bridge replays the kernel on host CUDA) + +PHASE III - COMPLETION + GPU engine authors the guest-visible QMD release word (5 -> 6 in the + acceptance program) and the channel release + host poll (200 ms) -> "GUEST GR LAUNCH OK ... qmd_rel=0x00000006" + -> guest release/event: gsp_rm_post_channel_event() -> MSI-X injection + -> counters work_gr: req/ok/fallback/refused/timeout/fault + guest semaphore observed -> CUDA continues -> verified output + fault recType=4 faultingEngine="GR" -> fwd_gr_fault; releases applied exactly + once, failed launch never retried + +PHASE IV - TEARDOWN + T1 drain mdev remove -> QEMU stop -> watchers stopped -> rmmod vgsp + T2 release gsp_phys_gpu_release(): UNLOADING_GUEST_DRIVER -> quiesce -> + WPR2 down -> free staging -> rebind distribution driver (only if + the board answers and WPR2 is gone) -> release D0 locks [live] + T3 verify rmmod rc=0 | "rebound to nvidia (rc=1)" | WPR2 down | 5 power + domains locked | pstore empty | nvidia-smi green | 0 Xid [live] + FAIL RmInitAdapter failed (no Xid) -> PCI function-level reset; + WPR2-locked wedge -> stop; reboot only per the reboot-approval + policy (written reason + post-reboot plan first) +``` + +The single open step on this path is I4's stage-3 GR context commit: the +server must accept a real engine-context construct (the reference client's +channel shape plus its atomic promote entry set) before stage 4 can latch +`REAL GR SEM OK`. Everything from I4-stage-4 onward is implemented and +fail-closed; until the latch is set, every forward refuses with +`exec_ok=0` and the guest work path behaves exactly as it does today +(see §1.1 and the M1 gate in §5 P6). + --- ## 1. Target architecture: runtime state machine |