Open-IOV vGPU-IO: host-side GSP emulation for unmodified NVIDIA guest drivers https://git.vgpu.io/open-iov.git/atom?h=cs-share 2026-10-06T21:37:50+00:00 2026-10-06T21:37:50+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-06T21:37:50+00:00 urn:sha1:930789eafcb2ed05929c5ca8f92b5fda3b58875c 2026-10-06T21:05:16+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-06T21:05:16+00:00 urn:sha1:c8b9b89b15c334ea61e8fba635763e406e7b9f0c 2026-10-06T20:29:29+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-06T19:41:01+00:00 urn:sha1:191100b51d1e6ed36bf4ff776107d551c4793621 2026-10-06T19:19:54+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-06T19:19:54+00:00 urn:sha1:dc5b6507889d07003cf01ed1d784c0faf48bec03 2026-10-06T15:57:33+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-06T15:57:33+00:00 urn:sha1:3f892b2924c0bdd15aaa04899b879601bd263981 - Mode 28 golden execution lane behind gsp_phys_rmcall_gr_promote=28 with a shape bitmask (gsp_phys_rmcall_golden_exec_shape: ring / entries=8 / sysmem USERD / USERD size / hObjectBuffer=VAS / skip replay TURING_A) and a USERD control-offset helper. Finding: the RM-managed golden channel hangs any GR object construct (-110, no assert) as soon as client submission wiring (ring or sysmem USERD) is attached, so only the captured server-owned shape constructs; the machinery is kept for future wiring ideas. - fwclient mask bit 11 (0x800): send each non-MAIN FWCLIENT promote entry as its own one-entry control and continue past refusals. V6 acceptance matrix: PM, PATCH, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP accepted, FECS_EVENT refused 0x1f. - fwclient mask bit 12 (0x1000): golden9 VA-only global bookings (BUNDLE_CB, PAGEPOOL, ATTRIBUTE_CB, RTV_CB_GLOBAL) accepted on V6; bits 13/14 (0x2000/0x4000): FECS PA-only (refused 0x1f) and FECS VA-only (accepted). - gsp_phys_rmcall_srv_main_ignore: ignore a server-reported MAIN in the srv-ctx commit and force the client INITIALIZE_CTX + VA-only fallback. - gsp_sema_service: pool semaphores never fall back to the learned map; queue the absolute release via svc_pending_add until svc_pool_pa_resolve succeeds, and judge pool slots through the verified resolver in svc_apply_pending and the release decode (UVM registration stall fix). 2026-10-05T21:16:18+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T21:16:18+00:00 urn:sha1:fc8d7923c83c6babd0ee8c23565ac18b709fa5a5 Ladder mode 25 reproduces the real 570.144 host client's golden-image construct on the golden replay handles: captured channel shape (flags 0x00200320, hObjectBuffer 0, gpFifoOffset 0, gpFifoEntries 0x20, engineType 1, host memdesc substitutions), a single atomic 9-entry GPU_PROMOTE_CTX on the golden channel (MAIN id0 and PATCH id2 combined, ids 3/4/5/6 VA-only globals, ids 9/10/11 FECS/PRIV/UNR combined), then the TURING_A 3D object. The golden VAS/channel/3D object stay alive and the existing V6 FWCLIENT promote follows on the golden-initialized GR-instance state; a failed golden step aborts with no retry and no follow-up promote. The captured VAS COPY_SERVER_RESERVED_PDES control is implemented gated (fwclient mask bit 8, zero-level no-op form) because its level PAs are server-internal. Mode clamp/name/help extended to 25; modes 1-24 unchanged. 2026-10-05T20:04:33+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T20:04:33+00:00 urn:sha1:7c1fceb5654cda3628531214934b93e96a507700 - gsp_phys_rmcall_fwclient_mask (default 0x3f): per-entry selection for the mode-24 FWCLIENT promote set; bit6 selects the MAIN UVM VA-only shape. - golden replay: tolerate the client-local system-memory PB step and continue with hObjectBuffer=0; on-ladder TURING_A 3D object retained best-effort. Live bisection on the KMD channel: MAIN alone reaches the per-buffer commit and stops at the engine-context memdesc state; PM-only and PATCH-only are accepted; any multi-entry control containing MAIN is refused 0x1f. All cycles release clean. 2026-10-05T19:58:43+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T19:58:43+00:00 urn:sha1:5a77352f78e6cd28280f2dc559f677d31ae25c38 - golden replay (gsp_phys_rmcall_golden, default off): replay the stock client's post-load golden channel allocations (separate VAS, pushbuffer objects, USERD, bare GR channel, TURING_A 3D object) before the ladder channel; best-effort: a failure warns and the ladder continues. - on-ladder TURING_A (0xc597) 3D object allocation before the compute object (best-effort) to initialize server-side GR-instance state. - promote mode 24: the reference FWCLIENT non-3D set (MAIN, PM, PATCH, FECS_EVENT, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP) as one GPU_PROMOTE_CTX on the KMD channel, with per-entry logging and explicit omission semantics; mode clamps/descriptions extended to 24. Live: mode 23 soft-stops 0x1f on the server-managed shared channel and reaches the per-buffer commit on the KMD channel, where it stops at the missing engine-context memdesc; mode 24 emits all six entries and is refused with 0x1f before BIND. All cycles release clean, GPU green. 2026-10-05T18:03:48+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T18:03:48+00:00 urn:sha1:aff698d003d256a41bf7d4062bb81792457d3219 Make gsp_phys_rmcall_srv_ctxcommit an int mode: 1 (default) = map + INITIALIZE_CTX + UVM-shape VA-only promote; 2 = V6 client variant, no INITIALIZE_CTX and one combined PA+VA MAIN entry with bInitialize=1 and bNonmapped=0 so the first ctxsw initializes from the golden image; 0 = pure server-managed. Lets the next supervised cycle bisect both entry shapes without a rebuild. Build clean; not hardware-tested (GPU wedged). 2026-10-05T17:40:24+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T17:06:39+00:00 urn:sha1:d3f061966bb9e23fdba7f6cc0aa231acdf31f36e Add a default-off guest GR (compute) forwarding path so a guest CUDA launch submitted on its GR channel can be executed by the physical GR engine on the ladder channel, with fail-closed fallback to the host CUDA bridge and the unchanged software release path. - New module parameter gsp_phys_gr_forward (default 0) and accessor. - New GR execution gate: boot_stage.rmcall_gr_exec_ok, latched only after a real GR submission logs REAL GR SEM OK; the backend refuses while it is unset. - Sema capture: a completed 64-word inline QMD on a trusted, non-replay GR decode becomes an svc_gr_cand (push identity, decoded release0, parameter/program/pointer fields, up to 8 captured inline-write records); the CUDA bridge stays authoritative while the parameter is off and is reached in order forward -> bridge -> software release. - Backend: gsp_phys_backend_guest_gr_submit() maps pages at their own guest VAs through fresh Tier-1 small-PT pages/PDE installs or relocates inline-captured pages to ladder shadow pages (Tier 2), builds SET_OBJECT + the inline-QMD method run + REPORT_SEMAPHORE on the compute subchannel, reuses the shared GPFIFO/USERD/doorbell publish tail (extracted from the CE submit path, CE bytes unchanged) and polls the QMD release and/or module semaphore, draining the epoch-gated fault queue on timeout. - Sema wiring: GR requests reuse the INFLIGHT/hold/deferral machinery so the guest channel release applies only on the submit's success/failure edge; hold shims preserve the channel engine type. - Counters and a work_gr state line plus a stats accessor. - Bounds are fixed: 16 pages, 8 inline records, 64 QMD words; every pointer/size checked, pool exhaustion and stale mappings fail closed. - New ABI layouts pinned with static_asserts. Compile-only change: no device test performed. (cherry picked from commit ae507e6f241c7e3498728f9d1ffc2d41ede5cd64)