Open-IOV vGPU-IO: host-side GSP emulation for unmodified NVIDIA guest drivers https://git.vgpu.io/open-iov.git/atom?h=gr-forwarding 2026-10-05T17:06:39+00:00 2026-10-05T17:06:39+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T17:06:39+00:00 urn:sha1:ae507e6f241c7e3498728f9d1ffc2d41ede5cd64 Add a default-off guest GR (compute) forwarding path so a guest CUDA launch submitted on its GR channel can be executed by the physical GR engine on the ladder channel, with fail-closed fallback to the host CUDA bridge and the unchanged software release path. - New module parameter gsp_phys_gr_forward (default 0) and accessor. - New GR execution gate: boot_stage.rmcall_gr_exec_ok, latched only after a real GR submission logs REAL GR SEM OK; the backend refuses while it is unset. - Sema capture: a completed 64-word inline QMD on a trusted, non-replay GR decode becomes an svc_gr_cand (push identity, decoded release0, parameter/program/pointer fields, up to 8 captured inline-write records); the CUDA bridge stays authoritative while the parameter is off and is reached in order forward -> bridge -> software release. - Backend: gsp_phys_backend_guest_gr_submit() maps pages at their own guest VAs through fresh Tier-1 small-PT pages/PDE installs or relocates inline-captured pages to ladder shadow pages (Tier 2), builds SET_OBJECT + the inline-QMD method run + REPORT_SEMAPHORE on the compute subchannel, reuses the shared GPFIFO/USERD/doorbell publish tail (extracted from the CE submit path, CE bytes unchanged) and polls the QMD release and/or module semaphore, draining the epoch-gated fault queue on timeout. - Sema wiring: GR requests reuse the INFLIGHT/hold/deferral machinery so the guest channel release applies only on the submit's success/failure edge; hold shims preserve the channel engine type. - Counters and a work_gr state line plus a stats accessor. - Bounds are fixed: 16 pages, 8 inline records, 64 QMD words; every pointer/size checked, pool exhaustion and stale mappings fail closed. - New ABI layouts pinned with static_asserts. Compile-only change: no device test performed. 2026-10-05T16:50:44+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T16:50:44+00:00 urn:sha1:d647c93e9375dc47613e0a554445e4b49467f119 - NV2080_CTRL_GR_CTX_BUFFER_INFO is 80 bytes (NvBool is NvU8: flags at 56..59, uuid at 60) and the params are 5136 bytes, not 5648; the previous ABI mismatch is why GR_GET_CTX_BUFFER_INFO returned NV_ERR_INVALID_ARGUMENT (0x1f) on every shared-VAS cycle. - Under gsp_phys_srv_ctxbuf the GSP-RM server allocates/maps/commits the engine context itself: do not install the client GR context-window PDEs and do not map the client ctx VA in the PDB, so a client mapping cannot clobber the server's own context-window page table. - Dump the server-written INSTANCE/RAMFC chunks after BIND/SCHEDULE on the shared GR path so the committed engine-context pointer/VA can be located. - Emit the full NOCAT diag buffer as 16-byte hex rows (the printable copy is capped at 192 dotted bytes and hides binary fault dumps). Untested on hardware at commit time; the next supervised GR cycle validates. 2026-10-05T16:47:47+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T16:47:47+00:00 urn:sha1:0f227d063e281c4b8ec21e61132ae1ddf0920053 The 708a3802 per-type VRAM change stripped an nvidia- prefix from mdev->type->sysfs_name, but sysfs_name is the registered type name verbatim; the mdev core only prefixes the type DIRECTORY with the parent driver string. Every lookup failed, so all mdevs silently fell back to 512 MiB, nvidia-vgpu-turing-1g was unreachable and available_instances capped at 1. Match sysfs_name directly in get_available, show_description and mdev_create. Also canonicalise the NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE encoder (BAR0+0x100ce0) by stripping factors of two from MAG so 512 MiB encodes 0x19 again (1 GiB 0x1a, 2 GiB 0x1b) instead of the non-canonical 0x204. Default-512M behaviour is now byte-equivalent to the known-good pre-VRAM module except max_instances=2. Module sha256 cb1724d3. 2026-10-05T16:25:05+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T16:25:05+00:00 urn:sha1:708a3802824358c1770d5ad0b3c2e24fae114eb8 - nv_vgpu_type_info/ioctl gain vram_mb; nv_vgpu_device gains fb_emul_size bound from the mdev type at create (default 512 MiB). - New nvidia-vgpu-turing-1g type (1024 MiB); turing max_instances 1 -> 2; show_description reports the per-type framebuffer and max_instance. - Thread fb_emul_size through GSCI fb_length/region/frtsOffset, WPR2, NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE, the vidmem allocator bound and the BAR1 shadow; bar_info callback uses the per-device size. - UNTESTED at commit time: builds clean; validation blocked by the Xid 140 incident (reboot pending). See Handoff-vGPU-IO.md ADDENDUM 48. 2026-10-05T16:15:30+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T16:15:30+00:00 urn:sha1:f90b1a123a60dea44ac98f05529ec2fd545c26eb - UPDATE_PDE_2 now also installs the GR context window PT pages (PDE 17+). - The shared-path fallback promotes PATCH then MAIN PA-only (mode-13 order, the real client's set); the mapped (bNonmapped=0) entry still hits the known 0x40 structural gate and is not sent. - Shared+srvctx: channel/BIND/schedule/token/GPFIFO fetch all green; the remaining failure is ROBUST_CHANNEL_GR_FAULT_DURING_CTXSW (0x2c). 2026-10-05T16:12:47+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T16:12:47+00:00 urn:sha1:a9fd600193b2cb4d4393a2600f8ff9fc973fecdf - DMA_UPDATE_PDE_2 client control installs the ladder leaf PT through the server's own MMU walker on the shared-management VAS (the raw-PDB entries are not preserved by the root migration); FLUSH_PDE_CACHE + post-install refill + PT diagnostics. - srvctx registry profile also disables split VAS management; MAP_MEMORY_DMA is not served by GSP-RM for this client (NV_ERR_INVALID_FUNCTION), so the PDE control replaces it. - shared-path context commit helper: GPU_INITIALIZE_CTX + PA-only GPU_PROMOTE_CTX for the server-reported (or client fallback) MAIN buffer; GR_GET_CTX_BUFFER_INFO sends bufferCount=64. - Result: shared VAS + srvctx no longer wedges; channel/BIND/schedule/token green, PBDMA fetches the GPFIFO (GPGet=1); remaining wall is ROBUST_CHANNEL_GR_FAULT_DURING_CTXSW (0x2c). 2026-10-05T11:00:45+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T11:00:45+00:00 urn:sha1:c36ecc19cf19d760c070db6bdd29227b0fadc12d An unreadable CE entry whose channel already had a decoded release VA was faked immediately and consumed with no log line, before any rescue attempt. A later pin-batch rewind then executed the skipped copy against a staging buffer the client had since refilled - the deterministic vadd input shift seen in runs 94/122/151/173b/215b, each with a silent pre-rescue fake. Now every unreadable CE entry with an expected payload is held for the bounded grace and the rescue is retried on each pass, so a translation that is still coming wins and the work runs before the release. Only an entry whose release was already delivered (the channel moved past entry_want) is consumed at once; the post-rescue fake remains the bounded, visible escape. The CE grace default moves 15000 -> 25000 ms (a translation arrived ~17 s after first examination in 215b). Under live campaign 229-236 (module 97eba5b7; green so far). 2026-10-05T04:12:56+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T04:12:56+00:00 urn:sha1:a9d776627f5057fd11c8d0a94bbfcca2358fe507 A consumed entry's work is never run; if its release lands via the fallback before the entry's copies have executed, the client refills the staging slot the copies then read (observed as the deterministic one-chunk vadd input shift). Keep untranslatable CE entries pending longer so their copies run before any release reaches the client. 8-cycle pilot (141-148): zero corruption; a red-fingerprint cycle (held=1165, ring_auto=6) healed to ring_auto=26 with faked=0 and the exact checksum, where the 8 s window recorded 1 corruption in 36. 2026-10-05T00:54:03+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-05T00:54:03+00:00 urn:sha1:0e073122a56cb2b8498aaacb79968bdf8309b309 The grace-expiry fake path consumed a held CE entry without marking its GPFIFO slot as consumed for the put epoch, and the replay guard only ran while the channel was not deferring. A later pin-batch rewind then re-read the slot, found no recorded identity, and executed the push's copies against a staging buffer the client had refilled after the faked release - a deterministic vadd input shift (a[k] == h_a[k + 2 MB]). - grace-fake consume path now calls svc_replay_note() before advancing; - the svc_replay_seen() replay skip is evaluated unconditionally: a slot consumed in the current epoch is skipped on every replay, deferral or not, until a put wrap clears the mark; - the once-only identity gate gains a content witness (FNV-1a hash of the recorded push bytes): the same entry words with different push content is a new push (clients rewrite push buffers in place) and runs its ops with a cleared done bitmap; suppression requires identical content; - snapshot write-back persists the hash array; new drain counter reident= counts content-mismatch cancellations; - the grace window stays at 8 s: a faked entry's work is never run, so the hold is that work's only chance (a 2.5 s experiment made red-class runs complete with corrupt data instead of timing out). Verified: module 8e90fc7e..., three consecutive end-to-end acceptance runs (MAIL + VADD bad=0 checksum=392344090.500000 + STATUS OK). 2026-10-04T22:42:34+00:00 Arthur Hanson Rasmusson arthur@vgpu.io 2026-10-04T22:42:34+00:00 urn:sha1:c1adf2727983f5cace8ca0f5d1119ac2256c348e The entry-identity gate suppresses every operation on a replayed entry, but an entry held because one operand's targeted scan was still running is decoded again on retry, and the ops that already succeeded would re-run against a staging buffer the client has since refilled (observed: a retried HtoD wrote the second vadd input half into the first). Track a per-slot bitmap of the operations (0..63) already executed for the slot's current entry identity, plus a complete flag for the pass that finished with no operand pending. On replay, a complete entry suppresses all ops; otherwise the bitmap decides per operation. Successful fills/copies set their bit, and the skipped count is published as ops_once in the drain line.