summaryrefslogtreecommitdiffstats
path: root/kernel
Commit message (Collapse)AuthorAge
* vgsp: point inline-QMD launch at the guest QMD buffer + GR post-mortem loggingArthur Hanson Rasmusson6 days
|
* vgsp: alias-map indirect high-VA GR pages and fix up shadow pointersArthur Hanson Rasmusson6 days
| | | | | | | | | The mail kernel's output buffer has no QMD pointer word; the engine reaches it through a pointer inside the parameter bank, so the Tier-1 refusal at its high VA killed the launch. Alias-map such a page's guest HPA at a fresh ladder line and rewrite every matching 8-byte pointer inside the shadowed copy pages (param/program) to the alias, so engine writes land in the guest's own page and the guest's DtoH sees them.
* vgsp: track shadow ranges per record for multi-page GR buffersArthur Hanson Rasmusson6 days
| | | | | | | The inline-record path appends pages after the pointer pages, so the adjacent-entry heuristic missed the second param page. Record each shadowed range (base guest VA, line, mapped PTE count) and extend any page that continues a recorded range, up to 16 pages per 128 B line.
* vgsp: contiguous shadow PTEs for multi-page GR buffers (R4)Arthur Hanson Rasmusson6 days
| | | | | | | | | The mail launch's param buffer spans two pages but the QMD names only the base page. The second page has no pointer word, so the Tier-2 shadow path skipped it and the launch was refused. Map a page with no pointer word that is contiguous with the previous shadow-mapped page into the next PTE of the same 128 B fetch line (each line covers up to 16 consecutive 4 KB pages), keeping the rewritten base-pointer buffer contiguous for the engine.
* vgsp: raise GR forward shadow pool to 24 pagesArthur Hanson Rasmusson6 days
| | | | | | | The mail launch has 8 pointer/QMD pages plus up to two release/sem shadow lines; the 4-page pool exhausted and refused the launch even with the HPA-copy fallback. 24 covers the 16-page launch cap plus the two release shadows.
* vgsp: GR forward shadow fallback for high-VA guest pages (R4)Arthur Hanson Rasmusson6 days
| | | | | | | | | | | | The M1 forward captured the guest's mail launch but was refused at Tier-1: the guest's UVM program/param pages live at 0x7d6e... VAs whose 2 MiB window indices are outside this VAS's PDE range, so DMA_UPDATE_PDE_2 returns 0x1f and the launch fell back (bridge disabled) with the guest polling. When the page HPA was resolved on the identity domain, copy the full guest page into the Tier-2 shadow chunk and rewrite the QMD pointer pair, so pages the engine only reads need no inline coverage. Release/sem pages keep their existing direct-alias shadow lines.
* vgsp: golden9k - size every promoted GR ctx buffer from ↵golden-replayArthur Hanson Rasmusson6 days
| | | | | | | | | | | | | | | | | | GET_ENGINE_CONTEXT_PROPERTIES (REAL GR SEM OK) The golden replay promoted 0x1000 placeholders for FECS_EVENT/PRIV/UNR and 1-page VA-only mappings for the four 3D globals; the server's own geometry reports FECS_EVENT 0x10000, PRIV/UNR 0x80000 each, PAGEPOOL 0x20000, ATTRIBUTE_CB 0x7d7400 and RTV_CB_GLOBAL 0x80000 on this TU104. The girmap sizes are wrong for the engine's ctx-init access-map copies and the ctxsw bundle, producing INJECTED_BUNDLE_ERROR + FECS ctxsw TDR on every VAS shape. fwclient-mask bit 10 (0x400) sizes every promoted buffer from the server's GET_ENGINE_CONTEXT_PROPERTIES (0x801707), re-spaces the VAs to per-buffer caps, extends the ctx small-PT pages to six 2 MB windows and promotes the real sizes. Result (golden9k2): golden promote status 0, FWCLIENT MAIN promote status 0, stage 3 complete, REAL GR SEM OK sem=0xc0ffee01 GPGet=1 GPPut=1, clean teardown, no wedge.
* vgsp: golden9j - FBMEM backing + VAS mappings for the four 3D global ctx buffersArthur Hanson Rasmusson6 days
|
* vgsp: golden9i - shared-management ladder VAS binding + walker ctx window ↵Arthur Hanson Rasmusson6 days
| | | | for the golden channel
* vgsp: golden9g - fabricated golden-VAS page-table levels + walker ctx windowArthur Hanson Rasmusson6 days
|
* vgsp: golden9f - golden channel on the golden VAS with the client PDBArthur Hanson Rasmusson6 days
| | | | | | | | | | | | The golden9e ctxsw TDR persisted with capture-faithful FBMEM buffers and mapped VAs. Source dig: the ladder VAS is IS_EXTERNALLY_OWNED, and on that path the server's promote defers channel-context binding to UVM (which the ladder never does), so the engine ctxsw never becomes usable. The real golden channel runs on the PTETABLE_HEAP_MANAGED golden VAS (client-owned page tables, not externally owned), which takes the full client-managed commit path. golden9a/b only failed because that VAS had no page directory: install the ladder root PDB on the golden VAS via DMA_SET_PAGE_DIRECTORY before the channel alloc, and bind the golden channel to it.
* vgsp: golden9e - all ctx buffers FBMEM and every promoted VA PDB-mappedArthur Hanson Rasmusson6 days
| | | | | | | | | | | | | | | The golden9c cycle reached the server context init and then the GR ctxsw timed out (FECS_A/B TDR) with host-sysmem MAIN/PATCH/FECS/PRIV/UNR and unmapped promoted VAs; the golden9d FBMEM test ran on a boot left dirty by golden9c and its 0x1f is not attributable. The real golden wire carried FBMEM (physAttr=4) for every ctx buffer, so: - allocate PATCH/FECS/PRIV/UNR as server FBMEM (PATCH sized by the engine geometry query) in gr_ctx_prepare before the PDB build (mode 25 only); - map PATCH/FECS/PRIV/UNR at their promoted VAs with VIDMEM PTEs in the same 2 MB window as MAIN; - move the FECS/PRIV/UNR and VA-only 3D global VAs into that window and give the PA-backed entries PHYS_ATTR_FB. golden9d's FBMEM channel memdescs are kept.
* vgsp: golden9d - FBMEM channel memdescs and FBMEM MAINArthur Hanson Rasmusson6 days
| | | | | | | | | | | The golden9c cycle reached the promote commit and hung the GR engine in the ctxsw wait (FECS_A/B TDR records then a fatal assert). The real golden capture carried client-local FBMEM instance/USERD/RAMFC/mthd memdescs and FBMEM ctx buffers; the host-sysmem substitution is the remaining aperture difference. Allocate one FBMEM block for the four channel memdescs (sliced 0x1000/0x200/ 0x1000/0x1000 like the client memmgr) and back the promoted MAIN with the server FBMEM allocation (VIDMEM physAttr 0x4, VIDMEM PDB leaf PTEs) for mode 25. PATCH/FECS/PRIV/UNR stay host sysmem in this iteration.
* vgsp: golden9c - bind golden channel to the ladder's proven VASArthur Hanson Rasmusson7 days
| | | | | | | | | | | The captured golden VAS cannot be fully constructed by the ladder: its COPY_SERVER_RESERVED_PDES (0x90f10106) consumes client-RM-reserved page-table level PAs that only exist in the real client's memmgr (and are boot-specific). Bind the golden channel to the ladder's own FERMI_VASPACE_A, which already has server page tables installed by the DMA_SET_PAGE_DIRECTORY + PDB build, so the pVAS validation in kchannelConstruct_IMPL is satisfied. hObjectBuffer stays 0 (host sysmem PB is client-local) and the USERD keep the proven V5 memdesc pairing.
* vgsp: add golden-replay mode 25 (captured client construct + atomic 9-entry ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | promote) Ladder mode 25 reproduces the real 570.144 host client's golden-image construct on the golden replay handles: captured channel shape (flags 0x00200320, hObjectBuffer 0, gpFifoOffset 0, gpFifoEntries 0x20, engineType 1, host memdesc substitutions), a single atomic 9-entry GPU_PROMOTE_CTX on the golden channel (MAIN id0 and PATCH id2 combined, ids 3/4/5/6 VA-only globals, ids 9/10/11 FECS/PRIV/UNR combined), then the TURING_A 3D object. The golden VAS/channel/3D object stay alive and the existing V6 FWCLIENT promote follows on the golden-initialized GR-instance state; a failed golden step aborts with no retry and no follow-up promote. The captured VAS COPY_SERVER_RESERVED_PDES control is implemented gated (fwclient mask bit 8, zero-level no-op form) because its level PAs are server-internal. Mode clamp/name/help extended to 25; modes 1-24 unchanged.
* vgsp: fix host crash in FWCLIENT promote order (MAIN first, non-MAIN after)Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | The 2026-10-05 16:05 crasher sent an accepted non-MAIN PROMOTE_CTX (PM/PATCH/FECS/PRIV/UNR) before a MAIN control that failed 0x40; the writing task then Oopsed in vma_interval_tree_remove with a corrupted VMA rbtree and module list (host kernel memory corruption). Restore the reference order (kgrobjGetPromoteIds_FWCLIENT: MAIN first) and make MAIN-alone the abort point: GPU_INITIALIZE_CTX (mask bit7) then the single-entry MAIN promote; build and send the non-MAIN entries only after MAIN commits (or when MAIN is deselected). The previous accepted-non-MAIN-before-failing-MAIN order is forbidden. Build is unchanged otherwise; mask 0xbf now reproduces the proven-clean mask-0x81 shape when MAIN hits the memdesc gate.
* vgsp: FWCLIENT entry-selection mask and golden-replay hardeningArthur Hanson Rasmusson7 days
| | | | | | | | | | | | - gsp_phys_rmcall_fwclient_mask (default 0x3f): per-entry selection for the mode-24 FWCLIENT promote set; bit6 selects the MAIN UVM VA-only shape. - golden replay: tolerate the client-local system-memory PB step and continue with hObjectBuffer=0; on-ladder TURING_A 3D object retained best-effort. Live bisection on the KMD channel: MAIN alone reaches the per-buffer commit and stops at the engine-context memdesc state; PM-only and PATCH-only are accepted; any multi-entry control containing MAIN is refused 0x1f. All cycles release clean.
* vgsp: GR golden-replay groundwork and FWCLIENT promote mode 24Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | - golden replay (gsp_phys_rmcall_golden, default off): replay the stock client's post-load golden channel allocations (separate VAS, pushbuffer objects, USERD, bare GR channel, TURING_A 3D object) before the ladder channel; best-effort: a failure warns and the ladder continues. - on-ladder TURING_A (0xc597) 3D object allocation before the compute object (best-effort) to initialize server-side GR-instance state. - promote mode 24: the reference FWCLIENT non-3D set (MAIN, PM, PATCH, FECS_EVENT, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP) as one GPU_PROMOTE_CTX on the KMD channel, with per-entry logging and explicit omission semantics; mode clamps/descriptions extended to 24. Live: mode 23 soft-stops 0x1f on the server-managed shared channel and reaches the per-buffer commit on the KMD channel, where it stops at the missing engine-context memdesc; mode 24 emits all six entries and is refused with 0x1f before BIND. All cycles release clean, GPU green.
* vgsp: GR promote mode 23 (hVirtMemory FBMEM server-managed commit shape)Arthur Hanson Rasmusson7 days
| | | | The hVirtMemory no-entry promote backed by the server-owned FBMEM context object (GSP_RM_ALLOC class 0x40) on the server-managed shared V5 channel. A physical memdesc bypasses the server-booked engine-ctx vaList and hands the memdesc straight to the per-buffer commit 0x5b00d2c, the one remaining promote shape on the stable shared+srvctx path. Runs grv23/grv23v6 soft-stop at PROMOTE_CTX hVirtMemory status=0x1f before BIND with clean releases.
* vgsp: srv_ctxcommit mode 2 (V6 client combined init+VA entry)Arthur Hanson Rasmusson7 days
| | | | | | | | | Make gsp_phys_rmcall_srv_ctxcommit an int mode: 1 (default) = map + INITIALIZE_CTX + UVM-shape VA-only promote; 2 = V6 client variant, no INITIALIZE_CTX and one combined PA+VA MAIN entry with bInitialize=1 and bNonmapped=0 so the first ctxsw initializes from the golden image; 0 = pure server-managed. Lets the next supervised cycle bisect both entry shapes without a rebuild. Build clean; not hardware-tested (GPU wedged).
* vgsp: V6 client-owned ctx-buffer VA commit path (srvctx)Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | Source-confirmed: with the SKIP_CTXBUFFER_ALLOC channel flag the server does not allocate MAIN, does not map/book it, and its internal init/promote helpers return early for KMD virtual contexts. The client therefore owns the buffer. When no server MAIN is reported on a V6 channel, map the prepared client MAIN at the ladder ctx VA (sysmem PTEs), send GPU_INITIALIZE_CTX to install the memdesc, then the UVM-shape VA-only promote; with no server pre-booking the vaListSetManaged gate should pass and the per-buffer commit program the engine context pointer. V5 keeps the historical PA-only fallback. Build clean; not hardware-tested (GPU wedged).
* vgsp: GR forwarding static-review fixes (GR/CE slot bit, release VA->PA, ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | | | | | shadow refusal) Five defects found in a static review of the default-off GR forwarding path (no hardware test possible; GPU wedged): - clear the shared request slot's GR mode bit when a CE request is filled, so a prior GR request can never make a later CE push run the GR work function with stale fields; - resolve the QMD release0 guest VA to a guest PA on the service thread (counter pin/derive, else the trusted resolver) and only convert PA->HPA in the work item; the previous code fed the VA to the PA-based translator and would write release0 into an unrelated guest page; - take the refusal path (goto gr_page_unmapped) on every Tier-2 shadow failure instead of continuing to submit a half-built method stream; - consume the QMD run on replay passes when the GR forward is enabled so replayed QMD words are never reinterpreted as CE method headers; - only queue a captured GR candidate against the channel that captured it. Build clean (make -C kernel, W=1 only pre-existing warnings).
* vgsp: guest GR forwarding backend and capture (gsp_phys_gr_forward, default off)Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a default-off guest GR (compute) forwarding path so a guest CUDA launch submitted on its GR channel can be executed by the physical GR engine on the ladder channel, with fail-closed fallback to the host CUDA bridge and the unchanged software release path. - New module parameter gsp_phys_gr_forward (default 0) and accessor. - New GR execution gate: boot_stage.rmcall_gr_exec_ok, latched only after a real GR submission logs REAL GR SEM OK; the backend refuses while it is unset. - Sema capture: a completed 64-word inline QMD on a trusted, non-replay GR decode becomes an svc_gr_cand (push identity, decoded release0, parameter/program/pointer fields, up to 8 captured inline-write records); the CUDA bridge stays authoritative while the parameter is off and is reached in order forward -> bridge -> software release. - Backend: gsp_phys_backend_guest_gr_submit() maps pages at their own guest VAs through fresh Tier-1 small-PT pages/PDE installs or relocates inline-captured pages to ladder shadow pages (Tier 2), builds SET_OBJECT + the inline-QMD method run + REPORT_SEMAPHORE on the compute subchannel, reuses the shared GPFIFO/USERD/doorbell publish tail (extracted from the CE submit path, CE bytes unchanged) and polls the QMD release and/or module semaphore, draining the epoch-gated fault queue on timeout. - Sema wiring: GR requests reuse the INFLIGHT/hold/deferral machinery so the guest channel release applies only on the submit's success/failure edge; hold shims preserve the channel engine type. - Counters and a work_gr state line plus a stats accessor. - Bounds are fixed: 16 pages, 8 inline records, 64 QMD words; every pointer/size checked, pool exhaustion and stale mappings fail closed. - New ABI layouts pinned with static_asserts. Compile-only change: no device test performed. (cherry picked from commit ae507e6f241c7e3498728f9d1ffc2d41ede5cd64)
* vgsp: add gsp_phys_rmcall_channel to force one GR channel variantArthur Hanson Rasmusson7 days
| | | | | | | | | | Lets a bisection pair any promote shape with any channel shape, in particular V6 (SKIP_CTXBUFFER_ALLOC, KMD-managed) with the server-managed VA-only promote: with skip-ctxbuffer the server's kgrctxMapCtxBuffers does not pre-book the engine context vaList, so the VA branch's vaListSetManaged(NV_FALSE) gate should pass and the per-buffer commit can program the engine context VA. Default 0 keeps the mode-selected walk byte-identical. Not hardware-tested (GPU wedged; see Handoff ADDENDUM 51).
* vgsp: GR server-managed context commit experiments (EVICT, VA promote, ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | | | | | | | | external path) Rung-2 bisection after the GET_CTX_BUFFER_INFO ABI fix made the server ctx geometry visible (8 buffers; MAIN 0x133000 FBMEM, PATCH 0x5000): - map the server MAIN/PATCH at the ladder ctx VAs with FBMEM PTEs and send the UVM-shape VA-only promote (bInitialize=0, bNonmapped=0). On the shared VAS the per-entry VA branch calls vaListSetManaged(NV_FALSE) and returns 0x40 because the server pre-booked MAIN as managed; EVICT_CTX (status 0) does not release that booking. - try the same VA promote on the external VAS where the server did not book (kgrctxMapCtxBuffers early-returns): the promote never replies (-110) and leaves the device in the locked fail-safe (WPR2 up, unbound). Do not repeat this shape without a new hypothesis. - FBMEM physAttr selection for the server MAIN fixed (aperture 2 -> 0x4); gsp_phys_rmcall_srv_ctxcommit parameter added (0 = pure server-managed, no client promote). - query prints all 16 entries and records PATCH; query ABI 80-B/5136. Hardware state after the external-VAS run: GPU wedged, reboot required (no automated reboot). All shapes stay default-off behind gr_promote=17 + gsp_phys_srv_ctxbuf.
* vgsp: GR ctx query ABI fix, server-managed ctx map exclusion, raw diag captureArthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | - NV2080_CTRL_GR_CTX_BUFFER_INFO is 80 bytes (NvBool is NvU8: flags at 56..59, uuid at 60) and the params are 5136 bytes, not 5648; the previous ABI mismatch is why GR_GET_CTX_BUFFER_INFO returned NV_ERR_INVALID_ARGUMENT (0x1f) on every shared-VAS cycle. - Under gsp_phys_srv_ctxbuf the GSP-RM server allocates/maps/commits the engine context itself: do not install the client GR context-window PDEs and do not map the client ctx VA in the PDB, so a client mapping cannot clobber the server's own context-window page table. - Dump the server-written INSTANCE/RAMFC chunks after BIND/SCHEDULE on the shared GR path so the committed engine-context pointer/VA can be located. - Emit the full NOCAT diag buffer as 16-byte hex rows (the printable copy is capped at 192 dotted bytes and hides binary fault dumps). Untested on hardware at commit time; the next supervised GR cycle validates.
* vgsp: fix per-type VRAM type lookup and canonical FB-size encodingArthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | The 708a3802 per-type VRAM change stripped an nvidia- prefix from mdev->type->sysfs_name, but sysfs_name is the registered type name verbatim; the mdev core only prefixes the type DIRECTORY with the parent driver string. Every lookup failed, so all mdevs silently fell back to 512 MiB, nvidia-vgpu-turing-1g was unreachable and available_instances capped at 1. Match sysfs_name directly in get_available, show_description and mdev_create. Also canonicalise the NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE encoder (BAR0+0x100ce0) by stripping factors of two from MAG so 512 MiB encodes 0x19 again (1 GiB 0x1a, 2 GiB 0x1b) instead of the non-canonical 0x204. Default-512M behaviour is now byte-equivalent to the known-good pre-VRAM module except max_instances=2. Module sha256 cb1724d3.
* vgsp: per-mdev emulated VRAM size (1.0 multiple mediate sizes)Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | - nv_vgpu_type_info/ioctl gain vram_mb; nv_vgpu_device gains fb_emul_size bound from the mdev type at create (default 512 MiB). - New nvidia-vgpu-turing-1g type (1024 MiB); turing max_instances 1 -> 2; show_description reports the per-type framebuffer and max_instance. - Thread fb_emul_size through GSCI fb_length/region/frtsOffset, WPR2, NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE, the vidmem allocator bound and the BAR1 shadow; bar_info callback uses the per-device size. - UNTESTED at commit time: builds clean; validation blocked by the Xid 140 incident (reboot pending). See Handoff-vGPU-IO.md ADDENDUM 48.
* vgsp: GR shared path ctx-window PDE install and PATCH+MAIN PA-only promote ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | fallback - UPDATE_PDE_2 now also installs the GR context window PT pages (PDE 17+). - The shared-path fallback promotes PATCH then MAIN PA-only (mode-13 order, the real client's set); the mapped (bNonmapped=0) entry still hits the known 0x40 structural gate and is not sent. - Shared+srvctx: channel/BIND/schedule/token/GPFIFO fetch all green; the remaining failure is ROBUST_CHANNEL_GR_FAULT_DURING_CTXSW (0x2c).
* vgsp: GR ladder shared-VAS page-table install and server-managed ctx commit pathArthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | - DMA_UPDATE_PDE_2 client control installs the ladder leaf PT through the server's own MMU walker on the shared-management VAS (the raw-PDB entries are not preserved by the root migration); FLUSH_PDE_CACHE + post-install refill + PT diagnostics. - srvctx registry profile also disables split VAS management; MAP_MEMORY_DMA is not served by GSP-RM for this client (NV_ERR_INVALID_FUNCTION), so the PDE control replaces it. - shared-path context commit helper: GPU_INITIALIZE_CTX + PA-only GPU_PROMOTE_CTX for the server-reported (or client fallback) MAIN buffer; GR_GET_CTX_BUFFER_INFO sends bufferCount=64. - Result: shared VAS + srvctx no longer wedges; channel/BIND/schedule/token green, PBDMA fetches the GPFIFO (GPGet=1); remaining wall is ROBUST_CHANNEL_GR_FAULT_DURING_CTXSW (0x2c).
* vgsp: hold every unreadable CE entry for the grace; delete the silent ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | pre-rescue fake An unreadable CE entry whose channel already had a decoded release VA was faked immediately and consumed with no log line, before any rescue attempt. A later pin-batch rewind then executed the skipped copy against a staging buffer the client had since refilled - the deterministic vadd input shift seen in runs 94/122/151/173b/215b, each with a silent pre-rescue fake. Now every unreadable CE entry with an expected payload is held for the bounded grace and the rescue is retried on each pass, so a translation that is still coming wins and the work runs before the release. Only an entry whose release was already delivered (the channel moved past entry_want) is consumed at once; the post-rescue fake remains the bounded, visible escape. The CE grace default moves 15000 -> 25000 ms (a translation arrived ~17 s after first examination in 215b). Under live campaign 229-236 (module 97eba5b7; green so far).
* vgsp: widen the CE pending-entry grace to 15 sArthur Hanson Rasmusson7 days
| | | | | | | | | | | A consumed entry's work is never run; if its release lands via the fallback before the entry's copies have executed, the client refills the staging slot the copies then read (observed as the deterministic one-chunk vadd input shift). Keep untranslatable CE entries pending longer so their copies run before any release reaches the client. 8-cycle pilot (141-148): zero corruption; a red-fingerprint cycle (held=1165, ring_auto=6) healed to ring_auto=26 with faked=0 and the exact checksum, where the 8 s window recorded 1 corruption in 36.
* vgsp: never run work for a slot consumed without it (epoch mark + ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | content-hash once gate) The grace-expiry fake path consumed a held CE entry without marking its GPFIFO slot as consumed for the put epoch, and the replay guard only ran while the channel was not deferring. A later pin-batch rewind then re-read the slot, found no recorded identity, and executed the push's copies against a staging buffer the client had refilled after the faked release - a deterministic vadd input shift (a[k] == h_a[k + 2 MB]). - grace-fake consume path now calls svc_replay_note() before advancing; - the svc_replay_seen() replay skip is evaluated unconditionally: a slot consumed in the current epoch is skipped on every replay, deferral or not, until a put wrap clears the mark; - the once-only identity gate gains a content witness (FNV-1a hash of the recorded push bytes): the same entry words with different push content is a new push (clients rewrite push buffers in place) and runs its ops with a cleared done bitmap; suppression requires identical content; - snapshot write-back persists the hash array; new drain counter reident= counts content-mismatch cancellations; - the grace window stays at 8 s: a faked entry's work is never run, so the hold is that work's only chance (a 2.5 s experiment made red-class runs complete with corrupt data instead of timing out). Verified: module 8e90fc7e..., three consecutive end-to-end acceptance runs (MAIL + VADD bad=0 checksum=392344090.500000 + STATUS OK).
* vgsp: per-op once-only execution bitmap for retried GPFIFO entriesArthur Hanson Rasmusson7 days
| | | | | | The entry-identity gate suppresses every operation on a replayed entry, but an entry held because one operand's targeted scan was still running is decoded again on retry, and the ops that already succeeded would re-run against a staging buffer the client has since refilled (observed: a retried HtoD wrote the second vadd input half into the first). Track a per-slot bitmap of the operations (0..63) already executed for the slot's current entry identity, plus a complete flag for the pass that finished with no operand pending. On replay, a complete entry suppresses all ops; otherwise the bitmap decides per operation. Successful fills/copies set their bit, and the skipped count is published as ops_once in the drain line.
* vgsp: end-to-end CUDA acceptance in the guestArthur Hanson Rasmusson8 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The no-claim emulation path now runs a stock CUDA program end to end: device properties, cudaMalloc, a GPU-authored string kernel read back byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd executed on the physical GPU through the host bridge, and the device-to-host copy-back verified element by element. Three consecutive supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 + STATUS OK with identical input/output hashes. Two data-path correctness fixes make the transfers sound: - Once-only op execution per GPFIFO entry. The pump replays recently consumed ring entries when translation batches are injected so missed releases can land; re-executing a CE copy is not idempotent because its source is a client staging buffer that may have been refilled. Slots now record their raw entry words and a replayed entry matching that identity suppresses operations while still processing releases. - Per-page translation of virtual operands. svc_copy resolved only the first page of a VA range and walked the PA linearly; client staging ranges are frequently split (observed 195 + 61 pages), so the tail of alternating chunks was read from unrelated memory. Multi-page copies now re-resolve each 4 KB page through the trusted resolver. Also carries the earlier uncommitted emulation fixes that got the client to the launch (QMD address-window gate, counter registration persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon INCHECK/OUTCHECK/dumps).
* vgsp: host CUDA bridge + sema release recovery; greeneye simulator waveArthur Hanson Rasmusson9 days
| | | | | | | | - kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean - tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests - tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode - debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding - STATE-CURRENT/docs updates
* kernel: capture the guest's PROMOTE_CTX entries; add promote mode 19 ↵Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | (guest-faithful 9-entry set) - gsp_rm_emulation: dump every guest GPU_PROMOTE_CTX (0x2080012b) params and entries so the real client's promote shape is observable (captured on the emulation path: MAIN 2 MB PA+VA, PATCH, BUNDLE_CB/PAGEPOOL/ATTRIBUTE_CB/ RTV_CB_GLOBAL VA-only, FECS, PRIV, UNRESTRICTED) - gsp_phys_booter: promote mode 19 replays that exact 9-entry shape on the V5 channel with host buffers (MAIN 2 MB + PDB map, small buffers, booking-only VAs); clamp 0..19 - docs/eng-ops: mode-19 result recorded - the server never replied (fn76 -110) and the release fail-safe held WPR2/GPU until an approved reboot; mode 19 is known-wedging
* kernel: GR promote mode 18 (client FERMI_CONTEXT_SHARE_A via hContextShare)Arthur Hanson Rasmusson10 days
| | | | | | | | | | | - gsp_phys_booter: mode 18 allocates a FERMI_CONTEXT_SHARE_A (0x9067) on the TSG with the ladder VAS and carries it as hContextShare on the V6 KMD channel, so the share construct can allocate the GraphicsSubcontext the single-MAIN promote needs; clamp 0..18 - docs/eng-ops: GR promote bisection row updated with the session-2 results (modes 12-18: PA-only accepted but no ENG_CTX_VALID; PATCH/PM/FBMEM and server-managed registry all leave the KMD promote at 0x40; wall is the commit/RAMFC path)
* kernel: GR context-promote bisection modes 4-17 and server-managed-ctxbuf ↵Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | registry variant; complete vgsp rename; docs/tooling refresh - gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA, batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel, old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only, PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM geometry queries and buffers; pre-boot SET_REGISTRY profile selection - gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0 - vgsp rename across kernel, scripts, tools and docs (module vgsp.ko, /sys/kernel/debug/vgsp/, vgsp-<subsystem>: log prefixes) - risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag tool updates; README/SBOM/install/Makefile sync Live status: CE ladder and P6 guest-copy forwarding green; GR engine context commit still open (all promote shapes accepted or cleanly refused; the channel stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).
* gsp-phys: GR engine context allocate/map/promote before the GR sem push ↵Arthur Hanson Rasmusson10 days
| | | | (default off; live-unverified at promote)
* gsp-phys: relay verify hardening and counter exposure (default off)Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | - the work_relay state line now reports the real backend data-compare verify_fail plus a verify_transient count, distinct from the sema-service hold re-read verify_fail (previously the backend counter was dropped) - a relayed copy whose host-side compare fails is re-read and re-compared once after a short delay; a transient mismatch counts as success and only a persistent mismatch falls back to the software apply as before - the final-mismatch diagnostic now logs the semaphore target/payload/poll time, the first differing byte and 16-byte dumps of live dst, live src and a submit-time analysis material, flagging source mutation vs snapshot/dst - the fault no_event path logs the post reason/return All changes are behind the default-off relay path.
* gsp-phys: H1 GR ladder - GR0 channel, compute object, GR sem push (default off)Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | New load-time gsp_phys_rmcall_engine (0=COPY0 default, 1=GR0) parameterizes the physical ladder: TSG and channel engine_type GR0, one-attempt channel alloc, TURING_COMPUTE_A engine object (class 0xc5c0, handle 0xcaf0002c), BIND engineType=1, and a GR method push (SET_OBJECT 0xc5c0 on subchannel 1 plus SET_REPORT_SEMAPHORE) submitted through the existing GPFIFO/USERD doorbell tail, polling the module semaphore. CE work-forwarding refuses cleanly while a GR ladder is active. A debugfs trigger re-runs the GR push. With the parameter at its default the ladder emits byte-identical COPY0 requests (only the state line gains engine=COPY0). Live (first GR context on the live server): every GR RPC accepted - channel variant V5, TURING_COMPUTE_A alloc status=0x0 -> hObject=0xcaf0002c, BIND status=0x0, token=0x1 - and the engine fetched the push (GPGet 0->1), but raised a GR exception instead of executing the release: exceptType=0x20, NOCAT faultingEngine="GR", intr1=0x80000000, no rpc_result errors. The failure is downstream of all RPCs; no mutating retry was made and the device released cleanly (WPR2 down, rebound, 5 locks, pstore empty, same boot). A default-engine control cycle stayed green (REAL COPY OK). Follow-up: bisect the GR alloc params, the release method D word, and the method order/context init using the drained GR fault records as the oracle.
* gsp-phys: P6c C3 - relay forwarded-engine faults to the guest (default off)Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | New default-off gsp_phys_fault_relay: when a relayed copy times out, the GSP-RM status queue is drained and the first OS_ERROR_LOG (0x1006) recorded after the submission's doorbell is decoded (NOCAT corroboration), matched against the ladder channel's runlist/chid, and delivered to the guest by writing its channel notifier (info32=exceptType, info16=engine type, status=0xffff) and posting the channel event. A late-release recheck skips delivery if the engine actually completed, one delivery per hold, and the existing software fallback always runs first, so no release or data is lost. - pure decode helpers extracted from the status printers; prints unchanged - status-queue walks serialized with a mutex; unmatched 0x1006 records are no longer silently dropped - fault epoch snapshot at the doorbell prevents stale records from matching - fault counters and faults:/fault_relay: state lines - default-off gsp_phys_fault_inject debug knob forces the next relayed pair's source PTE to zero to produce a real engine fault - pump notifier latch prevents clobbering a delivered fault Live: injection run produced a real CE0 FAULT_PTE (0x1f @0x200000), decoded with errString, chid/runlist matched, notifier delivered, software fallback applied (applied+fallback==engaged), clean release; the guest RM reacted to the delivered fault by tearing down its channels (faithful semantics). Healthy control: faults all zero, relay requested=40 ok=40, smoke matched=64/64, clean release. Caveat: the auxiliary event post failed (no_event=1, no MSI-X) and the guest teardown means fault injection is not smoke-compatible; the parameter stays off.
* gsp-phys: P6c C0+C1 - engine-authored guest CE semaphore (default off)Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New default-off gsp_phys_relay: for a forwarded guest CE copy whose push ends in a one-word release, the release page is resolved at forward time, mapped on a fresh scratch line, and the real engine writes the payload directly into the guest-visible semaphore page; the deferred-release hold then verifies that page and skips only that record's software apply. Any failure (unresolvable target, refused kind, pool exhaustion, poll timeout, data mismatch, channel teardown) falls back to applying every release in software, so a release is never lost or duplicated. - new exported gsp_phys_backend_guest_ce_copy_ex() + relay counters; gsp_phys_backend_guest_ce_copy() behavior unchanged (wrapper) - new work_relay: counters line in gsp-phys-state - relay limited to CE channels and one-word absolute releases; counter and reduction releases stay on the existing paths Live: param-off regression matched the wave-2 baseline exactly (req=101 ok=101 verify_ok=101, copy_skipped=119, smoke matched=64/64, relay zeros). With the parameter on: requested=68 ok=67 skipped_sw=67, verify_fail=0, resolve_fail=0, timeouts=0, smoke 64/64, and the engine-authored guest semaphore was observed directly (sem=0x00000003 / 0x0000007e at the guest page). One copy in 68 released its semaphore but failed the host-side data compare; it fell back to the software apply (fallback accounting applied+fallback==engaged) with the smoke green - residual to triage, which is why the parameter stays default-off. A forced-unresolvable run (requested=103, fallback=105, no relay submissions) kept the smoke green. Every cycle: 5 power-domain locks held, driver rebound, WPR2 down, no rail cycles, pstore empty, same boot.
* gsp-phys: P6a debug triggers resolve the guest page HPA at trigger timeArthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | Triggers 6 (W2G) and 7 (G2H) previously used page-table slots pre-built once during the ladder's stage 3. A guest page's host physical address can change between translations (observed 0x13a591000 -> 0x3b8318000 for the same GPA), and a runtime PTE rewrite into an already-fetched 128-byte line is invisible (no invalidate path), so the later write-direction trigger validated against the wrong page and failed its data check. Both triggers now translate at trigger time and run through the same fresh-scratch-line path as forwarding, via a shared internal copy body; the exported forwarding API is unchanged and still requires gsp_phys_work_forward. A changed HPA is now an informational once-per-boot note rather than a failure; the probe-pattern guard still refuses stale pages. Trigger 8 and the stage-3 pre-built slots are unchanged. Live (run p6w3): GUEST G2H COPY OK gpa=0x12423d000 hpa=0x56ab3a000; GUEST W2G COPY OK with verify_ok=1; PTE selftest OK; guest SEEN_W2G; ladder REAL COPY OK; clean latched release (5 power-domain locks, rebound, WPR2 down, no rail cycles, pstore empty, same boot). The full smoke run hit the known guest cuInit/WPR2-already-up wedge at cdt; the trigger path itself is green.
* gsp-phys: P6b waves 1.5 and 2 - deferred release and hardware-authored guest ↵Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | | copies - new default-off gsp_phys_work_defer_release: a forwarded guest push's releases are recorded and its GPFIFO entry is held (no consume, no GP_GET write, no counter publish/stall) until the work item publishes completion; the service thread then delivers the releases, with a fallback on failure/timeout (500 ms cap enforced every pump pass)/ pause/quiesce/channel-delete - new default-off gsp_phys_work_hw_primary: the software copy is skipped for forwarded ops, so the physical engine authors the data; a once-only software copy runs before the releases on any hardware failure or when the request cannot be queued - release-apply helpers extracted; exactly-once latches; channel-delete and teardown handling; counters and a work_guard state line - live: deferred-release run engaged=75 applied=75 timeouts=0 with the full smoke green; hardware-primary run req=100 ok=100 verify_ok=100 fallback=0 copy_skipped=119 copy_fail=0, smoke ALL OK 64/64 (guest read data the engine authored), with 19 fail-closed inline software fallbacks under single-inflight contention; a range-unregistered run proved the fallback (121 skipped copies, all recovered, smoke green) - defaults off; the software path and all existing guards unchanged
* gsp-phys: P6b wave 1 - shadow-forward guest CE copies to the physical engineArthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | - the sema service captures the first eligible copy per push from the software resolver's own operands (the real push shape mixes guest RAM with emulated-VRAM shadow addresses, including physical destinations) - HPA resolution: guest RAM through the shared guest-RAM file page cache, emulated VRAM through the mapping of its kernel shadow - new scratch page-table pool: 16 zeroed page-table pages with their page-directory entries built before any engine access, handing out one whole 128-byte GMMU PTE fetch line per copy pair (16 PTEs, 512 pairs per boot, never reused; a PTE written into an already-fetched line is not observed and there is no invalidate path) - the backend submits the copy on the module-owned channel and verifies the result host-side; every miss falls back to the ladder shadow copy, and the software pump remains authoritative - live (full smoke, both parameters on): req=42 ok=42 verify_ok=42 verify_fail=0 fallback=0, guest payload roundtrip ALL OK 64/64, clean latched release; defaults stay off
* gsp-phys: P6a - real copy engine reads/writes guest RAM pagesArthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | - file-backed guest RAM (launcher memory-backend-file share=on) and in-kernel GPA->HPA translation through the shared file page cache (read_cache_page + page_to_phys + pfn_valid), opened O_LARGEFILE; new default-off gsp_phys_guest_mem parameter - debugfs gsp-phys-guest-ram (path + gpa/off/size ranges) and gsp-phys-guest-gpa (one pinned GPA per boot); rm-call triggers 6/7 run host->guest and guest->host 4 KiB CE copies; trigger 8 is a PTE self-test through a fresh slot - guest page-table slots are pre-built during ladder stage 3: runtime PTE writes into an already-walked page-table line are not observed without a TLB invalidate - guest RAM layout (q35 4.2, 4 GiB): below-4G GPA == file offset, above-4G offset = GPA - 0x80000000 (2 GiB PCI hole) - tests: tools/test/gpage_test.c guest probe; docs + STATE-CURRENT record the live green (G2H/W2G with semaphore release, guest observed the write, smoke ALL OK 64/64, clean latched release)
* docs: add complete SBOM, ignore local deployment resources, remove stale ↵Arthur Hanson Rasmusson10 days
| | | | third-party references
* gsp-phys: freeze-safe release, boot-proof READY, and guest-wiring P2/P3/P5-wave1Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Host-freeze fix bundle (validated 2/2 supervised cycles): - power unlock now latches D0/D3cold by default (new gsp_phys_repower, default 0); releasing the claim can no longer power-gate the board and hard-freeze the host on resume - bounded post-rebind readiness gate before dropping the locks; bounded D0 settle/verify before the unbind - READY means a live GSP-RM session: the async worker and the module-owned join only expose READY when init-done is seen and no unload was sent; otherwise the claim is released fail-open - FWSEC reads DMEM0/IMEM back before STARTCPU and fails fast when the staging is not visible - the claim harness leaves the board latched by default and takes --restore-power to opt back into the old restore Guest-wiring: - P2: generation-keyed fn65 static-config snapshot cache plus a default-off identity overlay (new gsp_phys_static_live) for name/short-name/VBIOS; the probe cycle runs once per backend generation at READY - P3: default-off read-only RM_CONTROL forwarding whitelist (new gsp_phys_ctrl_forward) with emulated-handle translation to the adopted server handles and full emulation fallback - P5 wave 1: default-off shadow execution of guest CE pushes on the ladder channel (new gsp_phys_work_forward) via a dedicated ordered work item; the ladder stage-4 submission is refactored into a shared bounded helper and the software release path stays authoritative Live: module-owned boot to init-done, guest smoke ALL OK 64/64 with P2/P3 enabled, 514 shadow copies observed on the physical CE; all cycles released cleanly with WPR2 down and no reboot.
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-12 10:42:08 +0000