open-iov.git/kernel/gsp_phys_booter.h, branch golden-replay

vgsp: guest GR forwarding backend and capture (gsp_phys_gr_forward, default off)

2026-10-05T17:40:24+00:00

Add a default-off guest GR (compute) forwarding path so a guest CUDA launch submitted on its GR channel can be executed by the physical GR engine on the ladder channel, with fail-closed fallback to the host CUDA bridge and the unchanged software release path. - New module parameter gsp_phys_gr_forward (default 0) and accessor. - New GR execution gate: boot_stage.rmcall_gr_exec_ok, latched only after a real GR submission logs REAL GR SEM OK; the backend refuses while it is unset. - Sema capture: a completed 64-word inline QMD on a trusted, non-replay GR decode becomes an svc_gr_cand (push identity, decoded release0, parameter/program/pointer fields, up to 8 captured inline-write records); the CUDA bridge stays authoritative while the parameter is off and is reached in order forward -> bridge -> software release. - Backend: gsp_phys_backend_guest_gr_submit() maps pages at their own guest VAs through fresh Tier-1 small-PT pages/PDE installs or relocates inline-captured pages to ladder shadow pages (Tier 2), builds SET_OBJECT + the inline-QMD method run + REPORT_SEMAPHORE on the compute subchannel, reuses the shared GPFIFO/USERD/doorbell publish tail (extracted from the CE submit path, CE bytes unchanged) and polls the QMD release and/or module semaphore, draining the epoch-gated fault queue on timeout. - Sema wiring: GR requests reuse the INFLIGHT/hold/deferral machinery so the guest channel release applies only on the submit's success/failure edge; hold shims preserve the channel engine type. - Counters and a work_gr state line plus a stats accessor. - Bounds are fixed: 16 pages, 8 inline records, 64 QMD words; every pointer/size checked, pool exhaustion and stale mappings fail closed. - New ABI layouts pinned with static_asserts. Compile-only change: no device test performed. (cherry picked from commit ae507e6f241c7e3498728f9d1ffc2d41ede5cd64)

kernel: GR context-promote bisection modes 4-17 and server-managed-ctxbuf registry variant; complete vgsp rename; docs/tooling refresh

2026-10-02T19:13:06+00:00

- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA, batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel, old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only, PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM geometry queries and buffers; pre-boot SET_REGISTRY profile selection - gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0 - vgsp rename across kernel, scripts, tools and docs (module vgsp.ko, /sys/kernel/debug/vgsp/, vgsp-: log prefixes) - risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag tool updates; README/SBOM/install/Makefile sync Live status: CE ladder and P6 guest-copy forwarding green; GR engine context commit still open (all promote shapes accepted or cleanly refused; the channel stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).

gsp-phys: GR engine context allocate/map/promote before the GR sem push (default off; live-unverified at promote)

2026-10-02T14:58:17+00:00

gsp-phys: relay verify hardening and counter exposure (default off)

2026-10-02T12:41:17+00:00

- the work_relay state line now reports the real backend data-compare verify_fail plus a verify_transient count, distinct from the sema-service hold re-read verify_fail (previously the backend counter was dropped) - a relayed copy whose host-side compare fails is re-read and re-compared once after a short delay; a transient mismatch counts as success and only a persistent mismatch falls back to the software apply as before - the final-mismatch diagnostic now logs the semaphore target/payload/poll time, the first differing byte and 16-byte dumps of live dst, live src and a submit-time analysis material, flagging source mutation vs snapshot/dst - the fault no_event path logs the post reason/return All changes are behind the default-off relay path.

gsp-phys: H1 GR ladder - GR0 channel, compute object, GR sem push (default off)

2026-10-02T12:29:45+00:00

New load-time gsp_phys_rmcall_engine (0=COPY0 default, 1=GR0) parameterizes the physical ladder: TSG and channel engine_type GR0, one-attempt channel alloc, TURING_COMPUTE_A engine object (class 0xc5c0, handle 0xcaf0002c), BIND engineType=1, and a GR method push (SET_OBJECT 0xc5c0 on subchannel 1 plus SET_REPORT_SEMAPHORE) submitted through the existing GPFIFO/USERD doorbell tail, polling the module semaphore. CE work-forwarding refuses cleanly while a GR ladder is active. A debugfs trigger re-runs the GR push. With the parameter at its default the ladder emits byte-identical COPY0 requests (only the state line gains engine=COPY0). Live (first GR context on the live server): every GR RPC accepted - channel variant V5, TURING_COMPUTE_A alloc status=0x0 -> hObject=0xcaf0002c, BIND status=0x0, token=0x1 - and the engine fetched the push (GPGet 0->1), but raised a GR exception instead of executing the release: exceptType=0x20, NOCAT faultingEngine="GR", intr1=0x80000000, no rpc_result errors. The failure is downstream of all RPCs; no mutating retry was made and the device released cleanly (WPR2 down, rebound, 5 locks, pstore empty, same boot). A default-engine control cycle stayed green (REAL COPY OK). Follow-up: bisect the GR alloc params, the release method D word, and the method order/context init using the drained GR fault records as the oracle.

gsp-phys: P6c C3 - relay forwarded-engine faults to the guest (default off)

2026-10-02T12:07:10+00:00

New default-off gsp_phys_fault_relay: when a relayed copy times out, the GSP-RM status queue is drained and the first OS_ERROR_LOG (0x1006) recorded after the submission's doorbell is decoded (NOCAT corroboration), matched against the ladder channel's runlist/chid, and delivered to the guest by writing its channel notifier (info32=exceptType, info16=engine type, status=0xffff) and posting the channel event. A late-release recheck skips delivery if the engine actually completed, one delivery per hold, and the existing software fallback always runs first, so no release or data is lost. - pure decode helpers extracted from the status printers; prints unchanged - status-queue walks serialized with a mutex; unmatched 0x1006 records are no longer silently dropped - fault epoch snapshot at the doorbell prevents stale records from matching - fault counters and faults:/fault_relay: state lines - default-off gsp_phys_fault_inject debug knob forces the next relayed pair's source PTE to zero to produce a real engine fault - pump notifier latch prevents clobbering a delivered fault Live: injection run produced a real CE0 FAULT_PTE (0x1f @0x200000), decoded with errString, chid/runlist matched, notifier delivered, software fallback applied (applied+fallback==engaged), clean release; the guest RM reacted to the delivered fault by tearing down its channels (faithful semantics). Healthy control: faults all zero, relay requested=40 ok=40, smoke matched=64/64, clean release. Caveat: the auxiliary event post failed (no_event=1, no MSI-X) and the guest teardown means fault injection is not smoke-compatible; the parameter stays off.

gsp-phys: P6c C0+C1 - engine-authored guest CE semaphore (default off)

2026-10-02T11:43:05+00:00

New default-off gsp_phys_relay: for a forwarded guest CE copy whose push ends in a one-word release, the release page is resolved at forward time, mapped on a fresh scratch line, and the real engine writes the payload directly into the guest-visible semaphore page; the deferred-release hold then verifies that page and skips only that record's software apply. Any failure (unresolvable target, refused kind, pool exhaustion, poll timeout, data mismatch, channel teardown) falls back to applying every release in software, so a release is never lost or duplicated. - new exported gsp_phys_backend_guest_ce_copy_ex() + relay counters; gsp_phys_backend_guest_ce_copy() behavior unchanged (wrapper) - new work_relay: counters line in gsp-phys-state - relay limited to CE channels and one-word absolute releases; counter and reduction releases stay on the existing paths Live: param-off regression matched the wave-2 baseline exactly (req=101 ok=101 verify_ok=101, copy_skipped=119, smoke matched=64/64, relay zeros). With the parameter on: requested=68 ok=67 skipped_sw=67, verify_fail=0, resolve_fail=0, timeouts=0, smoke 64/64, and the engine-authored guest semaphore was observed directly (sem=0x00000003 / 0x0000007e at the guest page). One copy in 68 released its semaphore but failed the host-side data compare; it fell back to the software apply (fallback accounting applied+fallback==engaged) with the smoke green - residual to triage, which is why the parameter stays default-off. A forced-unresolvable run (requested=103, fallback=105, no relay submissions) kept the smoke green. Every cycle: 5 power-domain locks held, driver rebound, WPR2 down, no rail cycles, pstore empty, same boot.

gsp-phys: P6a debug triggers resolve the guest page HPA at trigger time

2026-10-02T11:10:28+00:00

Triggers 6 (W2G) and 7 (G2H) previously used page-table slots pre-built once during the ladder's stage 3. A guest page's host physical address can change between translations (observed 0x13a591000 -> 0x3b8318000 for the same GPA), and a runtime PTE rewrite into an already-fetched 128-byte line is invisible (no invalidate path), so the later write-direction trigger validated against the wrong page and failed its data check. Both triggers now translate at trigger time and run through the same fresh-scratch-line path as forwarding, via a shared internal copy body; the exported forwarding API is unchanged and still requires gsp_phys_work_forward. A changed HPA is now an informational once-per-boot note rather than a failure; the probe-pattern guard still refuses stale pages. Trigger 8 and the stage-3 pre-built slots are unchanged. Live (run p6w3): GUEST G2H COPY OK gpa=0x12423d000 hpa=0x56ab3a000; GUEST W2G COPY OK with verify_ok=1; PTE selftest OK; guest SEEN_W2G; ladder REAL COPY OK; clean latched release (5 power-domain locks, rebound, WPR2 down, no rail cycles, pstore empty, same boot). The full smoke run hit the known guest cuInit/WPR2-already-up wedge at cdt; the trigger path itself is green.

gsp-phys: P6b wave 1 - shadow-forward guest CE copies to the physical engine

2026-10-02T09:04:44+00:00

- the sema service captures the first eligible copy per push from the software resolver's own operands (the real push shape mixes guest RAM with emulated-VRAM shadow addresses, including physical destinations) - HPA resolution: guest RAM through the shared guest-RAM file page cache, emulated VRAM through the mapping of its kernel shadow - new scratch page-table pool: 16 zeroed page-table pages with their page-directory entries built before any engine access, handing out one whole 128-byte GMMU PTE fetch line per copy pair (16 PTEs, 512 pairs per boot, never reused; a PTE written into an already-fetched line is not observed and there is no invalidate path) - the backend submits the copy on the module-owned channel and verifies the result host-side; every miss falls back to the ladder shadow copy, and the software pump remains authoritative - live (full smoke, both parameters on): req=42 ok=42 verify_ok=42 verify_fail=0 fallback=0, guest payload roundtrip ALL OK 64/64, clean latched release; defaults stay off

gsp-phys: P6a - real copy engine reads/writes guest RAM pages

2026-10-02T07:31:06+00:00

- file-backed guest RAM (launcher memory-backend-file share=on) and in-kernel GPA->HPA translation through the shared file page cache (read_cache_page + page_to_phys + pfn_valid), opened O_LARGEFILE; new default-off gsp_phys_guest_mem parameter - debugfs gsp-phys-guest-ram (path + gpa/off/size ranges) and gsp-phys-guest-gpa (one pinned GPA per boot); rm-call triggers 6/7 run host->guest and guest->host 4 KiB CE copies; trigger 8 is a PTE self-test through a fresh slot - guest page-table slots are pre-built during ladder stage 3: runtime PTE writes into an already-walked page-table line are not observed without a TLB invalidate - guest RAM layout (q35 4.2, 4 GiB): below-4G GPA == file offset, above-4G offset = GPA - 0x80000000 (2 GiB PCI hole) - tests: tools/test/gpage_test.c guest probe; docs + STATE-CURRENT record the live green (G2H/W2G with semaphore release, guest observed the write, smoke ALL OK 64/64, clean latched release)