From 6f7afe700f0919da50533568d48a44d5d158adb7 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Sun, 18 Oct 2026 03:15:35 -0400 Subject: vgsp: m181 relocation-page unification - launch-internal pages (param/program/banks) resolve through the landing table; zero-write diagnostics Audit of the QMD relocation/mirror path vs the m180 landing table (r210 evidence: /tmp dmesg, 0 ledger DIVERGE, four multi-page clients still wrong, 3 zero-write NO-STORES accepts): The relocation itself does NOT pick PAs: gsp_phys_gr_rewrite_data_vars() matches a device-VA-shaped 8-byte value against req->data[x].va (the fill's gr_data list, which m180 fed through the ledger) and rewrites it to req->data[x].reloc_va. The bounce page for reloc_va is the same page the tracking wait and the copy-out use, so a *matched* pointer is ledger- consistent by construction. The un-unified halves of that path are the page the relocation READS and the page the m129 mirror PATCHES: both come from pg->pa, resolved by svc_gr_page_note() with the independent sources (pin/lat/map/walks/scan), never the landing table. The CE inline writes that author those launches' param/program/bank pages go through svc_grce_inline_write(), which (m180) records its landing and diverges to it; svc_gr_page_note could still hand the win0 shadow copy and the mirror a stale page. Consequences, both observed classes: relocation matches nothing (the page holds another session's pointers) -> the engine runs against unrelocated guest VAs it cannot walk -> grid aborts with zero stores while the ack still lands; or the engine executes on a previous client's parameter block -> plausible-but-wrong results invisible to the ledger. Fix (kernel/gsp_sema_service.c): - svc_gr_page_note() consults svc_ledger_lookup() first for every launch-internal page. On a hit it uses the recorded landing (and pins it, so later resolves and the win0 copy path agree); when the independent sources would answer differently it logs (bounded) "sema-ledger: RELOC-DIVERGE va=... ledger_pa=... _pa=... (ledger wins)" - the requested relocation-vs-ledger witness. Non-hit pages keep the historical svc_resolve_va path unchanged. Diagnostics (kernel/gsp_phys_booter.c): - the m135 below-0x100 guard now shape-checks first and logs "bank N RELOC-SKIP @off: value is operand page ... (left guest-VA)". r210 shows the same offset-0 bank pointer in EVERY client (green ones included: the driver control block), so the guard is unchanged until a dereferenced pointer in that range is proven; the log makes that provable in one cycle. - the zero-write accept now also dumps the deposit QMD words (gsp_phys_gr_deposit_rb_log) so the next NO-STORES event shows whether the engine consumed the header words, and the dual-bank sentinel line is reworded to state facts only (the other bank is normally the previous launch's legitimately-consumed deposit; the earlier "older deposit consumed" reading was misleading). r210 classification: all three NO-STORES events had THIS deposit fetched (sentinel overwritten) and 0/968..0/967 stores even at +1 s, i.e. engine-side non-execution of a fetched deposit - consistent with the stale-launch-page abort mechanism above; the deposit-readback words will discriminate further. Not changed (deliberate): the m135 guard semantics, the relocation targets, the sentinel contract, retry/accept policy, and every semaphore path (mm2/gate/rewind effects untouched). make kernel -j green (srcversion DAB7E3AA75D364382DD6A3C; only the pre-existing gsp_phys_rmcall_stage3 printf warning). Main tree untouched; nothing pushed. --- kernel/gsp_phys_booter.c | 82 +++++++++++++++++++++++++++++++----------------- 1 file changed, 54 insertions(+), 28 deletions(-) (limited to 'kernel/gsp_phys_booter.c') diff --git a/kernel/gsp_phys_booter.c b/kernel/gsp_phys_booter.c index f60f6c5..c5870ce 100644 --- a/kernel/gsp_phys_booter.c +++ b/kernel/gsp_phys_booter.c @@ -16755,7 +16755,7 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words, const struct gsp_phys_gr_submit *req, u32 bank, u32 *mirror) { - u32 rew = 0, k; + u32 rew = 0, skipped = 0, k; if (!req->n_data) return; @@ -16771,19 +16771,6 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words, u64 wb; u32 x; - /* - * m135: never relocate below offset 0x100. The param-page - * header (grid/block dims and driver bookkeeping at 0x0..0xff) - * can contain server-region VAs (e.g. 0x100000100) that are - * NOT kernel parameters; relocating them corrupted the - * vadd launch's blockDim word (read as 0x170100 instead of - * 0x100, so every CTA past 0 computed out-of-range indices - * and exited: VADD bad=32512 exactly one block correct). - * Kernel parameters live at >= 0x140 (JIT sm_75 layout: - * a/b/c at 0x160/0x168/0x170, n at 0x178). - */ - if (k * 4u < 0x100u) - continue; if (v < 0x10000000ULL || v >= (1ULL << 47) || (v & 0xff)) continue; wb = v & ~0xfffULL; @@ -16791,6 +16778,34 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words, if (req->data[x].va != wb || !req->data[x].reloc_va) continue; + /* + * m135: never relocate below offset 0x100. The + * param-page header (grid/block dims and driver + * bookkeeping at 0x0..0xff) can contain + * server-region VAs (e.g. 0x100000100) that are NOT + * kernel parameters; relocating them corrupted the + * vadd launch's blockDim word (read as 0x170100 + * instead of 0x100, so every CTA past 0 computed + * out-of-range indices and exited: VADD bad=32512 + * exactly one block correct). Kernel parameters live + * at >= 0x140 (JIT sm_75 layout: a/b/c at + * 0x160/0x168/0x170, n at 0x178). + * + * m181 diagnostic: a below-0x100 value whose page is + * an operand VA of this launch is still logged (the + * observed class is the driver control block at + * bank offset 0, present in green clients too); the + * guard itself is unchanged until a dereferenced + * pointer in that range is proven. + */ + if (k * 4u < 0x100u) { + if (skipped < 16) { + skipped++; + pr_info("vgsp-gr: bank %u RELOC-SKIP @0x%x: 0x%llx is operand page 0x%llx (below 0x100, left guest-VA)\n", + bank, k * 4, v, wb); + } + break; + } /* m120: log the hit site so coverage is provable. */ if (rew < 16) pr_info("vgsp-gr: bank %u hit @0x%x: 0x%llx -> 0x%llx\n", @@ -19249,16 +19264,16 @@ gr_page_unmapped: fetched ? "OK" : "NONE"); /* - * m180 (lottery discriminators, agent-order H1/H2). - * - * (a) Deposit identity without changing the sentinel - * contract: read BOTH qmdwin banks' sentinel words. - * Ours intact + the other bank overwritten means an - * OLDER deposit was consumed instead of this one - * (stale/mis-targeted deposit, H2); ours overwritten - * means the engine really fetched THIS deposit - * (lost-dispatch vs read-only kernel, H1); both - * intact means no deposit ever reached the engine. + * m180 (lottery discriminator, agent-order H1/H2). + * Read BOTH qmdwin banks' sentinel words. Facts only: + * the other bank is normally the previous launch's + * deposit (legitimately consumed), so only THIS + * launch's sentinel classifies the event: intact = + * this deposit never reached the engine; overwritten + * = the engine consumed it (or something zeroed the + * word), so the zero stores are an engine-side + * non-execution of a fetched deposit, not a lost + * deposit. */ if (qmdwin_on && boot_stage.rmcall_gr_qmdwin.pages && qmd_dst_mem) { @@ -19266,18 +19281,29 @@ gr_page_unmapped: &boot_stage.rmcall_gr_qmdwin, qw_bank ^ 1u); - pr_warn("vgsp-gr: zero-write deposit stamp: bank=%u sentinel=0x%08x other-bank=%u sentinel=%s\n", + pr_warn("vgsp-gr: zero-write deposit stamp: bank=%u sentinel=0x%08x (%s) other-bank=%u %s\n", qw_bank, get_unaligned_le32( qmd_dst_mem + GSP_PHYS_GR_QMD_SENTINEL_WORD * 4), + fetched ? "this deposit fetched" : + "this deposit NOT fetched", qw_bank ^ 1u, om ? (gsp_phys_gr_qmd_fetched(om) ? - "overwritten (older deposit consumed)" : - "intact") : "n/a"); + "sentinel overwritten (previous deposit)" : + "sentinel intact") : "n/a"); } /* - * (b) Late-store check: a store ramp that retires after + * (b) Deposit readback at the accept (m181): the same + * QMD-word dump the first-timeout path logs, so a + * zero-write launch's deposit content (w0/w4/w12/.../ + * w63) shows whether the engine consumed the header + * words or the page is untouched, without needing a + * timeout to have happened first. + */ + gsp_phys_gr_deposit_rb_log(req, qmd_dst_mem); + /* + * (c) Late-store check: a store ramp that retires after * the host's window is indistinguishable from a lost * dispatch at the accept moment. Re-scan once at +1 s * and report whether the tracked pages changed in the -- cgit v1.2.3