vgsp: m181 relocation-page unification - launch-internal pages (param/program/banks) resolve through the landing table; zero-write diagnostics

Audit of the QMD relocation/mirror path vs the m180 landing table (r210
evidence: /tmp dmesg, 0 ledger DIVERGE, four multi-page clients still
wrong, 3 zero-write NO-STORES accepts):

The relocation itself does NOT pick PAs: gsp_phys_gr_rewrite_data_vars()
matches a device-VA-shaped 8-byte value against req->data[x].va (the
fill's gr_data list, which m180 fed through the ledger) and rewrites it to
req->data[x].reloc_va. The bounce page for reloc_va is the same page the
tracking wait and the copy-out use, so a *matched* pointer is ledger-
consistent by construction. The un-unified halves of that path are the
page the relocation READS and the page the m129 mirror PATCHES: both come
from pg->pa, resolved by svc_gr_page_note() with the independent sources
(pin/lat/map/walks/scan), never the landing table. The CE inline writes
that author those launches' param/program/bank pages go through
svc_grce_inline_write(), which (m180) records its landing and diverges to
it; svc_gr_page_note could still hand the win0 shadow copy and the mirror
a stale page. Consequences, both observed classes: relocation matches
nothing (the page holds another session's pointers) -> the engine runs
against unrelocated guest VAs it cannot walk -> grid aborts with zero
stores while the ack still lands; or the engine executes on a previous
client's parameter block -> plausible-but-wrong results invisible to the
ledger.

Fix (kernel/gsp_sema_service.c):
- svc_gr_page_note() consults svc_ledger_lookup() first for every
  launch-internal page. On a hit it uses the recorded landing (and pins
  it, so later resolves and the win0 copy path agree); when the
  independent sources would answer differently it logs (bounded)
  "sema-ledger: RELOC-DIVERGE va=... ledger_pa=... <src>_pa=... (ledger
  wins)" - the requested relocation-vs-ledger witness. Non-hit pages keep
  the historical svc_resolve_va path unchanged.

Diagnostics (kernel/gsp_phys_booter.c):
- the m135 below-0x100 guard now shape-checks first and logs
  "bank N RELOC-SKIP @off: value is operand page ... (left guest-VA)".
  r210 shows the same offset-0 bank pointer in EVERY client (green ones
  included: the driver control block), so the guard is unchanged until a
  dereferenced pointer in that range is proven; the log makes that
  provable in one cycle.
- the zero-write accept now also dumps the deposit QMD words
  (gsp_phys_gr_deposit_rb_log) so the next NO-STORES event shows whether
  the engine consumed the header words, and the dual-bank sentinel line
  is reworded to state facts only (the other bank is normally the
  previous launch's legitimately-consumed deposit; the earlier "older
  deposit consumed" reading was misleading). r210 classification: all
  three NO-STORES events had THIS deposit fetched (sentinel overwritten)
  and 0/968..0/967 stores even at +1 s, i.e. engine-side non-execution of
  a fetched deposit - consistent with the stale-launch-page abort
  mechanism above; the deposit-readback words will discriminate further.

Not changed (deliberate): the m135 guard semantics, the relocation
targets, the sentinel contract, retry/accept policy, and every semaphore
path (mm2/gate/rewind effects untouched).

make kernel -j green (srcversion DAB7E3AA75D364382DD6A3C; only the
pre-existing gsp_phys_rmcall_stage3 printf warning). Main tree untouched;
nothing pushed.

1 files changed, 54 insertions, 28 deletions

diff --git a/kernel/gsp_phys_booter.c b/kernel/gsp_phys_booter.c
index f60f6c5..c5870ce 100644
--- a/kernel/gsp_phys_booter.c
+++ b/kernel/gsp_phys_booter.c
@@ -16755,7 +16755,7 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words,
 					  const struct gsp_phys_gr_submit *req,
 					  u32 bank, u32 *mirror)
 {
-	u32 rew = 0, k;
+	u32 rew = 0, skipped = 0, k;
 
 	if (!req->n_data)
 		return;
@@ -16771,19 +16771,6 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words,
 		u64 wb;
 		u32 x;
 
-		/*
-		 * m135: never relocate below offset 0x100. The param-page
-		 * header (grid/block dims and driver bookkeeping at 0x0..0xff)
-		 * can contain server-region VAs (e.g. 0x100000100) that are
-		 * NOT kernel parameters; relocating them corrupted the
-		 * vadd launch's blockDim word (read as 0x170100 instead of
-		 * 0x100, so every CTA past 0 computed out-of-range indices
-		 * and exited: VADD bad=32512 exactly one block correct).
-		 * Kernel parameters live at >= 0x140 (JIT sm_75 layout:
-		 * a/b/c at 0x160/0x168/0x170, n at 0x178).
-		 */
-		if (k * 4u < 0x100u)
-			continue;
 		if (v < 0x10000000ULL || v >= (1ULL << 47) || (v & 0xff))
 			continue;
 		wb = v & ~0xfffULL;
@@ -16791,6 +16778,34 @@ static void gsp_phys_gr_rewrite_data_vars(u32 *w, u32 n_words,
 			if (req->data[x].va != wb ||
 			    !req->data[x].reloc_va)
 				continue;
+			/*
+			 * m135: never relocate below offset 0x100. The
+			 * param-page header (grid/block dims and driver
+			 * bookkeeping at 0x0..0xff) can contain
+			 * server-region VAs (e.g. 0x100000100) that are NOT
+			 * kernel parameters; relocating them corrupted the
+			 * vadd launch's blockDim word (read as 0x170100
+			 * instead of 0x100, so every CTA past 0 computed
+			 * out-of-range indices and exited: VADD bad=32512
+			 * exactly one block correct). Kernel parameters live
+			 * at >= 0x140 (JIT sm_75 layout: a/b/c at
+			 * 0x160/0x168/0x170, n at 0x178).
+			 *
+			 * m181 diagnostic: a below-0x100 value whose page is
+			 * an operand VA of this launch is still logged (the
+			 * observed class is the driver control block at
+			 * bank offset 0, present in green clients too); the
+			 * guard itself is unchanged until a dereferenced
+			 * pointer in that range is proven.
+			 */
+			if (k * 4u < 0x100u) {
+				if (skipped < 16) {
+					skipped++;
+					pr_info("vgsp-gr: bank %u RELOC-SKIP @0x%x: 0x%llx is operand page 0x%llx (below 0x100, left guest-VA)\n",
+						bank, k * 4, v, wb);
+				}
+				break;
+			}
 			/* m120: log the hit site so coverage is provable. */
 			if (rew < 16)
 				pr_info("vgsp-gr: bank %u hit @0x%x: 0x%llx -> 0x%llx\n",
@@ -19249,16 +19264,16 @@ gr_page_unmapped:
 				fetched ? "OK" : "NONE");
 
 			/*
-			 * m180 (lottery discriminators, agent-order H1/H2).
-			 *
-			 * (a) Deposit identity without changing the sentinel
-			 * contract: read BOTH qmdwin banks' sentinel words.
-			 * Ours intact + the other bank overwritten means an
-			 * OLDER deposit was consumed instead of this one
-			 * (stale/mis-targeted deposit, H2); ours overwritten
-			 * means the engine really fetched THIS deposit
-			 * (lost-dispatch vs read-only kernel, H1); both
-			 * intact means no deposit ever reached the engine.
+			 * m180 (lottery discriminator, agent-order H1/H2).
+			 * Read BOTH qmdwin banks' sentinel words. Facts only:
+			 * the other bank is normally the previous launch's
+			 * deposit (legitimately consumed), so only THIS
+			 * launch's sentinel classifies the event: intact =
+			 * this deposit never reached the engine; overwritten
+			 * = the engine consumed it (or something zeroed the
+			 * word), so the zero stores are an engine-side
+			 * non-execution of a fetched deposit, not a lost
+			 * deposit.
 			 */
 			if (qmdwin_on && boot_stage.rmcall_gr_qmdwin.pages &&
 			    qmd_dst_mem) {
@@ -19266,18 +19281,29 @@ gr_page_unmapped:
 					&boot_stage.rmcall_gr_qmdwin,
 					qw_bank ^ 1u);
 
-				pr_warn("vgsp-gr: zero-write deposit stamp: bank=%u sentinel=0x%08x other-bank=%u sentinel=%s\n",
+				pr_warn("vgsp-gr: zero-write deposit stamp: bank=%u sentinel=0x%08x (%s) other-bank=%u %s\n",
 					qw_bank,
 					get_unaligned_le32(
 						qmd_dst_mem +
 						GSP_PHYS_GR_QMD_SENTINEL_WORD * 4),
+					fetched ? "this deposit fetched" :
+						  "this deposit NOT fetched",
 					qw_bank ^ 1u,
 					om ? (gsp_phys_gr_qmd_fetched(om) ?
-					      "overwritten (older deposit consumed)" :
-					      "intact") : "n/a");
+					      "sentinel overwritten (previous deposit)" :
+					      "sentinel intact") : "n/a");
 			}
 			/*
-			 * (b) Late-store check: a store ramp that retires after
+			 * (b) Deposit readback at the accept (m181): the same
+			 * QMD-word dump the first-timeout path logs, so a
+			 * zero-write launch's deposit content (w0/w4/w12/.../
+			 * w63) shows whether the engine consumed the header
+			 * words or the page is untouched, without needing a
+			 * timeout to have happened first.
+			 */
+			gsp_phys_gr_deposit_rb_log(req, qmd_dst_mem);
+			/*
+			 * (c) Late-store check: a store ramp that retires after
 			 * the host's window is indistinguishable from a lost
 			 * dispatch at the accept moment. Re-scan once at +1 s
 			 * and report whether the tracked pages changed in the