gr/sema: r420 pause completion deadlines under the backend wedge-quiet hold

The r419 soak reproduced the user's mid-boot degradation with the wedge fix
in tree: victory_matmul printed bad=65536 while its launch was held by the
booter's WEDGE-QUIET window - the client was released ~115 s before the
post-hold accept. Root cause: two sema-service deadlines are shorter than
the deliberate quiet hold and fire under it, authoring the launch's
completion release (and applying its recorded releases) while the engine has
not run and the copy-out has not happened:

  - the 12 s GR release hold (SVC_FWD_GR_HOLD_TIMEOUT_MS) expiry
    (GR-STALL-fwd-hold ... releases + QMD word software-applied), and
  - the 3 s per-launch consumption bound (gsp_phys_gr_hold_max_ms) firing
    the busy-defer consume-with-software-release path, whose inline apply
    authors the capture's QMD release (GR-REL-AUTHOR src=hold).

Evidence (r419 live): push be94 captured 131333.687, WEDGE-QUIET holds
40 s + 80 s until 131345.598, accept 131346.552 - while the sema hold
expired at 131230.978 (GR-STALL-fwd-hold waited_ms=12032) and the inline
apply authored va=0x204027f7c pay=6 at 131341.734 (h=0x5c00001f). The
client's verify completed on the authored word and read stale data.

Fix: the booter publishes gsp_phys_gr_quiet_hold_active() exactly around
the quiet msleep (bounded by construction). The sema service pauses under
it: svc_fwd_hold_expired() pushes the GR deadline (bounded by a 600 s
defence-in-depth cap, announced once per hold as GR-HOLD-PAUSE);
svc_gr_hold_bound_expired() pushes the per-launch bound
(GR-HOLD-BOUND-PAUSE); and the busy-defer one-shot window re-holds while
the flag is set so the consume-with-sw-release path cannot run during a
quiet hold. The held launch is then forwarded and copied out after the
hold, and its completion is authored only on the hold's done edge - after
the data write-back.

1 files changed, 38 insertions, 0 deletions

diff --git a/kernel/gsp_phys_booter.c b/kernel/gsp_phys_booter.c
index d20e042..f7ad153 100644
--- a/kernel/gsp_phys_booter.c
+++ b/kernel/gsp_phys_booter.c
@@ -16324,6 +16324,29 @@ static u64 gsp_phys_gr_wedge_heals;
  * either heals (engine writes, client data correct) or is loudly classified.
  */
 static bool gsp_phys_gr_wedge_episode;
+/*
+ * r420: true exactly while a WEDGE-QUIET hold sleeps with NO submission
+ * outstanding. The hold is a deliberate, bounded wait for the sticky wedge to
+ * clear (r410/r411); during it the in-flight forwarded request is NOT done.
+ * The sema service's completion deadlines - the 12 s GR release hold
+ * (SVC_FWD_GR_HOLD_TIMEOUT_MS) and the 3 s per-launch consumption bound
+ * (gsp_phys_gr_hold_max_ms) - are shorter than this hold, so without the
+ * publish they fire UNDER it and author the launch's release word before the
+ * post-hold copy-out: the client then verifies stale data (r419 soak:
+ * `victory_matmul bad=65536` while its launch was held; the authoring paths
+ * are GR-STALL-fwd-hold / the busy-defer inline software release). The
+ * accessor lets both deadlines pause instead; the flag is bounded by the
+ * msleep itself and by the pause caps on the sema side. Single-inflight
+ * submit path, so a plain static.
+ */
+static bool gsp_phys_gr_quiet_hold_on;
+
+bool gsp_phys_gr_quiet_hold_active(void)
+{
+	return READ_ONCE(gsp_phys_gr_quiet_hold_on);
+}
+EXPORT_SYMBOL_GPL(gsp_phys_gr_quiet_hold_active);
+
 /* m86: program-page override blob (see gsp-phys-gr-override). */
 static u8 gsp_phys_gr_override[GSP_PHYS_PAGE_SIZE];
 static u32 gsp_phys_gr_override_len;
@@ -20579,7 +20602,22 @@ gr_page_unmapped:
 						quiet_ms, nw_attempts,
 						req->push_va,
 						READ_ONCE(gsp_phys_gr_nw_streak));
+					/*
+					 * r420: publish the deliberate hold so
+					 * the sema service's completion
+					 * deadlines (the 12 s GR release hold
+					 * and the 3 s per-launch consumption
+					 * bound) pause instead of authoring
+					 * this launch's release word under it -
+					 * the client must not be released
+					 * before the post-hold copy-out
+					 * (gsp_phys_gr_quiet_hold_active).
+					 */
+					WRITE_ONCE(gsp_phys_gr_quiet_hold_on,
+						   true);
 					msleep(quiet_ms);
+					WRITE_ONCE(gsp_phys_gr_quiet_hold_on,
+						   false);
 					quiet_used = quiet_ms;
 					pr_warn("vgsp-gr: WEDGE-QUIET hold done (push=0x%llx); lanes re-armed by the retry below\n",
 						req->push_va);