From be566f9579f32497003966e495543974cd93d20d Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Wed, 21 Oct 2026 13:50:43 -0400 Subject: gr/sema: r420 pause completion deadlines under the backend wedge-quiet hold The r419 soak reproduced the user's mid-boot degradation with the wedge fix in tree: victory_matmul printed bad=65536 while its launch was held by the booter's WEDGE-QUIET window - the client was released ~115 s before the post-hold accept. Root cause: two sema-service deadlines are shorter than the deliberate quiet hold and fire under it, authoring the launch's completion release (and applying its recorded releases) while the engine has not run and the copy-out has not happened: - the 12 s GR release hold (SVC_FWD_GR_HOLD_TIMEOUT_MS) expiry (GR-STALL-fwd-hold ... releases + QMD word software-applied), and - the 3 s per-launch consumption bound (gsp_phys_gr_hold_max_ms) firing the busy-defer consume-with-software-release path, whose inline apply authors the capture's QMD release (GR-REL-AUTHOR src=hold). Evidence (r419 live): push be94 captured 131333.687, WEDGE-QUIET holds 40 s + 80 s until 131345.598, accept 131346.552 - while the sema hold expired at 131230.978 (GR-STALL-fwd-hold waited_ms=12032) and the inline apply authored va=0x204027f7c pay=6 at 131341.734 (h=0x5c00001f). The client's verify completed on the authored word and read stale data. Fix: the booter publishes gsp_phys_gr_quiet_hold_active() exactly around the quiet msleep (bounded by construction). The sema service pauses under it: svc_fwd_hold_expired() pushes the GR deadline (bounded by a 600 s defence-in-depth cap, announced once per hold as GR-HOLD-PAUSE); svc_gr_hold_bound_expired() pushes the per-launch bound (GR-HOLD-BOUND-PAUSE); and the busy-defer one-shot window re-holds while the flag is set so the consume-with-sw-release path cannot run during a quiet hold. The held launch is then forwarded and copied out after the hold, and its completion is authored only on the hold's done edge - after the data write-back. --- kernel/gsp_phys_booter.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) (limited to 'kernel/gsp_phys_booter.c') diff --git a/kernel/gsp_phys_booter.c b/kernel/gsp_phys_booter.c index d20e042..f7ad153 100644 --- a/kernel/gsp_phys_booter.c +++ b/kernel/gsp_phys_booter.c @@ -16324,6 +16324,29 @@ static u64 gsp_phys_gr_wedge_heals; * either heals (engine writes, client data correct) or is loudly classified. */ static bool gsp_phys_gr_wedge_episode; +/* + * r420: true exactly while a WEDGE-QUIET hold sleeps with NO submission + * outstanding. The hold is a deliberate, bounded wait for the sticky wedge to + * clear (r410/r411); during it the in-flight forwarded request is NOT done. + * The sema service's completion deadlines - the 12 s GR release hold + * (SVC_FWD_GR_HOLD_TIMEOUT_MS) and the 3 s per-launch consumption bound + * (gsp_phys_gr_hold_max_ms) - are shorter than this hold, so without the + * publish they fire UNDER it and author the launch's release word before the + * post-hold copy-out: the client then verifies stale data (r419 soak: + * `victory_matmul bad=65536` while its launch was held; the authoring paths + * are GR-STALL-fwd-hold / the busy-defer inline software release). The + * accessor lets both deadlines pause instead; the flag is bounded by the + * msleep itself and by the pause caps on the sema side. Single-inflight + * submit path, so a plain static. + */ +static bool gsp_phys_gr_quiet_hold_on; + +bool gsp_phys_gr_quiet_hold_active(void) +{ + return READ_ONCE(gsp_phys_gr_quiet_hold_on); +} +EXPORT_SYMBOL_GPL(gsp_phys_gr_quiet_hold_active); + /* m86: program-page override blob (see gsp-phys-gr-override). */ static u8 gsp_phys_gr_override[GSP_PHYS_PAGE_SIZE]; static u32 gsp_phys_gr_override_len; @@ -20579,7 +20602,22 @@ gr_page_unmapped: quiet_ms, nw_attempts, req->push_va, READ_ONCE(gsp_phys_gr_nw_streak)); + /* + * r420: publish the deliberate hold so + * the sema service's completion + * deadlines (the 12 s GR release hold + * and the 3 s per-launch consumption + * bound) pause instead of authoring + * this launch's release word under it - + * the client must not be released + * before the post-hold copy-out + * (gsp_phys_gr_quiet_hold_active). + */ + WRITE_ONCE(gsp_phys_gr_quiet_hold_on, + true); msleep(quiet_ms); + WRITE_ONCE(gsp_phys_gr_quiet_hold_on, + false); quiet_used = quiet_ms; pr_warn("vgsp-gr: WEDGE-QUIET hold done (push=0x%llx); lanes re-armed by the retry below\n", req->push_va); -- cgit v1.2.3