From e77efad422961e5266ef30ef5ce9f49ae1fd76c1 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Wed, 21 Oct 2026 11:39:25 -0400 Subject: gr: r410 wedge quiet window - bounded idle hold before retrying a lane whose previous launch also no-wrote (r416: 58-launch sticky wedge clears after ~50 s quiet); wedge counters in state; wedge_recover v2 unmask The r416 boot reproduced the sticky no-execute wedge inside 60 s: 58 consecutive launches accepted zero writes while the lane stayed busy (wedge_recover v1's deterministic verify masked every one of them), and the first launches were green again only after the lane sat idle ~52 s. Both r409 (~37 s idle heal) and r416 therefore show the wedge clears on idleness and persists under back-to-back retries (r154/r409), so the retry loop now holds a quiet window (gsp_phys_gr_wedge_quiet_ms, default 20000, doubling on the second quiet retry, 0 disables) before the normal fresh-bank + resched retry when the previous launch also no-wrote. The whole-launch stall budget is relaxed only on this bounded recovery path. NOWRITE lines now carry the consecutive streak and quiet/heal counters, and the state file shows nw_streak/wedge_quiet/wedge_heals. wedge_recover v2 mutates the inputs every iteration so a stale (never executed) launch can no longer match the expected values, and classifies the readback STALE-PREV(NOWRITE) vs OTHER/PARTIAL. --- tools/cuda-ab/run/wedge_recover | Bin 821024 -> 825120 bytes 1 file changed, 0 insertions(+), 0 deletions(-) (limited to 'tools/cuda-ab/run') diff --git a/tools/cuda-ab/run/wedge_recover b/tools/cuda-ab/run/wedge_recover index 2238fc1..3d9351c 100755 Binary files a/tools/cuda-ab/run/wedge_recover and b/tools/cuda-ab/run/wedge_recover differ -- cgit v1.2.3