open-iov.git/tools/cuda-ab/run/wedge_recover, branch masterOpen-IOV vGPU-IO: host-side GSP emulation for unmodified NVIDIA guest drivers
https://git.vgpu.io/open-iov.git/atom?h=master2026-10-21T15:39:32+00:00gr: r410 wedge quiet window - bounded idle hold before retrying a lane whose previous launch also no-wrote (r416: 58-launch sticky wedge clears after ~50 s quiet); wedge counters in state; wedge_recover v2 unmask2026-10-21T15:39:32+00:00Arthur Hanson Rasmussonarthur@vgpu.io2026-10-21T15:39:25+00:00urn:sha1:e77efad422961e5266ef30ef5ce9f49ae1fd76c1
The r416 boot reproduced the sticky no-execute wedge inside 60 s: 58
consecutive launches accepted zero writes while the lane stayed busy
(wedge_recover v1's deterministic verify masked every one of them), and
the first launches were green again only after the lane sat idle ~52 s.
Both r409 (~37 s idle heal) and r416 therefore show the wedge clears on
idleness and persists under back-to-back retries (r154/r409), so the
retry loop now holds a quiet window (gsp_phys_gr_wedge_quiet_ms, default
20000, doubling on the second quiet retry, 0 disables) before the normal
fresh-bank + resched retry when the previous launch also no-wrote. The
whole-launch stall budget is relaxed only on this bounded recovery path.
NOWRITE lines now carry the consecutive streak and quiet/heal counters,
and the state file shows nw_streak/wedge_quiet/wedge_heals.
wedge_recover v2 mutates the inputs every iteration so a stale (never
executed) launch can no longer match the expected values, and classifies
the readback STALE-PREV(NOWRITE) vs OTHER/PARTIAL.
tools/cuda-ab: memset_probe + wedge_recover forensics clients (session 10)2026-10-21T14:58:58+00:00Arthur Hanson Rasmussonarthur@vgpu.io2026-10-21T14:58:58+00:00urn:sha1:9f68e5e304416a7d2148903eada9caffd7b0779e
memset_probe: linear 8 MB cudaMemset + D2H with first-bad offset and a
4 KB page map (correct / other / malloc-prefill) - separates 'fill
under-delivered' from 'readback delivered wrong pages'; standalone green,
bisects the memcpy_roundtrip memset phase.
wedge_recover: launch+verify loop that on failure sleeps N seconds and
retries the same iteration (same context), reporting per-iteration
attempts and launch times - tests whether an idle gap heals the per-boot
no-execute wedge (r409 log shows a ~37 s client-side gap healed a wedged
run).
Also ships the missing inplace_scale run binary (its source landed in
7685d1b3).