diff options
| -rw-r--r-- | debug/issue/issue-number-1.md | 32 |
1 files changed, 32 insertions, 0 deletions
diff --git a/debug/issue/issue-number-1.md b/debug/issue/issue-number-1.md index f23721f..e7b04d4 100644 --- a/debug/issue/issue-number-1.md +++ b/debug/issue/issue-number-1.md @@ -126,3 +126,35 @@ the CE push/pool windows) so the watchers are accelerators only. * 2026-10-08: campaigns p1–p13 + smiley r1–r3 all stall; evidence shows the missing completion + missing pool-page translation; `fa9296af` PROMOTE_CTX change tested and disproven (and is itself corrected here). + +## Resolution update (2026-10-08 evening) — root cause found, gate in progress + +**Root cause of the reproducible stall (fixed):** commit `1c5864ad` (m143, +"BAR1 size = fb_emul_size") made the guest see a 512 MB BAR1 while the +physical aperture is 256 MB. Every build at/after m143 stalls the CUDA +client (spin inside its first `cudaMalloc`); the pre-m143 m140 reference +reports 256 MB and is green. `26263e1d` restores the physical exposure +(forward-fix under design for the multi-size types: see +`/tmp/opencode/BAR1-512-ANALYSIS.md`). Merged with two companion reverts +(`8791acc3` on master): the walk-pin fix A revert (`2e1c9d2c`, it gated +guest GR pushes) and the end-of-pass GR counter publish restore +(`ed6d458d`, H3's early publish read put=0). + +**Interim gate result:** 9/10 green, then a wedge on run #10 (`rep10-r10`, +2026-10-08 17:25 EDT): GSP-RM ladder completed host-side, but the guest's +first UVM push was never serviced (`ring_auto=1`, `gpfifo done=1`, zero +pushes decoded), the client produced no output, and the guest hard-died +~10 min later. Guest logs (extracted to `/tmp/opencode/r10-guest-logs/`, +analysis `/tmp/opencode/R10-WEDGE-ANALYSIS.md`): the driver sequence is +identical to green runs until 14 ms after the 16 MB UVM pushbuffer system- +page allocation, where it goes silent — before the GPU-VA window mappings +every green run performs next. Leading hypothesis: ring discovery missed +the client channel's rings on that boot (host `ring_auto=1`), so the first +completion wait never resolved. Watchers/kprobes were armed normally. +Discriminating check: `uvtw`/`uvsv` kprobe traces during a stall (a uvtw +loop with value 0 and no `uvbp/uvch` submit = the push never left the +guest; submission present = the ring decode/emulation missed it). + +Wedge rate seen so far: 1 in 10 monitored cold boots. A clean 10/10 gate +re-run is in progress; the wedge stays open under this issue until a +10/10 consecutive-green streak is recorded. |