diff options
Diffstat (limited to 'docs/04-implementation/04-real-render-walls.md')
| -rw-r--r-- | docs/04-implementation/04-real-render-walls.md | 83 |
1 files changed, 83 insertions, 0 deletions
diff --git a/docs/04-implementation/04-real-render-walls.md b/docs/04-implementation/04-real-render-walls.md index 3bad12e..82d9cf0 100644 --- a/docs/04-implementation/04-real-render-walls.md +++ b/docs/04-implementation/04-real-render-walls.md @@ -467,3 +467,86 @@ content appears exactly when the driver writes entry0). 3. With the ring latched and the VAS at +5 s, verify the inline copies execute (`sema-lda`/`sema-opadd`/`sema-copyent` with the code dsts) and the code page gets content; then the `n_data` descriptor wiring (section 8). + +## 14. UPDATE 2026-10-26 (SESSION 134/135): the CE first-submission capture landed and its rank-blind first-hit caused a CUDA-client stall regression (fixed); the Vulkan client's CE rings are allocated ABOVE the capture scan's 64 MB bound and the frame chain reduces to that un-latched ring + +**SESSION 134 (commits `36642718`, `928c0730`).** A new instrument fires in +the notifier-completion path for CE channels *before* the clear write - the +one instant the ring content and the USERD are both live - reads entry0 +through the channel translation and the trusted walk, then scans the shadow +(0x100 step, first 64 MB) for a ring-shaped page (entry0 naming a 1 MB +aligned push VA in `[0x120000000, 0x121000000)` with a sane length) and runs +the standard USERD gate at `+0x10000/+0x2000/+0x1000/+0x0`; a validating +candidate is pinned/latched exactly like the regular CE autopin. `928c0730` +fixed the entry decode: the push VA is 40 bits, the scan had declared it +`u32`, so `scan_hits` was zero by construction. + +**SESSION 135 (commit `a674c21c`) - the rank-blind first-hit regression.** +The scan started at offset 0 and accepted the first ring-shaped page for +*every* CE channel. On r779 both `caf00012` (rank 0) and `caf0001d` +(rank 1) latched `off=0x80000 upa=0x90000`: rank 1 read rank 0's +ring/page, its first release decoded to the wrong semaphore word (+0 +instead of +4) and was dropped as stale (`cur=1`, already written by +rank 0), so the client's rank-1 tracker wait never completed. victory_mail +then spun at 100% CPU with `pub_calls=0` on three consecutive boots +(r777/r778/r779: battery pass=0 fail=7). A control run of the r746-era +module (`25523f84`) on a fresh boot passed the same client (VADD +`bad=0 checksum=12204516.000000`, `STATUS OK`), isolating this to the +SESSION 133/134 code. The fix tries the rank's convention offset +(`0x80000 + 0x20000 * rank`) first, scans the rest bounded as before, and +never accepts an offset another CE rank already latched this session; an +unlatched rank stays for the regular per-pass latch. Verified on r781: +ranks 0..5 latched correctly (0x80000/0xa0000/0xc0000/0xe0000/0x100000/ +0x120000, USERD +0x10000 each), rank 1's own semaphore word applied +(`pool-wr va=0x121000004 cur=0 -> val=1 ok`), and the client progressed +(`pub_calls=185132+`, no spin). + +**The Vulkan CE ring on the same boots.** r781 (classic autopin, rank 0): +`auto-pinned ring h=0xbeef0101 rank=0 va=0x4100000 -> fb 0x80000 (entry0 +va=0x120000000 len=40)` - a live entry - but the client's pushes decoded +`unread=1 injected=0`: the guest pin scanner covered CUDA clients +(`cdt,va,victory`) and never `vk_frame`. Extending the scanner to `vk` +engaged the tools (demand pins active) but r782/r783's captures reported +`scan_hits=0 latch=0`: no ring-shaped page anywhere in the shadow's first +64 MB, and no classic autopin either. + +**Where the Vulkan rings actually are.** The client's own vidheap ioctl +log (`PREV149-OUT` lines, r783) names the CE/GR channel allocations: +`off=0x4020000/0x4100000/0x4160000` - matching the registered gpfifos +(`0x4020000/0x4100000/0x4160000`) and lying just ABOVE the capture's +`lim = min(shadow, 0x4000000)` (64 MB). Direct dumps of those offsets (and +of `0x80000`) read zero, and a byte search of the full 4 GB guest-RAM image +for the entry pattern (`00 00 00 20 01 28 00 00` / the `0x20100000` +sibling) finds nothing - the ring content reaches neither the shadow at its +allocation offset nor guest RAM. + +**The frame chain (r783, live).** The Vulkan client completes three frames +(`VKFRAME_DONE frames=3`, `VKFRAME_IDLE ok`) but all PPMs are pure black. +The live GR channel (`h=0xbeec0100`) shows the dependency explicitly: +`sema-gr: launch captured ... prog=0xdff0feb900 ... inline=8` together +with `sema-gr: copy pend h=0xbeec0100 push=0x4f4580c src=0xdff0d60000 +dst=0x50b0000 row=0xa00 rows=480 ...` repeated ~20/s. The trusted walk +maps the copy source to VIDMEM (`walk_probe 0xdff0d60000 -> rc=0 +pa=0x570000 aperture=vidmem`) and the destination `0x50b0000` is the +framebuffer. The shadow at `0x570000` reads zero: the data pages are the +ones the CE inline copies are supposed to fill, and the GR launch is held +on their absence (the designed `dstpend` dependency). So the whole Vulkan +chain reduces once more to the un-latched CE ring: no ring -> no inline +i2m copies -> no data/code pages -> held launches -> black frame. + +**Next steps (ranked).** 1. Cover the client's real allocation in the ring +capture: raise the capture scan bound above the recorded channel +allocations (e.g. `0x6000000`) and/or seed candidates from the client's +vidheap offsets, then re-check whether the entry arrives at the +submission instant (a live `sema_vidmem_dump` at `0x4100000` while the +client is streaming would settle it; the post-run dumps read zero). 2. If +the shadow still reads zero at the allocation offset during live +submissions, the client-side write watch is the remaining forensic: find +the CPU pointer the driver writes GPFIFO entries through, its PA via the +guest pagemap, and why the GPU PTE/walk does not name that PA. 3. With a +latched ring, verify the inline copies execute (`sema-lda`/`sema-opadd` +with the code dsts), the code/data pages fill, then the `n_data` +descriptor wiring (section 8) → non-black frames. 4. Re-run the CUDA +battery on the fixed module (the r781 boot was a slow-crawl boot; the +spin stall class is gone) and revisit `sema_gr_xlate`'s multikernel +forward timeout. |
