wall-2 session 74: the client's 11s stalls are userspace completion-slot spins; the next wall is the polled RM control 0x2080a097

Root-caused the vk_frame client's exact 11.00s gaps: a pure sched_yield +
clock_gettime userspace spin inside libnvidia-eglcore's EGL/GL init, waiting
for completion slots (u16 at slot+0xe == 0xffff) in /dev/nvidiactl mappings.
Guest-side filling of the slot flags lets the client proceed immediately, but
vkCreateDevice still fails -4: with the slots filled the client settles into
an NVOS54 poll of RM control 0x2080a097 (paramsSize=1168) which our emulation
answers with the generic all-zero reply, so the poll never completes.

Also captured: sema_push_raw dumps proving the GR channel-init push contains
NO release (entry0 decodes 2 methods then zeros; entry2 SET_OBJECT 0xc597 then
zeros), and the negative result that neither channelFillGpFifo nor
nvWriteGpEntry runs for this channel's kick.

New probes: gdbpoll6/gdbu16/gdbfill/gdbwatch/gdbparam/gdbgap3/gdbspin (slot
check, flag fill, NVOS54 param dump, backtraces), slotfill{,2}.sh (guest slot
watcher), stracevk/fastrun{,2}.sh (strace+perf recipes), find_ring_va.py and
vmap_watch2.py (vmap/kcore watchers), README-session74.md.

1 files changed, 45 insertions, 0 deletions

diff --git a/tools/guest-diag/wall2/find_ring_va.py b/tools/guest-diag/wall2/find_ring_va.py
new file mode 100644
index 0000000..c02ea40
--- /dev/null
+++ b/tools/guest-diag/wall2/find_ring_va.py
@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+# Guest-side: read the nv_vm_map_pages vmap areas via /proc/kcore and find the
+# GPFIFO entry pattern (entry0 raw 0x0005980004480000 for va=0x4480000 len=1432).
+import struct, sys
+pat = bytes.fromhex("0000480400980500")  # 0x0005980004480000 LE
+class Kcore:
+    def __init__(self):
+        self.fd = open("/proc/kcore", "rb")
+        hdr = self.fd.read(64)
+        e_phoff = struct.unpack_from("<Q", hdr, 32)[0]
+        e_phentsize, e_phnum = struct.unpack_from("<HH", hdr, 54)
+        self.segs = []
+        for i in range(e_phnum):
+            self.fd.seek(e_phoff + i * e_phentsize)
+            ph = self.fd.read(56)
+            p_type, _fl, p_off, p_va, _pa, p_sz = struct.unpack_from("<IIQQQQ", ph, 0)
+            if p_type == 1:
+                self.segs.append((p_va, p_va + p_sz, p_off))
+    def read(self, va, n):
+        for lo, hi, off in self.segs:
+            if lo <= va and va + n <= hi:
+                self.fd.seek(off + (va - lo))
+                return self.fd.read(n)
+        return None
+kc = Kcore()
+areas = []
+for line in open("/proc/vmallocinfo"):
+    p = line.split()
+    if len(p) < 2: continue
+    if "nv_vm_map_pages" not in line: continue
+    lo, hi = p[0].split("-")
+    areas.append((int(lo, 16), int(hi, 16)))
+print("nv_vm_map_pages areas:", len(areas))
+for lo, hi in areas:
+    size = hi - lo
+    d = kc.read(lo, size)
+    if not d:
+        print("  0x%x-0x%x unreadable" % (lo, hi)); continue
+    idx = d.find(pat)
+    print("  0x%x-0x%x size=0x%x hit=%s" % (lo, hi, size, ("+0x%x -> VA 0x%x" % (idx, lo + idx)) if idx >= 0 else "no"))
+    if idx >= 0:
+        # dump the page around the hit
+        base = lo + (idx & ~0xfff)
+        page = kc.read(base, 0x100)
+        print("     page head: %s" % page.hex())