1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
|
# WALL #2 / device-init completion — session 74 findings (r675)
## The client's "11 s stalls" fully root-caused (this is NOT the channel-init release)
strace (ioctl-only) shows THREE exact 11.00 s gaps with NO ioctl in flight; a
full strace shows the client doing a pure `sched_yield` spin (~57K/s) +
`clock_gettime` (perf) inside **libnvidia-eglcore.so.570.144** — i.e. a
userspace timeout loop. gdb backtraces place the spin inside the Vulkan ICD's
EGL/GL init (`vkCreateInstance` -> libGLX_nvidia vk_icdNegotiate... ->
libEGL -> eglcore wait loop).
## The wait target: completion slots in /dev/nvidiactl mappings
- The wait loop (eglcore file offsets ~0xb284b0 delay, ~0xb33d00 wait loop,
~0xb1c750 slot check) polls a slot table on the client's heap object:
table at object+0x1428, count at object+0x22efc.
- Slot check: `esi` is offset by 0x43; for esi-0x43 > 4 it reads a **u16 at
slot+0xe** and treats `0xffff` as "empty/not complete".
- gdb sampling during the stall: 13164 checks with `u16@slot+0xe = 0xffff`,
1 with `0` — the client spins until the flag changes.
- The slot pointer = a **/dev/nvidiactl rw-s mapping** (page-sized; base+0x20
= slot). These are the client's mappings of RM memory (the channel
notifier pages: service logs `notif=0x11627f060` per channel).
## Filling the slots (guest-side) WORKS mechanically but is not the -4 cause
- `slotfill.sh` / `slotfill2.sh`: while vk_frame runs, write 0 to
`base+0x2e` of every small /dev/nvidiactl rw-s mapping when it reads 0xffff.
- Result: the 11 s gaps vanish (5 fills within ~2 s; the client proceeds
immediately) — **but vkCreateDevice still fails -4**. So the 11 s waits are
not the failure; they are an init-side slowdown the client tolerates.
## The next wall: the polled RM control 0x2080a097
- With the slots filled, the client settles into **NVOS54 (ioctl nr=0x2a)
polling every ~1.25 s** (strace), and the host dmesg shows
`GSP_RM_CONTROL cmd=0x2080a097 hClient=0xc1d000bb hObject=0xbeef0004
paramsSize=1168 flags=0x0` -> `unknown cmd=0x2080a097 -> generic SUCCESS`
(all-zero reply), 36 polls in 45 s.
- The NVOS54 struct (gdb dump): {hClient=0xc1d000c0, hObject=0xbeef0004,
cmd=0x2080a097, flags=0, params=ptr, paramsSize=0x490, status=ptr, ...}.
- The params buffer = 1168 bytes of 32-byte records with a `0x25` field at
+0x10 (dump in /tmp/opencode evidence; gdbparam.sh captures it live).
- The client polls until the control's status/reply signals completion; our
generic all-zero reply never does. **This is the wall that must be solved to
get vkCreateDevice past -4** (either a real reply for 0x2080a097 or making
the polled status field change).
## Other session-74 results
- The GR ring-entry writer was NOT found by disassembly-guessing: nvidia.ko
`channelFillGpFifo` and nvidia-modeset `nvWriteGpEntry` never fire for this
channel's kick (gdb hbreaks); nvidia/modeset kprobes are `notrace`; QEMU gdb
hbreak works.
- `sema_push_raw` dumps (module param) captured the service's reads of the
push: entry0 decodes only 2 methods then zeros (`consumed=0x288/1432`),
entry2 starts `SET_OBJECT 0xc597` then zeros; **no release in the push**
(`have_rel=0` for all three entries) — the channel-init release theory for
these waits is dead.
- `find_ring_va.py` / `vmap_watch2.py`: the ring entries are NOT in any
vmallocinfo area during the run; the nv_vm_map_pages areas hold the CE
pushes; the GR push content is not found by content searches.
## Files
- gdbpoll6.gdb, gdbu16.gdb, gdbfill.gdb, gdbwatch.gdb, gdbparam.sh,
gdbgap3.sh, gdbspin.sh — the gdb probes (slot check, u16 flag, fill,
NVOS54 param dump, backtraces).
- slotfill.sh / slotfill2.sh — the guest-side slot-fill watcher.
- stracevk.sh, fastrun.sh, fastrun2.sh — the strace/perf recipes.
- perfstall5.sh — perf sampling with map-relative offsets.
- find_ring_va.py, vmap_watch2.py — the vmap/kcore watchers.
|
