1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
|
# SESSION 73 notes (wall #2 live validation, r674 + r675)
## r674 cycle results (8 client runs, all VKFAIL -4)
- Client vk_frame reproducibly fails at vkCreateDevice (-4) as before.
- wall2_resolver2 (content-first CE group 0x20038090 + tuple arrays): no candidates found
during the stall; tuple arrays contain the ring PA but not the push.
- livecap/livecap2/livecap3 (0x20038090 shape / 0x20050017 host-release shape): NO data
hits in guest RAM during the stall; only driver CODE pages match.
- Conclusion: the GR channel-init push content is NOT findable in /dev/shm/vgpu-guest-ram
by these patterns, even live. (The ring page content IS there and matches the service's
decode.)
## Structural findings
- The push is RM-built (nvidia.ko), NOT UVM: kprobe on
uvm_hal_turing_host_semaphore_release never fired during the client run.
- nvidia.ko functions are `notrace` -> kprobe_events refuses them
("Could not probe notrace function ... [nvidia]"). UVM module probes DO work.
QEMU gdbstub hardware breakpoints work fine (hbreak *addr).
- Service rescue matcher is CE-shaped only (hdr 0x20038090, hi==1, lo&0xfff==slot,
w5/v5 shape) -> can never match a GR push. Entries 0/1/2 are consumed as "stale"
after ~150 failed rescues each; gr_wedge=1.
- channelFillGpFifo(channel, idx, len) disasm: rdi=channel; +0x178=gpu;
+0xe8=cached ring CPU VA; +0xb8=push GPU VA base; +0x48=stride;
+0x110=class (0xc5b5 seen for a CE channel; GR = 0xc46f).
Entry encoding: [0]=va&~3 (u32), [1]=(va>>32 &0xff)|((len<<8)&0x7ffffc00).
- r674 ring PAs per run: 0x12a783000(?) no: run1 0x1050b6000, run2 0x10297d000,
later 0x102317000 (ring line format: "h=0x... gpfifo=0x44a0000 ent=32768 userd=0xffe00 as=2 put=3 get=0 ring rc=0 pa=0x... ap=1").
- Push VA = 0x4480000, ring VA = 0x44a0000 (delta 0x20000); entry0 len=1432,
entry1 {va=0x4480598,len=4}, entry2 {va=0x448059c,len=976}.
- The pushbuffer (UVM) = 0x20000 (32 pages) aligned; UVM_PUSHBUFFER_SIZE=0x20000
(UVM_MAX_PUSH_SIZE=0x400). UVM_RM_MEM_TYPE_SYS with CPU map.
- RM memdesc records {size=0x20000, flags=0x03008000, first_PA, ptr} exist in RAM
(11 found post-run) but their PAs did not contain the release group.
- RM window record {0x4480000, 0x20000} + head/tail list nodes exist in RAM.
- The r673 VA-table record {0x10, 0x4500000, 0x786fd9598ab0, 0xa3bf20} has a
"cpu" field that may be a user VA of the client (0x64..-0x78.. range).
## Ops
- r674 wedged after the gdb-interrupted vk31 run (SSH + guest agent dead, vCPUs idle).
Fresh cycle r675 started. gdb674d still attached to the dead QEMU (never kill gdbs).
- /dev/shm 53%.
## r675 plan
1. gdb675.gdb: hbreak channelFillGpFifo if class(+0x110)==0xc46f; on hit dump the
channel struct pointer fields (see gdb675.gdb) -> identify the push's CPU mapping
(direct-map or vmalloc) by content.
2. Compute PA (direct-map: VA - 0xffff8916c0000000 style base) and read the push
content from the RAM file; find the release group.
3. Inject 0x4480000 -> PA via sema_map_add BEFORE/DURING the stall; watch the service
apply the release and vkCreateDevice pass.
4. Durability: integrate into pin_gr_stream.py demand path or a service hook.
|
