1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
|
# session 77 (r683/r684) — vkCreateDevice PASSES: the 0x7E event validation and the readiness-word completion
## The two-part unblock
1. **r682 fix validated** (`e3c1cdc5`, loaded on r683/r684): the
nvkms/Vulkan path allocates its completion event as
NV01_EVENT_KERNEL_CALLBACK_EX (class 0x7E); the emulation now
registers it alongside 0x79. Live proof (r683, sema_dbg=1):
GSP_RM_ALLOC class=0x7e paramsSize=24 hClient=0xc1e00011
hParent=0xcaf00001 hObject=0xcaf00003
event registered hEvent=0xcaf00003 hSrc=0x0
notifyIndex=1006632972 (off 8, slot 13)
params=1100e0c1000000007e0000000c00003c0000000000000000
EVPOST all ev[13] cl=0xc1e00011 ev=0xcaf00003 nidx=0x3c00000c rc=0
(notifyIndex 0x3c00000c = subdevice 0x3c, index 0xc =
NV2080_NOTIFIERS_GRAPHICS). The guest consumes the records
(`guest_read == our_write` in the rmstate dump), but the guest's
event dispatch does NOT write the event's notification word — the
residual delivery gap.
2. **The completion word found and driven (the actual unblock)**. The
main-thread vkCreateDevice wait (eglcore 0xdceb80, session 76) reads
a u32 readiness word: `should-wait = (word != 1)`; the word lives at
`+0x80` of a **16 KB rw-s /dev/nvidiactl mapping** (the event
notification buffer; holder chain sub -> obj2 -> holder, resolved
live by t79.py). The host can write it directly in the guest RAM
file:
PA = pagemap PFN << 12 (+0x80); file offset = PA - 0x80000000
(above-4G RAM; below-4G would be PA itself per the cycle's
guest-RAM registration: gpa 0x0/0x100000000 -> off 0x0/0x80000000)
Writing 1 flipped the wait and **vkCreateDevice returned success**
(`VKDEVICE ok family=0`) — reproduced three times:
* r683: manual write after a t79 probe;
* r684: nudger (all 16 KB mapping words) + slotfill restart;
* r684d: probe -> single-word write (the reliable recipe).
## The reliable recipe (per run)
1. r682+ module; `sema_dbg=1 gsp_rm_ev_log=300` optional.
2. Launch `vk_frame` + `slotfill2.sh` (EGL completion-slot fills; the
slots re-arm, so re-run slotfill if the client is stuck in the
sched_yield spin before the word-wait).
3. Wait for the main thread in `do_poll` (the word-wait).
4. `t79.py` (gdb) -> `WAIT word=0x...`; guest pagemap -> PFN -> PA;
host write `01 00 00 00` at `PA-0x80000000`.
5. `VKDEVICE ok` follows within ~10 s. The client then proceeds to
image/memory/pipeline setup; its next wait is NOT the same word
(the word stays 1) — next session: probe the follow-on state
(attach fails while the client spins in the post-device phase;
use the /proc-only checks + a longer observation window).
## Cautions learned
* Writing ALL 16 KB mapping words (the multi-page nudger) can trip the
-4 give-up path (an RM error flag word gets written): restrict nudges
to the single probed word.
* Attaching gdb to the client while it is in the tight post-device spin
can hang the attach (leave a `timeout`-wrapped gdb, then kill that
gdb by PID and `kill -CONT` the client — the pending SIGSTOP from a
dead tracer otherwise wedges it).
* zsh truncates 64-bit hex literals in `$(( ))`: compute PFN/PA in
python, not zsh.
## Tools
* `t79.py` — resolves the wait chain (sub -> obj2 -> holder -> word VA)
and prints the client's nvidiactl rw-s mapping sizes.
* `guest_words.sh` — prints VA/PA of the +0x80 word for every 16 KB
nvidiactl mapping (pagemap; run in the guest as root).
* `nudge.sh` — host-side loop: poll guest_words.sh and write 1 to any
zero word (multi-page variant; prefer the single-word recipe).
## ADDENDUM (r685, session 78) — POST-DEVICE WAIT IDENTIFIED: the NvOgl (GL ucode) engine init; push windows now READABLE (resolver6)
* The post-device spin is the **GL module bring-up**: the main thread's
A8-layer retry loop (eglcore 0xa85d24) works on a work item whose
embedded name is ASCII **"NvOgl_m0"** (+0xf04: "NvOgl_m40"); the D6
submit layer (0xd624e0) ORs **0x200000** into its pending word (the
same engine-pending bit SESSION 75/76 watched on E+0x150), then the
A8 loop yields via the B284B0 spin helper (sched_yield) waiting for
the GL ucode init to complete on the GR engine.
* **The push windows ARE now readable** (first time): the resolver6
recipe (window-record pair {push_va,0x20000}+{ring_va,0x40000} ->
same-page 0x20000 memdesc PA -> `sema_map_add`) injected
`0x4480000 -> 0x120cfb000` (content `2001c00d 70000000 2001c00d
68000000` = the 2-method semaphore-like push) and `0x4ca0000 ->
0x12e51e000` (the 1432-byte GR init stream: SET_OBJECT 0xc5c0/0xc5b5,
`0x200121ed`, `0x200120a9`, bulk methods, tail `2001c00d
70/68`). The service now decodes the streams (`sema-push ... tail=0dc0
012000000070...`) where SESSION 73-74 saw "unread".
CAUTION: the resolver injects every candidate; a zero-page candidate
injected last replaces the good mapping for that VA (re-inject the
good page after the scan).
* **Remaining gate**: the service still reports `have_rel=0` for every
GR push (the CE-shaped release matcher) and — decisively — the
GR-forward machinery is NOT engaging: `drain held=4 starved=149
forwarded=0 ... gr_nocap=4`. The GL ucode init work is never
forwarded/executed, so the completion (and the 0x200000 bit clear)
never arrives. Next: the GR forward/capture path for these channels
(operand capture + release decode for the GR shape), then the client
proceeds to VKFRAME.
|
