1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
# session 76 notes (r680) — the 0x2080a097 engine-monitor re-examined
## What the firmware says (ground truth for the a097 family)
The shipped GSP-RM 570.144 image embeds the host-side control
descriptor table (file offset 0x41df8..0x42450, entry = 0x20 bytes
{u32 cmd, u32 params_size, u32 0x4049580, u32 0, u32 handler_ptr,
u32 0, u32 out_size, u32 0}). Entries seen:
0x2080a07f size 0x408 handler vaddr 0x551b2c0
0x2080a079 size 0x14804
0x2080a097 size 0x490 handler vaddr 0x55166f4
0x2080a0d1 size 0x7e8
...
The real 0x2080a097 handler (0x55166f4) writes up to FOUR {u32,u32}
mode pairs into params+4..+0x23 (params[0] low byte = the request's
count flag): for i=1..4 it calls an engine-status helper
(0x5516334 / 0x55d7f04 / 0x55d7c1c / 0x55d7d60) with mode=i and emits
{pair} = {engine value, 1000 * u16(engine+0x1bc0)}; the loop stops at
the first helper error. So the real reply overwrites the client's
request record {1,0,0,0}.
## What the client does with the reply
libnvidia-eglcore's local handler (file offset 0xecd2e0, the userspace
implementation of the internal cmd 0x2080a040) copies back:
result+0 = reply+0 (count)
result+0x10 = reply+0x204 (flag)
result+0x20 = reply+0x488
result+0x24 = reply+0x48c
count*16 bytes from reply+4 -> *(result+8)
For the engine tick (file offset 0xd5e3e0, called from the wait loop
0xd6ef40 per iteration) the result struct IS the caller's request at
rsp+0x10, and *(result+8) points at a 16-byte "inner" buffer
(rsp+0). The tick then reads inner+4 and stores it to session+0xe104
(the per-engine "progress" cache; poll interval ~1.25 s from the same
function). A second consumer of the tick's return value compares it
against per-boot config at lib+0x2019320:
0xfc8 = 1000, 0xffc = 95 (progress threshold),
0xfec = 10, 0xfed = 20 (bit-history cutoffs),
0x1000 = 500
and ORs the comparison bit into a round-robin bitfield (+0xb4) of the
state machine object.
## Live evidence on r680 (fresh boot, full-param recipe)
* The 0x2080a097 polls (NVOS54 0x2a, 1.25 s cadence) come from a
WORKER thread (gdb bt: start_thread -> eglcore a11b2a -> wait loop
0xd6ef40 -> tick 0xd5e3e0). The engine object E (0x604101365e90)
sits at E+0x150 = 0x00200000 (bit 21 only), E+0x1b038 = 1,
E+0x1b039 = 1, *(E+0x1b030)+8 != 0 — all static for 5+ minutes.
* MAIN thread (vkCreateDevice) is in a poll(fd=/dev/nvidia0, events
POLLIN|POLLPRI, timeout=10s) loop inside eglcore (bt: poll <-
eglcore+0xa12de4 <- 0xd69159 <- 0xd6a02b <- 0xdcebc6 <- ... <-
libvulkan vkCreateDevice). No -4 is ever printed in this state: the
client waits indefinitely (killed by its own timeout in runs).
* GR channel kicks continue concurrently (sema-entry lines for
h=0xbeec0100 va=0x4ca0000 len=1432 etc.) and end as
"skipping stale entry ... after N failed rescues" (gr_wedge>0).
* Reply sweep on r680 (module params) — flag=0/2/1/1..0xffffffff,
v488, v48c AND the new record matrix count/rec0..rec7 =
0/{1,0,0,0}/100/1000/100000/1000000/0xdeadbeef — produced ZERO
observable change: same a097 cadence, no writers to
E+0x150/E+0x1b038/E+0x1b039 ever caught, no client progress. The
hw watchpoints on those words fired zero times over ~5 min.
## Tools
* t75g.py — in-guest gdb python: dispatcher-type log, a097 issuer
dump, tick (d5e449) state + cfg dump, E/S capture, hw watchpoints on
E+0x150 / E+0x1b038 / E+0x1b039 / S+0x120, POST a040 consumption
dump.
* diag76.py — in-guest gdb python: thread-apply bt + map-relative
address resolution (used to find the main-thread poll wait).
* slots_probe.sh — read the u16 completion-slot words of every small
/dev/nvidiactl rw-s mapping (SESSION 74's slotfill ABI).
* run75g.sh — trace + record-word sweep driver.
## Verdict
The a097 monitor is a background worker, not an obviously blocking
gate: every knob the client reads from it can be varied with no
observable state change, while the main thread blocks elsewhere
(/dev/nvidia0 poll loop) and the GR channel-init pushes still cannot
be read by the sema service. Next lever is the main thread's wait
(what would make the /dev/nvidia0 poll return readable) or the GR
channel-init completion — not the a097 reply words.
## ADDENDUM (r681) — THE MAIN-THREAD vkCreateDevice WAIT DECODED
Live gdb on the main thread (r681, fresh boot) resolved the loop at
eglcore 0xdceb80 (frame chain: poll <- a12de4 <- d69159 <- d6a02b <-
dcebc6 <- d3838a <- ... <- libvulkan vkCreateDevice):
* rbx (waitable, vt 0x...3f58f8): vt+0x18 = d69ee0 = "should wait"
(returns `(*0x48(*(rbx+0x70)) != 1)`); vt+0x28 = d69f50 = "wait"
(builds a pollfd array from the object's fd holder and poll()s up to
16 fds with the 10 s timeout).
* rbp (loop object, vt 0x...3f71f8): vt+0x38 = dcee20 = the give-up
predicate. dcee20 returns nonzero (-> return -4 = the client's
VKFAIL -4 origin) when either object at rbp+0x110 / rbp+0x120 shows
an error (b29d60(...) != 0 or *(obj+0x70cc) != 0); otherwise 0 and
the loop repeats. It also throttles itself at 1e6 (units) via
rbp+0x1b8.
* The readiness value: sub = *(rbx+0x70) (0x...f7f0c90); sub+0x58 ->
obj2; obj2's vt+0x10 = 0xce71f0 = `read u32 at (*(obj2+0x58)) +
*(u32*)(obj2+0x60)` — i.e. a u32 word at holder_base + 0x80 (the
live object had obj2+0x60 = 0x80, obj2+0x64 = 0x1e). Observed: 0x0
every poll (39/39) -> should-wait = 1 forever.
* The word lives in a **/dev/nvidiactl rw-s mapping**
(0x7623d0480000-0x7623d0484000) — a VM_IO driver page, not plain
guest RAM: /proc/PID/mem reads of it return zeros, and writing the
guest-RAM file at the pagemap-derived PAs (0x1036dd080 and both
hole-shifted candidates) did NOT change the client's behaviour.
The writer is therefore the guest kernel driver/RM when the awaited
operation completes (the same class as the SESSION-74 completion
slots) — not a host RAM poke.
* All 20 sampled polls of the wait helper returned 0 (timeout): the
fd (/dev/nvidia0) never becomes readable (nv_poll reports
POLLPRI|POLLIN only when an nv_event is pending on the file).
So vkCreateDevice waits for (a) the readiness word == 1, and the wait
ends only when the driver/RM signals that word (or an nv_event arrives
on /dev/nvidia0). The give-up path (-4) fires only if the RM error
flags appear. Next: identify the operation whose completion writes the
word (the object chain sub -> obj2 -> holder is a "signal/semaphore"
object; dump the holder object and its creating control), then make
the emulation drive the guest kernel to complete it (the engine/
channel-init completion the sema service already tries to deliver).
|
