1 files changed, 134 insertions, 0 deletions
diff --git a/tools/guest-diag/wall2/README-session76.md b/tools/guest-diag/wall2/README-session76.md
new file mode 100644
index 0000000..828b3b9
--- /dev/null
+++ b/tools/guest-diag/wall2/README-session76.md
@@ -0,0 +1,134 @@
+# session 76 notes (r680) — the 0x2080a097 engine-monitor re-examined
+
+## What the firmware says (ground truth for the a097 family)
+
+The shipped GSP-RM 570.144 image embeds the host-side control
+descriptor table (file offset 0x41df8..0x42450, entry = 0x20 bytes
+{u32 cmd, u32 params_size, u32 0x4049580, u32 0, u32 handler_ptr,
+u32 0, u32 out_size, u32 0}). Entries seen:
+
+    0x2080a07f size 0x408 handler vaddr 0x551b2c0
+    0x2080a079 size 0x14804
+    0x2080a097 size 0x490 handler vaddr 0x55166f4
+    0x2080a0d1 size 0x7e8
+    ...
+
+The real 0x2080a097 handler (0x55166f4) writes up to FOUR {u32,u32}
+mode pairs into params+4..+0x23 (params[0] low byte = the request's
+count flag): for i=1..4 it calls an engine-status helper
+(0x5516334 / 0x55d7f04 / 0x55d7c1c / 0x55d7d60) with mode=i and emits
+{pair} = {engine value, 1000 * u16(engine+0x1bc0)}; the loop stops at
+the first helper error. So the real reply overwrites the client's
+request record {1,0,0,0}.
+
+## What the client does with the reply
+
+libnvidia-eglcore's local handler (file offset 0xecd2e0, the userspace
+implementation of the internal cmd 0x2080a040) copies back:
+
+    result+0     = reply+0        (count)
+    result+0x10  = reply+0x204    (flag)
+    result+0x20  = reply+0x488
+    result+0x24  = reply+0x48c
+    count*16 bytes from reply+4 -> *(result+8)
+
+For the engine tick (file offset 0xd5e3e0, called from the wait loop
+0xd6ef40 per iteration) the result struct IS the caller's request at
+rsp+0x10, and *(result+8) points at a 16-byte "inner" buffer
+(rsp+0). The tick then reads inner+4 and stores it to session+0xe104
+(the per-engine "progress" cache; poll interval ~1.25 s from the same
+function). A second consumer of the tick's return value compares it
+against per-boot config at lib+0x2019320:
+
+    0xfc8 = 1000, 0xffc = 95 (progress threshold),
+    0xfec = 10, 0xfed = 20 (bit-history cutoffs),
+    0x1000 = 500
+
+and ORs the comparison bit into a round-robin bitfield (+0xb4) of the
+state machine object.
+
+## Live evidence on r680 (fresh boot, full-param recipe)
+
+* The 0x2080a097 polls (NVOS54 0x2a, 1.25 s cadence) come from a
+  WORKER thread (gdb bt: start_thread -> eglcore a11b2a -> wait loop
+  0xd6ef40 -> tick 0xd5e3e0). The engine object E (0x604101365e90)
+  sits at E+0x150 = 0x00200000 (bit 21 only), E+0x1b038 = 1,
+  E+0x1b039 = 1, *(E+0x1b030)+8 != 0 — all static for 5+ minutes.
+* MAIN thread (vkCreateDevice) is in a poll(fd=/dev/nvidia0, events
+  POLLIN|POLLPRI, timeout=10s) loop inside eglcore (bt: poll <-
+  eglcore+0xa12de4 <- 0xd69159 <- 0xd6a02b <- 0xdcebc6 <- ... <-
+  libvulkan vkCreateDevice). No -4 is ever printed in this state: the
+  client waits indefinitely (killed by its own timeout in runs).
+* GR channel kicks continue concurrently (sema-entry lines for
+  h=0xbeec0100 va=0x4ca0000 len=1432 etc.) and end as
+  "skipping stale entry ... after N failed rescues" (gr_wedge>0).
+* Reply sweep on r680 (module params) — flag=0/2/1/1..0xffffffff,
+  v488, v48c AND the new record matrix count/rec0..rec7 =
+  0/{1,0,0,0}/100/1000/100000/1000000/0xdeadbeef — produced ZERO
+  observable change: same a097 cadence, no writers to
+  E+0x150/E+0x1b038/E+0x1b039 ever caught, no client progress. The
+  hw watchpoints on those words fired zero times over ~5 min.
+
+## Tools
+
+* t75g.py — in-guest gdb python: dispatcher-type log, a097 issuer
+  dump, tick (d5e449) state + cfg dump, E/S capture, hw watchpoints on
+  E+0x150 / E+0x1b038 / E+0x1b039 / S+0x120, POST a040 consumption
+  dump.
+* diag76.py — in-guest gdb python: thread-apply bt + map-relative
+  address resolution (used to find the main-thread poll wait).
+* slots_probe.sh — read the u16 completion-slot words of every small
+  /dev/nvidiactl rw-s mapping (SESSION 74's slotfill ABI).
+* run75g.sh — trace + record-word sweep driver.
+
+## Verdict
+
+The a097 monitor is a background worker, not an obviously blocking
+gate: every knob the client reads from it can be varied with no
+observable state change, while the main thread blocks elsewhere
+(/dev/nvidia0 poll loop) and the GR channel-init pushes still cannot
+be read by the sema service. Next lever is the main thread's wait
+(what would make the /dev/nvidia0 poll return readable) or the GR
+channel-init completion — not the a097 reply words.
+
+## ADDENDUM (r681) — THE MAIN-THREAD vkCreateDevice WAIT DECODED
+
+Live gdb on the main thread (r681, fresh boot) resolved the loop at
+eglcore 0xdceb80 (frame chain: poll <- a12de4 <- d69159 <- d6a02b <-
+dcebc6 <- d3838a <- ... <- libvulkan vkCreateDevice):
+
+* rbx (waitable, vt 0x...3f58f8): vt+0x18 = d69ee0 = "should wait"
+  (returns `(*0x48(*(rbx+0x70)) != 1)`); vt+0x28 = d69f50 = "wait"
+  (builds a pollfd array from the object's fd holder and poll()s up to
+  16 fds with the 10 s timeout).
+* rbp (loop object, vt 0x...3f71f8): vt+0x38 = dcee20 = the give-up
+  predicate. dcee20 returns nonzero (-> return -4 = the client's
+  VKFAIL -4 origin) when either object at rbp+0x110 / rbp+0x120 shows
+  an error (b29d60(...) != 0 or *(obj+0x70cc) != 0); otherwise 0 and
+  the loop repeats. It also throttles itself at 1e6 (units) via
+  rbp+0x1b8.
+* The readiness value: sub = *(rbx+0x70) (0x...f7f0c90); sub+0x58 ->
+  obj2; obj2's vt+0x10 = 0xce71f0 = `read u32 at (*(obj2+0x58)) +
+  *(u32*)(obj2+0x60)` — i.e. a u32 word at holder_base + 0x80 (the
+  live object had obj2+0x60 = 0x80, obj2+0x64 = 0x1e). Observed: 0x0
+  every poll (39/39) -> should-wait = 1 forever.
+* The word lives in a **/dev/nvidiactl rw-s mapping**
+  (0x7623d0480000-0x7623d0484000) — a VM_IO driver page, not plain
+  guest RAM: /proc/PID/mem reads of it return zeros, and writing the
+  guest-RAM file at the pagemap-derived PAs (0x1036dd080 and both
+  hole-shifted candidates) did NOT change the client's behaviour.
+  The writer is therefore the guest kernel driver/RM when the awaited
+  operation completes (the same class as the SESSION-74 completion
+  slots) — not a host RAM poke.
+* All 20 sampled polls of the wait helper returned 0 (timeout): the
+  fd (/dev/nvidia0) never becomes readable (nv_poll reports
+  POLLPRI|POLLIN only when an nv_event is pending on the file).
+
+So vkCreateDevice waits for (a) the readiness word == 1, and the wait
+ends only when the driver/RM signals that word (or an nv_event arrives
+on /dev/nvidia0). The give-up path (-4) fires only if the RM error
+flags appear. Next: identify the operation whose completion writes the
+word (the object chain sub -> obj2 -> holder is a "signal/semaphore"
+object; dump the holder object and its creating control), then make
+the emulation drive the guest kernel to complete it (the engine/
+channel-init completion the sema service already tries to deliver).