diff options
Diffstat (limited to 'docs/04-implementation/04-real-render-walls.md')
| -rw-r--r-- | docs/04-implementation/04-real-render-walls.md | 1216 |
1 files changed, 1216 insertions, 0 deletions
diff --git a/docs/04-implementation/04-real-render-walls.md b/docs/04-implementation/04-real-render-walls.md new file mode 100644 index 0000000..cc16b04 --- /dev/null +++ b/docs/04-implementation/04-real-render-walls.md @@ -0,0 +1,1216 @@ +# Real-render walls: Vulkan and Xorg on the emulated vGPU + +Status: OPEN (2026-10-29, SESSION 64). This page records the measured state +of the guest's *display/graphics* consumers - the Vulkan ICD and the Xorg +server - on the emulated vGPU, with the exact evidence and the next +subsystem to implement. It complements +[05-operations/06-guest-frame-path-and-looking-glass.md](../05-operations/06-guest-frame-path-and-looking-glass.md) +(which covers the CUDA -> ivshmem -> host frame transport that is already +proven). + +## 1. What works today + +* The guest's shipped 570.144 Vulkan userspace **loads and enumerates**: + `vkCreateInstance` succeeds and two physical devices are reported - + `NVIDIA GeForce RTX 2080 SUPER` (discrete, Vulkan 1.4.303) and the + software `llvmpipe` - from `/etc/vulkan/icd.d/nvidia_icd.json`. Probes: + `tools/vk/vk_probe.py`, `tools/vk/vk_probe2.py`. +* `eglInitialize` succeeds with `vendor=NVIDIA` on the guest EGL device + platform (the EGL probe `tools/guest-diag/egl_retry.c` and the older + `egl_frame`). +* CUDA is unaffected (its clients run through the same emulated GSP; see + the battery evidence in the handoff). + +## 2. Wall 1 - `vkCreateDevice` fails in nvidia-modeset's ALLOC_DEVICE + +`vkCreateDevice` returns `VK_ERROR_INITIALIZATION_FAILED`. An +`LD_PRELOAD` ioctl logger (`tools/guest-diag/nvkmstrace.c`) shows the +ICD's first nvKms ioctl failing: + +``` +NVKMS_IOCTL cmd=0 size=1488 addr=0x... rc=-1 # NVKMS_IOCTL_ALLOC_DEVICE +``` + +i.e. the kernel's `nvKmsAllocDevice` (in `nvidia-modeset`) returned a +non-success status; the userspace frees everything and reports the +generic initialization error. Reading the driver's own modeset source +(`nvkms-rm.c`) gives the alloc order and the two gates we hit and fixed: + +1. `ProbeHeadCountAndWindowAssignment()` -> `NV0073_CTRL_CMD_SYSTEM_GET_NUM_HEADS` + (`0x730102`): a zero reply logs `No heads found on board!` and fails the + alloc. The emulated GSP RSP answered all NV0073 controls through the + generic zero-reply path. +2. After the heads: `NoValidDisplays()` -> `NV0073_CTRL_CMD_SYSTEM_GET_SUPPORTED` + (`0x730107`) must report at least one display id, and the connector + path (`SPECIFIC_OR_GET_INFO` `0x73028b`, `SPECIFIC_GET_CONNECTOR_DATA` + `0x730250`, `DFP_GET_INFO` `0x731140`) must describe a connector. + +The host GSP-RM emulation now implements a minimal headless display: +one head (`0x730102`/`0x730287`), flexible window assignment (`0x7302ad`), +one static HDMI connector on display id 0 (SOR + single-link TMDS, which +avoids the DP-AUX library), plus DDC support. With this shim the alloc +gets past heads and `NoValidDisplays` and proceeds through the connector +queries and the display-object allocations (classes `0xc570`, `0xc57d`, +`0xc372`, controls `0x50700709`, `0x73010c`, `0xc3700201`) before failing +later in the display-core/modeset-channel path. That remaining subsystem +is the wall: the modeset core channel and the per-display dpy objects +must be emulated (a coupled object + channel + memory set whose exact +wire forms should be captured from a real-GPU session). + +A related experiment ruled out one shortcut: booting the guest with +`nvidia-modeset`/`nvidia-drm` blacklisted (no `/dev/nvidia-modeset`) does +not avoid the wall - the ICD's nvKms client still issues ALLOC_DEVICE and +fails, so the display path cannot be sidestepped by disabling KMS. + +## 3. Wall 2 - Xorg ScreenInit fails in nvidia-push's channel test + +Xorg starts (the GPU is detected, `Virtual screen 1280x1024`) and then: + +``` +(EE) NVIDIA(GPU-0): Failed to initialize DMA. +(EE) NVIDIA(0): Failed to allocate push buffer +``` + +`nvidia-push` logs `Failed to initialize DMA.` from +`__nvPushTestPushBuffer()`: right after allocating a push buffer the +driver writes a NOP and waits (10 s) for the channel to consume it. Host +evidence shows why it never does: + +* the X driver's channel registers (`class 0xc46f` GPFIFO channel, + `gpfifo=0x120090000`, `userd=0x350000` sysmem) and lives with + **`submissions=0`** - its published GP_PUT never advances in the host's + view, because the host needs the channel's VA->PA translations to read + its USERD/GPFIFO and that resolution comes from the guest-side scanner + that currently follows *CUDA clients only* (`targeted resolve + va=0x20400f000: full RAM scan found nothing`); +* on a busy boot (Xorg started while a CUDA battery ran) the channel's + completion-counter slot is rejected: `counter auto h=0x1000013 rank=0 + pa=0x11c827ff0 rejected (slot owned by another channel)` - two + concurrent clients both derive rank 0's slot from the per-device + structural formula. + +Fixing Xorg therefore needs (1) a platform-wide (non-CUDA) VA-resolution +path for the scanner or the host, and (2) counter-slot ownership keyed by +client identity (or a per-client slot domain), not just rank. + +## 4. Wall 3 - the EGL/glsi engine-init gate (unchanged) + +`eglMakeCurrent` returns `EGL_BAD_ALLOC (0x3003)`; the glsi engine-init +entry returns a "busy" status once, sets a state bit, and the eglcore +caller does not retry. Ten in-process retries (`egl_retry`) all fail, so +the state machine fails on a real input - the return-value trace of its +callees remains the way in (see +[userspace/cuda/570.144/docs/ANALYSIS-egl-engine-selection.md](../../userspace/cuda/570.144/docs/ANALYSIS-egl-engine-selection.md)). + +## 5. Why this matters for the frame path + +Everything downstream of a working GL/Vulkan context is already in place: +a guest publisher writes frames into the kvmfr ivshmem BAR +(`tools/vk/vk_frame.c`, `tools/guest-diag/xshm_pub.c` use the same layout +as the proven CUDA `frame_producer`), the host reader decodes them +(`tools/lg/frame_reader.py`) and ffmpeg assembles video. The missing +piece is a *working* graphics API on the guest GPU - the two subsystems +above. +## 6. UPDATE 2026-10-25 (RESUME #56): the wall is now fully bisected + +Three new instruments settled the remaining questions: + +* **An `LD_PRELOAD` nvKms ioctl logger** (`tools/guest-diag/nvkmstrace.c`) + decodes the failing `NVKMS_IOCTL_ALLOC_DEVICE` reply: status **5 = + `NVKMS_ALLOC_DEVICE_STATUS_CORE_CHANNEL_ALLOC_FAILED`**. +* **The QEMU gdb stub (`-gdb tcp::1234`) as a guest-kernel microscope** + (the guest kernel's `nvidia_modeset` symbols are in `/proc/kallsyms`): + breaking inside `nvAllocCoreChannelEvo` (the blob's offsets were mapped + from a disassembly of the shipped `nv-modeset-kernel.o_binary`) dumped + the driver's class list (42 classes; all `NVC57x` display classes are + present; **`NV927C_BASE_CHANNEL_DMA` (0x927c) is not** - it exists in no + HAL class list, so the "base channel" fallback path can never succeed). + Breaking at the capability parse showed the **capability notifier buffer + is all zeros** (nothing emulates the display engine that would fill it). +* **The caps-fill experiment**: with gdb writing `SYS_CAP = HEAD0_EXISTS` + and `SYS_CAPB = WINDOW0..7_EXISTS` into the caps buffer right before the + parse, the driver flips from the dead base path to the **window path** + (`nvRMAllocateWindowChannels`) - the hypothesis that `numWindows > 0` + unblocks the flow is confirmed. + +The window path then fails one step later: `nvRmApiAlloc(display, +class=0xc57e NVC57E_WINDOW_CHANNEL_DMA)` returns **`NV_ERR_NOT_SUPPORTED` +(0x56) locally in the guest kernel** (it never reaches the emulated +GSP - no ALLOC RPC is logged). This matches the shipped driver's vGPU +architecture: the guest runs in vGPU mode (`IS_VIRTUAL`), where the +display/modeset channel classes are rejected (the reference source's +`_rpcAllocObjectPrologue` documents "Display classes are added for DCE, +vGPU Guest does not support those") - a vGPU guest's display is meant to +be emulated by the **host-side vGPU display service** (the vGPU RPC +display path), which the emulation does not implement. + +**Consequence for the roadmap.** Guest Xorg/GL/Vulkan on this emulated +vGPU needs (a) the display capability fill on the host side (mechanics +proven by the gdb experiment) and (b) a vGPU display service: the +display-engine MMIO/RPC backend the guest expects from its host (the +equivalent of NVIDIA's vGPU display plugin), or the specific vGPU +configuration that makes the kernel accept the display channel class. +Until (b) exists the guest's NVIDIA display path is closed by design, and +GZDOOM-class workloads cannot render on the guest GPU. +## 7. UPDATE 2026-10-25 #2 (SESSION 66): the 0x56 pinned to the RS alloc path + +A new instrument closed the gap: the guest's `nvidia.ko` (DKMS build) is +**not stripped** - copy it out (`/lib/modules/<kver>/updates/dkms/nvidia.ko`), +`objdump -t`/`-dr` it, compute the module's `.text` runtime base from any +`/proc/kallsyms` symbol, and every guest-kernel function/offset becomes +addressable for live gdb breakpoints on the QEMU stub. With it: + +* `rpcAllocObject_v29_06`/`_rpcAllocObjectPrologue` are **exonerated** for + 0xc57e (their reject list is 0x73/0xc67*/0xc770/0xc77d only; the register + mapping `hClass = r8d` was verified live). +* The failing chain is pinned exactly: `rmapiAllocWithSecInfo` **enters** + with class 0xc57e, calls `serverAllocResourceUnderLock`, and that + function returns **0x56** - *before* the GSP-RM alloc RPC + (`rpcRmApiAlloc_GSP`, called from under-lock+0x447; the emulation never + sees an ALLOC for 0xc57e). The same probe shows 0xc57d/0xc570/0xc573 + return 0. +* So the last blocker for the window path is a **local RS-layer validation + inside `serverAllocResourceUnderLock`** (candidates: the inlined + privilege/class-support checks, `rmapiFixupAllocParams`' per-class fixup, + or `clientAllocResource`'s construct dispatch). The 0x56-producing + instruction is not yet pinned - a `stepi` loop over the QEMU stub is far + too slow and wedged the guest (recovered by a fresh cycle); the next + session should use conditional breakpoints on the function's call-return + sites (the approach in `gdbdiag19` works; the earlier run had truncated + addresses and needs the `0xffffffff00000000` base restored) or a + watchpoint on the status word. + +**gdb-stub field notes**: the stub is single-client; killing an attached +gdb leaves the VM stopped (recover with a detach-only connect); +`stepi` loops over the stub freeze the guest's userspace for minutes - +prefer breakpoints; every probe run with the caps-fill breakpoint changes +the driver's path, so keep the fill in each experiment. + +## 8. UPDATE 2026-10-25 #3 (SESSION 126): the fence fold landed - the service authors the Vulkan fence releases from the driver's own push bytes; the compute-dispatch shape decoded (direct C5C0 method stream, no inline QMD) + +**The fence wall is closed.** The client's per-frame fence wait polls a +16 KB BO word that a submit-push INC-5 semaphore group targets +(`[2005c017][0x45ba090][0][payload 1][0][0x02100001]`). The group sits in a +GPFIFO entry beyond the driver's kick boundary (GP_PUT=11 while the ring +already carries the entries it pre-built for the next submits, idx12..46), +so the GP_PUT-bounded pump walk never decoded it and a host-side assist had +to write the word. The service now sweeps that pre-built continuation +itself: `sema_gr_ahead` (default 64) walks entries beyond GP_PUT through the +ring's validated translation, accepts only pushes that pass the strict +method-stream gate, extracts the addr-0x5c RELEASE groups and applies them +forward-only through the standard pending ledger. Live evidence: the sweep +finds the groups and the ledger applies the fence word once per frame; +`VKFRAME seq=1..3/DONE` completes with no assist. The frames are still +black - the shader work itself is unexecuted. + +**The compute-dispatch shape (why nothing executes).** The workload is a +single compute dispatch (640x480 storage image). Its push is a DIRECT +method-stream programming of the Turing compute class: `SET_OBJECT 0xC5C0` +on subchannel 1, then the context windows (`0x7b4` local-memory window B, +`0x2a4` shared-memory window B, `0x270`), an immediate-config block, and +the code/const data runs (`0x3800`/`0x39C8` pairs plus a subchannel-0 data +port stream of `0x11c`/`0x114` runs in a separate 17 KB push). There is NO +inline-QMD write (no `0x318/0x31c/0x320..0x41c` run), so the launch capture +that feeds the physical-engine forward path never fires - and the CE-shaped +ops path is not even consulted for GR-class channels by construction. Real +pixels therefore require a translator that recognizes the C5C0 section, +builds the vendor launch the physical channel expects (the same prelude/QMD +shape the CUDA path already executes) from the section's fields and data +runs, and forwards it with the existing operand/relocation machinery. The +method vocabulary overlaps the physical prelude's existing knowledge +(`0x7b0/0x7b4`, `0x2a4`), so the landmarks for that translator are known. + +## 9. UPDATE 2026-10-26 (SESSION 130): the utility-channel LOAD_INLINE_DATA arm fires; the payloads are the driver's zero buffer; the SASS delivery is the remaining non-black gate + +**The arm was dead by branch order.** The utility channel's engine-side +uploads ride subchannel 2 (the A140/KEPLER_INLINE_TO_MEMORY_B lane), and +their operand group (`0x188/0x18c` destination, `0x180` line length, +`0x184` line count) was parsed only by an `else if (subch == 1 || subch == +2)` branch that could never run: the preceding `if (xlate && subch >= 2)` +TURING-copy parser also matches subchannel 2 and swallowed the group +unread. With `have_out`/`line_len` unset, the `0x1b4 +LOAD_INLINE_DATA` arm never opened (`saw=0`). The 0x180..0x18c group is now +parsed first for subchannel 1|2; the arm reaches its write decision and +authors (`writes=2` on r744: 256 B -> vidmem 0x1e0000, 12288 B -> vidmem +0x1c0000). + +**The payloads are stably zero - and that is the driver's zero buffer.** +Four independent reads (the sweep's channel path, the direct +registered-VAS walk - 22k attempts, never non-zero - the guest's own +`/proc/<pid>/mem` view of the client's mapping, and the guest's +`/proc/kcore` physical read) plus a whole-run 1 MB staging-BO poller and a +gdb hardware write-watchpoint that never fires agree: the driver never +writes the payload chunks for these entries. The 256 B size equals +`GF100_ZERO_BUFFER_SIZE`, and the driver's `gf100DownloadZeroBuffer()` +uploads exactly 256 B of zeros through this path. The arm therefore +applies zero payloads under a variant once-key (`h | 1`) - idempotent over +an empty destination - while a later non-zero payload for the same entry +still lands under its own key. + +**The non-black gate, precisely.** The launch's `prog=0xdfd8feb900` +resolves to vidmem `0x62b000`, which is empty; the compute shader SASS is +not delivered by any decoded entry. The utility channel's ring holds +exactly 14 entries (`GP_PUT=14`), of which two carry LOAD_INLINE_DATA - +both the zero objects above. The Vulkan (NVN) download path uploads via a +separate download state and points its destination view at the pitch-pool +allocation, so the SASS/constants i2m shape is a different destination +family than the observed zero-buffer uploads. Next steps: find the SASS +upload shape (pitch-pool destination, possibly on another channel or a +non-LID copy) and then the `n_data` descriptor/operand wiring +(section 8); with both, the physical forward can execute the dispatch. + +## 10. UPDATE 2026-10-26 (SESSION 131): the utility-ring translation is the live problem - the decoded LID entries are from a STALE ring page; the live ring is in vidmem + +**The ring is in vidmem, and its content varies per client.** The +utility channel's ring VA (0x4020000) resolves through the service's +learned map to a **vidmem** offset (PA 0xa0000, next to the channel's +USERD at 0x8fe00), and the pump's own entry read shows the live ring's +content: on the r750-era clients `entry[0] va=0x120100000 len=0x28` +(`sema-chan: entry[0] h=0xbeef0100 eng=0 raw=0000280120100000 +va=0x120100000 len=40`). A dump of the vidmem ring (dump_vidmem 0xa0000) +confirms exactly one entry there. + +**The decoded LID entries (the 14-entry ring at 0x4000000/0x4250000) are +from a STALE page.** The sweep's ring read (`svc_read_va` of the ring VA) +can resolve a **guest-RAM** page (e.g. PA 0x13003d000) that holds an +EARLIER client's ring - a coherent 14-entry ring whose LID uploads target +the Vulkan client's own allocations (dst 0x423f000 -> vidmem 0x1e0000, +dst 0x4448000 -> vidmem 0x1c0000, 256 B = GF100_ZERO_BUFFER_SIZE and +12288 B). Those entries are real, but they belong to a dead session: the +payload chunks they point at (VA 0x4250000/0x4250110 -> the session's +1 MB staging BO) read zero through EVERY path - the sweep's channel read, +the direct registered-VAS walk, the learned map, the guest's own +`/proc/<pid>/mem` view of the BO, `/proc/kcore`, a whole-run +high-frequency poller of every 1 MB BO the client maps, and a gdb +hardware write-watchpoint that never fires - because the session ended +and the BO's pages were freed/reused. **The payload is not missing from a +live upload; it is a stale-session artifact.** + +**The CUDA contrast proves the mechanism.** A CUDA client's LID rows on +the same boot show the arm working with real payloads: +`h=0x5c000019 i=2 va=0x2004002c0 len=0xb8 ... tail_va=0x200400378 +tail=0x1c t8=200125a6 00000001 ... saw=1 fired=1 zero=0 +dst=0xd2e00000` - the tail (the next GPFIFO entry) is CONTIGUOUS with the +push and holds a real method stream, the arm materializes it into the +high vidmem family, and the CUDA battery passes. + +**Next steps (ranked).** +1. Read the LIVE client's ring through the map (vidmem 0xa0000) during a + client run and decode its entries - that is the live upload path; the + sweep's `svc_read_va` source preference must not answer with a stale + RAM page (verify the ring page by content: the entries must lie in the + client's current VA allocations, and the ring must be the same page the + pump reads). +2. With the live ring decoded, the SASS upload shape follows; the + descriptor/operand wiring (`n_data=0`) remains the second gate. +3. `sema_util_dec_log` (96/boot) and `svc_gr_util_log` (24/boot) budgets + were exhausted early in multi-client boots, hiding later rows; make + them per-session for diagnosis. + +## 11. UPDATE 2026-10-26 (SESSION 131b): the A/B that names the frontier - CUDA's uploads land in the emulated vidmem, the Vulkan client's do not + +`dump_vidmem map` (new: one line per 1 MB block with non-zero content, +`nz` = non-zero dwords) on a fresh boot, same module: + +* **CUDA client (`victory_mail`, STATUS OK)** - the shadow fills up: + `blk 0x200000 nz=40811`, `blk 0x1400000 nz=178868`, + `blk 0x1500000 nz=96346`, `blk 0x2a00000 nz=22880`, + `blk 0x7800000 nz=98257`, ... (~450k dwords total). The CUDA driver's + uploads (LID i2m writes with real payloads + CE copies) materialize in + the emulated vidmem. +* **Vulkan client (`vk_frame120`)** - the shadow stays at the 14 + baseline dwords (the fence/counter words at 0xfee88, the GR param bank + at 0x385e08, one word at 0x4d0b10). **Nothing the Vulkan client uploads + reaches the emulated vidmem at all.** + +That is the frontier in one measurement: the Vulkan (NVN/GL) driver's +upload path is not among the writes the emulation can see. Combined with +section 10 (the utility ring lives in vidmem via the learned map but the +sweep can read a stale RAM page for it; the live ring holds a single +`0x120100000/40` entry, not the 14-entry LID ring), the next session must: +(1) capture the LIVE ring through the same source the pump uses while the +client is up and decode its entries (the 14-entry LID ring is a +stale-session page); (2) trace where the live client's upload writes go +(the map shows they are NOT in the first 256 MB of the shadow - check the +guest's own view of the client's staging BOs during the upload window); +(3) the descriptor/operand wiring (`n_data=0`) after that. + +## 12. UPDATE 2026-10-26 (SESSION 132): THE LIVE RING IS PROVEN (guest RAM, walk-confirmed); the vidmem "ring" was a false-positive pin whose dead entry the pump skipped 39 times; the two LID uploads are the driver's zero buffers by design; the code VA's mapping rotates and the shadow's lat cache can go stale + +**The live ring is in guest RAM and the sweep's decode was right.** +`walk_probe 0x4020000` (the client's own page tables, vas[0] +`h=0xfade0002 pdb=0x114039000`) resolves the utility channel's GPFIFO VA +to **guest RAM** (PA `0x11623d000` on r759; the page moves per run: +`0x104b55000` on r761) and its content is the 14-entry ring the sweep +decodes: entry0 -> push `0x4000000` len `0x8bc` (the methods), entry1 -> +the 256 B payload slot `0x4250000`, entry8 -> the 12288 B payload slot +`0x4250110`, ... - all pushes inside THIS client's own allocation window +(`PREV149-OUT ... off=0x4000000/0x4020000/0x4060000` in the client's own +VIDHEAP log). The decoded ring is live content, not a stale page. + +**The vidmem "ring" (PA 0xa0000) is a false-positive autopin.** On r759 +the VRAM shape scan found the channel's REAL USERD at vidmem `0xb0000`; +the userd-derived candidate loop then tried `0xb0000 - 0x10000 = +0xa0000`, whose entry0 (`push 0x120100000/40` - the CE-pool push +pattern) passed the plausibility gate and whose USERD probe hit the real +0xb0000 block, so the pin landed. The pinned entry's push reads all-zero +through every source; the pump retried 39 times and logged `skipping +stale entry h=0xbeef0100 va=0x120100000 len=40 after 39 failed rescues`, +while `sema_probe 0x4020000` then answered the wrong page +(`pa=0xa0000 aperture=vidmem`) and poisoned every later read (the sweep +included). Section 10's "stale session page" conclusion was this pin, not +the sweep. + +**Fixes landed (commit `72de0bed`).** (1) `svc_ring_entry_dead` + +`svc_ring_walk_live`: a candidate whose entry0 push provably reads +all-zero through the client's page tables is rejected when the VAS walk +resolves the ring VA to a page with a live entry (applied at the +userd-derived, FB-convention and existing-pin paths; the walk cannot vote +for itself); (2) `svc_gr_ring_candidate` rejects dead-entry candidates +and accepts the channel's known USERD cross-aperture (the driver places +GPFIFO and USERD independently: ring in guest RAM, USERD in vidmem); (3) +the util sweep arbitrates a dead channel-path ring entry against the +direct VAS walk per slot; (4) the util-dec/util-sweep witness budgets and +their dedup keys are per-client now (reset in `svc_session_flush`); (5) +`sema-copyent` logs once per copy OP, not per row - one 480-row frame +copy had exhausted the 128/session budget and hidden every later copy. +Verified on r760/r761: `sema-chan: h=0xbeef0100 ... ring rc=0 +pa=0x108cba000 ap=1` (the live RAM page), `sema-rdw: h=0xbeef0100 +va=0x4000000 len=2236 rc=0`, the sweep decodes the live 14 entries, the +fence fold releases fire, and the client completes (`VKFRAME_DONE +frames=3 640x480 saved=3`, `VKFRAME_IDLE ok`). + +**The two utility-channel LID uploads are the driver's zero buffers by +design, not the SASS.** Live decode (r760): i=0 -> `dst=0x423f000 ll=256 +lc=1 zero=1` (256 B = `GF100_ZERO_BUFFER_SIZE`) and i=8 -> +`dst=0x4448000 ll=12288 lc=1 zero=1` (12288 B = the CE fault method +buffer size). The payload slots (`0x4250000` -> RAM `0x116219000`; +`0x4250110`) read zero for the whole client run - the driver never +writes them. The GR channel's own LID entry DOES carry a real payload +(`dst=0x4b86020 ll=1568 zero=0` - the launch param bank at `param= +0x4b86000 size=0x1900`), which proves the arm works with real bytes on +this boot too. + +**Where the code page stands.** `prog=0xdfd8feb900` (the captured +launch): `sema_probe` answered vidmem `0x62b900` early in the run while +`walk_probe` answered vidmem `0xc8b000` later - the driver's code-buffer +mapping ROTATES (a code pool), and the shadow's learned lat entry for the +VA can go stale. Both pages read zero in the shadow: the driver's code +upload has not been observed on any decoded path (no LID, no CE copy - +the only `sema-copyent` op of the whole run is the 480-row frame +copy-out `dst=0x50b0000 src=0xdfd8d60000`, which executes `rc=0 +bytes=0x12c000` and correctly copies the still-empty image). The +remaining question is the SASS delivery shape for the C5C0 compute path +(section 6's `0x3800/0x39C8` code-run/data-port semantics) and the stale +lat preference for the rotating code VA. + +**Next steps (ranked).** +1. Resolve the code VA through the VAS walk at launch time (the walk is + the live truth; the lat entry goes stale as the pool rotates) and + watch the code page the QMD actually names. +2. Find the SASS delivery: dump the C5C0 section's `0x3800/0x39C8` runs + with a bounded methods witness while the client is live (the params + arrive by LID; the code must arrive by a sibling path). +3. Then the `n_data=0` descriptor/operand wiring (section 8). + +## 13. UPDATE 2026-10-26 (SESSION 133): THE SASS DELIVERY SHAPE IS FOUND - the driver embeds the program bytes INLINE in a CE LAUNCH_DMA push and copies them to vidmem; the CE channel's ring/USERD resolution is the boot-lottery gate that keeps the copies from running + +**The delivery shape (decoded from the CE channel).** A CE-class channel +(eng=11) carries, as entry[4] of its ring, a push whose method stream is a +LAUNCH_DMA copy with the SASS bytes INLINE: + +``` +sema-chan: entry[4] h=0xcaf00012 eng=11 raw=0021340120a00000 va=0x120a00000 len=8500 +sema-bigrun: h=0xcaf00012 push=0x120a00000 off=0x0 hdr=0x64000002 op=3 cnt=1024 subch=0 addr=0x8 +sema-lda: h=0xcaf00012 eng=11 push=0x120a00000 dma=0x00002181 xfer=1 dstphys=1 srcphys=0 out=0x205000 in=0x120a00004 line=4096 have_out=1 have_in=1 +sema-opadd: h=0xcaf00012 type=1 dst=0x205000 src=0x120a00004 bytes=4096 ap=0 src_ap=0 phys=1 src_phys=0 +``` + +The source is the push itself (`in=0x120a00004`, right after the method +header) and the destination is a vidmem VA (`out=0x205000`, then +`0x206000`, ... 4096 B per run) - the engine-side "download program +range" path. The Vulkan client's CE channels carry the SAME shaped +entries (r763: `sema-utildec: h=0xbeef0101 i=4 va=0x120a00000 len=0x2134`, +19 entries swept) - so the SASS delivery is a CE inline copy, not the +utility-channel LID entries (those are the driver's zero buffers, +section 12). + +**Why the copies never ran (three gates).** (1) The CE channel's ring and +USERD live in VIDMEM and resolve only by boot-lottery: the FB convention +matched on r763 (`auto-pinned ring h=0xbeef0101 rank=0 va=0x4100000 -> +fb 0x80000 (entry0 va=0x120000000 len=40)`; entries then decoded and the +frame path ran) but read zeros on r765/r767/r768 - the USERD shape scan +instead finds the channel blocks at 0xfee00/0xff000/0xff200 (r767/r768), +so the rings sit at unaligned offsets like 0xefe00. (2) The CE push VAs +(0x120000000+) need the client's VAS registered: the parked +COPY_SERVER_PDES candidate only registered at +15 s (age-out, +`sema_vas_cand_ms`) while the entry hold grace (`sema_ce_grace_ms`, +8000 ms from the cycle) expired first - r763 logged `skipping stale +entry h=0xbeef0101 va=0x120000000 len=40 after 8000 ms grace` and +`sema-pushfail: va=0x120a00000 ... trusted_rc=-2 ipin=0` for the copy +pushes. Setting `sema_vas_cand_ms=5000` registers the VAS at +5 s +(verified: `VAS 0xfade0002 ... registered UNVERIFIED after 5000 ms +parked`) but the ring gate still blocks. (3) The shadow ring scan +fallback (commits `52ef3f9d` + `2735180a`: 0x100 stepping, rate-limited +retry, entry0 shape + USERD probe at +0x10000/+0x2000/+0x1000) found no +candidate on r768: the ring/USERD content is volatile at scan instants +(a `find_userd` read shows `off=0xfee00 get=11 put=11` while a +`dump_vidmem 0xfee00` one second later reads zero), so a periodic scan +misses the window. + +**Next steps (ranked).** 1. Derive the CE ring from a USERD found by its +own shape (put at +0x8c, get at +0x88 - the `find_userd` scan already +does this) instead of scanning for ring entries: for every USERD-shaped +block, try ring = userd - {0x10000, 0x2000, 0x1000} and validate entry0 ++ the probe. 2. Or latch the ring at the first CE submission (the ring +content appears exactly when the driver writes entry0). 3. With the ring +latched and the VAS at +5 s, verify the inline copies execute +(`sema-lda`/`sema-opadd`/`sema-copyent` with the code dsts) and the code +page gets content; then the `n_data` descriptor wiring (section 8). + +## 14. UPDATE 2026-10-26 (SESSION 134/135): the CE first-submission capture landed and its rank-blind first-hit caused a CUDA-client stall regression (fixed); the Vulkan client's CE rings are allocated ABOVE the capture scan's 64 MB bound and the frame chain reduces to that un-latched ring + +**SESSION 134 (commits `36642718`, `928c0730`).** A new instrument fires in +the notifier-completion path for CE channels *before* the clear write - the +one instant the ring content and the USERD are both live - reads entry0 +through the channel translation and the trusted walk, then scans the shadow +(0x100 step, first 64 MB) for a ring-shaped page (entry0 naming a 1 MB +aligned push VA in `[0x120000000, 0x121000000)` with a sane length) and runs +the standard USERD gate at `+0x10000/+0x2000/+0x1000/+0x0`; a validating +candidate is pinned/latched exactly like the regular CE autopin. `928c0730` +fixed the entry decode: the push VA is 40 bits, the scan had declared it +`u32`, so `scan_hits` was zero by construction. + +**SESSION 135 (commit `a674c21c`) - the rank-blind first-hit regression.** +The scan started at offset 0 and accepted the first ring-shaped page for +*every* CE channel. On r779 both `caf00012` (rank 0) and `caf0001d` +(rank 1) latched `off=0x80000 upa=0x90000`: rank 1 read rank 0's +ring/page, its first release decoded to the wrong semaphore word (+0 +instead of +4) and was dropped as stale (`cur=1`, already written by +rank 0), so the client's rank-1 tracker wait never completed. victory_mail +then spun at 100% CPU with `pub_calls=0` on three consecutive boots +(r777/r778/r779: battery pass=0 fail=7). A control run of the r746-era +module (`25523f84`) on a fresh boot passed the same client (VADD +`bad=0 checksum=12204516.000000`, `STATUS OK`), isolating this to the +SESSION 133/134 code. The fix tries the rank's convention offset +(`0x80000 + 0x20000 * rank`) first, scans the rest bounded as before, and +never accepts an offset another CE rank already latched this session; an +unlatched rank stays for the regular per-pass latch. Verified on r781: +ranks 0..5 latched correctly (0x80000/0xa0000/0xc0000/0xe0000/0x100000/ +0x120000, USERD +0x10000 each), rank 1's own semaphore word applied +(`pool-wr va=0x121000004 cur=0 -> val=1 ok`), and the client progressed +(`pub_calls=185132+`, no spin). + +**The Vulkan CE ring on the same boots.** r781 (classic autopin, rank 0): +`auto-pinned ring h=0xbeef0101 rank=0 va=0x4100000 -> fb 0x80000 (entry0 +va=0x120000000 len=40)` - a live entry - but the client's pushes decoded +`unread=1 injected=0`: the guest pin scanner covered CUDA clients +(`cdt,va,victory`) and never `vk_frame`. Extending the scanner to `vk` +engaged the tools (demand pins active) but r782/r783's captures reported +`scan_hits=0 latch=0`: no ring-shaped page anywhere in the shadow's first +64 MB, and no classic autopin either. + +**Where the Vulkan rings actually are.** The client's own vidheap ioctl +log (`PREV149-OUT` lines, r783) names the CE/GR channel allocations: +`off=0x4020000/0x4100000/0x4160000` - matching the registered gpfifos +(`0x4020000/0x4100000/0x4160000`) and lying just ABOVE the capture's +`lim = min(shadow, 0x4000000)` (64 MB). Direct dumps of those offsets (and +of `0x80000`) read zero, and a byte search of the full 4 GB guest-RAM image +for the entry pattern (`00 00 00 20 01 28 00 00` / the `0x20100000` +sibling) finds nothing - the ring content reaches neither the shadow at its +allocation offset nor guest RAM. + +**The frame chain (r783, live).** The Vulkan client completes three frames +(`VKFRAME_DONE frames=3`, `VKFRAME_IDLE ok`) but all PPMs are pure black. +The live GR channel (`h=0xbeec0100`) shows the dependency explicitly: +`sema-gr: launch captured ... prog=0xdff0feb900 ... inline=8` together +with `sema-gr: copy pend h=0xbeec0100 push=0x4f4580c src=0xdff0d60000 +dst=0x50b0000 row=0xa00 rows=480 ...` repeated ~20/s. The trusted walk +maps the copy source to VIDMEM (`walk_probe 0xdff0d60000 -> rc=0 +pa=0x570000 aperture=vidmem`) and the destination `0x50b0000` is the +framebuffer. The shadow at `0x570000` reads zero: the data pages are the +ones the CE inline copies are supposed to fill, and the GR launch is held +on their absence (the designed `dstpend` dependency). So the whole Vulkan +chain reduces once more to the un-latched CE ring: no ring -> no inline +i2m copies -> no data/code pages -> held launches -> black frame. + +**Next steps (ranked).** 1. Cover the client's real allocation in the ring +capture: raise the capture scan bound above the recorded channel +allocations (e.g. `0x6000000`) and/or seed candidates from the client's +vidheap offsets, then re-check whether the entry arrives at the +submission instant (a live `sema_vidmem_dump` at `0x4100000` while the +client is streaming would settle it; the post-run dumps read zero). 2. If +the shadow still reads zero at the allocation offset during live +submissions, the client-side write watch is the remaining forensic: find +the CPU pointer the driver writes GPFIFO entries through, its PA via the +guest pagemap, and why the GPU PTE/walk does not name that PA. 3. With a +latched ring, verify the inline copies execute (`sema-lda`/`sema-opadd` +with the code dsts), the code/data pages fill, then the `n_data` +descriptor wiring (section 8) → non-black frames. 4. Re-run the CUDA +battery on the fixed module (the r781 boot was a slow-crawl boot; the +spin stall class is gone) and revisit `sema_gr_xlate`'s multikernel +forward timeout. + +## 15. UPDATE 2026-10-26 (SESSION 135b): every "Vulkan CE ring latch" on record was a STALE CUDA BLOCK - the Vulkan CE ring content has never been observed; the write path is not BAR1 and the content is in no shadow, no RAM, and not at the PTE target + +**Forensics on four fresh boots (r782-r786; commits `a674c21c` in tree, `c3cdcf4b` new).** + +*The write path works; the driver does not use it for the ring.* A direct +BAR1 test - the guest writing a known pattern at `BAR1+0x4100000` through +`/sys/bus/pci/devices/0000:01:00.0/resource1` - lands in the emulated +vidmem shadow verbatim (`RINGTEST` + `AAAA...` read back by both the guest +and `sema_vidmem_dump`), so the mdev's BAR1 region path (mmapped, direct +into `bars[1].vaddr`) is sound at those offsets and at `0x80000`. A +high-frequency watcher (12 candidate offsets, ~2.5 Hz, including +`0x80000/0xa0000/.../0x4020000/0x4100000/0x4160000` and the +0x10000 +USERDs) recorded ZERO nonzero dumps across the whole live streaming window +of a clean Vulkan run (r786; three frames, `VKFRAME_RATE fps=0.05`, +`VKFRAME_DONE frames=3`, black PPMs). The `shadow_scan` over all three BAR +shadows (0/1/3) for the CE push header `{0x120,0xb5c50000,...}` and the +ring-entry pair `{0x20000000,0x2801}` finds nothing, and a byte search of +the full 4 GB guest-RAM image for both patterns (little-endian, corrected +from an earlier reversed search) finds nothing either. The walk and the +learned map resolve the CE ring VA (`0x4100000`) to a guest-RAM page +(per-boot PA, e.g. `0x135726000`) that reads all zero, and the vidheap +ioctl log (`PREV149-OUT`) names the channel allocations +`0x4020000/0x4100000/0x4160000` on every boot - so the ring content is +written neither to the shadow at its allocation offsets nor anywhere in +RAM reachable through the guest's own page tables. + +*A stale-block trap explains every historical "latch".* On CUDA-first +boots the capture latched `fb 0x80000 / userd 0x90000` for the Vulkan +channel `beef0101` with `put=1 get=1` - the dead CUDA client's `caf00012` +block surviving in VRAM after its session. The decoded entry +(`va=0x120000000 len=40`) is the old session's (both clients' first CE +pushes are shape-identical), the expectations seed from the stale pool +words (`want=2/3` vs `pay=1`; the word page read 3-6), and the Vulkan +client stalls into its fence timeout (`VKFAIL vkWaitForFences -> 2`) on +every run. r763's and r781's "auto-pinned ring h=0xbeef0101 -> fb 0x80000" +rows are the same artefact: both were CUDA-first boots. All genuine +first-submission latches on record read `put=1 get=0` at the capture +instant. `c3cdcf4b` rejects a consumed-state candidate (`get == put > 0`) +in the first-submission capture - a just-submitted entry cannot be +consumed at the notifier-clear instant. + +*The frontier is now precise.* The Vulkan client's CE submissions complete +only through the fabricated completion machinery (the driver never reads +its ring back), its CE ring content is invisible to every host source, and +the GR launch for each frame is held on the unexecuted CE i2m copies +(`copy pend src=0xdff0d60000 dst=0x50b0000`; walk: src -> vidmem +`0x570000`, zero). Next: the client-side write watch - find the CPU +pointer the guest driver writes GPFIFO entries through, its PA via the +guest pagemap, and why neither the GPU page table target nor any shadow +holds the result (guest-kernel gdb via the QEMU gdbstub or a kprobe on +the driver's ring-write helper). + +## 16. UPDATE 2026-10-26 (SESSION 137): THE CE-RING HYPOTHESIS FOR THE VULKAN CLIENT IS DISPROVEN BY DIRECT WRITE WATCHES - no kernel and no client writer ever touches the ring page + +**The write-watch toolchain (all three instrument classes).** (1) kprobes: +`channelFillGpFifo` (the kernel's GPFIFO-entry writer, called from the CE +and SEC2 submission paths per the open driver sources) is marked +`notrace` - `trace_kprobe: Could not probe notrace function +channelFillGpFifo [nvidia]` - so kprobes and ftrace can never attach +(this is why the named helpers "never fired" in earlier sessions: they +could never register). (2) QEMU-gdbstub hardware breakpoints work: a +verification `hbreak *__x64_sys_ioctl` fired on guest ioctls +(`IOCTL#1 pc=0xffffffffbc316160`). An `hbreak` on the exact entry store +(`channelFillGpFifo+0xb5`, `mov %edi,(%rdx)`, where `%rdx = +pbCpuVA + channelPbSize + lastSubmittedEntry*8` per the disassembly and +the source) stayed silent across a complete Vulkan run - even though that +same run produced the CE channel's notifier clear +(`sema-chan: notifier cleared h=0xbeef0101 info32=0x5`) and the +first-submission capture (`sema-cefirst: h=0xbeef0101 rank=1 ... +scan_hits=0 latch=0`). (3) A client-side hardware watchpoint: the ring +VA resolves through the guest page tables to a guest-RAM page that is +mapped into the client process (`/dev/nvidiactl` mapping; e.g. PA +`0x101021000` -> client VA `0x7fffee69b000`; a later run PA +`0x10087b000` -> VA `0x7fffee7d8000`), and the walk and the emulation's +own read agree the page is ZERO. An in-client gdb (the t122 recipe plus +a file-fed watchpoint) armed `watch *(unsigned long*)0x7fffee7d8000` +from the client's FIRST ioctl (`Hardware watchpoint 2` + `RGWATCH-SET` +logged before `GLBASE`) caught ZERO writes across the whole run +(3 frames, `VKFRAME_DONE`, IDLE). + +**Consequences.** On vk-first boots the client creates its CE channel but +never submits CE work: no GPFIFO entry is written by the kernel +(`channelFillGpFifo` never executes), nor by the client (hardware watch, +zero hits), nor is any content found in any BAR shadow or in RAM. The +historical "Vulkan CE ring latches" were the dead CUDA client's stale +blocks (section 15). The image/upload path for the Vulkan client rides +its GR channels (`launch captured ... inline=8`; the frame copy held as +`sema-gr: copy pend ... src=0xdff0d60000 dst=0x50b0000 rows=480`), and +the non-black gate must be attacked on that path - the data/code pages +those launches depend on are never filled because no CE i2m copies are +ever queued, and the launch is held on their absence (dstpend/n_data). +Next: instrument the GR launch's dependency resolution (which pages the +dstpend gate waits for, and whether the inline records the capture holds +are sufficient to fill them) instead of hunting a CE ring that nothing +writes. + +## 17. UPDATE 2026-10-26 (SESSION 138): THE GR LAUNCH'S WIRING IS PRINTED - one QMD-window page carries prog+pointer data (deduped and never scanned), its only fill candidate is an unresolvable inline word, and the backend refuses `n_data=0 (min 16)` + +**The witness (commit `10b09217`, sema_dbg-gated).** Per launch, bounded: +the candidate handle, `param_va`, `prog_va`, `n_pointer` + QMD pointer +pairs, inline count, and the resolved `gr_pages` list; then the fill's +built candidate set with the arg flag. + +**First capture (r788, Vulkan `h=0xbeec0100`, push=0x4f4509c).** +``` +sema-gr: cand h=0xbeec0100 param=0x4b86000 prog=0xdff0feb900 n_ptr=1 inline=8 n_pages=2 push=0x4f4509c +sema-gr: ptr[0] va=0xdff0febc00 word=34 +sema-gr: page[0] va=0x4b86000 pa=0x386000 ap=2 qmd_word=32 +sema-gr: page[1] va=0xdff0feb000 pa=0x56b000 ap=2 qmd_word=48 +sema-gr: fill cand n=1 pages=2 push=0x4f4509c +sema-gr: win[0] va=0x43f00000 arg=1 +``` +Reading: the launch's page list is the param page and the +prog/QMD-window page (both resolving as `ap=2`, i.e. vidmem). +`ptr[0]` (QMD word 34, `0xdff0febc00`) shares its 4K page with the program +page, so `svc_gr_page_note` dedupes it into the `qmd_word=48` entry - and +the fill's scan passes cover only `qmd_word==32` and `32..47 even`, so +that page's content is never scanned for data pointers. The fill's single +candidate comes from the inline bytes instead (`0x43f00000`, arg=1) and +resolves in **no** vaspace (`svc-gr arg-miss base=0x43f00000 probe=none +... vas[0..3] rc=-2`), so `n_data` stays 0 and the backend refuses: +`vgsp-gr: GR submit refused: degenerate data set n_data=0 (min 16) +launch_va=0x4f4509c`. + +**Content state.** The shadow at the param page (`0x386000`), the arg +region (`0x386150`), the program area (`0x56b900`) and the pointer/data +area (`0x56bc00/0x56bd00`) reads all zero on this boot - the launch's +data and code bytes exist in no host-visible source (consistent with +section 16: nothing queues the CE i2m copies for the Vulkan client). + +**Next.** (1) Scan the deduped QMD-window page (or scan a page once under +every qmd_word that names it) so the pointer-region data words enter the +candidate set; (2) resolve the inline/dst candidates through the vidmem +serving lane (the pages ARE resolved to vidmem PAs - the missing piece is +the candidate list, not the resolver); (3) drive `n_data` toward the +launch's `min 16` (the window's expected page count) - and where the +window content must come from is the open question the byte-state above +poses. + +## 18. UPDATE 2026-10-26 (SESSION 139): THE FILL'S CANDIDATE SOURCE IS DISPROVEN - `0x43f00000` IS THE FLOAT 480.0f; after the float-shape filter the candidate set is EMPTY and the wall is content, not wiring + +**The false candidate (commit `add22329`).** A full dump of the launch's +param page (vidmem `0x386000`, r789) holds exactly two nonzero spots: +`+0x30: 08 00 00 00` and `+0x630: cd cc 4c 3e | 00 00 20 44 | 00 00 f0 +43 | 00 00 00 00` - i.e. the f32 scalars **0.2f, 640.0f, 480.0f** (the +640x480 viewport!). `0x43f00000` is 480.0f: the fill's only candidate +("arg base=0x43f00000 resolved 0 pages") has been a **float scalar +misread as a device VA** by the scan's shape filter (`>= 0x10000000`, +4-aligned). The GRCE inline write that does fire (`dst=0x4b86020 -> +pa=0x386020 bytes=1568 data8=0000000000000000`) writes the driver's +zero stub (the push carries only an 8-byte nonzero inline fragment at +`+0xb0`; the rest reads zero - the SESSION 130 zero-buffer finding). + +**The fix and verification.** All four scan sites (the three page scans +plus `svc_gr_scan_cand`) now reject float-shaped words (f32 exponent byte +in ~`[0x6c..0x8f]` = 2^-19..2^16; device VAs as f32 land in subnormal +exponents), and the bank-page scan covers word 48 (the deduped program +page). r790: **`sema-gr: fill cand n=0`** - no false candidates, no false +demand-loop traffic; the launch still refuses (`GR submit refused: +degenerate data set n_data=0 (min 16)`). + +**The wall, precisely.** The launch's real data pointers are not in any +scanned source: the param page holds scalars; the QMD pointer word +(`0xdff0febc00`) is one window buffer whose content is never written; the +inline payload is the zero stub. The content could only arrive via the +driver's i2m/LOAD_INLINE_DATA copies (which the Vulkan client never +queues - SESSION 137) or from the real GPU (physical forward, which the +n_data gate refuses first). This is now a pure content-source problem: +either find/queue the i2m that fills the QMD window, or break the +chicken-and-egg by forwarding a launch with the window zero (letting the +real engine read it) behind an explicit parameter. + +## 19. UPDATE 2026-10-26 (SESSION 140): THE n_data WIRING - the QMD's own pointer values ARE the launch's data set; the fill now reaches them (candidates appear, the page is correctly launch-internal), and the degenerate-set guard's bypass reaches the backend only to oops in the empty-set walk + +**Three wiring corrections (commit `5c0fcc21`).** (1) The QMD page itself +is noted into the request page list (word 34) - the pointer values live +inside the QMD (r790: word 34 = `0xdff0febc00`) and no scan ever covered +it. (2) The fill seeds candidates directly from the launch's own QMD +pointer pairs (`gr_qmd` words 34..47, decoded from the push's inline-QMD +run): r793 `sema-gr: qmd-ptr seed va=0xdff0feb000 (word 34)` - the values +need no page scan (the page shadows read empty; the +invisible-vidmem-write class). (3) The scans read pin-aware +(`svc_read_va` first) - the staging copy the decode reads is a pinned +page; a raw mapping of the vidmem pa sees zeros. + +**Result.** `fill cand n=1` with the QMD window page seeded; +`sema_probe 0xdff0feb000 -> pa=0x56b000 aperture=vidmem` resolves - but +the R3 hygiene filter classifies the page launch-internal, so `n_data` +stays 0: **this launch's QMD declares exactly one pointer pair and it +names its own QMD window - a launch with no external data, which the +"degenerate set" refusal class wrongly covers.** + +**The egg-break test + the oops.** With the r564 guard bypassed +(`gsp_phys_gr_min_data=0` - and the value clamp removed for the test -) +the refusal disappears and the forward ENTERS the backend: +`Oops: 0000 [#2] PREEMPT SMP NOPTI`, `Workqueue: gsp-phys-work +svc_fwd_work_fn [vgsp]`, `RIP: gsp_phys_backend_guest_gr_submit+0x4394` +(NULL-base table walk: CR2=0x1e93c, R14=0, RAX=0x1e000, RSI=0xdff0feb900 +- the program VA). The empty-data walk needs that guard before +`min_data=0` can be used; the clamp revert was withdrawn (crash +enabler). The copy pipeline is unaffected: `sema-gr: copy done +h=0xbeec0100 push=0x4f4580c dst=0x50b0000 bytes=0x12c000 rows=480 rc=0` +runs cleanly in the same runs. + +**Next.** Guard the empty-set walk at `gsp_phys_backend_guest_gr_submit` +(the struct walk reading 0x04/0x10/0x38/0x3c off a NULL base with the +program VA in RSI), then re-test `min_data=0` for the no-external-data +launch class: the forward should reach the real engine, and the copy-out +(and the frame copy that already executes) can carry rendered pixels +back. + +## 20. UPDATE 2026-10-26 (SESSION 143): THE r800 GUARD HOLDS AND THE `min_data=0` BYPASS REACHES THE REAL ENGINE - IT FETCHES THE PROGRAM AND MMU-FAULTS WITH `FAULT_PDE` AT THE PROGRAM VA; THE WALL IS NOW THE PROGRAM PAGE'S MAPPING AND CONTENT, NOT THE DATA-SET GATE + +**The test (r801b).** `gsp_phys_gr_min_data=0` is now an explicit bypass +(commit `34e611f4`; 0 previously clamped to 1). On a clean cycle +(module from `34e611f4`, full params, `sema_gr_xlate=1`, +`sema_gr_util_scan=0`, `sema_vas_cand_ms=5000`, `sema_ce_grace_ms=20000`, +`sema_ce_hold_ms=0`, `sema_ops_log=8`, `sema_dbg=Y`, `min_data=0`) the +Vulkan client ran once: + +* **No oops.** `grep -c Oops` = 0 for the whole run; the r794 + `gsp_phys_backend_guest_gr_submit+0x4394` crash class is gone. +* **The refusal is gone.** Zero `GR submit refused: degenerate data set` + lines; the launch is submitted: + `GR LAUNCH submitted h=0xbeec0100 ... pages=3 shadow=1 + qmd_va=0x4b85f00 slot=3 words=239 ... qmdwin=0x108000`. +* **The engine ran and fetched.** After ~2.5 s: + `GR LAUNCH FAILED: timeout after 2500 ms (qmd_rel=0x00000000 + want=0x600df00d ...)`, and the real firmware reported: + `vgsp-fault: func=0x1006 ... errString="Ch 00000001, intr 00000000. + MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_GCC faulted @ 0xdf_d8feb000. + Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ"`. +* **The copy-out still works:** `sema-gr: copy done h=0xbeec0100 + push=0x4f4580c dst=0x50b0000 bytes=0x12c000 rows=480 rc=0`. +* **The client now fails earlier than the black-frame runs:** it reports + `VKDEVICE ok family=0` then `VKFAIL vkWaitForFences ... -> 2` and no + frame is saved (the service synthesizes a client-visible fault for the + failed forward, where the refusal path used to software-complete and + let three black frames through). + +**The launch wiring measured on this boot.** +``` +page[0] va=0x4b86000 pa=0x386000 ap=2 qmd_word=32 (param) +page[1] va=0xdfd8feb000 pa=0x62b000 ap=2 qmd_word=48 (program) +page[2] va=0x4b85000 pa=0x385000 ap=2 qmd_word=34 (QMD window bank) +``` +The program page resolves in the emulated vidmem at `0x62b000`, whose +backing host page (`0x5e0504000`) reads ZERO; `CODEHASH ... h0=0 ... +SUSPECT (first 16B)` and `FAULT-CAP ... prog_w48=0xd8feb900 +prog_pa=0x5e0504000 code_src=guest-hpa off=0x900 code64=000...` confirm +the source is empty. The QMD deposit keeps the guest program VA +(`w48/49=d8feb900 000000df`); in the retry only the `qmd_word 34` page is +copied to the window-0 shadow (`qmd_word 34 page copied to window-0 +shadow VA=0x1e8000`), so the program pointer is left at the guest VA and +the engine's page tables have no PDE for it. The post-mortem program +target reads `slot=30 ... w0..15 = 0`. + +**Reading.** The data-set gate and the r800 guard are settled: the +no-external-data launch is legitimately forwarded and the engine's +channel/context are alive (it fetched the QMD from the host deposit and +issued a real MMU fault). The remaining wall is exactly two coupled +facts: +1. **Mapping** - the engine's page tables do not map the guest program VA + (`FAULT_PDE`), and the host keeps that VA because the code source is + an empty page (the shadow rewrite is only used for sources it can + trust). +2. **Content** - the program page is empty in every host-visible source. + The driver's program-upload path stages the ucode and copies it to the + code heap (the observed CE `LAUNCH_DMA`/inline shape); that copy has + not been observed executing for the Vulkan client. + +**Next.** (1) Find and execute the Vulkan client's program-upload copy +(the staging->code-heap CE copy) so the program page becomes non-zero - +the utility/GR sweeps are the instruments, and the code page becoming +non-zero is the milestone. (2) Then map/relocate the program VA for the +engine (or install the PDE) so the fetch succeeds. (3) Then non-black +frames -> Xorg/GZDOOM -> KVMFR/LG -> CUDA 15/15 + soak -> the 1.0.0 +video -> Windows. Until (1)/(2) land, keep `min_data` at its default 16: +the bypass forwards a launch the engine cannot complete, which is worse +for the client than the software-completion fallback. + +## 21. UPDATE 2026-10-26 (SESSION 143b): THE COPY-CLASS CHANNELS ARE REGISTERED, SWEPT AND NEVER PUSHED; NO CHANNEL CARRIES PROGRAM BYTES OR A LAUNCH_DMA; THE CODE PAGE NEVER FILLS - AND THE REFUSAL PATH NO LONGER PRODUCES THE BLACK-FRAME BASELINE + +**The instrument (commit `c5510744`).** The util sweep now logs one-shot +`sema-utilsweep: begin|skip|read=none` for every channel plus a 128-row +per-pass cap, so "absent from the log" can no longer be confused with +"absent from the channel table". + +**Channel visibility (r803, full params, `util_scan=64`, `min_data=16`).** +Ten channels in the table, all with gpfifo/entries, **all USERD +unresolved** at first sight: +``` +begin h=0xbeef0101 eng=11 gpfifo=0x4100000 entries=32768 userd=0 +begin h=0xbeef0100 eng=0 gpfifo=0x4020000 entries=32768 userd=0 +begin h=0xbeef0102 eng=12 gpfifo=0x4160000 entries=32768 userd=0 +begin h=0xbeee0101 eng=11 gpfifo=0x4500000 ... +begin h=0xbeee0102 eng=12 gpfifo=0x4560000 ... +begin h=0xbeed0101 eng=12 gpfifo=0x49c0000 ... +begin h=0xbeee0100 eng=0 gpfifo=0x44a0000 ... +begin h=0xbeec0104 eng=0 gpfifo=0x4d20000 ... +begin h=0xbeec0100 eng=0 gpfifo=0x4cc0000 ... +``` +The Vulkan client's copy-class channels **0xbeef0101 (eng 11) and +0xbeef0102 (eng 12) are in the table and swept** - and their rings carry +nothing all boot: `read=none` on first sight, `swept=0` on every pass, no +`utildec`, no LID, no copy. The client posted one event on 0xbeef0101 at +registration (`post_event hSrc=0xbeef0101`) and never pushed. + +**What did decode.** All boot: 3 LID writes only - the driver's two zero +buffers (256 B -> vidmem 0x423f000, 12288 B -> 0x4448000, both +`zero=1`) and the launch's own param bank (1568 B -> 0x4b86020, +`zero=0`). No `sema-lda`/`sema-opadd` rows at all, no `copy-in` rows, no +`utildec` on the two CE channels. Global swept=1652, writes=3. + +**The code page never fills.** `0x62b000` (the launch's program page for +`prog=0xdfd8feb900`) and the rotating-pool page `0xc8b000` read ALL-ZERO +before the client, after it, and at a final re-dump; the whole 256 MB +aperture holds only 12 non-zero dwords boot-wide (the viewport scalars +`0x44200000`/`0x43f00000` at 0x386634/0x386638, the releases, the +baseline). Nothing anywhere writes program bytes. + +**Client outcome.** `VKDEVICE ok family=0` then +`VKFAIL vkWaitForFences ... -> 2`; no frames, no PPMs. The service +answered `GR-COMPLETE-SW` 10 347 times for the same refused push while +the client kept waiting. This is the same outcome as the r801b forward +test, and it differs from the earlier black-frame baseline +(`VKFRAME seq=1..3`, three black PPMs): on a refused/failed forward the +service authors a client-visible `SYNTH-FAULT` (commit `c6c152cc`, +`sema_refusal_fault=1` since r618), and the current completion/release +path does not get the client past its fence. `util_scan` is excluded as +the cause (r801b failed with it off). + +**Reading.** The Vulkan client *believes* it uploaded its compute shader +(it dispatches with a valid program VA), but no CPU write, no CE LID, no +CE copy and no GR i2m in any observed channel ever delivers the program +bytes to the emulated vidmem. The driver's upload path is a +`staging ring -> code heap` copy (CPU memcpy into a staging BO, then a +GPU copy to the code heap); the copy is not among the decoded pushes and +the staging BO is not visible in the emulated vidmem. Two separate +questions now stand: +1. **Delivery** - where the driver queues that copy (a CE channel it + never pushes, a GR subchannel/method shape the parser does not + decode, or a sysmem staging BO that a different read path would + expose). +2. **Completion regression** - restore the refusal-path software + completion (or gate the synthetic fault) so the client reaches the + fence-released black-frame baseline again; this is needed for every + downstream observation. + +**Next.** (a) Dump and decode the client's push method stream around the +upload (the `staging ring -> code heap` copy) rather than more ring +sweeps; the staging BO's CPU write is the most likely host-visible copy +of the SASS. (b) Instrument the completion path for one refused Vulkan +launch (release ordering vs the synth-fault) to restore the black-frame +baseline. (c) Only then resume the mapping work (the r801b `FAULT_PDE` at +the program VA). + +## Session 154 (2026-11-02): formation protocol, execution regression, first engine-final stores + +Three durable findings from the r1414-r1427 experiment series (host module +`vgsp.ko` unchanged; insmod-config + protocol only): + +1. **Client formation needs one X-client teardown + re-create ("F1").** The + boot's first DDX registration leaves a stale X-client VAS whose page + directory never resolves the completion-counter page (`counter page + 0x20400f000 unreadable via vaspaces/discovered/targeted; slot stays + pending`); the GL client then never allocates its channel group + (`class=0xa06c` / `chan alloc hObject=0xbeef0100`). Terminating Xorg once + and restarting it re-offers a fresh `VAS 0x10005` page directory; the + counter derives ~36-64 s later and formation completes on the next client + draws. Seven consecutive boots without this step never formed; five boots + with it formed every time. +2. **The IRAM-gate readback default regressed macro execution.** With both + IRAM-gate sites enabled (`sema_gl_mme_iram_gate_break=1` + `sema_gl_mme_iram_gate_build=1`), the first window-bearing launch after the + pack wedges the front end: grfifo frozen FULL, launch `events=0`, + `macro-status=0`, `GP_GET-frozen`, no macro ever executes. A fresh boot + with both sites off (`=0`) executes macros again (`macro-status` with + run=1 progressions, `events>0` ends, SURF/ATTR preludes emitted) - the + pre-CONT-107 behaviour. Runtime flips do not recover an already-wedged + boot (the lane state is sticky), so the sites must be off from insmod. + (`sema_gl_mme_keep_clamp=0` must NOT be used: it breaks the capture path, + `sa=0` / IMMATURE-SKIP flood.) +3. **Golden overlay + gates off produces the first engine-final stores.** + With `sema_gl_mme_golden=1` and both gate sites off from insmod, the + called-but-uncaptured slots (DRAW_IAB slot 34 `mac=0x22`, slot 57 + `mac=0x39`) run with zero call drops, and the engine writes guest pages: + `output retirement wait: 3/900 pages changed after 217 ms (settled)`, + `GUEST-WIT retire ... changed (direct shadow/alias stores)` - first + non-zero store retirements on this lane. The screen surface itself stays + unchanged (GATE-1 still black); the lane still poisons + (`badf1201` -> WEDGE-RECOVER) after ~60 s, and the stores land on + relocated-operand pages rather than the render target. + +## Session 154b (2026-11-03): the SPH wall - draw executes, DS rejects at SPH validation + +Follow-up state after the first store retirements, three further levers: + +1. **The macro "run bit" is a completion latch, not an arming gate.** The + engine's macro-status sampler reads back `0x4044b4`; a completed + slot-34 body latches `mac=0x22 mpc=637` with `run=0` (637 = the body's + last instruction). The earlier belief that the DRAW never ran was a + sampler artefact - the slot-34 body is a short straight-line program the + transition-gated logger never caught mid-flight. +2. **`sema_gl_sph_bind` (pipeline-stage bind emission) was 0 on every boot + to r1433 and is the bind lever.** Enabling it emits the per-window + pipeline-stage binds (`SPH-BIND emitted ... win-binds=2 j1-type=0x11 + initblk=17`), zeroes the previous SPH11 signature, and produces the first + observed slot-34 execution (`mac=0x22 run=1`) with live draw-surface + retirements (`retirement wait: 3/2/1 of 901`, OUT-PAGE-WIT on + 0x120700000-0x120705000 = draw-surface rows 614-618, operand/uniform + values). +3. **The residual wall is the DS stage validation.** DS-ESR decode: + bits 23:0 = SPH<n>_ERR, bits 29:24 = pipeline stage. `a2040800` = + SPH11|SPH18; with binds on, `a2040a00` = SPH9 (the vertex SPH fetched + from the window's own j=1 bind alias is rejected; the DS `region=` + readback is 0/0). Variants: `sema_sph_version_val=0x00040003` flips the + SPH9 attribution but not the wall; `sema_sph_region_legacy=1` emits the + legacy 0x1608/0x160c pair (base 0x121201000) but is refused at the method + level (`Cl 0000c597 Off 00001608`); `sema_sph_region_base=0x01500000` + propagates but the launch is refused by an MME class error + (`Cl 0000902d Off 0x2000`) before the DS stage is reached. + +Current best lineage: `golden=1 golden_force=1 hold_calls=1` + +IRAM-gate break/build 0 + `sa_rearm=0` + `sph_bind=1` (binds + draw +execution + draw-surface retirements; still no rasterized pixels - the +region record for the synthesized stage slots remains the named gap). + +## Session 154c (2026-11-04): whole-shadow page-hash bracket closes the observation gap + +The copy-out samples the launch's tracked output pages, so "0/900 pages +changed" could not distinguish "the engine rasterizes but into an untracked +page" from "the engine rasterizes nothing". A whole-shadow bracket now hashes +every 4 KB page of the emulated vidmem before the forwarded GR publish and +diffs after the output-retirement wait (`sema_gl_vidmem_bracket`, plus the +manual `vidmem_hash_snap`/`vidmem_hash_diff` parameters). On the r1457 boot, +across 11 forwarded launches (10 `NOWRITE-CONFIRMED`, `n_data`/`tracked` +900-902, `wrote=0`), only USERD/ring/PBDMA-progress pages changed (0x80, +0x100, 0x18f, 0x1b0, 0xbc0); no page in the render-target range +0x200000-0x500000 and none at the bounce VAs 0xa000000/0xc000000. The +render-target surface and its bounce candidates are byte-stable, so the +no-pixel state is an execution gap, not an observation gap. + +The live window was also decoded against the class method tables. Present: +the colour-target record (0x840 = SET_COLOR_TARGET, 1280x720, X8R8G8B8, +blocklinear), raster enable (0x37c), shader pipeline binds (vertex 0x11, +tessellation-init 0x21, pixel 0x51), vertex attributes and stream formats, +the index buffer, the slot-2 vertex-stream macro, the slot-0x22 draw call +(0x3910 = CALL_MME_MACRO(34)), and Begin/End. Absent: the viewport family +(0x0a00-0x0a14 scale/offset, 0x0c08/0x0c0c clip, 0x1924 pixel centre, +0x192c/0x193c clip control), scissor, blend and depth/stencil. In a fresh +replayed context a missing viewport clips all geometry, which matches the +zero-fragment observations. The next experiment synthesizes a gated viewport +prelude at the window preamble and re-runs the bracket; a pass is any changed +page inside the render-target range. + +The synthesized viewport was then closed with the driver's own semantics: the +live viewport macro is MME slot 33 (CALL method 0x3908), the clip rectangle is +0x0c00/0x0c04, and the clip control must read 0x00021800 (Z_CLIP_RANGE +ZERO_ONE | GEOMETRY_CLIP FRUSTUM_XYZ) - the earlier value 1 left the geometry +clip in its WZERO mode. The colour-target closure was also found: the client's +pixel shader has MRT_ENABLE=0 and writes target 0, while the window binds only +target 1. Both a corrected viewport/clip prelude and a target-0 bind with a +two-target SET_CT_SELECT are now gated behind module parameters, and the +pair-state orphan END fix stopped the EC0x0d poison after held windows. With +all of that in place and the DRAW macro executing (DRAW_VERTEX_ARRAY, 60/46 +vertices, QUAD_STRIP), the whole-shadow bracket still reports no render-target +page change. + +The remaining input is the shader content. The program pages carried valid SPH +headers but their code was a 144-byte SPH+EXIT test image supplied through the +`vs_hex`/`ps_hex` debug parameters, which overwrote the client's real SASS; +that image writes no position, so every vertex is degenerate. Removing the +test images lets the real SASS survive at the bound program addresses, but the +engine still stores nothing. The open checks are: whether the real vertex +shader writes position and reads the attribute stream; why the client's own +upload accounting still reports zero applied bytes; whether the vertex fetch +bases hold the geometry the guest writes; and a positive control that draws a +constant-position full-screen triangle with a colour pixel shader. + +The shader-content wall was then broken open: the driver's real program +images were recovered from the init-push payload (the guest had already freed +the page, so the bytes were read from the host RAM backing store). Disassembly +of the 128-byte SPH headers plus SASS shows a genuine pixel shader (IPA/MUFU/ +FMUL/EXIT) and two genuine vertex shaders (ALD attribute loads, FFMA/FMUL, +MUFU.RSQ transform path; 1408 and 1536 bytes). Feeding the pixel image through +the existing per-stage program path (`ps_hex`) and switching the synthesized +viewport to raw scale/offset words produced the first engine render-target +writes on record: three of the 900 render-target pages changed, three pages +were copied back from the bounce buffer, the whole-vidmem page hash saw a +launch-page change, and the captured root window contains a handful of +non-black pixels. The pipeline now runs vertex -> raster -> pixel -> render +target end to end; what remains is coverage - the viewport/clip transform +collapses the geometry to a few isolated pixels - plus a delivery detail: a +window whose vertex-shader upload was never captured has an empty VTG page, +and a vertex image applied only to the host shadow is lost when the window-0 +shadow is re-copied from the guest page. The next work is to deliver the +captured vertex image into the guest page for every window and to calibrate +the viewport against the client's own CALL(33) arguments or a synthetic +full-screen control draw. + +## Session 154d (r1494-r1496): fixed-CB stability and contiguous VTG streams + +Two delivery defects on the path from the client's constants and vertex buffers +to the engine were isolated and fixed; neither alone produces fragments yet. + +1. **The fixed constant buffer must be re-delivered every draw.** The MME + constant-buffer base is forced to a page inside the client's screen + allocation and the engine reads it through the launch's bounce alias. That + page is ordinary guest memory: the client (and the guest console) can + overwrite it between launches. The previous fallback wrote an identity + matrix whenever the source VA failed to resolve, which wiped an already + validated real CB (`cb-copy ... src=synthetic-id`). The copy now remembers + the last page that resolved through a SYS (engine-visible) source and passed + the matrix gate, and reuses it for the rest of the boot when a later + resolution misses. The synthetic identity is used only when no real CB was + ever resolved. The copy is not skipped on an unchanged hash, because the + seed page can have been rewritten by the guest since the previous launch + (receipts: `src=vaswalk` -> `src=cached-real` with the client's MVP in + `m[0x80..0xbc]`, and a guest-witness readback of the real matrix words). + +2. **A VTG stream must be one contiguous engine-visible range.** The vertex + stream macro payload is `[stream, end_hi, end_lo, base_hi, base_lo]`, where + `end - base` is the stream length. Relocating `end` and `base` as two + independent single-page operands put a page-crossing stream's ends in + unrelated shadow windows: the computed length went negative and the fetch + past the base page read an invalid PTE (the `miss=1` poison seen once the + real vertex shader started executing). The payload pair is now mapped into + one fresh contiguous fetch line (up to 16 x 4 KB) and both words are + rewritten from that line; unresolvable pages map the module zero page so a + read is a degenerate vertex, never a fault. The receipt + (`MME-VSTREAM-LINE ... pages=N dummy=... line=... first8=...`) reads the + client's real vertex floats back through the mapped page. + +**Remaining wall (measured, r1496).** The active indexed draw's vertex and +index buffers are vidmem (BAR1-aperture) allocations: the launch page set +resolves them to FB page `0xb9f000`, but the emulated vidmem shadow for that +page is all zero (`dump_vidmem` proof), while the same vertex/index data is +present in guest system memory as the client's staging/backing copy. The +engine therefore fetches zeros for the active draw even though the stream +mapping is correct, and the copy-out still reports `0/900` changed pages. The +fixed-CB source VA also moves between boots (resolved in r1492/r1493/r1495, +unresolved in r1496), so a single hard-coded source VA is not sufficient. The +DS continues to latch SPH stage errors on some windows. The next work is to +source the active vertex/index page from the sysmem backing (or to make the +vidmem upload deterministic) and to make the CB source discoverable per boot. + +## Session 154e (r1497-r1499, 2026-11-05): real-FB fetch delivery, per-boot CB, DS post-clear + +Three changes closed the "engine fetches zeros for the active draw" wall and +moved the measured state to "the engine clears the target but draws no +geometry". + +1. **Empty-shadow FB fetch pages are delivered from the real frame buffer.** + The active indexed draw's vertex/index buffers are mapped by the client's + own page tables with the vidmem aperture, but no emulated path ever + populated the BAR1 shadow copy of those pages (the upload did not pass + through an inline write, a decoded CE copy or a CPU aperture write), so + the engine fetched zeros. Two delivery paths now cover the page: + * when every byte of the shadow page is zero, a launch fetch page (a QMD + pointer, not the data-fill surface) is mapped with a VID-aperture leaf + to the real frame-buffer offset the guest page table names + (`FETCH-VID-MAP va=... fb=... qmd=...`); + * a vertex-stream page uses the same rule inside the contiguous fetch + line (`MME-VSTREAM-LINE ... vid=N fb0=...`); + * if the same VA also has a sysmem mapping with nonzero bytes, those + bytes are copied into the shadow page (`FETCH-SYSBACK`), which covers + boots where the client keeps a CPU-visible staging copy. + A page whose shadow already has content is never touched, so the working + sysmem streams, the fixed-CB seed and every engine-written page keep + their identity. Render-target/surface pages (QMD word NONE) are + deliberately excluded: their stores must land in the emulated page so the + copy-out can ship them back to the guest. + +2. **The fixed CB is discovered per boot.** When the client's + `SET_MME_SHADOW_SCRATCH(0x18)` write reaches a decoded push it is used as + the fixed-CB base; when that capture is absent the historically observed + base is tried as a candidate and the matrix gate decides. In r1499 the + candidate resolved through the VAS walk to the client's real page + (`first8=3ff35a31 be800000 be9e377b 3f737870 ...`, the client's MVP), and + 42 of 96 copies used the real page with the rest served from the + remembered validated page. + +3. **The DS HWW ESR latch is cleared after each replayed window as well as + before it.** The latch is W1C; the pre-clear made each window's validation + set readable, the post-clear keeps a failed window's pending report from + poisoning the next window's entry state. The poison count stayed at 0-1 + per run with the mask knob enabled. + +**Measured state (r1499).** The active draw's streams now read the client's +real vertex floats (`MME-VSTREAM-LINE base=0x44fea00 first8=3f800000 00000000 +bf000000 bf800000 ...` and `base=0x44ff000 first8=4069999a ...`), and the +engine executes without a fetch fault. It writes an opaque-black clear +(`0xff000000`) into the render-target shadow page at FB offset `0x200000` +and into the forced RT seed page at `0x504000` (`dump_vidmem` readback), the +bounce copy-out ships the changed seed pages, and the guest's window capture +is uniformly black (`xwd -id`, 800x600, non-black pixels 0, bbox 0,0,0,0). +The clear reaching the target proves the RT0 bind and the color-store path; +no gear fragments are produced. The next wall is geometry coverage: the +index-buffer bind, the vertex-shader position output and the viewport/clip +transform must be checked against the client's own draw arguments (the +active index page currently reads as vertex floats at its bound VA, so the +index/vertex role or the index count is also a candidate). + +## Session 154 (r1515-r1517): the fixed-CB upload is decoded but its bulk is not in the method stream + +The client publishes the fixed-CB base once per context +(`SET_MME_SHADOW_SCRATCH(0x18)`, method `0x3460`, value = VA >> 8) and then +writes the constant store through the C597 methods +`SET_CONSTANT_BUFFER_SELECTOR_A/B/C` (`0x2380/0x2384/0x2388`), +`LOAD_CONSTANT_BUFFER_OFFSET` (`0x238c`) and `LOAD_CONSTANT_BUFFER(i)` +(`0x2390`). The emulator now scans every decoded push of the client's GL +channel for those runs, assembles the first 4 KB of the constant buffer, +validates it with the strict projection gate, and delivers a passing image +to the forced MME seed page; a receipt reports the push, base, selector, +run/word counts, the `c[0x0][0x80..0xbc]` block and whether the projection +shape passed (`CB-UPLOAD`, `CB-UPLOAD-SCAN`, `cb-copy src=const-upload`). + +Measured on a validated boot (r1517, glxgears formed, draws submitted): the +60216-byte setup push is read and the scratch-18 base is captured +(VA `0x4460000`), but its `0x2390` content is only 10 state-patch words +(selector/offset pairs, no matrix rows). The base page resolves to the +emulated vidmem offset `0xb10000` and reads zero; the neighbouring +`0x43e0000` page is client vertex data, and the guest-RAM projection-shape +scanner finds only transient copies. The draw therefore still runs with the +synthetic identity (r1517 `GL-COVER` clip space inside 50/64 sampled +vertices) and the window capture is black (`xwd -id`, non-black 0). The +remaining work is to capture the constant-buffer *bulk* transfer itself +(the staging buffer to the vidmem CB page), not the method-stream patches. + + +## Session 155 cont 8 (2026-11-06): DS SPH bitmap semantics and the BEGIN/END placement wall + +Two results worth keeping: + +1. **The DS SPH bitmap is `stage*4+check` over the six GF100 pipeline stages** + (VSA, VSB, TCS, TES, GS, PS). The six per-stage SPH start registers + (`DS_HWW_ESR_SPH_INFO_STAGE0..5`, 0x405c00..0x405c14) and the live + `DS-FAST` stage VAs plus the fetched SPH header types prove the stage + index: stage1 0x…900 is a V4T1 `SHADER_TYPE=1` VTG image, stage2 0x…200 a + V4T1 `TYPE=2` tess-init image, stage5 0x…0 a V4T2 `TYPE=5` pixel image. + The four checks per stage are the four SPH-V4 validation groups: header + type/version, mpipe/stage binding, attribute map (IMAP/OMAP + register + count + stream formats), and stream-out/topology. A clean boot with only + VSB/PS bound reported `a0040a06` = SPH1|2 (VSA checks 1/2), SPH9|11 (TCS + checks 1/3), SPH18 (GS check2) - i.e. the unbound stages are the failures. + Enabling the TCS bind added SPH10 inside the TCS group (r1554c), which is + the live proof of the `stage*4+check` split. + +2. **Synthesizing the missing VSA/GS stage records clears those bits but + their method run lands inside BEGIN/END.** Binding VSA/GS to the client's + own VTG image (`mask=0x35`) moved the bitmap from `a0040a06` to + `900400a0` (SPH1|2|9|11 gone; SPH5|7|18 remain), then the next window + raised `Class Error Offset 0x2018 Data 019a0900 EC 0x0d` - + `ILLEGAL_METHOD_IN_BEGIN_END`. The value is accepted; the placement is + the wall (the synth block is prepended to a window whose previous END was + dropped). Next: make the stage-record emission BEGIN/END-safe (emit after + an explicit END when the FE begin state is still open) and keep the + client's own TYPE/REGISTER_COUNT-before-address order. + +`SET_SPH_VERSION`/`CHECK_SPH_VERSION` (0x16a4/0x16a8) is not a lever for +this bitmap: blob-derived, `0x10004` and `0x40004` all produce the same DS +value on the same boot. |