summaryrefslogtreecommitdiffstats
path: root/kernel/gsp_sema_service.c
Commit message (Collapse)AuthorAge
* vgsp-sema: validate the CE ring entry0 before trusting the layout USERD ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | (ISSUE-0002) r41 c1 regressed: svc_lookup answered rank4's fresh gpfifo VA with an unrelated FB offset (0x90000) and the idle acceptance cemented the wrong USERD (ring+0x10000 of the bogus base). Gate the layout-derived USERD on the ring's entry0 decoding to a CE push VA + plausible len, the same check svc_ring_autopin already uses; a bogus ring now falls through to the previous candidate list.
* vgsp-sema+guest-diag: fix CE USERD discovery for re-registered sessions ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | (ISSUE-0002) svc_chan_userd now prefers the layout-derived CE USERD (ring_pa + 0x10000 in the ring aperture) and accepts it while idle, so a second session cannot latch onto the registered pool-relative offset when that alias holds a DIFFERENT rank's stale USERD: r40 c2 rank0 read rank3's stale 1/1 at FB 0xf0000 and the pump never followed rank0's own GP_PUT (guest GPUT mapping for gpFifoGpuVa proves ring+0x10000). guest_pin_stream.va_to_pa drops translations outside the registered guest-RAM windows so BAR1-phys addresses (0x70000xxxxx) are no longer injected as SYS pins; r40 rank2's ring resolved to the bogus pa=0x70000c0000 ap=SYS and read 66/66.
* vgsp-sema: per-second GP_PUT snapshots for CE channels (ISSUE-0002 diag)Arthur Hanson Rasmusson3 days
|
* vgsp-sema: per-session entry-outcome and stream probes for the CE->GR ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | transition (ISSUE-0002) r31 experiment: log every CE GPFIFO entry read (budget raised, reset per session), every svc_decode_stream call, and the two exits that consume an entry without a decoded release (norel hold / unread slow path). Names the exact exit taken by the second session's follow-up pushes.
* vgsp-sema: fall back to the displaced injected pair for push reads (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | The conflicting-injection race handled for pool releases also poisons push pages: when the later store wins the pin table, the pump reads a page the client never wrote, the push fails the method-stream plausibility check and its releases never decode. When the primary read is implausible (or unreadable through every source), try the recorded displaced translation once before the rescue/hold paths; content plausibility stays the arbiter. Bounded sema-pushalt evidence lines (32/session).
* vgsp-sema: free the release-apply budget and re-arm the push decode budget ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | per session (ISSUE-0002) The ipin store/lookup probes served their purpose (the displaced-pin race is understood and handled); keeping them consumed the shared per-session counter_rel_dbg budget before the release path could log, which left r25's second-client stall blind. Remove them and reset svc.entry_log in the session flush so the next session's sema-push decode lines are visible.
* vgsp-sema: keep displaced injected pins and fan pool releases out to them ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | (ISSUE-0002) Two host tools can inject conflicting PAs for the same pool-semaphore VA page; the pin table is one entry per page, so the later store won and the release written through the loser was invisible while the client polled the winner forever (observed r22/r24: fresh va=0x121000000 -> pa matched the client pagemap; a later stale pair overwrote it, the single release write landed on the stale page, and the second client spun in UVM registration). Remember the displaced pair and write the release to it as well: forward-only and read-back proven, best effort, sequential clients only (documented in the helper).
* vgsp-sema: bust the verified pool cache on every fresh injection + ipin ↵Arthur Hanson Rasmusson3 days
| | | | store/lookup probes (ISSUE-0002)
* vgsp-sema: defer pool releases decoded before the guest base/injection lands ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | (ISSUE-0002) Hold pool-VA releases for 5s after a channel registration so the fresh session injection supersedes the previous client's translation, defer a readable-zero pool word instead of absorbing the write, and keep an INC pending entry alive until the word is non-zero. Second-session releases still resolved to the previous client's page (observed: write to 0x122090000 while the fresh injection at 0x134590000 stayed zero) - the stale-pin source is the remaining lead; diagnostic pool-rel candidates added.
* vgsp-sema: never drop a decoded pool release in the counter path (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | A CE (pool) release that reaches svc_counter_release_apply finds no counter slot and was discarded. The launch path normally writes pool releases via svc_sem_write, but skips it whenever it believes the value was already applied (or the word was unreadable at decode time); the second session's early releases were lost exactly there. Apply forward-only absolute through the pool writer instead - idempotent for a monotonic tracker, so a duplicate can never double-count.
* vgsp-sema: hoist the GR-ring discovery bitmap declaration for the session flushArthur Hanson Rasmusson3 days
|
* vgsp-sema: re-arm push-buffer and GR-ring discovery with the session ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | (ISSUE-0002) The push-buffer pairing bitmap (svc_pb_done), its pass budgets (svc_pb_passes/svc_pb_idle/svc_pb_last*) and the GR ring bitmap (svc_gr_ring_done) are boot-scoped, but svc_session_flush() only wiped the pins they stood for. The next client reuses the same CE/GR ranks, so discovery skipped every rank, its push pages could never be paired, and its decoded releases were dropped ('no slot registered'): the guest waited on a tracking semaphore that could never advance - the second-client hang. Re-arm the discovery state in the flush.
* vgsp-sema: re-arm diagnostic budgets per session and log the release-apply ↵Arthur Hanson Rasmusson3 days
| | | | silent exits (ISSUE-0002 diagnosis)
* vgsp-sema: log release writes into the UVM sysmem semaphore range ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | (ISSUE-0002 diagnosis) Bounded instrumentation: every svc_sem_write into 0x7924_xxxx_xxxx..0x7926 records (h, va, pa, aperture, resolver provenance, value, write rc, read-back ok/value, pool) so the second session's translations for the same VAs can be compared against the first session's. The second client spins in uvm_gpu_retain_by_uuid -> uvm_page_tree_init -> uvm_push_end_and_wait -> wait_for_entry_with_spin (guest kernel stack, sysrq-l) waiting on the channel tracking semaphore; these lines will name the PA the release landed at.
* vgsp-sema: flush host-injected pins with the session (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | The pin-stream tool follows the live client's pid, but its injected entries survived the session flush. The next client reuses the same VAs with new PAs, so stale injections resolved it through the previous client's pages - the second-client exit hang. The 10/10 gate never ran the stream (green); the handover harness does, which is why r3-r10 saw the hang. The queue is drained on svc_thread (svc_handle_map_add), the same thread as this flush, so the clear cannot race it.
* vgsp-sema: synchronize the session flush with channel registration; stop ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | | | | | | | sleeping in the VA-space walker (ISSUE-0002) Two host-side defects from the r9 capture: 1. The session-end flush was executed at the top of svc_rebuild without checking whether the next session had already started registering: in r9 it ran between the second client's first and later channel registrations and wiped its freshly learned state mid-setup. The flush now runs on svc_thread only (the sole writer of the discovery state), at the top of its loop; a first channel of a new session waits on svc_sess_flush_done before registering, so the wipe can never land after that session's state exists and the new session still starts from a clean slate. Scans abort on the pending flag so the wait is bounded, and channel_del wakes svc_thread so the flush runs promptly instead of waiting out a pass. 2. svc_walk_vaspaces held svc.vas_lock (spinlock) across svc_gmmu_walk, whose guest reads go through vfio_dma_rw (mutex, may sleep) - the 'BUG: scheduling while atomic' in the r9 service log. The walker now snapshots the published VA spaces under the lock and walks unlocked.
* vgsp-sema: include the learned pin table, counter-derived cache and hot map ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | in the session flush (ISSUE-0002) The second client still resolved through the first client's pages because three session-scoped tables survive svc_session_flush: svc.pin (the primary learned VA->PA table, consulted before every other source), svc_cnt_der (serves a cached PA without re-proving it) and svc_hot (scan bitmap). Clear all three with the rest; svc.ipin is left alone (injected pins are the external pin tool's domain).
* vgsp-sema: only a real session end may request the discovery flush (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | | A redundant channel delete (a FREE for a handle that never registered, or a repeated delete after the session already ended) found the channel set empty while the next client was already starting and re-armed the flush, which then executed mid-startup and wiped that client's discovery state (and dropped its pending releases): the client spun at its first channel. Track whether a session's channel has registered since the last flush and request the flush only on the true existed->empty transition; redundant deletes are inert.
* vgsp-sema: flush per-session discovery state when the last channel closes ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | | (ISSUE-0002) A second CUDA client in the same guest runs in a new UVM VA space that reuses the same VAs backed by new pages. The service's discovery state (merged scan map, pinned push-buffer pairs, discovered page-directory latch, direct_base, pool-PA cache, CE ring-done ranks, retry queue, forward-release records, GRCE dst ring records, pending releases, VA spaces) is learned from the live client and was never reset, so the next session inherited translations for the previous session's pages: its rings never auto-pinned, releases queued unresolved in svc_pending and the client spun at its first channel. One clean boot always works, so make the next session look like one: when the channel set empties mark a flush; the next rebuild (forced immediately) resets the same state a fresh boot starts with, and the new client re-discovers from its own content.
* vgsp-sema: reject identity pool-semaphore translations (the wedge class)Arthur Hanson Rasmusson3 days
| | | | | | | | | | | Wedges (WEDGE-COMPARE-r7-vs-r10.md) wrote the pool completion to pa==va (0x121000000, src=trusted) instead of the guest's real page (greens use src=pin); the read-back proof cannot catch identity because the SYS aperture reads/writes accept any address, so it cached and the guest's first tracker wait never woke (kernel logging stops at client init, guest wedges). Never accept or cache a pool VA->PA resolved to an identity page: guard lookup/cache/store and every resolver source; a rejected identity misses, so the release queues in svc_pending[] until the real pin lands.
* vgsp-sema: publish GR channels at end-of-pass again (regression fix for the ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | cudaMalloc spin) m140 published every channel with the pass-computed put and GR clients advanced (green counter publish val=put). The H3 change restricted the end-of-pass site to CE and moved GR to the safety-net early call, which reads GP_PUT from the snapshot USERD and returned 0 on live GR channels while the pass's own put was 1. GR never published; victory_mail spun at 100% CPU in the first cudaMalloc (wchan=0). Publish GR here again; the early call remains the fallback for early-continue passes and both are monotonic.
* Revert "vgsp-sema: walk-authoritative GR ring pin when content not yet ↵Arthur Hanson Rasmusson4 days
| | | | | | readable; walk-verdict telemetry (FORENSICS-grv6-ring fix A)" This reverts commit d246c02500a6eb41f88e7aae1febb6e800668d73.
* vgsp-sema: split release slot from completion counter (F1, SA-COUNTER ↵Arthur Hanson Rasmusson4 days
| | | | FORENSICS-grv7-counter): release targets the learned release VA, publish targets the ascending counter slot
* vgsp-sema: remove pub-mirror (guest-perturbing; SA-GATE ↵Arthur Hanson Rasmusson4 days
| | | | FORENSICS-grv7-gate); keep read-only dbg telemetry
* vgsp-sema: walk-authoritative GR ring pin when content not yet readable; ↵Arthur Hanson Rasmusson4 days
| | | | walk-verdict telemetry (FORENSICS-grv6-ring fix A)
* vgsp-sema: debug-gated GR counter mirror + pub-info + GP_GET readback ↵Arthur Hanson Rasmusson4 days
| | | | (FORENSICS-grv5 E1/E2)
* vgsp-sema: periodic GR discovery re-arm + FB-first fast path (FORENSICS-grv3 R1)Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FORENSICS-grv3: the FB discovery worked for ranks 0..2 (content-discovered + auto-pinned ap=2, ring_auto 6->9, gpfifo done 89->110, gr_rel 0->3), but the guest only ever wrote ring entry0 for those three ranks during the scan window, and the discovery epoch took its 60-pass cap and retired - the re-arm existed only on channel registration - so the remaining ~8 minutes of guest life had no scanning at all (890 "capped" messages, no pass after ~4189). The first pin was also ~3.5 s late because the 6 GiB RAM walk ran before the FB scan. Changes: 1. Periodic re-arm (FORENSICS-grv3 sections 4-5A). Taking the epoch cap no longer retires discovery: the pass count resets, discovery rests SVC_GR_CONTENT_COOLDOWN (30 s) and then starts a fresh epoch for as long as any registered GR channel still lacks a ring. A registration batch still re-arms immediately (and now also clears the cooldown). When every registered GR channel has a ring, discovery stops with a single "discovery idle" log line. Passes stay one per 2 s and the cap/cooldown bounds the scanning duty cycle, so a permanently missing ring cannot hot-loop. 2. FB scan first + fast path (FORENSICS-grv3 section 5A). The emulated-VRAM scan (direct host reads, milliseconds, bounded to SVC_GR_CONTENT_FB_LIMIT at a page stride) now runs before the RAM walk and records ring candidates without requiring a push candidate. Push candidates are kept across passes (the ctx-init pushes are content-stable), so when every ring-carrying rank already has a push the pass pairs right after the FB scan and skips the RAM walk entirely: a late ring submission becomes visible within one 2 s pass instead of after a full re-scan. 3. Descending RAM walk with early exit. The push (release-group) walk now starts at the top of guest RAM - the same reasoning as the CE content rescue: the guest's work allocations live in the high RAM block (grv3 pushes at 0x1028f...0x12df8..., 4.0-4.8 GiB) - and stops as soon as every ring-carrying rank has a push candidate, so the pass ends in about a second instead of a full 6 GiB walk. RAM ring candidates are still recorded here for boots whose ring is not in FB. The pairing gate is unchanged: entry0 exact, the push parses end-to-end, the release group sits at len-20 with this rank's slot and payload 1, and the USERD at ring+0x2000 reads plausibly. Wrong or stale candidates pin nothing. No module parameters were added or changed. Not included (deliberately): the notifier/event completion path from FORENSICS-grv3 section 5B. Section 3 lists it as the leading hypothesis only, never observed live (the relevant logs are debug-gated), and 5B gates it on this change showing the guest posts nothing more; the sema_dbg=Y experiment is the discriminator. No speculative change was made. Evidence: FORENSICS-grv3 sections 0, 4, 5A; 11-hostdmesg-grv3-r1.log 2727-2735 (FB pins), 2820-2823 (cap), 4060-4062 (capped tail); m104 17-host-m104-r1.txt:164-283.
* vgsp-sema: FB-aperture GR ring discovery + per-registration discovery re-arm ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (FORENSICS-grv1 R1) FORENSICS-grv1: the grv1 campaign ran the H1 content pass to its 60-pass cap with zero pairs. svc_read() is vfio_dma_rw over guest DMA addresses, so the 4 GiB guest-RAM scan found all eight GR ctx-init push candidates (len 0x2a4) but no page-aligned ring entry0; the ring lives in the emulated vidram aperture (vgpu->bars[1].vaddr), where m104's green ring was FB 0x140000 + R*0x3000. The trusted walk also missed (the ring VAS is unpublished: VAS[0] root empty, VAS[1] has no usable PD0 for the window), and the H3 early publish was a no-op (put=0 from the pool-relative registered USERD at 0xc32000, which reads zeros in FB). Changes (one coherent commit): 1. svc_gr_content_pin() takes the ring's aperture. The ring page and the USERD at ring+0x2000 are read through that aperture; the push page and its release group stay guest RAM (SYS); the hot-page bitmap (guest-PA indexed) is only touched for a SYS ring. FB rings log "content-discovered GR ring ... -> fb 0x...", RAM rings keep "-> pa=... ap=1". The gate is unchanged: entry0 exact (push VA + len), push parses end-to-end, release group exactly at len-20 with this rank's slot and payload 1, USERD readable (active pair or readable idle) - otherwise nothing is pinned. 2. svc_gr_content_discover() scans the emulated VRAM mapping after the RAM pass for every rank that has a RAM push candidate but no guest-RAM ring candidate, at a page stride, bounded to SVC_GR_CONTENT_FB_LIMIT (512 MiB, matching the existing one-shot fbs scan bound), with the same entry0 filters and the plausible-tail check. Candidates pair with the RAM pushes by length (0x2a4) and rank geometry, then go through the same full gate. 3. The 60-pass one-shot cap is now a per-epoch bound that re-arms on channel registration: gsp_sema_service_channel_add() sets svc_gr_content_rearm for GR channels (the guest recreates channels on every context retry), and svc_gr_content_discover() consumes it and resets the epoch pass count. The cap message is ratelimited and names the re-arm condition, so discovery can no longer retire for the boot while the guest is still trying. 4. The registered userdMem base (pool-relative, reads zeros) is superseded by the ring+0x2000 USERD probe through the ring's aperture; this covers the grv1 "probe rc=-11 from the wrong base" case without a separate path. No module parameters added or changed; the accepted recipe is untouched. Evidence: FORENSICS-grv1 sections 0-5, m104 17-host-m104-r1.txt:164/169, 11-hostdmesg-grv1-r1.log:2308/2339-2347/2429-2435/2796. 5. The GR pin-branch "auto-pinned GR ring ..." line now carries ap=%d, so the latch is unambiguous in dmesg (FB=2 for the content-discovered ring).
* vgsp-sema: reachable GR counter publish + injected-pin override at autoreg ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (FORENSICS-v2 H3 + anomaly 6) FORENSICS-v2 H3: svc_counter_publish() was reachable only at the end of a fully pumped channel; when a GR channel's ring or USERD cannot be read the per-channel loop early-`continue`s above that call (USERD not resolvable, GP_PUT unreadable), so pub_calls stayed 0 even with all 20 GR counter slots registered and the client's tracker waiting on them. Duplicate the publish into the safety-net block at the top of the per-channel loop, next to svc_counter_autoreg/svc_counter_flush_pending: - the value is a real GPFIFO put read from the channel's registered USERD; when the USERD is not readable nothing is written, so a guessed put can never advance the client past work the pump cannot see; - the target is the proof-gated slot registration (injected pin or the read-back-validated derive), never an unproven page; - the same relay-hold guard as the end-of-pass call applies. The end-of-pass site now publishes for CE channels only, so a GR pass does not publish twice. Anomaly 6: an earlier-derived counter_pa could disagree with a later-arriving injected pin (r1: derived 0x123a43ff0 vs pin-backed 0x123a28ff0) and nothing re-checked the pin while counter_pa was already registered - a publish would then advance a page the client never polls. svc_counter_autoreg() now re-checks the rank's slot against the injected pin on every pass and re-registers (under chan_lock) on a mismatch, which also makes the new publish proof-gated through the pin. Evidence: FORENSICS-v2 H3 + anomaly 6; 11-hostdmesg-v2-r1.log:2429-2714 (20x "not at fb ... probe rc=-11", pub_calls=0, gr_rel=0)."
* vgsp-sema: kernel-native GR ring/USERD discovery (both apertures, trusted ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | walk, content fallback) (FORENSICS-v2 H1+H2) The v2 campaign confirmed the CE-side fixes work (89/89 releases applied, pool page resolved src=trusted) but the stall moved one phase later: the guest authors its eight GR context-init pushes and the host never reads a single GR GPFIFO entry (ring_auto=6 CE-only, pub_calls=0, gr_rel=0, work_gr req=0). The GR ring VA 0x200200000+R*0x3000 and its USERD are never resolved: svc_gr_ring_autopin only tried FB conventions, userd-derived FB candidates and existing pins (FORENSICS-v2 H1/H2). Three bounded, content-gated extensions: 1. Trusted walk (H1): try svc_lookup_trusted() for the ring VA - the driver-published VAS walk / self-discovered directory chain that resolved the CE pool page in v2 ("pool-pa ... src=trusted", 11-hostdmesg-v2-r1.log:2313). 2. Both apertures (H2): every ring-derived and registered USERD candidate (svc_chan_userd) and every userd-derived GR ring candidate (svc_gr_ring_autopin) is tried in the registered aperture first and the other one second; v2 read the GR USERD through FB and got 0/0 while the same boot's pushbuffer pins are sysmem. 3. Content fallback (H1 last resort): one self-gated, 2 s-rate-limited, 60-pass-capped 64 KB chunked scan of registered guest RAM locates, per rank 0..7, the release-group page ("200426c0 00000002 0400ffXX 00000001", group at push len-20, page-aligned push at VA 0x200400000+0x400000*rank, m104/handoff 2576-2597 geometry) and the ring page whose entry0 is the exact (push VA, len) pair. A pair is pinned only when the full gate passes: entry0 bytes exact, the push parses end-to-end as a method stream, the release group is exactly at len-20 with this rank's slot and payload 1, and the USERD at +0x2000 reads a plausible get/put pair (or a readable idle 0/0). Anything else pins nothing. All candidates share the existing rule: a wrong guess costs a failed probe, never a wrong pin. Evidence: FORENSICS-v2 H1/H2, m104 17-host-m104-r1.txt:169-171, v2 dmesg 11-hostdmesg-v2-r1.log:2313-2714.
* vgsp-sema: multi-page content rescue pins a whole CE push from RAM content ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (ISSUE-0001) svc_rescue_try() substitutes exactly one candidate page and requires every other page of the entry to resolve through svc_lookup_trusted(). For a multi-page push whose pages have no translations yet that precondition can never hold: the content scan finds the release-group page by content, but the pages that need a candidate are exactly the ones missing from every trusted source. That is the stalled-boot class named by SA-STALL (entry va=0x120021db8, len=12816, release #39; 17-host-m84-r1.txt:841-842), where the userspace pin stream was the only supplier of those translations. Add svc_rescue_run_try(): once content has anchored one page of the entry (release-group candidate from the matcher, or the verified window base), read every page of the entry directly from the same signed page delta in both physical directions, validate the assembled buffer exactly like the single-page rescue (full method-stream parse + the channel's exact next release), and only then pin every page of the run. A failed direction pins nothing; a passed validation is content proof for every page. Wire it into the svc_pb fast path, the hot-page scan and the descending RAM scan, ahead of the unchanged single-substitution try. This makes the pump able to consume a starved CE push with no watcher pins and no pre-existing per-page translations; the watchers become pure accelerators. New counter resc_run is printed when non-zero. Evidence: ISSUE-0001, SA-STALL 3.2/3.4; 17-host-m104-r1.txt entry[38] and 17-host-m84-r1.txt rescue rejects.
* vgsp-sema: keep push-buffer base discovery alive until the CE windows drain ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | (ISSUE-0001) svc_pb_discover() ran with a flat 12-pass budget for the whole boot and then retired, even when a registered CE channel had no content-discovered push-buffer base yet. A channel that appeared after the budget was spent (including one reappearing with a new client session) then had no paired window, so every push went through the slower per-page rescue and the userspace watcher pin stream. Stop the pass only after SVC_PB_IDLE_PASSES consecutive passes with no new base AND with every registered CE channel either paired or having a base recorded for its release slot, with a hard cap (SVC_PB_PASS_MAX) so a channel that genuinely has no base page cannot scan forever. The 3 s rate limit and chunked walk keep the service thread responsive; the per-pass log now carries the idle count. Evidence: ISSUE-0001; v2-r1 counter freeze (pb=6/12, see backups/evidence-20261002-victory/11-hostdmesg-v2-r1.log) and the SA-STALL class where the CE window was reached only after discovery had stopped.
* vgsp-sema: retryable page-directory discovery from fresh guest-truth pairs ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | (ISSUE-0001) svc_pd_discover() latched svc_pd_tried before its first scan, so a call that ran before the client's page tables existed (or before enough verified push-buffer pairs were known) disabled the one kernel-native source that translates pool/push windows the content-scanned map and the published directories miss, for the rest of the boot. That source is what svc_lookup_trusted() and svc_pool_pa_resolve() fall back to when the userspace watcher pins race. Make discovery retryable and evidence-driven: an attempt that cannot run does not latch, a scan runs only when the sample-set fingerprint changed, and the total is capped at SVC_PD_ATTEMPTS_MAX per boot - the same order as the old per-pairing re-arm, which this supersedes (the stale re-arm in svc_pb_pair is removed). Evidence: ISSUE-0001, SA-STALL 3.2/3.4 (the deferred pool release of the last CE push with no translation when watchers race).
* vgsp-sema: kernel-native pool release translation + starvation telemetry ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | (ISSUE-0001) svc_pool_pa_resolve() could translate an UVM pool/waiter page only from watcher pins, the read-back cache, the self-discovered directory and a targeted rm_mem scan. When the userspace watcher races a boot and the self-discovered directory has no entry yet, a starved completion release sits in svc_pending[] forever (p13-r4: applied frozen at 89/90, gr_rel=0), the guest tracker never wakes and CUDA blocks at DEVICE. Add: - svc_walk_vaspaces() (the driver-published page directories - the walker svc_lookup_trusted always used, previously absent here); - a final proof-gated svc_lookup_trusted() fallback (also the only path that triggers svc_pd_discover()). Both read-back proven like every other source. Plus per-entry starvation telemetry (tries + bounded logs) so a future stall names the exact va.
* sema: fix GR counter-slot mapping to fixed pool total (20)Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | The counter slot for GR rank r is 0x20400fec0 + (TOTAL-1-r)*0x10 with TOTAL the fixed pool total. The autoreg used a live high-water channel count instead; channels register progressively, so an early pass with a partial count (16) registered the ranks present at the wrong slots and every later rank's correct slot was then rejected as owned. Result: ranks 10..19 never got a counter, no counter was ever published, and the client's userspace tracker never advanced (cudaMalloc stall while the guest RM polled MC_SERVICE_INTERRUPTS). Validated on a live boot: 20/20 slots registered with 0 collisions (previously ranks 6..9 installed at the n_gr=16 mapping and ranks 10..19 rejected).
* vgsp+guest-diag: kcore-truth pin resolver (pool/pushbuffer/channel ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | | | | | | ring+userd) + GR ring userd-derived candidates guest_pin_stream.py now resolves translations entirely from guest truth via /proc/kcore + a manual page-table walk (no QMP races): the CE semaphore pool and pushbuffer memory from the UVM rm_mem structures, and each channel's GPFIFO ring + GPPUT from the tracking semaphore's embedding uvm_channel_t (channel_info located by its gpFifoGpuVa value). This removes the watcher pipeline's flakiness on the cudaMalloc stall class (missing pool/pushbuffer pins). gsp_sema_service.c: svc_gr_ring_autopin tries the guest-registered USERD memdesc base as a ring candidate (the fixed FB convention only matches when the guest's VRAM allocator happens to reproduce the old layout), plus an optional (gsp_sema_dbg) one-shot FB scan + raw-word diagnostics when every candidate fails. tools/cuda-ab/p2: self-checking arbitrary-kernel acceptance suite (10 programs: sequential launches, shapes, params, smem/GEMM, streams, malloc churn, printf+error, FMA benchmark, mixed, managed probe) with host oracle expectations and a guest runner.
* vgsp: m140 - resolve 33 pages/pointer (128KB buffers with 0x400 in-page ↵Arthur Hanson Rasmusson4 days
| | | | offset span 33 pages; tails read the next reloc record)
* vgsp: m137 - CE-hold liveness watchdog (bound deferred CE waits across ↵Arthur Hanson Rasmusson4 days
| | | | passes; targets the guest cudaMalloc stall class)
* vgsp: m121 - resolve param-page kernel-argument pointers (offsets >= 0x150) ↵Arthur Hanson Rasmusson5 days
| | | | before the bounce cap
* vgsp: m120 - data-fill 32 pages/ptr under the 100 cap + rewrite hit/coverage ↵Arthur Hanson Rasmusson5 days
| | | | logging + param 0x150 dump + main-PT slot dump
* vgsp: m118 - data fill candidates at page granularity (runs start at the ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | pointer page) m110 collected candidate windows at 2 MB granularity, so the 64-page runs started at the window base and never covered the pointer pages themselves - the bank/param pointers matched no record and were never rewritten (m117: only 2 relocations, trap unchanged). Collect candidate pages instead.
* vgsp: m114 - pre-commit bounce window for the kernel's data (native run w/ ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | | copy-in/out) Decisive evidence: the SM walker resolves only the commit-time tree (every late write/install fails: m94, m99-m113; UNSET refused 0x1f). Plant a 112-page bounce window into the main small-PT at stage 3 (slots 0x110..0x17f - the proven win0 pattern), copy the launch's resolved data pages in, pointers are already rewritten to those VAs, and copy results back after the release. The mail message lives at bounce page 100 (c[0x4][0x0]=0x174000). Data cap 100 pages (440 KB), matching the shrunk victory_mail (3 x 128 KB).
* vgsp: m110 - relocate data into the engine-walkable main-PT window; shrink ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | the demo All UPDATE_PDE_2 installs (post- or pre-commit) are invisible to the SM walker; the only window it honours is the commit-time main small-PT page (VA 0x100000+, the proven win0/const window). Relocate each candidate data pointer's first 64 pages into slots 0x110..0x1f0 of that window (fresh PTE lines, direct leaf writes), rewrite bank pointers per page record, and put the mail page at the reserved slot 0x1f0 (c[0x4][0x0]=0x1f0000). Shrink victory_mail's buffers (n=1<<15, 128 KB each) to fit the 2 MB window.
* vgsp: m106 - pre-install the data PT window pool in stage 3 (pre-commit)Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | | Post-commit UPDATE_PDE_2 installs are accepted (status 0x0) but ineffective (m94's scratch windows, m101-m105's reloc windows: the SM never resolves them), while the stage-3 ctx-window installs are honoured. Add a fixed 32-window pool at GSP_PHYS_RMCALL_VA_GR_DATA_BASE (0x100000000 + k<<21, inside the server-managed region), each backed by a module PT page installed through UPDATE_PDE_2 during stage 3; the engine data map then only writes leaf PTEs into those fresh PT pages at launch (the P6b-proven 'direct leaf write into a walker-installed fresh table' pattern). The mail injection uses the reserved last window (k=31); service data windows cap at 31 and use the same fixed layout.
* vgsp: m102 - relocate guest data into the server-managed VA region and ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | | rewrite bank pointers The server refuses UPDATE_PDE_2 (status 0x1f) for guest CUDA VAs: they are client-managed, while the golden VAS reserves the server-managed region [0x100000000, 0x11fffffff) (gsp_phys_booter.c:8455). Map each resolved data page there at an affine-relocated VA (identity HPA, no copies), rewrite every 8-byte pointer value inside the shadowed const-bank pages (banks 0..7) that falls inside a mapped guest window to the relocated VA, and raise the Tier-1 window pool to 32 windows x 512 PTEs so the relocated pages fit. The kernel now reaches its real data through the server-installed windows.
* vgsp: m100 - scan every const-bank page for data VAs; data PTEs kind 0 by ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | default The +0x880 trap instruction disassembles (nvdisasm on the m98b prologue bytes) to LDG.E.U8.CONSTANT.SYS R5, [R2.64+UR4] with UR4 = c[0x4][0x0] - the first global load's base comes from const bank 4, not the c[0x0] param page the m99 scan covered. svc_gr_data_fill now scans every bank page the launch carries (QMD words 32..47, even LO words: banks 0..7), deduping up to 16 windows. Data PTEs use the gr_pte_kind param (default 0 = the kind proven SM-visible via the const reads) instead of a hardcoded 6.
* vgsp: m99 - identity-map the forwarded kernel's guest data pages into the ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | | | | | | | | | | | | | | engine tree The m97/m98 decode proved the SM instruction fetch and const reads work; the wall is the kernel's first global data access (prologue instruction at +0x80) to a guest-VA device pointer (param page words 6..9 carry the demo's two 16MB buffers at 0x763dXXXXXXXX), which is not mapped in the engine's page tables. The walker walks OUR root->PD2->PD1->PD0->PT tree (SC PDB == our VASPDB chunk, proven by matching the same-run INST base to the PDB-root DMA log), and fresh direct-planted tables are visible at first fetch. Service side (gsp_sema_service.c): svc_gr_data_fill() scans the param (c[0x0]) page for plausible device VAs, resolves the containing 2MB windows (-1..+16 windows, early stop when a window resolves nothing) via svc_resolve_va on the service thread, and carries {va, pa} in svc_fwd_req.gr_data; svc_fwd_gr_work() converts PA->HPA and attaches the list to the submit. Backend (gsp_phys_booter.c): gsp_phys_rmcall_gr_data_map() plants VA->HPA leaf PTEs (GENERIC_MEMORY kind 0x06, like UVM/RM on Turing) into lazily allocated PD1/PD0/PT chunk branches under root[0] (VAs < 2^47), caching branches per index; the submit maps all req->data pages before publish and logs the count.
* vgsp: TSG-sibling mode 30, UVM-stall Patch C, QMD shadow-window ensure + ↵Arthur Hanson Rasmusson6 days
| | | | launch diagnostics
* vgsp: PRIV VA-only booking bit + verify the exact slot before consuming ↵Arthur Hanson Rasmusson6 days
| | | | | | | | | | | | | unreadable CE entries - fwclient mask bit 15 (0x8000): book the PRIV_ACCESS_MAP VA (the FWCLIENT entry form is init-only; the golden9 form carries the VA at VA_GOLDEN9_PRIV). Accepted on V6 (status 0), completing the maximal promoted context. - gsp_sema_service: svc_entry_want_reached() verifies the exact tracking slot against entry_want before the three unreadable-CE consume shortcuts treat the entry as delivered; when the slot lags, re-seed the expectation and let svc_fake_release author or queue the exact slot (UVM registration stall pair with the pool-VA verified-resolver queue).
* vgsp: golden-exec shape bisection + fwclient per-entry promotion + pool-VA ↵Arthur Hanson Rasmusson6 days
| | | | | | | | | | | | | | | | | | | | | | | | | stall fix - Mode 28 golden execution lane behind gsp_phys_rmcall_gr_promote=28 with a shape bitmask (gsp_phys_rmcall_golden_exec_shape: ring / entries=8 / sysmem USERD / USERD size / hObjectBuffer=VAS / skip replay TURING_A) and a USERD control-offset helper. Finding: the RM-managed golden channel hangs any GR object construct (-110, no assert) as soon as client submission wiring (ring or sysmem USERD) is attached, so only the captured server-owned shape constructs; the machinery is kept for future wiring ideas. - fwclient mask bit 11 (0x800): send each non-MAIN FWCLIENT promote entry as its own one-entry control and continue past refusals. V6 acceptance matrix: PM, PATCH, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP accepted, FECS_EVENT refused 0x1f. - fwclient mask bit 12 (0x1000): golden9 VA-only global bookings (BUNDLE_CB, PAGEPOOL, ATTRIBUTE_CB, RTV_CB_GLOBAL) accepted on V6; bits 13/14 (0x2000/0x4000): FECS PA-only (refused 0x1f) and FECS VA-only (accepted). - gsp_phys_rmcall_srv_main_ignore: ignore a server-reported MAIN in the srv-ctx commit and force the client INITIALIZE_CTX + VA-only fallback. - gsp_sema_service: pool semaphores never fall back to the learned map; queue the absolute release via svc_pending_add until svc_pool_pa_resolve succeeds, and judge pool slots through the verified resolver in svc_apply_pending and the release decode (UVM registration stall fix).
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-12 10:03:17 +0000