summaryrefslogtreecommitdiffstats
path: root/kernel/gsp_sema_service.c
Commit message (Collapse)AuthorAge
* vgsp-sema: SESSION 132 ring-latch truth + per-client util witnesses - the ↵Arthur Hanson Rasmusson5 hours
| | | | r759 false positive is fixed at three points: (1) a ring candidate whose entry0 push provably reads all-zero through the client's own page tables is rejected when the VAS walk resolves the ring VA to a page with a live entry (svc_ring_entry_dead + svc_ring_walk_live, applied at the userd-derived, FB-convention and existing-pin paths; the walk cannot vote for itself); (2) svc_gr_ring_candidate now rejects dead-entry candidates too and accepts the channel's known USERD cross-aperture (r759: ring in guest RAM 0x11623d000, USERD in vidmem 0xb0000, so the walk-resolved ring failed the same-aperture USERD probe); (3) the util sweep arbitrates a dead channel-path ring entry against the direct VAS walk per slot. Also: util-dec/util-sweep witness budgets and their dedup keys become per-client (reset in svc_session_flush) and the sema-copyent witness logs once per copy op instead of per row (a 480-row frame copy alone exhausted the 128/session budget and hid every later copy, including any SASS upload)
* vgsp-sema: SESSION 130 xlate regression bisect - the r727 translator ↵Arthur Hanson Rasmusson6 hours
| | | | regressions were re-tested on a fresh boot: with the translator ON and the utility sweep OFF, victory_saxpy and victory_math PASS with checksums identical to the gate-off baseline (saxpy csum=981440890.500000), so the SESSION 129/130 sweep changes fixed the saxpy/math half; the remaining xlate-on regression is victory_multikernel, whose physical forward times out repeatedly (GR LAUNCH FAILED ... qmd_rel=0xc0ffee01, hold active), so the default stays 0 until that forward path is fixed. The UTILITY SWEEP is the other regression source (util_scan=24 broke saxpy bad=1047527 and hung math) and stays opt-in until its channel reads prefer the direct walk
* vgsp-sema: SESSION 130 zero-payload application + tail-arbitration witness - ↵Arthur Hanson Rasmusson7 hours
| | | | a zero-headed tail-served payload is now applied under a variant once-key (h | 1) instead of being refused forever (the destination is empty, so authoring zeros is idempotent and correct for the driver's internal zero buffer, whose 256 B i2m equals GF100_ZERO_BUFFER_SIZE exactly), while a later non-zero payload for the same entry still matches its own (h) key and lands; the utildec witness now carries alt=success/tries so a walk-read that fails is distinguishable from one that returns zero
* vgsp-sema: SESSION 130 util-channel LOAD_INLINE_DATA arm FIXED - the subch-2 ↵Arthur Hanson Rasmusson7 hours
| | | | GRCE operand group was swallowed by the TURING-copy branch (its `subch >= 2` guard also matches subchannel 2, leaving have_out/line_len unset so the LID arm never fired: r739 witness `lidn=1 subch=2 saw=0 dst=0 ll=0`); the 0x180..0x18c group now parses FIRST for subch 1|2, plus tail-content arbitration (a zero-headed channel tail is re-read through the direct registered-VAS walk before the arm's zero-refusal can stall the upload) and the witness now carries the served tail's first two words + the alt-read counter
* vgsp-sema: SESSION 129 util-channel LOAD_INLINE_DATA arbitration + witness - ↵Arthur Hanson Rasmusson16 hours
| | | | the subchannel-2 arm never fired (r737 writes=0) because the utility channel's read path could not produce a parseable methods entry (the tail-served split shape); the util sweep now arbitrates attempt A (channel path, unchanged) against attempt B (direct registered-VAS walk, the walk_probe-proven path) and lets the strict parse choose the stream, a still-zero tail-served payload is refused (no once-store, retried every pass; in-push payloads untouched), and a bounded deduplicated sema-utildec witness names per distinct entry: winning attempt, both parses, tail VA/length, LID header count/subch and the arm's saw/fired/zero state
* vgsp-sema: SESSION 128 utility/copy-channel inline-download sweep ↵Arthur Hanson Rasmusson16 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (sema_gr_util_scan, default 0) - decode the driver's engine-side MME/i2m upload path and materialize LOAD_INLINE_DATA downloads and pitch copy-ins from channels the put-bounded entry loop never decodes The Vulkan client's code/constant uploads leave the utility channels (the MEM2MEM/A140 composite at h=0xbeef0100 and the eng=11/12 copy channels) and were invisible to the emulation: with an unresolved USERD the put-bounded entry loop skips those channels entirely (the !userd_ok branch continues), and with a resolved USERD the main decode's operand group and LOAD_INLINE_DATA arms only accept subchannel 1 while the uploads ride the mem2mem/copy subchannels (2/4). svc_gr_util_sweep scans up to sema_gr_util_scan (default 0 = off; requires sema_gr_xlate) GPFIFO entries of every channel directly, reads each push plus its successor as the tail (the driver's source-view GPFIFO call carries the payload of a trailing LOAD_INLINE_DATA run - observed beef0100 idx00->idx01 and idx08->idx09), and feeds them through svc_gr_ahead_push in a new inline-only mode: the GRCE/INLINE_TO_MEMORY operand group now also accepts the mem2mem subchannel, the QMD window, release and copy-pend arms are skipped, and a TURING_DMA_COPY_A launch on the copy subchannels materializes the engine copy-in (src from the client's staging buffer through the trusted read, dst through the same svc_grce_inline_write the GR sweep uses) instead of queueing a launch copy-out. Every materialization is once-only keyed by (push_va, content hash, dst, bytes); SVC_AHEAD_TAIL_INLINE_MAX rises 4 KB -> 32 KB so the 12 KB code-staging payload is not truncated. Verified on r737 (gated): the sweep runs for all ten channels (utility, CE and GR) with bounded reads and no release/QMD state changes; the utility channel is swept 14 entries per pass. The LOAD_INLINE_DATA arm on the utility channel's subchannel-2 runs still does not fire (payload/parse witness is the next debug item, see Handoff SESSION 128).
* vgsp-sema: SESSION 127 follow-up - gate the C5C0 translator behind ↵Arthur Hanson Rasmusson17 hours
| | | | | | | | | | | | | | | | | | | | | | | | sema_gr_xlate (default 0) The r727 full battery with the translator active but ungated regressed three clients (saxpy bad=1047527, math sums=0, multikernel hang) while r729 with the translator's semantics gated off is clean on the same subset (mail, saxpy, math OK; multikernel still stalls in its pre-existing flaky class). All translator behaviour is now behind sema_gr_xlate (default 0 = exactly the historical sweep): - the tail-aware method parse (tail passed only when enabled), - the sweep's inline-QMD accumulation, LOAD_INLINE_DATA materialization, C5B5 copy capture and candidate-owned release withholding, - the client-VAS walk fallbacks in svc_gr_page_note / svc_gr_req_fill, - the inline-rec filter at capture (keeps resolvable or QMD-named recs when enabled; historical keep-all when off), - the pump's sweep-captured-launch request and copy fallback. sema_gr_xlate=1 restores the full translator (r730 re-verified: launch captured, page-notes, copy pend/done, submit refused n_data=0, release authored, client completes). Flip the default once the two CUDA-side hazards (stale-VAS resolution widening, rec-filter interaction) are eliminated and the battery is proven clean with it on.
* vgsp-sema: SESSION 127 C5C0 TRANSLATOR v1 - the Vulkan compute launch ↵Arthur Hanson Rasmusson18 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | captured, filled and forwarded (and its copy-out executed) through the CUDA QMD machinery The Vulkan client's compute dispatch is an inline-QMD launch that lives in the PRE-BUILT ring continuation beyond GP_PUT (the earlier analysis only inspected the put-bounded pushes' idx7/idx12 and concluded 'no inline QMD'): beec0100 idx23 carries SET_INLINE_QMD_ADDRESS_B (0x31c = 0x4b85f, the upper half absent) plus the 64-word LOAD_INLINE_QMD_DATA (0x320) run - grid 40x30, prog=0xdfd8feb900 (the SASS page), param=0x4b86000, release0=0x4b85e08/1 - and idx24 carries the TURING_DMA_COPY_A (0xC5B5, subch 4) image->buffer copy-out (src=0xdfd8d60000 dst=0x050b0000, 640x480 RGBA rows) plus the SEND_PCAS schedule (0x2b4 = qmd>>8, 0x2bc = SCHEDULE). Pieces landed (all service-thread, all bounded/logged): - svc_gr_ahead_push (the sema_gr_ahead sweep parser) is now a strict tail-aware translated decode of one continuation entry: it accepts a method stream whose final inline run is completed from the NEXT ring entry (the driver splits big LOAD_INLINE_DATA payloads into a dedicated data slot entry: beef0100 idx00->idx01 256 B, idx08->idx09 12288 B), feeds the C5C0 inline-QMD window to svc_qmd_accum (capture via the standard CUDA QMD candidate), materializes GRCE/KEPLER_INLINE_TO_MEMORY_B LOAD_INLINE_DATA writes into the VA shadow exactly like the main decode (once-only keyed; recorded in the dst ring for the fill's inline-served pages), parses the C5B5 copy operands (0x400/0x404/0x408/0x40c/0x410/0x414/0x418/0x41c, 0x300 LAUNCH_DMA, 0x700/0x708 remap) into a pending copy list, and queues the release groups minus a release a live launch candidate owns (that word belongs to the completion edge, so the client's fence can never clear on a pre-render image). - The parser reads the run data words correctly (w3 fix; the GRCE 0x18c operand was being read from the next run's header word, which sent the 1568 B parameter-bank write to a bogus destination). - svc_qmd_accum: accept the ADDRESS_B-only inline-QMD form (implicit zero upper half) so qmd_dev_va is real for the Vulkan launch. - svc_gr_page_note + svc_gr_req_fill: resolve a client-VAS page through the registered-VAS walk when the trusted sources do not carry it (the Vulkan QMD's program/parameter/descriptor pages are client-managed vidmem; the fill refused the whole launch with -ENOENT before this). - The QMD capture's inline recs are filtered to the pages the captured QMD names (a stale session-wide rec destination made the fill refuse with ENOENT va=0x80412000 qmd_word=NONE). - Pending GRCE copy-out list executed on the launch completion edge (svc_fwd_hold_gr_release and svc_fwd_outcome_check) before the guest release is authored, with a bounded no-forward fallback at the sweep; the copy itself runs through the standard svc_copy executor (observed: 'copy done h=0xbeec0100 dst=0x50b0000 bytes=0x12c000 rows=480 rc=0'). - The pump offers a sweep-captured launch every pass (svc_gr_request) - the continuation launch's only dispatch path - and retires pending copies when no forward is owed. - gsp_phys_gr_min_data (module param, default 16): the degenerate-data-set refusal threshold is now tunable; the Vulkan dispatch's operand set is a handful of records, not the CUDA-era 968. - Bounded witnesses for the request paths (first-N/periodic: no-candidate, backend-not-ready, hold-busy, inflight-busy, fill-ENOENT) and the capture tail dump is now sema_dbg-gated with a 4/boot budget (it flooded dmesg). Live status (r726, cycles r719..r726): the client runs green through the fence (VKFRAME seq/DONE/IDLE, 3 PPMs) with no in-flight client assist; the sweep captures the launch ('launch captured ... qmd=0x4b85f00 prog=0xdfd8feb900 param=0x4b86000'); the fill resolves the QMD pages (page-note src=walk) and now passes its ENOENT gate; the copy-out executes for every frame. The physical launch itself is still refused: the fill yields n_data=0 because the launch's operand CONTENT (the SASS at prog=0xdfd8feb900, the descriptor data) is delivered by the driver through the MME / inline-to-memory engine upload stream (beec0100 idx7/idx12, beef0100 idx08), which the emulation does not execute - the pages the QMD names stay zero in the emulated vidmem. That engine-side upload path is the next work item; sessions r719..r726, evidence in the handoff. No host reboots; 0 oopses; the fetch of the CUDA battery on a fresh boot follows separately.
* vgsp-sema: SESSION 126 FENCE FOLD - channel-anchored sweep of a GR channel's ↵Arthur Hanson Rasmusson20 hours
| | | | PRE-BUILT ring continuation beyond GP_PUT (sema_gr_ahead, default 64): the live Vulkan client's fence waits never clear because the driver's kick boundary stops at the first batch (put=11) while the ring already carries the continuation entries (12..47) whose INC-5 RELEASE groups target the fence words - r716 ring dumps: beec0100 idx17 0x49035b0 len=1020 (group at +0x374 -> 0x45ba080), idx25/33/41/46 at 0x4903f5c/0x4904594/0x4904bb4/0x49051d4 (len 136/112 -> 0x45ba090), all beyond the GP_PUT-bounded walk; the sweep walks entries beyond put through the ring's validated GPFIFO translation, accepts only entries passing the strict method-stream gate (all fence entries parse), extracts addr-0x5c count>=5 RELEASE/REDUCTION groups and queues them forward-only via the standard pending ledger - no consumption bookkeeping touched, idempotent, gated by gr_pub_armed (live epoch) and rrc==0; also: sema-pushdec budget 512->2048/client (boot-time driver-init reads exhausted it before the client's first submit on r716) and chan_dump gains armed=%d
* vgsp-sema: THE CONSUME-WITHOUT-DECODE BRANCH FIXED - the strict ↵open-iov21 hours
| | | | | | | | method-stream parser rejected the real driver's push shapes (op 0 = NVC06F_DMA_SEC_OP_GRP0_USE_TERT, observed 0x0001fff0 SET_SUBDEVICE_MASK; zero NOP padding mid-stream), so every large GR submit push read implausible and was skipped unread; plus the sema-pushdec branch witness (va/src/pa/plausible/injected/ipin/zero_nop/displaced/unreadable/first8, 512/client), the injected flag now follows the source that ACTUALLY answered the read (src 0 only - a page-granularity ipin lookup could take the injected branch while the read used the trusted walk src 4), the PA of the winning read source is recorded (svc_read_last_pa), and svc_push_prefix_ok accepts the same header-only ops r715 in-run validation (mdev b4ccb179, t122 recipe + content-located PDB anchor via pdbtick + rewind): the 1432/976/17316-byte GR pushes now reach svc_decode_stream (sema-pushall rows) and decode with releases: 'sema-push: h=0xbeec0100 va=0x4ca0f1c len=17316 rel=0x4b17c30 pay=3 have_rel=1', 'va=0x4ca0000 len=1432 rel=0x4923000 pay=1 have_rel=1', 'va=0x4ca059c len=976 rel=0x4b17bb0 pay=3 have_rel=1', CE 'va=0x4d00000 len=204 rel=0x4b19b10', plus the same shape on h=0xbeee0100; service applied=29 miss=0; the release words read back 1 in guest RAM (sema_probe 0x4923000..0x49230a0, 0x4b17xxx, 0x4213xxx all advanced). The sema-pushdec rows name the branch and content on every read (e.g. 'va=0x4ca097c len=4 ... plausible=0 zero_nop=0 unreadable=1 first8=00000000 00000000' for the all-zero 4-byte kicks). REMAINING GATE (session 125): vkWaitForFences still times out (~70 s of the client's 120 s bound, UMD-internal cap) even with all push releases applied - the fence wait word is NOT among the eleven released semaphores (released words verified 1 in RAM; the client's wait does not go through the eglcore+0xDCEBD9 device-wait-word chaser that carries the init wait at +0x80 of a 16 KB nvidiactl BO; 16 blind writes to zero wait-slots did not flip it). Next: enumerate the UMD wait-check helper via the vtable at the 0xDCEBD9 site (checkfn=*(*(void**)rbx+0x18)) and breakpoint it for every wait word, or trace the guest kernel poll(NV_DEVICE) handler for the fence object.
* vgsp-sema: raise the sema-rdw witness budget to 6000/client and log the ↵Arthur Hanson Rasmusson21 hours
| | | | viawalk retry outcome (walk rc, pa, plausibility) - session 124 evidence: for the GR submit push 0x4ca0000 the read returns rc=0 src=4 (trusted) even AFTER the user VAS registers and walk_probe rc=0, yet NO viawalk row and NO decode row appear and the entry skips after ~250 rescues: the entry is consumed between the read and the decode (plausibility passed but the decode never ran, or an earlier branch took it) - next: witness the branch taken after the read (plausibility verdict, injected/zero_nop/displaced flags)
* vgsp-sema: per-client unconditional push-read attempt witness - sema-rdw ↵Arthur Hanson Rasmusson21 hours
| | | | (h/va/len/rc/winning-src, budget 400/client, reset in svc_session_flush) at the pump read site plus the r707x src out-param (svc_read_last_src) inside svc_read_va_push_ex; also per-client reset of the sema-pushfail budget. r713: the run reproduced VKDEVICE+flips+VKFAIL but the witness rows scrolled out of the 2 KB dmesg ring before capture - the next run must read dmesg immediately after the first skips (or raise the ring buffer)
* vgsp-sema: on an implausible primary push read for a GR entry, retry via the ↵Arthur Hanson Rasmusson22 hours
| | | | DIRECT registered-VAS walk (svc_walk_vaspaces + svc_read_ap, the exact path walk_probe proves) instead of svc_lookup_trusted - whose pin/lat hits would answer with the same wrong page again; r711/r712 evidence: the user VAS registers and walk_probe rc=0, the rewind replays, but the retry/log never appears and the entries still skip after ~250 failed rescues (the sema-pushfail witness budget is per-boot and was exhausted earlier, so the read-failure detail was invisible this boot)
* vgsp-sema: every-pass VAS candidate verify with the last-good ring pair ↵Arthur Hanson Rasmusson22 hours
| | | | (r707e) - the r710 boot shows the host content-located trigger registers the user VAS and the walk resolves (walk_probe va=0x4ca0000: vas[3] h=0xfade0002 pdb=0x101d63000 -> rc=0 pa=0x11fa20000), while the pump's own svc_lookup pair only registered the three internal VASes (all rc=-2); the working order is: content-located trigger FIRST (registers the user VAS), then sema_chan_rewind so the replay re-reads the entries with the working VAS
* vgsp-sema (FOLD): verify the parked COPY_SERVER_PDES VAS candidates the ↵Arthur Hanson Rasmusson22 hours
| | | | moment any channel's ring resolves through the channel object, so the user-VAS page directory can register BEFORE the submit's first push reads (the host-triggered path could not win that window: a failed GR entry read is never retried with the fresh VAS); r709 evidence: three VASes registered (h=0xc/0xbaba0042/0xcaf00000, pdb=0x11b8db000/0x11b8e6000/0x11b8ef000) and walk_probe still rc=-2 for the client's push VAs - the registered directories do not cover the user VAS, so the content-pair verify against the ring must still land for that one
* vgsp-sema: bounded channel rewind (sema_chan_rewind: hex h_object or 'all' ↵Arthur Hanson Rasmusson22 hours
| | | | -> clears slot_done_epoch + last_get + the per-entry rescue state so entries skipped before the VAS pdb anchor re-decode) + bounded per-chunk push-read failure witness (sema-pushfail: va/chunk/page/last_rc/trusted_rc/ipin/alt/win per failed chunk); wall2: runbook.py (trigger the anchor then rewind). Session 118: the anchor+rewind path is wired; the automated rewind scheduler missed (dmesg churn) and the pushfail trace still needs one clean capture
* vgsp-sema: host-triggered VAS pdb discovery (sema_pdb_discover va,pa -> ↵Arthur Hanson Rasmusson22 hours
| | | | verify the parked COPY_SERVER_PDES candidates against a content-verified ring pair, then fall back to the vidmem walk-root sweep) + runtime-tunable candidate age-out (sema_vas_cand_ms; the default 15 s ages the client's VAS candidate out exactly when its channels first register, so the verified walk can never anchor it by itself)
* vgsp-sema: bind the scrub slot to the EXACT 4096-entry channel ↵Arthur Hanson Rasmusson25 hours
| | | | (NUM_COPY_BLOCKS) - the >=4096 test also matched every client channel (32768 entries in this driver version), so the pass authored for whichever client channel registered last while the real scrubber's finish payload stayed zero (the vkAllocateMemory -2 gate, r701 live evidence: slot held h=0xbeee0102/entries=32768 and the pass logged 'scrub author put=0')
* vgsp-sema: scrub-pass diagnostics - stash bind/MISS logs (4096+-entry ↵Arthur Hanson Rasmusson25 hours
| | | | channels that fail the userd/va condition were silent), pass idle/read-fail/junk-put logs; the r697 session proved the direct authoring works (fin/host sema advance unblocks scrubCheckAndWaitForSize permanently) but the in-service pass never fired
* vgsp-sema r686: accept the low aperture (>=0x4000000) for GR/GL release ↵Arthur Hanson Rasmusson31 hours
| | | | | | | | | | | | | | | decode in both release forms (0x5c-form and SET_REPORT_SEMAPHORE) The GR/GL ucode init pushes carry their releases on the RM's low aperture (the same 0x4000000-0x4ffffff VA family as the push/ring windows): the live GL init push decodes as rel=0x4b19b10 pay=1 (REPORT_SEM group 200426c0 00000000 04b19b10 00000001 10000000) and rel=0x4b17bb0 pay=3 / 0x4b17e30 pay=1 (0x5c-form 2005c017 04923060 ...), all rejected by the sysmem-only SEMA_VA_LO bound - the NvOgl engine-init wait then spun forever. With the bound relaxed (and the push windows injected via the window-record/memdesc recipe) the releases decode live (have_rel=1, applied rising) and the un-translated ones queue in svc_pending (417 at the observed steady state) awaiting the semaphore-VA translations.
* vgsp-sema r640: raise the wait-gate episode extension cap 25 s -> 45 s - the ↵Arthur Hanson Rasmusson43 hours
| | | | r639 streams2 transcript has the readback's acquire giving up at waited_ms=28378 while the launch whose release it waits for only forwarded ~9 s later (watermark defer + arg-partial defer + backend poll chained past the 25 s bound), so the readback ran out of order and the client read zeros (bad=65534); 45 s keeps >15 s of headroom under the clients' >= 60 s timeouts and only engages while a GR forward is unoffered (green boots never reach it)
* vgsp-sema r638: break the CE-watermark/wait-gate mutual citation - a channel ↵Arthur Hanson Rasmusson43 hours
| | | | whose unconsumed head entry is wait-gate held is a consumer-side wait, not a producer backlog; the watermark skips it while the candidate's operands are fully resolved and no decoded device write is pending, so the deferred launch forwards and its release installs (the r612/r636 streams chain: readback acquire op=3 pay=N sits behind the very launch whose release it wants, the two defer each other for the full 15 s cap and the copy runs ~20 s late); module param sema_mutual_skip (default 1, 0 = r636 behaviour)
* vgsp-sema r635: host-side operand resolution when the scanner misses - ↵Arthur Hanson Rasmusson44 hours
| | | | per-shadow-page REAL-content tracking (r->content set on every landed write) anchors a client's own H2D-shadowed pages and arms a launch-level content anchor, so an answerless output candidate (contiguous tail of a shadowed input, or a non-contiguous allocation) is served from a bounded synthetic page set (1024/candidate, 4096/launch, 3/4-table guard) instead of failing the launch closed; synthetic pages can never re-anchor (the first draft self-fed on its own minted pages and filled the shared table to the guard - 6144/8192 measured on a green boot - content gating restored 0 minted pages there and the historical green shape); trusted pin/walk/ledger answers keep the r633 anchored budgets and no longer arm the cross mint; fresh r636 boot: best-equal battery 9 OK / 2 clean FAIL / 1 HANG (streams2 bad=65534 - its H2D dsts are replay-denied, the GR forward defers on the CE stream-order watermark, degenerate refusal faults cleanly; streams_events hang->clean CUDA_ERROR via the auto-fault; smiley first-wait crawl)
* vgsp-sema r633b diag: log the hold-engage record set (nrel/va0/pay0/gr) and ↵Arthur Hanson Rasmusson46 hours
| | | | raise the non-pool device-VA release-write log budget 512 -> 4096/boot - the r633b battery's streams2 spin (userspace, launches completed, QMD releases applied) needs the held records and the release target PA visible; the 512-line budget was exhausted by mid-battery and hid every later application
* vgsp-sema r633: park a held GR launch's push-level release records and ↵Arthur Hanson Rasmusson46 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deliver them on the eventual forward (the r632 mail hang), gate synthetic shadow pages on an anchored span with per-episode budgets (the r632 junk-candidate flood), and name the release0 redirect cause The r632 first boot's mail client spun forever in cudaDeviceSynchronize after its vadd launch: the push-level tracking release (0x20480fff0 pay=112) never landed while the client's word stayed at 111, even though the launch itself completed (GUEST GR LAUNCH OK) and its QMD release was software-applied. Root cause: the release records (svc_fwd_rel[]) are per-decode state (reset at the top of svc_decode_stream), and the two GR hold paths (busy-defer hold, m150 data-stalled hold) break out of the pump entry loop BEFORE the release block (hold-engage / inline apply) runs - so a held launch's records are dropped at the pass boundary. The eventual forward (queued from the capture ring at the defer cap, without a re-decode of the entry) then engaged the release hold with whatever records that pass happened to carry (often none, or another entry's). Live evidence: read_proc_va on the spinning client - 0x20480fff0 = 111 (payload 112 never applied), 0x204827f74 = 6 (QMD release delivered); host log - no sema-sw line for 0x20480fff0 val=0x70 anywhere. Fix (parked releases, keyed by vgpu/channel/push VA): both hold paths park the pass's records (svc_grpark_put) before their break; svc_gr_request restores the queued push's parked set into svc_fwd_rel[] at the consume point so the pump's hold-engage holds exactly those records; the two fail-closed REFUSED paths deliver the parked set inline; the session flush clears the table. Bounded log line "GR parked releases restored" (32/boot). Proven live: r633 first boot restored push=0x20043c1c8 n=1 va0=0x20480fff0 pay0=100; r633 mail now completes (VADD bad=0 checksum=12204516.000000, STATUS OK) on two consecutive fresh boots. Also r633: the r632 synthetic shadow caps were per-FILL-CALL and the fill re-runs on every retry pass, so one probe=none junk argument candidate could mint another 1024 synthetic pages per pass - the 8192-page shadow table filled within seconds of the r632 first launch (va-shadow 8192/8192, all synthetic, from the two 16 MB guessed spans of candidates no resolver ever answered), which then killed the lane for every later real page ("shadow table full ... write dropped"). Two changes: - ANCHOR GATE: a synthetic page is created only when the candidate's span already has >= 1 page answered by ipin/resolve/ledger. Existing H2D shadow content still serves unconditionally (the red-draw chain), and a real buffer's unresolved tail stays anchored by its resolved head. A candidate whose every page is answerless takes the historical drop path - its guessed span is not a buffer; - PER-EPISODE BUDGETS: the per-candidate (1024) and per-launch (4096) caps now persist per (push VA, span base) across the episode's passes, not per call. Measured r633: shadow table peak 2048/8192 (vs 8192/8192 full), 32 synthetic pages for the mail launch (vs 4041 junk-inflated n_data), the lane stays available for content serves. Diagnostics (bounded, one boot of evidence): "vgsp-work: gr-rel0 ..." names the release0 resolution chain (rel_pa/ap/hpa/word/pay/unobs) and "vgsp-gr: qmd-rel redirect: ..." names the backend redirect cause. The first live reading answers the release-target question: rel_hpa resolves (0x2fc495000), map_ret=0, unobs=0 - the redirect is the deliberate gsp_phys_rmcall_gr_rel_mod=1 default (m64 forced module-semaphore release), not a resolver failure. The r632 hang was the push-level release loss, fixed above. Build clean (kernel + r65535 warnings only).
* vgsp-sema r632: the GR data fill serves unresolved operand pages from the VA ↵Arthur Hanson Rasmusson47 hours
| | | | shadow lane (existing H2D content, or a synthetic page for an argument candidate under per-candidate/per-launch caps) instead of dropping them - the submit build hands the shadow page's host address to the bounce machinery, so copy-in stages the client's own bytes and copy-out writes the engine's results into the same page the D2H copies read (red-draw chain: walk=none -> H2D shadow -> fill fail-closes -> degenerate refusal -> SYNTH-FAULT); write guard keeps a shadowed VA's writeback; sema_no_shadow=1 keeps the historical drop path
* vgsp-sema r627: the copy-landing parity pass no longer clobbers per-page ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | landings - the r623c wrong-write site The post-loop parity store (r254-era) recorded a large virtual-destination copy's whole range contiguously (dst + po) AFTER the per-page loop had already stored each page's true resolved PA (m180/r391). For a scattered UVM host buffer the contiguity guess overwrote the true entries; the next copy of the same VA then followed the poisoned ledger (the non-pin DIVERGE branch keeps the ledger) and wrote client c-data into guest pages the kernel had since re-used. r623c evidence chain (saxpy D2H readback of 0x70e64e700000): sema-ledger: rewrite va=0x70e64e77f000 old=0x12e700000 new=0x115800000 504x sema-ledger: DIVERGE write ... prev=0x1158xxxxx cur=0x12e7xxxxx guest panic: kchannelGetGfid_IMPL+0x12, garbage 0x44560800445590f0 (IEEE754 floats 854.25/856.125 = client c-data in nvidia channel structs) Fix: the parity pass skips pages the per-page loop already stored (lookup guard); it only seeds pages the loop did not reach. True per-page landings are never overwritten.
* vgsp-sema r626: sema_no_shadow - A/B knob to disable the VA shadow lane ↵Arthur Hanson Rasmusson2 days
| | | | (copy-dst + fill fallbacks hold instead); isolates the lane as a variable for the fatal corruption class (nvidia channel structs overwritten with PTE-like patterns -> panic in kchannelGetGfid_IMPL via nvGpuOpsStopChannel; r613c/r616/r623c)
* vgsp-sema r620: r619's device-VA sem-write witness range used the 1 TiB ↵Arthur Hanson Rasmusson2 days
| | | | SVC_VAS_FLOOR and silently excluded the 0x200000000-0x20fffffff VAS window (the release words it exists for) - use the window base so the r618-style write-target divergence is actually logged
* vgsp-sema r619 diag: log the resolved target (pa/ap/src/rc/rb) of NON-pool ↵Arthur Hanson Rasmusson2 days
| | | | device-VA semaphore writes - the r618 hung math client read 0 at 0x20480fef0 while the host's deferred releases advanced cur 97..105 on its own target; this makes the write-target divergence measurable (bounded 512/boot)
* vgsp-sema r618: a failed/refused GR forward now delivers a synthetic channel ↵Arthur Hanson Rasmusson2 days
| | | | error edge (notifier status=0xffff + channel event, sema_refusal_fault=1 default) - proven live by the r616 grfaultpoke test: the r616b matmul wedge ended with CUDA_ERROR unspecified launch failure instead of spinning forever; rollback = sema_refusal_fault=0
* vgsp-sema r616 diag: grfaultpoke - manually write a channel's error notifier ↵Arthur Hanson Rasmusson2 days
| | | | (relayed-fault shape: info32=except, info16=engine, status=0xffff, notifier_err latch) and post its channel event, to test whether a synthetic engine-error edge ends a refused launch's cudaDeviceSynchronize spin (the authored sw release lands - measured 0x6 - but the kernel never completes for a never-executed launch)
* vgsp-sema r614 diag: name the first shadow-lane device-dst copies and ↵Arthur Hanson Rasmusson2 days
| | | | untruncate the arg-miss walk list - the r613b matmul wedge began with an H2D taking the shadow silently (no ledger/pin/walk) and the post-mortem could not see which registered pdb answered what (240B buffer, 3 of 8 entries)
* vgsp-sema r613: a wait-gate give-up is deferred while a GR forward is owed - ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | the release sits behind our own launch deferral The r34x acquire wait gate gives up (fail-open, ops run) after gsp_sema_wait_hold_ms with no satisfiable value, on the premise that the value cannot be installed by firmware alone. That premise is false while we are holding the launch that authors the release: the r612 streams2 capture gave up the D2H readback's acquire (ch=0x5c00005f va=0x20480ffe0 op=3 pay=4 cur=2 waited_ms=10333) while the vadd launches that release that semaphore sat deferred by the CE watermark for the full 15 s cap; the readback then ran BEFORE the kernels and the client read zeros (STREAMS bad=65534 even with the launches forwarded and GUEST GR LAUNCH OK). While any captured candidate is still unoffered (svc_gr_forward_pending), extend the episode up to wait-hold + SVC_GR_DEFER_CAP_MS + 5 s margin and log 'extend' instead of giving up. Every client-side timeout is >= 60 s, so a <= 24 s episode cannot strand the client; when no forward is pending the give-up is unchanged.
* vgsp-gr r612: refuse the defer cap only for a DEGENERATE argument fill - ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | stalled decoy candidates must not kill a launch whose real operands resolved r586's fail-closed refused any launch still carrying unresolved argument pages at the 15 s cap. streams2 as the first client of a fresh boot (r611) shows the cost: its two launches carry candidates 0x783299000000 and 0x783297000000 that stall in EVERY source from the first fill (no pin, no walk, no ledger; 'stalled VA' listed, never resolving in the run) while 969 real pages resolve, the r543 arg-defer is pre-empted by the CE watermark defers, and at the cap both kernels are refused - STREAMS bad=65534 with the engine never running. Refuse only a degenerate fill (resolved < dropped - the r584 matmul shape, 2 resolved / 18 dropped, where the engine has no usable window). A non-degenerate fill at the cap forwards historically and loudly (fwd_gr_argfw witness): waiting longer cannot resolve a candidate stalled in every source, and by this point every copy-related guard (dstpend, CE watermark) has already had its chance and returned above when it fired.
* vgsp-gr r611: the stream-order watermark considers only op-carrying channels ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | - sibling launch rings caused mutual deferral r607 made the CE submission watermark watch every other ring. That caught the copy channel the r568 class needs, but it also caught sibling GR LAUNCH channels, whose rings legitimately hold the client own future launches: streams2s two stream channels (h=0x5c00001f/23) each saw the others published entry as backlog, deferred for the full 15 s cap, and the r586 fail-closed then refused BOTH kernels (r610c streams2 as the first client of a fresh boot: every launch REFUSED, STREAMS n=32768 bad=65534). A channel that carries the client stream-ordered memory work is exactly a channel that has decoded a transfer op; measured across r606/r609/r610c, ops appear only on the 0x5c000059 copy family and the CE ranks, and the GR launch channels decode none. Mark it in the op-append block (ever_ops) and scope the watermark to those channels: the r606 red-draw copy channel stays in scope, the sibling launch rings no longer defer.
* vgsp-sema r610: a replay pass must not re-mark a copy entry as ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | operand-pending - the held-chain livelock The fail4 r302 block marks a denied device-destination copy op_tr_pending so the m163/opdef hold retries it. It fires on EVERY decode pass with allow=false - including a REPLAY pass, which is the pump re-walking an entry whose ops were already recorded complete for this identity (entry_replay = same && slot_ops_complete). The re-mark therefore un-completes a slot whose copy HAS run: the next pass is fresh, re-executes the op, the next pin-batch rewind re-marks it again - a replay/fresh/replay cycle that re-arms a 3 s opdef hold every round and stalls the whole copy channel behind it. Measured: 100% of the op_tr_pending marks in the r604-r609 boots came from replay passes (27/27 in r609; 71/71 in r606, whose captured held chain idx=90/92 is this shape; 72/72 in r608). In the r609 boot the d_b H2D entry (idx=94) was re-marked and re-run every ~0.3 s from 368841 while the vadd launch waited on it; the launch then hit the r586 fail-closed cap at 15 s and the engine never wrote (VADD bad=32767 csum=0). r308's rationale for the replay re-mark (pre-r595 the eng==0 copy channel's fresh passes never admitted ops, so the re-mark was the only retry hatch) no longer holds: r595's cap_trust admission runs a plausible copy on its fresh pass, and a genuinely failed copy keeps its slot incomplete via op_owed / the svc_run_ops rc!=0 pending rule. The whole denied accounting block (dstpend note, staging re-snapshot, pending mark, hold) is now skipped on replay; a fresh (non-replay) denied pass marks exactly as before.
* vgsp-sema r609: age the device-dst guard per channel burst - the r606 ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | | | | | held-chain serialization The m169 device-dst first-touch window is per PAGE, but the risk it covers is per client table build. The driver's H2D relocation burst on the eng==0 copy channel writes N distinct first-write pages back to back; because the pump holds the first held entry and the rest of the chain decodes only after it drains, each page paid its own 3 s window, serialized (r606 red draw: hold idx=90 362167.7->362170.8, idx=92 362177.7->362182.9, the b H2D entry ran 362182.882, 326 ms AFTER the launch forward -> VADD bad=32735 csum=8139264, c = a + 0). Every observed release logs 'without a pin': the pin the window waits for never arrives for this class. After ONE un-pinned release on a channel, age it (gsp_sema_copy_dst_age_ms, default 3000): further first-write device-dst pages of that channel inside the window run immediately with the walk translation - the exact translation the expired guard would have used anyway. The first page of every burst still pays the full window; only the N x window serialization is removed (~20 s of chain -> one window). Keyed by h_object like every channel table and cleared by the session flush (svc_dst_guard_reset; also clears the per-page slots, which are keyed by VA alone and must not pre-age the next client's first touch - the r604 lifetime lesson). 0 restores per-page windows (pre-r609).
* vgsp-gr r607: the stream-order watermark must watch EVERY ring, not ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | engine_type!=0 (final form of r590b, on the r606 red-draw capture) The r606 fresh-boot first mail client drew the classic red VADD (bad=32735 csum=8139264, c = a + 0) with the peek armed, and the capture names the blindness exactly: 362177.927 GR-DEC push=0x20043c394 dropped=9 partial=1 dstpend=0 copypend=0 ceback=0 defer_exp=0 362178.001 peek-backlog h=0x5c000059 eng=0 put=95 last=92 362182.145 peek-backlog h=0x5c000059 eng=0 put=95 last=92 362182.556 GR LAUNCH submitted (one-shot arg-defer retry) 362182.882 ce-soft-copy va=d_b executed - 326 ms AFTER the forward -> VADD bad=32735 csum=8139264 The driver's H2D relocation copy sat three entries behind on the COMPUTE-class channel h=0x5c000059 (eng=0) for 4.7 s, held by 'sema-pend: hold h=0x5c000059 idx=90 ... (operand pending)' - the wait group (0x20480fff0 pay=84) passed every pass, so the only guard blindness was the engine_type filter: ceback=0 while the copy channel was provably behind. The r590 watermark now considers every other channel of the device (the launching channel itself is excluded: its ring legitimately runs ahead with future entries). Same pass snapshot, same bounded busy-defer; this guard never refuses - the r586 fail-closed branch is separate and fires only when argument pages never resolve. r595's GR-class ops admission has since cut hold resolution to 0.5-4.7 s (the captured 4.7 s), well inside the 15 s cap, so the deferral ends by forwarding with the copy executed. Companion to r605 (newest-registered-first walk within the class): r605 drains consumable published entries before the launch decision in the same pass; r607 defers the decision while any other ring still holds unconsumed entries (held chains included).
* vgsp-gr r605: walk the eng==0 class newest-registered first - the ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | stream-order fix completed (the write-pointer drain, by ordering) The r507o stream-order fix (walk CE channels before GR ones) assumed the client's H2D copies ride CE channels. r595 established they also ride the compute-class channel h=0x5c000059 (eng=0), and r604 gave the decision-time capture: at the vadd's second decision on a fresh boot, 'peek-backlog h=0x5c000059 eng=0 put=95 last=94' - the b-copy entry was already written and PUBLISHED (GP_PUT ahead of the pump) while the launch decision ran, invisible to every guard (the r590 watermark skips engine_type==0; the r541/r568 gates only see DECODED copies). The pump's stable partition walked the launch channel (registered early, 0x5c000019) before the copy channel (registered later, 0x5c000059), so the copy was consumed 8 ms AFTER the decision and ~320 ms before the launch's copy-in; the run was green only because the partial-argument defer had stretched the window. Pins landing one pass earlier would have forwarded against the pre-copy page (c = a + 0, the r568 red draw). Fix the order within the class: newest-registered first (descending h_object) walks the driver's later-registered copy / push-library channels before its compute channels, so every launch decision sees the copy ring consumed to its live write pointer (GP_PUT) - the write-pointer drain, by ordering. No deferral, no cap interaction, no speculative execution: the pump still consumes each entry exactly once through its normal per-slot identity bookkeeping (slot_ops_ident/done/complete/hash), so no identity mirroring is needed. A client whose compute channel registers last keeps the historical order (no worse).
* vgsp-gr r604: session-scope the per-VA defer latches; peek the whole channel ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | set incl. eng==0 copy channels Two fixes, both from the SESSION 49 r603-family capture. 1. The r543 arg-defer latch and the r556 absolute defer cap are keyed by push VA ALONE, and the UVM pool reuses the same launch push VAs for every client of a boot - the exact lifetime bug the session flush already fixes for the r565 exactly-once memories (svc_gr_cap_hist etc.). Carried across sessions, the r556 stamp armed once at the previous client's first sighting, so a new client's first decision for a reused VA already read defer_expired()=true: every partial-argument defer failed closed (r586 path: QMD release authored in software, no engine launch) and the client read stale/zero buffers. Witness (r603-family boot): the first mail client (fresh VAs 0x20043be94/0x20043c364) deferred and launched (GUEST GR LAUNCH OK ok=1/ok=2); every later mail client (VAs 0x20043beac/0x20043c37c reused) saw defer_exp=1 at its FIRST decision with GR forward REFUSED at defer cap for both launches, and its 'green' was the previous client's results still in the reused device pages. Reset both tables with the session; within a session the original bounds stand unchanged. 2. gsp_phys_gr_peek now logs every drained channel (cap 8 -> 24; the six CE ranks came first and always spent the cap, so the eng==0 copy-carrying channels were never in the logged set - Session 48's 'the copy channel's lookahead read empty' was an artifact of that), names the channel's engine_type, deepens the lookahead to 6 slots, and logs peek-backlog rows for channels whose live GP_PUT reads ahead of the pass's consumed position (the r590 watermark skips engine_type==0, so a published backlog on a GR-class copy channel is invisible to every existing guard; the capture must show it). r604 evidence (fresh boot, peek armed, mail first client): both launches ran (ok=1/ok=2) and green; at the vadd's second decision 'peek-backlog h=0x5c000059 eng=0 put=95 last=94' - the driver's H2D b-copy entry was already written+published on the eng==0 copy channel and unconsumed, executed 8 ms after the decision and ~320 ms before the launch's copy-in; zero write-ahead (+j) rows at any decision.
* vgsp-gr r598 diag: peek the ring slots past GP_PUT at the launch decisionArthur Hanson Rasmusson2 days
| | | | | | | | | The r568 lever is the driver's ring WRITE pointer, not the GPFIFO put: the b copy entry exists in the ring before the client publishes GP_PUT. Add a bounded diagnostic (gsp_phys_gr_peek) that, when a channel reads as drained, dumps up to four slots past last_get with {va,len} and whether the slot already executed that exact entry - so the next fix can be grounded in what is actually written ahead of the put.
* Revert "Reapply "vgsp-gr r590b: the stream-order watermark must watch every ↵Arthur Hanson Rasmusson3 days
| | | | | | ring, not engine_type!=0"" This reverts commit 668c22768abce559ab756f2f998260e08044a3d1.
* Revert "vgsp-gr r595b: bound the stream-order guard to a settle window, then ↵Arthur Hanson Rasmusson3 days
| | | | | | forward (never refuse)" This reverts commit 9333e84deed78464a7448c9faaf6841261a802d8.
* vgsp-gr r595b: bound the stream-order guard to a settle window, then forward ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | (never refuse) With r595 the relocation copies execute within 0.5-1.5 s, but the r590b watermark deferred through the shared 15 s cap; by then the r541/r583 dstpend guard took its fail-closed branch and refused a launch whose data was already on its way (r596 mail MAILDUMP zeros, 2 refusals). Track the settle start per push VA and defer only for gsp_phys_gr_cewm_settle_ms (default 2000). Each deferral is one pump pass and the pump drains the copy rings before the launching channel, so the window is enough for the r595 copies; after it the launch forwards with a witness. The guard can no longer reach any cap-refusal branch.
* Reapply "vgsp-gr r590b: the stream-order watermark must watch every ring, ↵Arthur Hanson Rasmusson3 days
| | | | | | not engine_type!=0" This reverts commit 64cca43699d6b74a1f2f2c5e087d33c7aba59de1.
* vgsp-sema r595: run a plausible GR push's ops - the driver's relocation ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | copies live on a compute channel The r568 b-copy class is rooted in the op gate: allow_ops was (ch->engine_type != 0), so a GR-class channel's own memory copies (LAUNCH_DMA on h=0x5c000059, eng=0) never executed unless the push page came from an injected pin. Their releases were still applied by the op-hold path, the client's blocking cudaMemcpy returned, and the launch read the pre-copy page (c = a + 0). Admit ops when the push content is a plausible method stream - the same content arbiter (cap_trust) that already gates GR launch capture - never on a replay or a rehold. Measured: the d_a/d_b relocation copies now run ~0.5-1.5 s after the driver submits them instead of tens of seconds.
* Revert "vgsp-gr r590b: the stream-order watermark must watch every ring, not ↵Arthur Hanson Rasmusson3 days
| | | | | | engine_type!=0" This reverts commit f3225a81ca384248180d77f78c3614c73dcf289d.
* vgsp-gr r590b: the stream-order watermark must watch every ring, not ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | engine_type!=0 The r591 draw reproduced the r568 signature (VADD bad=32735 csum=8139264, c = a + 0) with ceback=0: the driver submits its relocation/H2D copies on a COMPUTE channel (h=0x5c000059 registers eng=0), so the engine_type filter skipped exactly the backlog the guard exists for. Consider every other channel of the device (the launching channel is excluded: its own ring legitimately runs ahead with future entries), and read each one's live USERD GP_PUT against this pass's consumed position. r590's validated part is unchanged: same pass snapshot, same bounded defer, historical forward at the absolute cap (never a refusal).
* vgsp-gr r590: CE submission watermark - read the ring's live GP_PUT against ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | | this pass's consumption before forwarding a launch The r589 mail VADD drew c = a + 0: the h_b H2D entries sat behind a translation-held CE entry while the pump's last read of the ring's GP_PUT was 92, so the GR launch (processed later in the same pass) forwarded against the still-zero b window. The r541 guard needs a decoded copy and the r568 opdef gate needs a decoded hold; an entry the pump has not reached yet is invisible to both. Record this pass's channel snapshot (CE channels first), read each CE ring's live USERD GP_PUT at the GR decision, and defer while it differs from the snapshot's consumed position. Both sides of the comparison come from the same channel and the same pass, so it cannot reproduce r588's stale-epoch over-fire; the guard only defers (bounded, witnessed) and at the absolute cap the historical forward stands, never a refusal.
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-26 20:08:25 +0000