summaryrefslogtreecommitdiffstats
path: root/kernel/gsp_phys_booter.c
Commit message (Collapse)AuthorAge
* vgsp: re-arm on MISS-ABORT (candidate re-offered unwon, releases re-parked, ↵Arthur Hanson Rasmusson12 hours
| | | | 1500ms tick, 55s/64-try episode bound; GSP_PHYS_GR_RC_MISSABORT -ENOMSG), junk-operand filter (0x10000000..0x100000000 reloc misses not demanded - float constants), MME prelude latch cleared on WEDGE-RECOVER - **FIRST-EVER nonzero beef submit+retire** (GR LAUNCH submitted push=0xcf3d724 len=5016 pages=9 words=389, poll break ch=0xc0ffee02 after 64us, 12 poll breaks, 0 poisons; client 0.077->287-436 FPS); still NO-STORES (bounce 0 pages changed by engine); next: SKED CTA counters for prove dispatch-vs-degenerate + fix vertex/index operand state - SESSION 153 r1109-r1111
* vgsp: demand set 64->512 + ring-overwrite (no silent drop; ↵Arthur Hanson Rasmusson12 hours
| | | | head/total/evicted in debugfs sema_setup_demand), EBADMSG return authors software release (suppresses synth), ZEROWIN-SKIP (all-zero 3D window software-released, no engine submit) - verified: demand 197 VAs past cap, MISS-ABORT convergence (submit only at miss=0), ZEROWIN-SKIP no-poison; store-bearing window still not achieved (r1106 retire root-caused zero window); r1108 slow-death (Wait for channel idle timed out + red screen on SKIP edge); next: re-arm candidate on MISS-ABORT (not consume) - SESSION 153 r1107-r1108
* vgsp: PBENTRY first-sighting watch (all 12 PBDMA intr0 bit18, ↵Arthur Hanson Rasmusson13 hours
| | | | PB_HEADER/COUNT/DATA0/HDR_SHADOW dump pre-rearm, 8/boot) + PB-FAULT-DUMP voff/tail/past-end; sema_gl_sdm_drop (op-0/2/6 single-word SDM entries from preludes+window); reloc level-2 fallback; sema_gl_miss_abort (pre-doorbell abort on residual MISS, feeds pin demand); gsp_sema_demand_note + attr-prelude ptr pages demand-recorded + sema_attr_settle_ms=8000 - **FIRST BEEF WINDOW RETIRES ON REAL HW** (poll break ch=0xc0ffee02 after 68us, 737-word block) but no-store retire (0/4 pages changed, NO-STORES); residual MISS flood (miss=11..17) blocked by FULL demand set (64/64 MAX) - SESSION 153 r1103-r1106
* vgsp: sema_gl_attr_zero_fix (default 1) - torn-read zero guard drops ↵Arthur Hanson Rasmusson14 hours
| | | | exactly-zero data words in SET_VERTEX_ATTRIBUTE_A/B INC runs (span=(wr-wout)+1), FE-decode trap regs (0x40a630..c MME_PC/MACRO/GENERATED) in sked sampler - 0x1168 fault root-caused to client's own 20030458 38400000 38200040 00000000 run (A(2)=0 ErrorCode 0x5); r1102: ZERO Class Errors (fault class cleared); next wall: PBENTRY fault intr0 bit18 (exceptType=0x20, pbfetch=0x101ebc block end, gp_put=5) - SESSION 153 r1101-r1102
* vgsp+diag: attr/pipeline prelude (families 0x1100-0x1200/0x2000-0x2060, ↵Arthur Hanson Rasmusson14 hours
| | | | dedup, 512w, program-pair VA record; emitted via patch_window relocation, run-aligned after MME prelude), mature-prelude gate (>=64 0x114 code words, sema_gr_prelude_gate_ms=60000 busy-defer), pin_gr_stream 0.75s pass, trap regs in sked sampler - first clean DS+MME (ds_esr=80000000 mme_esr=80000000, a2040a00 gone); 0x1168 CLASS ERROR persists as MME-MACRO-GENERATED WRITE (no 0x1168 header in emitted stream; macro(2) args [0,0,600000,0,610800]); next: disassemble macro (2) at pc 0xa + trapped-addr read pre-latch - SESSION 153 r1095-r1100
* vgsp+diag: debugfs sema_setup_demand (armed setup-hold VA set, 0444 ↵Arthur Hanson Rasmusson15 hours
| | | | seq_file) + pin_gr_stream demand loop reads it (kills dmesg-ring race; 20s resend/300s expiry), bounded 3D raw dump of pre-drop window - MME-PRELUDE emitted gen=2 words=512 (context alive submissions=760); NEW WALL: first beef window loads context + clean MME (mme_esr=80000000) then Class Error ChId 0001 Class 0xc597 Offset 0x1168 (SET_VERTEX_ATTRIBUTE_A(2)) ErrorCode 0x5 + poison; emitted attr runs byte-faithful; next: attr/pipeline prelude (0x1100-0x1200/0x2000-0x2060) + gate first launch on mature prelude - SESSION 153 r1091-r1094
* vgsp: MME setup prelude - capture client MME context-setup runs ↵Arthur Hanson Rasmusson17 hours
| | | | (0x114..0x124/0x234/0x550..0x568/0x3400..0x34fc) verbatim, exact-run dedup, 512-word buf + gen counter (sema_mme_prelude_cap); one-time-per-generation preplay in GR launch build (sema_gl_mme_prelude); capture verified - full macro program 11 SA/IR pairs ~206 code words, MME-PRELUDE emitted gen=5 words=444; no further mme_esr=80000004 after prelude (r1079); remaining: setup push unreadable=1 skip before pin demand; r1082-class content-independent dispatch stall - SESSION 153 r1078-r1082
* vgsp: in-flight MME/fault capture (ds_esr/mme_esr/mme_pc/mac/ir/mth regs at ↵Arthur Hanson Rasmusson17 hours
| | | | 5ms sked sampler), PB-FAULT-DUMP+WINPROBE+window-dump budgets raised, BLOCKVAL stream validator, subdrop (multi-word runs subchannel!=0 dropped), sema_mme_census - ROOT CAUSE: real MME ILLEGAL_OPCODE on CALL_MME_MACRO(2)=0x3810 with no macro program loaded (mme_esr=80000004 mme_pc=0 mme_ir=1); client's MME setup (0x120 LOAD_MME_START_ADDRESS_RAM, 0x124, 0x3400 scratch) never forwarded; 0x1c00+j*16=SET_VERTEX_STREAM_A_FORMAT(j); next: MME setup prelude injection - SESSION 153 r1073-r1077
* vgsp: svc_push_page_heal (beef push all-zero read re-resolves via ↵Arthur Hanson Rasmusson18 hours
| | | | launch-page chain, pin/cache, src=10; sema-pushzero len cap 1024), mme-reloc-rsv (MME-family payload miss resolves via RELOC-FIXED ledger/pin/walk shadow-map), pin_gr_stream pushzero 4K-page bulk demand (48/batch); HW retire only with A+0x34a8+C dropped - draw payloads live in exactly the refused families (0x1c00-0x1c20 MME macro-code uploads / 0x3810 CALL_MME_MACRO); Graphics Exception ChID 0001 Class 0x0000c597 Offset 0 - SESSION 153 r1070-r1072
* vgsp: sema_gl_mme_reloc (CALL_MME_MACRO/DATA payload relocation via launch ↵Arthur Hanson Rasmusson20 hours
| | | | pages, raw-operand fault fix), MME-page collection + 4KB ptr dedup, 0x1b00 pair-relocation, sema_gr_wedge_recover (TSG-preempt re-arm on 0xbadf, revived channel: REAL GR SEM OK 0xc0ffee01), win_drop_auto += 0x1b00 - BEEF LAUNCHES RETIRE (h=0xbeef0100 ch_rel=0xc0ffee02 elapsed_ms=0 ok=6, 3 consecutive); full-perimeter windows still wedge in [0x1200,0x7ffc); client registration 5/5 with keeper - SESSION 153 r1058-r1063
* vgsp: sema_gl_idx_reloc knob (idx-buffer/vertex-stream shadow-VA relocation, ↵Arthur Hanson Rasmusson21 hours
| | | | default 0 = r1035-r1041 set), win_drop lo-hi ranges, engine-entry logging (e0/e1 in GR LAUNCH, ring[3] in poll break) - entry encoding IDENTICAL X vs beef (e0=0x00101000); discriminator = raw operand VA in window content faulting GR once and wedging ladder (cwd0=badf1201, fecsirq=0x10); next: relocate CALL_MME_DATA(j)=0x3804+j*8 operands + wedge recovery - SESSION 153 r1042-r1057
* vgsp: 3D flush gate fix (built-in chunk bound, not bisect knob), pushzero ↵Arthur Hanson Rasmusson22 hours
| | | | zero-content rescue (probe PROMOTE_CTX, src=9), CHUNK1 dump, SET_INDEX_BUFFER+vertex-stream collection; publish GPFIFO entry at engine GP_GET (PUT-GET=1); win_drop 0x parser fix - beef chunks still PBDMA-stop at 0x1160 attr run (hdr=20030458 INC m=0x1160 cnt=3) while X retires ch_rel=0xc0ffee02 same boot - SESSION 153 r1034-r1041
* vgsp: 3D stream chunking for client pushes - GR push words 512->2048, ↵Arthur Hanson Rasmusson24 hours
| | | | candidates 4->8, method-aligned <=384-word chunks in ring order with 8-pending backpressure (sema_gl_chunk_words knob), zero-head guard, PBDMA chunk-end fault fix, 3DSTREAM dump; first client src=stream capture (h=0xbeef0100 push=0x54eefd4 len=1580 nprog=2 recs=2), ch_rel=0xc0ffee02 submitted; beef windows still don't retire (scheduler stall vs X retiring same boot) - SESSION 153 r1028-r1033
* vgsp: RUSD INIT_USER_SHARED_DATA (0x20800afe) handler + gsp-phys-shared-bit0 ↵Arthur Hanson Rasmusson43 hours
| | | | knob - direct client channel alloc gate broken open (c46f hObject=0xbeef0101, err@ReadPixels=0x0 px=255,0,255,0) - SESSION 153 r999-r1002
* vgsp: full-frame strip assembly (origins+per-channel CE state, delta push ↵Arthur Hanson Rasmusson45 hours
| | | | fold), FERMI_TWOD executor (screen fills rc=0), RT-as-screen surface alias + win-drop auto, client replay retires (faulted=0, GR forward failed=0) - SESSION 153 r984-r997
* vgsp: sked fe0/fehost/fecsirq lanes + FECS post-mortem readback + ↵Arthur Hanson Rasmusson2 days
| | | | sema_gl_win_patch knob (SESSION 153 r959-r961)
* vgsp: per-channel C597 attachment table, sema_gl_win_drop strip knob, ↵Arthur Hanson Rasmusson2 days
| | | | WINPROBE; strip experiment: binds-only windows retire, full draws die in FECS switch-in (SESSION 153 r955-r958)
* vgsp: PB-FAULT-DUMP + pb block preservation (warm-sem probe off=0x800), ↵Arthur Hanson Rasmusson2 days
| | | | attach-on-BEGIN; init replay retires ch_rel=0xc0ffee02; draw windows remain unreleased (SESSION 153 r951-r954)
* vgsp: staged program delivery (V3 SPH accepted by engine, 0 SPH faults/40 ↵Arthur Hanson Rasmusson2 days
| | | | launches), inline bytes 256, per-VA typed slots (SESSION 153 r947-r950)
* vgsp: sticky C597 attachment + BEGIN recognizer + draw-scan window; draws ↵Arthur Hanson Rasmusson2 days
| | | | captured and replayed on 3D lane (engine reaches program header validation); guest-page blob delivery; two mdevs coexist (SESSION 153 r944-r946)
* vgsp: 3D method-stream replay lane (verbatim C597 window + shadow relocation ↵Arthur Hanson Rasmusson3 days
| | | | + release tail; engine retires real 3D methods, no QMD) (SESSION 153 r932)
* vgsp: mode-35 plan-F 3D class object accepted on V7; GL client C597 ↵Arthur Hanson Rasmusson3 days
| | | | section-open + draw-synth path; push dumps (SESSION 153 r923-r926)
* vgsp: GL section capture + synthesized launch reaches the engine (SESSION ↵Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | | 149 r888) - M1.1a: preserve the bracketing release across the section-open reset; allow hash-verified replay capture (exactly-once guard). First trusted pass now captures the GL setup section (push=0x120010004, prog=0x121201b00, rel=0x120001000, replay=0). - M1.1b: per-channel ring scan 16->24 slices + put-triggered keep-alive. - M2-S1: booter skips w48 Tier-1 when shadow_win0 is set; flush attaches the observed LID payload as an inline record. - Evidence (r888): 'qmd_word 48 page copied to window-0 shadow'; 'GR LAUNCH submitted pages=5 shadow=1 qmdwin=0x108000'; FAULT-CAP code_src=win0, code64 non-zero, FAULT_PDE count=0 (engine fetched the shadowed code). - Remaining wall: SKEDCHECK06_REGISTER_COUNT + SKEDCHECK16_CTA_THREAD_DIM_ZERO (synthesized QMD scheduling fields) -> bounded fail + SW release. Feature gated by sema_gl_xlate (default off).
* vgsp: perf tracing + map-verified rescue + us RPC quanta (session 148)Arthur Hanson Rasmusson4 days
| | | | | | | | | | | | | | | | | - sema_trace_h + sema-perf logs: per-second channel state (put/last/armed/ counter) and publish-gate reasons, plus maprescue attempt outcomes. Instrument added after window tuning failed to move the 22.4 s/frame glxgears cadence; it located the real blocker in one run. - svc_rescue_push_page: content-verified full-chain map rescue (works for GR channels, whose rescue was previously refused outright). - gsp_phys_booter: RPC reply waits now usleep_range(150-350us) instead of msleep(5) (termbin review suggestion, in-tree constant verified; no FPS change measured, kept as correctness-neutral latency reduction). Evidence: r843-r847 dmesg_live logs; measured constant 22.4 s/frame for glxgears across all param windows (grace/poll/first-poll), showing the pacer is elsewhere - traced to client pushbuffer pages (va=0x3f06270-class) failing svc_lookup (-ENOENT) while the client is alive.
* vgsp r801: gsp_phys_gr_min_data=0 is an explicit bypass (the ↵Arthur Hanson Rasmusson5 days
| | | | | | | | | | | | | | | no-external-data launch class), not a clamp to 1 The Vulkan compute launch's single QMD pointer pair names its own QMD window and its params are scalars (0.2/640/480), so n_data=0 is correct for it; the bare n_data>=min threshold refused it as degenerate. The bypass is safe now that gsp_phys_chunk_page_va answers NULL for an unallocated chunk (r800) - before that guard the empty-set post-mortem print dereferenced a garbage pointer (Oops at gsp_phys_backend_guest_gr_submit+0x4394). Default stays 16; 0 is opt-in for the egg-break test that forwards the launch to the real engine so the copy-out can carry pixels back. Proper launch-internal predicate next.
* r800 SESSION 141 walk guard: gsp_phys_chunk_page_va returns NULL for an ↵Arthur Hanson Rasmusson5 days
| | | | unallocated chunk (no per-page array AND no base va) - it previously returned (u8*)NULL + page*PAGE_SIZE, a small non-NULL garbage pointer, so the caller's NULL guard passed and the empty-set post-mortem print dereferenced it (Oops #2 at gsp_phys_backend_guest_gr_submit+0x4394, CR2=0x1e93c); with the guard, min_data=0's no-external-data forward can proceed
* vgsp-sema: SESSION 127 C5C0 TRANSLATOR v1 - the Vulkan compute launch ↵Arthur Hanson Rasmusson6 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | captured, filled and forwarded (and its copy-out executed) through the CUDA QMD machinery The Vulkan client's compute dispatch is an inline-QMD launch that lives in the PRE-BUILT ring continuation beyond GP_PUT (the earlier analysis only inspected the put-bounded pushes' idx7/idx12 and concluded 'no inline QMD'): beec0100 idx23 carries SET_INLINE_QMD_ADDRESS_B (0x31c = 0x4b85f, the upper half absent) plus the 64-word LOAD_INLINE_QMD_DATA (0x320) run - grid 40x30, prog=0xdfd8feb900 (the SASS page), param=0x4b86000, release0=0x4b85e08/1 - and idx24 carries the TURING_DMA_COPY_A (0xC5B5, subch 4) image->buffer copy-out (src=0xdfd8d60000 dst=0x050b0000, 640x480 RGBA rows) plus the SEND_PCAS schedule (0x2b4 = qmd>>8, 0x2bc = SCHEDULE). Pieces landed (all service-thread, all bounded/logged): - svc_gr_ahead_push (the sema_gr_ahead sweep parser) is now a strict tail-aware translated decode of one continuation entry: it accepts a method stream whose final inline run is completed from the NEXT ring entry (the driver splits big LOAD_INLINE_DATA payloads into a dedicated data slot entry: beef0100 idx00->idx01 256 B, idx08->idx09 12288 B), feeds the C5C0 inline-QMD window to svc_qmd_accum (capture via the standard CUDA QMD candidate), materializes GRCE/KEPLER_INLINE_TO_MEMORY_B LOAD_INLINE_DATA writes into the VA shadow exactly like the main decode (once-only keyed; recorded in the dst ring for the fill's inline-served pages), parses the C5B5 copy operands (0x400/0x404/0x408/0x40c/0x410/0x414/0x418/0x41c, 0x300 LAUNCH_DMA, 0x700/0x708 remap) into a pending copy list, and queues the release groups minus a release a live launch candidate owns (that word belongs to the completion edge, so the client's fence can never clear on a pre-render image). - The parser reads the run data words correctly (w3 fix; the GRCE 0x18c operand was being read from the next run's header word, which sent the 1568 B parameter-bank write to a bogus destination). - svc_qmd_accum: accept the ADDRESS_B-only inline-QMD form (implicit zero upper half) so qmd_dev_va is real for the Vulkan launch. - svc_gr_page_note + svc_gr_req_fill: resolve a client-VAS page through the registered-VAS walk when the trusted sources do not carry it (the Vulkan QMD's program/parameter/descriptor pages are client-managed vidmem; the fill refused the whole launch with -ENOENT before this). - The QMD capture's inline recs are filtered to the pages the captured QMD names (a stale session-wide rec destination made the fill refuse with ENOENT va=0x80412000 qmd_word=NONE). - Pending GRCE copy-out list executed on the launch completion edge (svc_fwd_hold_gr_release and svc_fwd_outcome_check) before the guest release is authored, with a bounded no-forward fallback at the sweep; the copy itself runs through the standard svc_copy executor (observed: 'copy done h=0xbeec0100 dst=0x50b0000 bytes=0x12c000 rows=480 rc=0'). - The pump offers a sweep-captured launch every pass (svc_gr_request) - the continuation launch's only dispatch path - and retires pending copies when no forward is owed. - gsp_phys_gr_min_data (module param, default 16): the degenerate-data-set refusal threshold is now tunable; the Vulkan dispatch's operand set is a handful of records, not the CUDA-era 968. - Bounded witnesses for the request paths (first-N/periodic: no-candidate, backend-not-ready, hold-busy, inflight-busy, fill-ENOENT) and the capture tail dump is now sema_dbg-gated with a 4/boot budget (it flooded dmesg). Live status (r726, cycles r719..r726): the client runs green through the fence (VKFRAME seq/DONE/IDLE, 3 PPMs) with no in-flight client assist; the sweep captures the launch ('launch captured ... qmd=0x4b85f00 prog=0xdfd8feb900 param=0x4b86000'); the fill resolves the QMD pages (page-note src=walk) and now passes its ENOENT gate; the copy-out executes for every frame. The physical launch itself is still refused: the fill yields n_data=0 because the launch's operand CONTENT (the SASS at prog=0xdfd8feb900, the descriptor data) is delivered by the driver through the MME / inline-to-memory engine upload stream (beec0100 idx7/idx12, beef0100 idx08), which the emulation does not execute - the pages the QMD names stay zero in the emulated vidmem. That engine-side upload path is the next work item; sessions r719..r726, evidence in the handoff. No host reboots; 0 oopses; the fetch of the CUDA battery on a fresh boot follows separately.
* vgsp-sema r633: park a held GR launch's push-level release records and ↵Arthur Hanson Rasmusson7 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deliver them on the eventual forward (the r632 mail hang), gate synthetic shadow pages on an anchored span with per-episode budgets (the r632 junk-candidate flood), and name the release0 redirect cause The r632 first boot's mail client spun forever in cudaDeviceSynchronize after its vadd launch: the push-level tracking release (0x20480fff0 pay=112) never landed while the client's word stayed at 111, even though the launch itself completed (GUEST GR LAUNCH OK) and its QMD release was software-applied. Root cause: the release records (svc_fwd_rel[]) are per-decode state (reset at the top of svc_decode_stream), and the two GR hold paths (busy-defer hold, m150 data-stalled hold) break out of the pump entry loop BEFORE the release block (hold-engage / inline apply) runs - so a held launch's records are dropped at the pass boundary. The eventual forward (queued from the capture ring at the defer cap, without a re-decode of the entry) then engaged the release hold with whatever records that pass happened to carry (often none, or another entry's). Live evidence: read_proc_va on the spinning client - 0x20480fff0 = 111 (payload 112 never applied), 0x204827f74 = 6 (QMD release delivered); host log - no sema-sw line for 0x20480fff0 val=0x70 anywhere. Fix (parked releases, keyed by vgpu/channel/push VA): both hold paths park the pass's records (svc_grpark_put) before their break; svc_gr_request restores the queued push's parked set into svc_fwd_rel[] at the consume point so the pump's hold-engage holds exactly those records; the two fail-closed REFUSED paths deliver the parked set inline; the session flush clears the table. Bounded log line "GR parked releases restored" (32/boot). Proven live: r633 first boot restored push=0x20043c1c8 n=1 va0=0x20480fff0 pay0=100; r633 mail now completes (VADD bad=0 checksum=12204516.000000, STATUS OK) on two consecutive fresh boots. Also r633: the r632 synthetic shadow caps were per-FILL-CALL and the fill re-runs on every retry pass, so one probe=none junk argument candidate could mint another 1024 synthetic pages per pass - the 8192-page shadow table filled within seconds of the r632 first launch (va-shadow 8192/8192, all synthetic, from the two 16 MB guessed spans of candidates no resolver ever answered), which then killed the lane for every later real page ("shadow table full ... write dropped"). Two changes: - ANCHOR GATE: a synthetic page is created only when the candidate's span already has >= 1 page answered by ipin/resolve/ledger. Existing H2D shadow content still serves unconditionally (the red-draw chain), and a real buffer's unresolved tail stays anchored by its resolved head. A candidate whose every page is answerless takes the historical drop path - its guessed span is not a buffer; - PER-EPISODE BUDGETS: the per-candidate (1024) and per-launch (4096) caps now persist per (push VA, span base) across the episode's passes, not per call. Measured r633: shadow table peak 2048/8192 (vs 8192/8192 full), 32 synthetic pages for the mail launch (vs 4041 junk-inflated n_data), the lane stays available for content serves. Diagnostics (bounded, one boot of evidence): "vgsp-work: gr-rel0 ..." names the release0 resolution chain (rel_pa/ap/hpa/word/pay/unobs) and "vgsp-gr: qmd-rel redirect: ..." names the backend redirect cause. The first live reading answers the release-target question: rel_hpa resolves (0x2fc495000), map_ret=0, unobs=0 - the redirect is the deliberate gsp_phys_rmcall_gr_rel_mod=1 default (m64 forced module-semaphore release), not a resolver failure. The r632 hang was the push-level release loss, fixed above. Build clean (kernel + r65535 warnings only).
* vgsp-gr r597: emit the captured prelude's SEM_EXECUTE ACQUIRE groups ahead ↵Arthur Hanson Rasmusson8 days
| | | | | | | | | | | | | | | | | of the rebuilt window (default off) The verbatim prelude replay breaks the launch (r599: qmd_rel=0 after an 8.9 s stall), so the acquire cannot be taken by replaying the prelude. This takes only the wait method groups - C46F addr 0x5c count>=5 with EXECUTE&7 != 1, and C5C0 SET_REPORT_SEMAPHORE A..D with a non-release op - and copies them verbatim before the proven init+window+PCAS+release block, leaving the window untouched. Gate: gsp_phys_rmcall_gr_acq (default 0). Live A/B on r601: the mail launch's prelude carries ZERO acquire groups (the C5C0 prelude is inline parameter data + release only), so emitted=0 and the VADD race is unchanged - the driver assumes its relocation copies precede the launch without an engine-visible wait.
* r596 diag: print the raw-prelude geometry the verbatim gate tests (bounded)Arthur Hanson Rasmusson8 days
| | | | | | | | | | | | The r595 A/B could not be judged from the log: with prelude=1 the r593 boot never reached the build path (r590b deferred first) and no geometry line existed. Print raw presence/raw_words/hdr/end/want_end for the first eight prelude-gated launches so the gate is verifiable. r599 measurement: raw=1 raw_words=210 hdr=138 end=205 want_end=205 - the gate IS met; the verbatim prelude is replayed (words=205) but the launch then completes with qmd_rel=0x00000000 after an 8.9 s stall (engine never ran; software release), so prelude=1 breaks the launch.
* vgsp-gr r564: refuse a degenerate launch data set before the engine sees itArthur Hanson Rasmusson8 days
| | | | | | | | | | | | | r565 evidence: the fill dropped 4 argument bases (GRFILL-DROP), RDW showed nrecs=2 where healthy launches run 968-969, the launch was still submitted and the backend timed out after ~19 s (GR forward failed rc=-110); the tail then authored the software release and the client kept spinning on a completion for a kernel that never ran - 20 s of backend occupancy plus a false completion, and a plausible wedge source for the following clients. gsp_phys_gr_submit now refuses n_data < GSP_PHYS_GR_MIN_DATA_RECS (16) with a loud line, failing fast into the same software-completion fallback without touching the engine.
* vgsp-gr r552: retire-wait settle window 300 -> 100 ms (launch p50 1119 -> ↵Arthur Hanson Rasmusson8 days
| | | | | | | | | | | | | | | | | | | | | 651 ms) Measured A/B on real boots (same module, same client set, only gsp_phys_gr_settle_ms varied): - r549 (default 300): 32 launches, min 862 / p50 1119 / p90 2363 ms; retire wait 680-901 ms per launch ("(settled)", 1..64 of 1000+ tracked pages changed). - r551 (100): 22 launches, min 476 / p50 651.5 / p90 1010 / max 1428 ms; retire wait 231-247 ms. Battery 12/15, 0 hangs (unchanged profile; the 3 fails are the streams2/streams_events/memset2d readback class). Why the wait was the dominant cost: a read-mostly kernel writes far fewer pages than the launch's tracked set, so the "changed >= n_all" shortcut never fires and every launch settles through the content-stable rule, which paid the full window plus scan overhead. The stability verdict is still hash-gated (m180); only the minimum stable duration shrinks, and the battery correctness profile is unchanged. 300 remains available through the module parameter.
* r494 the GL 3D-channel classification: NV2080_ENGINE_TYPE_GRAPHICS (1) is a ↵Arthur Hanson Rasmusson9 days
| | | | | | | | | | | | | | | | | | | | | GR channel With the real-GSP KGR snapshot in place the X driver's 3D-engine selection succeeds; the next wall was 'Failed to initialize DMA' / 'Failed to allocate push buffer' - the X driver's pushbuffer NOP test never saw its GPFIFO entry consumed. Root: the wire's NV_CHANNEL_ALLOC_PARAMS engineType carries NV2080_ENGINE_TYPE_* (NULL=0, GR0/GRAPHICS=1, COPY0=9); CUDA's kernel-RM GR channels arrive with NULL=0 and the progress service's convention is 'engine_type == 0 means GR', so the X/GL 3D channel's real GR0 value 1 was misclassified as a CE - the CE path then demanded a ring translation the channel does not have (rrc=-2 'norrng'), never probed the USERD, and the pump never executed a single entry (submissions=0 at the 10 s timeout). Normalize GR0..GR7 (1..8) to 0 at channel registration; every other engine id is >= COPY0 (9) so no CE can be captured. Also fixes the KGR snapshot max-size constant: KGR_GET_INFO is 3712 bytes (8 engines x 58 x 8), which the 1664-byte cap silently dropped - the one snapshot the engine selection actually needs.
* r493 THE 3D-ENGINE WALL: serve the live GSP's own KGR static data to the guestArthur Hanson Rasmusson9 days
| | | | | | | | | | | | | | | | | | | | | The X driver's nvSelect3DClass -> nv3dAllocDevice maps GR_GET_INFO index SM_VERSION (0x0c) through an SM-version -> SPA-version table; the emulated KGR_GET_INFO fill never set index 0x0c, so SM_VERSION read back as SM_VERSION_NONE (0) and every Xorg/EGL session died at 'Failed to select a 3D engine' (the class list and the engine list were never the problem: the X log carries neither 'Failed to determine graphics capabilities' nor 'Failed to determine engines list'). The physical backend already asks the LIVE GSP the real client's whole KGR query set at mode-35 boot (gsp_phys_rmcall_user_lane_kgr_queries) and threw the replies away. Retain them (generation-keyed, invalidated with the backend static cache) and serve the identical bytes in the guest-side GSP_RM_CONTROL dispatch for the same commands, so the guest's GL/3D stack reads the device's true GR static values (SM_VERSION 0x75, warps, caps, topology, ctx-buffer tables) instead of fabricated approximations. Falls through to the emulated handlers unchanged when no capture exists (backend not READY, other boot mode, cmd not captured) and never answers with more bytes than the guest requested.
* r452 change-detection fingerprint fix: the 32-bit ADDITIVE page checksum ↵Arthur Hanson Rasmusson10 days
| | | | used for the copy-out 'engine changed this page' decision is provably blind to power-of-two in-place scales - doubling every word of a 4 KB page shifts each exponent by +1 (+2^23/word), and 1024 words give 1024*2^23 = 2^33 == 0 (mod 2^32), so c[i] *= 2.0f left csum(post) == csum(pre), the page classified 'unchanged', the copy-out was skipped and the client read the PRE-scale values (the deterministic streams2 / streams_events / in-place-scale class, reproduced by ip_probe2: the chunk holds 0x43010000 (129.0) while the guest page holds 0x42810000 (64.5) and csum post==pre). Replace every change-decision fingerprint (scan_pages, retire wait, copy-out, gap copy-out, window witness, prev-out late-store witness, idempotent launch fp) with FNV-1a 64 over the page; the additive sum survives only as the all-zero filter (gsp_phys_gr_page_sum32). New r452 diagnostics: OUT-UNCH witness (unchanged page: chunk vs guest first words, the line that cracked it) and gsp_phys_gr_scan (whole-chunk pattern scan)
* r449 shader-constant-cache invalidate experiment (gsp_phys_gr_ccache_inv, ↵Arthur Hanson Rasmusson10 days
| | | | default off): the forwarded QMD's cache-control block (w5) always carries bits 26..29 and the M91 fix adds bit 30 (instruction), but the V01_06-layout SHADER_CONSTANT_CACHE position (bit 31) is never set - and the forwarded param page is rewritten in place between launches (two-bank win0 pool), so a stale constant-cache line can serve the PREVIOUS launch's parameters (stale pointers -> the current launch's output pages untouched = the post-dispatch no-store signature). Mode 1 sets bit 31, mode 2 forces bits 24..31; only applied when the standard signature is present
* r445b wedge hard re-arm trigger: the r445 battery shows every post-dispatch ↵Arthur Hanson Rasmusson10 days
| | | | no-store episode leaves the warm ladder sem probe ALIVE (the ladder block runs; only the client launch no-writes), so the re-arm now fires on the FINAL no-write retry - TSG PREEMPT bWait=1 (forced context save/unload) + lane-enable/schedules re-run - before the last re-publish; default off
* r445 wedge hard re-arm experiment (gsp_phys_gr_wedge_rearm, default off): on ↵Arthur Hanson Rasmusson10 days
| | | | the proven engine-wedged warm-sem-probe failure, one bounded recovery attempt - TSG PREEMPT bWait=1 (forced context save/unload) + the m155 extended re-arm (mode-35 lane-enable tail + GPFIFO schedules + doorbell) + a repeated probe - before the historical WEDGED break
* r441 zero the guest-visible release words on the FIRST attempt too (not only ↵Arthur Hanson Rasmusson10 days
| | | | outcome retries): the r439 evidence shows a first attempt breaking its poll on a stale value (qmd_rel=0x5 from the previous launch's engine write, want=6) while the engine never fetched the deposit (sentinel intact) - the host then shipped a completion for a launch that did not run; zeroing forces the poll to wait for this launch's own engine write, so the timeout ladder and the software-authored completion are the honest path and the wedge classification can no longer be satisfied by a stale value
* r436 full sysmem PTE aperture/VOL matrix: gsp_phys_rmcall_pte_coh becomes a ↵Arthur Hanson Rasmusson10 days
| | | | 4-mode selector (0=0xf aperture3+VOL historical, 1=0xd aperture2+VOL r431, 2=0x5 aperture2 non-volatile, 3=0x7 aperture3 non-volatile) applied consistently to the leaf PTE and the single PDE (bit0 = IS_PDE there), so the r431 finding (aperture 2 with VOL still 1 is a pair the reference RM never emits) can be A/B'd against the consistent encodings
* r434 wedge instrumentation + CWD-slot path: post-retire QMD64 dump ↵Arthur Hanson Rasmusson10 days
| | | | (hardware-only w50/w51 INNER get/put = the SKED's CTA dispatch counters, which the submission-time dump cannot show) so the healthy-vs-wedged diff proves whether the grid was dispatched and truncated; post-OK qmdwin reads the launch's own bank; gsp_phys_rmcall_gr_cwd_slot >= 0 emits the CUDA driver's CWD-slot PRI write (SKED_DEBUG_1 ENABLE_CWD_SLOT_COUNT via falcon scratch) + SET_CWD_SLOT_COUNT in the init preamble for the mission's slot-count A/B (-1 = off)
* r433 RELEASE_MEMBAR_TYPE A/B: gsp_phys_gr_rel_membar forces QMD word11 bit14 ↵Arthur Hanson Rasmusson10 days
| | | | (FE_SYSMEMBAR) on forwarded GR launches so the engine flushes pending sysmem stores before writing the release semaphore - the post-dispatch store-loss signature (release host-visible, data stuck in L2) is what an unflushed write-back L2 looks like from the host; 0 keeps the guest's captured FE_NONE for a one-flip A/B
* r431b revert the coherent-PTE default: the clean-boot A/B showed aperture 2 ↵Arthur Hanson Rasmusson10 days
| | | | (SYSTEM_COHERENT) is worse on this platform (the first client of the boot wedged with 0/983 and the wedge persisted across the sweep), so the historical NON_COHERENT aperture stays the default; gsp_phys_rmcall_pte_coh=1 remains for experiments (it did make the engine's partial 1-page writes visible sooner)
* r431 sysmem leaf-PTE aperture fix (root-cause candidate): map every host ↵Arthur Hanson Rasmusson10 days
| | | | dma_alloc_coherent page as SYSTEM_COHERENT (aperture 2) instead of SYSTEM_NON_COHERENT (3) - the reference RM picks the aperture from the allocation's CPU cache attribute, and non-coherent L2 let engine stores linger invisibly (the no-execute wedge) while re-staged copy-in content stayed invisible to the engine; gsp_phys_rmcall_pte_coh=0 restores the historical aperture for a boot-time A/B
* r430 wedge dispatch-window sampling: skedrec samples every poll iteration ↵Arthur Hanson Rasmusson10 days
| | | | (the release lands ~7ms after the doorbell; the old 8-iteration gate saw at most one sample), full 64-word QMD64 dump per launch (hardware-only w50/w51/w52/w53 included) for the healthy-vs-wedged diff
* r427 idempotent fast-accept: a zero-write launch whose identity (QMD minus ↵Arthur Hanson Rasmusson10 days
| | | | release0 + staged content checksums) matches an earlier writing launch accepts immediately (no retries, no quiet holds); recorded for every writing launch; wedge diagnostics retained
* r425 wedge diagnostics + init-once A/B: SKED/FECS/TPC/CWD register witnesses ↵Arthur Hanson Rasmusson10 days
| | | | (healthy vs wedged baselines in-log), prev-out late-store witness, gsp_phys_rmcall_gr_init_once (emit the compute init preamble once per boot vs per launch)
* gr: r420b author the guest QMD release on every outcome when release0 is ↵Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | redirected With gsp_phys_rmcall_gr_rel_mod=1 (release0 -> module semaphore, the ordering fix for the engine-release-before-copy-out race) the engine never writes the guest's own QMD release word, so the host must author it on EVERY backend outcome. The hold's success/fail edges do; a launch with no deferred release records (no hold engaged) fell through and its client hung on a word nothing would ever write - the r414 saxpy f0 cur=5 stall. The backend now reports qmd_rel_redirected in the copy result whenever it points release0 at the module semaphore (rel_mod, or no targetable guest release page). svc_fwd_gr_work folds that into the existing gr_rel_unobserved latch, which makes the bounded outcome check author the word on done (success or failure) - the same forward-only write the hold edges perform. rel_mod=0 paths are unchanged (the engine still authors the word and the latch is only set on an unresolved release page).
* gr/sema: r420 pause completion deadlines under the backend wedge-quiet holdArthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The r419 soak reproduced the user's mid-boot degradation with the wedge fix in tree: victory_matmul printed bad=65536 while its launch was held by the booter's WEDGE-QUIET window - the client was released ~115 s before the post-hold accept. Root cause: two sema-service deadlines are shorter than the deliberate quiet hold and fire under it, authoring the launch's completion release (and applying its recorded releases) while the engine has not run and the copy-out has not happened: - the 12 s GR release hold (SVC_FWD_GR_HOLD_TIMEOUT_MS) expiry (GR-STALL-fwd-hold ... releases + QMD word software-applied), and - the 3 s per-launch consumption bound (gsp_phys_gr_hold_max_ms) firing the busy-defer consume-with-software-release path, whose inline apply authors the capture's QMD release (GR-REL-AUTHOR src=hold). Evidence (r419 live): push be94 captured 131333.687, WEDGE-QUIET holds 40 s + 80 s until 131345.598, accept 131346.552 - while the sema hold expired at 131230.978 (GR-STALL-fwd-hold waited_ms=12032) and the inline apply authored va=0x204027f7c pay=6 at 131341.734 (h=0x5c00001f). The client's verify completed on the authored word and read stale data. Fix: the booter publishes gsp_phys_gr_quiet_hold_active() exactly around the quiet msleep (bounded by construction). The sema service pauses under it: svc_fwd_hold_expired() pushes the GR deadline (bounded by a 600 s defence-in-depth cap, announced once per hold as GR-HOLD-PAUSE); svc_gr_hold_bound_expired() pushes the per-launch bound (GR-HOLD-BOUND-PAUSE); and the busy-defer one-shot window re-holds while the flag is set so the consume-with-sw-release path cannot run during a quiet hold. The held launch is then forwarded and copied out after the hold, and its completion is authored only on the hold's done edge - after the data write-back.
* gr: r411 sticky wedge episode - quiet the first retry of any zero-write ↵Arthur Hanson Rasmusson10 days
| | | | | | | | | | | | | | launch once the boot has seen a quiet heal (or a no-write accept) r418 with the r410 quiet path showed the wedge alternating one launch at a time: every second push no-wrote and was accepted (streak was reset by the healed launch in between, so the quiet trigger never armed), e.g. c584 NOWRITE / c92c healed / d424 NOWRITE / d7cc healed / ... 6 holds, 6 heals, 7 accepted no-writes. A zero-write accept (or a successful quiet heal) now arms a sticky wedge episode for the boot; any later zero-write launch takes the quiet path on its first retry, so in an episode every wedged launch either heals (engine writes, client data correct) or is classified loudly instead of riding through the quick-retry accept.
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-31 21:25:46 +0000