| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
logging
|
| |
|
|
| |
landing resolver (RELOC-FIXED), param gsp_phys_gr_reloc_resolve default 1
|
| |
|
|
| |
relocation coverage counter, zero-write register dump (diagnostic)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(param/program/banks) resolve through the landing table; zero-write diagnostics
Audit of the QMD relocation/mirror path vs the m180 landing table (r210
evidence: /tmp dmesg, 0 ledger DIVERGE, four multi-page clients still
wrong, 3 zero-write NO-STORES accepts):
The relocation itself does NOT pick PAs: gsp_phys_gr_rewrite_data_vars()
matches a device-VA-shaped 8-byte value against req->data[x].va (the
fill's gr_data list, which m180 fed through the ledger) and rewrites it to
req->data[x].reloc_va. The bounce page for reloc_va is the same page the
tracking wait and the copy-out use, so a *matched* pointer is ledger-
consistent by construction. The un-unified halves of that path are the
page the relocation READS and the page the m129 mirror PATCHES: both come
from pg->pa, resolved by svc_gr_page_note() with the independent sources
(pin/lat/map/walks/scan), never the landing table. The CE inline writes
that author those launches' param/program/bank pages go through
svc_grce_inline_write(), which (m180) records its landing and diverges to
it; svc_gr_page_note could still hand the win0 shadow copy and the mirror
a stale page. Consequences, both observed classes: relocation matches
nothing (the page holds another session's pointers) -> the engine runs
against unrelocated guest VAs it cannot walk -> grid aborts with zero
stores while the ack still lands; or the engine executes on a previous
client's parameter block -> plausible-but-wrong results invisible to the
ledger.
Fix (kernel/gsp_sema_service.c):
- svc_gr_page_note() consults svc_ledger_lookup() first for every
launch-internal page. On a hit it uses the recorded landing (and pins
it, so later resolves and the win0 copy path agree); when the
independent sources would answer differently it logs (bounded)
"sema-ledger: RELOC-DIVERGE va=... ledger_pa=... <src>_pa=... (ledger
wins)" - the requested relocation-vs-ledger witness. Non-hit pages keep
the historical svc_resolve_va path unchanged.
Diagnostics (kernel/gsp_phys_booter.c):
- the m135 below-0x100 guard now shape-checks first and logs
"bank N RELOC-SKIP @off: value is operand page ... (left guest-VA)".
r210 shows the same offset-0 bank pointer in EVERY client (green ones
included: the driver control block), so the guard is unchanged until a
dereferenced pointer in that range is proven; the log makes that
provable in one cycle.
- the zero-write accept now also dumps the deposit QMD words
(gsp_phys_gr_deposit_rb_log) so the next NO-STORES event shows whether
the engine consumed the header words, and the dual-bank sentinel line
is reworded to state facts only (the other bank is normally the
previous launch's legitimately-consumed deposit; the earlier "older
deposit consumed" reading was misleading). r210 classification: all
three NO-STORES events had THIS deposit fetched (sentinel overwritten)
and 0/968..0/967 stores even at +1 s, i.e. engine-side non-execution of
a fetched deposit - consistent with the stale-launch-page abort
mechanism above; the deposit-readback words will discriminate further.
Not changed (deliberate): the m135 guard semantics, the relocation
targets, the sentinel contract, retry/accept policy, and every semaphore
path (mm2/gate/rewind effects untouched).
make kernel -j green (srcversion DAB7E3AA75D364382DD6A3C; only the
pre-existing gsp_phys_rmcall_stage3 printf warning). Main tree untouched;
nothing pushed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
device VA (kills the write/read translation split of the always-failing class)
Context (r209, fc2d1ae5): multikernel/memcpy_roundtrip/streams2/
victory_streams_events still fail with self-consistent, silent wrongness
while every launch reports OK. Evidence: streams2's bad data is exactly the
PREVIOUS client's h_in pattern (engine read stale inputs); the memcpy
window logs 16x "sema-ledger: DIVERGE read ... wrote=0x7f83000
read=0x8783000" (D2H resolved a different page than the write landed on);
the failing four's device operands are all sysmem-backed (ap=2,
pa=0x7800000/0x8800000 in every svc-gr cand[] line), i.e. exactly the
aperture the m171/m172 ledger never recorded.
Root cause class: a data-bearing guest VA had NO single authoritative PA.
Write paths (H2D svc_copy dst, fills, inline writes, engine write-back)
each landed wherever their own resolution said; read paths (GR launch
fill, D2H svc_copy src) resolved independently again. The ledger patched
up some of that, but (a) it recorded only VID-aperture copies >=64 KiB, so
the dominant sysmem case was invisible; (b) it was a 2048-entry RING -
one launch's submission record set (2505-4553 pages in r209) evicted the
H2D records of the same client before its own D2H read them, so the
readback fell back to its own (different) resolution; (c) several
resolution failures returned -ENOENT even when the landing was known; and
(d) copy paths deferred to the forwarded-request machinery skipped the
in-loop record entirely.
Fix - one landing authority, consulted everywhere:
1. The ledger is now a keyed hash table (16384 slots, 16-probe, one entry
per VA page, in-place rewrite). No cross-VA eviction: a client's whole
working set (8 MB in + 8 MB out + engine records) survives until its
readers have run. Rewrites are counted + logged (bounded) because a
write path moving an existing page is the write-side half of DIVERGE.
2. Every guest-visible write records its landing page:
- svc_copy dst_virt: SYS as well as VID (the sysmem half was the hole);
- svc_fill: SYS as well as VID;
- svc_grce_inline_write: unconditionally;
- small copies that are deferred to the forwarded path: recorded at
candidate capture (the software loop never runs for them);
- engine launch data pages: unchanged (submission-time record).
3. Every read/write consults it:
- GR fill: already diverted (m171); now actually has SYS records to
find;
- D2H svc_copy src: per-page divert (m171) + NEW fallback to the
recorded landing when the resolver fails (previously -ENOENT);
- svc_copy dst / svc_fill / svc_grce_inline_write: NEW - resolve,
then land on the recorded page when one exists (one page per VA,
readers already follow it), and fall back to it when resolution
fails. This also spares the m169 stale-walk hold for destinations
whose page is already known (no pin needed).
Effect on the failing class: the H2D/fill write landing becomes the page
the engine fill, the copy-out and the D2H all use; where a stale and a
fresh resolution disagree, the readers follow the landing and the other
translation is logged as DIVERGE (read/write) or rewrite - bounded, once.
streams2's c1 (engine read stale) is healed by the new SYS records;
streams2's c2 (readback of an engine output after later launches) is
healed by the hash table, which no longer evicts the first launch's
records.
Zero-write lottery diagnostics (secondary, booter; no behavior change on
the green path):
- content-hash settle rule in gsp_phys_gr_output_retire_wait: the
"settled" verdict now requires the per-page checksum FOLD to be stable
for ~300 ms, not just the changed-page COUNT (a continuing store ramp
with a constant count used to read as settled - the r209 "17/32
settled"/"2048/4552 settled" partial copy-outs);
- zero-write accept now logs both qmdwin banks' sentinel words
(ours intact + other bank overwritten = an older deposit was consumed,
i.e. mis-targeted deposit; ours overwritten = this deposit was fetched)
and re-scans the tracked pages once at +1 s: LATE-STORES (copy-out will
ship them) vs NO-STORES (dispatch lost or read-only kernel). The +1 s is
bounded to the rare zero-write accept path only.
The sentinel constant itself is left intact (it is load-bearing for the
m179 post-mortem classification); the dual-bank read answers the same
mis-targeting question without changing the engine-visible contract.
Untouched on purpose: mm2 publish arm/gate, GR fresh-put rewind, the
release machinery and every semaphore path (no regression surface there).
make kernel -j green (new srcversion); main tree and the running VM were
not touched; nothing pushed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
zero-write test + loud classified accept
The r200-r202 memcpy_roundtrip/multikernel all-wrong class: a forwarded GR
launch retires with zero tracked engine writes (NOWRITE, qmd_rel satisfied),
the m153 outcome-retry loop either never ran (it shared the timeout ladder's
attempt counter, which the fast re-arm / fresh entries had already consumed)
or exhausted, and the accept path silently restored the release words,
handing the client an all-wrong output with a fabricated completion.
Changes (kernel/gsp_phys_booter.c, gsp_phys_gpu.c, gsp_phys_gpu.h):
1. gsp_phys_gr_nowrite_retries (default 3, clamped [0,4]): the no-write
re-execution loop now has a DEDICATED budget, independent of the timeout
ladder's attempts. Each retry keeps the existing backoff mechanics (zero
release words, fresh qmdwin/win0 bank, refill bounce, rebuild PB,
re-publish + resched, fresh-ack poll, fetch witness, retire wait). Worst
case stays within the 8-entry GPFIFO ring (initial + fast re-arm + 2 fresh
entries + 3 retries = 7 publishes).
2. Specific zero-write test: gsp_phys_gr_output_retire_wait() now reports
n_tracked (the number of data entries that resolve to a bounce page); the
no-write retry/accept path only triggers when n_tracked > 0, so a launch
whose data entries resolve nothing can no longer false-positive as a lost
launch.
3. The accept (last resort after the budget) is now LOUD (pr_warn) and
classified by the QMD fetch sentinel: NOWRITE-CONFIRMED = deposit never
reached the engine (sweep/hold-faked-ack class); NOWRITE-AMBIGUOUS =
engine consumed the deposit, zero tracked writes may be a legitimately
read-only kernel. It also runs when the retry master gate is off, so a
zero-write outcome is never silent.
4. Ordering: explicit wmb() after the retry's fresh qmdwin deposit (the
first-attempt path and the publish tail already carry the barriers;
dma_wmb before GP_PUT advance + wmb before the doorbell in
gsp_phys_rmcall_publish_locked) - deposit->kick order is now explicit on
both paths.
5. Frame bound: the ~370-byte struct gsp_phys_fault_info moved to static
scratch (the function's documented single-inflight contract, same as
methods[]/qmd[]) with a per-submission memset; this clears the
pre-existing >1024-byte frame warning in the submit function instead of
growing it.
Fast re-arm (gsp_phys_gr_retry_fast) and the timeout ladder are untouched.
Build: make kernel -j - green, vgsp.ko; no warnings in touched code.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Kernel: authorship instrumentation + swrel gate for in-flight GR release VAs;
first-poll shortening + fast extended re-arm (params gr_retry_fast=1,
gr_first_poll_ms=2500); per-attempt win0 bank recompute + shadow recopy in the
outcome-retry loop; poll-break/deposit-readback diagnostics; C1-C10 sema
diagnostics (crel-entry, relx, opdef-clear, walk, pub-adv, hold-fallback,
busy-defer, CAND-OVERWRITE, inline recs); busy-defer hold instead of
software-release consume; counter fast path; QMD word 48/49 skip.
Tools: heartbeat thread, scanner-alive probe on reinstall, full periodic
re-assert, #nopid handling, per-client reset fix (debounce), dead-pid drop,
supervisor grace+threshold.
|
| |
|
|
| |
resolution; hold failed device-copy ops) + m164 CLASS-2 GR no-capture entry hold + m165 pool release applies at fresh-pin time + m166 timeout-ladder warm-sem fail-fast; scanner: first-class pool VA resolution + periodic re-assert + demand routing
|
| |
|
|
| |
stability (1024-page outputs were copied out with 2 pages written) + m161 fetch-witness logging
|
| |
|
|
| |
fresh qmdwin bank + rebuilt PB, restore-on-no-write), event-table slot reclaim (free-slot reuse + loud full log), GR capture trust for fresh plausibility-verified ring entries
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
push on zero-write retry (E5)
- Merge bigpath (B-2 64 MiB grown data-bounce region for large operands)
and dual (per-mdev separation G1-G8a) into master.
- E3: the inline-QMD destination is now double-buffered (0x107000/0x108000,
same proven PTE fetch line); launches alternate banks by submission epoch
so a QMD-unit look-ahead read of the previous launch's page can never race
this launch's rewrite.
- E5: on the first zero-write outcome retry the module pushes the proven
stage-4 GR semaphore block; OK = engine alive (retry can recover), timeout
= engine wedged for the boot and further retries are skipped (saves ~20 s),
counted in work_gr sem_wedged.
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Goal 1.0: two mdevs + two QEMU guests running CUDA simultaneously.
Single-mdev behaviour is kept identical: the legacy guest-RAM record
still serves the first/only open mdev, the channel table degenerates to
the historical key, and the session flush fires at the same
last-channel-closed transition.
- G6: guest-RAM backing {file, path, ranges} per mdev (legacy record +
up to 4 vgpu-bound records); debugfs gsp-phys-guest-ram accepts an
mdev UUID as an extra first token and keeps the 4-token form; records
released on mdev cleanup and physical release.
- G7: gsp_phys_gpa_to_hpa_vgpu(vgpu, gpa, *hpa); NULL keeps the
historical first-open resolution. Forwarding operands pass the
request's vgpu; the single-guest boot-ladder pin passes NULL.
- G1: the sema-service thread services every open mdev each pass.
- G2: channel table keyed by (vgpu, h_object); per-guest ranks;
counter/scrub/opdef/pump snapshot+writeback matches per vgpu;
scrubber records per guest.
- G3: per-guest session records drive the flush; the shared learned
tables (staged G4) are only flushed when no guest has live channels.
- G5: hold/abort keyed by (vgpu, handle); dead-owner guards on the
hold resolution paths; single-inflight serialization kept.
- G8a: guest-GPA pin refused while more than one mdev is open.
Build: make -j8 clean success (vgsp.ko). No module load. See
DUAL-REPORT.md for the per-item status and live-validation plan.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replace the 205-page / 68-pages-per-pointer GR operand window with a
boot-planted 16384-page (64 MiB) region at VA 0x0a000000 (PD0 80..111):
one dual PD0 entry per 2 MB window backed by one fresh 4 KB PT page,
every leaf PTE written at stage 3 before any engine fetch (the GMMU has
no PTE invalidate, so no leaf is rewritten at runtime). The 208-page
window, mail slot 206 and marker 207 stay as-is (mail injection and
copy diagnostics unchanged) but carry no operand data when the grown
region is on.
Sema service: reloc base and page cap derive from the shared header
constants; per-pointer span becomes a 4096-page runaway guard; a total
cap hit is fail-closed (launch stalled for the bounded hold, loud
CAP-HIT log, no silent truncation); the launch's own param/bank pages
are skipped as data candidates (budget hygiene). Copy-in, copy-out,
retirement wait and the window diagnostic index the active region
through one helper/chunk selector; checksum and dedup scratch move off
the kernel stack.
gsp_phys_gr_data_region=0 (insmod, boot-fixed) flips the whole path
back to the legacy 205-page window for A/B trialing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
r154: after the second client's idle period the FECS saved/unloaded the
context (SKED rec: cwd0=0x12486 fecs_cur=b0633ee3 -> cwd0=0x2086 fecs_cur=0
and no change ever after) and every later forwarded launch acked
(qmd_rel/ch_rel) while dispatching nothing - including all three outcome
retries with fresh ring entries (0/205 changed, 720 ms no-writes, x4).
Re-issuing only GPFIFO_SCHEDULE+TSG_GPFIFO_SCHEDULE+doorbell did not
revive dispatch. The extended re-arm now replays the mode-35 lane-enable
tail first (GR_SET_CTXSW_PREEMPTION_MODE CILP + SET_TIMESLICE +
DEVICE_SCHED - the proven-accepted stage-4 sequence, HF-3 design), then
the schedules and the doorbell. Poison variants (explicit PREEMPT/tail,
BIND, PROMOTE) remain excluded.
|
| |
|
|
|
|
|
|
|
|
|
| |
Live r153 evidence: the highest-reloc data entry (bi=204) is a
UVM-internal buffer that correct launches never write, so the
'top operand untouched' condition fired on correct executions and
re-executed them up to 3x - a correctness hazard for side-effecting
kernels (atomics/printf) and pure waste otherwise. The honest execution
witness is the retirement wait's changed count; only zero-writes retries
now (safe by construction: a kernel that wrote nothing has nothing to
duplicate). Read-only kernels pay the bounded extra polls.
|
| |
|
|
|
|
|
|
|
| |
The server refuses multi-page client memdescs for these buffers (a 48-page
chunk aborted stage 2 with rpc_result 0x1f + firmware ASSERT 0x5ab0214 at
ALLOC_MEMORY buffer 30); the only accepted shape is the existing
single-page-per-PD0-slot pattern. GUEST_PT_EXT0..47 are 48 separate
single-page chunks; BUF_COUNT 33 -> 81 (handle range still inside the
meta assert bound).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
launch retry
m152: the Tier-2 shadow alias pool consumed one never-reused 128 B fetch
line per mapping per launch (512 lines/boot), so a normal launch mix
exhausted it permanently: every later launch that needed a new mapping
was refused (the deterministic r152 'launch refused: no Tier-1 mapping'
loop). Two fixes: (a) an HPA-keyed alias cache in map_shadow_locked that
returns the existing never-rewritten VA for a page already mapped, so
repeat launches of the same buffers/sem pages consume nothing; a hit
reports line_out = ~0 so contiguity bookkeeping never extends an old
line; (b) a 48-window extension pool at VA 0x04000000 (PD0 indices
32..79, one 48-page PT chunk) raising capacity to 2048 lines/boot.
m153: the intermittent per-boot mis-execution classes (engine wrote
nothing; or wrote into a lower operand window while the output operand
stayed untouched - both complete with qmd_rel/ch_rel satisfied, live
evidence in r152 smiley: passes wrote b into the a window / nothing /
correct-after-10s-retry) are now detected by the m151 retirement wait
and retried: refill the bounce window from the guest inputs, re-publish
the same method block on a fresh ring entry, re-issue schedule, re-poll,
re-wait - bounded by the timeout path's attempt budget. Gate param
gsp_phys_rmcall_gr_nowrite_retry (default 1).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Footprint-by-inference (max-reloc contiguous run, launch-page exclusion)
failed twice: the fill's candidate list ends with the launch's own bank/
param pages, so the inferred output run was 3-4 pages (r150/r151: '0/3
pages changed after 72 ms (no writes)') and the wait exited before any
real output landed. Replace with a whole-window count over every resolved
data page plus cheap adaptive exits:
- all pages changed -> done (one poll in the normal case),
- partial count stable for 3 polls -> stores settled,
- no change for 600 ms -> no output for this launch,
- hard cap 1.5 s.
No inference, no exclusion list, bounded cost.
|
| |
|
|
|
|
|
|
|
|
|
| |
r150's first cut excluded only QMD-word pages; the fill also relocates the
0xffffffff (non-QMD) forwarded pages (pushbuffer/indirect/bank pages) as
high-VA candidates, so out_base landed on the bank-page group, out_pages
came out 3 and the wait exited after 72 ms - losing the late-write capture
it exists for (smiley r150: '0/3 pages changed after 72 ms (no writes)'
followed by mixed 8-changed/0-changed/32-changed copy-outs and one wrong
pass). Now every forwarded page VA is excluded, so the footprint is the
real output operand (32 pages for smiley, 64 for matmul).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
adaptive exits
- Footprint: exclude data entries whose guest VA is a forwarded launch page
(qmd_word pages) before picking out_base/out_pages. The fill relocates the
launch's own param/bank pages as high-VA candidates, which made out_pages
the whole 205-page budget and forced the full 1.5 s wait on every launch
(r147: '64/205 pages changed ... (TIMEOUT)' on green matmul).
- Adaptive exits: no writes at all after 60 ms (identical-rewrite benchmark
launches / no-output footprints) or a partial count settled for 3 polls
(masked kernels) end the wait early; genuinely late writes are still
caught. Log now carries the reason suffix (no suffix = full footprint).
Live evidence from r147 with the first cut: the exact failing order
mail->matmul->smiley ran green->green->green (smiley recovered to 32/205
changed, C window written; earlier boots showed 0-8 changed and STATUS
FAIL). victory_math/vector_add still fail on the >256 KB per-operand
window cap (chunked data path is the next wall, not this class).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two independent hardening changes for the per-boot 'all-wrong' class:
1. Output-footprint retirement wait (gsp_phys_gr_output_retire_wait): qmd_rel
is a schedule/fetch ack, not a data-path retirement - the engine's stores
into the bounce pages can land after it. Copy-out (and the next launch's
copy-in) before those stores land loses them: the guest observes zeros
for a completed launch (matmul csum=0, r146) or stale/mixed content
(smiley b-into-a, r143/r145). The wait polls up to 300x5 ms, success path
only, until every page of the output footprint (contiguous descending
reloc run from the max reloc_va) differs from its copy-in checksum;
logs 'output retirement wait: N/N pages changed after T ms'.
2. Two-bank win0 rotation: pool grows 8 -> 16 slots (VAs 0x1e0000..0x1effff,
below the mail-injection slot 0x1f0000), bank selected by submission epoch
(epoch & 1). A launch's copy into one bank can no longer clobber the
pages the previous launch's engine index/data fetches may still read.
3. Evidence: full win0 param page dump rows + FNV-1a pre/post hashes of the
win0 param page and the guest param page, so a torn/mixed read source is
provable from the log on the next occurrence.
static_asserts: pool < mail slot and SLOTS == 2 x BANK. Build clean.
Live validation pending (next cycle: mail->matmul->smiley->victory_math->
vector_add).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rows 0..31)
m150 grew the GR data bounce window to 208 pages at main-PT slots
0x110..0x1df but left the win0 QMD shadow pool at VA 0x190000 (slots
0x190..0x197) - inside the new window. The win0 plant is unconditional on
every gsp_phys_rmcall_pdb_build() and runs after the one-shot bounce
plant, so it hijacked the PTEs for bounce pages 128..135, which hold the
first 8 pages of the third operand. victory_matmul's C rows 0..31
(8 pages x 256 floats = 8192 elements) were written into the shadow pages
and lost: MATMUL bad=8192, copy-out '56 changed by engine (first bi 136)'.
Move GSP_PHYS_RMCALL_VA_GR_SHADOW0_BASE to 0x1e0000 (slots 0x1e0..0x1e7),
directly above the last bounce/mail/marker page (0x1df000) and below the
0x200000 guest scratch window; both the plant and the launch-time shadow
VA derive from the macro. Add a static_assert so any future bounce growth
cannot silently re-introduce the overlap, and correct the stale slot-range
comments/logs (0x108..0x10f -> the current range).
Evidence: /tmp/opencode/dmesg-r143-mailmatmul.log (main PT[190..197] =
win0 chunk DMAs; 'window[160] reloc=0x1b0000 changed=1', 'window[128]
reloc=0x190000 changed=0'; MATMUL bad=8192). Live validation pending: a
fresh cycle's victory_matmul must print bad=0 STATUS OK.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
victory_matmul (256x256 float: 256 KB per operand) failed with all-wrong
results while 128 KB workloads passed: the relocation bounce window was
112 pages with a 100-page data cap and 33 pages per pointer, so each 64-page
operand was truncated and the engine ran on partial/mismatched windows.
Grow the planted bounce window to 208 pages (main-PT slots 0x110..0x1df),
move the mail page to the last data-free slot (206, marker 207), and raise
the per-pointer cap to 68 pages and the data cap to 205 (three 256 KB
operands = 3x68 = 204 <= 205). Operands larger than 256 KB still hit the cap
and need a chunked data path (documented at the define).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A green forwarded launch completes in ~170-195 ms, but an intermittent
subset times out with qmd_rel stuck at 5 (cold context save/unload, or an
entry the PBDMA never fetched). The existing recovery only replays the
schedule verbs, which cannot help once the entry was fetched or never
latched, so the client fell back to the software release and read
stale/zero results.
On a timeout, after the schedule re-arm, re-publish a FRESH ring entry with
the same PB/QMD content and re-issue schedule+doorbell, up to
GSP_PHYS_RMCALL_GR_ATTEMPTS (3) with a GSP_PHYS_RMCALL_GR_RETRY_MS (2000 ms)
poll. Safe against double execution: the timeout path never copies the
result out to the guest, so a timed-out attempt did not reach the client -
only the eventually-successful attempt is visible. New counter
gsp_phys_gr_retries and a 'retry N/M fresh ring entry' log line.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The host publishes every forwarded launch into the one-page ladder ring
modulo GSP_PHYS_RMCALL_GPFIFO_ENTRIES (8). Commit 3b82047c declared the
mode-35 user-lane channel with 1024 entries for capture fidelity; the
engine's GET/PUT then wrap at 1024 while the host wraps at 8, so at the
host's first wrap (GP_PUT 7 -> 0) the engine desyncs and never fetches
another entry: launches 1..7 green, the 8th and all later ones silent
(qmd_rel=5, ch_rel=0), which is the 'launch lottery'.
Live proof (r138): the failure fires exactly at slot=7 (first wrap) and
every launch after it fails; vecadd_loop (100 launches) dies at the wrap.
M1's two-launch acceptance never wrapped, so the regression was latent.
Set the declared count back to the host ring constant; the wrap-only
fidelity difference against the captured real-client ring is documented at
the define.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registration; scanner streams anonymous high-VA buffers
Second-client wedge fix: a new GSP session re-registers the same channel
handles/ranks, but the per-rank one-shot ring/push-buffer discovery latches
stayed set from the previous session, so the new session's ring was never
re-autopinned (ring_auto=0 pb=0/0, nomap growing) and the guest spun in its
channel-creation wait. svc_chan_rearm_rank() clears the latches on every
channel registration; all post-fix cycles launch the second client.
Scanner: guest_pin_stream.py now streams the client's anonymous high-VA
mappings (cudaMalloc buffers were skipped by the old len(parts)<6 check),
so the host can learn their pages.
Diagnostics: PT-window lines, per-window first words, post-OK qmdwin
w48..59. Remaining: the fill-time svc_ipin_lookup miss for the second
client's a/b/c pins (they exist in the pin log but are not seen at fill
time) - next session.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(second-client fix)
Second-client launches scanned a stale/reused guest param page that still
held the FIRST client's device VAs: zero new candidates resolved, nothing
was relocated, and the engine executed with unrelocated guest pointers
(r97 evidence: client#1 '15 candidate ptrs, 100 pages resolved' and win0
param 160=110400; client#2 '12 candidates, 1 page' with 160=75552ec00000
unrelocated). The driver-authored inline param block is captured from the
pushbuffer at request time, so scanning it makes pointer discovery
independent of guest-page reuse timing. Adds svc_gr_scan_cand()/
svc_gr_scan_inline() and the DIAG win0/guest-param proof line. Client #1
stays green (4 cycles); the remaining second-client stall is the pool
semaphore residue (see next commit).
|
| |
|
|
|
| |
Discriminates engine-side writes of float data (kernel ran with c on this
buffer) from non-kernel writes (pointers/QMD-shaped words).
|
| |
|
|
|
|
|
| |
8 changed pages x 4 passes for smiley vs 33 for victory's vadd - the
indexes name the relocated range (bi >= 0x40 = the c-pointer output
window 0x150000+) and discriminate engine-side partial dispatch from a
guest write-back/read attribution failure.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
save/unload lottery)
r90 evidence: the 3rd session's forwarded launch entered SKED (CWD VALID)
and was then saved/unloaded by the FECS before dispatch; qmd_rel stayed at
5/6, GP_GET froze, and every later launch on the boot was never fetched
either. The service owns no repair path, so the channel stayed dead until
reboot.
On the first poll timeout, re-issue the two GPFIFO_SCHEDULE controls the
stage-4 sequence used (channel-level 0xa06f0103 + TSG-level 0xa06c0101,
both proven accepted) and re-ring the work-submit doorbell, then poll one
more full window before the postmortem. No BIND, no PROMOTE (m76-m81
proved the explicit PREEMPT/tail variants stop the QMD enqueuing).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ipin-first data resolution
Two fixes and one diagnostic for the all-zero-results class (r89: smiley's
4 vadd launches reported GUEST GR LAUNCH OK yet every output word read 0):
- gsp_phys_booter: copy the guest data pages into the bounce window BEFORE
gsp_phys_rmcall_publish_locked() rings the doorbell. The fill used to run
after the kick; the engine front end starts within ~7 ms, so the kernel
could read the window before it was filled.
- gsp_phys_booter: per-page checksums at copy-in, compared at copy-out;
the copy-out log now reports 'N changed by engine'. An all-unchanged
window after a reported completion is the front-end-ran/grid-never-
dispatched signature (completion semaphores are front-end authored).
The first 3 data (va,hpa,reloc) triples are logged per launch.
- gsp_sema_service: svc_gr_data_fill resolves data pages through
svc_ipin_lookup() first (guest-truth pins), falling back to
svc_resolve_va, mirroring the pool resolver order; a wrong page here
silently runs the kernel on zeros. First 16 resolved (va,pa,ap) triples
are logged for post-mortem comparison against the guest's own mapping.
|
| |
|
|
| |
corrupted vadd blockDim -> one-CTA result); pre-rewrite/final header dumps
|
| |
|
|
| |
replay (inline records restored the original guest pointers into the engine-mapped page; root cause of the param-read wall)
|
| |
|
|
| |
0x190..0x197) + win0 content and PT-slot diagnostics (the slot-0x108 line reads stale to the SM data path)
|
| |
|
|
| |
reach for probes)
|
| |
|
|
| |
patch land?)
|
| |
|
|
| |
c[0x0] reads the guest page, not the shadow)
|
| |
|
|
| |
success path)
|
| |
|
|
| |
collided with the bounce window)
|
| |
|
|
| |
logging + param 0x150 dump + main-PT slot dump
|
| |
|
|
|
|
|
| |
m116's param dump exposed the m103 4-byte-stride scan corrupting small
adjacent fields (0x40/1 rewritten into relocated-looking values). Use
8-byte stride, require 256-byte-aligned values (all device pointers), and
skip records without a reloc VA.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
copy-in/out)
Decisive evidence: the SM walker resolves only the commit-time tree (every
late write/install fails: m94, m99-m113; UNSET refused 0x1f). Plant a
112-page bounce window into the main small-PT at stage 3 (slots
0x110..0x17f - the proven win0 pattern), copy the launch's resolved data
pages in, pointers are already rewritten to those VAs, and copy results
back after the release. The mail message lives at bounce page 100
(c[0x4][0x0]=0x174000). Data cap 100 pages (440 KB), matching the shrunk
victory_mail (3 x 128 KB).
|
| |
|
|
|
|
|
|
|
|
| |
(re-commit probe)
All late engine-tree writes are invisible to the SM walker (only the
commit-time snapshot resolves); the mail injection at a main-window fresh
slot also fails (m112). Probe whether UNSET+SET_PAGE_DIRECTORY on the
ladder VAS after planting forces the server walker to re-snapshot the tree;
gated by gr_recommit.
|
| |
|
|
|
| |
m111's mret=-34: mva was BASE+0x1f0000 = 0x200000, outside the 2 MB main
window. Use the absolute VA 0x1f0000.
|
| |
|
|
|
|
| |
m110 ran without gr_data_prewin, and the mail prep was nested inside that
block - mail.pages stayed 0 and the injection failed with EINVAL. Move the
mail page preparation out; it is 4 KB and harmless.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
the demo
All UPDATE_PDE_2 installs (post- or pre-commit) are invisible to the SM
walker; the only window it honours is the commit-time main small-PT page
(VA 0x100000+, the proven win0/const window). Relocate each candidate data
pointer's first 64 pages into slots 0x110..0x1f0 of that window (fresh PTE
lines, direct leaf writes), rewrite bank pointers per page record, and put
the mail page at the reserved slot 0x1f0 (c[0x4][0x0]=0x1f0000). Shrink
victory_mail's buffers (n=1<<15, 128 KB each) to fit the 2 MB window.
|
| |
|
|
|
|
| |
m108's mret=-34 (ERANGE): the shift bound tighter than expected in the
written expression, making mva = (0x100000000 + 31) << 21, outside the
pool. Parenthesize.
|
| |
|
|
|
|
|
|
| |
m107's inject entry showed mail.pages=0 after the launch-time
dma_alloc_coherent attempt: the allocation fails silently there. Allocate
and fill the message page in stage 3 alongside the data PT pool (proven to
allocate fine), and at launch only map it into pool window 31 and patch
c[0x4][0x0]; log the map result either way.
|
| |
|
|
|
|
|
|
|
|
| |
unconditional inject
m106's bank-4 injection silently did not fire although the insmod args
carried mail_patch=1 and the stage-3 data pool installed. Print the getter
values at launch and log the injection entry (w0/w1/mail.pages/pt31.pages),
then inject unconditionally when bank-4 is processed so the pool-backed
mail page (window 31) is exercised either way.
|
