| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
with the walk translation; r173: the unbounded guard deadlocked mail's first H2D - device pins only appear via the launch/fill paths, which the held copy blocked)
|
| |
|
|
| |
by walk/RAM-scan are held until a pin confirms; r172 streams2_diag stream-1 H2D wrote through a stale walk entry while the launch read the true page) + session-scope the op-trace budget + bigcopy witness covers device-dst copies >=64KB
|
| |
|
|
| |
line r171: cached=0x1247b5000 new=0x1321fa000); a cross-session cache served the previous client's dead page and silently absorbed releases. scanner: pool demand resolves against every known tracker sem (seen + resolved chains)
|
| |
|
|
| |
every use read-back proven; pins stay first priority with conflict logging) - r170 clients 4-6 starved minutes on the pool/waiter page after the flush wiped the only translation
|
| |
|
|
| |
fired with smiley's channels live 7s after registration, dropped its queued launch release, client spun in uvm_channel_is_value_completed) + m164 hardening: GR no-capture hold applies only to fresh entries, never replays (old-session pushes were held and re-counted)
|
| |
|
|
| |
resolution; hold failed device-copy ops) + m164 CLASS-2 GR no-capture entry hold + m165 pool release applies at fresh-pin time + m166 timeout-ladder warm-sem fail-fast; scanner: first-class pool VA resolution + periodic re-assert + demand routing
|
| |
|
|
| |
stability (1024-page outputs were copied out with 2 pages written) + m161 fetch-witness logging
|
| |
|
|
| |
fresh qmdwin bank + rebuilt PB, restore-on-no-write), event-table slot reclaim (free-slot reuse + loud full log), GR capture trust for fresh plausibility-verified ring entries
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
hold-and-reprocess)
r159: the m156/m158 hold kept the entries but the pump never re-processed
them before the client tore down (held entry re-decode is a replay; the
one-shot window and teardown timing conspired), and the channel died with the
launches unforwarded.
New design: on a busy refusal with the bridge disabled the capture is NOT
consumed and the entry loop waits (bounded 4 s) for the slot to free -
resolving the active release hold through its own channel on this same pump
thread - then retries svc_fwd_request inline. Multi-launch clients now
serialize correctly: kernel 2/3 of streams2 wait for kernel 1's forward and
then run. On timeout the historical consume+soft-release path stands.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
continuation)
r158: the m156 hold fired (entries held, no fabricated release) but the held
entry's next decode is a replay, which suppresses the launch capture, and the
one-shot hold flag consumed the entry on the following pass - the deferred
launches were still lost (channel unregistered before they ran).
Two changes:
- the hold now stays armed for its whole window (re-hold while
time_before(gr_hold_until)) instead of one-shot per slot;
- the hold arms ch->gr_rehold_idx, and the entry decode forces exactly one
fresh (non-replay) pass for that slot with ops still suppressed and the
capture re-enabled, so the deferred launch is captured and forwarded for
real once the engine's in-flight launch completes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
them (R1)
r157 streams2: kernels 2 and 3 were captured while kernel 1's forward was
INFLIGHT, refused by svc_gr_request, and their releases were fabricated in
software 36 us later (the F2.3 fail-open path) - the client read all zeros.
Now a busy refusal (hold active / INFLIGHT / paused, bridge disabled) sets
svc_gr_busy_defer and the entry loop holds the entry under the m150 one-shot
window instead of consuming it, so the pump re-captures and forwards it for
real once the in-flight launch completes. With the bridge enabled the old
behaviour is kept (the bridge job owns the release).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
push on zero-write retry (E5)
- Merge bigpath (B-2 64 MiB grown data-bounce region for large operands)
and dual (per-mdev separation G1-G8a) into master.
- E3: the inline-QMD destination is now double-buffered (0x107000/0x108000,
same proven PTE fetch line); launches alternate banks by submission epoch
so a QMD-unit look-ahead read of the previous launch's page can never race
this launch's rewrite.
- E5: on the first zero-write outcome retry the module pushes the proven
stage-4 GR semaphore block; OK = engine alive (retry can recover), timeout
= engine wedged for the boot and further retries are skipped (saves ~20 s),
counted in work_gr sem_wedged.
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The host CUDA bridge is the GR fallback path when a forward request is
refused under the single-inflight slot (the expected steady state with
two guests). Its FIFO request and pending records dropped the owning
vgpu, so guest B's fallback job could be marshalled (VAs resolved, guest
memory read/written) through guest A's address space.
- cubb_req/cubb_pend gain the owning mdev; the launch stores it and it
is part of the dedup identity.
- Only the owning guest's service pass marshals its job or drives its
staged completion; a head/record whose mdev is gone is dropped instead
of ever being dereferenced.
- One guest: head_owner is always the pumped vgpu, so behaviour is
unchanged.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Goal 1.0: two mdevs + two QEMU guests running CUDA simultaneously.
Single-mdev behaviour is kept identical: the legacy guest-RAM record
still serves the first/only open mdev, the channel table degenerates to
the historical key, and the session flush fires at the same
last-channel-closed transition.
- G6: guest-RAM backing {file, path, ranges} per mdev (legacy record +
up to 4 vgpu-bound records); debugfs gsp-phys-guest-ram accepts an
mdev UUID as an extra first token and keeps the 4-token form; records
released on mdev cleanup and physical release.
- G7: gsp_phys_gpa_to_hpa_vgpu(vgpu, gpa, *hpa); NULL keeps the
historical first-open resolution. Forwarding operands pass the
request's vgpu; the single-guest boot-ladder pin passes NULL.
- G1: the sema-service thread services every open mdev each pass.
- G2: channel table keyed by (vgpu, h_object); per-guest ranks;
counter/scrub/opdef/pump snapshot+writeback matches per vgpu;
scrubber records per guest.
- G3: per-guest session records drive the flush; the shared learned
tables (staged G4) are only flushed when no guest has live channels.
- G5: hold/abort keyed by (vgpu, handle); dead-owner guards on the
hold resolution paths; single-inflight serialization kept.
- G8a: guest-GPA pin refused while more than one mdev is open.
Build: make -j8 clean success (vgsp.ko). No module load. See
DUAL-REPORT.md for the per-item status and live-validation plan.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replace the 205-page / 68-pages-per-pointer GR operand window with a
boot-planted 16384-page (64 MiB) region at VA 0x0a000000 (PD0 80..111):
one dual PD0 entry per 2 MB window backed by one fresh 4 KB PT page,
every leaf PTE written at stage 3 before any engine fetch (the GMMU has
no PTE invalidate, so no leaf is rewritten at runtime). The 208-page
window, mail slot 206 and marker 207 stay as-is (mail injection and
copy diagnostics unchanged) but carry no operand data when the grown
region is on.
Sema service: reloc base and page cap derive from the shared header
constants; per-pointer span becomes a 4096-page runaway guard; a total
cap hit is fail-closed (launch stalled for the bounded hold, loud
CAP-HIT log, no silent truncation); the launch's own param/bank pages
are skipped as data candidates (budget hygiene). Copy-in, copy-out,
retirement wait and the window diagnostic index the active region
through one helper/chunk selector; checksum and dedup scratch move off
the kernel stack.
gsp_phys_gr_data_region=0 (insmod, boot-fixed) flips the whole path
back to the legacy 205-page window for A/B trialing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
r154: after the second client's idle period the FECS saved/unloaded the
context (SKED rec: cwd0=0x12486 fecs_cur=b0633ee3 -> cwd0=0x2086 fecs_cur=0
and no change ever after) and every later forwarded launch acked
(qmd_rel/ch_rel) while dispatching nothing - including all three outcome
retries with fresh ring entries (0/205 changed, 720 ms no-writes, x4).
Re-issuing only GPFIFO_SCHEDULE+TSG_GPFIFO_SCHEDULE+doorbell did not
revive dispatch. The extended re-arm now replays the mode-35 lane-enable
tail first (GR_SET_CTXSW_PREEMPTION_MODE CILP + SET_TIMESLICE +
DEVICE_SCHED - the proven-accepted stage-4 sequence, HF-3 design), then
the schedules and the doorbell. Poison variants (explicit PREEMPT/tail,
BIND, PROMOTE) remain excluded.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r153 root cause (3rd-client userspace spin for the rest of the boot): the
guest scanner's merged CE-window run (va=0x121000000 len=0x40000; the
merged length varies per session) covered a CE channel's GPFIFO ring VA
0x121010000. svc_ipin_store took precedence in the ring read path, so the
ring read the guest page at the wrong PA (0x102e02000): an all-zero entry
was silently consumed, the pool release was never authored
(pushes frozen at 1643, crel frozen), and victory_matmul spun in
userspace for 50+ min. The ring VA translation must come from the
channel's own discovery path; an injected pin inside any live channel's
ring range is always wrong - drop it (counted + logged, first 8 per
session). Validated on the next cycle: 'injected pin ... DROPPED' lines
plus 'entry[1] raw!=0', advancing pushes, MATMUL bad=0 STATUS OK.
|
| |
|
|
|
|
|
|
|
|
|
| |
Live r153 evidence: the highest-reloc data entry (bi=204) is a
UVM-internal buffer that correct launches never write, so the
'top operand untouched' condition fired on correct executions and
re-executed them up to 3x - a correctness hazard for side-effecting
kernels (atomics/printf) and pure waste otherwise. The honest execution
witness is the retirement wait's changed count; only zero-writes retries
now (safe by construction: a kernel that wrote nothing has nothing to
duplicate). Read-only kernels pay the bounded extra polls.
|
| |
|
|
|
|
|
|
|
| |
The server refuses multi-page client memdescs for these buffers (a 48-page
chunk aborted stage 2 with rpc_result 0x1f + firmware ASSERT 0x5ab0214 at
ALLOC_MEMORY buffer 30); the only accepted shape is the existing
single-page-per-PD0-slot pattern. GUEST_PT_EXT0..47 are 48 separate
single-page chunks; BUF_COUNT 33 -> 81 (handle range still inside the
meta assert bound).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
launch retry
m152: the Tier-2 shadow alias pool consumed one never-reused 128 B fetch
line per mapping per launch (512 lines/boot), so a normal launch mix
exhausted it permanently: every later launch that needed a new mapping
was refused (the deterministic r152 'launch refused: no Tier-1 mapping'
loop). Two fixes: (a) an HPA-keyed alias cache in map_shadow_locked that
returns the existing never-rewritten VA for a page already mapped, so
repeat launches of the same buffers/sem pages consume nothing; a hit
reports line_out = ~0 so contiguity bookkeeping never extends an old
line; (b) a 48-window extension pool at VA 0x04000000 (PD0 indices
32..79, one 48-page PT chunk) raising capacity to 2048 lines/boot.
m153: the intermittent per-boot mis-execution classes (engine wrote
nothing; or wrote into a lower operand window while the output operand
stayed untouched - both complete with qmd_rel/ch_rel satisfied, live
evidence in r152 smiley: passes wrote b into the a window / nothing /
correct-after-10s-retry) are now detected by the m151 retirement wait
and retried: refill the bounce window from the guest inputs, re-publish
the same method block on a fresh ring entry, re-issue schedule, re-poll,
re-wait - bounded by the timeout path's attempt budget. Gate param
gsp_phys_rmcall_gr_nowrite_retry (default 1).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Footprint-by-inference (max-reloc contiguous run, launch-page exclusion)
failed twice: the fill's candidate list ends with the launch's own bank/
param pages, so the inferred output run was 3-4 pages (r150/r151: '0/3
pages changed after 72 ms (no writes)') and the wait exited before any
real output landed. Replace with a whole-window count over every resolved
data page plus cheap adaptive exits:
- all pages changed -> done (one poll in the normal case),
- partial count stable for 3 polls -> stores settled,
- no change for 600 ms -> no output for this launch,
- hard cap 1.5 s.
No inference, no exclusion list, bounded cost.
|
| |
|
|
|
|
|
|
|
|
|
| |
r150's first cut excluded only QMD-word pages; the fill also relocates the
0xffffffff (non-QMD) forwarded pages (pushbuffer/indirect/bank pages) as
high-VA candidates, so out_base landed on the bank-page group, out_pages
came out 3 and the wait exited after 72 ms - losing the late-write capture
it exists for (smiley r150: '0/3 pages changed after 72 ms (no writes)'
followed by mixed 8-changed/0-changed/32-changed copy-outs and one wrong
pass). Now every forwarded page VA is excluded, so the footprint is the
real output operand (32 pages for smiley, 64 for matmul).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
adaptive exits
- Footprint: exclude data entries whose guest VA is a forwarded launch page
(qmd_word pages) before picking out_base/out_pages. The fill relocates the
launch's own param/bank pages as high-VA candidates, which made out_pages
the whole 205-page budget and forced the full 1.5 s wait on every launch
(r147: '64/205 pages changed ... (TIMEOUT)' on green matmul).
- Adaptive exits: no writes at all after 60 ms (identical-rewrite benchmark
launches / no-output footprints) or a partial count settled for 3 polls
(masked kernels) end the wait early; genuinely late writes are still
caught. Log now carries the reason suffix (no suffix = full footprint).
Live evidence from r147 with the first cut: the exact failing order
mail->matmul->smiley ran green->green->green (smiley recovered to 32/205
changed, C window written; earlier boots showed 0-8 changed and STATUS
FAIL). victory_math/vector_add still fail on the >256 KB per-operand
window cap (chunked data path is the next wall, not this class).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two independent hardening changes for the per-boot 'all-wrong' class:
1. Output-footprint retirement wait (gsp_phys_gr_output_retire_wait): qmd_rel
is a schedule/fetch ack, not a data-path retirement - the engine's stores
into the bounce pages can land after it. Copy-out (and the next launch's
copy-in) before those stores land loses them: the guest observes zeros
for a completed launch (matmul csum=0, r146) or stale/mixed content
(smiley b-into-a, r143/r145). The wait polls up to 300x5 ms, success path
only, until every page of the output footprint (contiguous descending
reloc run from the max reloc_va) differs from its copy-in checksum;
logs 'output retirement wait: N/N pages changed after T ms'.
2. Two-bank win0 rotation: pool grows 8 -> 16 slots (VAs 0x1e0000..0x1effff,
below the mail-injection slot 0x1f0000), bank selected by submission epoch
(epoch & 1). A launch's copy into one bank can no longer clobber the
pages the previous launch's engine index/data fetches may still read.
3. Evidence: full win0 param page dump rows + FNV-1a pre/post hashes of the
win0 param page and the guest param page, so a torn/mixed read source is
provable from the log on the next occurrence.
static_asserts: pool < mail slot and SLOTS == 2 x BANK. Build clean.
Live validation pending (next cycle: mail->matmul->smiley->victory_math->
vector_add).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rows 0..31)
m150 grew the GR data bounce window to 208 pages at main-PT slots
0x110..0x1df but left the win0 QMD shadow pool at VA 0x190000 (slots
0x190..0x197) - inside the new window. The win0 plant is unconditional on
every gsp_phys_rmcall_pdb_build() and runs after the one-shot bounce
plant, so it hijacked the PTEs for bounce pages 128..135, which hold the
first 8 pages of the third operand. victory_matmul's C rows 0..31
(8 pages x 256 floats = 8192 elements) were written into the shadow pages
and lost: MATMUL bad=8192, copy-out '56 changed by engine (first bi 136)'.
Move GSP_PHYS_RMCALL_VA_GR_SHADOW0_BASE to 0x1e0000 (slots 0x1e0..0x1e7),
directly above the last bounce/mail/marker page (0x1df000) and below the
0x200000 guest scratch window; both the plant and the launch-time shadow
VA derive from the macro. Add a static_assert so any future bounce growth
cannot silently re-introduce the overlap, and correct the stale slot-range
comments/logs (0x108..0x10f -> the current range).
Evidence: /tmp/opencode/dmesg-r143-mailmatmul.log (main PT[190..197] =
win0 chunk DMAs; 'window[160] reloc=0x1b0000 changed=1', 'window[128]
reloc=0x190000 changed=0'; MATMUL bad=8192). Live validation pending: a
fresh cycle's victory_matmul must print bad=0 STATUS OK.
|
| |
|
|
|
|
|
|
|
| |
r141 showed the 68-pages-per-pointer change starved the operands: the
launch's own QMD VA (0x204608000) and other low-VA candidates each consumed
68 window pages, so a 256 KB operand resolved 0 pages before the 205-page
cap. Give the full span only to high, cudaMalloc-shaped VAs, skip the QMD
VA entirely, and give every other candidate one page. Three 256 KB operands
(3x68=204) now fit.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
victory_matmul (256x256 float: 256 KB per operand) failed with all-wrong
results while 128 KB workloads passed: the relocation bounce window was
112 pages with a 100-page data cap and 33 pages per pointer, so each 64-page
operand was truncated and the engine ran on partial/mismatched windows.
Grow the planted bounce window to 208 pages (main-PT slots 0x110..0x1df),
move the mail page to the last data-free slot (206, marker 207), and raise
the per-pointer cap to 68 pages and the data cap to 205 (three 256 KB
operands = 3x68 = 204 <= 205). Operands larger than 256 KB still hit the cap
and need a chunked data path (documented at the define).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A green forwarded launch completes in ~170-195 ms, but an intermittent
subset times out with qmd_rel stuck at 5 (cold context save/unload, or an
entry the PBDMA never fetched). The existing recovery only replays the
schedule verbs, which cannot help once the entry was fetched or never
latched, so the client fell back to the software release and read
stale/zero results.
On a timeout, after the schedule re-arm, re-publish a FRESH ring entry with
the same PB/QMD content and re-issue schedule+doorbell, up to
GSP_PHYS_RMCALL_GR_ATTEMPTS (3) with a GSP_PHYS_RMCALL_GR_RETRY_MS (2000 ms)
poll. Safe against double execution: the timeout path never copies the
result out to the guest, so a timed-out attempt did not reach the client -
only the eventually-successful attempt is visible. New counter
gsp_phys_gr_retries and a 'retry N/M fresh ring entry' log line.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The host publishes every forwarded launch into the one-page ladder ring
modulo GSP_PHYS_RMCALL_GPFIFO_ENTRIES (8). Commit 3b82047c declared the
mode-35 user-lane channel with 1024 entries for capture fidelity; the
engine's GET/PUT then wrap at 1024 while the host wraps at 8, so at the
host's first wrap (GP_PUT 7 -> 0) the engine desyncs and never fetches
another entry: launches 1..7 green, the 8th and all later ones silent
(qmd_rel=5, ch_rel=0), which is the 'launch lottery'.
Live proof (r138): the failure fires exactly at slot=7 (first wrap) and
every launch after it fails; vecadd_loop (100 launches) dies at the wrap.
M1's two-launch acceptance never wrapped, so the regression was latent.
Set the declared count back to the host ring constant; the wrap-only
fidelity difference against the captured real-client ring is documented at
the define.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
freeze)
The guest PMA scrubber (in-guest CeUtils CE channel, 4096 GPFIFO entries)
waits on its channel buffer's finish payload (offset 0x6c004) after the
third CUDA session's pages must be scrubbed; nothing in the emulation
authored it, so cudaMalloc froze in scrubCheckAndWaitForSize.
- channel_add: detect the 4096-entry scrubber channel and stash its USERD
PA/ring geometry (the channel itself is left untouched: no rank change,
no pump change - the colliding numeric-handle FREE may still unregister
it, the geometry survives).
- sema_scrubbuf_add debug param: register the finish/host-semaphore word
PAs the guest scanner resolves (vmallocinfo + kcore walk), because the
RM kernel VA space collides numerically with the client CE window in the
global VA map and cannot be resolved through it.
- svc_scrub_pass (service thread, per pass): from the channel's own USERD
GPPut, track submitted scrub blocks across wraps, then author USERD
GP_GET = put, host semaphore = put and finish payload = submitted.
Forward-only, idempotent, zero writes while the scrubber idles.
Live-validated mechanism (r137 VM): authoring the finish word at the kcore-
resolved PA let the frozen victory_matmul leave scrubCheckAndWaitForSize
and proceed to its launch.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
displaced pin pair
The r129 cross-page evidence: a 216-byte CE push whose SET_SEMAPHORE_A +
LAUNCH_DMA tail fell in the next 4 KB page, whose injected pin resolved to
an all-0xFF page while the first page decoded perfectly. A continuation
chunk read from an unvalidated source (injected pin, its displaced pair, or
the linear window guess) is now rejected when the whole chunk is 0xFF, the
displaced ipin-alt pair is tried as a new source, and a bounded line names
the rejected (va, pa, src) at the fix site.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
release
The entries the pump holds for the 8 s CE grace are exactly the ones whose
decode found no release. Under SEMA_DBG this logs the decode state
(have_sem, rel va/payload, ops) and the first 64 push bytes for every
content-bearing CE push that decoded no release (2048-line budget,
all-zero reads are filtered so they cannot wash it out). Used in r129 to
show that a push whose tail crosses the CE page boundary reads 0xFF
garbage for its second page - the sem+LAUNCH_DMA live exactly in that
tail, so only boundary-crossing pushes were undecodable.
|
| |
|
|
|
|
|
|
|
| |
The pump works on a snapshot copy of the channel table; without the
end-of-pass write-back the hold window was re-armed every pass and a
launch whose argument pointers never translate was never consumed (r122:
smiley's launch held forever). Persist gr_hold_until/gr_hold_idx so the
hold is exactly one bounded window per entry, then the entry is consumed
as before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
derive the CE vmap pairing per allocation
Two second-client zeros/stall countermeasures:
- svc_gr_data_fill now reports when a launch carries high (device-space)
argument pointers and none of them resolves through any trusted source.
svc_gr_request refuses to forward such a launch and leaves the capture
unconsumed; the pump holds the ring entry for one bounded window per
entry (gsp_sema_gr_grace_ms) and retries it, so a translation that
appears a pass later (scanner pin, published directory, learned pair)
is used instead of executing the kernel against unresolved pointers.
On window expiry the entry is consumed exactly as before (bounded
zeros, never a hang). Bounded per-session fill diagnostics name the
source that serves each candidate (ipin/pin/lat/map/vas/disc/win) and
dump the published vaspace walk results for misses.
- The CE push-buffer kcore chain resolver derives its vmap page offset
per allocation instead of using a fixed shift: the rm_mem size is read
from its real offset (570.144 layout) and paired with the vmallocinfo
area page count, so a leading header page is only skipped when it
actually exists. With an unreadable size the two candidate offsets are
content-probed with the host's method-stream plausibility test and an
ambiguous result keeps the status-quo pairing. This removes the
off-by-one that fed wrong push pages to the pump (content-rescue
crawl).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rescues repair injections
The CE push-buffer uvm_rm_mem is vmalloc'd as a 4097-page area: one
leading page plus the 4096-page (16 MB) GPU data window, so GPU data
page k lives at vmap page k+1. The kcore chain resolver paired page k
with vmap page k, pinning the adjacent push for every page; the pump
then read the wrong push, decode failed, and the host fell into the
content rescue which crawled one entry per ~2 s (r115: rescue_run=4202,
entry_va=0x12001b18c repeating). Verified against five independent
content-validated rescue anchors (k=20..28, all delta +1), and the vmap
area boundary confirms the extra page (nv_vm_map_pages 4097 pages).
Five fixes:
- resolve_mem_pins() takes page_off; the CE chain and uvbp push-buffer
paths pass mem_page_off() (1 when the vmap is exactly 16 MB + 1 page;
0 for direct-mapped mems like the semaphore pool).
- a content-validated rescue page now also replaces the injected-pin
entry (svc_rescue_try / svc_rescue_run_try), so a stale or wrong
injection can no longer shadow the validated page and re-run the
rescue on every pass.
Evidence: r115 rescue anchors va 0x120014000->pa 0x12a2e2000 .. va
0x12001c000->pa 0x12a2de000 all at vmap offset k+1; the candidate pages
decode as the adjacent push (wrong semaphore payload) before the fix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
stall)
The second CUDA client's GPFIFO entries in the CE push-buffer window
(0x120000000) had no injected pin: that window is never mapped into the
client's pagemap, so the scanner's window scan cannot see it, and its
only supplier - the one-shot uvbp kprobe event - is raced by two trace
clears (the scanner's pid-switch clear and semfix's trace-tail clear).
When the event loses the race the entry reads fall back to the ~250 ms
content rescue, which is the second-client stall and the same
resolution-miss class as the ISSUE-0001 lottery.
The guest scanner now walks channel->pool->manager->pushbuffer->rm_mem
in kernel memory with kcore and pins the push-buffer pages directly (no
trace event to lose), and the trace reader's dedup marks reset on a
client switch so a reused kptr cannot suppress the new client's #pb.
Boot evidence (r112): rescue reject ipin=0 for va_page=0x120023000,
rescue scans 175 ms - 1 s each, want incrementing to 39. Smoke test of
this fix on the stalled VM: 478 pin runs (~7955 pages) streamed within
seconds, first va=0x120000000 -> pa=0x108090000. A bounded kernel log
now prints the first CE push-buffer pins of each session.
|
| |
|
|
|
|
|
|
|
|
|
| |
(second-client zeros)
The watcher's pins for the second client's a/b/c buffers can land between
the service loop's svc_handle_map_add() and the launch capture; the fill
then misses them via svc_ipin_lookup, resolves 0 pages, relocates nothing,
and the win0 param page keeps the driver's guest VAs (rk1: candidate[0..2]
-> 0 new pages; DIAG win0[0] 160=753e4ec00000). Drain the queue right
before svc_gr_req_fill() (same service thread, no race).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registration; scanner streams anonymous high-VA buffers
Second-client wedge fix: a new GSP session re-registers the same channel
handles/ranks, but the per-rank one-shot ring/push-buffer discovery latches
stayed set from the previous session, so the new session's ring was never
re-autopinned (ring_auto=0 pb=0/0, nomap growing) and the guest spun in its
channel-creation wait. svc_chan_rearm_rank() clears the latches on every
channel registration; all post-fix cycles launch the second client.
Scanner: guest_pin_stream.py now streams the client's anonymous high-VA
mappings (cudaMalloc buffers were skipped by the old len(parts)<6 check),
so the host can learn their pages.
Diagnostics: PT-window lines, per-window first words, post-OK qmdwin
w48..59. Remaining: the fill-time svc_ipin_lookup miss for the second
client's a/b/c pins (they exist in the pin log but are not seen at fill
time) - next session.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
(second-client fix)
r99/r100: the fresh session's reused pool page reads the previous session's
residue (cur=2) before the first genuine release (pay=1) is translated; the
translated read set ch->pool_rel_seen, so every later residue release
(payload <= cur) was silently dropped by the final else and the client spun
at DEVICE forever. A read that applied or deferred nothing now leaves the
flag clear so the r63/r64 residue re-seed still fires for the first genuine
release of the session.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(second-client fix)
Second-client launches scanned a stale/reused guest param page that still
held the FIRST client's device VAs: zero new candidates resolved, nothing
was relocated, and the engine executed with unrelocated guest pointers
(r97 evidence: client#1 '15 candidate ptrs, 100 pages resolved' and win0
param 160=110400; client#2 '12 candidates, 1 page' with 160=75552ec00000
unrelocated). The driver-authored inline param block is captured from the
pushbuffer at request time, so scanning it makes pointer discovery
independent of guest-page reuse timing. Adds svc_gr_scan_cand()/
svc_gr_scan_inline() and the DIAG win0/guest-param proof line. Client #1
stays green (4 cycles); the remaining second-client stall is the pool
semaphore residue (see next commit).
|
| |
|
|
|
| |
Discriminates engine-side writes of float data (kernel ran with c on this
buffer) from non-kernel writes (pointers/QMD-shaped words).
|
| |
|
|
|
|
|
| |
8 changed pages x 4 passes for smiley vs 33 for victory's vadd - the
indexes name the relocated range (bi >= 0x40 = the c-pointer output
window 0x150000+) and discriminate engine-side partial dispatch from a
guest write-back/read attribution failure.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
save/unload lottery)
r90 evidence: the 3rd session's forwarded launch entered SKED (CWD VALID)
and was then saved/unloaded by the FECS before dispatch; qmd_rel stayed at
5/6, GP_GET froze, and every later launch on the boot was never fetched
either. The service owns no repair path, so the channel stayed dead until
reboot.
On the first poll timeout, re-issue the two GPFIFO_SCHEDULE controls the
stage-4 sequence used (channel-level 0xa06f0103 + TSG-level 0xa06c0101,
both proven accepted) and re-ring the work-submit doorbell, then poll one
more full window before the postmortem. No BIND, no PROMOTE (m76-m81
proved the explicit PREEMPT/tail variants stop the QMD enqueuing).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ipin-first data resolution
Two fixes and one diagnostic for the all-zero-results class (r89: smiley's
4 vadd launches reported GUEST GR LAUNCH OK yet every output word read 0):
- gsp_phys_booter: copy the guest data pages into the bounce window BEFORE
gsp_phys_rmcall_publish_locked() rings the doorbell. The fill used to run
after the kick; the engine front end starts within ~7 ms, so the kernel
could read the window before it was filled.
- gsp_phys_booter: per-page checksums at copy-in, compared at copy-out;
the copy-out log now reports 'N changed by engine'. An all-unchanged
window after a reported completion is the front-end-ran/grid-never-
dispatched signature (completion semaphores are front-end authored).
The first 3 data (va,hpa,reloc) triples are logged per launch.
- gsp_sema_service: svc_gr_data_fill resolves data pages through
svc_ipin_lookup() first (guest-truth pins), falling back to
svc_resolve_va, mirroring the pool resolver order; a wrong page here
silently runs the kernel on zeros. First 16 resolved (va,pa,ap) triples
are logged for post-mortem comparison against the guest's own mapping.
|
| |
|
|
| |
This reverts commit f5961087aa17111c6cdb7f6641626c0c877eee6b.
|
| |
|
|
|
|
| |
(ISSUE-0002)"
This reverts commit 32a9123b087c67c2a8126c6e9789ce36f7371045.
|
| |
|
|
|
|
| |
closing (ISSUE-0002)"
This reverts commit b839a12de675818f41a8cb388e5663d804c2a1f7.
|
| |
|
|
|
|
| |
payload from GP_PUT (ISSUE-0002)"
This reverts commit cf560b257eac2394da84825f12ba0e4192dc74db.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GP_PUT (ISSUE-0002)
The CeUtils scrubber channel (entries=4096, GPU-global) stays registered
across client sessions now that deletion is client-scoped, but its channel
buffer memory is reused by later clients' push buffers: entry-pumping its
stale ring invented work and perturbed the session (r79/r80: the first
client wedged on an rwsem). The pump now skips channels with the
scrubber layout (entries >= 4096) and instead publishes the word the pma
scrub actually waits on - the channel buffer finish payload at
gpfifo_va + 8*entries + 4 (live-verified: 0x120064000 + 0x8000 + 4 =
0x12006c004, the address from the hung client's register dump) -
forward-only from the channel's own GP_PUT (one payload per submitted
push, the same counter the guest's ceutilsUpdateProgress polls).
Skipping the scrub's actual memset work is acceptable for the emulated
vidmem: the guest's scrub clears pages of its own shadow, and the
emulation's job here is the completion contract. Revisit if per-tenant
vidmem scrubbing is required.
|
