| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
queue 4k
- counter_pa autoreg result is written back to the channel table so later
passes cannot re-target a different slot when the live GR count changes
during teardown; re-registration of the same channel preserves the slot.
- counter slot layout uses the max GR count ever seen in the generation
instead of the shrinking live count.
- sema_map_add queue 64 -> 4096 entries so a full-window pin burst drains
without ENOSPC retries.
- pin_gr_stream: one batched sudo pass per flush (was one process per run,
~4 s for 400 runs), retrying only queue-rejected runs.
- semfix_uvmt: SEMFIX_SKIP_PB=1 skips the blocking push-buffer chain walk
when the vsock window stream already supplies those pages.
|
| |
|
|
| |
GR GPFIFO rings)
|
| | |
|
| |
|
|
| |
last release VA; larger push log budget
|
| |
|
|
| |
corruption); tooling guards
|
| |
|
|
| |
semfix+uvmt watcher
|
| | |
|
| |
|
|
|
|
|
| |
The final cuCtxCreate stall is a UVM tracking semaphore whose completed
value stays 0; the pool payload is the ground truth. semfix now prints
idx/+0x10/+0x28/payload for every tracking sem it sees, whenever a field
changes, so the last stall before a failure is visible in its log.
|
| |
|
|
|
|
|
|
|
| |
The first deltas of a 15k-page window took tens of seconds to arrive while
the client was mapping under load, so the pins still landed after the
client's cuCtxCreate deadline. The scanner now reads only the first 64 KB
of every 2 MB region - the pushes sit at the per-channel allocation heads -
which cuts the per-loop page-table work ~50x, and it logs scan timing so
the latency stays visible.
|
| |
|
|
|
|
|
|
|
|
| |
The guest-agent exec path is too slow under driver-init load (a 15k-page
window scan took 17-35 s to complete), so GR window pins landed after the
channels' entries had been consumed and after the client's cuCtxCreate
timeout. The guest now scans /proc/<pid>/pagemap locally and streams only
delta runs over vsock (no agent on the hot path); the host listener pins
them as they arrive. The scanner writes a pidfile so restarts do not kill
the new instance.
|
| |
|
|
|
|
|
|
| |
Under load the semfix poll grepped the whole (growing) trace file and the
track_waits push block spawned three guest processes per sample, both of
which slowed the pin path into the client's failure window. The trace read
now tails 128 KB; track_waits.sh gates the push dump behind PUSH=0; the
pagemap helper reads 32 KB chunks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pin output
The first GR window pin landed ~2 s after the pump consumed the freshly
created channels' GPFIFO entries on the first failed read, so the pushes
(and the tracking-semaphore releases they carry) were lost. GR entries with
an unresolvable push page now stay pending for sema_gr_grace_ms (default
8 s) before the stale-entry consume; a pin landing in that window makes the
entry readable and it executes normally. The completion counter is still
published from put at the end of every pass, so the client's counter wait
is not delayed.
guest_pmap_win.py coalesces contiguous VA+PA pages into run lines, so a
15k-page window transfers as ~200 lines instead of ~15k and the pin batch
lands within a second of the pages appearing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
injected-pin table
The GR channel GPFIFO entries point at pushes in the client GPU VA window
(observed 0x200400000 + stride, one allocation per channel) and the pump
cannot read them: pushes_nomap grows and the tracking-semaphore releases
they carry never land. The pushes are sysmem pages mapped into the client
process, so /proc/<pid>/pagemap in the guest gives the GPA.
New guest helper guest_pmap_win.py batch-reads the pagemap of a VA range
(512 entries per pread). New host tool pin_gr_window.py rescans the window
on an interval, follows cdt pid changes, and injects VA->PA runs through
sema_map_add (same mechanism as the counters and the CE pushbuffer pins).
The injected-pin table grows 8k -> 32k slots so the CE pushbuffer (4096)
plus the GR window pages fit without drops.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two fixes for the CE data path. (1) svc_handle_map_add rewound every
channel by 8 ring entries on every injected mapping; a 16 MB push-buffer pin
is thousands of page injections, so the pump's last_get was pinned in place
and it re-read the same entries for the whole batch (3645 submissions, zero
ops). The rewind now runs once per drained batch. (2) The push-start
heuristic (op=INC_METHOD at address 0) rejects UVM's real CE memop pushes,
which start with an IMM method at address 8 (observed 0x60020002, PTE data
words) or a release method at 0x700 (0x200381c0); a buffer read through a
host-injected (ipin) translation is now decoded fully trusted, ops included.
Result on the next boot: memset=2718/0/0 ringops=1794 (first sustained CE
work), false 12k->1.9k. semfix_watch: bulk-pin contiguous runs via
sema_map_add len, read rm_mem size, 16 MB (4096 page) cap, non-blocking QMP
read (the fixed 30 ms sleep made 4096 gva2gpa calls take two minutes), and
KEEP_TRACE=1 to preserve the guest trace for post-mortems.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The CE push-buffer page 0x120000000 resolved to a zero-filled page even after
sema_map_add injected the correct translation: the shared pin table silently
dropped inserts once a hash cluster exceeded the 32-probe window (the page
hashes to slot 0, a hot cluster), and lookups fell through to the
content-scanned map and answered with a stale page - so every CE push read as
implausible and memset stayed 0. Injected translations now live in a separate
authoritative table checked before every other source and are never written by
self-discovery; the general pin table is 4x larger with a 4x probe window and
counts drops (pin_drop/ipin_drop) in the stats line. guest_kprobes.sh disables
the kprobe group before removal (EBUSY) and matches p:kprobes/<name>; the
guest wait sampler can run counters-only with GDB=0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
chunk-read diagnostics
semfix_watch.py now resolves the CE pushbuffer from guest truth instead of
scanning a wide raw window: it walks uvm_channel_t +0=pool, pool+0=manager,
manager+8=pushbuffer, pushbuffer+8=rm_mem, rm_mem+0x10=vas[] and prefers the
rm_mem whose GPU VA is in the 0x120000000 CE pushbuffer range. Three entry
points: uvch probe on __uvm_push_begin_acquire_on_channel_with_info, uvbp
probe on uvm_pushbuffer_begin_push, and the uvmt tracking-sem kptr (channel
scan back from sem). Misses now print structured hop counters so a failed
chunk read is distinguishable from a failed chain. guest_kprobes.sh
(re)installs uvmt/uvch/uvbp/uvpbf after every guest boot (plain, module:
and bare spellings).
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
candidates)
|
| |
|
|
| |
GPA layout)
|
| | |
|
| |
|
|
| |
injector marker-based handle discovery
|
| |
|
|
| |
injector, wait sampler
|
| |
|
|
|
|
| |
The client times out the context-init counter wait in ~30 s; the previous
poll loop spent ~50 s on chan_dump round-trips before injecting. Count the
per-channel first-pump log lines straight from dmesg every 0.5 s instead.
|
| | |
|
| |
|
|
|
|
|
|
| |
Live verification: the trackers waiting on counter slots idx2/idx3 belong
to the channels 2nd/3rd from the top of the ascending handle list, and the
only channel whose GPFIFO put advanced to 2 was the 4th from the top,
whose tracker polled idx3. Zip descending handles with ascending counter
indices by default; ORDER=asc restores creation order.
|
| |
|
|
|
|
|
| |
Restart the VM, push the guest tools, start cdt holding on failure, wait
for the 20 GR channels, register/pin the completion counters and arm
scanerr.py, so each iteration is a single command. scanerr now loops over
scanner hits until a hit carries a non-zero RM error record.
|
| |
|
|
|
|
|
|
| |
The libcuda error scanner (FUN_0054fa00) walks a context list and returns
999 when its force flag is set and any context's RM error record has a
non-zero class. ctx999 only shows the first (zeroed) record; scanerr walks
the whole list and prints every non-zero record so the failing control or
alloc is named. Pushed by push_guest_tools.sh.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The counter enumeration/injection workflow lived in the guest's /tmp and was
lost on every restart. Make it durable under tools/guest-diag:
- sem_counters.py: translate a process's per-channel u32 completion counters
(16-byte stride in the /dev/nvidiactl counter page) to guest PAs through
/proc/<pid>/pagemap; --pairs emits handle,va,pa triples.
- sem_slots.py: read/write the counter slots via /proc/<pid>/mem (fallback
for the host counter publish).
- read_va.py: hex dump a process VA (UVM sysmem GPU VAs are usually valid
CPU VAs in the owning process); --u32 for method streams.
- inject_counters.sh: host-side one-shot: push the tool, take the GR handles
from the live chan_dump, enumerate, then register each counter with
sema_counter and pin its VA with sema_map_add.
- push_guest_tools.sh: push the new tools after every VM start.
- launch_qemu_direct.sh: parameterized mdev UUID/paths instead of a stale
hardcoded one; committed so it survives checkouts.
|
| |
|
|
|
|
|
|
|
| |
The 15 s pinner loses the race against the client's internal deadline in
most boots: in the one boot where UVM's page-table write pushes were
submitted the pin had landed early, and later pins (60 s+) left the client
dead before submitting anything. Poll the guest pagemap every 2 s and pin
0x20400f000 once per cdt pid (re-pinning the same pid also stirred the
pump's rewind repeatedly, which is why the guard exists).
|
| |
|
|
|
|
|
|
|
|
|
| |
ctx999.py arms the scanner entry/returns plus cuCtxCreate and dumps the
ctx->mgr->reg->record chain at the stop. First live capture: the scanner is
called with param_2=1 during normal context creation and correctly returns 0
on a zero record (mgr[+0], the channel handle, and the record walk are now
confirmed live). pin_miss_pages.sh must match the exact VA pmapall printed:
guest_pmap_all.py returns the first nvidia page at-or-above va-min, so a page
that is not present resolves to an unrelated mapping (it pinned a UVM page
for 0x20400f000).
|
| |
|
|
|
|
|
|
|
|
| |
pin_miss_pages.sh watches the host journal for release-miss VAs, resolves
each page for the live cdt pid through the guest pagemap and injects it (the
injection rewinds the pump so releases consumed under a stale pin are
re-read); it re-pins when a new cdt pid appears. cuda_drv_test prints
cuGetErrorName/String on failure and, with CDT_HOLD=<seconds>, stays alive so
the RM error record and tracker state can be inspected after cuCtxCreate has
given up.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
record
A freed client's notifier events (hSrcResource 0) stayed valid in the event
table, so the fallback wake-up nudge kept posting POST_EVENT records for a
(hClient, hEvent) pair the guest RM no longer knows; every one asserts in
CliGetEventInfo (kernel_gsp.c:466) and is dropped (171 failures in the
15:24 window of the last run, zero after this fix). FREE now retires the
matching event record (and a client's records when its root object is freed).
bp999.py now walks rdi -> ctx -> ctx[0x1291] -> +0x10 -> +0x10 and prints the
RM error notification record, and push_guest_tools.sh ships it to the guest.
|
| |
|
|
|
|
|
|
|
|
| |
The guest RM's expiring waits are kernel busy-polls and all of their
functions are notrace, so ftrace/kprobes cannot name them. sysrq-l NMI
backtraces can: this sampler triggers them and keeps only the Comm and
[nvidia] frame lines per round, so the wait/teardown stacks survive dmesg
ring wraps. Caught the adapter-shutdown scrubber stack
(channelReadChannelMemdesc <- ceutilsUpdateProgress <- scrubberDestruct <-
memmgrDestroyInternalChannels <- gpuStatePreUnload <- nv_shutdown_adapter).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The content sweep reads arbitrary guest RAM and ran the full push decoder on
every page, so fabricated method streams in unrelated data executed
fills/copies: one boot scanned 65M pages, 'decoded' 124k pushes and ran
dozens of memsets from false positives - writes into live guest memory (a
guest kernel crashed in nvidia_uvm right after such a boot).
- Swept pages now only permit the bounded semaphore-release writes; fills
and copies execute only for pushes the pump read through a trusted ring
translation.
- Pushes read for a compute (GR) channel apply releases only: their method
space is not the copy engine's, so aliased encodings must never become
memory writes.
- sempages.py derives the pages to pin from the client's tracker objects
(sem object + sem memory) instead of matching bare sem-object shapes, which
pinned 140 pages where the client has ~53 sems; shape scan remains as a
fallback when no tracker resolves.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- The CE semaphore/memset method groups also arrive on subchannel 0 (CUDA's
push library binds the copy class on compute channels to carry the
work-submission releases), so the stream decoder accepts both 0 and 4.
- GR (compute) channel rings live in the same video-memory allocation as the
CE rings, 0x3000 bytes per channel after them: ring (0x2000) at
ce_end + 0x3000*rank, USERD at +0x2000. Discover and pin them once the CE
ranks are known; verified against ring entry 0 (1 MB aligned push VA in
the client window, sane length) and the USERD's put/get, so a layout
change costs a failed pin, not a wrong one. Gated by sema_gr_rings
(default on).
- memprobe: require the sem-VA triple in the sem-object scanner (a bare
qword in the window matches thousands of unrelated pointers once the
ring/pushbuffer windows are part of the range).
Live evidence: ring entries read through the guest's /proc/pid/mem decode to
the GR channels' first pushes; pinning the ring (fb 0x140000+0x3000*rank),
USERD (ring+0x2000) and first-push pages made all 20 GR channels execute
their entry 0 for the first time (get=1 sub=1).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The content sweep now:
- decodes a release whose method stream continues with any INC_METHOD,
not only the CE LAUNCH_DMA group, so the releases at the head of a
compute/GR channel's first push are applied too;
- remembers releases whose target VA had no translation (deferred list)
and retries them every service pass, so a VA->PA injection lands them
immediately instead of after the next full sweep;
- re-arms the fast sliced sweep for a few seconds after an injection;
- treats a channel whose USERD/ring cannot be resolved as stalled, so the
sweep keeps looking for the releases its unreadable pushes carry.
tools: add per-cdt client-pin helper (sem pages + /dev/nvidia0 windows),
sem-page scanner and memprobe fixes (short-read guard, --brief).
Live-verified: injecting a client sem page applied 12 deferred releases
within a second; the service kept applying (89 -> 202) while cdt ran.
|
| |
|
|
| |
replies in the ioctl tracer
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
- Add expected_sem_payload tracking to svc_channel for CE channels
- Implement svc_rescue_push_page() to scan guest RAM for push with
expected semaphore release when VA translation fails
- Add svc_init_expected_payload() to initialize expected payload from pool
- Add rate-limited logging of distinct stalled VAs (svc_log_stalled_va)
- Advance expected_sem_payload on successful push decode
- Pushes SEMA_MAP_MAX to 65536, SVC_PIN_SLOTS to 32768 for scale
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mappings into the service
The pin table became the primary VA->PA store once push-buffer regions and
bridged client mappings started adding thousands of pages per boot; at
8192 slots with an 8-probe window, inserts were silently dropped and
lookups missed at half load. 32768 slots and a 32-probe window keep the
mapped set intact.
tools/guest-diag/{guest_pmap_all.py,bridge_sems.sh} translate the CUDA
client's /dev/nvidia* mappings through the guest's own pagemap and inject
each present page into the service as a sysmem pin. With the client's
semaphore pages pinned the service applies their releases into the page
the client polls (verified: applied 67 -> 109, misses stop growing for
those VAs), where before every write went to the vidmem shadow.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the libcuda tracker wait
The vidmem window convention turns any VA above the window base into a
vidmem offset, but the CUDA client's GPU semaphores are sysmem objects
mapped into its address space at those same VAs. A release applied through
the convention therefore landed in the vidmem shadow instead of the page
the client polls, and the completion was silently lost: the libcuda
tracker it waits on never advanced. Semaphore reads and writes now use
trusted translations only (pins, learned cache, discovered map, page
walks), so an unresolved semaphore is counted as a miss and stays visible
instead of corrupting a shadow.
tools/guest-diag/trkinfo.py dumps the libcuda tracker the client spins on:
the tracker object behind p+0x9410, the semaphore object it references,
the u32 semaphore memory it polls (a /dev/nvidiactl mapping backed by
guest RAM) and the VMA that memory lives in.
|
| |
|
|
|
|
|
| |
The probe dumps {type, CPU VA, GPU VAs, size} on every
uvm_rm_mem_get_cpu_va call (and the first 64 bytes of the object's CPU
mapping with nested fetchargs), which is how the host-side learned map
can be checked against what the guest actually created.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- admit {GPU VA -> PA} pairs only when the preceding bytes form a valid
client memory descriptor header (type 0/1, CPU+GPU mapped bits), prefer
validated pairs over headerless candidates under first-wins dedup, and
invalidate the translation cache on every rebuild; fixes stale identity
entries shadowing live pool pages and lets the guest UVM channel
bring-up complete on its own
- capture OS event allocations (class 0x79) and post event records
(function 0x1003) into the guest status queue when a channel's busy
notification is consumed, with a broadcast fallback, serialized with
replies via a queue lock
- add BAR0 shadow introspection (dump_bar0/scan_bar0) and guest-side
diagnostic scripts for the VM cycle
|
