| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The one-shot #sem kprobe event that drives the CE push-buffer chain
resolution can be lost (trace rotation, scanner restart, slab kptr reuse),
and the chain resolver then never learns a client - every GPFIFO entry of
its CE channels is grace-skipped and the client spins on its tracker
semaphore (the copy-heavy stall class: streams2/memcpy/multikernel/saxpy
in r148; smiley as a later client in r147 took ~90 s to recover).
New demand channel, both ends here:
- pin_gr_stream: _demand_loop scans the host dmesg for 'stalled VA #N:
0x...' (the service logs each distinct unreadable push VA once), tracks
each sighting, and re-sends every 20 s for up to 5 min to
/tmp/guest_want_vas.txt in the guest (via the guest agent).
- guest_pin_stream: the main loop reads new VAs from that file and resolves
each by CONTENT against the nv_vm_map_pages vmap areas
(resolve_va_by_content: candidate leaf page at the VA's in-window offset,
+1/-1 neighbours for the header-page offset ambiguity, host-plausible
method-stream prefix). On a match the exact pair is emitted; only the
requested VA is pinned, so a wrong candidate cannot poison other VAs.
Also fix the scanner start: the kill-first pattern must be anchored
(^python3 /tmp/guest_pin_...) - the unanchored pattern matched the deploy
shell's own cmdline and every start attempt killed its own shell before
launching the scanner (r149: no scanner for the whole boot).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r148 ran THREE guest scanners (pin_gr_stream install_and_start retried
while a partially-failed first attempt had already started one, and the
pid-file kill missed it). Each scanner re-asserted the whole CE pair set
every 2 s; with a fully fragmented pair set (descending PAs, one page per
line) that is ~2 k debugfs writes/s per scanner, and the guest hit load 19
with clients wedged spinning in userspace.
- pin_gr_stream._start_cmd: kill EVERY existing scanner first (pattern
assembled at runtime with printf so the command's own cmdline cannot
self-match), then start exactly one.
- guest_pin_stream._emit_ce_mem_delta: the full re-assert keeps its 1.5 s
cadence for the first STREAM_FULLREM_FAST_S seconds of a mem's life (the
client-switch flush race), then backs off to STREAM_FULLREM_SLOW_S (10 s)
as a safety net. Configurable via env.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The host wipes its whole sema map when the previous client's last channel
closes ('session flush (all channels closed)'). A pin batch that raced in
for the new client (observed: the 1-page 0x120000000 pin emitted in the
same second as the flush) was lost, and the scanner's content-gated delta
never re-emitted the unchanged pairs - so the second client's CE pushes
were permanently unreadable (sema-probe 'NO ENTRY for page 0x120000000'),
every GPFIFO entry was grace-skipped (r144/r145 matmul and mail stalls),
and the guest spun in uvm_tracker_wait_for_entry.
Fix (scanner, guest side):
- _ce_mem_pages: walk the whole vmalloc area (minus header off), not the
size-derived prefix that stopped one page short of the write frontier.
- _emit_ce_mem_delta: full re-assert mode every KC_FULLREM_S (1.5 s) plus
every fresh_map pass; emits every translatable pair content-independent,
merged into contiguous runs. Zero-content pages are pinned on purpose:
the pin carries the translation, the host reads live content.
- missing pages retried with the negative PA cache popped (bounded 512/pass).
Fix (host tool):
- pin_gr_stream._flush: never dedupe single-page repeats of the CE window
(0x120000000..0x121000000) so the re-asserts land.
Live-validated in r145 (same-boot, no rebuild): mail green as 4th client,
matmul green as 2nd client (bad=0, 64-changed acceptance of 857fe5f9),
both with zero 'stalled VA'/'skipping stale entry' lines; pre-fix the same
sequences stalled. Headless smiley all-wrong (bad=131072) reproduced
independently and is the next target (engine-side constant class).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
scrub freeze)
guest_pin_stream: find the scrubber channel buffer among the nvidia vmap
areas (size in the CeUtils channel set, 906F ring shape at +0x64000 when
non-empty), walk its pages with kcore and emit '#scrub <fin_pa>,<hsem_pa>'
(the finish payload at +0x6c004 and the host semaphore at +0x6c000).
pin_gr_stream: handle '#scrub' lines and register the PAs through the
sema_scrubbuf_add module parameter, so the kernel's svc_scrub_pass can
author the guest's scrub progress.
|
| |
|
|
|
|
|
|
| |
The one-shot CE mem emission pinned the whole 16 MB window including
pages whose content was zero at the time; the host then read zeros for
the other CE ranks' pushes. Sweep the mem every 2 s, skip zero pages,
emit only new/changed {va,pa} pairs, and burst the Kcore PA cache every
30 s so a remap is re-emitted instead of shadowed by the cache.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
tracking-sem case
The second-client stall leaves exactly one live uvmt event: a tracking
semaphore spinning with val=0 (the other channel/one-shot probes are inlined
in this driver build and never fire). The sem is a member of a uvm_channel_t
whose records sit in the sem's own window and carry the CE ring GPU VA
(0x121010000+) plus back-to-back object handles; the channel's pool pointer is
the first field at channel+0 (channel = sem-0x80), and the pool chain already
satisfies _chain_mems_at once it is tried. The resolver now expands the
candidate set from every ring-VA word's surrounding records and their targets
before scoring. Verified live: the r134 spinning sem resolves to the CE
push-buffer rm_mem with 413 pin runs, where the bare sem-window scan had
produced none.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
freshness on an unknown page PA
Two scanner defects seen in r131: (1) a resolved CE mem whose first vmap
page could not be translated at store time stored page_pa=0, and the
freshness check then failed on every probe, dropping the latch and
re-resolving in a tight loop - and each failed re-resolution left the host
with no pins for the channel at all; (2) the stale path discarded the latch
before knowing the re-resolution would produce anything. Unknown PA now
counts as fresh (the {cpu,gpu} object check is the evidence), and a stale
refresh keeps the previous latch unless a new resolution actually emits
runs.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
clear_trace() on a pid switch dropped exactly the one-shot uvmt/uvch
events of the new client when they landed in the switch instant; the
pipe reader then never saw them and the whole session ran with an empty
sem set (r126: vecadd_loop sem=0 for minutes while its CE entries paid
the 8 s grace). Scan the ring tail once per switch and seed the sets
instead - stale kptrs that are also re-learned resolve to no content and
are deferred by the candidate scoring.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The kptr-keyed one-shot latches assumed object identity, but UVM frees and
re-creates CE push-buffer rm_mems and channels mid-session and the slab
reuses the same addresses; a resolution taken before such a move pinned
pages that read all-zero when the host used them, so every CE ring entry
stalled in the 8 s grace (r125: the live pushes sat at new vmap pages of
the same rm_mem kptr). Candidates are now scored by their window content
(a live push buffer is full of method streams, a freed range reads zeros)
and the best is pinned - all-zero candidate sets emit nothing instead of
poisoning the host map. A resolved mem is re-validated every ~2 s against
its own {cpu,gpu} fields and first-page PA; when they move, the latches
drop and the current truth is re-emitted.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
derive the CE vmap pairing per allocation
Two second-client zeros/stall countermeasures:
- svc_gr_data_fill now reports when a launch carries high (device-space)
argument pointers and none of them resolves through any trusted source.
svc_gr_request refuses to forward such a launch and leaves the capture
unconsumed; the pump holds the ring entry for one bounded window per
entry (gsp_sema_gr_grace_ms) and retries it, so a translation that
appears a pass later (scanner pin, published directory, learned pair)
is used instead of executing the kernel against unresolved pointers.
On window expiry the entry is consumed exactly as before (bounded
zeros, never a hang). Bounded per-session fill diagnostics name the
source that serves each candidate (ipin/pin/lat/map/vas/disc/win) and
dump the published vaspace walk results for misses.
- The CE push-buffer kcore chain resolver derives its vmap page offset
per allocation instead of using a fixed shift: the rm_mem size is read
from its real offset (570.144 layout) and paired with the vmallocinfo
area page count, so a leading header page is only skipped when it
actually exists. With an unreadable size the two candidate offsets are
content-probed with the host's method-stream plausibility test and an
ambiguous result keeps the status-quo pairing. This removes the
off-by-one that fed wrong push pages to the pump (content-rescue
crawl).
|
| |
|
|
|
|
|
|
|
|
|
| |
The +1 vmap pairing was content-validated on one boot's CE push-buffer
allocation (4097-page vmap: five rescue anchors at k+1), but r116/r117
show the leading header page is not universal: with the shift applied
client #1 stalls mid-run (rescues resume) where it was green without it.
Reverting the shift restores the previous behaviour; the kernel-side
repair from 16124929 stays (a content-validated rescue replaces the
injected pair, so a wrong pin is corrected once per page instead of
being re-rescued on every pass).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rescues repair injections
The CE push-buffer uvm_rm_mem is vmalloc'd as a 4097-page area: one
leading page plus the 4096-page (16 MB) GPU data window, so GPU data
page k lives at vmap page k+1. The kcore chain resolver paired page k
with vmap page k, pinning the adjacent push for every page; the pump
then read the wrong push, decode failed, and the host fell into the
content rescue which crawled one entry per ~2 s (r115: rescue_run=4202,
entry_va=0x12001b18c repeating). Verified against five independent
content-validated rescue anchors (k=20..28, all delta +1), and the vmap
area boundary confirms the extra page (nv_vm_map_pages 4097 pages).
Five fixes:
- resolve_mem_pins() takes page_off; the CE chain and uvbp push-buffer
paths pass mem_page_off() (1 when the vmap is exactly 16 MB + 1 page;
0 for direct-mapped mems like the semaphore pool).
- a content-validated rescue page now also replaces the injected-pin
entry (svc_rescue_try / svc_rescue_run_try), so a stale or wrong
injection can no longer shadow the validated page and re-run the
rescue on every pass.
Evidence: r115 rescue anchors va 0x120014000->pa 0x12a2e2000 .. va
0x12001c000->pa 0x12a2de000 all at vmap offset k+1; the candidate pages
decode as the adjacent push (wrong semaphore payload) before the fix.
|
| |
|
|
|
|
|
|
|
|
|
| |
Two robustness fixes for the sequential client battery. cdt_pid() now
picks the highest-pid CUDA candidate: preferring a name-prefix match
kept the scanner pinned to a previous client that was still tearing
down while the new client had already started pushing, so its first
pushes were unreadable, the release expectation drifted past the real
payloads and the stall followed. The CE push-buffer chain resolver now
also runs for uvch channel kptrs, so the pins do not depend on the uvmt
semaphore events alone.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
stall)
The second CUDA client's GPFIFO entries in the CE push-buffer window
(0x120000000) had no injected pin: that window is never mapped into the
client's pagemap, so the scanner's window scan cannot see it, and its
only supplier - the one-shot uvbp kprobe event - is raced by two trace
clears (the scanner's pid-switch clear and semfix's trace-tail clear).
When the event loses the race the entry reads fall back to the ~250 ms
content rescue, which is the second-client stall and the same
resolution-miss class as the ISSUE-0001 lottery.
The guest scanner now walks channel->pool->manager->pushbuffer->rm_mem
in kernel memory with kcore and pins the push-buffer pages directly (no
trace event to lose), and the trace reader's dedup marks reset on a
client switch so a reused kptr cannot suppress the new client's #pb.
Boot evidence (r112): rescue reject ipin=0 for va_page=0x120023000,
rescue scans 175 ms - 1 s each, want incrementing to 39. Smoke test of
this fix on the stalled VM: 478 pin runs (~7955 pages) streamed within
seconds, first va=0x120000000 -> pa=0x108090000. A bounded kernel log
now prints the first CE push-buffer pins of each session.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registration; scanner streams anonymous high-VA buffers
Second-client wedge fix: a new GSP session re-registers the same channel
handles/ranks, but the per-rank one-shot ring/push-buffer discovery latches
stayed set from the previous session, so the new session's ring was never
re-autopinned (ring_auto=0 pb=0/0, nomap growing) and the guest spun in its
channel-creation wait. svc_chan_rearm_rank() clears the latches on every
channel registration; all post-fix cycles launch the second client.
Scanner: guest_pin_stream.py now streams the client's anonymous high-VA
mappings (cudaMalloc buffers were skipped by the old len(parts)<6 check),
so the host can learn their pages.
Diagnostics: PT-window lines, per-window first words, post-OK qmdwin
w48..59. Remaining: the fill-time svc_ipin_lookup miss for the second
client's a/b/c pins (they exist in the pin log but are not seen at fill
time) - next session.
|
| |
|
|
|
|
|
|
|
|
|
| |
failure)
The watcher's resolve_sem_pool read sem->page->mem->vas through a
process-lifetime cache: a transient zero captured while UVM was still
initialising the structures was served forever, the pool pin never
landed, and the second-session client hung in cudaMallocManaged
(r88 evidence: 'pool unresolved (retry)' from the watcher while the
same walk succeeded from a fresh QMP client). Cache is now opt-in.
|
| |
|
|
|
|
|
|
|
| |
Two failure modes found when the scanner detects clients by device fd:
a transient boot nvidia-smi was picked and its exited pagemap read back
EOF, and scan_window advanced by got=0 forever (busy spin, no stream);
and non-client nvidia fd holders must never be committed to. Break on
got<=0 in scan_window (scan_range already did), skip nvidia-smi/
persistenced/Xorg, and prefer /dev/nvidia-uvm holders.
|
| |
|
|
|
|
|
|
|
|
| |
The guest pin scanner only tracked processes whose comm matched
CLIENT_PREFIXES (cdt,va,victory). An arbitrary CUDA binary (e.g.
/tmp/smiley) was invisible: no pages streamed, no VA->PA pins injected,
the service could not read its pushes and the client hung in
cudaMalloc/uvm_tracker_wait. Detect compute clients by the CUDA device
they hold open (/dev/nvidia-uvm preferred, then any /dev/nvidia*), with
the prefix match kept as the first-choice path.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The launch line carried both an explicit rtl8139 on netdev net0 and a
stray '-net user,hostfwd=tcp::2222-:22 -net nic' pair, creating a SECOND
slirp network with a second NIC and a second 2222 forward. The SSH
connection landed on one slirp instance while the guest's replies left
through the other interface, stalling the handshake before auth. The
explicit rtl8139 now owns the only user netdev with the 2222 forward.
The generated launch_qemu_rootport.run.sh is patched in place by the
harness from the same line.
Verified: sshpass -p ubuntu ssh -p 2222 ubuntu@127.0.0.1 -> nvidia-smi
green. Guest-side companion fix (persisted in the qcow2 image): the
cloud image's /etc/ssh/sshd_config.d/60-cloudimg-settings.conf set
PasswordAuthentication no; it is now yes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(ISSUE-0002)
svc_chan_userd now prefers the layout-derived CE USERD
(ring_pa + 0x10000 in the ring aperture) and accepts it while idle, so
a second session cannot latch onto the registered pool-relative offset
when that alias holds a DIFFERENT rank's stale USERD: r40 c2 rank0 read
rank3's stale 1/1 at FB 0xf0000 and the pump never followed rank0's own
GP_PUT (guest GPUT mapping for gpFifoGpuVa proves ring+0x10000).
guest_pin_stream.va_to_pa drops translations outside the registered
guest-RAM windows so BAR1-phys addresses (0x70000xxxxx) are no longer
injected as SYS pins; r40 rank2's ring resolved to the bogus
pa=0x70000c0000 ap=SYS and read 66/66.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
#put (ISSUE-0002 diag)
CE channels register gpfifo=0x121010000+0x30000*rank, outside the
GR-only window [0x200200000,0x200400000) the #put resolver searched, so
no #put line ever fired for the CE channels the second session stalls
on. Resolve channel_info from both windows with layout validation, keep
re-resolving the channel after the pool mem has resolved, reset the
#put map on client switch, and add bounded resolver diagnostics to the
guest log.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Sweep of the 155 tracked C/Python/shell files under userspace/ (excluding
the MIT-licensed vendored userspace/extern/), tools/ and scripts/ that had
no per-file license identifier. Each gains the minimal two-line header in
the area's existing form: 'SPDX-License-Identifier: BSD-3-Clause' plus
'Copyright (c) 2026 Open-IOV contributors'; shebang and coding lines stay
first. No other content touched. kernel/** is deliberately out of scope
(recorded as a follow-up in CODESTD-REPORT-2026-10-08.md): its SPDX
coverage and MODULE_LICENSE normalization need their own GPL-2.0 pass.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
window
The guest's first RmInitAdapter intermittently fails: GspStatusQueueInit
gives the status-queue link only 4 s (message_queue_cpu.c: timeoutUs =
4000000) and the first boot attempt can burn it, after which the RM tears
down, leaks the device and retries; the surviving retry leaves the GR/ctx
state degraded, which is the persistent cudaMalloc stall class.
Changes:
- gsp_rm_emu_poll: parse-retry cadence 250 ms -> 50 ms (the 250 ms tick
consumed a fifth of the guest's budget per attempt).
- gsp_rm_emu_reset: clear last_parse_retry so a new boot gets a fresh
retry budget; take stat_lock while clearing queue cursors so a sema
event post cannot race the reset.
- gsp_rm_emu_on_booter_done: on GSP_INIT_DONE send failure re-arm the
retry path (parsed/active were already set, so the poll loop would
never retry).
- gsp_rm_parse_queue_chain: free the previous ptes before re-alloc so
retries and post-reset re-parses do not leak.
- gsp_rm_post_channel_event: never post events across a queue reset
(parsed gate).
guest_kprobes.sh: add rpcs/rpcp transport probes (the direct RPC-wait
symbol is notrace in 570.144) and ccset entry/return probes for
rmapiControlCacheSet so a stall names the control whose reply breaks the
guest's control-cache bookkeeping.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ring+userd) + GR ring userd-derived candidates
guest_pin_stream.py now resolves translations entirely from guest truth via
/proc/kcore + a manual page-table walk (no QMP races): the CE semaphore pool
and pushbuffer memory from the UVM rm_mem structures, and each channel's
GPFIFO ring + GPPUT from the tracking semaphore's embedding uvm_channel_t
(channel_info located by its gpFifoGpuVa value). This removes the watcher
pipeline's flakiness on the cudaMalloc stall class (missing pool/pushbuffer
pins).
gsp_sema_service.c: svc_gr_ring_autopin tries the guest-registered USERD
memdesc base as a ring candidate (the fixed FB convention only matches when
the guest's VRAM allocator happens to reproduce the old layout), plus an
optional (gsp_sema_dbg) one-shot FB scan + raw-word diagnostics when every
candidate fails.
tools/cuda-ab/p2: self-checking arbitrary-kernel acceptance suite (10
programs: sequential launches, shapes, params, smem/GEMM, streams, malloc
churn, printf+error, FMA benchmark, mixed, managed probe) with host oracle
expectations and a guest runner.
|
| |
|
|
|
|
|
|
| |
The hostfwd tcp::2222-:22 sat on the rtl8139 NIC (net0, guest enp0s7,
DOWN); the guest networks over the legacy -net user NIC (enp0s2,
10.0.2.15), so SSH to the guest timed out during banner exchange. Move
the forward to the legacy NIC (net0 loses it) - ssh -p 2222 now reaches
the guest and key auth works.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
second-client stall race)
The guest scanner's TraceReader tailed the last 256 KB of the trace ring
every 0.3 s. A one-shot uvmt sem= event could age out of that window
before the next poll (the client's spin floods the ring), so a second
client in the same VM was never discovered: semfix re-pinned the previous
client's freed sem while the new client's push page stayed untranslated
(host nomap, drain held, guest spins forever).
- guest_pin_stream.py: consume /sys/kernel/tracing/trace_pipe instead
(each event delivered exactly once; no ring-window race), with
reconnect on read error. The old ring-tail read is kept only as
semfix's fallback.
- clear the trace ring on every client switch so the fallback cannot
re-learn the previous client's stale events.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pushbuffer pins
Root cause of the cudaMalloc/UVM-init stall on our bring-up path:
guest_kprobes.sh ran while nvidia_uvm was not yet loaded (it loads lazily
with the first CUDA client), so every probe add failed silently (symbols
not in kallsyms) and the host never got the #sem -> pushbuffer-pin
pipeline. The CE push page (0x120021db8) then never translated and the
guest spun forever at DEVICE.
- guest_kprobes.sh: modprobe nvidia_uvm, wait (bounded 30 s) for the
uvmt symbol, install, and print KP_ARMED/KP_FAIL for bring-up checks.
- semfix_watch.pin_pushbuffer: retry transient/partial gva2gpa misses
in-call (bounded passes with a short sleep) instead of aborting on the
first all-miss block; report unresolved page counts.
Validated: 2/2 cold boots (vmkeep6/7) arm the probes and run
victory_mail green in 15-20 s (MAIL + VADD bad=0 checksum=12204516 +
STATUS OK, oracle-matched); pre-fix boots with unarmed probes stalled
4/4.
|
| |
|
|
|
|
|
|
| |
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean
- tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests
- tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode
- debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding
- STATE-CURRENT/docs updates
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registry variant; complete vgsp rename; docs/tooling refresh
- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA,
batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel,
old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only,
PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM
geometry queries and buffers; pre-boot SET_REGISTRY profile selection
- gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf
parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0
- vgsp rename across kernel, scripts, tools and docs (module vgsp.ko,
/sys/kernel/debug/vgsp/, vgsp-<subsystem>: log prefixes)
- risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag
tool updates; README/SBOM/install/Makefile sync
Live status: CE ladder and P6 guest-copy forwarding green; GR engine context
commit still open (all promote shapes accepted or cleanly refused; the channel
stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- file-backed guest RAM (launcher memory-backend-file share=on) and
in-kernel GPA->HPA translation through the shared file page cache
(read_cache_page + page_to_phys + pfn_valid), opened O_LARGEFILE;
new default-off gsp_phys_guest_mem parameter
- debugfs gsp-phys-guest-ram (path + gpa/off/size ranges) and
gsp-phys-guest-gpa (one pinned GPA per boot); rm-call triggers 6/7 run
host->guest and guest->host 4 KiB CE copies; trigger 8 is a PTE
self-test through a fresh slot
- guest page-table slots are pre-built during ladder stage 3: runtime
PTE writes into an already-walked page-table line are not observed
without a TLB invalidate
- guest RAM layout (q35 4.2, 4 GiB): below-4G GPA == file offset,
above-4G offset = GPA - 0x80000000 (2 GiB PCI hole)
- tests: tools/test/gpage_test.c guest probe; docs + STATE-CURRENT
record the live green (G2H/W2G with semaphore release, guest observed
the write, smoke ALL OK 64/64, clean latched release)
|
| |
|
|
| |
third-party references
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Host-freeze fix bundle (validated 2/2 supervised cycles):
- power unlock now latches D0/D3cold by default (new gsp_phys_repower,
default 0); releasing the claim can no longer power-gate the board and
hard-freeze the host on resume
- bounded post-rebind readiness gate before dropping the locks; bounded D0
settle/verify before the unbind
- READY means a live GSP-RM session: the async worker and the module-owned
join only expose READY when init-done is seen and no unload was sent;
otherwise the claim is released fail-open
- FWSEC reads DMEM0/IMEM back before STARTCPU and fails fast when the
staging is not visible
- the claim harness leaves the board latched by default and takes
--restore-power to opt back into the old restore
Guest-wiring:
- P2: generation-keyed fn65 static-config snapshot cache plus a default-off
identity overlay (new gsp_phys_static_live) for name/short-name/VBIOS;
the probe cycle runs once per backend generation at READY
- P3: default-off read-only RM_CONTROL forwarding whitelist (new
gsp_phys_ctrl_forward) with emulated-handle translation to the adopted
server handles and full emulation fallback
- P5 wave 1: default-off shadow execution of guest CE pushes on the ladder
channel (new gsp_phys_work_forward) via a dedicated ordered work item;
the ladder stage-4 submission is refactored into a shared bounded helper
and the software release path stays authoritative
Live: module-owned boot to init-done, guest smoke ALL OK 64/64 with P2/P3
enabled, 514 shadow copies observed on the physical CE; all cycles released
cleanly with WPR2 down and no reboot.
|
| |
|
|
| |
harness
|
| | |
|
| | |
|
| |
|
|
| |
--boot is the only device-writing mode (full boot lifecycle: FWSEC, Booter Load, RISC-V start, FALCON_OS, RM-init poll). --boot-dryrun rehearses the host staging read-only. --keep leaves a green claimed session running. --fwsec is retired and refused at parse time: a FWSEC-only carve is not host-recoverable on this part.
|
| | |
|
| | |
|
| |
|
|
| |
abort checks)
|
| |
|
|
|
|
|
|
| |
Reproducible TASK 2 harness: fresh boot, sema_dbg=Y with
sema_copy_dst=0x200000 / sema_copy_dst2=0x208000, cdt2 parked at
CDT_CP_PAUSE so the root PD/PT pages can be dumped and walk_probe run
while the client is live. Prints the offending write data (prov= names
the translation layer) and the post-run walk state.
|
| |
|
|
|
|
|
|
|
| |
The DtoH staging buffer lives in the client's high-VA /dev/zero|nvidia
mappings (e.g. devptr+0x3c00100), outside the scanner's fixed CE/GR
windows; only the host's pagemap read can see it. The guest scanner
now walks those VMAs and streams their present pages so the module
pins them. run_cycle.sh learns the sema_dbg master switch (auto-on
when any watch knob is set).
|
| |
|
|
| |
failure left a boot without pins and the client hung)
|
| |
|
|
|
|
| |
A fresh boot can leave /sys/kernel/tracing as an unmounted mount point;
the probe adds then fail silently (probe count 0) and the whole sem/kptr
pipeline goes blind.
|
| |
|
|
|
|
|
| |
A new CUDA client allocates fresh channel pools at the same GPU VAs; the
watcher kept the previous client's sem kptrs and seen-pin marks, so it
pinned the previous (freed) pool pages and the new client's release writes
went to dead memory. Clear the learned sems/marks on each pid change.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
HMP x/gva2gpa translate with the selected vCPU's CR3; while that CPU runs
user code under PTI the kernel half is hidden, so kernel-VA reads (the
tracking-sem chain and the CE pushbuffer CPU alias) fail and the pin pass
aborts. Try the other vCPUs before giving up: during an active CUDA client
at least one CPU is in kernel mode, so the chain/pin work no longer require
the client to be parked in sleep().
|
| |
|
|
|
|
|
| |
The GR-window pin scanner matched comm=="cdt" exactly, so runs using the
cuda_drv_test2 binary (comm "cdt2") never streamed their mappings: the
GR push pages stayed unresolvable and the client stalled in context
creation. Match the cdt prefix instead.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The 256-byte cuMemcpyHtoD payload never appears in any CE push: CUDA
submits it on the compute channel as a compute-class DMA whose source is
the push's own inline data (LINE_LENGTH_IN=256, OFFSET_OUT=<device VA>,
LAUNCH_DMA=0x41, LOAD_INLINE_DATA 64 dwords, then the channel's
SET_REPORT_SEMAPHORE release). The CE-shaped decoder skipped the data run
(opcode 3) and no GR work ran at all.
- Decode NVC5C0 GRCE methods on subchannel 1 (LINE_LENGTH_IN, LINE_COUNT,
OFFSET_OUT/UPPER, LAUNCH_DMA) and the opcode-3 LOAD_INLINE_DATA run.
- svc_grce_inline_write resolves the destination VA through trusted
sources (pins, map, published page directories, targeted rm_mem
resolve), pins the page and writes into vidmem or guest RAM; budgeted
sema-grce logs name the target or report UNRESOLVED.
- cdt2 test binary gains CDT_CP_PAUSE / CDT_HP_PAUSE for live inspection
of the copy path; pushed by push_guest_tools.sh.
|
