| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
novel-only activity accounting (tools-fix a4840952)
|
| |
|
|
|
|
|
|
|
|
| |
(tools-fix c8fb5377)
Dead-pid purge via in-band #nopid (zombie-aware probe), monotonic
(pid,starttime) client identity so trace-ring replays can never reset an
active client (fixes the m178 mid-run reset regression avenue), pre-capture
and post-capture stall watchdogs that re-assert pins and escalate loudly,
heartbeat threads for both tools, supervisor STALE 75s/GRACE 120s.
|
| |
|
|
|
|
|
|
|
|
|
| |
stays
The m178 tool latching changes caused a reproducible pre-capture hang of the
third CUDA client after two green clients (r199: smiley #3; r200: multikernel
#3; both recovered by reverting tools only, r201). Prime suspect: semfix
per-client reset re-firing mid-run (same pid reset twice 2 min apart while the
client was active). Tools restored to the m177-proven state until the reset
logic is fixed with a monotonic per-pid guard.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Kernel: authorship instrumentation + swrel gate for in-flight GR release VAs;
first-poll shortening + fast extended re-arm (params gr_retry_fast=1,
gr_first_poll_ms=2500); per-attempt win0 bank recompute + shadow recopy in the
outcome-retry loop; poll-break/deposit-readback diagnostics; C1-C10 sema
diagnostics (crel-entry, relx, opdef-clear, walk, pub-adv, hold-fallback,
busy-defer, CAND-OVERWRITE, inline recs); busy-defer hold instead of
software-release consume; counter fast path; QMD word 48/49 skip.
Tools: heartbeat thread, scanner-alive probe on reinstall, full periodic
re-assert, #nopid handling, per-client reset fix (debounce), dead-pid drop,
supervisor grace+threshold.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(1) svc_session_flush keeps svc.direct_base (per-boot constant; the pin-free
targeted scanner's gate/input) and a new gsp_sema_service_guest_reset()
hook wipes boot-scoped discovery state on guest RM reload;
(2) counter-page derive now takes the first READABLE source (vaspaces ->
discovered -> targeted) instead of failing -EIO on a walked-but-garbage
page, with one-shot diagnostics;
(3) new m177c UVM ring-space directory discovery: content-verified
(push_va -> push_pa) samples from the GR content discovery and CE push
pairing feed a second discovered-directory slot (walks the 0x2xxxxxxx
family), consulted by svc_walk_discovered;
(4) GR channels now deliver withheld opdef sets at entry consume (normal
tail and the stale-skip path) instead of waiting for the 10 s sweep;
(5) host tools hardened: semfix_uvmt QMP reconnect + pid-generation dedupe
+ stale-sem eviction + post-success pin latch + heartbeats; pin_gr_stream
accept timeout + guest-scanner reinstall backoff + restart loop +
subprocess timeouts; new pin_supervisor.sh (heartbeat watchdog, capped
restarts) wired into start_vm_keep.sh.
Evidence: r194-r196 cycles; pool resolves src=trusted pin-free; ring dir
0x10801c000 found 4/4; row: 'rel-apply ... written' with tools up.
|
| |
|
|
| |
resolution; hold failed device-copy ops) + m164 CLASS-2 GR no-capture entry hold + m165 pool release applies at fresh-pin time + m166 timeout-ladder warm-sem fail-fast; scanner: first-class pool VA resolution + periodic re-assert + demand routing
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The one-shot #sem kprobe event that drives the CE push-buffer chain
resolution can be lost (trace rotation, scanner restart, slab kptr reuse),
and the chain resolver then never learns a client - every GPFIFO entry of
its CE channels is grace-skipped and the client spins on its tracker
semaphore (the copy-heavy stall class: streams2/memcpy/multikernel/saxpy
in r148; smiley as a later client in r147 took ~90 s to recover).
New demand channel, both ends here:
- pin_gr_stream: _demand_loop scans the host dmesg for 'stalled VA #N:
0x...' (the service logs each distinct unreadable push VA once), tracks
each sighting, and re-sends every 20 s for up to 5 min to
/tmp/guest_want_vas.txt in the guest (via the guest agent).
- guest_pin_stream: the main loop reads new VAs from that file and resolves
each by CONTENT against the nv_vm_map_pages vmap areas
(resolve_va_by_content: candidate leaf page at the VA's in-window offset,
+1/-1 neighbours for the header-page offset ambiguity, host-plausible
method-stream prefix). On a match the exact pair is emitted; only the
requested VA is pinned, so a wrong candidate cannot poison other VAs.
Also fix the scanner start: the kill-first pattern must be anchored
(^python3 /tmp/guest_pin_...) - the unanchored pattern matched the deploy
shell's own cmdline and every start attempt killed its own shell before
launching the scanner (r149: no scanner for the whole boot).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r148 ran THREE guest scanners (pin_gr_stream install_and_start retried
while a partially-failed first attempt had already started one, and the
pid-file kill missed it). Each scanner re-asserted the whole CE pair set
every 2 s; with a fully fragmented pair set (descending PAs, one page per
line) that is ~2 k debugfs writes/s per scanner, and the guest hit load 19
with clients wedged spinning in userspace.
- pin_gr_stream._start_cmd: kill EVERY existing scanner first (pattern
assembled at runtime with printf so the command's own cmdline cannot
self-match), then start exactly one.
- guest_pin_stream._emit_ce_mem_delta: the full re-assert keeps its 1.5 s
cadence for the first STREAM_FULLREM_FAST_S seconds of a mem's life (the
client-switch flush race), then backs off to STREAM_FULLREM_SLOW_S (10 s)
as a safety net. Configurable via env.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The host wipes its whole sema map when the previous client's last channel
closes ('session flush (all channels closed)'). A pin batch that raced in
for the new client (observed: the 1-page 0x120000000 pin emitted in the
same second as the flush) was lost, and the scanner's content-gated delta
never re-emitted the unchanged pairs - so the second client's CE pushes
were permanently unreadable (sema-probe 'NO ENTRY for page 0x120000000'),
every GPFIFO entry was grace-skipped (r144/r145 matmul and mail stalls),
and the guest spun in uvm_tracker_wait_for_entry.
Fix (scanner, guest side):
- _ce_mem_pages: walk the whole vmalloc area (minus header off), not the
size-derived prefix that stopped one page short of the write frontier.
- _emit_ce_mem_delta: full re-assert mode every KC_FULLREM_S (1.5 s) plus
every fresh_map pass; emits every translatable pair content-independent,
merged into contiguous runs. Zero-content pages are pinned on purpose:
the pin carries the translation, the host reads live content.
- missing pages retried with the negative PA cache popped (bounded 512/pass).
Fix (host tool):
- pin_gr_stream._flush: never dedupe single-page repeats of the CE window
(0x120000000..0x121000000) so the re-asserts land.
Live-validated in r145 (same-boot, no rebuild): mail green as 4th client,
matmul green as 2nd client (bad=0, 64-changed acceptance of 857fe5f9),
both with zero 'stalled VA'/'skipping stale entry' lines; pre-fix the same
sequences stalled. Headless smiley all-wrong (bad=131072) reproduced
independently and is the next target (engine-side constant class).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
scrub freeze)
guest_pin_stream: find the scrubber channel buffer among the nvidia vmap
areas (size in the CeUtils channel set, 906F ring shape at +0x64000 when
non-empty), walk its pages with kcore and emit '#scrub <fin_pa>,<hsem_pa>'
(the finish payload at +0x6c004 and the host semaphore at +0x6c000).
pin_gr_stream: handle '#scrub' lines and register the PAs through the
sema_scrubbuf_add module parameter, so the kernel's svc_scrub_pass can
author the guest's scrub progress.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Sweep of the 155 tracked C/Python/shell files under userspace/ (excluding
the MIT-licensed vendored userspace/extern/), tools/ and scripts/ that had
no per-file license identifier. Each gains the minimal two-line header in
the area's existing form: 'SPDX-License-Identifier: BSD-3-Clause' plus
'Copyright (c) 2026 Open-IOV contributors'; shebang and coding lines stay
first. No other content touched. kernel/** is deliberately out of scope
(recorded as a follow-up in CODESTD-REPORT-2026-10-08.md): its SPDX
coverage and MODULE_LICENSE normalization need their own GPL-2.0 pass.
|
| |
|
|
|
|
|
|
| |
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean
- tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests
- tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode
- debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding
- STATE-CURRENT/docs updates
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registry variant; complete vgsp rename; docs/tooling refresh
- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA,
batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel,
old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only,
PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM
geometry queries and buffers; pre-boot SET_REGISTRY profile selection
- gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf
parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0
- vgsp rename across kernel, scripts, tools and docs (module vgsp.ko,
/sys/kernel/debug/vgsp/, vgsp-<subsystem>: log prefixes)
- risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag
tool updates; README/SBOM/install/Makefile sync
Live status: CE ladder and P6 guest-copy forwarding green; GR engine context
commit still open (all promote shapes accepted or cleanly refused; the channel
stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).
|
| |
|
|
| |
failure left a boot without pins and the client hung)
|
| |
|
|
|
|
|
|
|
|
| |
The first CE channel's push-buffer was only pinned ~1 minute after the
client's failure: uvbp fired at t+4s but semfix's trace read (guest agent,
every 5 loops) picked it up at t+70s, so the tracker wait for value 3 spun
for 16 s. The guest scanner now emits '#pb 0x<kptr>' from the uvbp trace
lines as soon as they appear; the host mirrors them into the sem stream and
semfix resolves/pins the push-buffer through the UVM chain on the next loop
(sub-second).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
queue 4k
- counter_pa autoreg result is written back to the channel table so later
passes cannot re-target a different slot when the live GR count changes
during teardown; re-registration of the same channel preserves the slot.
- counter slot layout uses the max GR count ever seen in the generation
instead of the shrinking live count.
- sema_map_add queue 64 -> 4096 entries so a full-window pin burst drains
without ENOSPC retries.
- pin_gr_stream: one batched sudo pass per flush (was one process per run,
~4 s for 400 runs), retrying only queue-rejected runs.
- semfix_uvmt: SEMFIX_SKIP_PB=1 skips the blocking push-buffer chain walk
when the vsock window stream already supplies those pages.
|
| |
|
|
| |
GR GPFIFO rings)
|
| |
|
|
| |
last release VA; larger push log budget
|
| |
|
|
| |
corruption); tooling guards
|
|
|
The guest-agent exec path is too slow under driver-init load (a 15k-page
window scan took 17-35 s to complete), so GR window pins landed after the
channels' entries had been consumed and after the client's cuCtxCreate
timeout. The guest now scans /proc/<pid>/pagemap locally and streams only
delta runs over vsock (no agent on the hot path); the host listener pins
them as they arrive. The scanner writes a pidfile so restarts do not kill
the new instance.
|
