open-iov.git/tools/guest-diag, branch master

guest-diag: semfix_watch read64 uncached by default (stale-cache pool-pin failure)

2026-10-12T01:53:46+00:00

The watcher's resolve_sem_pool read sem->page->mem->vas through a process-lifetime cache: a transient zero captured while UVM was still initialising the structures was served forever, the pool pin never landed, and the second-session client hung in cudaMallocManaged (r88 evidence: 'pool unresolved (retry)' from the watcher while the same walk succeeded from a fresh QMP client). Cache is now opt-in.

guest-diag: scanner EOF guard + exclude non-client nvidia fd holders

2026-10-12T01:17:55+00:00

Two failure modes found when the scanner detects clients by device fd: a transient boot nvidia-smi was picked and its exited pagemap read back EOF, and scan_window advanced by got=0 forever (busy spin, no stream); and non-client nvidia fd holders must never be committed to. Break on got<=0 in scan_window (scan_range already did), skip nvidia-smi/ persistenced/Xorg, and prefer /dev/nvidia-uvm holders.

guest-diag: scanner detects arbitrary CUDA clients by nvidia fds

2026-10-12T01:08:05+00:00

The guest pin scanner only tracked processes whose comm matched CLIENT_PREFIXES (cdt,va,victory). An arbitrary CUDA binary (e.g. /tmp/smiley) was invisible: no pages streamed, no VA->PA pins injected, the service could not read its pushes and the client hung in cudaMalloc/uvm_tracker_wait. Detect compute clients by the CUDA device they hold open (/dev/nvidia-uvm preferred, then any /dev/nvidia*), with the prefix match kept as the first-choice path.

tools/guest-diag: single user netdev for QEMU (fix SSH pre-auth stall)

2026-10-12T00:33:05+00:00

The launch line carried both an explicit rtl8139 on netdev net0 and a stray '-net user,hostfwd=tcp::2222-:22 -net nic' pair, creating a SECOND slirp network with a second NIC and a second 2222 forward. The SSH connection landed on one slirp instance while the guest's replies left through the other interface, stalling the handshake before auth. The explicit rtl8139 now owns the only user netdev with the 2222 forward. The generated launch_qemu_rootport.run.sh is patched in place by the harness from the same line. Verified: sshpass -p ubuntu ssh -p 2222 ubuntu@127.0.0.1 -> nvidia-smi green. Guest-side companion fix (persisted in the qcow2 image): the cloud image's /etc/ssh/sshd_config.d/60-cloudimg-settings.conf set PasswordAuthentication no; it is now yes.

vgsp-sema+guest-diag: fix CE USERD discovery for re-registered sessions (ISSUE-0002)

2026-10-09T18:45:12+00:00

svc_chan_userd now prefers the layout-derived CE USERD (ring_pa + 0x10000 in the ring aperture) and accepts it while idle, so a second session cannot latch onto the registered pool-relative offset when that alias holds a DIFFERENT rank's stale USERD: r40 c2 rank0 read rank3's stale 1/1 at FB 0xf0000 and the pump never followed rank0's own GP_PUT (guest GPUT mapping for gpFifoGpuVa proves ring+0x10000). guest_pin_stream.va_to_pa drops translations outside the registered guest-RAM windows so BAR1-phys addresses (0x70000xxxxx) are no longer injected as SYS pins; r40 rank2's ring resolved to the bogus pa=0x70000c0000 ap=SYS and read 66/66.

guest-diag: accept both GPFIFO ring windows and validate channel_info for #put (ISSUE-0002 diag)

2026-10-09T18:23:05+00:00

CE channels register gpfifo=0x121010000+0x30000*rank, outside the GR-only window [0x200200000,0x200400000) the #put resolver searched, so no #put line ever fired for the CE channels the second session stalls on. Resolve channel_info from both windows with layout validation, keep re-resolving the channel after the pool mem has resolved, reset the #put map on client switch, and add bounded resolver diagnostics to the guest log.

guest-diag: stream channel CPU-side GPUT on change (#put lines, ISSUE-0002 diag)

2026-10-09T18:07:11+00:00

spdx: add BSD-3-Clause identifiers to userspace/tools/scripts sources

2026-10-08T15:53:20+00:00

Sweep of the 155 tracked C/Python/shell files under userspace/ (excluding the MIT-licensed vendored userspace/extern/), tools/ and scripts/ that had no per-file license identifier. Each gains the minimal two-line header in the area's existing form: 'SPDX-License-Identifier: BSD-3-Clause' plus 'Copyright (c) 2026 Open-IOV contributors'; shebang and coding lines stay first. No other content touched. kernel/** is deliberately out of scope (recorded as a follow-up in CODESTD-REPORT-2026-10-08.md): its SPDX coverage and MODULE_LICENSE normalization need their own GPL-2.0 pass.

vgsp-rm: harden the boot-time queue link against the 4 s GspStatusQueueInit window

2026-10-08T13:38:39+00:00

The guest's first RmInitAdapter intermittently fails: GspStatusQueueInit gives the status-queue link only 4 s (message_queue_cpu.c: timeoutUs = 4000000) and the first boot attempt can burn it, after which the RM tears down, leaks the device and retries; the surviving retry leaves the GR/ctx state degraded, which is the persistent cudaMalloc stall class. Changes: - gsp_rm_emu_poll: parse-retry cadence 250 ms -> 50 ms (the 250 ms tick consumed a fifth of the guest's budget per attempt). - gsp_rm_emu_reset: clear last_parse_retry so a new boot gets a fresh retry budget; take stat_lock while clearing queue cursors so a sema event post cannot race the reset. - gsp_rm_emu_on_booter_done: on GSP_INIT_DONE send failure re-arm the retry path (parsed/active were already set, so the poll loop would never retry). - gsp_rm_parse_queue_chain: free the previous ptes before re-alloc so retries and post-reset re-parses do not leak. - gsp_rm_post_channel_event: never post events across a queue reset (parsed gate). guest_kprobes.sh: add rpcs/rpcp transport probes (the direct RPC-wait symbol is notrace in 570.144) and ccset entry/return probes for rmapiControlCacheSet so a stall names the control whose reply breaks the guest's control-cache bookkeeping.

vgsp+guest-diag: kcore-truth pin resolver (pool/pushbuffer/channel ring+userd) + GR ring userd-derived candidates

2026-10-08T08:15:24+00:00

guest_pin_stream.py now resolves translations entirely from guest truth via /proc/kcore + a manual page-table walk (no QMP races): the CE semaphore pool and pushbuffer memory from the UVM rm_mem structures, and each channel's GPFIFO ring + GPPUT from the tracking semaphore's embedding uvm_channel_t (channel_info located by its gpFifoGpuVa value). This removes the watcher pipeline's flakiness on the cudaMalloc stall class (missing pool/pushbuffer pins). gsp_sema_service.c: svc_gr_ring_autopin tries the guest-registered USERD memdesc base as a ring candidate (the fixed FB convention only matches when the guest's VRAM allocator happens to reproduce the old layout), plus an optional (gsp_sema_dbg) one-shot FB scan + raw-word diagnostics when every candidate fails. tools/cuda-ab/p2: self-checking arbitrary-kernel acceptance suite (10 programs: sequential launches, shapes, params, smem/GEMM, streams, malloc churn, printf+error, FMA benchmark, mixed, managed probe) with host oracle expectations and a guest runner.