open-iov.git/tools/guest-diag/semfix_watch.py, branch master

guest-diag: semfix_watch read64 uncached by default (stale-cache pool-pin failure)

2026-10-12T01:53:46+00:00

The watcher's resolve_sem_pool read sem->page->mem->vas through a process-lifetime cache: a transient zero captured while UVM was still initialising the structures was served forever, the pool pin never landed, and the second-session client hung in cudaMallocManaged (r88 evidence: 'pool unresolved (retry)' from the watcher while the same walk succeeded from a fresh QMP client). Cache is now opt-in.

spdx: add BSD-3-Clause identifiers to userspace/tools/scripts sources

2026-10-08T15:53:20+00:00

Sweep of the 155 tracked C/Python/shell files under userspace/ (excluding the MIT-licensed vendored userspace/extern/), tools/ and scripts/ that had no per-file license identifier. Each gains the minimal two-line header in the area's existing form: 'SPDX-License-Identifier: BSD-3-Clause' plus 'Copyright (c) 2026 Open-IOV contributors'; shebang and coding lines stay first. No other content touched. kernel/** is deliberately out of scope (recorded as a follow-up in CODESTD-REPORT-2026-10-08.md): its SPDX coverage and MODULE_LICENSE normalization need their own GPL-2.0 pass.

tools: arm UVM kprobes deterministically (modprobe+wait+verify) and retry pushbuffer pins

2026-10-08T04:01:54+00:00

Root cause of the cudaMalloc/UVM-init stall on our bring-up path: guest_kprobes.sh ran while nvidia_uvm was not yet loaded (it loads lazily with the first CUDA client), so every probe add failed silently (symbols not in kallsyms) and the host never got the #sem -> pushbuffer-pin pipeline. The CE push page (0x120021db8) then never translated and the guest spun forever at DEVICE. - guest_kprobes.sh: modprobe nvidia_uvm, wait (bounded 30 s) for the uvmt symbol, install, and print KP_ARMED/KP_FAIL for bring-up checks. - semfix_watch.pin_pushbuffer: retry transient/partial gva2gpa misses in-call (bounded passes with a short sleep) instead of aborting on the first all-miss block; report unresolved page counts. Validated: 2/2 cold boots (vmkeep6/7) arm the probes and run victory_mail green in 15-20 s (MAIL + VADD bad=0 checksum=12204516 + STATUS OK, oracle-matched); pre-fix boots with unarmed probes stalled 4/4.

kernel: GR context-promote bisection modes 4-17 and server-managed-ctxbuf registry variant; complete vgsp rename; docs/tooling refresh

2026-10-02T19:13:06+00:00

- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA, batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel, old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only, PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM geometry queries and buffers; pre-boot SET_REGISTRY profile selection - gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0 - vgsp rename across kernel, scripts, tools and docs (module vgsp.ko, /sys/kernel/debug/vgsp/, vgsp-: log prefixes) - risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag tool updates; README/SBOM/install/Makefile sync Live status: CE ladder and P6 guest-copy forwarding green; GR engine context commit still open (all promote shapes accepted or cleanly refused; the channel stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).

guest-diag: cycle monitor CPUs on kernel-VA translation failure

2026-09-30T13:32:48+00:00

HMP x/gva2gpa translate with the selected vCPU's CR3; while that CPU runs user code under PTI the kernel half is hidden, so kernel-VA reads (the tracking-sem chain and the CE pushbuffer CPU alias) fail and the pin pass aborts. Try the other vCPUs before giving up: during an active CUDA client at least one CPU is in kernel mode, so the chain/pin work no longer require the client to be parked in sleep().

guest-diag: pushbuffer pin survives PTI holes (full-range scan + bounded retries until complete)

2026-09-30T09:28:13+00:00

sema-svc: apply releases before push memsets; poller: bypass read cache

2026-09-29T19:34:18+00:00

guest-diag: print tracking-sem state (index, queued, payload) on change

2026-09-29T19:08:39+00:00

The final cuCtxCreate stall is a UVM tracking semaphore whose completed value stays 0; the pool payload is the ground truth. semfix now prints idx/+0x10/+0x28/payload for every tracking sem it sees, whenever a field changes, so the last stall before a failure is visible in its log.

guest-diag: cheaper trace read, optional push sampler, bigger pagemap batches

2026-09-29T18:47:17+00:00

Under load the semfix poll grepped the whole (growing) trace file and the track_waits push block spawned three guest processes per sample, both of which slowed the pin path into the client's failure window. The trace read now tails 128 KB; track_waits.sh gates the push dump behind PUSH=0; the pagemap helper reads 32 KB chunks.

kernel: batch the map-add channel rewind; trust injected streams for ops

2026-09-29T18:25:05+00:00

Two fixes for the CE data path. (1) svc_handle_map_add rewound every channel by 8 ring entries on every injected mapping; a 16 MB push-buffer pin is thousands of page injections, so the pump's last_get was pinned in place and it re-read the same entries for the whole batch (3645 submissions, zero ops). The rewind now runs once per drained batch. (2) The push-start heuristic (op=INC_METHOD at address 0) rejects UVM's real CE memop pushes, which start with an IMM method at address 8 (observed 0x60020002, PTE data words) or a release method at 0x700 (0x200381c0); a buffer read through a host-injected (ipin) translation is now decoded fully trusted, ops included. Result on the next boot: memset=2718/0/0 ringops=1794 (first sustained CE work), false 12k->1.9k. semfix_watch: bulk-pin contiguous runs via sema_map_add len, read rm_mem size, 16 MB (4096 page) cap, non-blocking QMP read (the fixed 30 ms sleep made 4096 gva2gpa calls take two minutes), and KEEP_TRACE=1 to preserve the guest trace for post-mortems.