diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-24 10:48:08 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-24 10:48:08 -0400 |
| commit | 93431bdd896d9e6d9a25bebef0fb1af308dd64cd (patch) | |
| tree | 2c6b44e8b3fca4edd24a6a904e370a5c3e899d22 /tools/guest-diag/bridge_sems.sh | |
| parent | 775c61c34fdccb32179d0cb6239db62b0b42e867 (diff) | |
| download | open-iov-93431bdd896d9e6d9a25bebef0fb1af308dd64cd.tar.gz open-iov-93431bdd896d9e6d9a25bebef0fb1af308dd64cd.zip | |
kernel: enlarge the pin table and its probe window; tools: bridge client mappings into the service
The pin table became the primary VA->PA store once push-buffer regions and
bridged client mappings started adding thousands of pages per boot; at
8192 slots with an 8-probe window, inserts were silently dropped and
lookups missed at half load. 32768 slots and a 32-probe window keep the
mapped set intact.
tools/guest-diag/{guest_pmap_all.py,bridge_sems.sh} translate the CUDA
client's /dev/nvidia* mappings through the guest's own pagemap and inject
each present page into the service as a sysmem pin. With the client's
semaphore pages pinned the service applies their releases into the page
the client polls (verified: applied 67 -> 109, misses stop growing for
those VAs), where before every write went to the vidmem shadow.
Diffstat (limited to 'tools/guest-diag/bridge_sems.sh')
| -rwxr-xr-x | tools/guest-diag/bridge_sems.sh | 42 |
1 files changed, 42 insertions, 0 deletions
diff --git a/tools/guest-diag/bridge_sems.sh b/tools/guest-diag/bridge_sems.sh new file mode 100755 index 0000000..067a9ee --- /dev/null +++ b/tools/guest-diag/bridge_sems.sh @@ -0,0 +1,42 @@ +#!/bin/bash +# bridge_sems.sh - bridge the guest's own page table view into the host service. +# +# The emulation cannot (yet) translate the CUDA client's sysmem semaphores +# (guest RAM pages mapped at VAs above the vidmem window base), so releases +# for them are counted as misses. This helper translates the client's +# /dev/nvidia* mappings through the guest's /proc/<pid>/pagemap (the guest +# kernel owns the mapping) and injects each present page into the service as +# a sysmem pin. +# +# Usage: tools/guest-diag/bridge_sems.sh [iterations] [prefix-hex] +set -u +cd "$(dirname "$0")/../.." + +ITER=${1:-1} +START=${2:-0} +VMEXEC=./scripts/vgpu_vm.py + +for i in $(seq 1 "$ITER"); do + pid=$(timeout 20 $VMEXEC exec --timeout 15 'pgrep -x cdt | head -1' 2>/dev/null | tail -1) + [ -n "$pid" ] || { echo "no cdt pid"; exit 1; } + + b64=$(base64 -w0 tools/guest-diag/guest_pmap_all.py) + timeout 30 $VMEXEC exec --timeout 25 "echo '$b64' | base64 -d > /tmp/pmapall.py" >/dev/null 2>&1 + + cursor=$START + total=0 + for chunk in $(seq 1 40); do + mapfile -t lines < <(timeout 60 $VMEXEC exec --timeout 50 "sudo python3 /tmp/pmapall.py $pid $cursor 100" 2>/dev/null | grep -aE '^0x[0-9a-f]+,0x[0-9a-f]+$') + [ "${#lines[@]}" -gt 0 ] || break + for l in "${lines[@]}"; do + va=${l%%,*} + pa=${l##*,} + echo "$va,$pa,0x1000,1" | sudo tee /sys/module/open_iov_vfio/parameters/sema_map_add >/dev/null + done + total=$((total + ${#lines[@]})) + last_va=${lines[-1]%%,*} + cursor=$(printf '0x%x' $(( 16#${last_va#0x} + 0x1000 ))) + done + echo "iter $i: pinned $total pages (pid $pid)" + sleep 2 +done
\ No newline at end of file |
