diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 04:15:24 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 04:15:24 -0400 |
| commit | 5bb2d53ec14f0d802726bc879b994c0286f85415 (patch) | |
| tree | a6b75f0fd7ab8dc21420041d282ea61049797713 /tools/cuda-ab/p2/README.md | |
| parent | 11820d73e35511a8dfe25c65ee2f81970f7ac757 (diff) | |
| download | open-iov-5bb2d53ec14f0d802726bc879b994c0286f85415.tar.gz open-iov-5bb2d53ec14f0d802726bc879b994c0286f85415.zip | |
vgsp+guest-diag: kcore-truth pin resolver (pool/pushbuffer/channel ring+userd) + GR ring userd-derived candidates
guest_pin_stream.py now resolves translations entirely from guest truth via
/proc/kcore + a manual page-table walk (no QMP races): the CE semaphore pool
and pushbuffer memory from the UVM rm_mem structures, and each channel's
GPFIFO ring + GPPUT from the tracking semaphore's embedding uvm_channel_t
(channel_info located by its gpFifoGpuVa value). This removes the watcher
pipeline's flakiness on the cudaMalloc stall class (missing pool/pushbuffer
pins).
gsp_sema_service.c: svc_gr_ring_autopin tries the guest-registered USERD
memdesc base as a ring candidate (the fixed FB convention only matches when
the guest's VRAM allocator happens to reproduce the old layout), plus an
optional (gsp_sema_dbg) one-shot FB scan + raw-word diagnostics when every
candidate fails.
tools/cuda-ab/p2: self-checking arbitrary-kernel acceptance suite (10
programs: sequential launches, shapes, params, smem/GEMM, streams, malloc
churn, printf+error, FMA benchmark, mixed, managed probe) with host oracle
expectations and a guest runner.
Diffstat (limited to 'tools/cuda-ab/p2/README.md')
| -rw-r--r-- | tools/cuda-ab/p2/README.md | 102 |
1 files changed, 102 insertions, 0 deletions
diff --git a/tools/cuda-ab/p2/README.md b/tools/cuda-ab/p2/README.md new file mode 100644 index 0000000..da4e896 --- /dev/null +++ b/tools/cuda-ab/p2/README.md @@ -0,0 +1,102 @@ +# P2 - "arbitrary CUDA kernels" acceptance suite + +A reusable, self-checking acceptance suite for the Open-IOV emulated vGPU. +It extends the single-workload `ab.sh` A/B harness (`victory_mail`, +`vector_add`) to ten independent programs that exercise launch shape, kernel +parameters, shared memory, streams, allocator churn, device printf, error +reporting, a compute-heavy benchmark, and a combined program - each of which +verifies its own results exactly and prints machine-checkable lines. + +``` +p2/ +├── build.sh # nvcc -O2 -arch=sm_75 --cudart=static -> bin/ +├── oracle.sh # host reference run (GPU) or deterministic emit fallback +├── run_guest.sh # push to guest, run with timeout, diff vs expected/, table +├── src/ # common.h + 10 .cu programs +├── bin/ # built binaries (rebuild with ./build.sh) +├── expected/ # <name>.txt stable lines (+ <name>.full.txt, SOURCE.txt) +└── guest/ # per-run guest output (created by run_guest.sh) +``` + +## Programs + +| Program | Covers | Device footprint | +|---|---|---| +| `p2_seq5` | 5 sequential launches in one process on the same buffers, state verified after each | 3 x 4 KB | +| `p2_shapes` | 1x1x1, odd sizes (1013, block 33), non-multiple n (1000/64), 2D grid (40x20), 3D grid (24x8x8) with bounds checks | 8 KB buffer | +| `p2_params` | kernel parameter counts 0..12, types int/float/double/device-pointer | 64 B + 8 B + 4 B | +| `p2_smem` | shared-memory tree reduction (4096 -> 1) and 16x16-tiled 32x32 int GEMM | peak 4 pages | +| `p2_streams` | 3 streams x 2 kernels, per-stream events and result verification | 3 x 4 KB | +| `p2_churn` | 14 cudaMalloc/cudaFree across 6 iterations plus a final fresh alloc | peak 32 KB | +| `p2_printf_err` | device printf (5 deterministic lines) + invalid grid/block launch error reporting + recovery | 4 B | +| `p2_bench` | 8192 threads x 250k correctly-rounded FMAs; fixed verifiable value; TIMING ms/GFLOP/s | 32 KB | +| `p2_mixed` | mail + vadd + int reduction + 2D grid + 2 streams in one process | peak 96 KB | +| `p2_managed_probe` | **expected-failure probe**: cudaMallocManaged outcome recorded, then normal alloc/launch must still work | 4 KB (+4 KB) | + +All arithmetic is exact: integer kernels, fp32 add/mul of small integers +(element comparisons are bit-exact), and correctly-rounded `__fmaf_rn` / +`fmaf`, so `bad=0` is exact and no tolerances are used. + +## Output conventions + +* Compared lines (filtered, sorted): `RESULT`, `CHECK`, `ERR`, `STATUS`, + `P2PRINT`, `PROBE`, `MAIL`. +* Ignored lines: `DEVICE`, `INFO`, `TIMING`, `MEMINFO`. +* Every strict program ends with `STATUS OK` and exits 0; any CUDA failure + prints `ERR cuda=...` + `STATUS FAIL`. + +## Usage + +```bash +./build.sh # build all 10 programs (sm_75, static cudart) + +./oracle.sh # host references -> expected/ + # GPU mode if nvidia-smi works, else + # deterministic --emit-expected fallback + +./run_guest.sh # push + run in the running guest, then + # per-program PASS/FAIL table (exit != 0 on FAIL) +./run_guest.sh p2_seq5 p2_smem # subset +P2_GUEST_TIMEOUT=600 ./run_guest.sh # longer timeout (default 300 s) +``` + +`run_guest.sh` requires the guest to be up (`scripts/vgpu_vm.py exec` must +answer) and `expected/` to exist. It pushes binaries with the same base64 +chunking method as `ab.sh` and never modifies the guest beyond `/tmp/p2_*`. + +## Reference outputs (`expected/`) + +`expected/<name>.txt` holds the stable lines the guest must reproduce; +`expected/<name>.full.txt` keeps the raw oracle output for humans. +`expected/SOURCE.txt` records how the current references were produced. + +* **GPU mode**: `host` runs on bare-metal; full `RUN output` captured. +* **EMIT mode**: when no usable host GPU is available (e.g. the campaign VM + owns the device, `nvidia-smi`/NVML fails), each program is run with + `--emit-expected`, a documented deterministic CPU computation of the + stable lines a correct GPU run must produce. Re-run `./oracle.sh` when a + host GPU is free to replace emit references with captured ones. + +The current `expected/` was generated in **EMIT mode** (see +`expected/SOURCE.txt`) because the host GPU was claimed by the running +guest at build time. + +## Bounce-window budget + +The emulated GR path relocates guest data pointers into bounce windows +(100 pages total, 112-page win0 shadow, 32 windows, ~33 pages per pointer +observed working). This suite stays well inside those caps: largest single +pointer 32 KB (8 pages), largest simultaneous footprint 96 KB (24 pages, +`p2_mixed` vadd phase), total live allocations per program <= 4. No managed +memory is used except in `p2_managed_probe`, which is informational. + +## Special cases + +* `p2_managed_probe` is the one deliberate expected-failure program: UVM + managed memory is a known unsupported path. `run_guest.sh` marks it + `INFO` and does not diff its content, but requires it to finish with + `STATUS OK` (clean managed-alloc failure, normal alloc still working). A + hang is caught by the timeout and fails the suite. +* `p2_printf_err` can legitimately be the most fragile test if the emulated + path does not support device printf buffering; a timeout/`FAIL` there is a + finding, not a harness bug. |
