From 7fdbe8713c8cf746cd1a15e93e0de39d333905c7 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Wed, 23 Sep 2026 06:31:44 -0400 Subject: kernel: execute pushed CE work (PTE memsets, copies) and track semaphores The progress service now decodes push buffers as method streams and performs the work the driver pushes, not just the tracking semaphore: - C5B5 method decoder: SET_REMAP_CONST_A/B + COMPONENTS + OFFSET_OUT + LINE_LENGTH_IN + LAUNCH_DMA = memset (into vidmem or guest RAM); OFFSET_IN + OFFSET_OUT + LAUNCH_DMA = copy; SET_SEMAPHORE_A/B/PAYLOAD + LAUNCH_DMA = release. Contiguous method runs (count 2/4) are applied in full - OFFSET_IN/OUT arrive as one run. - Pushes span page boundaries, so pages are decoded as method streams without a start marker. Collected operations execute only when the push's release is reached and still pending (payload ahead of the semaphore), so recycled buffers replaying old pushes are skipped and their stale memsets cannot overwrite newer state. - VA->PA map is first-wins: stale copies of freed allocations and unrelated adjacent u64 pairs produced bogus entries that hijacked a good mapping and sent semaphore writes to the wrong page. - dump_guest module parameter (hex dump of guest PA, kthread via completion; kthread_stop before the thread runs loses the work). With this the guest no longer stalls in uvm_tracker_wait and no longer oopses in the UVM path; the failure moved to a userspace segfault inside libcuda (NULL-0x38) after the driver's page allocations. Also add tools/cuda-ab: run one deterministic CUDA workload on the host (bare metal oracle) and in the guest, then compare the stable result lines. The binary links the CUDA runtime statically so the guest only needs the driver's libcuda.so. --- tools/cuda-ab/README.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 tools/cuda-ab/README.md (limited to 'tools/cuda-ab/README.md') diff --git a/tools/cuda-ab/README.md b/tools/cuda-ab/README.md new file mode 100644 index 0000000..db87149 --- /dev/null +++ b/tools/cuda-ab/README.md @@ -0,0 +1,32 @@ +# CUDA A/B harness + +Run one deterministic CUDA workload on the **host** (bare-metal GPU, the +oracle) and in the **guest** (Open-IOV emulated vGPU), then compare the +machine-readable results. This mirrors the earlier host/guest `nvidia-smi` +comparison, but exercises actual execution: allocation, HtoD/DtoH copies, +kernel launch, synchronisation and verification. + +``` +tools/cuda-ab/ab.sh all # build + host run + guest run + compare +tools/cuda-ab/ab.sh host # host only -> run/host.txt +tools/cuda-ab/ab.sh guest # push binary, run in the running VM -> run/guest.txt +tools/cuda-ab/ab.sh compare # diff the stable result lines +``` + +## Files + +| File | Purpose | +|---|---| +| `vector_add.cu` | the workload (`DEVICE`/`RESULT`/`STATUS` output lines) | +| `ab.sh` | host build/run, guest push/run, comparison | +| `run/` | generated outputs (`host.txt`, `guest.txt`) | + +## Notes + +* The binary links the CUDA runtime **statically** (`--cudart=static`), so the + guest needs no CUDA toolkit - only the guest driver's `libcuda.so`. +* The guest transfer uses the guest agent (base64 chunks); the VM must be + running (`scripts/vgpu_vm.py`). +* `DEVICE` lines are normalised before diffing (the guest reports the + emulated GPU name); `RESULT` and `STATUS` must match exactly. +* Host requirement: `nvidia-cuda-toolkit` (`nvcc`). -- cgit v1.2.3