summaryrefslogtreecommitdiffstats
path: root/tools/cuda-bridge/README.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
commitcc79b478843f0230fe6aa8ecd822de7ba71c891e (patch)
tree5de760ddec4e45cc23d39ce7502635f5c0711217 /tools/cuda-bridge/README.md
parent40351167dd209012ab485e6164ce94f0b8965422 (diff)
downloadopen-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.tar.gz
open-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.zip
vgsp: host CUDA bridge + sema release recovery; greeneye simulator wave
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean - tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests - tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode - debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding - STATE-CURRENT/docs updates
Diffstat (limited to 'tools/cuda-bridge/README.md')
-rw-r--r--tools/cuda-bridge/README.md104
1 files changed, 104 insertions, 0 deletions
diff --git a/tools/cuda-bridge/README.md b/tools/cuda-bridge/README.md
new file mode 100644
index 0000000..8ac0b6d
--- /dev/null
+++ b/tools/cuda-bridge/README.md
@@ -0,0 +1,104 @@
+# CUDA Bridge (host-side)
+
+Host daemon that executes guest CUDA kernels on the real physical GPU for
+Open-IOV. It is the execution backend for the endgame path: `vgsp.ko` detects a
+guest kernel launch, writes a **CUBB2 v2** job record to the debugfs job file,
+this daemon executes it with the CUDA driver API on the physical GPU and writes
+a **CBD2 v2** done record, and the module wakes the guest. The v1
+`CUBB`/`CUBD` packed/GPA mode remains for the offline tests.
+
+Host-only development tooling: it never touches the guest, QEMU, mdev,
+`vgsp.ko`, or the GPU reset path. It only maps the guest RAM file and talks to
+the NVIDIA driver.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `cuda_bridge.c` | daemon + v1/v2 protocol structs + bridge/self-test/bench/mutation modes |
+| `kernels.cu` | real kernels: `mail_kernel`, `vadd`, `vadd_mutated` |
+| `embed_ptx.py` | embeds nvcc-generated PTX into a C header |
+| `client_test.py` | external producer check of the v1 job/done transport |
+| `bridge_selftest.py` | offline plain-file self-test of `--bridge` (mail, vadd, stale seq, bad magic, failure, bounds) |
+| `check_module_layout.py` | mechanical CUBB2/CBD2 struct layout comparison vs `kernel/gsp_cuda_bridge.c` |
+| `Makefile` | build, test targets |
+| `PROTOCOL.md` | exact wire layout and semantics (v2 frozen; v1 appendix) |
+| `INTEGRATION.md` | how `vgsp.ko` uses it (paths, seq semantics, logging) |
+
+## Build
+
+```
+make # cuda_bridge
+make test # self-test on a fake 4 GiB RAM file
+make bench # timings (CUDA events + CPU baseline)
+make mutate-test
+```
+
+Kernel path: `nvcc` was available on the build host (CUDA 12.0,
+`/usr/bin/nvcc`), so the Makefile compiles `kernels.cu` to PTX
+(`-arch=sm_75`, PTX `.version 8.0`) and `embed_ptx.py` embeds it as a
+NUL-terminated byte array; `cuda_bridge` loads it with `cuModuleLoadData`, and
+the driver JITs it for Turing (CC 7.5). No PTX is loaded from disk at runtime
+and no CUDA toolkit is needed at runtime. If `nvcc` is unavailable on a build
+host, a pre-generated `kernels_ptx.h` can be copied in and the `cc` link step
+still succeeds. Hand-written PTX was therefore **not** needed.
+
+Requirements: `libcuda.so` (driver), `cc`, `nvcc` (build only), Python 3
+(embedding + `client_test.py`).
+
+## Run
+
+```
+# v2 daemon: poll the debugfs job file, execute, write out + done
+./cuda_bridge --bridge \
+ --job /sys/kernel/debug/vgsp/cuda_bridge_job \
+ --in /sys/kernel/debug/vgsp/cuda_bridge_in \
+ --out /sys/kernel/debug/vgsp/cuda_bridge_out \
+ --done /sys/kernel/debug/vgsp/cuda_bridge_done
+
+# v2 offline self-test with plain files (mail, vadd, stale seq, bad magic,
+# failure path, bounds)
+python3 bridge_selftest.py
+
+# v1 legacy daemon: poll the job file, execute, write the done file
+./cuda_bridge --job /path/job.bin --done /path/done.bin --ram /dev/shm/vgpu-guest-ram
+
+# v1 one-shot (tests / module bring-up)
+./cuda_bridge --job job.bin --done done.bin --ram ram.bin --once
+
+# fake-RAM self test: mail + vadd low GPA + vadd high GPA, byte-for-byte checks
+./cuda_bridge --test [--dir /tmp/opencode/cuda-bridge-test]
+
+# falsifiability: opcode 2 vs opcode 3 (+1 mutation) must give different results
+./cuda_bridge --mutation-test
+
+# timings and GPU utilization stress
+./cuda_bridge --bench [--big-iters N] [--verbose]
+
+# external transport check (starts/stops the daemon itself)
+python3 client_test.py
+```
+
+Every executed job prints one greppable line:
+
+```
+CUBB job seq=1000 op=2 n=1048576 gpu="NVIDIA GeForce RTX 2080 Super with Max-Q Design" kernel_ns=41567 status=ok checksum=392344090.500000 bad=0
+CUBB2 seq=42 op=2 n=1048576 gpu="NVIDIA GeForce RTX 2080 Super with Max-Q Design" kernel_ns=41919 status=ok checksum=392344090.500000 bad=0 out_bytes=4194304
+```
+
+`--test`, `--mutation-test` and `--bench` use sparse fake RAM files under
+`--dir` (default `/tmp/opencode/cuda-bridge-test`); they never touch
+`/dev/shm/vgpu-guest-ram`.
+
+## Evidence (2026-10-03)
+
+Under `open-iov/backups/evidence-20261003-bridge/`:
+
+| File | Content |
+|---|---|
+| `01-self-test.txt` | fake-RAM test: mail exact, vadd low + high GPA windows, bad=0, checksum exact |
+| `02-gpu-util.txt` | nvidia-smi @100 ms while 200 x n=2^24 vadd jobs run (0 % idle -> 27 % max, 298/308 samples > 0) |
+| `03-timings.txt` | CUDA-event kernel times + CPU baseline (n=2^20: 39.6 us vs 1.84 ms; n=2^24: 620 us vs 28.6 ms) |
+| `04-device.txt` | nvidia-smi identity and driver, plus the bridge's own DEVICE line |
+| `05-mutation.txt` | opcode 3 proves the real kernel ran: checksum +1048576, bad=1048576, sample c[0]=1.0 |
+| `06-daemon-transport.txt` | external producer, two seqs, done records parsed and verified |