diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-23 12:11:04 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-23 12:11:04 -0400 |
| commit | 21b8ebc9db1d0ae649d6406087ea22b54e3b5775 (patch) | |
| tree | 192aa5d3c76e0cf3a785936d9902c6848609d37e /userspace/cuda/docs/07-version-upgrade-playbook.md | |
| parent | 631bd20c63a4058775a4fa549b9473915c918020 (diff) | |
| download | open-iov-21b8ebc9db1d0ae649d6406087ea22b54e3b5775.tar.gz open-iov-21b8ebc9db1d0ae649d6406087ea22b54e3b5775.zip | |
userspace/cuda: add CUDA userspace driver analysis, tooling and name maps
Add the analysis of the NVIDIA CUDA userspace driver used by Open-IOV VMs:
* docs/01-08: architecture (layers, object model, threads, sync, memory,
channels, launch), build and execution model, the driver/kernel/GPU
interface contract, virtualization approaches, the 570.144 decompilation
map with the corrected libcuda crash path and the current engine-topology
blocker, tooling, and a version-upgrade playbook.
* tools/: version-agnostic toolchain - stdlib ELF/interface toolkit
(exports, unwind-table function map, wrapper resolution, version delta,
crash address decode), callgraph walker, corpus annotator, and Ghidra
analysis/decompilation/name-map scripts with resumable parallel drivers.
* 570.144/annotations/: our name map (exports + resolved wrappers + verified
internals) and subsystem index, tracked so naming work accumulates.
* READ-ME-FIRST.md: reading guide with the fast lookup recipes.
The full decompiled corpus and annotated corpus stay untracked (vendor-
derived, gitignored under the version run/ tree).
Diffstat (limited to 'userspace/cuda/docs/07-version-upgrade-playbook.md')
| -rw-r--r-- | userspace/cuda/docs/07-version-upgrade-playbook.md | 98 |
1 files changed, 98 insertions, 0 deletions
diff --git a/userspace/cuda/docs/07-version-upgrade-playbook.md b/userspace/cuda/docs/07-version-upgrade-playbook.md new file mode 100644 index 0000000..9e53514 --- /dev/null +++ b/userspace/cuda/docs/07-version-upgrade-playbook.md @@ -0,0 +1,98 @@ +# 07 — Version-upgrade playbook: re-running the CUDA analysis + +When the guest driver, CUDA runtime, GSP firmware or GPU generation changes, +this is the procedure that refreshes the analysis with minimal effort and +keeps the name map and interface knowledge accumulating. + +## 0. Before you start + +* You need the new `libcuda.so.<version>` (from the guest with + `570.144/scripts/fetch_from_guest.sh`, or from the driver package with + `--extract-only`). +* Work in a new version directory `userspace/cuda/<version>/` — never + overwrite an analysed version. Copy the structure from `570.144/` + (`scripts/`, `ghidra/`, `annotations/`) and create `run/`. +* Update `WORKING-HERE.md` so parallel agents know which tree is being + analysed. + +## 1. Cheap triage (minutes) + +``` +tools/cudatool.py info <lib> +tools/cudatool.py delta <previous_lib> <lib> > run/delta.txt +tools/cudatool.py crash <lib> <map_base> <ip> # if a guest crash is known +``` + +Read `delta.txt`: + +* **no export moves, small text growth** → patch release: most names and + offsets from the previous version transfer; spot-check the subsystems + touched by the release notes. +* **new exports** → feature release: identify the new API surface, map it to + engine functions via decompilation of the new stubs. +* **many moves** → reshuffle (toolchain/linker change): plan a full re-analysis + and re-apply the name map, expecting to fix a fraction. + +## 2. Full analysis (tens of minutes) + +``` +tools/ghidra_full_analysis.sh <lib> <version_dir>/run/ghidra-full cuda-full +tools/ghidra_export_index.sh <version_dir>/run/ghidra-full cuda-full <program> <version_dir>/run/analysis/elf-index +tools/ghidra_batch_decompile.sh <version_dir>/run/ghidra-full cuda-full <program> <version_dir>/run/analysis/decompiled \ + <text_start> <text_end> 0x100000 5 120 +tools/ghidra_index_merge.sh <version_dir>/run/analysis/decompiled +``` + +Get `<text_start>`/`<text_end>` from `cudatool.py info`. Confirm the image base +with `PrintProgramInfo.java` before citing addresses. + +## 3. Transfer names and knowledge + +``` +cp <previous>/annotations/names.tsv <new>/annotations/names.tsv +# translate addresses: the map is file-vaddr keyed, so for unchanged +# functions the offsets are usually identical; where the delta shows moves, +# adjust from the decompiled corpus. +analyzeHeadless <proj> cuda-full -process <program> -noanalysis \ + -scriptPath tools/ghidra -postScript ApplyNameMap.java <new>/annotations/names.tsv --dry-run +# then without --dry-run +``` + +Also transfer the *subsystem index* (`annotations/subsystems.tsv`: address, +our name, subsystem, evidence) and re-validate the entries that the delta +marked as moved. + +## 4. Validate against the guest + +1. Load the stack and run the guest smoke test (`tools/test/cuda_drv_test.c` + compiled as `/tmp/cdt`), with the ioctl tracer + (`tools/guest-diag/ioctl_trace.c`) attached. +2. Compare the observed ioctl/control sequence with the previous version's + `docs/05` sequence; new or reordered controls are the interesting deltas. +3. Run the host/guest A/B harness (`tools/cuda-ab/ab.sh all`) for a numerical + correctness check. +4. If the guest fails, use the crash decoder + corpus lookup (§1, §2) to get + from a dmesg `ip` to the exact function and line in the decompiled corpus, + then to the control reply the driver consumed. + +## 5. Update the documents + +* `docs/05-<version>-decompilation.md` — new version-specific map and crash + notes (copy the structure from `docs/05-570.144-decompilation.md`). +* `annotations/names.tsv` and `annotations/subsystems.tsv` — the deltas. +* `docs/04-virtualization-and-emulation.md` — only if the *interface* changed + (new controls, new capability bits, new object classes). +* `README.md` status table. + +## 6. What not to do + +* Do not patch the guest driver or add `LD_PRELOAD` shims to make things work; + that breaks the anti-fragility rule (`docs/04` §3). If a reply must change, + change the emulation. +* Do not commit decompiled output, extracted binaries or Ghidra projects; they + are gitignored for a reason (vendor-derived, large). +* Do not name non-public source locations in tracked docs; describe behaviour + and cite `libcuda+0x...` offsets. +* Do not run a second analysis in the same Ghidra project while another + process has it open (exclusive lock, even read-only); use the per-chunk + project copies in `ghidra_batch_decompile.sh`. |
