From c587eac377c1c5b720142edadebaace5cb1b17c0 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Wed, 23 Sep 2026 13:48:00 -0400 Subject: userspace/cuda: complete the analysis pass * Name recovery: name_candidates.py proposes the hand-written API-layer implementation for each exported wrapper by grouping version variants and intersecting their callees; 180 high-confidence proposals recorded in annotations/name-candidates.tsv, 29 of them body-verified and promoted to names.tsv with evidence in subsystems.tsv. * Engine records: identified the 0x14-byte per-engine descriptor, mapped its fields (kind, engine type, instance, class id, CE caps) and the existence callback, and documented how the missing async copy engine (COPY14) leaves the secondary record zeroed and blocks context creation (docs/05 section 3.7, public-safe). * Targeted type recovery: AnalyzeFunctions.java runs the parameter-ID analyzer over a curated list of 80 functions; 16 improved materially and the typed decompilations are kept under the version run tree. * Cross-version: cudatool.py fdematch measures how names transfer between patch versions (570.144 -> 570.211.01: exports 100%, verified internals ~70%, generated wrappers must be re-resolved by relocation, never by address arithmetic); documented in the upgrade playbook. * Corpus tooling: diff_corpus.py (version-to-version corpus diff) and subsystem_digest.py (per-subsystem call trees). * Deprecate the old per-version helpers whose address handling and analysis timeout produced wrong results. Generated corpora and typed outputs remain untracked under the version run tree. --- userspace/cuda/docs/06-tooling.md | 103 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) (limited to 'userspace/cuda/docs/06-tooling.md') diff --git a/userspace/cuda/docs/06-tooling.md b/userspace/cuda/docs/06-tooling.md index 85d1c8c..72fc8ac 100644 --- a/userspace/cuda/docs/06-tooling.md +++ b/userspace/cuda/docs/06-tooling.md @@ -8,6 +8,9 @@ tools are stdlib-only or Ghidra scripts; nothing needs network access. userspace/cuda/tools/ ├── cudatool.py ELF/interface toolkit (python3, no deps) ├── callwalk.py callee/caller trees and shortest call path +├── name_candidates.py propose impl names for wrapper callees +├── subsystem_digest.py per-subsystem call-tree digest +├── diff_corpus.py diff two decompiled corpus indexes ├── annotate_corpus.py annotated + name-substituted corpus ├── ghidra_full_analysis.sh import + full auto-analysis into a project ├── ghidra_export_index.sh functions/exports/callgraph/strings index @@ -19,6 +22,7 @@ userspace/cuda/tools/ ├── ExportFunctionIndex.java index exporter ├── BatchDecompile.java chunk decompiler ├── DecompileAt.java single function (file-vaddr aware) + ├── AnalyzeFunctions.java targeted Decompiler Parameter ID pass ├── ApplyNameMap.java apply/transfer a name map └── PrintProgramInfo.java address convention + stats ``` @@ -34,12 +38,33 @@ tools/cudatool.py crash # dmesg -> file vaddr tools/cudatool.py strings [--grep RE] tools/cudatool.py note # .note.cuda.fwd_compatibility JSON tools/cudatool.py delta # see §4 +tools/cudatool.py fdematch [--names names.tsv] [--window 0x100] ``` Useful for: checking what changed between two driver package versions before spending time on decompilation; converting a guest dmesg crash to a file vaddr without gdb; getting a function list in a second. +### 1b. Cross-version function matching — `fdematch` + +``` +tools/cudatool.py fdematch [--names ] [--window 0x100] +``` + +`fdematch` matches function ranges from the two `.eh_frame_hdr` tables. A +single global address shift is not enough across patch versions, so it uses +the common exports as **local anchors**: for each A function it looks for a B +function with the same FDE size within `--window` bytes of +`old_addr + delta(nearest common export)`. The report gives exact / window / +unique match counts and the overall match rate, and with `--names` it breaks +the name map down by its `source` column (exports, wrappers, verified, +wrapper-callee), which is exactly what you need to know before transferring +names to a new version. Example result for 570.144 → 570.211.01: 85.5% of +functions match (58.3% uniquely), **all 659 exports transfer exactly, but +only ~1% of the generated wrapper addresses do** — the wrapper region is +regenerated and must be re-resolved with `cudatool.py wrappers`, never by +address arithmetic. + ### Address conventions * `cudatool.py` prints **file virtual addresses** — the `libcuda+0x...` used @@ -102,6 +127,67 @@ tools/callwalk.py /callgraph.tsv chain 0x31b7f0 0x2cb940 Addresses are file vaddrs; `names.tsv` next to the callgraph is used for labels. +## 2d. Targeted type recovery + +The full analysis disables the Decompiler Parameter ID analyzer (§2), so +function signatures stay at the default `FUN_(...)`. `AnalyzeFunctions.java` +re-runs that analyzer over a caller-supplied function list instead of the whole +program: it takes an addresses file (one `0x` per line, comments +allowed), translates through the image base like `DecompileAt.java`, resolves +each address to its function and runs the analyzer on the resulting address +set (batched, with a per-function decompiler timeout). Run it against a +project copy, not the canonical one, because it commits parameters into the +project: + +``` +cp -a -typed +analyzeHeadless -typed -process -noanalysis \ + -scriptPath tools/ghidra \ + -postScript AnalyzeFunctions.java \ + --timeout 120 --batch 20 --out +``` + +Then re-decompile the affected functions with `ghidra_decompile_at.sh` from the +typed project; the report's `params_before/after` and `GAIN` columns show +which functions gained structure. `--list` resolves the list without running +the analyzer (dry run). + +## 2d. Subsystem call-tree digest — `subsystem_digest.py` + +``` +tools/subsystem_digest.py [--out FILE] + [--depth 2] [--max-per-node 8] [--subsystem NAME] +``` + +For every subsystem in `annotations/subsystems.tsv` it prints the annotated +members, the subsystem **entry functions** (members not called by another +member of the same subsystem) and a depth-limited call tree from each entry +whose direct callees are grouped by subsystem. It closes with a +cross-subsystem edge count matrix. This is the quickest way to see a +subsystem's shape and where its boundary with the rest of the driver is, +without opening Ghidra. Example output for 570.144 is saved under +`570.144/run/analysis/subsystem-digest.txt` (gitignored). + +## 2e. API-implementation candidates — `name_candidates.py` + +``` +tools/name_candidates.py + [--max-outside 2] +``` + +Exports are generated stubs → generated wrappers → the hand-written +implementation of that API. `name_candidates.py` groups the wrappers by base +name (stripping `_v`, `_ptsz`, `_ptds`), takes the intersection of their +direct callees and filters out helpers that are also called by wrappers +outside the group (the trace/init/lock helpers). A single surviving candidate +is `high` confidence when the group has several variants or the callee is +unique; otherwise the best-ranked candidate is `medium`. Output rows carry +the full candidate set and the outside-wrapper count as evidence, plus a +`status` column that starts as `unverified`. Verification means reading the +body in the corpus and marking `verified`/`corrected`; only verified entries +may be copied into `names.tsv` (source `wrapper-callee`). The 570.144 run is +tracked in `570.144/annotations/name-candidates.tsv`. + ## 3. Looking things up ``` @@ -131,6 +217,23 @@ first thing to run on a new driver release; it tells you immediately whether the update is a patch (nothing moved), a feature release (new exports) or a reshuffle (many moves). +### 4b. Corpus deltas — `diff_corpus.py` + +``` +tools/diff_corpus.py [--label-a A] [--label-b B] + [--out FILE] [--max-list 40] [--json] +``` + +Diffs two `decompiled_index.tsv` corpora (the merged output of +`ghidra_batch_decompile.sh` + `ghidra_index_merge.sh`) and reports +added/removed/moved/resized functions and a summary. Named functions match by +name; the address shift is estimated from them and anonymous `FUN_*` +functions match at `old_addr + shift` (a size difference there is a resize). +This makes a driver upgrade measurable at the function level: run it after +decompiling the new version to see how much of the previous corpus still +corresponds. It needs a decompiled index for the other version; if only the +binary exists, use `cudatool.py fdematch` (no decompilation required). + ## 5. Name transfer — `ApplyNameMap.java` ``` -- cgit v1.2.3