diff options
Diffstat (limited to 'userspace/cuda/docs/05-570.144-decompilation.md')
| -rw-r--r-- | userspace/cuda/docs/05-570.144-decompilation.md | 89 |
1 files changed, 69 insertions, 20 deletions
diff --git a/userspace/cuda/docs/05-570.144-decompilation.md b/userspace/cuda/docs/05-570.144-decompilation.md index 84b377e..4e23866 100644 --- a/userspace/cuda/docs/05-570.144-decompilation.md +++ b/userspace/cuda/docs/05-570.144-decompilation.md @@ -162,9 +162,18 @@ engine index >= 0x16 -> table + 0x351c + (index-0x16)*0x14 before the helper, the walker calls the callback at `table+0x4e00` with `(table, desc->[0xc])` — a -"does this engine/class exist" check — and only proceeds when it returns -true. The helper then uses `desc->[0]` (engine kind), `desc->[8]` (instance) -and `desc->[0xc]` (class) for the registration. +"does this engine/class exist" check (a class-list membership test; the +callback pointer is installed by the device ABI table setup at +`libcuda+0x458040`/`libcuda+0x4494d0`, target `libcuda+0x44a170`) — and only +proceeds when it returns true. The helper then uses `desc->[0]` (engine kind), +`desc->[8]` (instance) and `desc->[0xc]` (class) for the registration. + +The records are built by `libcuda+0x450ff0` from the engine list that the +`GET_ENGINES_V2` wrapper `libcuda+0x462770` returns (`{kind, engine type, +instance}` triples at 0x10-byte stride). The record layout and the full field +map are in §3.7. Note the walker's index ranges cover copy slots `1..0xa` and +`0xc..0x15`, with `0xb` used for the SEC2 slot and `>=0x16` for decompression +copy slots; the engine list is a different, dense array. ### 3.4 The engine create/register function @@ -232,8 +241,8 @@ path on the host's real driver): * libcuda asks the driver for the GPU's engines (`GET_ENGINES_V2`). The host oracle's filtered list has **7** engines: GR, CE0..CE4 and `COPY14`, the async copy engine. The guest list has only **6** — the same minus `COPY14`. -* `COPY14`'s engine descriptor is `0x793ceb0e`, instance 14, client engine - type `0x38`, paired with engine class `0xc9b5`. The GSP-RM firmware's +* `COPY14`'s engine descriptor is `0x793ceb0e`, instance 14, engine type + `0x38`, paired with engine class `0xc9b5`. The GSP-RM firmware's `{class, engDesc}` pair table pairs OBJCE instances `0..7` with class `0xcab5` and instances `0x0b..0x13` (including `0x0e` = 14) with class `0xc9b5`. @@ -256,24 +265,56 @@ static info and add a 13th device-info entry with descriptor `0x793ceb0e`, instance 14, RM engine type `0x17`, FIFO tag 6, runlist 5) did not change the guest list, so the missing input is elsewhere: what builds the driver's classDB entry for descriptor `0x793ceb0e` (class list/static-info class data -versus the device-info-table fields) is the open question. The record layout -the walker reads is in §3.3; the per-engine fields it consumes are -`+0` (kind), `+8` (instance) and `+0xc` (class). - -### 3.7 What the emulation should check first - -* **Engine-list/classDB consistency — the active blocker (§3.6).** The engine - records at the offsets in §3.3 must be present and self-consistent for every - engine index the walker visits, the "does this engine exist" callback - (`[table+0x4e00]`) must agree with them, and the whole set must agree with - the class-filtered engine list the driver builds from the GSP static info - (`GET_ENGINES_V2` + classDB). A missing entry is silent: the walker then - reads a zeroed record. +versus the device-info-table fields) is the open question. + +### 3.7 The engine descriptor record, and how it explains the blocker + +The device state holds a list of 0x10-byte engine entries plus a parallel set +of 0x14-byte **per-engine descriptor records**; the walker (§3.3) reads the +records, the helper and create hook consume them. The record was recovered +from the record builder (`libcuda+0x450ff0`), the `GET_ENGINES_V2` wrapper +(`libcuda+0x462770`), the create hook disassembly (`libcuda+0x4676f0`) and the +live guest dump (`run/gdb32b.out`): + +| Offset | Field | Meaning | Evidence | +|---|---|---|---| +| `+0x00` | engine kind | `1` = graphics (GR), `3` = copy (CE), `4` = SEC2, `5` = decompression copy. The helper normalizes `1` to `0` for the per-context slot index. | builder branches on it; `GET_ENGINES_V2` wrapper classifies each engine type into it; live: GR `t=1`, CE `t=3` | +| `+0x04` | engine type | raw `NV2080_ENGINE_TYPE` from the engine list (`1` = GR, `9..13` = COPY0..4, `0x38` = COPY14) | builder copies the first 8 bytes of the list entry into `+0/+4`; not read by the walker/helper | +| `+0x08` | instance | engine instance (`COPY0..4` → `0..4`, `COPY14` → `14`) | helper + create hook read it; create hook copies it into the RM alloc parameters only for kinds 2/3/5; live dump | +| `+0x0c` | class | RM allocation class for this engine: GR → the device's compute class (live `0xc5c0`), copy → the DMA copy class (live `0xc5b5`), SEC2/decomp copy → their classes | builder stores the per-kind class; the create hook passes `+0xc` straight to `libcuda+0x481360` as the `NV_ESC_RM_ALLOC` class; host capture shows `class=0xc5b5` | +| `+0x10` | engine caps | for copy records: the CE capability bitmask; **bit 0 (`CE_GRCE`) marks the async/GRCE copy engine**. A copy record with bit 0 set is also copied to the *secondary record* slot and does not take a normal CE slot. | builder tests `+0x10 & 1` and copies to the secondary; live caps `0xbc`/`0xfc` (bit 0 clear) | + +Secondary record: the GR context registers a second engine from a dedicated +0x14-byte slot (same layout) that is filled from the GRCE-capable copy record +and then normalized to kind `2` and the async copy class. GR's own record is +larger (a 0x20-byte slot with an unrecovered tail at `+0x10..+0x1f`). + +**How this explains the blocker.** The builder derives the records from the +engine list the driver returns for `GET_ENGINES_V2`; the guest list has 6 +entries (GR + CE0..4) because the class-database filter drops `COPY14`. With +no `COPY14` entry there is no copy record with the `CE_GRCE` capability bit +set, so the secondary record slot is never filled and the walker sees class +`0`. libcuda then allocates the engine object with class `0`, the escape path +rewrites that to `NV01_ROOT_CLIENT` (`0x41`), the allocation fails with status +`0x19`, the context rolls back and `cuCtxCreate` returns 999. In other words: +the guest is not merely missing one entry in a list — it is missing the +*async copy engine record that seeds the secondary record*. + +**What the emulation should check first (revised)**: + +* **`COPY14` in `GET_ENGINES_V2`** — the engine's classDB entry must exist so + that `gpuUpdateEngineTable`'s class-list filter keeps OBJCE instance 14. + The class/advertised-engine-table consistency is the root cause; the + secondary-record zeroing is its symptom. +* **CE capabilities.** The copy records' caps word must carry the `CE_GRCE` + bit for the async CE. If `COPY14` is restored but its emulated CE caps still + return bit 0 clear, the secondary record stays empty and the class-0 + allocation returns. * **Consumed-control reply fidelity.** The controls answered generically (the host log lists `0x20803039`, `0x20802a12`, `0x20808162`, `0x2080182b`, `0x20800a70`, `0x20800a93`, …) should be checked for consumed fields, because the walker's engine table is built from data the driver collected earlier - (device caps, engine list, class list). + (device caps, engine list, class list, CE caps). * The `NV_ESC_RM_ALLOC` parameter block built by `libcuda+0x481360` for class `0xc5c0` (`TURING_COMPUTE_A`) must be answered with a coherent object: echo the request's params back (§3.5) and return a coherent handle; the driver @@ -307,7 +348,15 @@ citations in docs and gdb sessions use the same numbers. holds our name map (exports are exact, internals grow over time). * The decompiler does not recover struct types; field offsets are named `[obj+0xNN]` and must be interpreted with the descriptions in `docs/01` - and `docs/03`. + and `docs/03`. A targeted Decompiler Parameter ID pass + (`tools/ghidra/AnalyzeFunctions.java`, see `docs/06` §2d) re-ran over 80 + key functions (crash/engine path, resolved wrapper implementations, largest + `.text`) and committed their signatures into a project copy + (`run/ghidra-typed/`); the re-decompiled bodies are in + `run/analysis/typed/` and the before/after report in + `run/analysis/typed/paramid_report.tsv`. 16 of 80 bodies changed materially + (parameter/type recovery), the rest now carry committed signatures that + improve downstream decompilation. * Indirect calls dominate the API dispatch; the callgraph contains direct edges only (77,742). Cross-referencing indirect targets requires reading the wrapper/stub relocations (as in §2) or live traces. |