diff options
Diffstat (limited to 'userspace')
| -rw-r--r-- | userspace/cuda/570.144/docs/ANALYSIS-cuctxcreate-segv.md | 170 |
1 files changed, 170 insertions, 0 deletions
diff --git a/userspace/cuda/570.144/docs/ANALYSIS-cuctxcreate-segv.md b/userspace/cuda/570.144/docs/ANALYSIS-cuctxcreate-segv.md new file mode 100644 index 0000000..95b6237 --- /dev/null +++ b/userspace/cuda/570.144/docs/ANALYSIS-cuctxcreate-segv.md @@ -0,0 +1,170 @@ +# libcuda 570.144 `cuCtxCreate` segfault - analysis (2026-09-23) + +First analysis note for the CUDA userspace driver under Open-IOV. Companion to +`../README.md` (tools) and the host/guest A/B harness (`tools/cuda-ab/`). + +## Status summary + +| Phase | Result | +|---|---| +| `cuInit(0)` | **OK** (previous sessions: failed) | +| `cuDeviceGetCount` / `cuDeviceGet` | **OK** (device 0) | +| `cuCtxCreate` | **SIGSEGV inside libcuda** - the current blocker | +| `cuMemAlloc` / copies / `ALL OK` | not reached yet | + +Guest: Ubuntu 24.04, driver **570.144** (open kernel modules + libcuda +570.144), emulated TU104 `10de:1ed3` via `open-iov-vfio.ko`. +Host oracle: RTX 2080 Super (cc 7.5), driver 570.211.01, CUDA toolkit 12.0. + +## The crash, precisely + +Guest kernel log: + +``` +cdt[912]: segfault at ffffffffffffffc8 ip 0000735cfce53f90 ... error 5 + in libcuda.so.570.144[735cfcb79000+d1c000] +``` + +* fault address `0xffffffffffffffc8` = **NULL - 0x38** +* file vaddr of the fault: `0x453f90` (dmesg's `[...]` base is the first + *executable* mapping, not the load base - see `scripts/`; the ELF exec + segment starts at vaddr `0x179000`, so the offset must be corrected: + `vaddr = ip - mapping_base + 0x179000`... here it lands in function + range `0x453ef0..0x45401b` found in `.eh_frame`). +* gdb registers at the fault: `rbp=0`, `rax=0`, `rbx=0x7ffff1035010`, + `rsi=0xc030462b` (an NV ioctl request = `_IOWR('F', 0x2b, 48)` = + `NV_ESC_RM_ALLOC`), `rdi=0xffffffff`. +* The faulting instruction reloads a stack slot relative to `rbp`: + `mov rax, [rbp-0x38]` - with `rbp = 0` this reads NULL-0x38. + +So a callee in the call chain ended up with a **clobbered/zero frame +pointer**; the function expects `rbp` to be its frame. + +## What libcuda is doing when it crashes (from the ioctl trace) + +`tools/guest-diag/ioctl_trace.c` (ptrace; libcuda uses raw syscalls) shows +`cuCtxCreate` performing, in order: + +``` +alloc class=0x0 (NV01_ROOT) +alloc class=0x80 (NV01_DEVICE_0) +alloc class=0x2080 (NV20_SUBDEVICE_0) +alloc class=0x40 (NV01_EVENT x several) +alloc class=0xa06c (KEPLER_CHANNEL_GROUP_A) +alloc class=0x9067 (channel group / runlist object) +alloc class=0x3e (NV01_MEMORY_*) +alloc class=0xc46f (TURING_CHANNEL_GPFIFO_A) <- the channel +control cmd=0x80170d (NV0080_CTRL_FIFO_GET_CHANNELLIST) <- driver-local +alloc class=0xc5c0 (TURING_COMPUTE_A) <- last alloc +[crash] +``` + +Host-side GSP-RM emulation log for the same window: + +``` +CE_GET_FAULT_METHOD_BUFFER_SIZE -> 12288 (handled) +GPU_PROMOTE_CTX (0x2080012b, 560 B) -> SUCCESS (generic fallback) +TURING_CHANNEL_GPFIFO_A alloc, TURING_COMPUTE_A alloc +``` + +## The crashing function (Ghidra, full-analysis project) + +`FUN_00453ef0` (disassembly is authoritative; Ghidra also flags it as using +the caller's frame - i.e. a local subroutine of the enclosing FDE range +`0x453ef0..0x45401b`): + +``` +rdi = object, rsi = per-engine descriptor (from caller) +r14d = [rsi]; if (r14d == 1) r14d = 0 +rax = [[rdi]+8]; rax = [rax+0x40]; [rbp-0x38] = rax (spill a table) +call 0x3a9c20(global 4413d60) (returns eax) +if (eax != 0) { + rax = global 4413ed8; rax = [rax+0x40] + rsi = [rdi+0x9488]; esi = *rsi + edi = global 4412f40; edx = r15d (the status); rcx = descriptor + call [rax+0x530] (indirect: log hook) + if (eax == 0) goto 0x453f90 (crash site) + ... +} +``` + +Callers: only two, both in `FUN_0045a670` (`0x45afc1` and `0x45b02b`), +which walks engines and calls this helper per engine with +`rsi = device + 0x2fec`, `device + 0x300c + (idx-1)*0x14`, ... - i.e. it is +a **per-engine context check/report** path. + +## Current hypothesis + +The crash happens *after* an engine-related check failed: the helper calls a +global logging/dispatch hook (`call [rax+0x530]`) and the frame is corrupted +on the way back, or the hook pointer itself is not a valid function because +the dispatch table was never (fully) initialised in this environment. Two +sub-hypotheses, both testable: + +1. **Emulation data gap**: our GSP static info advertises 11 engines (GR, + 5x CE, 2x NVDEC, NVENC, SEC2, NVJPG) but only the CE caps are emulated; + a real GSP-RM would let libcuda/driver bring each engine up (or fail in a + handled way). If the per-engine status/descriptor data we provide is + inconsistent, libcuda takes this error path. +2. **Log hook table**: the hook reached through two global pointer + dereferences (`[global+0x40]+0x530`) is not initialised, so the "callee" + is not a real function and clobbers `rbp`. + +`GPU_PROMOTE_CTX` returning generic-SUCCESS is also suspicious (on TU104 with +no NVLink/vGPU this may be expected to fail in a specific way); it is the +last *control* before the crash window. + +## Register trace (gdb, breakpoints in the live guest) + +Sequence observed (single helper invocation): + +``` +HELPER-ENTRY rbp=0x7fffffffdbf0 rsp=0x7fffffffd5d8 rax=1 +HOOK-ENTRY rbp=0x7fffffffd5d0 rsp=0x7fffffffd588 rax=<heap ptr> +HOOK-LEAVE-OK rbp=0x7fffffffd580 rsp=0x7fffffffd570 rax=0 +AFTER-HOOK rbp=0x0 rsp=0x7fffffffd590 rax=0 <- return from hook +CRASH rbp=0x0 rsp=0x7fffffffd590 +``` + +* The helper entered with a valid frame, and the hook's own frame is also + valid (`push rbp; mov rbp,rsp` ... `leave; ret` - the hook is at file + vaddr `0x4676f0` and does **not** by itself clobber rbp). +* Yet immediately after the hook returns (`0x453f65`), **rbp is 0**. The + saved frame pointer on the hook's stack must therefore have been + overwritten inside the hook call chain, or the hook returned through a + different path than its `leave`. +* The hook calls `0x481360` with `ecx = [descriptor+0xc]` (an engine class/id + switch over values 0x80, 0x79, 0x5, ...) and `r8 = &local`; that function + also has a normal frame. The descriptor is built by the caller from + `device+0x2fec` / `device+0x300c+(idx-1)*0x14`. +* Table values read live: `[global 4413ed8] = 0x7ffff7896d80`, + `[[global+0x40]] = 0x7ffff3a5f6c0` (libcuda+0x45f6c0), + `[[[[global]+0x40]+0x530]] = 0x7ffff3a676f0` (libcuda+0x4676f0, the hook + above) - i.e. the dispatch table *is* populated with real libcuda code. + +So the immediate mechanism is: **the engine-check path runs with an +unexpected engine descriptor/class and loses the caller's frame while +reporting it**. The engine descriptor contents (and what the emulation +advertised for that engine) are the next thing to inspect. + +## What is NOT (yet) analysed + +* No full auto-analysis output was reviewed beyond the functions above; + the Ghidra project exists (`run/ghidra-proj`) for cheap further + decompilation (`scripts/ghidra_decompile_project.sh <vaddr>`). +* The engine descriptor contents at the hook (`[descriptor+0x0/0x8/0xc]`) + were not yet dumped and compared with the expected engine table semantics. +* No comparison of our GSP static info engine table against a real TU104 + vGPU dump yet. + +## Reproduction + +``` +tools/guest-diag/vmcycle.sh +tools/guest-diag/push_guest_tools.sh +# in guest: sudo modprobe nvidia_uvm; setsid /tmp/cdt >/tmp/cdt.out 2>&1 & +``` + +`/tmp/cdt` prints `-> <call>` before each libcuda entry point (see +`tools/test/cuda_drv_test.c`), so the last marker identifies the crashing +call. Add `strace`-like ioctl detail with `/tmp/iot /tmp/cdt`. |