diff options
Diffstat (limited to 'userspace/cuda/docs/03-driver-and-gpu-interface.md')
| -rw-r--r-- | userspace/cuda/docs/03-driver-and-gpu-interface.md | 276 |
1 files changed, 276 insertions, 0 deletions
diff --git a/userspace/cuda/docs/03-driver-and-gpu-interface.md b/userspace/cuda/docs/03-driver-and-gpu-interface.md new file mode 100644 index 0000000..4c4caa2 --- /dev/null +++ b/userspace/cuda/docs/03-driver-and-gpu-interface.md @@ -0,0 +1,276 @@ +# 03 — What the CUDA userspace driver asks of the kernel and the GPU + +This document is the emulation contract: everything `libcuda` needs from +below itself. It is the checklist Open-IOV's host-side emulation has to +satisfy for unmodified CUDA to run, and the map used when a guest call fails. + +Sources: live ioctl traces from the guest (ptrace tracer), host-side GSP-RM +emulation logs, and decompilation of the RM backend in `libcuda` 570.144. The +ioctl numbers below were observed on the wire; the object classes and control +IDs are quoted from the trace and from public CUDA/RM headers. + +--- + +## 1. Device nodes and the ioctl vocabulary + +| Node | Used for | +|---|---| +| `/dev/nvidiactl` | process-wide control: version negotiation, client creation, GPU attach, OS events, exported-object file descriptors | +| `/dev/nvidiaN` | per-GPU: object alloc/free, RM controls, memory alloc/map, event allocation | +| `/dev/nvidia-uvm` | managed memory and page migration (UVM userspace API) | +| `/dev/nvidia-vgpuN` | vGPU plugin node (only inside NVIDIA's own vGPU stack; not used by stock CUDA paths) | + +Control-device ioctls (`_IOWR('F', cmd, …)` with magic `'F'`): + +| cmd | Name | Purpose | +|---|---|---| +| 200 (`0xc8`) | `NV_ESC_CARD_INFO` | enumerate GPUs and minors | +| 201 (`0xc9`) | `NV_ESC_REGISTER_FD` | bind a per-GPU fd to the control fd | +| 206/207 | `NV_ESC_ALLOC_OS_EVENT` / `FREE_OS_EVENT` | OS event objects (blocking waits, GPU events) | +| 209 (`0xd1`) | `NV_ESC_STATUS_CODE` | driver status on open failure | +| 210 (`0xd2`) | `NV_ESC_CHECK_VERSION_STR` | RM/driver version negotiation | +| 211 (`0xd3`) | `NV_ESC_IOCTL_XFER_CMD` | inter-GPU ioctl forwarding | +| 212 (`0xd4`) | `NV_ESC_ATTACH_GPUS_TO_FD` | attach GPUs to an fd (exported objects) | +| 213 (`0xd5`) | `NV_ESC_QUERY_DEVICE_INTR` | interrupt query | +| 214 (`0xd6`) | `NV_ESC_SYS_PARAMS` | system parameters | + +RM escapes (same magic, used on control and per-GPU fds): + +| cmd | Name | Purpose | +|---|---|---| +| 0x27 | `NV_ESC_RM_ALLOC_MEMORY` | allocate memory (older path; NVOS02) | +| 0x28 | `NV_ESC_RM_ALLOC_OBJECT` | allocate object (NVOS05) | +| 0x29 | `NV_ESC_RM_FREE` | free object (NVOS00) | +| 0x2a | `NV_ESC_RM_CONTROL` | issue an RM control (NVOS54) | +| 0x2b | `NV_ESC_RM_ALLOC` | allocate object (NVOS21/NVOS64) — **the main alloc path** | +| 0x34 | `NV_ESC_RM_DUP_OBJECT` | duplicate/import an object | +| 0x35 | `NV_ESC_RM_SHARE` | share object | +| 0x41 | `NV_ESC_RM_IDLE_CHANNELS` | idle channels | +| 0x4a | `NV_ESC_RM_VID_HEAP_CONTROL` | heap allocation (NVOS32) | +| 0x4e / 0x4f | `NV_ESC_RM_MAP_MEMORY` / `UNMAP_MEMORY` | map memory into the process (NVOS33/NVOS34) | +| 0x57 / 0x58 | `NV_ESC_RM_MAP_MEMORY_DMA` / `UNMAP_MEMORY_DMA` | map memory into a device VA space (NVOS46/NVOS47) | +| 0x5c / 0x5d | `NV_ESC_RM_EXPORT_OBJECT_TO_FD` / `IMPORT_OBJECT_FROM_FD` | cross-process object sharing | +| 0x52 | `NV_ESC_RM_GET_EVENT_DATA` | read event payload (NVOS41) | +| 0x32/0x33 | `NV_ESC_RM_CONFIG_GET/SET` | registry-style configuration | + +In a trace, the ioctl request word encodes the escape and the size; e.g. +`_IOWR('F',0x2b,48)` = `NV_ESC_RM_ALLOC` with a 48-byte payload. Seeing this +request in a crash register dump (`rsi=0xc030462b`) identifies "driver was +allocating an RM object" — that is exactly what happens at the `cuCtxCreate` +crash (see `docs/05`). + +## 2. The RM object model as CUDA uses it + +Every RM call names a **class**. The classes CUDA allocates, in the order +observed for 570.144 on an emulated TU104: + +| Class | Name (public RM header) | When | +|---|---|---| +| `0x0` | `NV01_ROOT` | process client (once, at `cuInit`) | +| `0x80` | `NV01_DEVICE_0` | per GPU, at attach | +| `0x2080` | `NV20_SUBDEVICE_0` | per GPU/subdevice | +| `0x40` | `NV01_EVENT` | events (several) | +| `0xa06c` | `KEPLER_CHANNEL_GROUP_A` | context channel group (TSG) | +| `0x9067` | channel-group/runlist object | context | +| `0x3e` | `NV01_MEMORY_*` | memory objects | +| `0xc46f` | `TURING_CHANNEL_GPFIFO_A` | the channel itself | +| `0xc5c0` | `TURING_COMPUTE_A` | compute engine object bound to the channel | +| `0xc5b5` | (compute class, second engine) | used by the host A/B run for a CE-style object | +| `0xa080` | `KEPLER_DEVICE_VGPU` | vGPU capability query (vGPU only) | +| `0x90f1`-family | VA space (`FERMI_VASPACE_A` etc.) | memory model | + +`cuCtxCreate`'s observed ioctl sequence (guest, 570.144): + +``` +NV_ESC_RM_ALLOC class=0x0 (NV01_ROOT) +NV_ESC_RM_ALLOC class=0x80 (NV01_DEVICE_0) +NV_ESC_RM_ALLOC class=0x2080 (NV20_SUBDEVICE_0) +NV_ESC_RM_ALLOC class=0x40 (events) +NV_ESC_RM_ALLOC class=0xa06c (KEPLER_CHANNEL_GROUP_A) +NV_ESC_RM_ALLOC class=0x9067 (channel group object) +NV_ESC_RM_ALLOC class=0x3e (memory) +NV_ESC_RM_ALLOC class=0xc46f (TURING_CHANNEL_GPFIFO_A) <- channel +NV_ESC_RM_CONTROL cmd=0x80170d (NV0080_CTRL_FIFO_GET_CHANNELLIST) +NV_ESC_RM_ALLOC class=0xc5c0 (TURING_COMPUTE_A) <- last alloc +[engine registration attempt; pre-fix this segfaulted — now rc=999, see docs/05 §3.5-3.6] +``` + +The first two allocs (`NV01_ROOT`, `NV01_DEVICE_0`) actually happen at +`cuInit`/device attach; the trace window shows them repeated per client setup. + +## 3. RM controls the CUDA path uses + +Controls are `NV_ESC_RM_CONTROL` with a 32-bit command ID and a fixed-size +parameter block. Families relevant to CUDA bring-up (IDs in the low 32 bits +are `(class << 16) | cmd`; the list below is from traces and decompiled call +sites, not exhaustive): + +* **Device/GPU info** — `NV0080_CTRL_CMD_GPU_GET_VIRTUALIZATION_MODE` + (`0x00800089`), `NV2080_CTRL_CMD_GPU_GET_INFO`, `GPU_GET_ENGINES`, + `GPU_GET_CLASSLIST`, `GPU_GET_SIMULATION_INFO`, `FB_GET_INFO`, + `BUS_GET_INFO`, `BUS_GET_PCI_INFO`, `BUS_GET_PCI_BAR_INFO`, + `MC_GET_ARCH_INFO`, `GR_GET_INFO`, `GR_GET_GPC_MASK`, `GR_GET_TPC_MASK`, + `GR_GET_GLOBAL_SM_ORDER`, `FB_GET_LTC_INFO_FOR_FBP`, `CE_GET_CAPS`, + `GSP_GET_FEATURES`, `GPU_GET_SW_FEATURES`, `GPU_GET_MEMOP_ENABLE`, + `FB_GET_CAPS`, `GR_GET_CAPS_V2`. +* **FIFO/channels** — `NV0080_CTRL_CMD_FIFO_GET_CHANNELLIST` (`0x80170d`), + `NV0080_CTRL_CMD_FIFO_GET_CAPS`, `NVA06F/NVA06C_CTRL_CMD_GPFIFO_SCHEDULE`, + `SET_ERROR_NOTIFIER`, `NVA06C_CTRL_CMD_SET/GET_TIMESLICE`, + `NV2080_CTRL_CMD_FIFO_DISABLE_CHANNELS`, per-class + `GET_CLASS_ENGINEID` (e.g. `NV906F`, `NVA06F`-family), + `NVC36F_CTRL_CMD_GPFIFO_GET_WORK_SUBMIT_TOKEN`. +* **GR/context** — `GR_CTXSW_PM_MODE`, `GR_CTXSW_SMPC_MODE`, + `GR_SET_CTXSW_PREEMPTION_MODE`, `GR_PC_SAMPLING_MODE`, + `GR_GET_CTX_BUFFER_SIZE`, `FLCN_GET_CTX_BUFFER_SIZE`. +* **Memory** — `FB_GET_INFO` (BAR1 info), `NV0000_CTRL_CMD_OS_UNIX_*` + (flush user cache, exported objects), `NV0041_CTRL_CMD_GET_SURFACE_INFO`, + `NV0000_CTRL_CMD_CLIENT_GET_ADDR_SPACE_TYPE`. +* **Events/interrupts** — `NV90E6_CTRL_CMD_MASTER_GET_VIRTUAL_FUNCTION_ERROR_CONT_INTR_MASK` + (interrupt-tree setup), `NV83DE_CTRL_CMD_DEBUG_*` (SM error states, debug + mode), `RC_ENABLE/DISABLE/SOFT_DISABLE_WATCHDOG`, `RELEASE_WATCHDOG_REQUESTS`. +* **vGPU/licensing** — `NVA080_CTRL_CMD_VGPU_GET_CONFIG` (`0xa0800301`), + `NVA080_CTRL_CMD_VGPU_SET_LICENSE_INFO`, + `NV2080_CTRL_CMD_GPU_GET_LICENSABLE_FEATURES`, + `GPU_GET_GRID_UNLICENSED_STATE_MACHINE_INFO` (`0x2080018c`). +* **Perf/clocks** — `GPU_GET_COMPUTE_POLICY_CONFIG`, boost/sync-boost + controls, `NV2080_CTRL_CMD_GPU_ACQUIRE/RELEASE_COMPUTE_MODE_RESERVATION`. + +Traces show the driver tolerating unknown-but-successful controls: it only +fails when a *specific* reply field is consumed and wrong. This is why the +emulation can start with generic success and then bisect the consumed fields +(documented in `docs/04` as the "reply-fidelity ladder"). + +## 4. Memory: allocation, mapping, VA + +CUDA's memory path uses three layers: + +1. **VA reservation** — `NV_ESC_RM_VID_HEAP_CONTROL` with a virtual + `NVOS32_FUNCTION_ALLOC_SIZE` request (or no reservation at all when UVM + owns the VA space). +2. **Backing allocation** — vidmem/sysmem `NVOS32` allocation with attributes: + location (`VIDMEM`/`PCI`/`SYSTEM`), page size, coherency, compression, + protection, fixed physical address (if requested), or pinning an existing + host address (`NV01_MEMORY_SYSTEM_OS_DESCRIPTOR`). +3. **Mapping** — `NV_ESC_RM_MAP_MEMORY` (`NVOS33`) to map into the process + (for host-visible/CPU access, flags include MAP_FIXED, RESERVE_ON_UNMAP, + DIRECT for incoherent sysmem) and `NV_ESC_RM_MAP_MEMORY_DMA` (`NVOS46`) to + map into a device VA space (flags include RW/RO, snoop, fixed DMA offset, + page size, peer flags). + +Related classes: `FERMI_VASPACE_A` (VA space; one is shared with UVM, one +legacy, optionally one for fabric-local aperture), `NV01_MEMORY_FABRIC*` / +`NV_MEMORY_FABRIC*` (NVLink/fabric), `NV01_MEMORY_SYSTEM_OS_DESCRIPTOR` +(pinned host memory), `NV01_MEMORY_LOCAL_USER` (vidmem). + +For Open-IOV the practical summary is: **all guest memory that CUDA uses is +either an RM allocation backed by emulated VRAM/BAR1, or pinned guest RAM that +the GPU must be able to DMA to.** The project's current model backs VRAM with +host RAM and executes push-buffer work in the host (see the parent handoff). + +## 5. Channels, GPFIFO, USERD and doorbells + +A CUDA channel is a GPFIFO (array of pushbuffer descriptors) plus one or more +engine objects: + +* GPFIFO memory: allocated by CUDA (vidmem or sysmem), passed to the kernel + driver at channel creation with its offset and entry count, together with + an error notifier object. +* **USERD**: on Volta and later, CUDA allocates the user-mode doorbell/control + area itself and passes its handles to the channel allocation; older + generations map the kernel-provided USERD. On Hopper, a second BAR1-mapped + usermode region is used for doorbells. +* **Doorbell**: submission writes the GPFIFO put pointer into USERD and rings + a doorbell token (obtained via `NVC36F_CTRL_CMD_GPFIFO_GET_WORK_SUBMIT_TOKEN` + on newer ABIs, or a static token on older ones) at a token/offset the + channel allocation returned. +* **Pushbuffers**: command streams in vidmem or sysmem; a push is a sequence + of methods (`SET_OBJECT`, engine methods, semaphore releases) terminated by + a tracking-semaphore release and a new GPFIFO entry. +* **Context share (Volta+)**: subcontexts are created by allocating a + `FERMI_CONTEXT_SHARE_A` object with the VA space handle and subcontext id; + channels then reference it instead of the VA space directly. + +Open-IOV relevance: the host emulation sees the GPFIFO and pushbuffer memory +the guest writes. The project already decodes push buffers as method streams +and executes memory ops (`kernel/gsp_sema_service.c`); this section defines +the contract those streams follow. + +## 6. Synchronization objects below the API + +* **Semaphores** — plain 32/64-bit memory values, allocated by CUDA in + GPU-visible memory. The GPU releases them (semaphore release methods or + inline memcpy of the payload), the CPU polls or waits. Tracking semaphores + count completed pushes; QMD semaphores count completed grids. +* **Notifiers** — 4 KB error-notifier memory per channel; the kernel driver + writes a notification record on channel errors, which CUDA reads when + synchronizing to map to `CUDA_ERROR_LAUNCH_*`. +* **OS events** — kernel objects (`NV_ESC_ALLOC_OS_EVENT`) used to wake + blocked threads; GPU events bind an engine event to an OS event so an + interrupt can signal user space. +* **Interrupts** — the kernel driver owns the GPU interrupt; CUDA's ISR + thread is woken through the OS event/monitored-fence mechanism and then + walks channel state, updates blocking-sync accounting and runs DPCs. +* **Trap handler** — for debuggers and `device-side assert`, CUDA installs a + trap handler (embedded device code) and reads a trace buffer in GPU memory. + +## 7. UVM interface + +When UVM is available, the userspace driver calls the UVM API (through +`/dev/nvidia-uvm`): + +* set the expected kernel driver version (forward compatibility); +* initialize UVM (sharing mode for MPS, HMM/pageable-memory options); +* register each GPU and each VA space; +* register each channel with its platform parameters (the RM handles and + context-buffer VA/size the kernel driver provided); +* allocate/register managed allocations, prefetch/advice calls; +* semaphores: UVM semaphore acquire/release are pushed into the same streams. + +If UVM is unavailable or the vGPU profile lacks UVM/fault capability, CUDA +disables kernel-mode UVM and degrades to zero-copy behavior. + +## 8. GSP-RM in the picture + +Modern drivers run the resource manager on the GSP (a RISC-V core on the +GPU). From the userspace driver's perspective this changes nothing: it still +issues RM ioctls. Inside the kernel driver, controls become GSP-RM RPCs; in +the Open-IOV emulation, the host answers those RPCs. The host log for the +`cuCtxCreate` window shows, in order: + +``` +CE_GET_FAULT_METHOD_BUFFER_SIZE -> 12288 +GPU_PROMOTE_CTX (0x2080012b, 560 B) -> generic SUCCESS +TURING_CHANNEL_GPFIFO_A alloc, TURING_COMPUTE_A alloc +``` + +The userspace failure now surfaces after the engine registration attempt: with +the alloc reply fixed (`docs/05` §3.5) the walker reads a zeroed secondary +engine record and `cuCtxCreate` returns `rc=999` (`docs/05` §3.6). The +function-level path is in `docs/05`; the emulation-side question is what input +builds the driver's class-filtered engine list. + +## 9. Emulation checklist (derived from the above) + +For an unmodified CUDA 12.x userspace driver in a guest to reach a working +context, the host emulation must provide, at minimum: + +1. **Control device semantics** — version negotiation, client creation, GPU + attach, OS events, per-GPU fds. +2. **Object allocation** for the classes in §2, with plausible sizes/handles, + and successful `NV_ESC_RM_FREE`. +3. **Device/engine topology** — a consistent engine list and class list + (`GPU_GET_ENGINES`, `GPU_GET_CLASSLIST`, static info) matching what the + caps/controls claim; per-engine records must be self-consistent (see the + engine-descriptor discussion in `docs/05`). +4. **Memory** — VA spaces, vidmem/sysmem allocations with the requested + attributes, device and host mappings, BAR1 info. +5. **Channels** — GPFIFO alloc with notifier and USERD handles, engine object + alloc, scheduling, channel ids, doorbell tokens/offsets. +6. **Interrupt/event plumbing** — interrupt tree or usermode region, OS + events, error notifiers. +7. **Reply fidelity for consumed controls** — the small set of controls whose + *values* (not just status) the driver reads: memory sizes, engine ids, + class engine ids, BAR1 info, channel ids, version info, UVM caps. +8. **Push-buffer execution** — the GPU work encoded by userspace must actually + happen (memsets, copies, semaphore releases, kernel grids) or the driver + will hang in synchronization rather than fail fast. |