From 14283fe73fb5e432a7af45de94961655714910f6 Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Tue, 22 Sep 2026 23:32:44 -0400 Subject: kernel: FB/BUS info-list controls + PCIe link caps for the UVM path CUDA (cuInit) now reaches UVM's device create: libcuda opens nvidia-uvm, UVM calls nvGpuOpsDeviceCreate/gpuGetGpuInfo, and the driver fails at getPCIELinkRateMBps -> calculatePCIELinkRateMBps ("Unknown PCIe speed", NV_ERR_INVALID_STATE) -> nv_open_device aborts -> cuInit returns CUDA_ERROR_NOT_INITIALIZED (3). Implemented along the way: - FB_GET_INFO_V2 (0x20801303): echo the requested FB info indices and answer TU104 values (256MB RAM/heap/BAR1, GDDR6, 8 LTCs, 4MB L2, ...). Fixes the kern_mem_sys_ctrl.c:911 asserts and gives CUDA/UVM real memory sizes. - BUS_GET_INFO_V2 (0x20801823): echo indices; PCIe Gen3 x16 link caps / GEN info / link status, SYSMEM_CONNECTION_TYPE=PCIE. UVM's gpuGetSysmemLinkInfo reads SYSMEM_CONNECTION_TYPE and the nominal link bandwidth from here. - PCIe Link Capabilities in the emulated config space (0x8C/0x92) and in the BAR0 PCFG mirror (gpuReadBusConfigReg -> NV_PCFG+0x84 = BAR0+ 0x88084): MAX_SPEED=3 (8GT/s Gen3), MAX_WIDTH=0x10 (x16); link status mirror at 0x88088. Also mirrored under the VF-window/low aliases. - Temporary change-watch on PCFG[0x88084] to see if the guest overwrites it (it does not - stays 0x103). Verified: guest nvidia-smi -L still works and reports PCIe Generation Max/Current = 3. cuInit still fails; the remaining question is which address the driver's kbifGetGpuLinkCapabilities read sees as zero (the PCFG shadow holds 0x103). Guest-side kprobe tracing of that function panicked the VM (free_swap_cache page fault on a corrupted pointer) - possible guest memory corruption to investigate separately. --- kernel/bar_emulation.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) (limited to 'kernel/bar_emulation.c') diff --git a/kernel/bar_emulation.c b/kernel/bar_emulation.c index eaa006c..3448b8c 100644 --- a/kernel/bar_emulation.c +++ b/kernel/bar_emulation.c @@ -279,6 +279,36 @@ int nv_vgpu_bar_emul_init(struct nv_vgpu_device *vgpu) */ *(u32 *)((u8 *)ctx->bar0_shadow + 0x100ce0) = 0x00000018; + /* + * NV_PCFG (BAR0+0x88000) - PCI config space mirror the driver reads + * via gpuReadBusConfigReg_HAL (GPU_BUS_CFG_RD32): + * 0x88084 NV_XVE_LINK_CAPABILITIES: MAX_LINK_SPEED 3:0 = 3 (8 GT/s, + * Gen3), MAX_LINK_WIDTH 9:4 = 0x10 (x16). + * 0x88088 NV_XVE_LINK_CONTROL_STATUS: negotiated speed 19:16 = 3, + * negotiated width 25:20 = 0x10 (x16). + * UVM's nvGpuOpsDeviceCreate -> getPCIELinkRateMBps -> + * kbifControlGetPCIEInfo -> kbifGetGpuLinkCapabilities reads 0x88084; + * zeros make calculatePCIELinkRateMBps log "Unknown PCIe speed" and + * return NV_ERR_INVALID_STATE, which fails cuInit + * (CUDA_ERROR_NOT_INITIALIZED in the guest). + * + * DEVICE_BASE(NV_PCFG) expands to 0 in this driver (nvmisc.h: + * "#define DEVICE_BASE(d) (0?d)"), so the read is actually + * gpuNvAddr + 0x84 - mirror the values at the low alias as well. + */ + *(u32 *)((u8 *)ctx->bar0_shadow + 0x88084) = 0x00000103; + *(u32 *)((u8 *)ctx->bar0_shadow + 0x88088) = 0x01030000; + *(u32 *)((u8 *)ctx->bar0_shadow + 0x00084) = 0x00000103; + *(u32 *)((u8 *)ctx->bar0_shadow + 0x00088) = 0x01030000; + /* Same values under the VF register window aliases, in case this + * driver reaches the config mirror through virtualRegPhysOffset. */ + *(u32 *)((u8 *)ctx->bar0_shadow + 0xB80084) = 0x00000103; + *(u32 *)((u8 *)ctx->bar0_shadow + 0xB80088) = 0x01030000; + *(u32 *)((u8 *)ctx->bar0_shadow + 0xB88084) = 0x00000103; + *(u32 *)((u8 *)ctx->bar0_shadow + 0xB88088) = 0x01030000; + *(u32 *)((u8 *)ctx->bar0_shadow + 0xC08084) = 0x00000103; + *(u32 *)((u8 *)ctx->bar0_shadow + 0xC08088) = 0x01030000; + /* * Serve a valid VBIOS at the PROM window (BAR0+0x300000). * -- cgit v1.2.3