summaryrefslogtreecommitdiffstats
path: root/kernel/bar_emulation.c
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-22 23:32:44 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-22 23:32:44 -0400
commit14283fe73fb5e432a7af45de94961655714910f6 (patch)
treee993bf56f3e2ef93195019ac0fd996d82204384a /kernel/bar_emulation.c
parentb34e4531d8f0b984dc15660d3ab406ae48822931 (diff)
downloadopen-iov-14283fe73fb5e432a7af45de94961655714910f6.tar.gz
open-iov-14283fe73fb5e432a7af45de94961655714910f6.zip
kernel: FB/BUS info-list controls + PCIe link caps for the UVM path
CUDA (cuInit) now reaches UVM's device create: libcuda opens nvidia-uvm, UVM calls nvGpuOpsDeviceCreate/gpuGetGpuInfo, and the driver fails at getPCIELinkRateMBps -> calculatePCIELinkRateMBps ("Unknown PCIe speed", NV_ERR_INVALID_STATE) -> nv_open_device aborts -> cuInit returns CUDA_ERROR_NOT_INITIALIZED (3). Implemented along the way: - FB_GET_INFO_V2 (0x20801303): echo the requested FB info indices and answer TU104 values (256MB RAM/heap/BAR1, GDDR6, 8 LTCs, 4MB L2, ...). Fixes the kern_mem_sys_ctrl.c:911 asserts and gives CUDA/UVM real memory sizes. - BUS_GET_INFO_V2 (0x20801823): echo indices; PCIe Gen3 x16 link caps / GEN info / link status, SYSMEM_CONNECTION_TYPE=PCIE. UVM's gpuGetSysmemLinkInfo reads SYSMEM_CONNECTION_TYPE and the nominal link bandwidth from here. - PCIe Link Capabilities in the emulated config space (0x8C/0x92) and in the BAR0 PCFG mirror (gpuReadBusConfigReg -> NV_PCFG+0x84 = BAR0+ 0x88084): MAX_SPEED=3 (8GT/s Gen3), MAX_WIDTH=0x10 (x16); link status mirror at 0x88088. Also mirrored under the VF-window/low aliases. - Temporary change-watch on PCFG[0x88084] to see if the guest overwrites it (it does not - stays 0x103). Verified: guest nvidia-smi -L still works and reports PCIe Generation Max/Current = 3. cuInit still fails; the remaining question is which address the driver's kbifGetGpuLinkCapabilities read sees as zero (the PCFG shadow holds 0x103). Guest-side kprobe tracing of that function panicked the VM (free_swap_cache page fault on a corrupted pointer) - possible guest memory corruption to investigate separately.
Diffstat (limited to 'kernel/bar_emulation.c')
-rw-r--r--kernel/bar_emulation.c30
1 files changed, 30 insertions, 0 deletions
diff --git a/kernel/bar_emulation.c b/kernel/bar_emulation.c
index eaa006c..3448b8c 100644
--- a/kernel/bar_emulation.c
+++ b/kernel/bar_emulation.c
@@ -280,6 +280,36 @@ int nv_vgpu_bar_emul_init(struct nv_vgpu_device *vgpu)
*(u32 *)((u8 *)ctx->bar0_shadow + 0x100ce0) = 0x00000018;
/*
+ * NV_PCFG (BAR0+0x88000) - PCI config space mirror the driver reads
+ * via gpuReadBusConfigReg_HAL (GPU_BUS_CFG_RD32):
+ * 0x88084 NV_XVE_LINK_CAPABILITIES: MAX_LINK_SPEED 3:0 = 3 (8 GT/s,
+ * Gen3), MAX_LINK_WIDTH 9:4 = 0x10 (x16).
+ * 0x88088 NV_XVE_LINK_CONTROL_STATUS: negotiated speed 19:16 = 3,
+ * negotiated width 25:20 = 0x10 (x16).
+ * UVM's nvGpuOpsDeviceCreate -> getPCIELinkRateMBps ->
+ * kbifControlGetPCIEInfo -> kbifGetGpuLinkCapabilities reads 0x88084;
+ * zeros make calculatePCIELinkRateMBps log "Unknown PCIe speed" and
+ * return NV_ERR_INVALID_STATE, which fails cuInit
+ * (CUDA_ERROR_NOT_INITIALIZED in the guest).
+ *
+ * DEVICE_BASE(NV_PCFG) expands to 0 in this driver (nvmisc.h:
+ * "#define DEVICE_BASE(d) (0?d)"), so the read is actually
+ * gpuNvAddr + 0x84 - mirror the values at the low alias as well.
+ */
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0x88084) = 0x00000103;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0x88088) = 0x01030000;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0x00084) = 0x00000103;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0x00088) = 0x01030000;
+ /* Same values under the VF register window aliases, in case this
+ * driver reaches the config mirror through virtualRegPhysOffset. */
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xB80084) = 0x00000103;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xB80088) = 0x01030000;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xB88084) = 0x00000103;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xB88088) = 0x01030000;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xC08084) = 0x00000103;
+ *(u32 *)((u8 *)ctx->bar0_shadow + 0xC08088) = 0x01030000;
+
+ /*
* Serve a valid VBIOS at the PROM window (BAR0+0x300000).
*
* The guest driver's physical GSP boot (kgspInitRm -> TU102 path)