summaryrefslogtreecommitdiffstats
path: root/kernel
Commit message (Collapse)AuthorAge
* kernel: serve CE caps + CE->PCE masks (unblocks UVM channel manager)archive/pre-squashArthur Hanson Rasmusson2026-09-23
| | | | | | | | | | | | | | | | | | | | | | | kceGetDeviceCaps (kernel_ce.c) copies the caps the client hands to UVM from NV2080_CTRL_CMD_CE_GET_PHYSICAL_CAPS (0x20802a07), which reaches GSP-RM and was answered with zeros. With capsTbl = {0,0} no CE had the SYSMEM bit, so UVM's channel manager found no usable CE and UVM_REGISTER_GPU failed with NV_ERR_NOT_SUPPORTED ("Failed to initialize the channel manager", uvm_gpu.c:1574 - visible with uvm_debug_prints=1). New handlers: - CE_GET_PHYSICAL_CAPS (0x20802a07): {ceEngineType, capsTbl[2]} with SYSMEM_READ|SYSMEM_WRITE|SYSMEM|P2P|BL_GT_64K (0xEC) for COPY0..COPY4, NOT_SUPPORTED otherwise (matches subdeviceCtrlCmdCeGetCaps_IMPL). - CE_GET_ALL_CAPS (0x20802a0a) / CE_GET_ALL_PHYSICAL_CAPS (0x20802a0b): capsTbl[64][2] + present bitmap with the five CEs. - CE_GET_CE_PCE_MASK (0x20802a02): pceMask = BIT(ceIdx) (one PCE per LCE). Verified in-guest progress: UVM now gets past the channel manager and creates channels; the register path's new blocker is the first channel push (channel_init, uvm_channel.c:2535) whose semaphore release the emulated GPU must complete ("Tracker entry ... stuck waiting for 90s", sema VA 0x121000000 on channel 1:3 CE 0).
* kernel: serve GMMU static info + per-VF fault/access-counter sizes for UVMArthur Hanson Rasmusson2026-09-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Three fixes that clear all driver-visible errors in UVM's device create (cuInit still fails later in UVM_REGISTER_GPU): 1. GSP_RM_CONTROL 0x20800a59 INTERNAL_GMMU_GET_STATIC_INFO (new handler): kgmmu.pStaticInfo->replayableFaultBufferSize feeds kgmmuFaultBufferReplayableAllocate (called by faultbufConstruct when UVM opens the device). The generic zero reply made it a 0-page allocation: "faultbufConstruct_IMPL: Failed to setup Replayable Fault buffer (status=0x51)". Report 64KB replayable / 4KB legacy, mirroring kgmmuSetAndGetDefaultFaultBufferSize_TU102 defaults. 2. VF-window NV_VIRTUAL_FUNCTION_PRIV_MMU_FAULT_BUFFER_SIZE(0/1) (BAR0+0xB83010/0xB83030, plus plain aliases) pre-filled with _VAL in NVC369_BUF_SIZE units (0x800=64KB, 0x80=4KB) so the local (non-RPC) path reads the same values the control handler reports. 3. VF-window NV_VIRTUAL_FUNCTION_PRIV_ACCESS_COUNTER_NOTIFY_BUFFER_SIZE (BAR0+0xB83110, plus plain alias) = 0x1000 (the value the host's physical TU104 reports; x32B = 128KB). uvmGetAccessCounterBufferSize_ TU102 multiplied our zero and uvmInitializeAccessCntrBuffer failed with NV_ERR_NO_MEMORY at mem_desc.c:1353. Also note (companion finding, no code change): the guest must be placed behind a pcie-root-port; on the Q35 root bus QEMU rewrites the PCIe cap to Root Complex Integrated Endpoint and zeroes LNKCAP/LNKCTL/DeviceCaps (vfio_setup_pcie_cap: "On a Root Complex bus Endpoints become Root Complex Integrated Endpoints, which changes the type and clears the LNK & LNK2 fields"), which is what made getPCIELinkRateMBps read speed 0.
* kernel: drop PCFG poison experiment and config-write spam, keep bus-info ↵Arthur Hanson Rasmusson2026-09-23
| | | | | | | | | | | | | | | | detail log The poison test (fill every dword of BAR0 0x88000-0x88FFF, 0xB80000- 0xB80EFF, 0xC08000-0xC08EFF with 0x103) did not change the CUDA "Unknown PCIe speed" failure, proving the failing BUS_GET_INFO_V2 index-3 data does not come from any BAR0 mirror read. Combined with the QEMU config-space behavior (link registers zeroed, type=0x92, 0x84 never read from us) the value must come from the VGPU_STATIC_INFO branch in getBusInfos (pVSI-> pcieGpuLinkCaps), i.e. the guest needs the host vGPU static-info payload. Cleanup: remove the poison loops and the per-write CFGWR log; keep the once-per-init PCIe cap print and add per-entry BUS_GET_INFO_V2 logging (index+data) for the ongoing CUDA/UVM work.
* kernel: real TU104 config-space layout (PCIe cap at 0x78) + CUDA/UVM tracingArthur Hanson Rasmusson2026-09-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | Move the emulated PCIe capability to config offset 0x78 and shift MSI to 0x48 / MSI-X to 0x68, matching the real TU104 (host lspci: PM 0x60, MSI 0x68, Express 0x78). This matters because the driver's NV_PCFG mirror (BAR0+0x88000, size 0x1000) is read as PCI config space: osDevReadReg032 -> vgpuDevReadReg032 intercepts [NV_PCFG, +0x30) (or the full 0x1000 for SRIOV) and forwards to osPciReadDword, so NV_XVE_LINK_CAPABILITIES (0x84) must be the PCIe Link Capabilities register, i.e. cap+0x0C with the cap at 0x78. Link Caps = Gen3 x16 (0x103), Link Status = 0x0103. Also set MCFG-visible caps: the guest now reports MSI [48], MSI-X [68], Express [78] exactly like the host GPU. Fixing the capability chain: the first attempt set the PCIe cap's NEXT to 0x0C, which broke enumeration (device 00:06.0 disappeared from the guest) - NEXT must terminate at 0. Unknown still open: QEMU presents type=0x92 (Root Complex Integrated Endpoint) and zeroes 0x84-0x8B in its config cache (never read from us, not refreshed by writes), and poisoning all BAR0 mirror windows with 0x103 does not change the driver's "Unknown PCIe speed" - so the value comes from the VGPU_STATIC_INFO branch (pVSI->pcieGpuLinkCaps, IS_VIRTUAL true via hypervisor detection) rather than any register read. Debug helpers added (temporary): CFGWR config-write log, PCFG change watch, PCIe-cap print at init.
* kernel: widen PCFG mirror watch to link-cap aliases (debug helper)Arthur Hanson Rasmusson2026-09-22
| | | | | | | | Logs changes to NV_PMC_BOOT_1 (0x4), the low PCFG alias, the BAR0 PCFG block (0x880000/0x880080/0x880084/0x880088/0x88008c) and the VF-window aliases. Observed during a CUDA run: the guest never writes any of them, so the link-cap values we pre-set (0x103 / 0x01030000) are not clobbered. Reads are not observable through the mmap'd BAR0.
* kernel: FB/BUS info-list controls + PCIe link caps for the UVM pathArthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CUDA (cuInit) now reaches UVM's device create: libcuda opens nvidia-uvm, UVM calls nvGpuOpsDeviceCreate/gpuGetGpuInfo, and the driver fails at getPCIELinkRateMBps -> calculatePCIELinkRateMBps ("Unknown PCIe speed", NV_ERR_INVALID_STATE) -> nv_open_device aborts -> cuInit returns CUDA_ERROR_NOT_INITIALIZED (3). Implemented along the way: - FB_GET_INFO_V2 (0x20801303): echo the requested FB info indices and answer TU104 values (256MB RAM/heap/BAR1, GDDR6, 8 LTCs, 4MB L2, ...). Fixes the kern_mem_sys_ctrl.c:911 asserts and gives CUDA/UVM real memory sizes. - BUS_GET_INFO_V2 (0x20801823): echo indices; PCIe Gen3 x16 link caps / GEN info / link status, SYSMEM_CONNECTION_TYPE=PCIE. UVM's gpuGetSysmemLinkInfo reads SYSMEM_CONNECTION_TYPE and the nominal link bandwidth from here. - PCIe Link Capabilities in the emulated config space (0x8C/0x92) and in the BAR0 PCFG mirror (gpuReadBusConfigReg -> NV_PCFG+0x84 = BAR0+ 0x88084): MAX_SPEED=3 (8GT/s Gen3), MAX_WIDTH=0x10 (x16); link status mirror at 0x88088. Also mirrored under the VF-window/low aliases. - Temporary change-watch on PCFG[0x88084] to see if the guest overwrites it (it does not - stays 0x103). Verified: guest nvidia-smi -L still works and reports PCIe Generation Max/Current = 3. cuInit still fails; the remaining question is which address the driver's kbifGetGpuLinkCapabilities read sees as zero (the PCFG shadow holds 0x103). Guest-side kprobe tracing of that function panicked the VM (free_swap_cache page fault on a corrupted pointer) - possible guest memory corruption to investigate separately.
* kernel: guest probe completes - nvidia-smi enumerates the emulated TU104Arthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Root cause of the golden-image NV_ERR_NO_MEMORY: STATIC_KGR_GET_CONTEXT_ BUFFERS_INFO (0x20800a32) fell through to the generic zeroed reply, so kgraphicsInitializeDeferredStaticData() stored a 0-byte main ctx buffer size and kgrctxAllocMainCtxBuffer() did a 0-page allocation (mem_desc.c:1353). Serve the TU104 per-engine ctx buffer table (26 engine IDs x 8 GR engines; NV_U32_MAX = unsupported, which the client skips). Second blocker after that: L2 cache-op and TLB-invalidate completion registers. Turing reaches them through the virtual-register window (GPU_VREG_* = BAR0 + sriovState.virtualRegPhysOffset + offset, 0xB80000 for a non-SRIOV virtual GPU): NV_VIRTUAL_FUNCTION_PRIV_MMU_INVALIDATE (0xB830B0, TRIGGER bit31) and the L2 invalidate/dirty-flush set (0x70000-0x70014 and 0xB80F00/0xB80F04). The driver soft-locked spinning in osDevReadReg032 on 0xB830B4 (kgmmuCheckPendingInvalidates_TU102) and timed out in kmemsysDoCacheOp_GM107; auto-clear all of them (both the plain and windowed aliases). Third: re-probe failed with GspStatusQueueInit msgqRxLink timeout because gsp_rm_emu_reset() was not called before a fresh boot, leaving rm->parsed set so the new libos args were never parsed. Reset GSP-RM state on the fresh FWSEC kick. Verified: guest nvidia-smi and nvidia-smi -L both exit 0 (full unload/reload cycle), GPU enumerated as "NVIDIA GeForce RTX 2080 SUPER" with 256MB, no NO_MEMORY / timeouts / soft lockups in the guest. Also add tools/rmq/grctx_oracle.c documenting that the host RM does not expose the ctx-buffer table to userspace (the real values are probed from the ctxsw ucode by GSP-RM), so the emulation serves synthesized sizes.
* BAR2 verify test fix: remove PRAMIN sync + contiguous ALLOC + vidmem VA ↵Arthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | | allocator Fixes the BAR2 verify test (kbusVerifyBar2_GM107): 1. Removed PRAMIN window sync from emulated BAR1 - the sync was overwriting test data written by the driver through the vBar2 aperture. The shadow BAR0 is already consistent (guest writes/reads hit the same shadow location). 2. Added contiguous page allocation for golden image (alloc_pages with order for large allocations, falling back to individual pages). 3. Added vidmem VA allocator - ALLOC now returns GPU VAs within the emulated GPU's vidmem range (0x100000 - fb_size) instead of default 4GB, so the memmgr accepts them. Verified: driver probes past RmInitNvDevice bus init, reaches golden image channel allocation (progress from fn-10 FREE timeout).
* PRAMIN window translation + queue state preservation on Booter-unloadArthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | Fixes the BAR2 verify test failure (kbusVerifyBar2_GM107): - GSP emulation thread now watches NV_PBUS_BAR0_WINDOW (BAR0+0x1700) for base/target changes and syncs the PRAMIN window (BAR0+0x700000) from the emulated BAR1 (vidmem) at (base << 16). This makes PRAMIN reads return correct vidmem content for the BAR2 verify test. Fixes the fn-10 FREE timeout during driver cleanup: - gsp_emu_clear_boot_state no longer resets the GSP-RM queue state on Booter-unload kick. The driver sends cleanup FREEs AFTER the Booter-unload kick; resetting the queue here caused those frees to time out. - Queue state is now only reset on: ENGINE reset (full GPU reset) or genuine queue death (insane writePtr). Both call gsp_rm_emu_reset explicitly. Also fixes rmq tool ioctl encoding: - Use _IOWR('F', nr, type) encoding for NVIDIA ioctls (was raw numbers) - Open /dev/nvidiactl (not /dev/nvidia0) for RM_ALLOC - Add NVOS64_PARAMETERS (48 bytes) for device/subdevice allocs Verified: BAR2 verify test passes, INSTLOC regkey read, driver probes past RmInitNvDevice bus init, reaches fn-10 FREE timeout during cleanup (same as handoff's characterized blocker).
* host stability: BAR3 shadow + skip submodule nvidia + teardown-safe garbage ↵Arthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | | | | handler Root causes of host GSP crash (WPR2 stuck, 'unexpected WPR2 already up'): 1. stack_manager load-nvidia tried to load our submodule nvidia.ko (610.57.04-base) alongside/over apt nvidia 570.211.01 -> GSP crash during driver conflict. Fix: skip submodule when apt nvidia is already loaded (not needed for QEMU). 2. BAR3 (32MB instance memory window) was mapped to the physical GPU's real BAR3, so guest BAR3 writes could corrupt the host GPU's instance memory. Fix: vzalloc'd shadow buffer (like BAR1 emulated vidmem), per-page mmap. 3. GSP-RM garbage element handler declared GSP death on teardown queue garbage, preventing driver cleanup frees from completing. Fix: stop polling quietly, let Booter-unload kick handle reset. Verified: full lifecycle (load, mdev create, QEMU guest boot, guest probe, mdev remove, unload, reload) with host nvidia-smi (apt 570.211.01) stable throughout. No GSP crash, no WPR2 stuck, no module unload hang.
* kernel: dedicated host-RAM vidmem buffer for BAR1 (not host GPU's real VRAM)Arthur Hanson Rasmusson2026-09-22
| | | | | | | Guest VRAM (BAR1) was mapped to the host GPU's real BAR1 (managed by nvidia.ko) - guest VRAM accesses hit the broken host GPU. Now a dedicated 256MB vzalloc buffer backs guest VRAM; BAR1 mmap uses per-page vmalloc_to_pfn (like BAR0 shadow). vfree on cleanup.
* kernel: revert fb to 256MB (512MB broke FWSEC WPR2-location check)Arthur Hanson Rasmusson2026-09-22
| | | | | | | | The FB=512MB change made the FWSEC WPR2-location check fail ('WPR2 initialized at an unexpected location: 0x1fe00 (expected 0xfe00)') because NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE must stay consistent with GSCI fb_length. Reverted to 256MB everywhere. The golden-image GR context buffer NO_MEMORY is a heap free-list issue, not fb size.
* kernel: floorsweeping masks + interrupt tree + WPR2 race + fn-10 handlerArthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | | | | | | - STATIC_KGR_GET_FLOORSWEEPING_MASKS(0x20800a26): TU104 gpcMask=0x3f (6 GPCs), 4 TPC/GPC. numGpcs==0 killed krcWatchdogInit -> RmInitAdapter bailed ('numGpcs > 0' assert at kernel_graphics_object.c:191). - CPU interrupt tree (osVerifyInterrupts): driver accesses VF window at virtualRegPhysOffset=NV_VIRTUAL_FUNCTION_FULL_PHYS_OFFSET(0xB80000)+off, NOT 0x1640 (was watching wrong offset). On LEAF_TRIGGER write, set LEAF/TOP pending bits + inject MSI-X so the sanity-test ISR runs - osVerifySystemEnvironment now passes. - WPR2 race: clear boot state BEFORE CPUCTL=HALTED on Booter-unload kick so WPR2 is down when driver checks kgspIsWpr2Up ('WPR2 is still up'). - Dedicated GSP_RM_FREE(10) handler: echo NVOS00_PARAMETERS + status=NV_OK (proper free_v03_00 form). fn-10 timeout persists despite this - characterized as driver-side reply-matching issue, not content.
* kernel: fn-10 delay tuned to 2ms (partial mitigation, not full fix)Arthur Hanson Rasmusson2026-09-22
| | | | | | | | | | | 2ms udelay before fn-10/47 replies. Mitigates but does not fully eliminate the intermittent fn-10 FREE timeout (driver-side reply- matching issue). Verified reply path correct: seqNum matches driver's rxSeqNum (poison experiment 'Expected 75 got 175'), checksum folds to 0, function=10, writePtr/readPtr land, driver drains 1:1. Root cause of the driver's intermittent non-acceptance of fn-10/47 replies during cleanup remains under investigation (needs real-GSP oracle or guest- kernel GDB inspection of the msgq reply-matching).
* kernel: CrashCat wayfinder suppression (prevent kgspHealthCheck loop)Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | kgspHealthCheck runs on every RPC drain; if the GSP falcon's DEBUGINFO register (BAR0+0x110094) holds a valid CrashCat wayfinder-L0 signature (0xdead), crashcatEngineGetNextCrashReport reads 'reports' from garbage and loops forever in nv_open_q (which then can't poll RPCs -> fn-10 timeouts). Force the wayfinder invalid - our emulated GSP has no crashcat reports to offer.
* kernel: STATIC_KGR_GET_INFO handler - TU104 litter topology + maxSubcontext=64Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | Driver caches the GR infoList as pGrInfo and reads MAX_SUBCONTEXT_COUNT (0x2c) - zero made channel-group numMax==0, asserting channel allocation (numMax == numFree && numMax != 0 -> NV_ERR_INVALID_STATE, blocking the per-device open -> NVML 'No devices'). Fill litter topology (6 GPC x 4 TPC x 2 SM = 48 SMs) and subcontext counts = 64 (NV_PGRAPH_PRI_FE_CHIP_DEF_INFO_MAX_VEID_COUNT_INIT).
* kernel: fn-10 timing-race mitigation (2ms reply delay) + remove seqNum-poisonArthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | The fn-10 FREE timeout is a driver-side reply-matching TIMING RACE, not a content bug. seqNum-poison experiment PROVED the driver's rxSeqNum matches our stat_seq exactly ('Expected 75 got 175'); checksum folds to 0; function=10; writePtr/readPtr land; driver drains 1:1. A 2ms udelay before fn-10/47 replies eliminates timeouts in most runs (matches real GSP-RM's natural reply latency). Removed the seqNum-poison diagnostic (served its purpose) and fn10 wire-bytes debug dumps. Race is not 100% eliminated - intermittently recurs during unload/cleanup cycles.
* kernel: fn-10 FREE timeout fixed - reply timing race (delay fn-10/47 replies)Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | The fn-10 (FREE) timeout was a driver-side reply-matching TIMING RACE, NOT a reply-content bug. Verified exhaustively: reply is well-formed, checksum valid (folds to 0), seqNum matches driver's rxSeqNum exactly ('Expected 75 got 175' via seqNum-poison experiment), function=10, writePtr/readPtr land correctly, driver drains 1:1 (no lapping). Yet rpcRecvPoll timed out only for fn-10/47-class RPCs during open/ cleanup. A 2ms delay before fn-10/47 replies eliminates all timeouts (0 in a full nvidia-smi run) - matches real GSP-RM's natural reply latency that the driver's reply-matching implicitly relies on. Also: fn10 reply wire-bytes debug dump, writePtr readback confirm, seqNum-poison experiment scaffolding.
* kernel: TLB invalidate auto-complete (MMU_INVALIDATE trigger) + fn-10 queue ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | logging - NV_VIRTUAL_FUNCTION_PRIV_MMU_INVALIDATE (BAR0+0x30B0, TRIGGER=bit31): driver sets TRIGGER then spins in kgmmuCheckPendingInvalidates_TU102 polling for it to clear. Our shadow held TRIGGER forever -> nv_open_q hard-wedged in RmInitAdapter (TLB invalidate during dmaAllocMapping). Auto-clear the trigger bit in gsp_emu_poll_boot (instant completion). - fn-10/47 handler logs hObjectOld + cmd_read/stat_write/drv_rp/stat_seq. Confirmed: drv_rp tracks stat_write 1:1 (driver drains cleanly, no lapping); frees complete; stat_seq increments per record. - Health-check/crashcat loop (kgspHealthCheck -> crashcatEngineGetNext CrashReport never returns NULL) is a SECONDARY symptom triggered by an RPC failure during RmInitAdapter - under investigation.
* kernel: Fix guest memory corruption from non-page-aware queue I/O + fn-10 ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | logging ROOT CAUSE (guest userspace segfaults): the GSP-RM msgq buffer is NV_MEMORY_NONCONTIGUOUS (129 independent 4K pages). gsp_rm_send wrote multi-element records CONTIGUOUSLY from a translated start GPA - a record crossing a page boundary scribbled into arbitrary guest RAM (pages are unrelated GPAs). Observed as pgrep/ls segfaulting in the guest. - gsp_rm_write_buf/gsp_rm_read_buf: split every shared-buffer access at page boundaries and translate each page via the buffer page table (gsp_rm_off_to_gpa). All queue record/element/header accesses converted. - fn-10 (FREE) / fn-47 (UNLOADING) handler now logs hClient/hObject + cmd_read for unload-cycle debugging. Diagnosis captured: driver sends UNLOADING(47) then final FREEs during teardown; queue-death detector can strand the last frees (fn-10 timeout blocks close -> nvidia-smi hangs in exit).
* kernel: Booter-Unload success + SM order table (unload/reload cycle works)Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | - gsp_emu_poll_boot generic SEC2 kick path: clear SEC2_MAILBOX0=0 (success) - Booter Unload during GPU shutdown read a stale 0xff, failing the unload and killing the subsequent re-init ('Booter failed with non-zero error code: 0xff' -> RmInitAdapter 0x40). A SEC2 kick after a completed boot is a GSP shutdown/reload: clear boot state. Driver unload/reload cycle (last close -> RmShutdown, next open -> RmInitAdapter) now works. - STATIC_KGR_GET_GLOBAL_SM_ORDER(0x20800a22): TU104 topology 6 GPC x 4 TPC x 2 SM = 48 SMs/24 TPCs on GR0. kgrStaticInit is init-critical (NV_CHECK_OK_OR_GOTO); zeros left numSm=0 and NVML spun on empty topology.
* kernel: msgq writePtr wrap semantics + retry backoff + CE fault buffer sizeArthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | | CRITICAL protocol fix (msgq.c): writePtr/readPtr are slot indices that WRAP at msgCount ('writePtr += n; if (>= msgCount) -= msgCount'), NOT monotonic counters. '62 -> 0' is a legitimate wrap, not queue death. - gsp_rm_poll_commands: consume while (cmd_read != writePtr), advance cmd_read = (cmd_read + elem_count) % msgcount. Death detection now only: writePtr >= msgCount (garbage header) / bad element signature / insane elemCount. Removes false deaths on healthy wraps that killed the emulator mid-session (fn 10 FREE timeouts). - gsp_rm_send: published status writePtr now wraps too (driver's free-space/slot math breaks on unbounded values >= 2*msgCount). - gsp_rm_emu_poll: back off on_booter_done retries to 250ms (was every 100us poll -> thousands of 'will retry' lines/sec, dmesg unusable); retry log now pr_warn_ratelimited. - CE_GET_FAULT_METHOD_BUFFER_SIZE(0x20802a08) = 12288 (GV100 formula inherited by Turing: 135*(13*5+2) page-aligned, 5 PCEs). Zero reply failed fault method buffer allocation (NV_ERR_NO_MEMORY) killing channel-group creation.
* kernel: INTR_GET_KERNEL_TABLE subtreeMap = intrInitSubtreeMap_TU102 valuesArthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | Driver's intrCacheIntrFields requires subtree(ACCESS_CNTR=72)=1 within subtreeMap[UVM_OWNED] and UVM_SHARED to be a single subtree. Zeroed map failed RmInitNvDevice state load ('unexpected subtree'). Map matches intrInitSubtreeMap_TU102 (what physical-path guests get from real GSP-RM): ESCHED{3,3} ESCHED_NOTIF{0,0} UVM_OWNED{1,1} UVM_SHARED{2,2}, DEFAULT/RUNLIST/RUNLIST_NOTIF=INVALID(0xFF). Struct retyped to 7x2B; out_params_size spans full struct (driver reads subtreeMap at offset 2052). Verified: probe passes state load, /dev/nvidia0 persists.
* kernel: GSP-RM handlers - ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | NUM_CHANNELS/PCI_BAR_INFO/DISP_STATIC_INFO/DISP_IP_VERSION/I2C - FIFO_GET_NUM_CHANNELS(0x20800a61)=64/runlist: zero tripped NV_ASSERT(numChannels>0), kdisp clientChannelTable alloc failed. - BUS_GET_PCI_BAR_INFO(0x20801803): report BAR0=16MB/BAR1=256MB/ BAR2=1MB instmem window@BAR0+0x700000/BAR3=32MB (was all zeros). - DISPLAY_GET_STATIC_INFO(0x20800a01): heads=1, numDispChannels=1 (36B wire layout; zero -> clientChannelTable[0] alloc failed). - DISPLAY_GET_IP_VERSION(0x20800a4b)=0x04000000 (Turing DISP v04.00; driver passes uninitialized input - must return a real version). - I2C_TRANSACTION(0x402c0105): echo request params + NOT_SUPPORTED. Variable-length FINN-serialized params: zeroed reply fails DeserializeUp and the calling ioctl never completes (nvidia-smi hang). - Default unknown-cmd policy back to zeroed params (echo is dangerous for queries with uninitialized input structs). Milestone: guest RmInitAdapter completed, /dev/nvidia0 created.
* kernel: Fix FIFO table fortify_panic + GSP boot-state reset on queue death + ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | ISO fbRegion - gsp_rm_handle_fifo_device_table: read baseIndex from the queue element buffer (req_data + hdr), not past the local 24-byte header struct - FORTIFY proved the old memcpy always overflows -> fortify_panic killed gsp_emu_0 mid RmInitNvDevice (guest timed out on fn 76). - gsp_emu_clear_boot_state: clear WPR2/RISCV/mailboxes/stage tracker and reset GSP-RM queue state. Turing guests produce no observable BAR0 writes on unload, so the next probe's _kgspBootGspRm failed with 'unexpected WPR2 already up'. Trigger on ENGINE reset and on detected queue death (insane writePtr jump / garbage element signature). - GspStaticConfigInfo fbRegion[0].supportISO=1: memmgrCalcReservedFbSpace asserts some region supports ISO 'even with no display'; without it kdispStateInitLocked couldn't allocate clientChannelTable. - .gitignore: scripts/git-push-vgpu.sh (local per-machine helper)
* kernel: Fix gsp_emu_cleanup crash on dead thread (refcount saturation)Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | Root cause: when the gsp_emu thread crashes (e.g. fortify_panic in gsp_rm_send), the kernel kills it. Later, gsp_emu_cleanup() calls kthread_stop() on the dead thread, hitting refcount_warn_saturate on the freed/saturated task_struct. Fixes: 1. pid_alive() check before kthread_stop() - skip if thread already dead 2. get_task_struct() after kthread_run() - hold reference so the task_struct stays valid even after a crash 3. put_task_struct() in cleanup - release the reference This prevents the module from getting stuck in 'going' state during mdev removal after a gsp_emu thread crash.
* kernel: Multi-element msgq support + USER_REGISTER_ACCESS_MAP handlerArthur Hanson Rasmusson2026-09-21
| | | | | | | | | - gsp_rm_send: Split large RPC replies across multiple queue elements (fixes fortify_panic when reply > 4016 bytes RPC data) - gsp_rm_build_record: Build multi-element records with proper elemCount, cross-element data continuation, wrap-around handling - Add INTERNAL_GPU_GET_USER_REGISTER_ACCESS_MAP (0x20800a41) handler returning empty register access map (8204 bytes)
* kernel: GSP-RM msgq emulation + TU104 hardware tables (GET_GSP_STATIC_INFO, ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GSP_RM_CONTROL) Implements the GSP-RM reply content layer for the msgq RPC channel: - kernel/gsp_rm_emulation.c/h: GSP-RM msgq emulation engine - Queue chain discovery (libos args → RMARGS → GSP_ARGUMENTS_CACHED) - Status queue link + GSP_INIT_DONE emission - Command queue polling + RPC dispatch - SET_GUEST_SYSTEM_INFO version negotiation (VGX 41.12) - GET_GSP_STATIC_INFO(65) with 1656-byte GspStaticConfigInfo - GSP_RM_CONTROL(76) sub-command dispatch: - FIFO_GET_DEVICE_INFO_TABLE (0x20801112) - INTERNAL_INTR_GET_KERNEL_TABLE (0x20800a5c) - GPU_GET_CONSTRUCTED_FALCON_INFO (0x208001b0) - INTERNAL_GET_DEVICE_INFO_TABLE (0x20800a40) - kernel/gsp_rm_tu104.h: TU104 hardware tables - 12-entry FIFO engine table (GR0, CE0-4, NVDEC0-1, NVENC0, SEC2, NVJPG, SW) - 8-entry interrupt table (from intr_phys_tu102.c) - 1-entry constructed falcon table (FBFLCN) - 11-entry internal device info table - GspStaticConfigInfo layout (1656 bytes, ABI-verified offsets) - Data from NVIDIA driver material (ext-tree) + hwref/turing/tu104 - tools/rmq/: Userspace RM API client (oracle for ground-truth tables) KNOWN ISSUE: gsp_rm_send crashes (fortify_panic) on multi-element messages when reply data exceeds one 4096-byte queue element. Affects cmd 0x20800a41 (paramsSize=8204). Multi-element splitting is the next fix.
* kernel: Present physical TU104 to guest + emulate GSP boot (FWSEC/Booter) + ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | VBIOS serving Guest driver probe fixes (unmodified 570.144 open driver on physical path): - kernel/main.c: add show_description to the mdev driver. Userspace mdev management tools parse 'framebuffer=<n>M' from every mdev type's description sysfs file before creating an mdev; the missing file made such tools fail to create the vGPU. Also fix create_mdev.sh for the real type dir name pci-nvidia-vgpu-* (mdev core names dirs dev_driver_string(parent)-<sysfs_name>; the parent registers before nvidia.ko binds so dev_driver_string == 'pci'). - kernel/bar_emulation.c: seed BAR0 shadow so guest sees a REAL, PHYSICAL, non-virtual TU104 (user directive: no PV/VF vGPU paths, unmodified driver): * PMC_BOOT_0/1/42 = physical TU104 values captured from host hardware (0x164000a1 / 0x0 / 0x164a1000). Without these, halmgrGetHalForGpu() fails and probe aborts with 'does not include the required GSP'. * Serve a valid VBIOS at the PROM window (BAR0+0x300000) loaded from /lib/firmware/tu104_vbios.bin (dump of host TU104 ROM). Guest's kgspExtractVbiosFromRom_TU102 needs it for the mandatory FWSEC ucode (kgspGetFrtsSize is hardwired to 1MB on Turing). * Pre-set GSP secure-boot (GFW) completion state: PGC6 PLM (0x118128)=1, GFW_BOOT progress (0x118234)=0xFF, GSP falcon CPUCTL HALTED (0x110100). The driver's post-FLR GFW wait only *waits* (never kicks), so the pre-set state satisfies it immediately. - kernel/gsp_emulation.c/h: physical GSP boot emulation. * Falcon boot-watch: poll GSP falcon/PGC6/WPR2/scratch registers and log changes (guest BAR0 is mmap-based, otherwise invisible to host). * gsp_emu_poll_boot: STARTCPU auto-complete state machine. Guest kicks FWSEC then Booter Load via GSP falcon CPUCTL; detect each kick and complete the stage (HALTED + FRTS err NONE + WPR2 + mailbox + RISCV-active) so the guest believes its ucodes ran. Diagnosis that led here (documented in Handoff-vGPU-IO.md): - Open nvidia.ko can never be an SR-IOV VF vGPU guest: os_is_grid_supported() is a stub returning FALSE in the open build (disassembled). PV path also rejected per user directive. Only the physical path with a fully emulated local GSP boot remains.
* kernel: Fix GSP emulation shadow binding, eventfd wiring, RPC livelocks + ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | docs suite + repo reorg CRITICAL FIXES (guest driver bring-up): - gsp_emulation.c: emulation engine bound to the WRONG BAR0 - it polled the host physical BAR0 (bars[0].vaddr ioremap) for the doorbell and read RPCs from a local dma_alloc_wc buffer, while the guest writes land in the vzalloc'd bar0_shadow. The RPC path could never work; PBA writes even corrupted the host GPU's physical BAR0+0x3000. gsp_emu_init now binds exclusively to bar0_shadow (doorbell/mailbox/PBA aliases; shared_mem_dma==0 marks the alias so cleanup never dma_free's it). - main.c SET_IRQS: DATA_EVENTFD eventfds were DISCARDED ('TODO' + kfree) - the guest never received MSI-X completions. Full wiring now: eventfd_ctx_fdget per vector, fd -1 clears, dev_close releases refs. gsp_emu_inject_completion_interrupt looks up vgpu->irqs[0].trigger LIVE (init-time copy was permanently NULL since QEMU wires after gsp_emu_init). - gsp_rpc_client.c: BAR_TRAP/MIGRATE in emulation mode called gsp_rpc_submit_raw which rings the SAME doorbell the thread polls - self-triggered RPC livelock + multi-second stalls. Now fail fast with -ENOTSUPP + pr_warn_ratelimited (guest BAR access is direct MMIO). MORE FIXES: - main.c: nv_vgpu_config_write returned SIZE_MAX (while(count--) wrap) and was dead code while the VFIO write handler did unfiltered byte copies (guest could rewrite vendor ID/BARs). Return fixed; VFIO write handler now routes through the per-register filter. - kernel/Makefile: M=$(PWD) used the env var which is stale when invoked via the top-level make (pointed at repo root -> kbuild silently built NOTHING; explains every 'stale .ko' incident). Now M=$(CURDIR). - Warnings sweep: header prototypes for mdev/type functions, format casts in gsp_comm.c DRY-RUN logs, unused variable. ROBUSTNESS (harness/tooling): - vgpu_vm.py: run had no timeout (start could block forever); guest_shell recv could block forever on a wedged guest. Both now bounded; guest_shell pre-flights guest-ping and reports wedge diagnostics. - stack_manager.sh: qemu/qemu-debug now launch via the vgpu_vm.py harness (workflow rule: never raw QEMU). REPO REORGANIZATION: - driver/ (legacy guest-side gsp-vgpu.ko) moved to legacy-driver-guest/ (reference only, not built; README explains). 3 legacy un-buildable test files moved to legacy-driver-guest/kernel-tests/. Top-level Makefile cleaned of the dead DRIVER_DIR reference. DOCUMENTATION SUITE (new): - docs/GSP_EMULATION_ARCHITECTURE.md - emulator deep-dive: shadow BAR0 model, doorbell protocol, dispatch table, MSI-X path, lifecycle, memory- model caveats - docs/DEBUGGING.md - QEMU GDB stub workflow, serial console, QMP NMI, failure taxonomy, full bring-up validation checklist, EPT-livelock case study, version-matrix discipline - docs/ONBOARDING.md - repo tour, coding standards (incl. virt_to_phys / dma_addr_t / get-put rules), first build & test - docs/VERSION_UPGRADE.md - end-to-end new-driver-version procedure (firmware fetch, Ghidra analysis, configs, signatures, validation) - docs/PERFORMANCE.md - bare-metal A/B methodology + targets - README.md doc index; STATE.md refreshed; DEVELOPMENT_WORKFLOW.md corrected (load order: open-iov-vfio BEFORE nvidia; stock-driver blacklist note)
* kernel: Fix guest-hang root cause (VFIO mmap garbage PFN) + rebuild mdev ↵Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | remove path VFIO mmap (main.c): virt_to_phys() was used on vmalloc-family addresses (vzalloc'd BAR0 shadow, ioremap'd BAR2/3), producing garbage PFNs. Guest BAR0 access then hit a KVM make_spte reserved-bits WARN and an EPT-violation livelock, hard-wedging the guest on every nvidia driver probe (confirmed via QEMU GDB stub). BAR0 now maps per-page via vmalloc_to_pfn(); BAR2/3/4/5 map bars[N].gpa directly (like BAR1); config space uses new nv_vgpu_mmap_virt_to_pfn() helper. mdev remove path (main.c): old path never list_del'd the device (available_instances leaked), never ran cleanup (leaked gsp_emu kthread), never unregistered VFIO (leaked /dev/vfio/N + iommu group), never put pdev, never freed the struct. New kvmgt-style path: cancel init_work -> wait fds -> vfio_unregister_group_dev -> cleanup work -> list_del -> pci_dev_put -> vfio_put_device. gsp_rpc_client.c: fix dma_map_page dma_addr_t->int truncation in gsp_rpc_alloc_emul (corrupts >=4GB IOVAs); export gsp_client_init_emulation. gsp_emulation.c: export gsp_emu_process_rpc and gsp_emu_get_stats. main.c: remove wrong MODULE_SOFTDEP("pre: nvidia") - module has zero nvidia imports, loads standalone, and must load BEFORE nvidia.ko (it exports nvidia_vgpu_vfio_* for nvidia.ko to import). nvidia-vgpu-vfio.h: NULL-safe nv_vgpu_{dbg,info,warn,err} macros. tests/gsp_emulation_test.c: rewritten against current API - drives gsp_emu_process_rpc() with crafted shared-memory messages (protocol validation, SCHEDULE/YIELD/NOTIFY/DEBUG dispatch, stress). Builds with KBUILD_EXTRA_SYMBOLS; depends: open-iov-vfio. stack_manager.sh: vfio_mdev doesn't exist on >=6.4 (probe per-module); load-nvidia detects foreign nvidia via /sys/module/nvidia/version and errors with guidance instead of silently skipping.
* kernel: Add get_available mdev callback (fix leaked available_instances)Arthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | The mdev core's shared parent->available_instances counter leaks whenever an mdev remove path fails to complete (as happened during the now-fixed remove crash): creates then fail with -EUSERS while sysfs reports 0 available instances with zero mdevs present. Implement mdev_driver.get_available to compute availability from the driver's own authoritative device list instead of the core counter. Handles the 'nvidia-' prefixed sysfs type names when looking up per-type max_instances (falls back to 1).
* Make repo fully path-portable and add reproducible installerArthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | | | | | | | | | - install.sh: one-command fresh-Ubuntu reproduction (deps, NVIDIA clone + patch, GSP firmware fetch from linux-firmware.git with objcopy .fwimage extraction + MD5 verify, ordered build, host config, load, mdev creation). Wrapper-neutral: works with libvirt/raw QEMU. - patches/nvidia: regenerated path-independent patch (KBUILD_EXTRA_SYMBOLS resolved via $(src)/$(CURDIR) relative to the NVIDIA tree). - scripts/stack_manager.sh and helper scripts auto-detect repo root (OPEN_IOV_ROOT override); no hardcoded developer paths anywhere. - firmware analysis scripts use script-relative paths (also fixes stale firmware/570-GSP references); docs/wiki updated to $OPEN_IOV_ROOT. - .github/workflows/ci.yml updated for standalone repo layout; firmware fetched from linux-firmware in CI. - Untrack remaining kbuild artifacts (Module.symvers, modules.order, *.o.d); gitignore them. - STATE.md: live project state, limitations, fixed/open bugs. - README.md: rewritten for clone-and-run reproducibility.
* kernel: Fix mdev PCI config space to match TU104 hardware layoutArthur Hanson Rasmusson2026-09-21
| | | | | | | | | | | - Fix vendor/device/subsys ID byte order from vGPU type info - BAR0: 16MB 32-bit non-prefetchable (matches physical TU104) - BAR1: 256MB 64-bit prefetchable VRAM (consumes BAR2 upper half) - BAR3: 32MB 64-bit prefetchable (consumes BAR4 upper half) - Subsystem vendor/device IDs at correct offsets 0x2C/0x2E - MSI-X: 32 vectors (table size 31), chain to PCIe cap at 0x80 - Capabilities pointer at 0x34 -> PM at 0x40 - Interrupt pin INTA at 0x3D
* VFIO device registration fix + IOMMUFD callbacks + VFIO ioctl handlersArthur Hanson Rasmusson2026-09-14
| | | | | | | | | | | | | | - Removed IOMMU group addition before vfio_register_emulated_iommu_dev() in nv_vgpu_mdev_create() - Added IOMMUFD callbacks (bind_iommufd, unbind_iommufd, attach_ioas, detach_ioas) using kernel's vfio_iommufd_emulated_* functions - Implemented VFIO ioctl handlers: GET_INFO, GET_REGION_INFO, GET_IRQ_INFO, SET_IRQS, MMAP - Added config space read/write handlers at offset 0x70000000000 - Mmap handler for config space, BAR0 (emulation shadow), BAR1-5 (physical VRAM/registers) - Interrupt handler for VFIO_DEVICE_SET_IRQS (DATA_EVENTFD, DATA_NONE, DATA_BOOL) - Added udev rules for vfio group permissions - Created stack_manager.sh unified tool - Fixed mdev remove crash with deferred cleanup in vfio_device_release callback - Kernel 6.8+ IOMMUFD compatibility - QEMU launches successfully with vGPU passthrough
* Phase 5: Fix MDEV creation blocker - mdev parent registration & ioctl-based ↵Arthur Hanson Rasmusson2026-09-13
| | | | | | | | | | | | | | | vGPU type registration - Implemented nv_vgpu_mdev_register_parent() using mdev_register_parent() API with nv_vgpu_mdev_types array for proper mdev parent registration - Added ioctl NV_VGPU_IOCTL_REGISTER_VGPU_TYPE for userspace vGPU type registration - Created separate header nv-vgpu-vfio-interface-nvidia.h for nvidia.ko with only extern declarations to avoid duplicate symbol exports - Used KBUILD_EXTRA_SYMBOLS to provide only needed symbols (nvidia_vgpu_vfio_*) - Updated create_mdev.sh to use correct mdev type names (pci-nvidia-vgpu-*) - Updated load_vgpu_stack.sh to load open-iov-vfio.ko before nvidia.ko Result: 4 mdev types now show available_instances=8, mdev creation successful
* Phase 4: Portable Architecture ImplementationArthur Hanson Rasmusson2026-09-13
| | | | | | | | | | | - Firmware Analyzer: Runtime firmware signature computation (CRC32, SHA256) and pattern-based RPC handler address discovery - Dynamic RPC Dispatcher: Version-aware dispatch with native firmware handlers (550/560/570/580) and emulation fallback - Hardware Architecture Layer: Turing, Ampere, Blackwell configs with BAR layouts, doorbell offsets, MSI-X vectors - Driver Version Adaptation Layer: Feature detection per driver version - Declarative YAML configs for hardware (turing/ampere/blackwell) and versions (550/560/570/580) - Kernel unit tests for GSP emulation (gsp_emulation_test.c) - Test infrastructure scripts (test_gsp_emu.sh, test_guest_driver.sh, test_cuda_workloads.sh, test_graphics_workloads.sh, test_windows_guest.sh, test_vgpu_stack.sh) - Updated kernel Makefile with new source files
* Phase 3: RPC Handler CompletionArthur Hanson Rasmusson2026-09-13
| | | | | | | | | | | | | - Enhanced gsp_rpc_client.c with proper emulation implementations: - ALLOC/FREE: VRAM allocation tracking with host page allocation and DMA mapping - MAP/UNMAP: GPU VA mapping management - PIN/UNPIN: DMA address retrieval for pinned pages - SCHEDULE/YIELD: Local workqueue-based scheduling - NOTIFY: Callback registration for async events - Added emulation function declarations to gsp_rpc_client.h - Exported nv_vgpu_simple_mapping struct in memory.h for cross-file use - Updated RPC handlers to use gsp_rpc_emulation_is_enabled() accessor - All 12 RPC types now have functional emulation fallbacks
* Phase 2: PCI Config Space & BAR EnhancementsArthur Hanson Rasmusson2026-09-13
| | | | | | | | | | | | | | | | - Enhanced PCI config space write handlers in main.c: - Added MSI-X Message Control (0x72-0x73) write support - Added MSI-X Table Offset/BIR (0x74-0x77) write support - Added MSI-X PBA Offset/BIR (0x78-0x7B) write support - Enhanced BAR emulation in bar_emulation.c: - Added MSI-X Table emulation (32 entries at BAR0+0x2000) - Added MSI-X PBA emulation (64-bit at BAR0+0x3000) - Added nv_vgpu_msix_set_pending/clear_pending for interrupt injection - Integrated with GSP emulation constants from gsp_emulation.h - Updated bar_emulation.h with MSI-X struct definitions and function declarations - Added GSP emulation constants to gsp_emulation.h (MSIX offsets, polling params)
* Phase 1: Add GSP Emulation Engine (host-side)Arthur Hanson Rasmusson2026-09-13
| | | | | | | | | | | | | - New kernel/gsp_emulation.h: GSP emulation context and API - New kernel/gsp_emulation.c: Per-mdev emulation thread with: - Adaptive polling of doorbell at BAR0+0x110C00 - Shared memory RPC processing (4KB at BAR0+0x1000) - Dispatch for all 12 RPC types via firmware-decompiled handlers - MSI-X completion interrupt injection via eventfd - Statistics tracking (processed/failed RPCs, latency) - Modified kernel/main.c: Integrate gsp_emu_init/cleanup in device lifecycle - Modified kernel/Makefile: Add gsp_emulation.o to build - Modified kernel/gsp_rpc_client.h: Export gsp_rpc_emulation_set/is_enabled
* Initial commit: Open-IOV vGPU-IO complete stackArthur Hanson Rasmusson2026-09-13
- Firmware decompilation: GSP 570.144W (28.5MB ELF, RV64IMAC), 132 decompiled C files - Kernel modules: nvidia.ko (34MB, OpenRM), open-iov-vfio.ko (2.8MB, VFIO mdev), gsp-vgpu.ko (1.8MB, guest driver) - Userspace: libiov.so/libvgpu.so (76KB), example_vgpu (28KB), rpc_emulator.c - PCI config space emulation (256B with PM, MSI, MSI-X, PCIe caps) - 12 RPC types fully mapped from firmware decompilation - BAR trap, WPR, scheduling, notification, migration emulation - Helper scripts: load_vgpu_stack, create_mdev, launch_qemu_vgpu, test_vgpu_stack, setup_guest_driver, unload_vgpu_stack - Architecture documentation - Wiki: 10 technical documents - Git LFS for large binaries (.ko, .bin, .iso, .img, .o, ghidra repos) - nvidia-open-gpu-kernel-modules as submodule
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-12 11:26:15 +0000