| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
detail log
The poison test (fill every dword of BAR0 0x88000-0x88FFF, 0xB80000-
0xB80EFF, 0xC08000-0xC08EFF with 0x103) did not change the CUDA "Unknown
PCIe speed" failure, proving the failing BUS_GET_INFO_V2 index-3 data does
not come from any BAR0 mirror read. Combined with the QEMU config-space
behavior (link registers zeroed, type=0x92, 0x84 never read from us) the
value must come from the VGPU_STATIC_INFO branch in getBusInfos (pVSI->
pcieGpuLinkCaps), i.e. the guest needs the host vGPU static-info payload.
Cleanup: remove the poison loops and the per-write CFGWR log; keep the
once-per-init PCIe cap print and add per-entry BUS_GET_INFO_V2 logging
(index+data) for the ongoing CUDA/UVM work.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move the emulated PCIe capability to config offset 0x78 and shift MSI to
0x48 / MSI-X to 0x68, matching the real TU104 (host lspci: PM 0x60, MSI
0x68, Express 0x78). This matters because the driver's NV_PCFG mirror
(BAR0+0x88000, size 0x1000) is read as PCI config space:
osDevReadReg032 -> vgpuDevReadReg032 intercepts [NV_PCFG, +0x30) (or the
full 0x1000 for SRIOV) and forwards to osPciReadDword, so
NV_XVE_LINK_CAPABILITIES (0x84) must be the PCIe Link Capabilities
register, i.e. cap+0x0C with the cap at 0x78. Link Caps = Gen3 x16
(0x103), Link Status = 0x0103.
Also set MCFG-visible caps: the guest now reports MSI [48], MSI-X [68],
Express [78] exactly like the host GPU. Fixing the capability chain: the
first attempt set the PCIe cap's NEXT to 0x0C, which broke enumeration
(device 00:06.0 disappeared from the guest) - NEXT must terminate at 0.
Unknown still open: QEMU presents type=0x92 (Root Complex Integrated
Endpoint) and zeroes 0x84-0x8B in its config cache (never read from us,
not refreshed by writes), and poisoning all BAR0 mirror windows with
0x103 does not change the driver's "Unknown PCIe speed" - so the value
comes from the VGPU_STATIC_INFO branch (pVSI->pcieGpuLinkCaps, IS_VIRTUAL
true via hypervisor detection) rather than any register read.
Debug helpers added (temporary): CFGWR config-write log, PCFG change
watch, PCIe-cap print at init.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CUDA (cuInit) now reaches UVM's device create: libcuda opens nvidia-uvm,
UVM calls nvGpuOpsDeviceCreate/gpuGetGpuInfo, and the driver fails at
getPCIELinkRateMBps -> calculatePCIELinkRateMBps ("Unknown PCIe speed",
NV_ERR_INVALID_STATE) -> nv_open_device aborts -> cuInit returns
CUDA_ERROR_NOT_INITIALIZED (3).
Implemented along the way:
- FB_GET_INFO_V2 (0x20801303): echo the requested FB info indices and
answer TU104 values (256MB RAM/heap/BAR1, GDDR6, 8 LTCs, 4MB L2, ...).
Fixes the kern_mem_sys_ctrl.c:911 asserts and gives CUDA/UVM real
memory sizes.
- BUS_GET_INFO_V2 (0x20801823): echo indices; PCIe Gen3 x16 link caps /
GEN info / link status, SYSMEM_CONNECTION_TYPE=PCIE. UVM's
gpuGetSysmemLinkInfo reads SYSMEM_CONNECTION_TYPE and the nominal link
bandwidth from here.
- PCIe Link Capabilities in the emulated config space (0x8C/0x92) and in
the BAR0 PCFG mirror (gpuReadBusConfigReg -> NV_PCFG+0x84 = BAR0+
0x88084): MAX_SPEED=3 (8GT/s Gen3), MAX_WIDTH=0x10 (x16); link status
mirror at 0x88088. Also mirrored under the VF-window/low aliases.
- Temporary change-watch on PCFG[0x88084] to see if the guest overwrites
it (it does not - stays 0x103).
Verified: guest nvidia-smi -L still works and reports PCIe Generation
Max/Current = 3. cuInit still fails; the remaining question is which
address the driver's kbifGetGpuLinkCapabilities read sees as zero (the
PCFG shadow holds 0x103). Guest-side kprobe tracing of that function
panicked the VM (free_swap_cache page fault on a corrupted pointer) -
possible guest memory corruption to investigate separately.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
allocator
Fixes the BAR2 verify test (kbusVerifyBar2_GM107):
1. Removed PRAMIN window sync from emulated BAR1 - the sync was overwriting
test data written by the driver through the vBar2 aperture. The shadow
BAR0 is already consistent (guest writes/reads hit the same shadow location).
2. Added contiguous page allocation for golden image (alloc_pages with order
for large allocations, falling back to individual pages).
3. Added vidmem VA allocator - ALLOC now returns GPU VAs within the emulated
GPU's vidmem range (0x100000 - fb_size) instead of default 4GB, so the
memmgr accepts them.
Verified: driver probes past RmInitNvDevice bus init, reaches golden image
channel allocation (progress from fn-10 FREE timeout).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
handler
Root causes of host GSP crash (WPR2 stuck, 'unexpected WPR2 already up'):
1. stack_manager load-nvidia tried to load our submodule nvidia.ko (610.57.04-base)
alongside/over apt nvidia 570.211.01 -> GSP crash during driver conflict.
Fix: skip submodule when apt nvidia is already loaded (not needed for QEMU).
2. BAR3 (32MB instance memory window) was mapped to the physical GPU's real BAR3,
so guest BAR3 writes could corrupt the host GPU's instance memory.
Fix: vzalloc'd shadow buffer (like BAR1 emulated vidmem), per-page mmap.
3. GSP-RM garbage element handler declared GSP death on teardown queue garbage,
preventing driver cleanup frees from completing.
Fix: stop polling quietly, let Booter-unload kick handle reset.
Verified: full lifecycle (load, mdev create, QEMU guest boot, guest probe,
mdev remove, unload, reload) with host nvidia-smi (apt 570.211.01) stable
throughout. No GSP crash, no WPR2 stuck, no module unload hang.
|
| |
|
|
|
|
|
| |
Guest VRAM (BAR1) was mapped to the host GPU's real BAR1 (managed by
nvidia.ko) - guest VRAM accesses hit the broken host GPU. Now a
dedicated 256MB vzalloc buffer backs guest VRAM; BAR1 mmap uses
per-page vmalloc_to_pfn (like BAR0 shadow). vfree on cleanup.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
VBIOS serving
Guest driver probe fixes (unmodified 570.144 open driver on physical path):
- kernel/main.c: add show_description to the mdev driver. Userspace mdev
management tools parse 'framebuffer=<n>M' from every mdev type's
description sysfs file before creating an mdev; the missing file made
such tools fail to create the vGPU. Also fix create_mdev.sh for the real
type dir name pci-nvidia-vgpu-* (mdev core names dirs
dev_driver_string(parent)-<sysfs_name>; the parent registers before
nvidia.ko binds so dev_driver_string == 'pci').
- kernel/bar_emulation.c: seed BAR0 shadow so guest sees a REAL, PHYSICAL,
non-virtual TU104 (user directive: no PV/VF vGPU paths, unmodified driver):
* PMC_BOOT_0/1/42 = physical TU104 values captured from host hardware
(0x164000a1 / 0x0 / 0x164a1000). Without these, halmgrGetHalForGpu()
fails and probe aborts with 'does not include the required GSP'.
* Serve a valid VBIOS at the PROM window (BAR0+0x300000) loaded from
/lib/firmware/tu104_vbios.bin (dump of host TU104 ROM). Guest's
kgspExtractVbiosFromRom_TU102 needs it for the mandatory FWSEC ucode
(kgspGetFrtsSize is hardwired to 1MB on Turing).
* Pre-set GSP secure-boot (GFW) completion state: PGC6 PLM (0x118128)=1,
GFW_BOOT progress (0x118234)=0xFF, GSP falcon CPUCTL HALTED (0x110100).
The driver's post-FLR GFW wait only *waits* (never kicks), so the
pre-set state satisfies it immediately.
- kernel/gsp_emulation.c/h: physical GSP boot emulation.
* Falcon boot-watch: poll GSP falcon/PGC6/WPR2/scratch registers and log
changes (guest BAR0 is mmap-based, otherwise invisible to host).
* gsp_emu_poll_boot: STARTCPU auto-complete state machine. Guest kicks
FWSEC then Booter Load via GSP falcon CPUCTL; detect each kick and
complete the stage (HALTED + FRTS err NONE + WPR2 + mailbox +
RISCV-active) so the guest believes its ucodes ran.
Diagnosis that led here (documented in Handoff-vGPU-IO.md):
- Open nvidia.ko can never be an SR-IOV VF vGPU guest: os_is_grid_supported()
is a stub returning FALSE in the open build (disassembled). PV path also
rejected per user directive. Only the physical path with a fully emulated
local GSP boot remains.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
docs suite + repo reorg
CRITICAL FIXES (guest driver bring-up):
- gsp_emulation.c: emulation engine bound to the WRONG BAR0 - it polled the
host physical BAR0 (bars[0].vaddr ioremap) for the doorbell and read RPCs
from a local dma_alloc_wc buffer, while the guest writes land in the
vzalloc'd bar0_shadow. The RPC path could never work; PBA writes even
corrupted the host GPU's physical BAR0+0x3000. gsp_emu_init now binds
exclusively to bar0_shadow (doorbell/mailbox/PBA aliases;
shared_mem_dma==0 marks the alias so cleanup never dma_free's it).
- main.c SET_IRQS: DATA_EVENTFD eventfds were DISCARDED ('TODO' + kfree) -
the guest never received MSI-X completions. Full wiring now:
eventfd_ctx_fdget per vector, fd -1 clears, dev_close releases refs.
gsp_emu_inject_completion_interrupt looks up vgpu->irqs[0].trigger LIVE
(init-time copy was permanently NULL since QEMU wires after gsp_emu_init).
- gsp_rpc_client.c: BAR_TRAP/MIGRATE in emulation mode called
gsp_rpc_submit_raw which rings the SAME doorbell the thread polls -
self-triggered RPC livelock + multi-second stalls. Now fail fast with
-ENOTSUPP + pr_warn_ratelimited (guest BAR access is direct MMIO).
MORE FIXES:
- main.c: nv_vgpu_config_write returned SIZE_MAX (while(count--) wrap) and
was dead code while the VFIO write handler did unfiltered byte copies
(guest could rewrite vendor ID/BARs). Return fixed; VFIO write handler
now routes through the per-register filter.
- kernel/Makefile: M=$(PWD) used the env var which is stale when invoked
via the top-level make (pointed at repo root -> kbuild silently built
NOTHING; explains every 'stale .ko' incident). Now M=$(CURDIR).
- Warnings sweep: header prototypes for mdev/type functions, format casts
in gsp_comm.c DRY-RUN logs, unused variable.
ROBUSTNESS (harness/tooling):
- vgpu_vm.py: run had no timeout (start could block forever);
guest_shell recv could block forever on a wedged guest. Both now bounded;
guest_shell pre-flights guest-ping and reports wedge diagnostics.
- stack_manager.sh: qemu/qemu-debug now launch via the vgpu_vm.py harness
(workflow rule: never raw QEMU).
REPO REORGANIZATION:
- driver/ (legacy guest-side gsp-vgpu.ko) moved to legacy-driver-guest/
(reference only, not built; README explains). 3 legacy un-buildable test
files moved to legacy-driver-guest/kernel-tests/. Top-level Makefile
cleaned of the dead DRIVER_DIR reference.
DOCUMENTATION SUITE (new):
- docs/GSP_EMULATION_ARCHITECTURE.md - emulator deep-dive: shadow BAR0
model, doorbell protocol, dispatch table, MSI-X path, lifecycle, memory-
model caveats
- docs/DEBUGGING.md - QEMU GDB stub workflow, serial console, QMP NMI,
failure taxonomy, full bring-up validation checklist, EPT-livelock case
study, version-matrix discipline
- docs/ONBOARDING.md - repo tour, coding standards (incl. virt_to_phys /
dma_addr_t / get-put rules), first build & test
- docs/VERSION_UPGRADE.md - end-to-end new-driver-version procedure
(firmware fetch, Ghidra analysis, configs, signatures, validation)
- docs/PERFORMANCE.md - bare-metal A/B methodology + targets
- README.md doc index; STATE.md refreshed; DEVELOPMENT_WORKFLOW.md corrected
(load order: open-iov-vfio BEFORE nvidia; stock-driver blacklist note)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
remove path
VFIO mmap (main.c): virt_to_phys() was used on vmalloc-family addresses
(vzalloc'd BAR0 shadow, ioremap'd BAR2/3), producing garbage PFNs. Guest
BAR0 access then hit a KVM make_spte reserved-bits WARN and an
EPT-violation livelock, hard-wedging the guest on every nvidia driver
probe (confirmed via QEMU GDB stub). BAR0 now maps per-page via
vmalloc_to_pfn(); BAR2/3/4/5 map bars[N].gpa directly (like BAR1);
config space uses new nv_vgpu_mmap_virt_to_pfn() helper.
mdev remove path (main.c): old path never list_del'd the device
(available_instances leaked), never ran cleanup (leaked gsp_emu kthread),
never unregistered VFIO (leaked /dev/vfio/N + iommu group), never put
pdev, never freed the struct. New kvmgt-style path: cancel init_work ->
wait fds -> vfio_unregister_group_dev -> cleanup work -> list_del ->
pci_dev_put -> vfio_put_device.
gsp_rpc_client.c: fix dma_map_page dma_addr_t->int truncation in
gsp_rpc_alloc_emul (corrupts >=4GB IOVAs); export gsp_client_init_emulation.
gsp_emulation.c: export gsp_emu_process_rpc and gsp_emu_get_stats.
main.c: remove wrong MODULE_SOFTDEP("pre: nvidia") - module has zero
nvidia imports, loads standalone, and must load BEFORE nvidia.ko
(it exports nvidia_vgpu_vfio_* for nvidia.ko to import).
nvidia-vgpu-vfio.h: NULL-safe nv_vgpu_{dbg,info,warn,err} macros.
tests/gsp_emulation_test.c: rewritten against current API - drives
gsp_emu_process_rpc() with crafted shared-memory messages (protocol
validation, SCHEDULE/YIELD/NOTIFY/DEBUG dispatch, stress). Builds with
KBUILD_EXTRA_SYMBOLS; depends: open-iov-vfio.
stack_manager.sh: vfio_mdev doesn't exist on >=6.4 (probe per-module);
load-nvidia detects foreign nvidia via /sys/module/nvidia/version and
errors with guidance instead of silently skipping.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The mdev core's shared parent->available_instances counter leaks whenever
an mdev remove path fails to complete (as happened during the now-fixed
remove crash): creates then fail with -EUSERS while sysfs reports 0
available instances with zero mdevs present.
Implement mdev_driver.get_available to compute availability from the
driver's own authoritative device list instead of the core counter.
Handles the 'nvidia-' prefixed sysfs type names when looking up per-type
max_instances (falls back to 1).
|
| |
|
|
|
|
|
|
|
|
|
| |
- Fix vendor/device/subsys ID byte order from vGPU type info
- BAR0: 16MB 32-bit non-prefetchable (matches physical TU104)
- BAR1: 256MB 64-bit prefetchable VRAM (consumes BAR2 upper half)
- BAR3: 32MB 64-bit prefetchable (consumes BAR4 upper half)
- Subsystem vendor/device IDs at correct offsets 0x2C/0x2E
- MSI-X: 32 vectors (table size 31), chain to PCIe cap at 0x80
- Capabilities pointer at 0x34 -> PM at 0x40
- Interrupt pin INTA at 0x3D
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Removed IOMMU group addition before vfio_register_emulated_iommu_dev() in nv_vgpu_mdev_create()
- Added IOMMUFD callbacks (bind_iommufd, unbind_iommufd, attach_ioas, detach_ioas) using kernel's vfio_iommufd_emulated_* functions
- Implemented VFIO ioctl handlers: GET_INFO, GET_REGION_INFO, GET_IRQ_INFO, SET_IRQS, MMAP
- Added config space read/write handlers at offset 0x70000000000
- Mmap handler for config space, BAR0 (emulation shadow), BAR1-5 (physical VRAM/registers)
- Interrupt handler for VFIO_DEVICE_SET_IRQS (DATA_EVENTFD, DATA_NONE, DATA_BOOL)
- Added udev rules for vfio group permissions
- Created stack_manager.sh unified tool
- Fixed mdev remove crash with deferred cleanup in vfio_device_release callback
- Kernel 6.8+ IOMMUFD compatibility
- QEMU launches successfully with vGPU passthrough
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
vGPU type registration
- Implemented nv_vgpu_mdev_register_parent() using mdev_register_parent() API
with nv_vgpu_mdev_types array for proper mdev parent registration
- Added ioctl NV_VGPU_IOCTL_REGISTER_VGPU_TYPE for userspace vGPU type registration
- Created separate header nv-vgpu-vfio-interface-nvidia.h for nvidia.ko
with only extern declarations to avoid duplicate symbol exports
- Used KBUILD_EXTRA_SYMBOLS to provide only needed symbols (nvidia_vgpu_vfio_*)
- Updated create_mdev.sh to use correct mdev type names (pci-nvidia-vgpu-*)
- Updated load_vgpu_stack.sh to load open-iov-vfio.ko before nvidia.ko
Result: 4 mdev types now show available_instances=8, mdev creation successful
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Enhanced PCI config space write handlers in main.c:
- Added MSI-X Message Control (0x72-0x73) write support
- Added MSI-X Table Offset/BIR (0x74-0x77) write support
- Added MSI-X PBA Offset/BIR (0x78-0x7B) write support
- Enhanced BAR emulation in bar_emulation.c:
- Added MSI-X Table emulation (32 entries at BAR0+0x2000)
- Added MSI-X PBA emulation (64-bit at BAR0+0x3000)
- Added nv_vgpu_msix_set_pending/clear_pending for interrupt injection
- Integrated with GSP emulation constants from gsp_emulation.h
- Updated bar_emulation.h with MSI-X struct definitions and function declarations
- Added GSP emulation constants to gsp_emulation.h (MSIX offsets, polling params)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- New kernel/gsp_emulation.h: GSP emulation context and API
- New kernel/gsp_emulation.c: Per-mdev emulation thread with:
- Adaptive polling of doorbell at BAR0+0x110C00
- Shared memory RPC processing (4KB at BAR0+0x1000)
- Dispatch for all 12 RPC types via firmware-decompiled handlers
- MSI-X completion interrupt injection via eventfd
- Statistics tracking (processed/failed RPCs, latency)
- Modified kernel/main.c: Integrate gsp_emu_init/cleanup in device lifecycle
- Modified kernel/Makefile: Add gsp_emulation.o to build
- Modified kernel/gsp_rpc_client.h: Export gsp_rpc_emulation_set/is_enabled
|
|
|
- Firmware decompilation: GSP 570.144W (28.5MB ELF, RV64IMAC), 132 decompiled C files
- Kernel modules: nvidia.ko (34MB, OpenRM), open-iov-vfio.ko (2.8MB, VFIO mdev), gsp-vgpu.ko (1.8MB, guest driver)
- Userspace: libiov.so/libvgpu.so (76KB), example_vgpu (28KB), rpc_emulator.c
- PCI config space emulation (256B with PM, MSI, MSI-X, PCIe caps)
- 12 RPC types fully mapped from firmware decompilation
- BAR trap, WPR, scheduling, notification, migration emulation
- Helper scripts: load_vgpu_stack, create_mdev, launch_qemu_vgpu, test_vgpu_stack, setup_guest_driver, unload_vgpu_stack
- Architecture documentation
- Wiki: 10 technical documents
- Git LFS for large binaries (.ko, .bin, .iso, .img, .o, ghidra repos)
- nvidia-open-gpu-kernel-modules as submodule
|
