open-iov.git/tools, branch archive/pre-squash

kernel: serve CE caps + CE->PCE masks (unblocks UVM channel manager)

2026-09-23T05:26:25+00:00

kceGetDeviceCaps (kernel_ce.c) copies the caps the client hands to UVM from NV2080_CTRL_CMD_CE_GET_PHYSICAL_CAPS (0x20802a07), which reaches GSP-RM and was answered with zeros. With capsTbl = {0,0} no CE had the SYSMEM bit, so UVM's channel manager found no usable CE and UVM_REGISTER_GPU failed with NV_ERR_NOT_SUPPORTED ("Failed to initialize the channel manager", uvm_gpu.c:1574 - visible with uvm_debug_prints=1). New handlers: - CE_GET_PHYSICAL_CAPS (0x20802a07): {ceEngineType, capsTbl[2]} with SYSMEM_READ|SYSMEM_WRITE|SYSMEM|P2P|BL_GT_64K (0xEC) for COPY0..COPY4, NOT_SUPPORTED otherwise (matches subdeviceCtrlCmdCeGetCaps_IMPL). - CE_GET_ALL_CAPS (0x20802a0a) / CE_GET_ALL_PHYSICAL_CAPS (0x20802a0b): capsTbl[64][2] + present bitmap with the five CEs. - CE_GET_CE_PCE_MASK (0x20802a02): pceMask = BIT(ceIdx) (one PCE per LCE). Verified in-guest progress: UVM now gets past the channel manager and creates channels; the register path's new blocker is the first channel push (channel_init, uvm_channel.c:2535) whose semaphore release the emulated GPU must complete ("Tracker entry ... stuck waiting for 90s", sema VA 0x121000000 on channel 1:3 CE 0).

tools: keep all guest-diag artifacts in-tree (push helper, runtime launch script, README)

2026-09-23T05:10:40+00:00

tools: add rmq_probe + kret attempts, document kprobe/notrace limitation

2026-09-23T05:04:38+00:00

tools: add guest-diag (ptrace ioctl tracer, vmcycle, root-port QEMU cmdline)

2026-09-23T04:51:55+00:00

The ptrace tracer resolves the current CUDA blocker: libcuda issues raw syscall instructions, so LD_PRELOAD shims and ltrace see nothing; following the child with PTRACE_SYSCALL + PTRACE_GET_SYSCALL_INFO does. It found UVM_REGISTER_GPU (ioctl 0x25) returning rmStatus=0x56 NV_ERR_NOT_SUPPORTED, with inputs identical to the working host run (rmCtrlFd=-1, hClient=0, hSmcPartRef=0). Also dumps NV_ESC_RM_CONTROL cmd/status and RM alloc/free statuses (all zero) and the UVM ioctl set (guest stops after 0x27,0x1,0x25,0x2 vs 141 ioctls on the host). launch_qemu_rootport.sh is the working QEMU command line with the pcie-root-port fix (see its header + the handoff PART 4); vmcycle.sh drives the module/mdev/VM reload cycle around it.

tools: minimal CUDA driver-API smoke test (dlopen, no toolkit)

2026-09-23T02:43:43+00:00

Documents the current CUDA blocker: cuInit returns CUDA_ERROR_NOT_INITIALIZED (3) on the emulated GPU even though every nvidia.ko ioctl succeeds and RmInitAdapter completes. libcuda opens /dev/nvidia-uvm before /dev/nvidia0, so the next milestone is making nvidia-uvm initialize against the emulated MMU/VAS model.

kernel: guest probe completes - nvidia-smi enumerates the emulated TU104

2026-09-23T02:40:26+00:00

Root cause of the golden-image NV_ERR_NO_MEMORY: STATIC_KGR_GET_CONTEXT_ BUFFERS_INFO (0x20800a32) fell through to the generic zeroed reply, so kgraphicsInitializeDeferredStaticData() stored a 0-byte main ctx buffer size and kgrctxAllocMainCtxBuffer() did a 0-page allocation (mem_desc.c:1353). Serve the TU104 per-engine ctx buffer table (26 engine IDs x 8 GR engines; NV_U32_MAX = unsupported, which the client skips). Second blocker after that: L2 cache-op and TLB-invalidate completion registers. Turing reaches them through the virtual-register window (GPU_VREG_* = BAR0 + sriovState.virtualRegPhysOffset + offset, 0xB80000 for a non-SRIOV virtual GPU): NV_VIRTUAL_FUNCTION_PRIV_MMU_INVALIDATE (0xB830B0, TRIGGER bit31) and the L2 invalidate/dirty-flush set (0x70000-0x70014 and 0xB80F00/0xB80F04). The driver soft-locked spinning in osDevReadReg032 on 0xB830B4 (kgmmuCheckPendingInvalidates_TU102) and timed out in kmemsysDoCacheOp_GM107; auto-clear all of them (both the plain and windowed aliases). Third: re-probe failed with GspStatusQueueInit msgqRxLink timeout because gsp_rm_emu_reset() was not called before a fresh boot, leaving rm->parsed set so the new libos args were never parsed. Reset GSP-RM state on the fresh FWSEC kick. Verified: guest nvidia-smi and nvidia-smi -L both exit 0 (full unload/reload cycle), GPU enumerated as "NVIDIA GeForce RTX 2080 SUPER" with 256MB, no NO_MEMORY / timeouts / soft lockups in the guest. Also add tools/rmq/grctx_oracle.c documenting that the host RM does not expose the ctx-buffer table to userspace (the real values are probed from the ctxsw ucode by GSP-RM), so the emulation serves synthesized sizes.

PRAMIN window translation + queue state preservation on Booter-unload

2026-09-22T14:59:59+00:00

Fixes the BAR2 verify test failure (kbusVerifyBar2_GM107): - GSP emulation thread now watches NV_PBUS_BAR0_WINDOW (BAR0+0x1700) for base/target changes and syncs the PRAMIN window (BAR0+0x700000) from the emulated BAR1 (vidmem) at (base << 16). This makes PRAMIN reads return correct vidmem content for the BAR2 verify test. Fixes the fn-10 FREE timeout during driver cleanup: - gsp_emu_clear_boot_state no longer resets the GSP-RM queue state on Booter-unload kick. The driver sends cleanup FREEs AFTER the Booter-unload kick; resetting the queue here caused those frees to time out. - Queue state is now only reset on: ENGINE reset (full GPU reset) or genuine queue death (insane writePtr). Both call gsp_rm_emu_reset explicitly. Also fixes rmq tool ioctl encoding: - Use _IOWR('F', nr, type) encoding for NVIDIA ioctls (was raw numbers) - Open /dev/nvidiactl (not /dev/nvidia0) for RM_ALLOC - Add NVOS64_PARAMETERS (48 bytes) for device/subdevice allocs Verified: BAR2 verify test passes, INSTLOC regkey read, driver probes past RmInitNvDevice bus init, reaches fn-10 FREE timeout during cleanup (same as handoff's characterized blocker).

kernel: GSP-RM msgq emulation + TU104 hardware tables (GET_GSP_STATIC_INFO, GSP_RM_CONTROL)

2026-09-21T22:01:55+00:00

Implements the GSP-RM reply content layer for the msgq RPC channel: - kernel/gsp_rm_emulation.c/h: GSP-RM msgq emulation engine - Queue chain discovery (libos args → RMARGS → GSP_ARGUMENTS_CACHED) - Status queue link + GSP_INIT_DONE emission - Command queue polling + RPC dispatch - SET_GUEST_SYSTEM_INFO version negotiation (VGX 41.12) - GET_GSP_STATIC_INFO(65) with 1656-byte GspStaticConfigInfo - GSP_RM_CONTROL(76) sub-command dispatch: - FIFO_GET_DEVICE_INFO_TABLE (0x20801112) - INTERNAL_INTR_GET_KERNEL_TABLE (0x20800a5c) - GPU_GET_CONSTRUCTED_FALCON_INFO (0x208001b0) - INTERNAL_GET_DEVICE_INFO_TABLE (0x20800a40) - kernel/gsp_rm_tu104.h: TU104 hardware tables - 12-entry FIFO engine table (GR0, CE0-4, NVDEC0-1, NVENC0, SEC2, NVJPG, SW) - 8-entry interrupt table (from intr_phys_tu102.c) - 1-entry constructed falcon table (FBFLCN) - 11-entry internal device info table - GspStaticConfigInfo layout (1656 bytes, ABI-verified offsets) - Data from NVIDIA driver material (ext-tree) + hwref/turing/tu104 - tools/rmq/: Userspace RM API client (oracle for ground-truth tables) KNOWN ISSUE: gsp_rm_send crashes (fortify_panic) on multi-element messages when reply data exceeds one 4096-byte queue element. Affects cmd 0x20800a41 (paramsSize=8204). Multi-element splitting is the next fix.

Make repo fully path-portable and add reproducible installer

2026-09-21T15:09:36+00:00

- install.sh: one-command fresh-Ubuntu reproduction (deps, NVIDIA clone + patch, GSP firmware fetch from linux-firmware.git with objcopy .fwimage extraction + MD5 verify, ordered build, host config, load, mdev creation). Wrapper-neutral: works with libvirt/raw QEMU. - patches/nvidia: regenerated path-independent patch (KBUILD_EXTRA_SYMBOLS resolved via $(src)/$(CURDIR) relative to the NVIDIA tree). - scripts/stack_manager.sh and helper scripts auto-detect repo root (OPEN_IOV_ROOT override); no hardcoded developer paths anywhere. - firmware analysis scripts use script-relative paths (also fixes stale firmware/570-GSP references); docs/wiki updated to $OPEN_IOV_ROOT. - .github/workflows/ci.yml updated for standalone repo layout; firmware fetched from linux-firmware in CI. - Untrack remaining kbuild artifacts (Module.symvers, modules.order, *.o.d); gitignore them. - STATE.md: live project state, limitations, fixed/open bugs. - README.md: rewritten for clone-and-run reproducibility.

repo: Untrack binaries/vendor/wrapper layers, add .gitignore + NVIDIA patches

2026-09-21T14:40:43+00:00

Restructure for clean mirroring to git.vgpu.io/open-iov.git: - Untrack GSP firmware blobs and decompiled output (firmware/**/*.bin, firmware/**/decompiled/) - large, license-restricted NVIDIA binaries. Analysis scripts, docs, and address maps remain tracked. - Untrack nvidia-open-gpu-kernel-modules submodule; our delta against upstream tag 610.57.04 is now tracked as a patch series in patches/nvidia/ and applied by install.sh after cloning. - Untrack kernel build artifacts (*.o, *.ko, Module.symvers, ...). - Untrack Ghidra project databases (*.rep/, *.gpr) - scripts remain. - Untrack VM images (*.iso, *.img, *.qcow2) and compiled helper binaries. - Drop git-lfs (.gitattributes) - no large objects remain in git. - Local QEMU-wrapper layers (scripts/vgpu_vm.py, qemu_control.py, qemu_harness.sh) are gitignored: the repo is hypervisor-management neutral (libvirt, raw QEMU all work). All files remain on disk; only git tracking changes. No source code modified.