| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
injector, wait sampler
|
| |
|
|
|
|
|
|
| |
The libcuda error scanner (FUN_0054fa00) walks a context list and returns
999 when its force flag is set and any context's RM error record has a
non-zero class. ctx999 only shows the first (zeroed) record; scanerr walks
the whole list and prints every non-zero record so the failing control or
alloc is named. Pushed by push_guest_tools.sh.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The counter enumeration/injection workflow lived in the guest's /tmp and was
lost on every restart. Make it durable under tools/guest-diag:
- sem_counters.py: translate a process's per-channel u32 completion counters
(16-byte stride in the /dev/nvidiactl counter page) to guest PAs through
/proc/<pid>/pagemap; --pairs emits handle,va,pa triples.
- sem_slots.py: read/write the counter slots via /proc/<pid>/mem (fallback
for the host counter publish).
- read_va.py: hex dump a process VA (UVM sysmem GPU VAs are usually valid
CPU VAs in the owning process); --u32 for method streams.
- inject_counters.sh: host-side one-shot: push the tool, take the GR handles
from the live chan_dump, enumerate, then register each counter with
sema_counter and pin its VA with sema_map_add.
- push_guest_tools.sh: push the new tools after every VM start.
- launch_qemu_direct.sh: parameterized mdev UUID/paths instead of a stale
hardcoded one; committed so it survives checkouts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
record
A freed client's notifier events (hSrcResource 0) stayed valid in the event
table, so the fallback wake-up nudge kept posting POST_EVENT records for a
(hClient, hEvent) pair the guest RM no longer knows; every one asserts in
CliGetEventInfo (kernel_gsp.c:466) and is dropped (171 failures in the
15:24 window of the last run, zero after this fix). FREE now retires the
matching event record (and a client's records when its root object is freed).
bp999.py now walks rdi -> ctx -> ctx[0x1291] -> +0x10 -> +0x10 and prints the
RM error notification record, and push_guest_tools.sh ships it to the guest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the libcuda tracker wait
The vidmem window convention turns any VA above the window base into a
vidmem offset, but the CUDA client's GPU semaphores are sysmem objects
mapped into its address space at those same VAs. A release applied through
the convention therefore landed in the vidmem shadow instead of the page
the client polls, and the completion was silently lost: the libcuda
tracker it waits on never advanced. Semaphore reads and writes now use
trusted translations only (pins, learned cache, discovered map, page
walks), so an unresolved semaphore is counted as a miss and stays visible
instead of corrupting a shadow.
tools/guest-diag/trkinfo.py dumps the libcuda tracker the client spins on:
the tracker object behind p+0x9410, the semaphore object it references,
the u32 semaphore memory it polls (a /dev/nvidiactl mapping backed by
guest RAM) and the VMA that memory lives in.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Analysis tooling for debugging CUDA execution in Open-IOV guests:
- userspace/cuda/570.144/: fetch a library from a running guest (nc),
objdump helpers, crash ip -> file vaddr + symbol mapping, and Ghidra
headless import/analysis/decompile scripts (project kept so single
functions decompile in seconds). The purpose is debugging correctness
and performance of CUDA in Open-IOV VMs (not a reimplementation);
extracted binaries and decompiled output are gitignored.
- tools/guest-diag/ioctl_trace.c: ptrace tracker for the NVIDIA driver
ioctls of a child (libcuda issues raw syscalls, so LD_PRELOAD sees
nothing). Decodes RM_ALLOC (class) and RM_CONTROL (cmd) arguments;
this is how the cuCtxCreate crash sequence was captured.
- tools/test/cuda_drv_test.c: unbuffered + per-call markers so a crash
identifies the CUDA call it happened in.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
