| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
| |
The host libcuda (570.211.01) has the identical engine-registration helper
instruction sequence (byte-pattern matched, host vaddr 0x454470 vs guest
0x453ef0). scripts/host_gdb_descriptor.sh runs cdt on the host under gdb and
prints the per-engine 6-dword registration record and the dispatch hook
target. Host registers a copy engine {3, 0xd, 4, 0xc5b5, 0x3e6, 3} and
cuCtxCreate succeeds; the guest registers GR {1, 1, 0, 0xc5c0, 0, 0} and
crashes - so the divergence is in the engine records the driver builds from
the emulated GSP static info engine table.
|
| |
|
|
|
|
|
|
| |
Records that libcuda builds an async copy-engine list from per-engine records
(engine kind, runlist, class) received with the device information, which is
derived from the GSP static info engine table this project emulates, and that
the crashing per-engine helper operates on those records. The adjacent local analysis notes (userspace/cuda/570.144/LOCAL-ANALYSIS.md) carry supporting
detail.
|
| |
|
|
|
|
|
|
|
| |
Documents the exact crash: cuInit/device enumeration now succeed; cuCtxCreate
faults in an engine-check/report path at libcuda+0x453f90 with rbp=0.
Register trace shows rbp valid at the helper and at the dispatch hook's
leave, but 0 immediately after the hook returns; the hook and the dispatch
table entries are all valid libcuda code. Adds the ioctl sequence observed
before the crash and lists what is still unanalysed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Analysis tooling for debugging CUDA execution in Open-IOV guests:
- userspace/cuda/570.144/: fetch a library from a running guest (nc),
objdump helpers, crash ip -> file vaddr + symbol mapping, and Ghidra
headless import/analysis/decompile scripts (project kept so single
functions decompile in seconds). The purpose is debugging correctness
and performance of CUDA in Open-IOV VMs (not a reimplementation);
extracted binaries and decompiled output are gitignored.
- tools/guest-diag/ioctl_trace.c: ptrace tracker for the NVIDIA driver
ioctls of a child (libcuda issues raw syscalls, so LD_PRELOAD sees
nothing). Decodes RM_ALLOC (class) and RM_CONTROL (cmd) arguments;
this is how the cuCtxCreate crash sequence was captured.
- tools/test/cuda_drv_test.c: unbuffered + per-call markers so a crash
identifies the CUDA call it happened in.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The progress service now decodes push buffers as method streams and performs
the work the driver pushes, not just the tracking semaphore:
- C5B5 method decoder: SET_REMAP_CONST_A/B + COMPONENTS + OFFSET_OUT +
LINE_LENGTH_IN + LAUNCH_DMA = memset (into vidmem or guest RAM);
OFFSET_IN + OFFSET_OUT + LAUNCH_DMA = copy; SET_SEMAPHORE_A/B/PAYLOAD +
LAUNCH_DMA = release. Contiguous method runs (count 2/4) are applied in
full - OFFSET_IN/OUT arrive as one run.
- Pushes span page boundaries, so pages are decoded as method streams
without a start marker. Collected operations execute only when the
push's release is reached and still pending (payload ahead of the
semaphore), so recycled buffers replaying old pushes are skipped and
their stale memsets cannot overwrite newer state.
- VA->PA map is first-wins: stale copies of freed allocations and
unrelated adjacent u64 pairs produced bogus entries that hijacked a
good mapping and sent semaphore writes to the wrong page.
- dump_guest module parameter (hex dump of guest PA, kthread via
completion; kthread_stop before the thread runs loses the work).
With this the guest no longer stalls in uvm_tracker_wait and no longer
oopses in the UVM path; the failure moved to a userspace segfault inside
libcuda (NULL-0x38) after the driver's page allocations.
Also add tools/cuda-ab: run one deterministic CUDA workload on the host
(bare metal oracle) and in the guest, then compare the stable result
lines. The binary links the CUDA runtime statically so the guest only
needs the driver's libcuda.so.
|
| |
|
|
|
|
|
|
|
| |
Relax the in-order rule to forward progress (payload > current), with the
jump bounded and counted: UVM tolerates forward jumps up to 2M
(2 * UVM_CHANNEL_NUM_GPFIFO_ENTRIES_MAX), and push buffers are recycled so
intermediate pushes can be missed. Add miss/stale/jump counters to the
rebuild log and a sema_probe module parameter to dump the service's map
entry and live slot values for a GPU VA (diagnostics for tracker stalls).
|
| |
|
|
|
|
|
|
|
| |
- patches/ -> legacy-patches/ (gitignored)
- legacy-driver-guest/ (old guest vGPU driver, gitignored)
- docs/old/ (superseded architecture write-up, gitignored)
- docs that only reference the legacy components stay tracked; their path
references now point at legacy-patches/
- install.sh PATCH_DIR updated
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
