| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
memsets)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pin output
The first GR window pin landed ~2 s after the pump consumed the freshly
created channels' GPFIFO entries on the first failed read, so the pushes
(and the tracking-semaphore releases they carry) were lost. GR entries with
an unresolvable push page now stay pending for sema_gr_grace_ms (default
8 s) before the stale-entry consume; a pin landing in that window makes the
entry readable and it executes normally. The completion counter is still
published from put at the end of every pass, so the client's counter wait
is not delayed.
guest_pmap_win.py coalesces contiguous VA+PA pages into run lines, so a
15k-page window transfers as ~200 lines instead of ~15k and the pin batch
lands within a second of the pages appearing.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- svc_handle_chan_dump()/svc_decode_stream(): static snapshot buffers. The
128-channel local array made an ~18 KB frame, which overflows the 16 KB
kernel stack in the function prologue and panicked the host 2-3 s after a
VM attached the vGPU (proven from EFI pstore: guard page hit at
svc_handle_chan_dump+0x20, sub rsp,0x4850).
- svc_pump_channels(): write-back now also persists expected_sem_payload,
rescue_last and rescue_fail. The pump works on a snapshot copy, so
without this the stale-entry skip and the rescue rate limit could never
fire and the payload expectation was reinitialized every pass.
- svc_targeted_resolve(): sliced scan (128 MB per call, one VA at a time,
failed-VA cache) instead of a full-RAM scan per miss; no longer appends
to the sorted map out of order.
- svc_rebuild(): merges new scan results into the previous map (fresh
translation wins in place, new VAs appended) instead of replacing it
wholesale, so live translations survive a scan-yield collapse.
- Warning fixes: misleading indentation, unused variable, jiffies
typecheck. Frame audit: largest frame now 872 B.
- Includes the previous session's uncommitted work: sema_counter /
shadow_scan / shadow_dump debug params, exit stage markers, pdev ref
fix, and the dynamic dispatcher request_firmware-out-of-spinlock fix.
|
| |
|
|
|
|
|
|
|
|
| |
New module parameter walk_probe <va> runs the recorded-va-space walks and
the discovered-directory walk directly (pin/lat/pair map bypassed) with
per-level tracing, and prints the result of each. This separates 'the
page tables are not walkable' from 'the VA->PA pair map already answers'
when a translation is missing. Baseline on a live boot: root PDB entries
are zero (L0 e=0 idx=0) so the writes UVM issues into the level pages
have no parent entry yet.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- The CE semaphore/memset method groups also arrive on subchannel 0 (CUDA's
push library binds the copy class on compute channels to carry the
work-submission releases), so the stream decoder accepts both 0 and 4.
- GR (compute) channel rings live in the same video-memory allocation as the
CE rings, 0x3000 bytes per channel after them: ring (0x2000) at
ce_end + 0x3000*rank, USERD at +0x2000. Discover and pin them once the CE
ranks are known; verified against ring entry 0 (1 MB aligned push VA in
the client window, sane length) and the USERD's put/get, so a layout
change costs a failed pin, not a wrong one. Gated by sema_gr_rings
(default on).
- memprobe: require the sem-VA triple in the sem-object scanner (a bare
qword in the window matches thousands of unrelated pointers once the
ring/pushbuffer windows are part of the range).
Live evidence: ring entries read through the guest's /proc/pid/mem decode to
the GR channels' first pushes; pinning the ring (fb 0x140000+0x3000*rank),
USERD (ring+0x2000) and first-push pages made all 20 GR channels execute
their entry 0 for the first time (get=1 sub=1).
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Debug injections could only teach sysmem translations, but the UVM CE
channel rings and USERDs live in the frame buffer window, and the
progress service had no candidate for their USERD (the allocation
parameters carry an offset relative to a client pool, not the FB
offset). Add an aperture field to the pinned translations and to the
sema_map_add debug parameter (va,pa[,len[,aperture]]), and probe the
observed UVM layout as well: the USERD sits one 0x10000 block after the
ring for the data-path channels.
With the rings pinned, the pump now resolves and decodes the CE rings
(the entries point at 1 MB pushbuffer regions); only the pushbuffer
VA->PA translation is still missing, so the entries are kept pending.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- admit {GPU VA -> PA} pairs only when the preceding bytes form a valid
client memory descriptor header (type 0/1, CPU+GPU mapped bits), prefer
validated pairs over headerless candidates under first-wins dedup, and
invalidate the translation cache on every rebuild; fixes stale identity
entries shadowing live pool pages and lets the guest UVM channel
bring-up complete on its own
- capture OS event allocations (class 0x79) and post event records
(function 0x1003) into the guest status queue when a channel's busy
notification is consumed, with a broadcast fallback, serialized with
replies via a queue lock
- add BAR0 shadow introspection (dump_bar0/scan_bar0) and guest-side
diagnostic scripts for the VM cycle
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The guest's CUDA stack was stuck inside context creation because the work it
submits is never executed by the emulated GPU. Two forms of pushed work were
missing, and the completion path had grown far too slow to be useful:
- Channel tracking: on a channel allocation parse the GPFIFO GPU VA and entry
count, the USERD address/aperture and the instance block, hand them to the
progress service and drop them again when the channel is freed.
- Host-class (C46F) semaphore releases: pushes on a compute channel signal
progress with a five-dword method run (SEM_ADDR_LO/HI, PAYLOAD_LO/HI,
EXECUTE) rather than the copy-engine group the service already understood,
so those releases were never applied. Decode both forms.
- Emulated vidmem: channel blocks (USERD/instance) and some client semaphores
live in the emulated frame buffer, whose VAs are a window over the BAR1
backing rather than guest RAM; access them through the right aperture.
- Latency: the service's full-RAM sweep took seconds per pass, so guests'
progress waits took whole seconds and the driver tore its own setup back
down in the meantime. Sweep guest RAM in bounded, chunked windows on every
pass, remember the pages pushes are actually found in and revisit those
first, and keep the VA->PA map complete (dedup, sort, binary search) so
semaphores resolve reliably.
Also add a vidmem poke parameter for bringing up and debugging emulated
frame-buffer state.
|
| |
|
|
|
|
|
|
|
| |
Relax the in-order rule to forward progress (payload > current), with the
jump bounded and counted: UVM tolerates forward jumps up to 2M
(2 * UVM_CHANNEL_NUM_GPFIFO_ENTRIES_MAX), and push buffers are recycled so
intermediate pushes can be missed. Add miss/stale/jump counters to the
rebuild log and a sema_probe module parameter to dump the service's map
entry and live slot values for a GPU VA (diagnostics for tracker stalls).
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
