| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
This reverts commit 5fa773c2cd4e166d2c31a40aff9fb63e7534fec3.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Handles are per-client, but the FREE path unregistered any channel whose
numeric handle matched the freed object, with no client check. r77 root
cause of the third-client pma scrub hang: the boot session's CeUtils
scrubber channel (h=0x2, gpfifo=0x120064000, entries=4096) was
unregistered when an unrelated class-0x2080 object with handle 0x2 in
client 0xc1d00011 was freed. The guest kept using the live channel for
the scrub; the service no longer pumped it, its pushes were never
decoded and the channel-buffer finish payload (pbGpuVA+0x6c004) was
never written, so scrubCheckAndWaitForSize polled forever (sysrq stack:
ceutilsUpdateProgress -> channelReadChannelMemdesc).
Channel registration now records h_client (the RM alloc header's client)
and deletion matches (h_object, h_client); a missing client on either
side keeps the handle-only behaviour. Re-registration keeps the
counter/release slots only for the same (handle, client) pair. Live
proof from the hung VM: sema_probe translates the finish-payload VA
0x12006c004 to pa=0x10f962004 aperture=ram - the word the guest polls is
reachable; the channel was simply invisible to the pump.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
BAR1 aperture (ISSUE-0002)
The pma scrub in a second/third CUDA session memsets a legitimate vidmem
offset at 262M (inside the 512M turing shadow) and the copy path rejected it
with -EINVAL because every FB bounds check compared against
vgpu->bars[1].size, the physical BAR1 aperture exposed to the guest (256M on
TU104). The op was held and retried forever (opdef), the GR scrub waited in
scrubCheckAndWaitForSize and the client froze.
All bounds checks that guard a dereference of the emulated vidmem shadow at
vgpu->bars[1].vaddr now use nv_vgpu_fb_emul_size() via svc_fb_shadow_size():
the copy/fill paths, svc_read_ap/svc_write_ap, svc_ap_ok, the page-directory
discovery scans, svc_lookup's vidmem VA translation, the GRCE inline write,
svc_fwd_operand_hpa and the debug scans. The 2 MB leaf path of svc_gmmu_walk
now validates through svc_ap_ok like every other leaf path so an
out-of-shadow translation falls through to the shadow fallback instead of
reaching the copy path. The RM page-directory debug peek is bounded the same
way.
Evidence: r75b live log: sema-bigcopy dst_va=0x76aa08e02200 -> pa=0x10602200
ap=0 prov=3; sema-bigop rc=-22 repeated; guest sysrq stack
scrubCheckAndWaitForSize -> pmaAllocatePages.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
kgrobjPromoteContext_IMPL marks each ctx buffer initialized when the
REPLY's promoteEntry[i].bInitialize is set:
if (!params.promoteEntry[i].bInitialize) continue;
kgrctxMarkCtxBufferInitialized(...);
The flag is the server's initialized-ACK, so the correct emulation reply is
a verbatim echo (the physical backend owns residency; the control answers
SUCCESS). 7ce387f3 cleared the flag believing the check inverted - that
skipped the mark and left the RM re-promoting the same buffers forever.
Restore echo semantics with the source-cited comment.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
kgrobjPromoteContext_IMPL marks a context buffer initialized only when the
REPLY's promoteEntry[i].bInitialize is clear; echoing the request back kept
the flag set so the RM re-promoted the same contexts indefinitely. Clear
the flag on the echoed reply (per-entry, 5 entries observed per call).
Note: this aligns the reply with the caller contract but did NOT by itself
unblock the observed cudaMalloc stall (client still reaches DEVICE then
spins; zero GR ring content in VRAM) - kept because the old behavior was
wrong and the flag gates kgrctxMarkCtxBufferInitialized.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
window
The guest's first RmInitAdapter intermittently fails: GspStatusQueueInit
gives the status-queue link only 4 s (message_queue_cpu.c: timeoutUs =
4000000) and the first boot attempt can burn it, after which the RM tears
down, leaks the device and retries; the surviving retry leaves the GR/ctx
state degraded, which is the persistent cudaMalloc stall class.
Changes:
- gsp_rm_emu_poll: parse-retry cadence 250 ms -> 50 ms (the 250 ms tick
consumed a fifth of the guest's budget per attempt).
- gsp_rm_emu_reset: clear last_parse_retry so a new boot gets a fresh
retry budget; take stat_lock while clearing queue cursors so a sema
event post cannot race the reset.
- gsp_rm_emu_on_booter_done: on GSP_INIT_DONE send failure re-arm the
retry path (parsed/active were already set, so the poll loop would
never retry).
- gsp_rm_parse_queue_chain: free the previous ptes before re-alloc so
retries and post-reset re-parses do not leak.
- gsp_rm_post_channel_event: never post events across a queue reset
(parsed gate).
guest_kprobes.sh: add rpcs/rpcp transport probes (the direct RPC-wait
symbol is notrace in 570.144) and ccset entry/return probes for
rmapiControlCacheSet so a stall names the control whose reply breaks the
guest's control-cache bookkeeping.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
- nv_vgpu_type_info/ioctl gain vram_mb; nv_vgpu_device gains fb_emul_size
bound from the mdev type at create (default 512 MiB).
- New nvidia-vgpu-turing-1g type (1024 MiB); turing max_instances 1 -> 2;
show_description reports the per-type framebuffer and max_instance.
- Thread fb_emul_size through GSCI fb_length/region/frtsOffset, WPR2,
NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE, the vidmem allocator bound and the BAR1
shadow; bar_info callback uses the per-device size.
- UNTESTED at commit time: builds clean; validation blocked by the Xid 140
incident (reboot pending). See Handoff-vGPU-IO.md ADDENDUM 48.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.
Two data-path correctness fixes make the transfers sound:
- Once-only op execution per GPFIFO entry. The pump replays recently
consumed ring entries when translation batches are injected so missed
releases can land; re-executing a CE copy is not idempotent because
its source is a client staging buffer that may have been refilled.
Slots now record their raw entry words and a replayed entry matching
that identity suppresses operations while still processing releases.
- Per-page translation of virtual operands. svc_copy resolved only the
first page of a VA range and walked the PA linearly; client staging
ranges are frequently split (observed 195 + 61 pages), so the tail of
alternating chunks was read from unrelated memory. Multi-page copies
now re-resolve each 4 KB page through the trusted resolver.
Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(guest-faithful 9-entry set)
- gsp_rm_emulation: dump every guest GPU_PROMOTE_CTX (0x2080012b) params and
entries so the real client's promote shape is observable (captured on the
emulation path: MAIN 2 MB PA+VA, PATCH, BUNDLE_CB/PAGEPOOL/ATTRIBUTE_CB/
RTV_CB_GLOBAL VA-only, FECS, PRIV, UNRESTRICTED)
- gsp_phys_booter: promote mode 19 replays that exact 9-entry shape on the V5
channel with host buffers (MAIN 2 MB + PDB map, small buffers, booking-only
VAs); clamp 0..19
- docs/eng-ops: mode-19 result recorded - the server never replied (fn76 -110)
and the release fail-safe held WPR2/GPU until an approved reboot; mode 19 is
known-wedging
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registry variant; complete vgsp rename; docs/tooling refresh
- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA,
batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel,
old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only,
PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM
geometry queries and buffers; pre-boot SET_REGISTRY profile selection
- gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf
parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0
- vgsp rename across kernel, scripts, tools and docs (module vgsp.ko,
/sys/kernel/debug/vgsp/, vgsp-<subsystem>: log prefixes)
- risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag
tool updates; README/SBOM/install/Makefile sync
Live status: CE ladder and P6 guest-copy forwarding green; GR engine context
commit still open (all promote shapes accepted or cleanly refused; the channel
stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Host-freeze fix bundle (validated 2/2 supervised cycles):
- power unlock now latches D0/D3cold by default (new gsp_phys_repower,
default 0); releasing the claim can no longer power-gate the board and
hard-freeze the host on resume
- bounded post-rebind readiness gate before dropping the locks; bounded D0
settle/verify before the unbind
- READY means a live GSP-RM session: the async worker and the module-owned
join only expose READY when init-done is seen and no unload was sent;
otherwise the claim is released fail-open
- FWSEC reads DMEM0/IMEM back before STARTCPU and fails fast when the
staging is not visible
- the claim harness leaves the board latched by default and takes
--restore-power to opt back into the old restore
Guest-wiring:
- P2: generation-keyed fn65 static-config snapshot cache plus a default-off
identity overlay (new gsp_phys_static_live) for name/short-name/VBIOS;
the probe cycle runs once per backend generation at READY
- P3: default-off read-only RM_CONTROL forwarding whitelist (new
gsp_phys_ctrl_forward) with emulated-handle translation to the adopted
server handles and full emulation fallback
- P5 wave 1: default-off shadow execution of guest CE pushes on the ladder
channel (new gsp_phys_work_forward) via a dedicated ordered work item;
the ladder stage-4 submission is refactored into a shared bounded helper
and the software release path stays authoritative
Live: module-owned boot to init-done, guest smoke ALL OK 64/64 with P2/P3
enabled, 514 shadow copies observed on the physical CE; all cycles released
cleanly with WPR2 down and no reboot.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
helper gates on
The static CUDA runtime's device init failed with cudaErrorInitializationError
even though every kernel ioctl returned 0: cuInit succeeded and then
cuDeviceGetAttribute(ECC_ENABLED=36) returned CUDA_ERROR_NOT_FOUND (500).
Tracing libcuda's clock-domain helper (libcuda+0xe8c410) shows it issues the
CLK legacy controls 0x20809009/0x20809064/0x20809001 and bails with 500 when
the replies do not carry the expected domain bits/entries; the dependent
attribute getters (ECC_ENABLED, TOTAL_CONSTANT_MEMORY) then abort init.
Reply to the three queries with the minimal fields the helper checks: bits
32|34 present in 0x20809009, non-zero domain entries at offsets 12/20/28/36
of 0x20809064, and a matching selector in 0x20809001. No frequencies are
fabricated (all other fields stay zero).
Verified in the guest with the stock 570.144 static-cudart vectorAdd client:
cudaGetDeviceCount -> rc=0 (ndev=1), cudaSetDevice, cudaFree, three
cudaMallocs and both HtoD copies all return rc=0; the client then parks in
cudaDeviceSynchronize waiting for the (not yet forwarded) kernel launch.
|
| |
|
|
|
|
|
|
|
|
| |
The control is cacheable in the guest kernel (RMCTRL_FLAGS_CACHEABLE)
and rmapiControlCacheSet validates paramsSize against the canonical
sizeof(NV2080_CTRL_BUS_GET_PCI_BAR_INFO_PARAMS) = 200. We replied only
8 + 4*24 = 104 bytes, so every reply tripped
NV_ERR_INVALID_PARAMETER in rmapiControlCacheSet (rpc.c:9998) and the
BAR-info cache entry was never populated. Zero-fill and report all
eight NV2080_CTRL_BUS_PCI_BAR_INFO entries (count still 4).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A healthy run left dmesg dominated by per-operation lines (sema-push,
sema-lda, sema-chan rel-apply, sema-grce, DOORBELL/BAR writes, GSPRM
per-RPC lines, the teardown garbage-element stream). Introduce the
module parameter sema_dbg (default 0) and route every high-frequency
diagnostic through it; the finer-grained watch knobs (sema_ops_log,
sema_copy_watch, sema_copy_pat, sema_copy_dst, sema_copy_dst2) only
take effect while it is on. Boot stats, channel registrations, pin
events and state changes stay unconditional.
Also carries the per-side translation provenance (op_tr_src, printed
as prov= in the destination-window watch) and the 16 KB destination
window used to observe the guest's page-table writes.
Verified: fresh boot + full cdt2 smoke test (64/64, ALL OK) leaves a
520-line dmesg with zero per-push/per-doorbell/per-RPC lines.
|
| |
|
|
| |
TPC count)
|
| |
|
|
| |
watcher reaping, wider CE stream window
|
| |
|
|
|
|
|
|
|
|
|
| |
NVA06C_CTRL_CMD_INTERNAL_PROMOTE_FAULT_METHOD_BUFFERS is sent at
channel-group allocation; it hands the GSP each runqueue's 12 KB CE fault
method buffer as {base (GPU physical), size, alignment, addressSpace,
cpuCacheAttrib} plus a bar2Addr. Parse and log them, and keep the last
set per channel-group object so the addresses (guest structures we cannot
otherwise enumerate) are available to the engine emulation. Live: two
12 KB buffers per group, base = guest PA, addressSpace = 1, buffers
zeroed (no faults posted yet).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
record
A freed client's notifier events (hSrcResource 0) stayed valid in the event
table, so the fallback wake-up nudge kept posting POST_EVENT records for a
(hClient, hEvent) pair the guest RM no longer knows; every one asserts in
CliGetEventInfo (kernel_gsp.c:466) and is dropped (171 failures in the
15:24 window of the last run, zero after this fix). FREE now retires the
matching event record (and a client's records when its root object is freed).
bp999.py now walks rdi -> ctx -> ctx[0x1291] -> +0x10 -> +0x10 and prints the
RM error notification record, and push_guest_tools.sh ships it to the guest.
|
| |
|
|
| |
shares GR's runlist/PBDMA
|
| | |
|
| |
|
|
|
|
| |
Guest driver was receiving SUCCESS replies as 'unexpected events' (0x48, 0x49).
These async init RPCs are fire-and-forget per GSP-RM protocol; driver doesn't
expect a reply on the status queue. CONTINUATION_RECORD still sends SUCCESS.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Added TU104_CTRL_INTERNAL_MEMSYS_L2_INVALIDATE_EVICT (0x20800a6c)
- Added TU104_CTRL_INTERNAL_GR_GET_FECS_TRACE_HW_ENABLE (0x20800a38)
- Added TU104_CTRL_SYSTEM_MAP_SHARED_DATA (0x730151)
- Fixed async RPCs (72 GSP_SET_SYSTEM_INFO, 73 SET_REGISTRY, 71 CONTINUATION_RECORD)
to send SUCCESS replies instead of silent consume
- Added explicit handlers for RPC 47 (UNLOADING_GUEST_DRIVER) and 70 (UPDATE_BAR_PDE)
with proper reply semantics
Guest driver was failing at GSP-RM level with unknown RPCs 47, 70, 72, 73.
Channels registered then immediately unregistered.
Now they should stay registered and complete initialization.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- re-post the completion event for a bounded window after a channel
completes work (notifier consumed or semaphore release applied), so a
client that starts waiting after the edge-triggered wake-up still gets
a wake-up; the OS event classes carry no source object, so the hint
broadcasts to the registered events
- add rmstate (dump msgq cursors, guest consumption position, event
table) and rmpost (post an event record by index or to all) debug
parameters, both executing in kthreads because the IOMMU read path
only works with a NULL current->mm or the mapping owner's mm
- report the underlying error codes from the event post path
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- admit {GPU VA -> PA} pairs only when the preceding bytes form a valid
client memory descriptor header (type 0/1, CPU+GPU mapped bits), prefer
validated pairs over headerless candidates under first-wins dedup, and
invalidate the translation cache on every rebuild; fixes stale identity
entries shadowing live pool pages and lets the guest UVM channel
bring-up complete on its own
- capture OS event allocations (class 0x79) and post event records
(function 0x1003) into the guest status queue when a channel's busy
notification is consumed, with a broadcast fallback, serialized with
replies via a queue lock
- add BAR0 shadow introspection (dump_bar0/scan_bar0) and guest-side
diagnostic scripts for the VM cycle
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The guest's CUDA stack was stuck inside context creation because the work it
submits is never executed by the emulated GPU. Two forms of pushed work were
missing, and the completion path had grown far too slow to be useful:
- Channel tracking: on a channel allocation parse the GPFIFO GPU VA and entry
count, the USERD address/aperture and the instance block, hand them to the
progress service and drop them again when the channel is freed.
- Host-class (C46F) semaphore releases: pushes on a compute channel signal
progress with a five-dword method run (SEM_ADDR_LO/HI, PAYLOAD_LO/HI,
EXECUTE) rather than the copy-engine group the service already understood,
so those releases were never applied. Decode both forms.
- Emulated vidmem: channel blocks (USERD/instance) and some client semaphores
live in the emulated frame buffer, whose VAs are a window over the BAR1
backing rather than guest RAM; access them through the right aperture.
- Latency: the service's full-RAM sweep took seconds per pass, so guests'
progress waits took whole seconds and the driver tore its own setup back
down in the meantime. Sweep guest RAM in bounded, chunked windows on every
pass, remember the pages pushes are actually found in and revisit those
first, and keep the VA->PA map complete (dedup, sort, binary search) so
semaphores resolve reliably.
Also add a vidmem poke parameter for bringing up and debugging emulated
frame-buffer state.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- GSP_RM_ALLOC: echo the request payload back with status forced to NV_OK.
The kernel driver copies the reply params over the caller's params buffer
using the class-expected size, so a zeroed reply smashed libcuda's stack
(SIGSEGV at the engine-registration continuation); echoing makes the
copy-back idempotent.
- CE_GET_PHYSICAL_CAPS / CE_GET_ALL(_PHYSICAL)_CAPS: report sysmem, P2P and
BL copy caps and mark the GR-synchronous CE (GRCE, COPY0). Without a GRCE
the client's context walk ends with an empty secondary-engine record and
the engine RM alloc is issued with class 0 (rejected -> cuCtxCreate fails).
- TU104 engine table: CE0 shares the GR PBDMA ids (the driver's GR/GRCE
runqueue validation requires it), plus an entry for the async copy engine
(COPY14) advertised in the GSP static info engine caps.
- tools/guest-diag/ioctl_trace: decode the RM status fields of alloc,
control and free so the first failing driver call is visible in traces.
gitignore: keep the CUDA analysis scratch tree (incl. WORKING-HERE.md) local.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
