| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
helper gates on
The static CUDA runtime's device init failed with cudaErrorInitializationError
even though every kernel ioctl returned 0: cuInit succeeded and then
cuDeviceGetAttribute(ECC_ENABLED=36) returned CUDA_ERROR_NOT_FOUND (500).
Tracing libcuda's clock-domain helper (libcuda+0xe8c410) shows it issues the
CLK legacy controls 0x20809009/0x20809064/0x20809001 and bails with 500 when
the replies do not carry the expected domain bits/entries; the dependent
attribute getters (ECC_ENABLED, TOTAL_CONSTANT_MEMORY) then abort init.
Reply to the three queries with the minimal fields the helper checks: bits
32|34 present in 0x20809009, non-zero domain entries at offsets 12/20/28/36
of 0x20809064, and a matching selector in 0x20809001. No frequencies are
fabricated (all other fields stay zero).
Verified in the guest with the stock 570.144 static-cudart vectorAdd client:
cudaGetDeviceCount -> rc=0 (ndev=1), cudaSetDevice, cudaFree, three
cudaMallocs and both HtoD copies all return rc=0; the client then parks in
cudaDeviceSynchronize waiting for the (not yet forwarded) kernel launch.
|
| |
|
|
| |
TPC count)
|
| |
|
|
| |
shares GR's runlist/PBDMA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Added TU104_CTRL_INTERNAL_MEMSYS_L2_INVALIDATE_EVICT (0x20800a6c)
- Added TU104_CTRL_INTERNAL_GR_GET_FECS_TRACE_HW_ENABLE (0x20800a38)
- Added TU104_CTRL_SYSTEM_MAP_SHARED_DATA (0x730151)
- Fixed async RPCs (72 GSP_SET_SYSTEM_INFO, 73 SET_REGISTRY, 71 CONTINUATION_RECORD)
to send SUCCESS replies instead of silent consume
- Added explicit handlers for RPC 47 (UNLOADING_GUEST_DRIVER) and 70 (UPDATE_BAR_PDE)
with proper reply semantics
Guest driver was failing at GSP-RM level with unknown RPCs 47, 70, 72, 73.
Channels registered then immediately unregistered.
Now they should stay registered and complete initialization.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The guest's CUDA stack was stuck inside context creation because the work it
submits is never executed by the emulated GPU. Two forms of pushed work were
missing, and the completion path had grown far too slow to be useful:
- Channel tracking: on a channel allocation parse the GPFIFO GPU VA and entry
count, the USERD address/aperture and the instance block, hand them to the
progress service and drop them again when the channel is freed.
- Host-class (C46F) semaphore releases: pushes on a compute channel signal
progress with a five-dword method run (SEM_ADDR_LO/HI, PAYLOAD_LO/HI,
EXECUTE) rather than the copy-engine group the service already understood,
so those releases were never applied. Decode both forms.
- Emulated vidmem: channel blocks (USERD/instance) and some client semaphores
live in the emulated frame buffer, whose VAs are a window over the BAR1
backing rather than guest RAM; access them through the right aperture.
- Latency: the service's full-RAM sweep took seconds per pass, so guests'
progress waits took whole seconds and the driver tore its own setup back
down in the meantime. Sweep guest RAM in bounded, chunked windows on every
pass, remember the pages pushes are actually found in and revisit those
first, and keep the VA->PA map complete (dedup, sort, binary search) so
semaphores resolve reliably.
Also add a vidmem poke parameter for bringing up and debugging emulated
frame-buffer state.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- GSP_RM_ALLOC: echo the request payload back with status forced to NV_OK.
The kernel driver copies the reply params over the caller's params buffer
using the class-expected size, so a zeroed reply smashed libcuda's stack
(SIGSEGV at the engine-registration continuation); echoing makes the
copy-back idempotent.
- CE_GET_PHYSICAL_CAPS / CE_GET_ALL(_PHYSICAL)_CAPS: report sysmem, P2P and
BL copy caps and mark the GR-synchronous CE (GRCE, COPY0). Without a GRCE
the client's context walk ends with an empty secondary-engine record and
the engine RM alloc is issued with class 0 (rejected -> cuCtxCreate fails).
- TU104 engine table: CE0 shares the GR PBDMA ids (the driver's GR/GRCE
runqueue validation requires it), plus an entry for the async copy engine
(COPY14) advertised in the GSP static info engine caps.
- tools/guest-diag/ioctl_trace: decode the RM status fields of alloc,
control and free so the first failing driver call is visible in traces.
gitignore: keep the CUDA analysis scratch tree (incl. WORKING-HERE.md) local.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
