| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
| |
The guest announces the display instance-memory block (8 KB
context-DMA hash table + entries, vidmem PA 0x1fff0000 on r1627) via
NV2080_CTRL_CMD_INTERNAL_DISPLAY_WRITE_INST_MEM (0x20800a49), but the
engine ignored it and scanned its FB shadow blindly, never finding the
hash entry - so SET_NOTIFIER_DMA handle 0x10013 resolved to pa=0 and
UPDATE never wrote FINISHED (r1627: UPDATE armed=1 notifier=0x0/0).
Handle 0xa49, record pa/size/address space per guest, read the block
through the FB shadow (or guest RAM for sysmem), and parse the hash
table from the announced location with bounded diagnostics.
|
| |
|
|
|
|
|
|
|
|
| |
nv3dAllocDevice reads SM_VERSION (GR info index 0x0c) through the 3D
library; with the emulated KGR reply leaving it 0 it gets
SM_VERSION_NONE, GetSpaVersion fails, nvHsAllocDevice returns FALSE
silently and nvidia-drm reports 'Failed to allocate NvKmsKapiDevice'
(r1626 gdb: nvHsAllocDevice -> nvHs3dAllocDevice -> nv3dAllocDevice).
Fill SM_VERSION=0x705 (7.5) plus MAX_WARPS_PER_SM=64,
MAX_THREADS_PER_WARP=32 and THREAD_STACK_SCALING_FACTOR=1.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
raster)
The guest nvidia-modeset drives the display through per-channel NV-DMA
push buffers kicked via the BAR0 UDISP control windows. Emulate that
engine contract host-side, unmodified guest:
- channel registration: display-channel RM objects (core 0xc57d, window
0xc57e, cursor 0xc57b, 0xc57a) are noted from the alloc stream, and the
internal control NV2080_CTRL_CMD_INTERNAL_DISPLAY_CHANNEL_PUSHBUFFER
(0x20800a58) records each push buffer (guest PA, size, class,
instance, validity);
- PUT polling + GET publication in the channel control pages (core
0x680000, window 0x690000+n*0x1000, cursor 0x6D8000+n*0x1000);
- NV-DMA method parser (METHOD/ONE_INC/NONINC/JUMP/SET_SUBDEVICE_MASK)
with the wrap-JUMP follow;
- completion: SET_CONTEXT_DMA_NOTIFIER / SET_NOTIFIER_CONTROL tracking,
exact notifier-surface resolution through the display instance hash
table (no heuristic writes), NV_DISP_NOTIFIER FINISHED on UPDATE
(0x200), and the channel completion event post;
- free-running 1080p60 raster line for NVC57E_GET_LINE.
All effects are behind independent knobs (disp_engine, disp_finish,
disp_vblank, disp_events, disp_notifier_pa) with verbose diagnostics
(disp_dbg, disp_dump), so the previous behaviour stays one load away.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
nvCEChooseEngine skips every CE without CE_SUPPORTS_NONPIPELINED_BL
(caps byte 1, 0x01) and, in the preferred pass, PIPELINED_BL (0x02). Our
emulated CE caps left byte 1 zero, so every CE failed the filter and the X
driver aborted with 'Failed to find a suitable copy object' - the wall
after the DMA pushbuffer test started passing (r503). Turing CEs are fully
capable (bug 200288104 case (a)): byte1 = 0x03.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
helper gates on
The static CUDA runtime's device init failed with cudaErrorInitializationError
even though every kernel ioctl returned 0: cuInit succeeded and then
cuDeviceGetAttribute(ECC_ENABLED=36) returned CUDA_ERROR_NOT_FOUND (500).
Tracing libcuda's clock-domain helper (libcuda+0xe8c410) shows it issues the
CLK legacy controls 0x20809009/0x20809064/0x20809001 and bails with 500 when
the replies do not carry the expected domain bits/entries; the dependent
attribute getters (ECC_ENABLED, TOTAL_CONSTANT_MEMORY) then abort init.
Reply to the three queries with the minimal fields the helper checks: bits
32|34 present in 0x20809009, non-zero domain entries at offsets 12/20/28/36
of 0x20809064, and a matching selector in 0x20809001. No frequencies are
fabricated (all other fields stay zero).
Verified in the guest with the stock 570.144 static-cudart vectorAdd client:
cudaGetDeviceCount -> rc=0 (ndev=1), cudaSetDevice, cudaFree, three
cudaMallocs and both HtoD copies all return rc=0; the client then parks in
cudaDeviceSynchronize waiting for the (not yet forwarded) kernel launch.
|
| |
|
|
| |
TPC count)
|
| |
|
|
| |
shares GR's runlist/PBDMA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Added TU104_CTRL_INTERNAL_MEMSYS_L2_INVALIDATE_EVICT (0x20800a6c)
- Added TU104_CTRL_INTERNAL_GR_GET_FECS_TRACE_HW_ENABLE (0x20800a38)
- Added TU104_CTRL_SYSTEM_MAP_SHARED_DATA (0x730151)
- Fixed async RPCs (72 GSP_SET_SYSTEM_INFO, 73 SET_REGISTRY, 71 CONTINUATION_RECORD)
to send SUCCESS replies instead of silent consume
- Added explicit handlers for RPC 47 (UNLOADING_GUEST_DRIVER) and 70 (UPDATE_BAR_PDE)
with proper reply semantics
Guest driver was failing at GSP-RM level with unknown RPCs 47, 70, 72, 73.
Channels registered then immediately unregistered.
Now they should stay registered and complete initialization.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The guest's CUDA stack was stuck inside context creation because the work it
submits is never executed by the emulated GPU. Two forms of pushed work were
missing, and the completion path had grown far too slow to be useful:
- Channel tracking: on a channel allocation parse the GPFIFO GPU VA and entry
count, the USERD address/aperture and the instance block, hand them to the
progress service and drop them again when the channel is freed.
- Host-class (C46F) semaphore releases: pushes on a compute channel signal
progress with a five-dword method run (SEM_ADDR_LO/HI, PAYLOAD_LO/HI,
EXECUTE) rather than the copy-engine group the service already understood,
so those releases were never applied. Decode both forms.
- Emulated vidmem: channel blocks (USERD/instance) and some client semaphores
live in the emulated frame buffer, whose VAs are a window over the BAR1
backing rather than guest RAM; access them through the right aperture.
- Latency: the service's full-RAM sweep took seconds per pass, so guests'
progress waits took whole seconds and the driver tore its own setup back
down in the meantime. Sweep guest RAM in bounded, chunked windows on every
pass, remember the pages pushes are actually found in and revisit those
first, and keep the VA->PA map complete (dedup, sort, binary search) so
semaphores resolve reliably.
Also add a vidmem poke parameter for bringing up and debugging emulated
frame-buffer state.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- GSP_RM_ALLOC: echo the request payload back with status forced to NV_OK.
The kernel driver copies the reply params over the caller's params buffer
using the class-expected size, so a zeroed reply smashed libcuda's stack
(SIGSEGV at the engine-registration continuation); echoing makes the
copy-back idempotent.
- CE_GET_PHYSICAL_CAPS / CE_GET_ALL(_PHYSICAL)_CAPS: report sysmem, P2P and
BL copy caps and mark the GR-synchronous CE (GRCE, COPY0). Without a GRCE
the client's context walk ends with an empty secondary-engine record and
the engine RM alloc is issued with class 0 (rejected -> cuCtxCreate fails).
- TU104 engine table: CE0 shares the GR PBDMA ids (the driver's GR/GRCE
runqueue validation requires it), plus an entry for the async copy engine
(COPY14) advertised in the GSP static info engine caps.
- tools/guest-diag/ioctl_trace: decode the RM status fields of alloc,
control and free so the first failing driver call is visible in traces.
gitignore: keep the CUDA analysis scratch tree (incl. WORKING-HERE.md) local.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
