| Commit message (Collapse) | Author | Age |
| |
|
|
|
|
|
|
|
|
|
|
| |
GPU VAs
Fill the external VAS root's GP10X v2 hierarchy (root/PD2/PD1/PD0-small/PT)
around a 2 MB VA window so the PDBMA/CE can fetch the GPFIFO and pushbuffer
by GPU VA; gpFifoOffset and the GPFIFO ring entry GET now carry those VAs.
Allocate the USERD page in FBMEM (class 0x40, matching the reference default)
with the sysmem chunk as fallback, and add the advisory TSG-level
GPFIFO_SCHEDULE 0xa06c0101 after the channel BIND/schedule. Log the window,
table DMAs and entries, and expose them in the debugfs state snapshot.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the notif index advisory
The class-0x40 NV_MEMORY_ALLOCATION_PARAMS reached the server with owner=0,
which the class constructor rejects with NV_ERR_INVALID_OWNER (0x39). Set
owner=HEAP_OWNER_RM_CLIENT_GENERIC (0xDEAF000C) and internalflags=
NVOS32_ALLOC_INTERNAL_FLAGS_SKIP_SCRUB (0x4), matching the reference notifier
allocation. Add static_asserts pinning owner/attr/size/offset/hVASpace/
internalflags to the reference 128-byte ABI so a layout drift breaks the build.
SET_WORK_SUBMIT_TOKEN_NOTIF_INDEX (0xc36f010a, index 3) now logs an advisory
warning on failure (no error-context object exists, so the server returns
NV_ERR_OBJECT_NOT_FOUND 0x57) and the stage continues to GET_WORK_SUBMIT_TOKEN:
the server writes the work-submit token before the notifier update, so the
token is independent of the notifier object.
|
| |
|
|
| |
submission
|
| |
|
|
|
|
|
|
|
| |
The TU104 GMMU root level has 2 index bits (4 PDEs of 2^47 bytes each), so
the server's vaLimitNew = (numEntries << 47) - 1 must stay within its
vaLimitMax = 2^49-1; numEntries=256 computed 2^55-1 and drew
NV_ERR_INVALID_ARGUMENT (0x1f). Send 4 entries (32 bytes of the 4 KB
page-aligned VASPDB chunk) and log the value on failure. The aperture/flags
word is unchanged (0x0a, SYSMEM_NONCOH + ALL_CHANNELS).
|
| |
|
|
| |
channel ladder (V5)
|
| | |
|
| |
|
|
| |
gsp-owned, server-allocated memdescs)
|
| |
|
|
| |
and decode rm-call statuses
|
| |
|
|
| |
static-config-handle ladder path
|
| |
|
|
| |
allocation and a first real copy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
identity, and request the full static-config payload
The RM log-ring init argument carried a malformed libos id (LOGM), so the
server never mapped the ring and its early failure path was invisible.
The pre-boot system-info payload carried unpacked 16-bit identity fields;
the server expects the raw config-space dwords (device<<16|vendor), so its
GPU device-id read back as zero and the static-config device-name lookup
returned NOT_SUPPORTED.
The static-config RPC must be sized to the full reply payload, and the
guest-system-info RPC is sent first, matching the reference order.
Live result: GSP-RM completes initialization (init-done rpc_result=0),
serves the 1656-byte static config (name, FB size, handles, PDE bases),
and shuts down cleanly through the unload RPC.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fill the GspSystemInfo chipset fields the production client syncs before
the GSP boot instead of leaving them zero:
- clPdbProperties (bitmask in the client's clSyncWithGsp order): bit 0
(config space accessible, from the OS MCFG/ECAM availability), bit 7
and bit 21 (Intel Skylake-family upstream-port setup: ASPM POR list,
root-port nosnoop WAR), bit 13 (upstream port ASPM L1 capable), bit 27
(LTR supported+enabled up to the root port) and bit 32 (IO coherent).
PDB_PROP_CL_UNSUPPORTED_CHIPSET is never assigned by the client, so the
faithful unmatched-chipset value is 0.
- Chipset = 0 (CS_UNKNOWN) for a host bridge with no chipsetInfo[] entry,
the reference's no-match fallback.
- FHBBusInfo/chipsetIDInfo: 10-byte BUSINFO blobs of the first host
bridge of the GPU's domain (device, vendor, subdevice, subvendor,
revision), with the subsystem-capability fallback on chipsetIDInfo.
acpiMethodData is intentionally sent all-zero (bValid=0): no ACPI/AML
calls are made and every consumer tolerates the no-data encoding. All
device lookups use refcounted pci_get_device()/pci_dev_put().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
experiment)
The working distribution driver runs with display enabled; the
RmDisableDisplay=1 deviation changes board/display state and may be
implicated in the devid/board validation failure blocking GSP-RM init.
Regenerate the profile without it: the 35 host default parameters plus
RMDevidCheckIgnore=1, 36 entries total, display brought up as a stock
boot would.
Same canonical packing: size 1344, numEntries 36, rpc.length 1376,
record 1392 (8-byte aligned), one 4 KB element. The module is unchanged
apart from comments and the prepared log line, which now reports
'host default profile + RMDevidCheckIgnore=1, display enabled'.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Append RMDevidCheckIgnore=1 (dword) to the pre-boot SET_REGISTRY
profile, after the 35 host default parameters and RmDisableDisplay=1.
The key is the registry escape for the static-config devid/board check
that currently fails GSP-RM init; every existing entry is unchanged.
Regenerated kernel/gsp_phys_registry_img.h with the same canonical
packing (values inline, names-only blob): size 1377, numEntries 37,
rpc.length 1409, one 4 KB element. The module code is unchanged apart
from comments: the submission and the prepared log line read the size
and entry count from the generated blob/macros.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GSP-RM init
The pre-boot SET_REGISTRY (function 73) submitted an empty
PACKED_REGISTRY_TABLE. Replace it with the default registry profile the
host driver sends before boot: the 35 default dword parameters from its
parameter table plus RmDisableDisplay=1. The display key makes the GSP-RM
display-presence check treat display as absent, so the display engine is
removed during pre-init instead of being brought up on a headless boot.
The payload is generated into kernel/gsp_phys_registry_img.h as a
PACKED_REGISTRY_TABLE blob (8-byte header, 36 16-byte dword entries,
NUL-terminated names after the array; values inline; no value bytes in
the name blob, matching the reference packer): size 1342, numEntries 36,
rpc.length 1374, one 4 KB element. fn72 is unchanged.
The payload length is not 8-byte aligned (record 1390 -> 1392); the TX
producer already zero-pads the kvzalloc'd staging record and folds the
XOR checksum over the padded length, exactly like the reference producer
zero-pads before _checkSum32, so no producer change is needed. The blob
is static const data: no heap, no new stack frame, and a submit failure
still aborts the boot. The prepared log line now prints the payload size
and entry count.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The real-GSP boot path staged the message queue but never queued the two
fire-and-forget init RPCs the production client places in it before the
booter starts. GSP-RM therefore created its GPU object without any system
information and aborted RM init (live GSP_INIT_DONE rpc_result=0x59), with
the resource-manager's object-database lookups asserting from the first
journal record.
Submit function 72 (GSP_SET_SYSTEM_INFO, 928-byte v17_00 payload carrying
the BAR physical bases, domain/bus/slot, PCI IDs, the BAR0 config-mirror
offset, link/FLR capabilities and upstream-port flags read from the
pci_dev) and function 73 (SET_REGISTRY, header-only empty table, which the
server explicitly accepts) through the existing submit-only queue producer
after all stage-3d preconditions and before the GSP falcon reset/Booter
Load kick. The wire layouts are re-declared in the module with per-field
static asserts against the reference ABI; payloads are heap-allocated, so
no new stack frames are introduced.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
A zero REG_POLL timeout dword is the reference's GPU_TIMEOUT_DEFAULT
sentinel: the reference substitutes the platform default (microseconds;
4 s in graphics mode, the default platform) before waiting. The executor
clamped zero to the 1 us floor, i.e. one jiffy, so a poll that was not
already satisfied failed the boot in 1 ms.
The live 0x1002 sequencer element contains exactly one REG_POLL with
timeout=0 (addr 0x110040); substitute the 4 s graphics-mode default and
then keep the existing 1 us .. 10 s clamp.
|
| |
|
|
|
|
|
|
|
| |
The live 0x1002 element carries bufferSizeDWord=0x3FE2 (16354) - the size
of GSP-RM's own command buffer, not the message. The executor's 0x1000
sanity cap refused it, so the sequencer was never run and GSP_INIT_DONE
was never observed. The reference imposes no upper bound (only nonzero and
greater than cmdIndex); raise the cap to a sanity ceiling and document
that the real execution bound is cmdIndex against the element payload.
|
| |
|
|
|
|
| |
Applies the adversarial review fixes for the command-queue TX and unload RPC: the unload RPC is gated on GSP-RM init-done, debugfs fops pin the module, REG_POLL timeouts use microseconds, the host status readPtr is published, TX free space is checked, swap flags and barriers are corrected, and the legacy payload fallback is opt-in.
Generalizes the transport to multi-element records and a reply-matching status-queue reader, and adds a debugfs probe that round-trips GET_GSP_STATIC_INFO (with a SET_GUEST_SYSTEM_INFO fallback), decodes the server configuration, and a read-only state snapshot file. All gated by gsp_phys_boot; the normal load path is unchanged.
|
| |
|
|
| |
Adds a minimal msgq producer on the command queue (element layout, checksum, sequence, dma_wmb before writePtr, doorbell) and submits the UNLOADING_GUEST_DRIVER RPC so GSP-RM shuts down and sets the scratch14 handoff bit the Booter Unload gate needs. Release attempts the RPC once before quiescing when a boot ran; on-demand testing is available through the debugfs file gsp-phys-unload-rpc while the module owns the GPU. GSP-RM processor suspend is polled on the GSP mailbox.
|
| |
|
|
|
|
|
|
| |
Stage 3d stages the GSP-RM boot from host sysmem (FMC, radix3 over the GSP-RM ELF, signature, message queues, libos LOGINIT/LOGRM/RMARGS and GSP arguments) and runs Booter Load on SEC2: GSP RISC-V reset, libos mailbox, IMEM/DMEM direct load, BOOTVEC=0, M0==0, FALCON_OS, RISC-V active. Stage 3e links the status queue and polls for GSP-RM events; it now also executes the inline run-sequencer command buffer (event 0x1002) against BAR0 (REG_WRITE/MODIFY/POLL, DELAY, STORE, CORE_* including CORE_RESUME) and then waits for GSP_INIT_DONE 0x1001.
Gated by gsp_phys_boot / gsp_phys_boot_dryrun (default off; the normal load path writes nothing). gsp_phys_fwsec=1 without gsp_phys_boot=1 is refused: a FWSEC-only carve cannot pass the booter-unload handoff gate. Release quiesces the GSP, runs the existing FWSEC-SB/booter-unload teardown and frees all staging.
tools/gen_gsp_bindata.py decodes the public generated archives into the Booter Load image (prod signature patched at 0x8700) and the GSP-RM boot (FMC) image; both are self-checked at build time.
|
| | |
|
| |
|
|
| |
release
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
bind-event hardening)
|
| |
|
|
| |
strict board check)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage 3b allocates the three coherent boot-argument pages and fills the
256-byte WPR metadata from the computed Turing FB layout; no device
registers are written (uncommitted work from the previous session,
committed as-is behind the new gate).
The physical claim unbinds the distribution NVIDIA driver; on this host
that power-gates the dGPU board and the re-bind can leave the GPU
unresponsive and hard-freeze the system (observed twice, 2026-09-30, no
crash record). The claim now refuses unless the new gsp_phys_hazard_ack
module parameter is also set, so a stale command line cannot re-trigger
it. See the hazard note at the top of gsp_phys_gpu.c.
|
| |
|
|
| |
(no device writes)
|
| |
|
|
|
|
|
| |
Logs both falcons' run/halt bits, the RISC-V active bit, WPR2 up/down and
the security-falcon mailboxes when the physical GPU is claimed. Verified
live: GSP and SEC2 halted, RISC-V inactive, WPR2 down (HI=0). This is the
read-only half of the real boot rung; no device state is written.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rung 1 (ownership): a default-off gsp_phys_gpu module param; on claim the
distribution driver is released from 10de:1ed3, our pci_driver binds it,
BAR0/1/3 are mapped and the falcon state is sampled; release unbinds ours
and rebinds the distribution driver. Verified live: claim reads
BOOT0=0x164000a1 with the falcon halted and mailboxes clear; rmmod rebinds
nvidia and nvidia-smi stays healthy.
Rung 2 (firmware): while the device is owned, gsp_phys_fw (default
nvidia/570.211.01/gsp_tu10x.bin) is fetched with request_firmware, checked
as ELF64 RISC-V, and its sections are logged (.fwimage, .fwversion,
.fwsignature_tu10x). The image is held for the boot rung.
The claim is default-off; the emulation path is unchanged.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The per-put-epoch consumed-slot skip must not suppress a slot whose entry
was consumed *without* delivering its release (the CE grace expiry in real
mode): the driver-API smoke test's CE pushes are consumed unreadable before
their ring pin lands and depend on the pin-batch rewind replaying them once
the translation appears. The skip now applies only to slots whose
consumption delivered a release:
- tail consumption records a slot when the iteration decoded a release
(svc_rel_count delta); the sweep-delivered entry_want consume and the
fake-release consume always record
- the grace-expiry consume records only when a recovery (fake or
sema_rel_recover) delivered the release
- svc_fake_release counts as a delivered release
So real-mode replays still re-process undelivered entries, while the
runtime-client case (readable pushes delivered immediately) keeps the
fast path that removed the 8 s stalls.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pump decided "this push carried a release" by comparing the last
decoded release VA before/after svc_decode_stream. Consecutive pushes on a
channel release to the SAME tracking-semaphore VA (UVM's tracker pattern),
so every push after the first read as release-less: the entry was held for
the full CE grace window (observed: exactly one push consumed per 8 s,
gating the whole runtime init; the release itself had already been applied
by the decode, only the pump's bookkeeping stalled).
svc_rel_count now increments on every decoded release (host sem block,
REPORT_SEM block, CE launch-DMA release) and the pump compares counts at
all three sites (initial decode, rescue re-route, rescued-page retry).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A pin-batch rewind replays recently consumed entries. An old entry whose
rescue cannot validate any candidate then sits in the CE grace path - and
because the client's put has advanced since, the replayed entry no longer
matches any "new" expectation: it waits the full 8 s grace, blocking every
later entry behind it on that channel. Observed live: the first push of
rank 0 re-processed and skipped twice, 38 s apart, while eight later pushes
queued behind it.
The pump now marks each consumed slot in a per-channel bitmap valid for the
current put epoch (cleared when put wraps, so a recycled slot in the next
epoch is processed normally) and consumes a replayed slot immediately.
Rewinds still refresh readable entries; they simply no longer resurrect
entries that were already handled.
Evidence: real-release watcher cycle, replay-after-put-advance no longer
produces 'skipping stale entry ... after 8000 ms grace' for old slots.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two stalls serialized the runtime client's init into one 8 s CE grace window
per push:
- a pin-batch rewind replays recently consumed GPFIFO slots; re-examining a
consumed slot runs the rescue and then holds it for the full grace window
again (observed: the same VA 'skipped after 8000 ms grace' twice, 38 s
apart). The pump now records each consumed slot with the put value it was
consumed at (32-entry per-channel replay ring, persisted across passes)
and consumes a replayed slot whose producing put has not advanced
immediately.
- the content sweep applies releases from RAM pages before the pump reads
their entries, so the pool advances past the entry's payload while it is
pending. entry_want (recorded on first examination) now detects that on
both the unreadable-rescue and the readable-no-release paths and consumes
the entry instead of waiting the grace window.
Evidence: real-release watcher cycle, pool semaphore advances no longer
gated at 8 s intervals once the replay skip engages.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the sweep already applied
The runtime client's second push on each UVM copy channel starts with a
NON_INC method (0x60080002, count 8) and its release method sits 0x50 bytes
in. The stream validator accepted only INC (op 1) headers, so the readable
page failed the plausibility gate and every content-rescue candidate for
that VA failed the read-back validation - the client's tracking semaphore
for the push never advanced and the runtime init parked.
- one shared parser (svc_stream_parse) accepts INC/NON_INC (op 1/3),
IMMD_DATA (op 4, header-only), ONE_INC (op 5) and END_PB_SEGMENT (op 7),
and still requires an exact method boundary over the whole entry plus
trailing zero padding - the strict end condition is what keeps the
content rescue safe against firmware/allocator pages
- svc_method_stream_plausible and svc_stream_full_ok both use it, so a real
readable push is decoded directly and a rescued page validates
- the decoder skips NON_INC/ONE_INC methods whole (including their data
words) instead of resyncing word-by-word through the payload, which could
misread data as headers and queue bogus work
Follow-on: the content sweep applies releases from RAM pages ahead of the
pump, so the pump's expected payload was already satisfied when it examined
the entry; the CE grace path then held each entry for the full 8 s window
before consuming it - one grace period per push, serializing the client's
runtime init. The pump now records the expected payload when it first
examines an entry (entry_want) and consumes the entry immediately once the
pool advances past it, on both the unreadable-rescue and the
readable-no-release paths.
Evidence (fresh boot, real releases, watchers + kprobes, no fakes):
cudaGetDeviceCount rc=0 in the runtime client, device-attribute walk
proceeds, pool semaphores advance; before this change the client sat in the
0x20801702 service-interrupt loop for >6 min and never left device init.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The runtime client's registration stalled when no guest-side pin scanner was
running: the content rescue refused to run before the pool semaphore was
translated (its expected payload was unlearned), and unreadable CE entries
were consumed after two failed rescues ~40 ms apart, dropping the client's
wait. The push pages then could not be found within the entry's grace window.
- rescue: drop the learned-payload gate (the matcher only needs the channel
rank and the release shape); keep unreadable CE entries pending for
sema_ce_grace_ms instead of consuming after a fixed attempt count
- expected payload: re-learn from the pool whenever it advanced (a deferred
release could land after the first read, leaving the expectation stale)
- matcher: require the expected payload exactly and validate the read-back
stream carries it, so a recycled page of an older push cannot be pinned
- scan: descending full-range pass (the work allocations live in the high
RAM block) plus a +/-512 MB near pass around the verified push-buffer base
- releases decoded with an untranslated semaphore are queued and applied
when the page appears (svc_pending_add on both decode paths)
- a CE push that reads as a plausible stream but carries no release for the
channel is re-routed through the rescue instead of silently consumed
- new module param sema_rel_recover (default off): after grace + rescue,
apply the channel's expected tracking value so the client's wait clears;
counter-visible as releases_recovered, distinct from sema_fake_rel
Evidence: fresh boot, real releases, no host watcher, no kprobes - the six
UVM CE first pushes are found by content rescue, their releases applied, and
the client reaches cudaGetDeviceCount rc=0 (it then waits for the third
push, which this change routes to the rescue).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
helper gates on
The static CUDA runtime's device init failed with cudaErrorInitializationError
even though every kernel ioctl returned 0: cuInit succeeded and then
cuDeviceGetAttribute(ECC_ENABLED=36) returned CUDA_ERROR_NOT_FOUND (500).
Tracing libcuda's clock-domain helper (libcuda+0xe8c410) shows it issues the
CLK legacy controls 0x20809009/0x20809064/0x20809001 and bails with 500 when
the replies do not carry the expected domain bits/entries; the dependent
attribute getters (ECC_ENABLED, TOTAL_CONSTANT_MEMORY) then abort init.
Reply to the three queries with the minimal fields the helper checks: bits
32|34 present in 0x20809009, non-zero domain entries at offsets 12/20/28/36
of 0x20809064, and a matching selector in 0x20809001. No frequencies are
fabricated (all other fields stay zero).
Verified in the guest with the stock 570.144 static-cudart vectorAdd client:
cudaGetDeviceCount -> rc=0 (ndev=1), cudaSetDevice, cudaFree, three
cudaMallocs and both HtoD copies all return rc=0; the client then parks in
cudaDeviceSynchronize waiting for the (not yet forwarded) kernel launch.
|
| |
|
|
|
|
|
|
|
|
| |
The control is cacheable in the guest kernel (RMCTRL_FLAGS_CACHEABLE)
and rmapiControlCacheSet validates paramsSize against the canonical
sizeof(NV2080_CTRL_BUS_GET_PCI_BAR_INFO_PARAMS) = 200. We replied only
8 + 4*24 = 104 bytes, so every reply tripped
NV_ERR_INVALID_PARAMETER in rmapiControlCacheSet (rpc.c:9998) and the
BAR-info cache entry was never populated. Zero-fill and report all
eight NV2080_CTRL_BUS_PCI_BAR_INFO entries (count still 4).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A healthy run left dmesg dominated by per-operation lines (sema-push,
sema-lda, sema-chan rel-apply, sema-grce, DOORBELL/BAR writes, GSPRM
per-RPC lines, the teardown garbage-element stream). Introduce the
module parameter sema_dbg (default 0) and route every high-frequency
diagnostic through it; the finer-grained watch knobs (sema_ops_log,
sema_copy_watch, sema_copy_pat, sema_copy_dst, sema_copy_dst2) only
take effect while it is on. Boot stats, channel registrations, pin
events and state changes stay unconditional.
Also carries the per-side translation provenance (op_tr_src, printed
as prov= in the destination-window watch) and the 16 KB destination
window used to observe the guest's page-table writes.
Verified: fresh boot + full cdt2 smoke test (64/64, ALL OK) leaves a
520-line dmesg with zero per-push/per-doorbell/per-RPC lines.
|
| |
|
|
| |
RAM scan (the DtoH dst staging VA needs multiple scan slices)
|