| Commit message (Collapse) | Author | Age |
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The m80 window shows PGRAPH_INTR=0x00000010 (ILLEGAL_METHOD, the reserved-SW
prelude methods, same as the real driver) clearing, then 0x00200002
(SEMAPHORE|EXCEPTION) pending exactly when the QMD activates, followed by the
FECS save/unload. The real driver's launch transient carries the same
ILLEGAL_METHOD without ever setting EXCEPTION, so the exception is the
ctx-restore difference. The one R1-booked ctx VA with no PTE is PM
(0x2370000): the golden9 range mapping covers PATCH/FECS/PRIV/UNR and the four
globals (windows 17..22 after the golden9k 1->6 extension) but never PM, and
R1-before-F2 makes the image carry that VA as the PM pointer. Map it through
the ctx PT window right after the mode-35 PM FBMEM allocation, before any
engine context load and before the PTE line's first fetch.
|
| |
|
|
|
|
|
|
|
|
| |
The m75/m78/m79 launch windows show FECS_CTXSW_STATUS_FE_0.GR_INTR_REQ
(bit17) asserted through the SKED CWD ACTIVE state and a following CTXSW_REQ
(bit0) that results in the context save/unload. The real driver carries
GR_INTR only transiently during its context load and clears it before ACTIVE.
The PGRAPH interrupt latch (0x400100) names what the engine is signaling
(e.g. ILLEGAL_METHOD bit4, SEMAPHORE bit1, NOTIFY); sampled by the external
sampler and dumped in the launch post-mortem.
|
| |
|
|
|
|
|
|
|
|
| |
aborts it)
gsp_phys_rmcall_gr_lane_skip: bit0 skips #1059 CILP mode, bit1 skips #1061
timeslice, bit2 skips #1063 device-sched. Used to bisect which lane-enable
control arms the forwarded QMD enqueue and which one leaves the scheduling
state that ends in the dispatch-time context save. Also makes the m76 lane
tail independent of #1063 (always refused 0x1f by the server).
|
| |
|
|
|
|
|
|
|
|
|
| |
The real client's context-creation tail after the lane-enable trio is
#1073 NVA06C_CTRL_CMD_PREEMPT on the TSG with bWait=1, #1075
NV0080 device-sched disable, #1077 PERF_BOOST_SET_2X. The explicit preempt
makes the RM perform a clean context save/switch-out at setup time so the
first dispatch loads the context normally; without it our lane's context sits
loaded from the ladder and task activation ends in a save/unload instead of
execution (m72/m73/m75 signature). Gated by gsp_phys_rmcall_gr_lane_tail
(default 0).
|
| |
|
|
|
|
|
|
|
|
|
|
| |
caches/tasktables
The dispatch-time comparison against the live driver needs the GR context's
floor-sweep state (CWD_FS expected 0x1806 on TU104) and the SKED debug
interiors (HWW_INFO_1/2/3, CWD_T2, CACHE_0/1, L15_CACHE_0, TASKTABLE_0/1/2).
Recorded 2026-10-07: the distro driver's executing steady state shows
CWD_FS=0x1806, CWD_T0=0x3f002400 (VEID 63), SKED_ACTIVITY=0x600000c0,
PGRAPH_STATUS=0x01242001, FECS_CURRENT valid and frozen for the kernel's
lifetime, TASKTABLE zero.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
CWD_FS readback
The one-shot GR context-image initializer (0x52a94f8, pEngCtx+0x135) runs in
F2's mapped-MAIN promote and consumes the per-VAS vaLists at that moment. With
R1 after F2 the image carries zero PATCH/PM/priv-map/global pointers; the
m72/m73 sampler signature (QMD ACTIVE then freed, FECS unloads exactly at
dispatch, no fault) is consistent with the engine context restore failing on
the empty image. gsp_phys_rmcall_gr_r1_first=1 sends R1 before F2 (default 0 =
legacy order). Post-mortem now also reads CWD_FS (0x405b00).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transition recorder, sysmem instance selector
m72: append the vendor turingLocalMemUpdate block (SET_SHADER_LOCAL_MEMORY_A/B
0x790/0x794, NON_THROTTLED_A/B 0x2e4/0x2e8, WINDOW_A/B 0x7b0/0x7b4) to the
forwarded GR launch prelude; without it the channel local-memory state is zero
and the m64-m69 sampler shows SKED_HWW_ESR bit12 = SKEDCHECK05
LOCAL_MEMORY_TOTAL_SIZE latching while the CWD entry is allocated then
deallocated. Base/per-TPC size are module params for A/B.
m72: change-detecting SKED transition recorder in the launch poll (CWD_T0/1/2,
HWW_INFO_1/2, ACTIVITY, HWW_ESR, GRFIFO, FECS current/new/ctxsw status) and
extended post-mortem readbacks (HWW_ESR_EN, INFO_1/2/3, CWD_T2, watermarks).
m72-r2 sampler result: the QMD enters SKED (CWD_T0=0x11300 STATUS=ACTIVE,
CWD_T1=0x1070) and HWW_ESR stays 0 with the lmem block, but the FECS context
load never completes: NEW_CTX=0x001f42b0 (V7 FBMEM instance, VALID=0) then
CURRENT=0; the task frees (CWD_T0=0x12000 DEP=PENDING_TASK_COMPLETE). m63
(sysmem instance) had VALID=1 on the same load. m73 adds the sysmem
instance/RAMFC selector (gsp_phys_rmcall_gr_inst_sys; default 0 = FBMEM
capture shape) for the A/B.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
forwarded QMD enqueues/schedules; engine-core execution is the remaining wall
(a) The TSG+share lane's compute-object construct returns success after a
10-entry VA-only booking promote books the server's per-VAS ctx-buffer maps
(a mapped MAIN promote alone leaves them unbooked because the construct
skips its own mapping once MAIN is initialized).
(b) The forwarded guest QMD now enqueues and is scheduled: the missing
real-client step was the lane-enable tail (CILP preemption mode, timeslice
0x800, device scheduling enable); the front-end deposit, the scheduler CWD
entry (QMD id) and the engine-context load (VALID) all appear.
Register ground truth from the working distribution driver during live CUDA
plus the read-only sampler/analyzer are committed under tools/gr-regs/.
Remaining wall: the GR engine core does not execute the scheduled QMD (no
release written anywhere, no visible fault); ranked next moves in the
local handoff. Host clean, GPU green, no reboots.
|
| |
|
|
|
|
|
|
|
|
| |
readback deviation)
The m62 sysmem-chunk instance/ramfc made the server's writes host-readable
(job done: FECS showed V7's instance loaded with VALID=1, TARGET=3 sysmem),
but the captured real lane uses FBMEM slices and the GR engine core's
context restore may require VIDMEM instance/RAMFC. Keep every fix
(R1/E2/m68 prefetch/m69 win0) and restore the capture shape.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
m68 proved the program page's shadow copy contains real SASS (SASS_version
0x75) and the prefetch pair is fixed, yet the engine still never completes
and never writes its release anywhere - including a window-0 module sem.
Best remaining suspect: the window-1 scratch VA range (PT chain installed by
DMA_UPDATE_PDE_2 pdeIndex=1) has never been proven readable by the GR engine
core, unlike the window-0 GPFIFO/PB/QMDWIN small-PT line. m69 plants 8
module pages at VAs 0x108000..0x10f000 (slots 0x108..0x10f, same 128 B PT
fetch line, planted before first fetch like qmdwin), copies the resolved
program/param page content into them at launch and rewrites the QMD pointer
pairs there (param=32, program=48). Param: gsp_phys_rmcall_gr_shadow_win0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
the program target
m67 decoded the QMD (v02_03, SASS_version 0x75, block 64x1x1, grid 1x1x1) and
showed the host rewrote PROGRAM_ADDRESS (w48/49) to a Tier-2 shadow copy but
left PROGRAM_PREFETCH_ADDR (w8 = LO32(prefetch>>8), w31[8:0] =
HI32(prefetch>>8)) pointing at the guest VA 0x7a90ed44c800 - unmapped in the
execution VAS. Rewrite the prefetch pair to the same shadow copy when the
prefetch lies in the shadowed page (preserving prefetch size and
SASS_VERSION), and dump the program target's 16 instruction words in the
post-mortem to prove the SASS landed.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
contents
m66's forced sysmem fault buffers were refused by the server (ALLOC_MEMORY
0x1f, assert 0x5ab0214), so the engine-fault path stays FBMEM. Instead dump
everything that decides whether the engine can execute the scheduled QMD:
the resolved page table (guest VA -> HPA per QMD pointer), the QMD's
w16..w31 (QMD/SASS version, thread dimensions, register count) and the
first words of every Tier-2 shadow page the QMD's param/program pointers
were rewritten to (proves whether the SASS/params actually landed there).
|
| |
|
|
|
|
|
|
|
| |
m65 proved the scheduled QMD never writes its release anywhere (module sem
stays 0, guest page stays 5) - the engine dies or is cancelled before
completion. The TSG fault-method buffers (0xa06c010a) are the hardware's
record of why; force them to host sysmem (the captured aperture, param
gsp_phys_rmcall_faultbuf_sys=1) and dump both buffers' first 8 words in the
post-mortem so an engine fault entry (faulting address/engine) is visible.
|
| |
|
|
|
|
|
|
|
|
|
| |
chunk DMA __va)
The forward's poll already falls back to the SEM chunk kernel VA when
qmd_rel_hpa is 0; the m64 change set qmd_rel_hpa to the chunk DMA address,
whose __va() alias is not the chunk mapping, so the m64 poll read garbage.
Revert to qmd_rel_hpa=0 for the module-release fallback and read the SEM
chunk via sem_va in the post-mortem relw (plus the release word at rel_off
in the release-page dump).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fix + post-mortem dumps
m63 (E2 lane enable) got the forwarded QMD into SKED: CWD_TABLE_0 left the
cold value, CWD_TABLE_1 carried the QMD id (0x1070), SKED_ACTIVITY fired,
FECS_CURRENT_CTX showed V7's instance with VALID=1 - but the release write
never landed (relw=5) and no engine fault is logged. m64 discriminates
'engine completed but the shadow release was lost' from 'engine never
completed': param gsp_phys_rmcall_gr_rel_mod=1 forces QMD release0 to the
module window-0 semaphore with a distinct payload (0x600df00d) and polls the
SEM chunk. Also fixes the latent fallback (it set no poll source, so a
module-sem release could never complete) and dumps the full QMD w2..w15 +
the guest release page words in the post-mortem.
|
| |
|
|
|
|
|
|
|
|
|
| |
reg_sampler.py: high-rate read-only MMIO sampler for the physical GPU's
PGRAPH/FECS/SKED/PFIFO registers (FECS current-context valid + pointer,
SKED CWD table occupancy/activity, GRFIFO, FE QMD-virt, PMC_ENABLE) with
analyze_regs.py for value distributions and transition timelines. Recorded
ground truth and register semantics in tools/gr-regs/README.md. Used to
capture the live distro-driver CUDA reference (FECS VALID=1, CWD table
0x3f002000 while running) and to observe the ladder's forwarded launch
(CWD entry + SKED processing after the mode-35 lane enable).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
m62 diagnostic: V7 instance/ramfc on host-visible sysmem chunks so the
server's WFI-pointer/TARGET-bit writes are decodable (captured lane uses
unreadable FBMEM slices); decode calls post-promotes and post-construct.
m63 E2: send the real CUDA lane-enable tail after stage-4 schedules
(R2 #1059 NV2080_CTRL_CMD_GR_SET_CTXSW_PREEMPTION_MODE flags=CILP_SET
hChannelGroup=TSG cilp=CILP on the subdevice, #1061
NVA06C_CTRL_CMD_SET_TIMESLICE 0x800 on the TSG, #1063 NV01 device
0x00802009 enable=1). Ground truth: live real CUDA runs with
FECS_CURRENT_CTX VALID=1; our m61 snapshots show V7's context never valid,
and the SKED CWD table never leaves the cold value. Best-effort, mode-35
only.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
0xc5c0 construct now returns 0x0
The construct's internal vaListFindVa (0x52cf9c0, KGC shared +0x370) needs
the channel VAS present in the per-context ctx-buffer VA maps; F2's mapped
MAIN promote only wrote the per-TSG ENGINE_CTX_DESCRIPTOR vaList. Send the
capture's #879-shaped 10-entry VA-only promote {5,11,9,10,4,6,0,3,1,2}
(va only, pa/size/attr/init/non all zero) after F2 and before the
TURING_COMPUTE_A construct. m61: promote accepted status=0x0; TURING_COMPUTE_A
on V7 alloc returns status=0x0 (hObject=0xcaf000a8); REAL GR SEM now fires on
V7. Forwarded guest QMD still does not schedule (qmd_rel=5, relw=5).
|
| |
|
|
| |
V6 only; m59 A+share promote rejected)
|
| |
|
|
| |
(V6+sibling A attach via params init)
|
| |
|
|
| |
keep the F3 KGR queries
|
| |
|
|
| |
before the channel batch (capture #388-402; a2a/a32 may prepare the server per-context tables the 0x57 companion needs)
|
| |
|
|
| |
(V6-proven construct path while the share stays attached)
|
| |
|
|
| |
(registered list object failed the promote resolver lookup 0x57)
|
| |
|
|
| |
hUserdMemory[0] (server construct worker looked one up; 0x57 without it)
|
| |
|
|
| |
promote to reach the per-buffer commit (sets pEngCtx+0x135; gate for the 0xc5c0 construct)
|
| |
|
|
| |
pass)
|
| |
|
|
| |
stage-3 -22 fix; lane continues to stage 4/forward)
|
| |
|
|
| |
golden D, NV_ERR_NO_MEMORY 0x51)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
3-entry PA-only promote, V7 execution swap; compile-verified, untested)
Replays the decoded real 570.144 GSP client compute-context sequence on the
ladder, additive to modes 0-34:
- TSG fault-method buffers: one 0xa06c010a on the ladder TSG with two
0x2000 {base,size,alignment=1,addressSpace,cache} descriptors + zero
bar2Addr words + numValidEntries=2; server FBMEM first, registered host
sysmem fallback (capture order TSG -> faults -> share -> channel).
- FERMI_CONTEXT_SHARE_A (0x9067) reuse of the mode-34 helper (ladder TSG,
SHARED_MANAGEMENT VAS, flags SYNC -> VEID 0).
- V7 channel 0xcaf000a0: class 0xc46f, TSG parent, hVASpace=0, hContextShare,
captured flags 0x00201320 and internalFlags 0x1d, engineType 0, sysmem
hObjectError/errorNotifierMem, FBMEM instance/ramfc, sysmem USERD.
- One atomic 3-entry GPU_PROMOTE_CTX on V7 (MAIN/PM/PATCH, PA-only,
physAttr 4, bInitialize=1, bNonmapped=1) before any 0xc5c0 construct.
- TURING_COMPUTE_A (0xc5c0) on V7 after the accepted promote; execution
channel swapped to V7 for stage 4 BIND/SCHEDULE/token/GR-sem and
forwarding (V6 allocated but unscheduled; its promote suppressed).
- Mode-35-only golden9 entry refresh: FECS_EVENT attr=5 with the ctx-window
VA in the physAddr field, PRIV_ACCESS_MAP va=0/bNonmapped=1.
- gr_promote clamp/param doc extended to 0..35.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
51/53 status
- sim/libos_port.py: port-role model + two-task network (extintr handoff,
cross-task send/reply, TASK_STATE_QUERY trap state)
- sim/msgq_attach.py: image task discovery, bounded two-task schedule,
second-task-frontier divergence fields (first ring poll still blocked on
post-init resume + GSP_ARGUMENTS_CACHED handoff; no fabricated polls)
- harness/state_decode.py: aggregate 'group: key=value' state parsing
(20-gsp-phys-state-* dumps) + kv diff classification
- tests: 127 green across the five observability modules
- docs/51/53: status and changelog updates
|
| |
|
|
| |
sysmem MAIN hang class)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Add a read-only BAR0 register-liveness probe at
/sys/kernel/debug/vgsp/gsp-phys-reg-probe that prints "name addr value"
for the PMC chip-id/clock gate, PGRAPH status/fault, SKED/HWW and
FECS/GPCCS falcon state, the front-end QMD virtual-window controls and
the GSP context registers, followed by decoded field lines.
Read-only (readl only); additive to gsp_phys_boot_debugfs_init(); no
ladder logic changed. Used to distinguish a clock/power-gated engine
from a halted or running one during a physical cycle.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Mode 34 allocates FERMI_CONTEXT_SHARE_A (0x9067) on the ladder TSG before the
golden replay; the replay's golden channel A (0xcaf00054) attaches to the share
with hVASpace=0 and keeps the captured unwired shape, runs its 9-entry promote
and carries TURING_A (0xc597) plus a TURING_COMPUTE_A (0xc5c0, handle
0xcaf00090). V6 (0xcaf00034) attaches to the same share, constructs no class
object of its own, and its FWCLIENT promote is suppressed; stage 4 and the
forwarded launches run on V6 in A's shared VEID/subcontext. Default off; all
paths gated by gsp_phys_rmcall_gr_promote=34. New pr_info cycle landmarks for
the share alloc, A alloc/promote, A construct (rpc_result), V6 alloc and the
instance/RAMFC decode. No hardware run.
|
| |
|
|
|
|
|
| |
rm_capture (chunked decode), launch_audit, state_seed/state_decode,
valprobe, rehost, contracts+run_contracts, host_traffic, gmmu_walk,
msgq_attach, libos_port, log_decode, server_snapshot; doc index updates
in the tool overview, operations pages and the repo README.
|
| |
|
|
| |
proven, method space exhausted, structural levers next
|
| | |
|
| |
|
|
| |
(compile-verified, untested)
|
| | |
|
| | |
|
| |
|
|
| |
+0x800)
|
| |
|
|
| |
ring-chunk page (compile-verified, untested)
|
| | |
|
| |
|
|
| |
with D)
|
| |
|
|
| |
without it)
|
| |
|
|
| |
golden construct)
|
| | |
|
| |
|
|
| |
without BASE)
|
| | |
|
| |
|
|
| |
0x29c, w63 sentinel
|