| Commit message (Collapse) | Author | Age |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transition recorder, sysmem instance selector
m72: append the vendor turingLocalMemUpdate block (SET_SHADER_LOCAL_MEMORY_A/B
0x790/0x794, NON_THROTTLED_A/B 0x2e4/0x2e8, WINDOW_A/B 0x7b0/0x7b4) to the
forwarded GR launch prelude; without it the channel local-memory state is zero
and the m64-m69 sampler shows SKED_HWW_ESR bit12 = SKEDCHECK05
LOCAL_MEMORY_TOTAL_SIZE latching while the CWD entry is allocated then
deallocated. Base/per-TPC size are module params for A/B.
m72: change-detecting SKED transition recorder in the launch poll (CWD_T0/1/2,
HWW_INFO_1/2, ACTIVITY, HWW_ESR, GRFIFO, FECS current/new/ctxsw status) and
extended post-mortem readbacks (HWW_ESR_EN, INFO_1/2/3, CWD_T2, watermarks).
m72-r2 sampler result: the QMD enters SKED (CWD_T0=0x11300 STATUS=ACTIVE,
CWD_T1=0x1070) and HWW_ESR stays 0 with the lmem block, but the FECS context
load never completes: NEW_CTX=0x001f42b0 (V7 FBMEM instance, VALID=0) then
CURRENT=0; the task frees (CWD_T0=0x12000 DEP=PENDING_TASK_COMPLETE). m63
(sysmem instance) had VALID=1 on the same load. m73 adds the sysmem
instance/RAMFC selector (gsp_phys_rmcall_gr_inst_sys; default 0 = FBMEM
capture shape) for the A/B.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
forwarded QMD enqueues/schedules; engine-core execution is the remaining wall
(a) The TSG+share lane's compute-object construct returns success after a
10-entry VA-only booking promote books the server's per-VAS ctx-buffer maps
(a mapped MAIN promote alone leaves them unbooked because the construct
skips its own mapping once MAIN is initialized).
(b) The forwarded guest QMD now enqueues and is scheduled: the missing
real-client step was the lane-enable tail (CILP preemption mode, timeslice
0x800, device scheduling enable); the front-end deposit, the scheduler CWD
entry (QMD id) and the engine-context load (VALID) all appear.
Register ground truth from the working distribution driver during live CUDA
plus the read-only sampler/analyzer are committed under tools/gr-regs/.
Remaining wall: the GR engine core does not execute the scheduled QMD (no
release written anywhere, no visible fault); ranked next moves in the
local handoff. Host clean, GPU green, no reboots.
|
| |
|
|
|
|
|
|
|
|
| |
readback deviation)
The m62 sysmem-chunk instance/ramfc made the server's writes host-readable
(job done: FECS showed V7's instance loaded with VALID=1, TARGET=3 sysmem),
but the captured real lane uses FBMEM slices and the GR engine core's
context restore may require VIDMEM instance/RAMFC. Keep every fix
(R1/E2/m68 prefetch/m69 win0) and restore the capture shape.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
m68 proved the program page's shadow copy contains real SASS (SASS_version
0x75) and the prefetch pair is fixed, yet the engine still never completes
and never writes its release anywhere - including a window-0 module sem.
Best remaining suspect: the window-1 scratch VA range (PT chain installed by
DMA_UPDATE_PDE_2 pdeIndex=1) has never been proven readable by the GR engine
core, unlike the window-0 GPFIFO/PB/QMDWIN small-PT line. m69 plants 8
module pages at VAs 0x108000..0x10f000 (slots 0x108..0x10f, same 128 B PT
fetch line, planted before first fetch like qmdwin), copies the resolved
program/param page content into them at launch and rewrites the QMD pointer
pairs there (param=32, program=48). Param: gsp_phys_rmcall_gr_shadow_win0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
the program target
m67 decoded the QMD (v02_03, SASS_version 0x75, block 64x1x1, grid 1x1x1) and
showed the host rewrote PROGRAM_ADDRESS (w48/49) to a Tier-2 shadow copy but
left PROGRAM_PREFETCH_ADDR (w8 = LO32(prefetch>>8), w31[8:0] =
HI32(prefetch>>8)) pointing at the guest VA 0x7a90ed44c800 - unmapped in the
execution VAS. Rewrite the prefetch pair to the same shadow copy when the
prefetch lies in the shadowed page (preserving prefetch size and
SASS_VERSION), and dump the program target's 16 instruction words in the
post-mortem to prove the SASS landed.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
contents
m66's forced sysmem fault buffers were refused by the server (ALLOC_MEMORY
0x1f, assert 0x5ab0214), so the engine-fault path stays FBMEM. Instead dump
everything that decides whether the engine can execute the scheduled QMD:
the resolved page table (guest VA -> HPA per QMD pointer), the QMD's
w16..w31 (QMD/SASS version, thread dimensions, register count) and the
first words of every Tier-2 shadow page the QMD's param/program pointers
were rewritten to (proves whether the SASS/params actually landed there).
|
| |
|
|
|
|
|
|
|
| |
m65 proved the scheduled QMD never writes its release anywhere (module sem
stays 0, guest page stays 5) - the engine dies or is cancelled before
completion. The TSG fault-method buffers (0xa06c010a) are the hardware's
record of why; force them to host sysmem (the captured aperture, param
gsp_phys_rmcall_faultbuf_sys=1) and dump both buffers' first 8 words in the
post-mortem so an engine fault entry (faulting address/engine) is visible.
|
| |
|
|
|
|
|
|
|
|
|
| |
chunk DMA __va)
The forward's poll already falls back to the SEM chunk kernel VA when
qmd_rel_hpa is 0; the m64 change set qmd_rel_hpa to the chunk DMA address,
whose __va() alias is not the chunk mapping, so the m64 poll read garbage.
Revert to qmd_rel_hpa=0 for the module-release fallback and read the SEM
chunk via sem_va in the post-mortem relw (plus the release word at rel_off
in the release-page dump).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fix + post-mortem dumps
m63 (E2 lane enable) got the forwarded QMD into SKED: CWD_TABLE_0 left the
cold value, CWD_TABLE_1 carried the QMD id (0x1070), SKED_ACTIVITY fired,
FECS_CURRENT_CTX showed V7's instance with VALID=1 - but the release write
never landed (relw=5) and no engine fault is logged. m64 discriminates
'engine completed but the shadow release was lost' from 'engine never
completed': param gsp_phys_rmcall_gr_rel_mod=1 forces QMD release0 to the
module window-0 semaphore with a distinct payload (0x600df00d) and polls the
SEM chunk. Also fixes the latent fallback (it set no poll source, so a
module-sem release could never complete) and dumps the full QMD w2..w15 +
the guest release page words in the post-mortem.
|
| |
|
|
|
|
|
|
|
|
|
| |
reg_sampler.py: high-rate read-only MMIO sampler for the physical GPU's
PGRAPH/FECS/SKED/PFIFO registers (FECS current-context valid + pointer,
SKED CWD table occupancy/activity, GRFIFO, FE QMD-virt, PMC_ENABLE) with
analyze_regs.py for value distributions and transition timelines. Recorded
ground truth and register semantics in tools/gr-regs/README.md. Used to
capture the live distro-driver CUDA reference (FECS VALID=1, CWD table
0x3f002000 while running) and to observe the ladder's forwarded launch
(CWD entry + SKED processing after the mode-35 lane enable).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
m62 diagnostic: V7 instance/ramfc on host-visible sysmem chunks so the
server's WFI-pointer/TARGET-bit writes are decodable (captured lane uses
unreadable FBMEM slices); decode calls post-promotes and post-construct.
m63 E2: send the real CUDA lane-enable tail after stage-4 schedules
(R2 #1059 NV2080_CTRL_CMD_GR_SET_CTXSW_PREEMPTION_MODE flags=CILP_SET
hChannelGroup=TSG cilp=CILP on the subdevice, #1061
NVA06C_CTRL_CMD_SET_TIMESLICE 0x800 on the TSG, #1063 NV01 device
0x00802009 enable=1). Ground truth: live real CUDA runs with
FECS_CURRENT_CTX VALID=1; our m61 snapshots show V7's context never valid,
and the SKED CWD table never leaves the cold value. Best-effort, mode-35
only.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
0xc5c0 construct now returns 0x0
The construct's internal vaListFindVa (0x52cf9c0, KGC shared +0x370) needs
the channel VAS present in the per-context ctx-buffer VA maps; F2's mapped
MAIN promote only wrote the per-TSG ENGINE_CTX_DESCRIPTOR vaList. Send the
capture's #879-shaped 10-entry VA-only promote {5,11,9,10,4,6,0,3,1,2}
(va only, pa/size/attr/init/non all zero) after F2 and before the
TURING_COMPUTE_A construct. m61: promote accepted status=0x0; TURING_COMPUTE_A
on V7 alloc returns status=0x0 (hObject=0xcaf000a8); REAL GR SEM now fires on
V7. Forwarded guest QMD still does not schedule (qmd_rel=5, relw=5).
|
| |
|
|
| |
V6 only; m59 A+share promote rejected)
|
| |
|
|
| |
(V6+sibling A attach via params init)
|
| |
|
|
| |
keep the F3 KGR queries
|
| |
|
|
| |
before the channel batch (capture #388-402; a2a/a32 may prepare the server per-context tables the 0x57 companion needs)
|
| |
|
|
| |
(V6-proven construct path while the share stays attached)
|
| |
|
|
| |
(registered list object failed the promote resolver lookup 0x57)
|
| |
|
|
| |
hUserdMemory[0] (server construct worker looked one up; 0x57 without it)
|
| |
|
|
| |
promote to reach the per-buffer commit (sets pEngCtx+0x135; gate for the 0xc5c0 construct)
|
| |
|
|
| |
pass)
|
| |
|
|
| |
stage-3 -22 fix; lane continues to stage 4/forward)
|
| |
|
|
| |
golden D, NV_ERR_NO_MEMORY 0x51)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
3-entry PA-only promote, V7 execution swap; compile-verified, untested)
Replays the decoded real 570.144 GSP client compute-context sequence on the
ladder, additive to modes 0-34:
- TSG fault-method buffers: one 0xa06c010a on the ladder TSG with two
0x2000 {base,size,alignment=1,addressSpace,cache} descriptors + zero
bar2Addr words + numValidEntries=2; server FBMEM first, registered host
sysmem fallback (capture order TSG -> faults -> share -> channel).
- FERMI_CONTEXT_SHARE_A (0x9067) reuse of the mode-34 helper (ladder TSG,
SHARED_MANAGEMENT VAS, flags SYNC -> VEID 0).
- V7 channel 0xcaf000a0: class 0xc46f, TSG parent, hVASpace=0, hContextShare,
captured flags 0x00201320 and internalFlags 0x1d, engineType 0, sysmem
hObjectError/errorNotifierMem, FBMEM instance/ramfc, sysmem USERD.
- One atomic 3-entry GPU_PROMOTE_CTX on V7 (MAIN/PM/PATCH, PA-only,
physAttr 4, bInitialize=1, bNonmapped=1) before any 0xc5c0 construct.
- TURING_COMPUTE_A (0xc5c0) on V7 after the accepted promote; execution
channel swapped to V7 for stage 4 BIND/SCHEDULE/token/GR-sem and
forwarding (V6 allocated but unscheduled; its promote suppressed).
- Mode-35-only golden9 entry refresh: FECS_EVENT attr=5 with the ctx-window
VA in the physAddr field, PRIV_ACCESS_MAP va=0/bNonmapped=1.
- gr_promote clamp/param doc extended to 0..35.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
51/53 status
- sim/libos_port.py: port-role model + two-task network (extintr handoff,
cross-task send/reply, TASK_STATE_QUERY trap state)
- sim/msgq_attach.py: image task discovery, bounded two-task schedule,
second-task-frontier divergence fields (first ring poll still blocked on
post-init resume + GSP_ARGUMENTS_CACHED handoff; no fabricated polls)
- harness/state_decode.py: aggregate 'group: key=value' state parsing
(20-gsp-phys-state-* dumps) + kv diff classification
- tests: 127 green across the five observability modules
- docs/51/53: status and changelog updates
|
| |
|
|
| |
sysmem MAIN hang class)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Add a read-only BAR0 register-liveness probe at
/sys/kernel/debug/vgsp/gsp-phys-reg-probe that prints "name addr value"
for the PMC chip-id/clock gate, PGRAPH status/fault, SKED/HWW and
FECS/GPCCS falcon state, the front-end QMD virtual-window controls and
the GSP context registers, followed by decoded field lines.
Read-only (readl only); additive to gsp_phys_boot_debugfs_init(); no
ladder logic changed. Used to distinguish a clock/power-gated engine
from a halted or running one during a physical cycle.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Mode 34 allocates FERMI_CONTEXT_SHARE_A (0x9067) on the ladder TSG before the
golden replay; the replay's golden channel A (0xcaf00054) attaches to the share
with hVASpace=0 and keeps the captured unwired shape, runs its 9-entry promote
and carries TURING_A (0xc597) plus a TURING_COMPUTE_A (0xc5c0, handle
0xcaf00090). V6 (0xcaf00034) attaches to the same share, constructs no class
object of its own, and its FWCLIENT promote is suppressed; stage 4 and the
forwarded launches run on V6 in A's shared VEID/subcontext. Default off; all
paths gated by gsp_phys_rmcall_gr_promote=34. New pr_info cycle landmarks for
the share alloc, A alloc/promote, A construct (rpc_result), V6 alloc and the
instance/RAMFC decode. No hardware run.
|
| |
|
|
|
|
|
| |
rm_capture (chunked decode), launch_audit, state_seed/state_decode,
valprobe, rehost, contracts+run_contracts, host_traffic, gmmu_walk,
msgq_attach, libos_port, log_decode, server_snapshot; doc index updates
in the tool overview, operations pages and the repo README.
|
| |
|
|
| |
proven, method space exhausted, structural levers next
|
| | |
|
| |
|
|
| |
(compile-verified, untested)
|
| | |
|
| | |
|
| |
|
|
| |
+0x800)
|
| |
|
|
| |
ring-chunk page (compile-verified, untested)
|
| | |
|
| |
|
|
| |
with D)
|
| |
|
|
| |
without it)
|
| |
|
|
| |
golden construct)
|
| | |
|
| |
|
|
| |
without BASE)
|
| | |
|
| |
|
|
| |
0x29c, w63 sentinel
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
across the tree
New page docs/04-implementation/03-execution-frontier.md describes the chain
that must fire for a guest CUDA launch (guest push -> GPFIFO/front-end ->
inline launch-descriptor install -> engine read/dispatch -> completion),
where the leading edge is (the descriptor install path), how we know
(front-end vs engine-authored observables and their limits), what is
exonerated (context validity at dispatch, destination window mapping,
fetch alignment, shared-buffer rule, parenting), the ranked issues (verbatim
prelude targets, missing descriptor-virtualization control, sentinel
hygiene), live investigation lines and labelled theories.
Align every page that still said GR execution is blocked on engine-context
validity to the current frontier, and add the new page to the README reading
order (row 15, later rows renumbered). Docs only; no code.
|
| |
|
|
| |
launch diagnostics
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sentinel diagnostic
- The Tier-2 shadow VAs (0x200000 + page<<21 + line<<16) had no PTE chain
in the engine's table view on the shared-management VAS: pdb_build's
client PD0 spine for the guest scratch PT pool is discarded by the
server's root migration, so every param/program/release/alias leg the
launch rewrites to a shadow VA was unreachable by the engine. Install
each scratch pool window lazily through DMA_UPDATE_PDE_2 (same control
shape as the Tier-1 installer, pool chunk as the leaf PT) the first
time a shadow line in it is consumed; shared VAS only, idempotent via
rmcall_guest_pt_win_mask. m17 confirmed the install is accepted
(pdeIndex=1 VA 0x200000 status 0x0).
- IQ2M sentinel: write 0xdeadbeef to the inline-QMD destination word 0
after the prewrite and report iq2m=ran/not-run in the post-mortem so
'engine stalled before the inline window' is distinguishable from 'QMD
deposited but not executed' without a server capture.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
control offset
- New mode 29 (GOLDEN_EXEC_LAST): identical to mode 28 but skips the
TURING_COMPUTE_A alloc on the client-wired golden channel (every GR object
construct there blocks RM); stage 3 completes with the golden handle
swapped and BIND/GPFIFO_SCHEDULE/token are accepted. Param clamp 0..29.
- Shape bits: bit6 TSG parent (alloc accepted, same no-fetch), bit7 host
USERD control offset 0 (THE FETCH FIX: the server applies the sysmem
userd_mem base directly and ignores the captured USERD_INDEX_VALUE=3, so
GP_PUT/GP_GET live at +0x8c/+0x88; m14: GP_GET advanced 1), bit8
pre-promote compute object (invalid order: RM blocked -110 + NOCAT
0x52a8b04).
- Failure diagnostics: REAL GR SEM FAILED now dumps both candidate USERD
control offsets (ctl and alt).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
unreadable CE entries
- fwclient mask bit 15 (0x8000): book the PRIV_ACCESS_MAP VA (the FWCLIENT
entry form is init-only; the golden9 form carries the VA at VA_GOLDEN9_PRIV).
Accepted on V6 (status 0), completing the maximal promoted context.
- gsp_sema_service: svc_entry_want_reached() verifies the exact tracking slot
against entry_want before the three unreadable-CE consume shortcuts treat the
entry as delivered; when the slot lags, re-seed the expectation and let
svc_fake_release author or queue the exact slot (UVM registration stall pair
with the pool-VA verified-resolver queue).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
stall fix
- Mode 28 golden execution lane behind gsp_phys_rmcall_gr_promote=28 with a
shape bitmask (gsp_phys_rmcall_golden_exec_shape: ring / entries=8 / sysmem
USERD / USERD size / hObjectBuffer=VAS / skip replay TURING_A) and a USERD
control-offset helper. Finding: the RM-managed golden channel hangs any GR
object construct (-110, no assert) as soon as client submission wiring
(ring or sysmem USERD) is attached, so only the captured server-owned shape
constructs; the machinery is kept for future wiring ideas.
- fwclient mask bit 11 (0x800): send each non-MAIN FWCLIENT promote entry as
its own one-entry control and continue past refusals. V6 acceptance matrix:
PM, PATCH, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP accepted,
FECS_EVENT refused 0x1f.
- fwclient mask bit 12 (0x1000): golden9 VA-only global bookings (BUNDLE_CB,
PAGEPOOL, ATTRIBUTE_CB, RTV_CB_GLOBAL) accepted on V6; bits 13/14
(0x2000/0x4000): FECS PA-only (refused 0x1f) and FECS VA-only (accepted).
- gsp_phys_rmcall_srv_main_ignore: ignore a server-reported MAIN in the
srv-ctx commit and force the client INITIALIZE_CTX + VA-only fallback.
- gsp_sema_service: pool semaphores never fall back to the learned map; queue
the absolute release via svc_pending_add until svc_pool_pa_resolve
succeeds, and judge pool slots through the verified resolver in
svc_apply_pending and the release decode (UVM registration stall fix).
|