| Commit message (Collapse) | Author | Age |
| |
|
|
|
|
|
|
|
|
|
|
|
| |
the program target
m67 decoded the QMD (v02_03, SASS_version 0x75, block 64x1x1, grid 1x1x1) and
showed the host rewrote PROGRAM_ADDRESS (w48/49) to a Tier-2 shadow copy but
left PROGRAM_PREFETCH_ADDR (w8 = LO32(prefetch>>8), w31[8:0] =
HI32(prefetch>>8)) pointing at the guest VA 0x7a90ed44c800 - unmapped in the
execution VAS. Rewrite the prefetch pair to the same shadow copy when the
prefetch lies in the shadowed page (preserving prefetch size and
SASS_VERSION), and dump the program target's 16 instruction words in the
post-mortem to prove the SASS landed.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
contents
m66's forced sysmem fault buffers were refused by the server (ALLOC_MEMORY
0x1f, assert 0x5ab0214), so the engine-fault path stays FBMEM. Instead dump
everything that decides whether the engine can execute the scheduled QMD:
the resolved page table (guest VA -> HPA per QMD pointer), the QMD's
w16..w31 (QMD/SASS version, thread dimensions, register count) and the
first words of every Tier-2 shadow page the QMD's param/program pointers
were rewritten to (proves whether the SASS/params actually landed there).
|
| |
|
|
|
|
|
|
|
| |
m65 proved the scheduled QMD never writes its release anywhere (module sem
stays 0, guest page stays 5) - the engine dies or is cancelled before
completion. The TSG fault-method buffers (0xa06c010a) are the hardware's
record of why; force them to host sysmem (the captured aperture, param
gsp_phys_rmcall_faultbuf_sys=1) and dump both buffers' first 8 words in the
post-mortem so an engine fault entry (faulting address/engine) is visible.
|
| |
|
|
|
|
|
|
|
|
|
| |
chunk DMA __va)
The forward's poll already falls back to the SEM chunk kernel VA when
qmd_rel_hpa is 0; the m64 change set qmd_rel_hpa to the chunk DMA address,
whose __va() alias is not the chunk mapping, so the m64 poll read garbage.
Revert to qmd_rel_hpa=0 for the module-release fallback and read the SEM
chunk via sem_va in the post-mortem relw (plus the release word at rel_off
in the release-page dump).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fix + post-mortem dumps
m63 (E2 lane enable) got the forwarded QMD into SKED: CWD_TABLE_0 left the
cold value, CWD_TABLE_1 carried the QMD id (0x1070), SKED_ACTIVITY fired,
FECS_CURRENT_CTX showed V7's instance with VALID=1 - but the release write
never landed (relw=5) and no engine fault is logged. m64 discriminates
'engine completed but the shadow release was lost' from 'engine never
completed': param gsp_phys_rmcall_gr_rel_mod=1 forces QMD release0 to the
module window-0 semaphore with a distinct payload (0x600df00d) and polls the
SEM chunk. Also fixes the latent fallback (it set no poll source, so a
module-sem release could never complete) and dumps the full QMD w2..w15 +
the guest release page words in the post-mortem.
|
| |
|
|
|
|
|
|
|
|
|
| |
reg_sampler.py: high-rate read-only MMIO sampler for the physical GPU's
PGRAPH/FECS/SKED/PFIFO registers (FECS current-context valid + pointer,
SKED CWD table occupancy/activity, GRFIFO, FE QMD-virt, PMC_ENABLE) with
analyze_regs.py for value distributions and transition timelines. Recorded
ground truth and register semantics in tools/gr-regs/README.md. Used to
capture the live distro-driver CUDA reference (FECS VALID=1, CWD table
0x3f002000 while running) and to observe the ladder's forwarded launch
(CWD entry + SKED processing after the mode-35 lane enable).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
m62 diagnostic: V7 instance/ramfc on host-visible sysmem chunks so the
server's WFI-pointer/TARGET-bit writes are decodable (captured lane uses
unreadable FBMEM slices); decode calls post-promotes and post-construct.
m63 E2: send the real CUDA lane-enable tail after stage-4 schedules
(R2 #1059 NV2080_CTRL_CMD_GR_SET_CTXSW_PREEMPTION_MODE flags=CILP_SET
hChannelGroup=TSG cilp=CILP on the subdevice, #1061
NVA06C_CTRL_CMD_SET_TIMESLICE 0x800 on the TSG, #1063 NV01 device
0x00802009 enable=1). Ground truth: live real CUDA runs with
FECS_CURRENT_CTX VALID=1; our m61 snapshots show V7's context never valid,
and the SKED CWD table never leaves the cold value. Best-effort, mode-35
only.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
0xc5c0 construct now returns 0x0
The construct's internal vaListFindVa (0x52cf9c0, KGC shared +0x370) needs
the channel VAS present in the per-context ctx-buffer VA maps; F2's mapped
MAIN promote only wrote the per-TSG ENGINE_CTX_DESCRIPTOR vaList. Send the
capture's #879-shaped 10-entry VA-only promote {5,11,9,10,4,6,0,3,1,2}
(va only, pa/size/attr/init/non all zero) after F2 and before the
TURING_COMPUTE_A construct. m61: promote accepted status=0x0; TURING_COMPUTE_A
on V7 alloc returns status=0x0 (hObject=0xcaf000a8); REAL GR SEM now fires on
V7. Forwarded guest QMD still does not schedule (qmd_rel=5, relw=5).
|
| |
|
|
| |
V6 only; m59 A+share promote rejected)
|
| |
|
|
| |
(V6+sibling A attach via params init)
|
| |
|
|
| |
keep the F3 KGR queries
|
| |
|
|
| |
before the channel batch (capture #388-402; a2a/a32 may prepare the server per-context tables the 0x57 companion needs)
|
| |
|
|
| |
(V6-proven construct path while the share stays attached)
|
| |
|
|
| |
(registered list object failed the promote resolver lookup 0x57)
|
| |
|
|
| |
hUserdMemory[0] (server construct worker looked one up; 0x57 without it)
|
| |
|
|
| |
promote to reach the per-buffer commit (sets pEngCtx+0x135; gate for the 0xc5c0 construct)
|
| |
|
|
| |
pass)
|
| |
|
|
| |
stage-3 -22 fix; lane continues to stage 4/forward)
|
| |
|
|
| |
golden D, NV_ERR_NO_MEMORY 0x51)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
3-entry PA-only promote, V7 execution swap; compile-verified, untested)
Replays the decoded real 570.144 GSP client compute-context sequence on the
ladder, additive to modes 0-34:
- TSG fault-method buffers: one 0xa06c010a on the ladder TSG with two
0x2000 {base,size,alignment=1,addressSpace,cache} descriptors + zero
bar2Addr words + numValidEntries=2; server FBMEM first, registered host
sysmem fallback (capture order TSG -> faults -> share -> channel).
- FERMI_CONTEXT_SHARE_A (0x9067) reuse of the mode-34 helper (ladder TSG,
SHARED_MANAGEMENT VAS, flags SYNC -> VEID 0).
- V7 channel 0xcaf000a0: class 0xc46f, TSG parent, hVASpace=0, hContextShare,
captured flags 0x00201320 and internalFlags 0x1d, engineType 0, sysmem
hObjectError/errorNotifierMem, FBMEM instance/ramfc, sysmem USERD.
- One atomic 3-entry GPU_PROMOTE_CTX on V7 (MAIN/PM/PATCH, PA-only,
physAttr 4, bInitialize=1, bNonmapped=1) before any 0xc5c0 construct.
- TURING_COMPUTE_A (0xc5c0) on V7 after the accepted promote; execution
channel swapped to V7 for stage 4 BIND/SCHEDULE/token/GR-sem and
forwarding (V6 allocated but unscheduled; its promote suppressed).
- Mode-35-only golden9 entry refresh: FECS_EVENT attr=5 with the ctx-window
VA in the physAddr field, PRIV_ACCESS_MAP va=0/bNonmapped=1.
- gr_promote clamp/param doc extended to 0..35.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
51/53 status
- sim/libos_port.py: port-role model + two-task network (extintr handoff,
cross-task send/reply, TASK_STATE_QUERY trap state)
- sim/msgq_attach.py: image task discovery, bounded two-task schedule,
second-task-frontier divergence fields (first ring poll still blocked on
post-init resume + GSP_ARGUMENTS_CACHED handoff; no fabricated polls)
- harness/state_decode.py: aggregate 'group: key=value' state parsing
(20-gsp-phys-state-* dumps) + kv diff classification
- tests: 127 green across the five observability modules
- docs/51/53: status and changelog updates
|
| |
|
|
| |
sysmem MAIN hang class)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Add a read-only BAR0 register-liveness probe at
/sys/kernel/debug/vgsp/gsp-phys-reg-probe that prints "name addr value"
for the PMC chip-id/clock gate, PGRAPH status/fault, SKED/HWW and
FECS/GPCCS falcon state, the front-end QMD virtual-window controls and
the GSP context registers, followed by decoded field lines.
Read-only (readl only); additive to gsp_phys_boot_debugfs_init(); no
ladder logic changed. Used to distinguish a clock/power-gated engine
from a halted or running one during a physical cycle.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Mode 34 allocates FERMI_CONTEXT_SHARE_A (0x9067) on the ladder TSG before the
golden replay; the replay's golden channel A (0xcaf00054) attaches to the share
with hVASpace=0 and keeps the captured unwired shape, runs its 9-entry promote
and carries TURING_A (0xc597) plus a TURING_COMPUTE_A (0xc5c0, handle
0xcaf00090). V6 (0xcaf00034) attaches to the same share, constructs no class
object of its own, and its FWCLIENT promote is suppressed; stage 4 and the
forwarded launches run on V6 in A's shared VEID/subcontext. Default off; all
paths gated by gsp_phys_rmcall_gr_promote=34. New pr_info cycle landmarks for
the share alloc, A alloc/promote, A construct (rpc_result), V6 alloc and the
instance/RAMFC decode. No hardware run.
|
| |
|
|
|
|
|
| |
rm_capture (chunked decode), launch_audit, state_seed/state_decode,
valprobe, rehost, contracts+run_contracts, host_traffic, gmmu_walk,
msgq_attach, libos_port, log_decode, server_snapshot; doc index updates
in the tool overview, operations pages and the repo README.
|
| |
|
|
| |
proven, method space exhausted, structural levers next
|
| | |
|
| |
|
|
| |
(compile-verified, untested)
|
| | |
|
| | |
|
| |
|
|
| |
+0x800)
|
| |
|
|
| |
ring-chunk page (compile-verified, untested)
|
| | |
|
| |
|
|
| |
with D)
|
| |
|
|
| |
without it)
|
| |
|
|
| |
golden construct)
|
| | |
|
| |
|
|
| |
without BASE)
|
| | |
|
| |
|
|
| |
0x29c, w63 sentinel
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
across the tree
New page docs/04-implementation/03-execution-frontier.md describes the chain
that must fire for a guest CUDA launch (guest push -> GPFIFO/front-end ->
inline launch-descriptor install -> engine read/dispatch -> completion),
where the leading edge is (the descriptor install path), how we know
(front-end vs engine-authored observables and their limits), what is
exonerated (context validity at dispatch, destination window mapping,
fetch alignment, shared-buffer rule, parenting), the ranked issues (verbatim
prelude targets, missing descriptor-virtualization control, sentinel
hygiene), live investigation lines and labelled theories.
Align every page that still said GR execution is blocked on engine-context
validity to the current frontier, and add the new page to the README reading
order (row 15, later rows renumbered). Docs only; no code.
|
| |
|
|
| |
launch diagnostics
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sentinel diagnostic
- The Tier-2 shadow VAs (0x200000 + page<<21 + line<<16) had no PTE chain
in the engine's table view on the shared-management VAS: pdb_build's
client PD0 spine for the guest scratch PT pool is discarded by the
server's root migration, so every param/program/release/alias leg the
launch rewrites to a shadow VA was unreachable by the engine. Install
each scratch pool window lazily through DMA_UPDATE_PDE_2 (same control
shape as the Tier-1 installer, pool chunk as the leaf PT) the first
time a shadow line in it is consumed; shared VAS only, idempotent via
rmcall_guest_pt_win_mask. m17 confirmed the install is accepted
(pdeIndex=1 VA 0x200000 status 0x0).
- IQ2M sentinel: write 0xdeadbeef to the inline-QMD destination word 0
after the prewrite and report iq2m=ran/not-run in the post-mortem so
'engine stalled before the inline window' is distinguishable from 'QMD
deposited but not executed' without a server capture.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
control offset
- New mode 29 (GOLDEN_EXEC_LAST): identical to mode 28 but skips the
TURING_COMPUTE_A alloc on the client-wired golden channel (every GR object
construct there blocks RM); stage 3 completes with the golden handle
swapped and BIND/GPFIFO_SCHEDULE/token are accepted. Param clamp 0..29.
- Shape bits: bit6 TSG parent (alloc accepted, same no-fetch), bit7 host
USERD control offset 0 (THE FETCH FIX: the server applies the sysmem
userd_mem base directly and ignores the captured USERD_INDEX_VALUE=3, so
GP_PUT/GP_GET live at +0x8c/+0x88; m14: GP_GET advanced 1), bit8
pre-promote compute object (invalid order: RM blocked -110 + NOCAT
0x52a8b04).
- Failure diagnostics: REAL GR SEM FAILED now dumps both candidate USERD
control offsets (ctl and alt).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
unreadable CE entries
- fwclient mask bit 15 (0x8000): book the PRIV_ACCESS_MAP VA (the FWCLIENT
entry form is init-only; the golden9 form carries the VA at VA_GOLDEN9_PRIV).
Accepted on V6 (status 0), completing the maximal promoted context.
- gsp_sema_service: svc_entry_want_reached() verifies the exact tracking slot
against entry_want before the three unreadable-CE consume shortcuts treat the
entry as delivered; when the slot lags, re-seed the expectation and let
svc_fake_release author or queue the exact slot (UVM registration stall pair
with the pool-VA verified-resolver queue).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
stall fix
- Mode 28 golden execution lane behind gsp_phys_rmcall_gr_promote=28 with a
shape bitmask (gsp_phys_rmcall_golden_exec_shape: ring / entries=8 / sysmem
USERD / USERD size / hObjectBuffer=VAS / skip replay TURING_A) and a USERD
control-offset helper. Finding: the RM-managed golden channel hangs any GR
object construct (-110, no assert) as soon as client submission wiring
(ring or sysmem USERD) is attached, so only the captured server-owned shape
constructs; the machinery is kept for future wiring ideas.
- fwclient mask bit 11 (0x800): send each non-MAIN FWCLIENT promote entry as
its own one-entry control and continue past refusals. V6 acceptance matrix:
PM, PATCH, PRIV_ACCESS_MAP, UNRESTRICTED_PRIV_ACCESS_MAP accepted,
FECS_EVENT refused 0x1f.
- fwclient mask bit 12 (0x1000): golden9 VA-only global bookings (BUNDLE_CB,
PAGEPOOL, ATTRIBUTE_CB, RTV_CB_GLOBAL) accepted on V6; bits 13/14
(0x2000/0x4000): FECS PA-only (refused 0x1f) and FECS VA-only (accepted).
- gsp_phys_rmcall_srv_main_ignore: ignore a server-reported MAIN in the
srv-ctx commit and force the client INITIALIZE_CTX + VA-only fallback.
- gsp_sema_service: pool semaphores never fall back to the learned map; queue
the absolute release via svc_pending_add until svc_pool_pa_resolve
succeeds, and judge pool slots through the verified resolver in
svc_apply_pending and the release decode (UVM registration stall fix).
|
| |
|
|
| |
acceptance, golden lane walled, engine dispatch still absent
|
| |
|
|
| |
buffers
|
| | |
|
| | |
|