| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
file existed); wedge_ab same-program vs second-launch discriminator
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Looking Glass status
- tools/cuda-ab/frame_producer.cu: renders an animated Julia frame on the
guest GPU and publishes it into the ivshmem BAR (the kvmfr 128 MiB window)
with a small header + RGBA payload.
- tools/lg/frame_reader.py: host-side reader of /dev/shm/kvmfr-<uuid> that
saves each published frame as a PPM (the transport half of the frame path).
- tools/guest-diag/egl_frame.c: EGL probe (device/surfaceless/default
platforms) - live result: eglQueryDevicesEXT sees the NVIDIA device and
eglInitialize succeeds, but eglMakeCurrent fails EGL_BAD_ALLOC (0x3003),
the same VASpace-allocation wall as Xorg's 'Failed to allocate VASpace
object'.
- docs/05-operations/06: the state, the blockers and the exact remaining
pieces (guest GL VASpace allocation; LGMP session producer).
|
| |
|
|
|
|
|
|
|
| |
inter-launch activity
Modes A/B/C/D add none / an extra H2D / an extra D2H / a 500 ms sleep
between the verify and the next launch of the wedge_recover shape. The
inputs mutate every iteration, so a no-execute launch is reported (bad>0),
never masked by deterministic content.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
redirected
With gsp_phys_rmcall_gr_rel_mod=1 (release0 -> module semaphore, the
ordering fix for the engine-release-before-copy-out race) the engine never
writes the guest's own QMD release word, so the host must author it on
EVERY backend outcome. The hold's success/fail edges do; a launch with no
deferred release records (no hold engaged) fell through and its client hung
on a word nothing would ever write - the r414 saxpy f0 cur=5 stall.
The backend now reports qmd_rel_redirected in the copy result whenever it
points release0 at the module semaphore (rel_mod, or no targetable guest
release page). svc_fwd_gr_work folds that into the existing
gr_rel_unobserved latch, which makes the bounded outcome check author the
word on done (success or failure) - the same forward-only write the hold
edges perform. rel_mod=0 paths are unchanged (the engine still authors the
word and the latch is only set on an unresolved release page).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
previous launch also no-wrote (r416: 58-launch sticky wedge clears after ~50 s quiet); wedge counters in state; wedge_recover v2 unmask
The r416 boot reproduced the sticky no-execute wedge inside 60 s: 58
consecutive launches accepted zero writes while the lane stayed busy
(wedge_recover v1's deterministic verify masked every one of them), and
the first launches were green again only after the lane sat idle ~52 s.
Both r409 (~37 s idle heal) and r416 therefore show the wedge clears on
idleness and persists under back-to-back retries (r154/r409), so the
retry loop now holds a quiet window (gsp_phys_gr_wedge_quiet_ms, default
20000, doubling on the second quiet retry, 0 disables) before the normal
fresh-bank + resched retry when the previous launch also no-wrote. The
whole-launch stall budget is relaxed only on this bounded recovery path.
NOWRITE lines now carry the consecutive streak and quiet/heal counters,
and the state file shows nw_streak/wedge_quiet/wedge_heals.
wedge_recover v2 mutates the inputs every iteration so a stale (never
executed) launch can no longer match the expected values, and classifies
the readback STALE-PREV(NOWRITE) vs OTHER/PARTIAL.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
memset_probe: linear 8 MB cudaMemset + D2H with first-bad offset and a
4 KB page map (correct / other / malloc-prefill) - separates 'fill
under-delivered' from 'readback delivered wrong pages'; standalone green,
bisects the memcpy_roundtrip memset phase.
wedge_recover: launch+verify loop that on failure sleeps N seconds and
retries the same iteration (same context), reporting per-iteration
attempts and launch times - tests whether an idle gap heals the per-boot
no-execute wedge (r409 log shows a ~37 s client-side gap healed a wedged
run).
Also ships the missing inplace_scale run binary (its source landed in
7685d1b3).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two fixes for the streams2 / multi-stream ordering class, both verified on
fresh boot r401 with the acquire gate enabled:
1. sema_wait_hold_ms (new module param, default 4000 ms) replaces the
hardcoded 100 ms acquire-wait bound in svc_run_ops. The old bound is
shorter than one emulated GR launch completion (0.5-1.6 s observed), so
a D2H readback's ACQ_CIRC_GEQ on the producer channel's progress
counter (op=3 pay=4 on 0x20400ffd0) gave up and ran BEFORE the scale
launch wrote d_c2; the client then verified pre-scale data
(bad=32767). r401 log: the same entry now holds ~1.2 s across
cur=1->4 and runs after GUEST GR LAUNCH OK (copy-back optrace
92315.594824 with dst16_after = the post-scale bytes). Fail-open is
preserved: a genuinely unsatisfiable acquire still expires, just late
enough that the dependency's work has landed (late-but-correct over
early-and-wrong).
2. svc_gr_seen: a session-scoped open-addressed (push_va, hash) table
behind svc_gr_cand_push_seen(). The 4-slot candidate ring + 8-VA
history evicted a real capture within ~12 later captures, so a true
replay of an already-forwarded launch was re-captured and re-executed
(r399: the same nine launch pushes re-submitted every ~10-11 s for
minutes). Cleared in svc_session_flush because a new client's
identical-content pushes restart their release counters - a
boot-scoped table would silently drop them (LAUNCH-SKIP).
Also adds tools/cuda-ab/streams2d.cu: streams2 with per-array/per-4KB-block
diff reporting plus the device pointer dump, which localised the remaining
failure to the D2H staging write path (our staging-page resolution vs the
guest's view).
|
| |
|
|
| |
expect LAUNCH_COUNT=1)
|
| |
|
|
|
|
|
|
| |
victory_saxpy, victory_reduce (shared+atomics), victory_matmul (tiled
shared), victory_strided, victory_memset2d (pitched), victory_atomics,
victory_streams_events (3 streams + events), victory_math (mixed
int/float/double). Each verifies via readback and prints STATUS OK|FAIL.
soak.sh now runs the full 14-client set (seq or sustained).
|
| |
|
|
| |
smiley push wiring, near-native perf harness (offline readiness; guest validation pending)
|
| |
|
|
|
|
|
|
| |
mail)
The run/ binary was the Oct 4 build (4 MB buffers, __device__ kMail);
ab.sh push_bin only builds when the binary is absent, so the guest shipped
the stale build. This is the rebuilt binary matching today's .cu.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.
Two data-path correctness fixes make the transfers sound:
- Once-only op execution per GPFIFO entry. The pump replays recently
consumed ring entries when translation batches are injected so missed
releases can land; re-executing a CE copy is not idempotent because
its source is a client staging buffer that may have been refilled.
Slots now record their raw entry words and a replayed entry matching
that identity suppresses operations while still processing releases.
- Per-page translation of virtual operands. svc_copy resolved only the
first page of a VA range and walked the PA linearly; client staging
ranges are frequently split (observed 195 + 61 pages), so the tail of
alternating chunks was read from unrelated memory. Multi-page copies
now re-resolve each 4 KB page through the trusted resolver.
Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).
|
| |
|
|
|
|
|
|
| |
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean
- tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests
- tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode
- debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding
- STATE-CURRENT/docs updates
|
|
|
The progress service now decodes push buffers as method streams and performs
the work the driver pushes, not just the tracking semaphore:
- C5B5 method decoder: SET_REMAP_CONST_A/B + COMPONENTS + OFFSET_OUT +
LINE_LENGTH_IN + LAUNCH_DMA = memset (into vidmem or guest RAM);
OFFSET_IN + OFFSET_OUT + LAUNCH_DMA = copy; SET_SEMAPHORE_A/B/PAYLOAD +
LAUNCH_DMA = release. Contiguous method runs (count 2/4) are applied in
full - OFFSET_IN/OUT arrive as one run.
- Pushes span page boundaries, so pages are decoded as method streams
without a start marker. Collected operations execute only when the
push's release is reached and still pending (payload ahead of the
semaphore), so recycled buffers replaying old pushes are skipped and
their stale memsets cannot overwrite newer state.
- VA->PA map is first-wins: stale copies of freed allocations and
unrelated adjacent u64 pairs produced bogus entries that hijacked a
good mapping and sent semaphore writes to the wrong page.
- dump_guest module parameter (hex dump of guest PA, kthread via
completion; kthread_stop before the thread runs loses the work).
With this the guest no longer stalls in uvm_tracker_wait and no longer
oopses in the UVM path; the failure moved to a userspace segfault inside
libcuda (NULL-0x38) after the driver's page allocations.
Also add tools/cuda-ab: run one deterministic CUDA workload on the host
(bare metal oracle) and in the guest, then compare the stable result
lines. The binary links the CUDA runtime statically so the guest only
needs the driver's libcuda.so.
|
