summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAge
* cuda-ab: victory-series coverage battery (8 new clients) + soak runner listArthur Hanson Rasmusson8 hours
| | | | | | | | victory_saxpy, victory_reduce (shared+atomics), victory_matmul (tiled shared), victory_strided, victory_memset2d (pitched), victory_atomics, victory_streams_events (3 streams + events), victory_math (mixed int/float/double). Each verifies via readback and prints STATUS OK|FAIL. soak.sh now runs the full 14-client set (seq or sustained).
* vgsp-gr: dump first words of changed bounce pages (m142 diagnostic)Arthur Hanson Rasmusson8 hours
| | | | | Discriminates engine-side writes of float data (kernel ran with c on this buffer) from non-kernel writes (pointers/QMD-shaped words).
* vgsp-gr: log the changed bounce page indexes at copy-out (m142 diagnostic)Arthur Hanson Rasmusson9 hours
| | | | | | | 8 changed pages x 4 passes for smiley vs 33 for victory's vadd - the indexes name the relocated range (bi >= 0x40 = the c-pointer output window 0x150000+) and discriminate engine-side partial dispatch from a guest write-back/read attribution failure.
* vgsp-gr: one bounded re-arm of a stalled forwarded launch (context ↵Arthur Hanson Rasmusson9 hours
| | | | | | | | | | | | | | | | save/unload lottery) r90 evidence: the 3rd session's forwarded launch entered SKED (CWD VALID) and was then saved/unloaded by the FECS before dispatch; qmd_rel stayed at 5/6, GP_GET froze, and every later launch on the boot was never fetched either. The service owns no repair path, so the channel stayed dead until reboot. On the first poll timeout, re-issue the two GPFIFO_SCHEDULE controls the stage-4 sequence used (channel-level 0xa06f0103 + TSG-level 0xa06c0101, both proven accepted) and re-ring the work-submit doorbell, then poll one more full window before the postmortem. No BIND, no PROMOTE (m76-m81 proved the explicit PREEMPT/tail variants stop the QMD enqueuing).
* vgsp: GR data path - pre-publish bounce fill, engine-change proof, ↵Arthur Hanson Rasmusson9 hours
| | | | | | | | | | | | | | | | | | | | | | ipin-first data resolution Two fixes and one diagnostic for the all-zero-results class (r89: smiley's 4 vadd launches reported GUEST GR LAUNCH OK yet every output word read 0): - gsp_phys_booter: copy the guest data pages into the bounce window BEFORE gsp_phys_rmcall_publish_locked() rings the doorbell. The fill used to run after the kick; the engine front end starts within ~7 ms, so the kernel could read the window before it was filled. - gsp_phys_booter: per-page checksums at copy-in, compared at copy-out; the copy-out log now reports 'N changed by engine'. An all-unchanged window after a reported completion is the front-end-ran/grid-never- dispatched signature (completion semaphores are front-end authored). The first 3 data (va,hpa,reloc) triples are logged per launch. - gsp_sema_service: svc_gr_data_fill resolves data pages through svc_ipin_lookup() first (guest-truth pins), falling back to svc_resolve_va, mirroring the pool resolver order; a wrong page here silently runs the kernel on zeros. First 16 resolved (va,pa,ap) triples are logged for post-mortem comparison against the guest's own mapping.
* guest-diag: semfix_watch read64 uncached by default (stale-cache pool-pin ↵Arthur Hanson Rasmusson10 hours
| | | | | | | | | | | failure) The watcher's resolve_sem_pool read sem->page->mem->vas through a process-lifetime cache: a transient zero captured while UVM was still initialising the structures was served forever, the pool pin never landed, and the second-session client hung in cudaMallocManaged (r88 evidence: 'pool unresolved (retry)' from the watcher while the same walk succeeded from a fresh QMP client). Cache is now opt-in.
* guest-diag: scanner EOF guard + exclude non-client nvidia fd holdersArthur Hanson Rasmusson10 hours
| | | | | | | | | Two failure modes found when the scanner detects clients by device fd: a transient boot nvidia-smi was picked and its exited pagemap read back EOF, and scan_window advanced by got=0 forever (busy spin, no stream); and non-client nvidia fd holders must never be committed to. Break on got<=0 in scan_window (scan_range already did), skip nvidia-smi/ persistenced/Xorg, and prefer /dev/nvidia-uvm holders.
* guest-diag: scanner detects arbitrary CUDA clients by nvidia fdsArthur Hanson Rasmusson11 hours
| | | | | | | | | | The guest pin scanner only tracked processes whose comm matched CLIENT_PREFIXES (cdt,va,victory). An arbitrary CUDA binary (e.g. /tmp/smiley) was invisible: no pages streamed, no VA->PA pins injected, the service could not read its pushes and the client hung in cudaMalloc/uvm_tracker_wait. Detect compute clients by the CUDA device they hold open (/dev/nvidia-uvm preferred, then any /dev/nvidia*), with the prefix match kept as the first-choice path.
* scripts: add ssh_guest.sh - reliable SSH into the running vGPU guestArthur Hanson Rasmusson11 hours
| | | | | | | | Password auth with key auth disabled outright: the host's id_rsa is passphrase-protected and a stale copy of its public key in the guest's authorized_keys made plain ssh hang on the passphrase prompt (the reported 'sshpass hangs'). Wrapper defaults to 127.0.0.1:2222 ubuntu/ubuntu, overridable via VGPU_GUEST_*.
* tools/guest-diag: single user netdev for QEMU (fix SSH pre-auth stall)Arthur Hanson Rasmusson11 hours
| | | | | | | | | | | | | | | | The launch line carried both an explicit rtl8139 on netdev net0 and a stray '-net user,hostfwd=tcp::2222-:22 -net nic' pair, creating a SECOND slirp network with a second NIC and a second 2222 forward. The SSH connection landed on one slirp instance while the guest's replies left through the other interface, stalling the handshake before auth. The explicit rtl8139 now owns the only user netdev with the 2222 forward. The generated launch_qemu_rootport.run.sh is patched in place by the harness from the same line. Verified: sshpass -p ubuntu ssh -p 2222 ubuntu@127.0.0.1 -> nvidia-smi green. Guest-side companion fix (persisted in the qcow2 image): the cloud image's /etc/ssh/sshd_config.d/60-cloudimg-settings.conf set PasswordAuthentication no; it is now yes.
* Revert "vgsp-sema: scope channel deletion to the owning RM client (ISSUE-0002)"Arthur Hanson Rasmusson14 hours
| | | | This reverts commit 5fa773c2cd4e166d2c31a40aff9fb63e7534fec3.
* Revert "vgsp-sema: scope CE/GR rank assignment to the owning client ↵Arthur Hanson Rasmusson14 hours
| | | | | | (ISSUE-0002)" This reverts commit 35ca086b20f04f6be5bd935fd53d08e43f67e869.
* Revert "vgsp-sema: session boundary is a client change, not the last channel ↵Arthur Hanson Rasmusson14 hours
| | | | | | closing (ISSUE-0002)" This reverts commit 12732c5894c893344422da4fd8d075c4c43c8e8a.
* Revert "vgsp-sema: keep the scrubber channel inert; publish its finish ↵Arthur Hanson Rasmusson14 hours
| | | | | | payload from GP_PUT (ISSUE-0002)" This reverts commit 2d0a65b8f10f8bfac935d880dd9704e7b14f5b29.
* vgsp-sema: keep the scrubber channel inert; publish its finish payload from ↵Arthur Hanson Rasmusson14 hours
| | | | | | | | | | | | | | | | | | | | | GP_PUT (ISSUE-0002) The CeUtils scrubber channel (entries=4096, GPU-global) stays registered across client sessions now that deletion is client-scoped, but its channel buffer memory is reused by later clients' push buffers: entry-pumping its stale ring invented work and perturbed the session (r79/r80: the first client wedged on an rwsem). The pump now skips channels with the scrubber layout (entries >= 4096) and instead publishes the word the pma scrub actually waits on - the channel buffer finish payload at gpfifo_va + 8*entries + 4 (live-verified: 0x120064000 + 0x8000 + 4 = 0x12006c004, the address from the hung client's register dump) - forward-only from the channel's own GP_PUT (one payload per submitted push, the same counter the guest's ceutilsUpdateProgress polls). Skipping the scrub's actual memset work is acceptable for the emulated vidmem: the guest's scrub clears pages of its own shadow, and the emulation's job here is the completion contract. Revisit if per-tenant vidmem scrubbing is required.
* vgsp-sema: session boundary is a client change, not the last channel closing ↵Arthur Hanson Rasmusson14 hours
| | | | | | | | | | | | | | | | | | (ISSUE-0002) r79 regression from keeping the CeUtils scrubber channel registered (client-scoped deletion): the session flush trigger required EVERY channel to close, and the scrubber never closes, so the per-session discovery reset never ran and the next client inherited the previous session's map/pins (the r9 class; the first client of r79 wedged the guest on an rwsem with no output). The session is now keyed by owning client: channel_add records svc_sess_client and requests the flush when a channel of a DIFFERENT client registers while the previous session is live (then waits for it, exactly like a pending flush); channel_del's flush trigger counts only channels of the current session client, so persistent kernel/GPU-global channels (the scrubber) no longer keep a dead session open.
* vgsp-sema: scope CE/GR rank assignment to the owning client (ISSUE-0002)Arthur Hanson Rasmusson15 hours
| | | | | | | | | | | | | | | r78 validation of the client-scoped deletion: the boot session's CeUtils scrubber channel (client != the UVM client, entries=4096) stayed registered as intended, but it consumed CE rank 0 and every UVM channel's rank shifted by one. Ring autopin and push-buffer pairing derive the ring and pool-semaphore offsets from the rank (0x121010000 + 0x30000*rank, 0x121000000 + 4*rank), so the first client's pairing went to the wrong offsets and the guest wedged on an rwsem (sysrq: rwsem_spin_on_owner). Rank is now counted only among live channels of the same owning client; the UVM client's channels keep ranks 0..5 and the kernel/GPU-global scrubber gets its own (unused for the pool layout) rank. Missing clients keep the old counting.
* vgsp-sema: scope channel deletion to the owning RM client (ISSUE-0002)Arthur Hanson Rasmusson15 hours
| | | | | | | | | | | | | | | | | | | | | Handles are per-client, but the FREE path unregistered any channel whose numeric handle matched the freed object, with no client check. r77 root cause of the third-client pma scrub hang: the boot session's CeUtils scrubber channel (h=0x2, gpfifo=0x120064000, entries=4096) was unregistered when an unrelated class-0x2080 object with handle 0x2 in client 0xc1d00011 was freed. The guest kept using the live channel for the scrub; the service no longer pumped it, its pushes were never decoded and the channel-buffer finish payload (pbGpuVA+0x6c004) was never written, so scrubCheckAndWaitForSize polled forever (sysrq stack: ceutilsUpdateProgress -> channelReadChannelMemdesc). Channel registration now records h_client (the RM alloc header's client) and deletion matches (h_object, h_client); a missing client on either side keeps the handle-only behaviour. Re-registration keeps the counter/release slots only for the same (handle, client) pair. Live proof from the hung VM: sema_probe translates the finish-payload VA 0x12006c004 to pa=0x10f962004 aperture=ram - the word the guest polls is reachable; the channel was simply invisible to the pump.
* vgsp-sema: bound emulated-vidmem shadow accesses by the shadow size, not the ↵Arthur Hanson Rasmusson16 hours
| | | | | | | | | | | | | | | | | | | | | | | | | BAR1 aperture (ISSUE-0002) The pma scrub in a second/third CUDA session memsets a legitimate vidmem offset at 262M (inside the 512M turing shadow) and the copy path rejected it with -EINVAL because every FB bounds check compared against vgpu->bars[1].size, the physical BAR1 aperture exposed to the guest (256M on TU104). The op was held and retried forever (opdef), the GR scrub waited in scrubCheckAndWaitForSize and the client froze. All bounds checks that guard a dereference of the emulated vidmem shadow at vgpu->bars[1].vaddr now use nv_vgpu_fb_emul_size() via svc_fb_shadow_size(): the copy/fill paths, svc_read_ap/svc_write_ap, svc_ap_ok, the page-directory discovery scans, svc_lookup's vidmem VA translation, the GRCE inline write, svc_fwd_operand_hpa and the debug scans. The 2 MB leaf path of svc_gmmu_walk now validates through svc_ap_ok like every other leaf path so an out-of-shadow translation falls through to the shadow fallback instead of reaching the copy path. The RM page-directory debug peek is bounded the same way. Evidence: r75b live log: sema-bigcopy dst_va=0x76aa08e02200 -> pa=0x10602200 ap=0 prov=3; sema-bigop rc=-22 repeated; guest sysrq stack scrubCheckAndWaitForSize -> pmaAllocatePages.
* vgsp-sema: rewind last_get on a fresh channel's second put drop (ISSUE-0002)Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | r73 c3 rank2: discovery read the previous session's residual put (67); the pump consumed the live entries 0..1, the r48 resync handled the first drop, but the guest's counter then settled at put=1 while last_get=2 and userd_wait_fresh was already consumed. The loop condition last_get != put then marched ~1023 slots the producer had not written, holding each stale entry for the full CE grace (one fake release per 8 s) - the client looked wedged and the exec timed out. When a drop leaves the pump ahead of the producer on a channel with fewer than half a ring of submissions, rewind last_get to put instead of marching. A genuine ring wrap needs a full ring of submissions, so this cannot skip a live entry.
* vgsp-sema: limit the failed-op hold to large ops (>= 256 KB) (ISSUE-0002)Arthur Hanson Rasmusson2 days
| | | | | | | | | r72 showed the broad rule holding entries on small failed ops too: two opdef holds on h=0x5c000059 (entries 82/84) were never resolved and withheld their releases for the whole session, while c2 hit the pre-existing GR launch forward timeout. The scrub class this rule exists for is always large (r58/ r71: 1 MiB / 297 KiB / 608 KiB), so only failures of ops >= 256 KB mark the pass op-pending; small failing ops keep the old consume-and-apply behaviour.
* vgsp-sema: hold an entry on ANY failed op, not only an unresolvable operand ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | | (ISSUE-0002) r71 c3 reached the pma scrub and froze in scrubCheckAndWaitForSize: three vidmem memsets (1 MiB / 297 KiB / 608 KiB) failed with -EINVAL because svc_op_translate resolved their client-VA destinations through a walk to an out-of-bounds vidmem translation before the scanner pinned the real sysmem pages. Only svc_op_translate's own failure sets op_tr_pending, so the pump consumed the entry and applied the release while the memset work was lost. svc_run_ops now marks the pass op-pending for any op that did not run (rc != 0, rc != SVC_COPY_FORWARDED): the pump holds the entry and withholds its releases (bounded by SVC_OPDEF_HOLD_MS) so the retried decode picks up the injected pin and runs the memset; on expiry the release is flushed and the op stays skipped, exactly like the -ENOENT path. Sessions with no failed ops (c1/c2 were 0 fails in r71) are unaffected.
* vgsp-sema: gate the pool-residue rule on the first translated pool read, not ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | submissions (ISSUE-0002) r70 c3 proved the submissions gate wrong: the content-rescue path consumes entry 0 (submissions++) while the pool page is still untranslated (rrc=-2), then the scanner's pin batch rewinds last_get (SVC_PIN_REWIND) and the entry is decoded again - this time with cur=134 residue, pay=1, but submissions already 1, so the residue branch was skipped, the hold branch deferred the release, and the pending retry dropped it (payload <= cur). The client's first tracker wait then never completed. Replace the freshness gate with a per-channel pool_rel_seen flag set on the first translated pool read (rrc2 == 0), persisted in the pump write-back. The residue rule now fires exactly on that first read, exactly once per channel per session.
* vgsp-sema: re-seed and apply the first release of a fresh CE channel when ↵Arthur Hanson Rasmusson2 days
| | | | | | | | | | | | | | | | | the pool word is residue (ISSUE-0002) A fresh session's first CE release could read a previous session's value through the just-injected pool pin (r64: cur=134 pay=1; r63: cur=114696). svc_init_expected_payload seeded the channel expectation from that residue, the release candidate then skipped as stale (payload <= cur) and the client's first tracker wait never completed. When the first GPFIFO entry of a fresh CE channel (submissions == 0) decodes a pool release whose word reads ahead of the payload, treat the word as residue: re-seed expected_sem_payload/entry_want from the payload and apply the release absolute instead of skipping. Monotonic trackers make the absolute write idempotent; displaced pins stay owned by the fan-out/pending paths. Bounded 'sema-chan: pool residue' diagnostic plus a residue counter in the drain line.
* vgsp-sema: forwarding owns the release when a candidate is captured (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | r65 showed the operand-hold release gate must not run before the forwarding gate: for a GR launch push whose software operand was pending, the operand hold applied the launch-completion release as soon as the software op ran, so the client passed the launch-completion wait while the physical launch was still in flight; when the physical launch later timed out (qmd_rel stuck at 5, want 6) the client read zeros (MAIL bad). Order the release sites fwd-first: when a forwarding candidate is captured, drop any operand-hold record for the entry (the fwd record set carries the same release) and record into the fwd set so the hardware completion (or its bounded fallback) is the single delivery edge. The operand hold keeps owning the release only when no candidate is captured, exactly as r62 intended.
* vgsp-sema: hold unresolved-operand entries and withhold their releases ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | | | (ISSUE-0002) The guest's pma scrub submits vidmem memsets whose operand translations (client buffer VAs pinned by the scanner) land about a second after the push. r58 lost those ops to the one-pass skip; r59 showed that retrying them without deferring the entry's releases corrupts the next client (the late re-run memset executes after the client passed the release and refilled the staging buffer). Hold the entry AND record its decoded releases in per-(device, channel, entry) slots while any op of the pass is untranslated; apply the recorded set when the retried decode completes every op, deliver it late on entry consume, or flush it when the bounded hold (10 s, inside the 25 s CE grace) expires with the op left skipped as before. Slots are dropped on session flush and for vanished channels. New opdef=held/applied/expired counters in the drain line.
* Revert "vgsp-sema: hold an entry on any unresolvable operand, not only ↵Arthur Hanson Rasmusson3 days
| | | | | | mid-scan (ISSUE-0002)" This reverts commit 61cc7629f989edd216c4381313c07f15ceaf1559.
* vgsp-sema: hold an entry on any unresolvable operand, not only mid-scan ↵open-iov-agent3 days
| | | | | | | | | | | | | | (ISSUE-0002) The guest driver's pma scrub submits vidmem memsets on a GR channel; the operand translations (client buffer VAs, pinned by the scanner) land about a second after the push. The one-pass skip lost those memsets forever and the guest spun in scrubCheckAndWaitForSize (r58 c3: three bigop memsets rc=-22 at 17:54:35, their pins logged at 17:54:36, the channel froze at get=89 with the client in pmaAllocatePages). Mark the operand path pending on any unresolved operand so the existing bounded entry hold retries the decode until the translation lands; raise the retry bound to 160 so the hold covers the observed pin latency.
* vgsp-sema: never seed last_get past a discovery put >= 2 (ISSUE-0002)open-iov-agent3 days
| | | | | | | | | | | | | | | A fresh CE channel can legitimately reach the service's first pass with a live put far above 1: the driver publishes the put once per submitted push ((cpu_put + 1) % entries) and clients stream pushes while the service is still discovering the channel. r56 measured healthy sessions at put=66/134 within seconds; r54's third-session rank2 discovery read p=67, g=67 (a LIVE put), the r48 seeding set last_get=67, and the client's first push (entry 0) was never decoded - the channel_pool_add spin. Always start at slot 0; a genuinely stale block is still handled by the existing put-drop resync. Also make the CE put-snapshot diagnostic per-rank (the channel struct iterated is a local copy, so the old per-channel stamp let the first rank spend the whole session budget) and raise its cap for a third-session rank2 to keep logging through a stall.
* vgsp-sema: software-author the GR launch completion release when the ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | physical forward fails (ISSUE-0002) r51/r52 root evidence: the second session's GR launch was forwarded to the physical engine; the backend timed out (qmd_rel=5 want=6 after 5 s) while the hold resolution applied only the *stream-recorded* channel release. The QMD launch completion semaphore (the word the client polls in its launch- completion wait) is carried by the GR candidate (gr_rel_va/gr_rel_payload), not by the release records, and nothing authored it on failure - the client spun forever in the completion wait and the pump went idle (pushes=1207, pending=499, crel frozen). Fix: - a GR hold waits for the backend's bounded outcome (12 s bound vs the backend's 5 s poll) instead of the 500 ms CE copy bound, so the release is never authored while the physical submission is still writing guest pages; - on hold resolution without a successful hardware copy (timeout, abort, expiry, or a release0 page the backend could not observe), the QMD release is software-applied forward-only (read/compare, absolute payload write) - idempotent against a late engine write and never rewinds the tracker; - a GR capture whose request did not queue (refused/backend busy/hold active) authors the same release inline when the CUDA bridge is disabled (the bridge job owns it otherwise); - new work_guard counter gr_sw_rel names how often the fallback authored a release, and each write logs its va/payload. Validated: c1 GREEN, c2 GREEN in the same VM (r54; CE->GR, mail + VADD STATUS OK). In that run the forward succeeded so gr_sw_rel stayed 0; the timeout branch is the one r52 proved is reachable.
* vgsp-sema: only ALL-ZERO window-guess push reads fall through (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | r50 regression: requiring method-stream plausibility for the window source rejected valid non-method push prefixes and corrupted c1 (VADD bad=32767), so that gate is reverted. Narrow it to the r49 failure mode: a window read returning all zeros cannot be a real page (the paired base page is real), so fall through to the learned/discovered resolvers. Non-zero window content is trusted exactly as before.
* vgsp-sema: implausible window-guess push reads fall through to per-page ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | resolvers (ISSUE-0002) r49: the second session's CE push pages beyond the paired base page read all-zero through the linear window guess; because the window read 'succeeded', the learned/discovered/targeted resolvers (src=3) were never tried and every follow-up push was held as unreadable (crel frozen, client spinning). A valid push's first chunk always parses as a method stream, so require that for the window source on the first chunk only and fall through otherwise. Correct window reads are unaffected.
* vgsp-sema: resync a CE channel past residual USERD state on counter reset ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | (ISSUE-0002) r48: the layout USERD of the second session's rank0 was accepted with the previous session's residual PUT=134; the pump consumed 34 stale slots, then the guest's fresh counter reset PUT to 1 and the pump wrapped: last_get never matched the guest's sequence and the live first push was not processed. The client was later seen pushing 216-byte no-release entries with crel frozen. A fresh channel's first push always publishes PUT<=1 and the client waits for its release before a second push, so a PUT>=2 at discovery is residual: seed last_get past it and resync to slot 0 when the guest's own counter reset drops PUT below put_prev. Clear the wait state once entries are consumed without a reset so later genuine wraps keep the normal wrap path.
* vgsp-sema: CE layout USERD accepts on PUT alone; GET is GPU-owned (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | r47 guest truth (channel_info walk in the hung second client): every CE channel's GP_PUT/GP_GET GPU VAs are ring+0x2008c / ring+0x20088 and the live rank0 USERD is the layout block at fb 0x90000 (the emulation's offsets were right all along). The coherence gate required g <= p, but the GET word is GPU-owned and holds leftovers (session 2 rank0: PUT=1, GET=136 from the previous session) - so the live block was rejected forever (r44 strict never accepted; r42/r46 latched the stale registered candidate). Accept the CE layout block when the PUT word is plausible; the pump tracks ch->last_get and rewrites GP_GET as it consumes entries.
* vgsp-sema: CE USERD layout-only + deferred FB sweep diag (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | | r46 evidence: in a second session the CE rank0 layout probe at ring+0x10000 read a transient incoherent pair (-EINVAL) at discovery and the old candidate list immediately latched the stale registered pool-relative offset (another rank's previous-session USERD, frozen 1/1), hiding every later GP_PUT. For CE channels enforce the ring-derived layout block only, retry until it reads coherent, and never fall back to the registered/neighbour candidates. A bounded (+3s, +33s) FB sweep logs the live put/get block when the layout never becomes coherent.
* Revert "vgsp-sema: CE USERD is layout-only" for now (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | r43/r44 showed the strict CE branch never engaged for the second session: no sema-chan/sema-put line for c2's CE channels at all (the layout probe did not accept), so c2 stalled before any USERD was selected. Restore the r42 validated-layout behaviour (c1 green, c2 progresses to the known crel=216 stall) until the layout probe's failure in session 2 is instrumented. The next experiment must log the CE branch's ring-validation and layout-probe rc/p/g per pass.
* vgsp-sema: CE USERD is layout-only; never latch the registered stale block ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | (ISSUE-0002) The pump calls svc_chan_userd before the ring is pinned; the old candidate list then accepted the registered pool-relative userdMem offset (stale cross-session 1/1) and userd_ok froze it for the session (r42 c2 rank0 userd=0xf0000 while the layout block 0x90000 is the one the guest writes). For CE channels: wait for the ring, validate its entry0 (a CE push VA + len), then accept the layout USERD at ring+0x10000 even while idle; anything else returns -ENOENT so the next pass retries. Registered-offset candidates remain for non-CE channels.
* vgsp-sema: validate the CE ring entry0 before trusting the layout USERD ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | (ISSUE-0002) r41 c1 regressed: svc_lookup answered rank4's fresh gpfifo VA with an unrelated FB offset (0x90000) and the idle acceptance cemented the wrong USERD (ring+0x10000 of the bogus base). Gate the layout-derived USERD on the ring's entry0 decoding to a CE push VA + plausible len, the same check svc_ring_autopin already uses; a bogus ring now falls through to the previous candidate list.
* vgsp-sema+guest-diag: fix CE USERD discovery for re-registered sessions ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | | | (ISSUE-0002) svc_chan_userd now prefers the layout-derived CE USERD (ring_pa + 0x10000 in the ring aperture) and accepts it while idle, so a second session cannot latch onto the registered pool-relative offset when that alias holds a DIFFERENT rank's stale USERD: r40 c2 rank0 read rank3's stale 1/1 at FB 0xf0000 and the pump never followed rank0's own GP_PUT (guest GPUT mapping for gpFifoGpuVa proves ring+0x10000). guest_pin_stream.va_to_pa drops translations outside the registered guest-RAM windows so BAR1-phys addresses (0x70000xxxxx) are no longer injected as SYS pins; r40 rank2's ring resolved to the bogus pa=0x70000c0000 ap=SYS and read 66/66.
* guest-diag: accept both GPFIFO ring windows and validate channel_info for ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | #put (ISSUE-0002 diag) CE channels register gpfifo=0x121010000+0x30000*rank, outside the GR-only window [0x200200000,0x200400000) the #put resolver searched, so no #put line ever fired for the CE channels the second session stalls on. Resolve channel_info from both windows with layout validation, keep re-resolving the channel after the pool mem has resolved, reset the #put map on client switch, and add bounded resolver diagnostics to the guest log.
* guest-diag: stream channel CPU-side GPUT on change (#put lines, ISSUE-0002 diag)Arthur Hanson Rasmusson3 days
|
* vgsp-sema: per-second GP_PUT snapshots for CE channels (ISSUE-0002 diag)Arthur Hanson Rasmusson3 days
|
* vgsp-sema: per-session entry-outcome and stream probes for the CE->GR ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | transition (ISSUE-0002) r31 experiment: log every CE GPFIFO entry read (budget raised, reset per session), every svc_decode_stream call, and the two exits that consume an entry without a decoded release (norel hold / unread slow path). Names the exact exit taken by the second session's follow-up pushes.
* vgsp-sema: fall back to the displaced injected pair for push reads (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | The conflicting-injection race handled for pool releases also poisons push pages: when the later store wins the pin table, the pump reads a page the client never wrote, the push fails the method-stream plausibility check and its releases never decode. When the primary read is implausible (or unreadable through every source), try the recorded displaced translation once before the rescue/hold paths; content plausibility stays the arbiter. Bounded sema-pushalt evidence lines (32/session).
* vgsp-sema: free the release-apply budget and re-arm the push decode budget ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | per session (ISSUE-0002) The ipin store/lookup probes served their purpose (the displaced-pin race is understood and handled); keeping them consumed the shared per-session counter_rel_dbg budget before the release path could log, which left r25's second-client stall blind. Remove them and reset svc.entry_log in the session flush so the next session's sema-push decode lines are visible.
* vgsp-sema: keep displaced injected pins and fan pool releases out to them ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | | (ISSUE-0002) Two host tools can inject conflicting PAs for the same pool-semaphore VA page; the pin table is one entry per page, so the later store won and the release written through the loser was invisible while the client polled the winner forever (observed r22/r24: fresh va=0x121000000 -> pa matched the client pagemap; a later stale pair overwrote it, the single release write landed on the stale page, and the second client spun in UVM registration). Remember the displaced pair and write the release to it as well: forward-only and read-back proven, best effort, sequential clients only (documented in the helper).
* vgsp-sema: bust the verified pool cache on every fresh injection + ipin ↵Arthur Hanson Rasmusson3 days
| | | | store/lookup probes (ISSUE-0002)
* vgsp-sema: defer pool releases decoded before the guest base/injection lands ↵Arthur Hanson Rasmusson3 days
| | | | | | | | | | | | | (ISSUE-0002) Hold pool-VA releases for 5s after a channel registration so the fresh session injection supersedes the previous client's translation, defer a readable-zero pool word instead of absorbing the write, and keep an INC pending entry alive until the word is non-zero. Second-session releases still resolved to the previous client's page (observed: write to 0x122090000 while the fresh injection at 0x134590000 stayed zero) - the stale-pin source is the remaining lead; diagnostic pool-rel candidates added.
* vgsp-sema: never drop a decoded pool release in the counter path (ISSUE-0002)Arthur Hanson Rasmusson3 days
| | | | | | | | | | A CE (pool) release that reaches svc_counter_release_apply finds no counter slot and was discarded. The launch path normally writes pool releases via svc_sem_write, but skips it whenever it believes the value was already applied (or the word was unreadable at decode time); the second session's early releases were lost exactly there. Apply forward-only absolute through the pool writer instead - idempotent for a monotonic tracker, so a duplicate can never double-count.
* vgsp-sema: hoist the GR-ring discovery bitmap declaration for the session flushArthur Hanson Rasmusson3 days
|
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-12 11:44:29 +0000