summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-18 00:50:35 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-18 00:53:00 -0400
commit5de33b14586b8669ebba9150b54c7ea182859214 (patch)
tree622907b13899c18b5f4925d9e39e6c0858f704ef
parentff11994cbc1cce45e5ce721c69c7957436031877 (diff)
downloadopen-iov-5de33b14586b8669ebba9150b54c7ea182859214.tar.gz
open-iov-5de33b14586b8669ebba9150b54c7ea182859214.zip
vgsp-sema: GR fresh-put-drop rewind - stop consuming the previous session's residual ring
r203 battery forensics (rank 0, sema_dbg=1): hang and progress alternate perfectly with the first USERD GP_PUT read of a fresh session. A session whose first read is residual (106/107/108 = the previous session's final put, GET==PUT) consumes the stale ring in its first pump pass, then never decodes the new client's live entries: its ctx-init completion word (the channel's descending counter slot) freezes at the first stale-decoded payload (1), publishes stop (its USERD put reads never advance past 1), and the client polls forever - multikernel (gen3), streams2 (gen5), victory_mail (gen7) and smiley (gen9) hung; memcpy (gen4), streams_events (gen6), victory_matmul (gen8) - each following a session whose final put was 1 - proceeded. The GR ring/USERD live in fixed VRAM slots (fb 0x140000 + rank*0x3000) that keep the dead session's state across svc_session_flush; m176 reset learned counter state but not the ring bookkeeping. m178/m179's r48 resync (last_get=0 on the first put drop) applies only when userd_wait_fresh is set, and the r73 rewind is CE-only (engine_type != 0), so a GR channel whose fresh put drops below put_prev keeps last_get ahead of the producer forever. Extend the r73 rewind shape to GR: rewind last_get to slot 0 (the fresh session restarts its push sequence at slot 0, and re-decoding slot 0 is idempotent - the forward-only apply drops the duplicate payload), gated by the same submissions < gpfifo_entries/2 freshness guard so a genuine wrap is unaffected. Live evidence (smiley, killed at 127105): counter page word 0x20400fff0 stuck at 1; ring put=107 get=107 at discovery; pub-adv 107->1; repeated rel-apply pay=1 -> skip once per pass. Worktree build: make kernel green (vgsp.ko linked).
-rw-r--r--kernel/gsp_sema_service.c41
1 files changed, 41 insertions, 0 deletions
diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c
index dd7a5e1..6a1595e 100644
--- a/kernel/gsp_sema_service.c
+++ b/kernel/gsp_sema_service.c
@@ -15323,6 +15323,47 @@ static void svc_pump_channels(struct nv_vgpu_device *vgpu)
ch->h_object, ch->ce_index, put,
(u32)ch->last_get);
ch->last_get = put;
+ } else if (!ch->engine_type && ch->gpfifo_entries &&
+ ch->last_get > put &&
+ ch->submissions < ch->gpfifo_entries / 2) {
+ /*
+ * m181 (r203 GR ctx-init stall): a GR channel's
+ * ring/USERD live in fixed VRAM slots
+ * (fb 0x140000 + rank*0x3000) that keep the
+ * PREVIOUS session's final state across the
+ * session flush (m176 reset learned counter
+ * state but not the ring-consumption
+ * bookkeeping). A new session's first pump
+ * pass therefore sees the residual put (e.g.
+ * 107 = the last session's final GP_PUT) and
+ * consumes the stale ring up to it, advancing
+ * last_get before the client has written its
+ * own entries. The client's fresh put then
+ * drops below put_prev, and without this
+ * rewind last_get stays ahead of the producer:
+ * the live ctx-init entries are never decoded,
+ * their REPORT_SEM releases are never applied
+ * (the completion word freezes at the first
+ * stale-decoded payload), and the client's
+ * ctx-init wait polls forever - r203 rank 0
+ * hung exactly so in multikernel / streams2 /
+ * victory_mail / smiley whenever the prior
+ * session had reached a high put (the sessions
+ * after a hung one, whose final put is 1,
+ * proceeded - a perfect alternation). Rewind
+ * to slot 0 like the r48 resync: a fresh
+ * session restarts its push sequence at slot
+ * 0, and re-decoding slot 0 is idempotent
+ * (the stale pass could only have authored the
+ * same payload, and the forward-only apply
+ * drops a duplicate). A genuine wrap cannot be
+ * this fresh (submissions guard), mirroring
+ * the r73 CE rewind.
+ */
+ pr_info_ratelimited("sema-chan: fresh put drop (GR) h=0x%x rank=%u put=%u last=%u -> rewind to 0\n",
+ ch->h_object, ch->gr_index, put,
+ (u32)ch->last_get);
+ ch->last_get = 0;
}
bitmap_zero(ch->slot_done_epoch, SVC_GPFIFO_MAX_BITS);
}