sema: r407 drop-stale must not deliver a held entry's releases on a rewind (only consumed entries are stale)

The r406 inplace_scale repro root cause: the H2D copy's operand
translation was unresolved (memcopy rc=-2) and the entry was held with
its release withheld (correct); a pin-batch rewind then re-opened lower
indices (83) while the held entry (91) sat ahead, and the unconditional
svc_opdef_drop_stale treated 91 as an already-consumed entry and applied
its withheld release at 104955.751. The copy itself only executed at
104957.418 - the client raced past its blocking cudaMemcpy, launched,
the launch's copy-in read the stale buffer (previous run's data) and
the copy landed after the launch's copy-out: wrong results with a
healthy-looking 32/1032 changed and no classifier firing.

An entry is only 'passed' if the ring consumed it (slot_done_epoch);
gate the stale drop on svc_replay_seen so a rewind can never deliver a
still-held entry's release early.

1 files changed, 17 insertions, 2 deletions

diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c
index a7f4b4e..53e2140 100644
--- a/kernel/gsp_sema_service.c
+++ b/kernel/gsp_sema_service.c
@@ -16009,9 +16009,22 @@ static void svc_opdef_drop(struct nv_vgpu_device *vgpu, struct svc_channel *ch,
  * The ring advanced past a slot's entry (a slot for another entry of the
  * same channel is being processed): its set belongs to a consumed entry.
  * Deliver late (forward-only, so it cannot double-apply) and drop it.
+ *
+ * r406: an entry is only "passed" if the ring actually CONSUMED it
+ * (slot_done_epoch). Without that proof the pin-batch rewind breaks the
+ * invariant: it re-opens lower indices (e.g. 83) while a held entry
+ * (e.g. 91) sits AHEAD in the same epoch, and the old unconditional drop
+ * treated the held entry as consumed and delivered its withheld releases
+ * early. Observed (r406 inplace_scale): the H2D copy's operand was
+ * unresolved (rc=-2), the entry held with its release withheld; a rewind
+ * pass at idx=83 fired drop-stale caller=4 for idx=91 and the release
+ * went out at 104955.751 - while the copy only executed at 104957.418.
+ * The client raced past its blocking memcpy, launched, the launch read
+ * the stale buffer (previous run's data), and the copy landed after the
+ * launch's copy-out: wrong results with 32/1032 "changed".
  */
 static void svc_opdef_drop_stale(struct nv_vgpu_device *vgpu,
-				 struct svc_channel *ch, u32 keep_idx)
+				 struct svc_channel *ch, u32 keep_idx, u32 put)
 {
 	int i;
 
@@ -16021,6 +16034,8 @@ static void svc_opdef_drop_stale(struct nv_vgpu_device *vgpu,
 		if (!e->active || e->vgpu != vgpu ||
 		    e->h_object != ch->h_object || e->entry_idx == keep_idx)
 			continue;
+		if (!svc_replay_seen(ch, e->entry_idx, put))
+			continue;	/* never consumed: not stale (rewind) */
 		if (e->n)
 			svc_fwd_rel_apply_all(vgpu, ch, e->rel, e->n);
 		svc_opdef_clear(e, SVC_OPDEF_CLR_STALE);
@@ -20139,7 +20154,7 @@ static void svc_pump_channels(struct nv_vgpu_device *vgpu)
 			 * (forward-only, so it cannot double-apply) and drop
 			 * it before this entry is examined.
 			 */
-			svc_opdef_drop_stale(vgpu, ch, (u32)idx);
+			svc_opdef_drop_stale(vgpu, ch, (u32)idx, put);
 
 			/*
 			 * A pin-batch rewind replays recently consumed