vgsp: fail4 - carry the channel into pending releases; hold the fence on denied passes

Two fixes for the cold-boot classes (CONT.60):

1) Pending releases starved forever: svc_apply_pending resolved the pool
semaphore with h=0, but the verified pool resolver is channel-keyed - it
never matched the per-channel pool entries while the very same lookup at
decode time succeeded (matmul r322: 'pool-rel cand ... rrc=0 cur=0' then
'pending release starved va=0x121000010 payload=1 tries=4/64...' forever,
the client hung and the boot died). Pending entries now record the channel
they were decoded for (struct svc_pending_release.h, set from svc.cur_ch
which svc_decode_stream stamps at its top) and svc_apply_pending resolves
with it. Also applies to the 'completion hint rc=-2' class.

2) The fence escaped on denied passes: in the P6b wave-2 overflow branch
the release was queued only when n_ops>0; a denied pass (n_ops==0) but
op-pending (a copy still owed) applied the release inline, letting the
driver refill the ping-pong staging before the copy ran - the cold-run
capture gap (vector_add run 1 bad=157696/193536). The queue condition is
now n_ops>0 || svc.op_tr_pending.

Measured (r323 fresh boot, full sequence): mail OK, nvidia-smi OK, matmul
OK (no death - the r322 killer), transformer OK (no stall), vector_add
cold-run bad=100352 (two chunks, down from 157696-193536; warm runs remain
bad=0).

1 files changed, 37 insertions, 2 deletions

diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c
index 1b86945..f55502b 100644
--- a/kernel/gsp_sema_service.c
+++ b/kernel/gsp_sema_service.c
@@ -7541,6 +7541,13 @@ static void svc_decode_stream(struct nv_vgpu_device *vgpu, const u8 *buf,
 			      u64 *ops_done)
 {
 	struct svc_push_state st;
+
+	/*
+	 * r316: pending releases record the channel they were decoded for;
+	 * the verified pool resolver is channel-keyed (see svc_pending_add
+	 * and the matmul r322 starved/hung note).
+	 */
+	svc.cur_ch = ch->h_object;
 	/*
 	 * Static for the same reason as svc_handle_chan_dump: a 32-entry ops
 	 * array is a ~2 KB frame, and this is called from the single service
@@ -9028,7 +9035,25 @@ static void svc_decode_stream(struct nv_vgpu_device *vgpu, const u8 *buf,
 							 * release through the pending
 							 * pass instead.
 							 */
-							if (n_ops > 0) {
+							if (n_ops > 0 ||
+							    svc.op_tr_pending) {
+								/*
+								 * r316 (CONT.60):
+								 * op_tr_pending covers
+								 * the denied pass - the
+								 * pass built no op
+								 * (allow=0) but a copy is
+								 * still owed; applying
+								 * the fence here (the old
+								 * n_ops==0 path) let the
+								 * driver refill the
+								 * ping-pong staging before
+								 * the copy ever ran - the
+								 * cold-run capture gap
+								 * (vector_add run 1
+								 * bad=157696/193536 vs
+								 * bad=0 warm).
+								 */
 								svc_pending_add(sem_va,
 										st.sem_payload,
 										inc ? 1u : 0u);
@@ -9168,6 +9193,12 @@ struct svc_pending_release {
 	u64 va;
 	u32 payload;
 	u32 op;		/* 0 = RELEASE, 1 = REDUCTION (add payload) */
+	u32 h;		/* r316: channel the release was decoded for; the
+			 * verified pool resolver is channel-keyed and h=0
+			 * never matches its per-channel entries (matmul
+			 * r322: pool-rel cand resolved at decode (rrc=0,
+			 * h=channel) while svc_apply_pending with h=0
+			 * starved forever and hung the client). */
 	/*
 	 * Resolution attempts that found no translation for this VA.
 	 * Telemetry for ISSUE-0001: a pending release that keeps failing to
@@ -9244,6 +9275,7 @@ static void svc_pending_add(u64 va, u32 payload, u32 op)
 	svc_pending[i].payload = payload;
 	svc_pending[i].op = op;
 	svc_pending[i].tries = 0;
+	svc_pending[i].h = svc.cur_ch;
 }
 
 /*
@@ -9571,8 +9603,11 @@ static void svc_apply_pending(struct nv_vgpu_device *vgpu)
 			 * resolver: the learned map can answer with a stale
 			 * page whose value makes this release look already
 			 * delivered and drops it (see svc_sem_write).
+			 * r316: resolve with the channel the release was
+			 * decoded for (h=0 never matches the per-channel
+			 * pool entries - matmul r322 starved/hung).
 			 */
-			if (svc_pool_pa_resolve(vgpu, 0, p->va, &ppa, &pap,
+			if (svc_pool_pa_resolve(vgpu, p->h, p->va, &ppa, &pap,
 						NULL)) {
 				/* not translatable yet: keep (ISSUE-0001) */
 				p->tries++;