vgsp-gr r605: walk the eng==0 class newest-registered first - the stream-order fix completed (the write-pointer drain, by ordering)

The r507o stream-order fix (walk CE channels before GR ones) assumed the
client's H2D copies ride CE channels. r595 established they also ride the
compute-class channel h=0x5c000059 (eng=0), and r604 gave the decision-time
capture: at the vadd's second decision on a fresh boot,
'peek-backlog h=0x5c000059 eng=0 put=95 last=94' - the b-copy entry was
already written and PUBLISHED (GP_PUT ahead of the pump) while the launch
decision ran, invisible to every guard (the r590 watermark skips
engine_type==0; the r541/r568 gates only see DECODED copies). The pump's
stable partition walked the launch channel (registered early, 0x5c000019)
before the copy channel (registered later, 0x5c000059), so the copy was
consumed 8 ms AFTER the decision and ~320 ms before the launch's copy-in;
the run was green only because the partial-argument defer had stretched
the window. Pins landing one pass earlier would have forwarded against
the pre-copy page (c = a + 0, the r568 red draw).

Fix the order within the class: newest-registered first (descending
h_object) walks the driver's later-registered copy / push-library channels
before its compute channels, so every launch decision sees the copy ring
consumed to its live write pointer (GP_PUT) - the write-pointer drain, by
ordering. No deferral, no cap interaction, no speculative execution: the
pump still consumes each entry exactly once through its normal per-slot
identity bookkeeping (slot_ops_ident/done/complete/hash), so no identity
mirroring is needed. A client whose compute channel registers last keeps
the historical order (no worse).

1 files changed, 54 insertions, 4 deletions

diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c
index 0e6a141..c5fafa1 100644
--- a/kernel/gsp_sema_service.c
+++ b/kernel/gsp_sema_service.c
@@ -22335,10 +22335,60 @@ static void svc_pump_channels(struct nv_vgpu_device *vgpu)
 		    svc.chan[i].engine_type)
 			local[n++] = svc.chan[i];
 	}
-	for (i = 0; i < SVC_MAX_CHANNELS; i++) {
-		if (svc.chan[i].valid && svc.chan[i].vgpu == vgpu &&
-		    !svc.chan[i].engine_type)
-			local[n++] = svc.chan[i];
+	/*
+	 * r605 stream-order, the eng==0 half of the r507o fix. The same
+	 * argument that made CE channels walk before GR ones applies WITHIN
+	 * the GR class: the driver's relocation/H2D copies ride a
+	 * later-registered compute-class channel (h=0x5c000059 family,
+	 * eng=0 - the r595 finding), while the launch channel registers
+	 * early (0x5c000019). The stable partition walked the launch first,
+	 * so at a launch decision the copy channel's PUBLISHED backlog was
+	 * unconsumed and invisible to every guard: the r590 watermark skips
+	 * engine_type==0, and the r541/r568 gates only see DECODED copies.
+	 * r604 capture (fresh boot, peek armed): at the vadd's second
+	 * decision `peek-backlog h=0x5c000059 eng=0 put=95 last=94` - the
+	 * b-copy entry was already in the ring with GP_PUT ahead of the
+	 * pump, executed 8 ms after the decision and 320 ms before the
+	 * launch's copy-in, and the run was green only because the
+	 * partial-argument defer had stretched the window; pins landing one
+	 * pass earlier would have forwarded against the pre-copy page
+	 * (c = a + 0, the r568 red draw). Newest-registered first
+	 * (descending h_object) walks the copy / push-library channels
+	 * before the compute channels, so every launch decision sees the
+	 * copy ring consumed to its live write pointer (GP_PUT): the
+	 * write-pointer drain, by ordering. No deferral, no cap
+	 * interaction, no speculative execution - the pump still consumes
+	 * each entry exactly once through its normal per-slot identity
+	 * bookkeeping (slot_ops_ident/done/complete/hash), so nothing needs
+	 * mirroring. A client whose compute channel registers last simply
+	 * keeps the historical order (no worse).
+	 */
+	{
+		u32 prev_h = 0xffffffffu;
+		int picked;
+
+		for (picked = 0; picked < SVC_MAX_CHANNELS; picked++) {
+			int best = -1;
+			u32 best_h = 0;
+
+			for (i = 0; i < SVC_MAX_CHANNELS; i++) {
+				struct svc_channel *c = &svc.chan[i];
+
+				if (!c->valid || c->vgpu != vgpu ||
+				    c->engine_type)
+					continue;
+				if (c->h_object >= prev_h)
+					continue;
+				if (best < 0 || c->h_object > best_h) {
+					best = i;
+					best_h = c->h_object;
+				}
+			}
+			if (best < 0)
+				break;
+			local[n++] = svc.chan[best];
+			prev_h = best_h;
+		}
 	}
 	spin_unlock_irqrestore(&svc.chan_lock, flags);