vgsp: fail4 - LAND ALL H2D CHUNKS: consume the staging snapshot; capture on first decode

vector_add was deterministic-wrong (bad=786335 checksum=359549561 identical
across boots) because the 1MB H2D chunk copies always read the driver's
ping-pong staging AFTER it had been refilled (the fence release applied long
before the late op execution; 60+ rc=-2 retry storms per chunk).

Two wiring fixes (r314-r315):

1) The decode-time snapshot was NEVER CONSUMED: svc_copy looked it up
(svc_copy_snap_lookup at the top) and freed it on success, but no code read
from it - every copy fell through to the live per-page source resolution
and read refilled staging. The source read loop now takes the bytes from
the snapshot when present (memcpy from snap + o). The copy also no longer
needs a live source translation, so it executes on its first allowed pass
instead of storming retries.

2) Capture on the DENIED pass too: the old capture gate lived in the
op-build block (allowed passes only - by then the fence fired and staging
was refilled). The denied device-dst copy branch now captures immediately
(pipeline-order moment), same translation-probe guard.

Measured r315 (fresh boot): vector_add first run bad=157696 (cold-run
capture coverage gap), second run bad=0 checksum=392344090.5 STATUS OK
with ALL EIGHT chunks clean and no retry storms (~75 s, guest survives).
Previously 786335 bad / identical checksum every boot.

1 files changed, 47 insertions, 1 deletions

diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c
index 7a7755f..09b8ee4 100644
--- a/kernel/gsp_sema_service.c
+++ b/kernel/gsp_sema_service.c
@@ -5803,7 +5803,22 @@ static int svc_copy(struct nv_vgpu_device *vgpu, u64 dst, u64 src, size_t total,
 			chunk = min_t(size_t, chunk,
 				      0x1000 - ((dst_orig + o) & 0xfff));
 
-		if (src_shadow) {
+		if (snap) {
+			/*
+			 * r314: consume the decode-time staging snapshot.
+			 * The capture machinery existed since cb50282b but
+			 * the execution never read it (lookup at svc_copy's
+			 * top, free on success, NO consumer) - every copy
+			 * read the live staging, which the driver had
+			 * refilled once the push's fence fired (the
+			 * deterministic 786335-bad vector_add output,
+			 * identical across boots). Reading from the
+			 * snapshot also needs no live source translation,
+			 * so the copies execute on their first allowed
+			 * pass instead of storming 60+ rc=-2 retries.
+			 */
+			memcpy(bounce, snap + o, chunk);
+		} else if (src_shadow) {
 			if (svc_vas_read(src_orig + o, bounce, chunk))
 				SVC_COPY_TRUNC_RET(-EIO);
 		} else if (src_virt) {
@@ -8500,6 +8515,37 @@ static void svc_decode_stream(struct nv_vgpu_device *vgpu, const u8 *buf,
 			    !DMA_REMAP_ENABLE(dma) &&
 			    st.offset_out >= SVC_VAS_FLOOR) {
 				/*
+				 * r314 (CONT.57 lead 2): capture the staging
+				 * source on the DENIED pass too. This is the
+				 * pipeline-order moment: the push was just
+				 * decoded, the driver has not seen this
+				 * push's fence (it applies later in this same
+				 * pass at the earliest), so the staging still
+				 * holds this chunk's bytes. The old capture
+				 * gate lived in the op-build block, which
+				 * only runs on ALLOWED passes - by then the
+				 * fence had long fired and the staging was
+				 * refilled (r302-r313: captures 0-3/8, the
+				 * late executions read refilled staging, the
+				 * deterministic 786335-bad output). Same
+				 * translation-probe guard as the build-time
+				 * gate: only an SYS-resolving source is
+				 * host staging.
+				 */
+				if (st.line_length <= SVC_COPY_SNAP_MAX) {
+					u64 spa = 0;
+					u32 sap = 0;
+
+					if (!svc_op_translate(vgpu,
+							      st.offset_in,
+							      &spa, &sap) &&
+					    sap == CE_APERTURE_SYS)
+						svc_copy_snap_capture(vgpu,
+								      st.offset_out,
+								      st.offset_in,
+								      st.line_length);
+				}
+				/*
 				 * r308: the `!replay` gate above kept this
 				 * dead on the sw-copy channel: after the first
 				 * consumed pass every later pass classifies as