vgsp: end-to-end CUDA acceptance in the guest

The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.

Two data-path correctness fixes make the transfers sound:

- Once-only op execution per GPFIFO entry. The pump replays recently
  consumed ring entries when translation batches are injected so missed
  releases can land; re-executing a CE copy is not idempotent because
  its source is a client staging buffer that may have been refilled.
  Slots now record their raw entry words and a replayed entry matching
  that identity suppresses operations while still processing releases.

- Per-page translation of virtual operands. svc_copy resolved only the
  first page of a VA range and walked the PA linearly; client staging
  ranges are frequently split (observed 195 + 61 pages), so the tail of
  alternating chunks was read from unrelated memory. Multi-page copies
  now re-resolve each 4 KB page through the trusted resolver.

Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).

1 files changed, 29 insertions, 2 deletions

diff --git a/kernel/main.c b/kernel/main.c
index 7001b3d..ca973b0 100644
--- a/kernel/main.c
+++ b/kernel/main.c
@@ -1285,15 +1285,34 @@ static int nv_vgpu_vfio_dev_mmap(struct vfio_device *vdev, struct vm_area_struct
 		return 0;
 	}
 
-	/* BAR2 region - ioremap'd host BAR; map backing physical address */
+	/* BAR2 region - emulated shadow (emulation-only) or real BAR (claim) */
 	region_base = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_BAR2_REGION_INDEX);
 	region_size = vgpu->bars[2].size;
 	if (offset >= region_base && offset < region_base + region_size) {
+		unsigned long bar2_off = offset - region_base;
+		unsigned long pos;
+
 		if (!vgpu->bars[2].vaddr || region_size == 0)
 			return -ENODEV;
 		if (offset + size > region_base + region_size)
 			return -EINVAL;
-		pfn = (vgpu->bars[2].gpa + (offset - region_base)) >> PAGE_SHIFT;
+		if (vgpu->bars[2].emulated) {
+			/* Emulation-only: map the private shadow's own pages,
+			 * never the physical GPU's BAR2 PFNs. */
+			vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
+			vma->vm_page_prot =
+				pgprot_writecombine(vma->vm_page_prot);
+			for (pos = 0; pos < size; pos += PAGE_SIZE) {
+				pfn = vmalloc_to_pfn(vgpu->bars[2].vaddr +
+						     bar2_off + pos);
+				if (remap_pfn_range(vma, vma->vm_start + pos,
+						    pfn, PAGE_SIZE,
+						    vma->vm_page_prot))
+					return -EAGAIN;
+			}
+			return 0;
+		}
+		pfn = (vgpu->bars[2].gpa + bar2_off) >> PAGE_SHIFT;
 		vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
 		return remap_pfn_range(vma, vma->vm_start, pfn, size, vma->vm_page_prot);
 	}
@@ -2643,6 +2662,14 @@ module_param_named(sema_ce_grace_ms, gsp_sema_ce_grace_ms, uint, 0644);
 MODULE_PARM_DESC(sema_ce_grace_ms, "CE pending-entry grace window in ms (default 8000)");
 
 /*
+ * Bounded work-before-release hold (ms) for an unreadable CE entry; the
+ * deferred retry probe shows translations/pins land 0.5-2 s late. Absolute
+ * per-entry deadline, never reset; 0 disables.
+ */
+module_param_named(sema_push_hold_ms, gsp_sema_push_hold_ms, uint, 0644);
+MODULE_PARM_DESC(sema_push_hold_ms, "CE unreadable-entry work-before-release hold in ms (default 2500, 0 disables)");
+
+/*
  * Grace window (ms) an untranslatable GR entry stays pending before the pump
  * consumes it as stale, so a late push-buffer pin can still make it readable.
  */