vgsp: end-to-end CUDA acceptance in the guest

The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.

Two data-path correctness fixes make the transfers sound:

- Once-only op execution per GPFIFO entry. The pump replays recently
  consumed ring entries when translation batches are injected so missed
  releases can land; re-executing a CE copy is not idempotent because
  its source is a client staging buffer that may have been refilled.
  Slots now record their raw entry words and a replayed entry matching
  that identity suppresses operations while still processing releases.

- Per-page translation of virtual operands. svc_copy resolved only the
  first page of a VA range and walked the PA linearly; client staging
  ranges are frequently split (observed 195 + 61 pages), so the tail of
  alternating chunks was read from unrelated memory. Multi-page copies
  now re-resolve each 4 KB page through the trusted resolver.

Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).

1 files changed, 34 insertions, 0 deletions

diff --git a/kernel/gsp_cuda_bridge.h b/kernel/gsp_cuda_bridge.h
new file mode 100644
index 0000000..2fb63d7
--- /dev/null
+++ b/kernel/gsp_cuda_bridge.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _GSP_CUDA_BRIDGE_H_
+#define _GSP_CUDA_BRIDGE_H_
+
+#include <linux/types.h>
+
+struct nv_vgpu_device;
+
+/*
+ * One decoded GRCE inline-data write: the destination VA and the payload bytes
+ * carried in the push itself. The sema service records these at decode time
+ * (the bytes are already in hand); the CUDA bridge uses them to locate a
+ * launch's argument block without a VA read, which is unavailable for
+ * UVM-managed VAs (no CPU pagemap, no published page directory, no reliable
+ * targeted resolve).
+ */
+#define GSP_BRIDGE_INLINE_MAX	16
+#define GSP_BRIDGE_INLINE_BYTES	64
+
+struct gsp_bridge_inline {
+	u64 dst_va;
+	u32 len;
+	u8 data[GSP_BRIDGE_INLINE_BYTES];
+};
+
+bool gsp_cuda_bridge_enabled(void);
+void gsp_cuda_bridge_launch(struct nv_vgpu_device *vgpu, u32 h_object,
+			    const u32 qmd[64], u64 qmd_dev_va, u64 push_va,
+			    const struct gsp_bridge_inline *recs, u32 n_recs);
+void gsp_cuda_bridge_service(struct nv_vgpu_device *vgpu);
+int gsp_cuda_bridge_init(void);
+void gsp_cuda_bridge_fini(void);
+
+#endif /* _GSP_CUDA_BRIDGE_H_ */