gsp-phys: P6b wave 1 - shadow-forward guest CE copies to the physical engine

- the sema service captures the first eligible copy per push from the
  software resolver's own operands (the real push shape mixes guest RAM
  with emulated-VRAM shadow addresses, including physical destinations)
- HPA resolution: guest RAM through the shared guest-RAM file page cache,
  emulated VRAM through the mapping of its kernel shadow
- new scratch page-table pool: 16 zeroed page-table pages with their
  page-directory entries built before any engine access, handing out one
  whole 128-byte GMMU PTE fetch line per copy pair (16 PTEs, 512 pairs
  per boot, never reused; a PTE written into an already-fetched line is
  not observed and there is no invalidate path)
- the backend submits the copy on the module-owned channel and verifies
  the result host-side; every miss falls back to the ladder shadow copy,
  and the software pump remains authoritative
- live (full smoke, both parameters on): req=42 ok=42 verify_ok=42
  verify_fail=0 fallback=0, guest payload roundtrip ALL OK 64/64,
  clean latched release; defaults stay off

1 files changed, 30 insertions, 0 deletions

diff --git a/kernel/gsp_phys_booter.h b/kernel/gsp_phys_booter.h
index b44d079..cc3b935 100644
--- a/kernel/gsp_phys_booter.h
+++ b/kernel/gsp_phys_booter.h
@@ -212,6 +212,36 @@ void gsp_phys_backend_work_counters(u64 *submitted, u64 *released,
 				    u64 *timed_out, u64 *refused);
 
 /*
+ * P6b wave-1 guest-operand forwarding (gsp_phys_work_forward=1 AND
+ * gsp_phys_guest_mem=1; both default off, either off keeps the P5 ladder
+ * shadow exactly as it was).
+ *
+ * gsp_phys_backend_guest_ce_copy() runs one bounded CE copy on the ladder
+ * channel with both operands mapped to real guest RAM by translated host PA:
+ * it re-checks the READY backend, the ladder channel, work forwarding, guest
+ * memory, the identity DMA domain and the active guest-RAM file; consumes one
+ * whole fresh, never-reused 128 B PTE fetch line per copy pair from the guest
+ * scratch PT pool built at stage 3 (no line reuse: the GMMU caches PTE
+ * fetches at 128 B granularity and there is no invalidate, so two pairs
+ * sharing a line would leave the second pair's PTEs stale); then submits
+ * src_off..+bytes from src_hpa to dst_hpa with the module semaphore, polls
+ * it lock-free for up to poll_ms (clamped to 200 ms) and verifies the copy
+ * with a direct-map memcmp (safe: identity domain and pfn_valid HPAs). Returns 0 on a verified copy,
+ * -ENOSPC when the scratch-PT pool is exhausted, -ETIMEDOUT on the poll
+ * bound, -EAGAIN when a gate is closed and -EINVAL/-EIO on operand/structural
+ * failures. No guest VA is ever submitted; the sema service falls back to its
+ * ladder copy on every failure, so the software release never depends on it.
+ *
+ * gsp_phys_backend_guest_counters() reads the booter-side guest counters
+ * (verified copies, direct-map verifies, verify failures, pool exhaustion,
+ * refusals including poll timeouts) for logging and the state file.
+ */
+int gsp_phys_backend_guest_ce_copy(u64 src_hpa, u32 src_off, u64 dst_hpa,
+				   u32 dst_off, u32 bytes, u32 poll_ms);
+void gsp_phys_backend_guest_counters(u64 *ok, u64 *verify_ok, u64 *verify_fail,
+				     u64 *pool_exhausted, u64 *refused);
+
+/*
  * Command-queue TX: build a msgq record with the 570.144 element/rpc header
  * layout, splitting it into 1..16 4 KB elements when it does not fit one
  * (reference elemCount = ceil((48 + rpc.length) / 4096), each element copied