gsp-phys: P6c C0+C1 - engine-authored guest CE semaphore (default off)

New default-off gsp_phys_relay: for a forwarded guest CE copy whose push
ends in a one-word release, the release page is resolved at forward time,
mapped on a fresh scratch line, and the real engine writes the payload
directly into the guest-visible semaphore page; the deferred-release hold
then verifies that page and skips only that record's software apply. Any
failure (unresolvable target, refused kind, pool exhaustion, poll timeout,
data mismatch, channel teardown) falls back to applying every release in
software, so a release is never lost or duplicated.

- new exported gsp_phys_backend_guest_ce_copy_ex() + relay counters;
  gsp_phys_backend_guest_ce_copy() behavior unchanged (wrapper)
- new work_relay: counters line in gsp-phys-state
- relay limited to CE channels and one-word absolute releases; counter
  and reduction releases stay on the existing paths

Live: param-off regression matched the wave-2 baseline exactly (req=101
ok=101 verify_ok=101, copy_skipped=119, smoke matched=64/64, relay zeros).
With the parameter on: requested=68 ok=67 skipped_sw=67, verify_fail=0,
resolve_fail=0, timeouts=0, smoke 64/64, and the engine-authored guest
semaphore was observed directly (sem=0x00000003 / 0x0000007e at the guest
page). One copy in 68 released its semaphore but failed the host-side data
compare; it fell back to the software apply (fallback accounting
applied+fallback==engaged) with the smoke green - residual to triage, which
is why the parameter stays default-off. A forced-unresolvable run
(requested=103, fallback=105, no relay submissions) kept the smoke green.
Every cycle: 5 power-domain locks held, driver rebound, WPR2 down, no rail
cycles, pstore empty, same boot.

1 files changed, 15 insertions, 0 deletions

diff --git a/kernel/gsp_phys_gpu.h b/kernel/gsp_phys_gpu.h
index 70b98e4..6bfe3f0 100644
--- a/kernel/gsp_phys_gpu.h
+++ b/kernel/gsp_phys_gpu.h
@@ -124,6 +124,21 @@ bool gsp_phys_work_defer_release_enabled(void);
 bool gsp_phys_work_hw_primary_enabled(void);
 
 /*
+ * True when the gsp_phys_relay module parameter is set (default 0: a
+ * forwarded guest CE copy polls the module-owned ladder semaphore and the
+ * software release path stays the sole author of the guest-visible release).
+ * When set with gsp_phys_work_forward, gsp_phys_guest_mem and
+ * gsp_phys_work_defer_release, one selected one-word release of a forwarded
+ * guest push is authored by the real CE engine directly into the
+ * guest-visible semaphore page (a third scratch PTE maps that page) and the
+ * service thread skips the software apply of that record once the guest word
+ * carries the payload; every failure, timeout, abort or failed verification
+ * falls back to the unchanged software apply, so a release is never lost or
+ * duplicated. One-line rollback: gsp_phys_relay=0.
+ */
+bool gsp_phys_relay_enabled(void);
+
+/*
  * Translate a guest GPA to the pinned host PA (gsp_phys_guest_mem ladder
  * support). The mdev is registered with vfio_register_emulated_iommu_dev():
  * its VFIO type-1 container keeps the guest DMA mappings privately, so