gsp-phys: P6b waves 1.5 and 2 - deferred release and hardware-authored guest copies

- new default-off gsp_phys_work_defer_release: a forwarded guest push's
  releases are recorded and its GPFIFO entry is held (no consume, no
  GP_GET write, no counter publish/stall) until the work item publishes
  completion; the service thread then delivers the releases, with a
  fallback on failure/timeout (500 ms cap enforced every pump pass)/
  pause/quiesce/channel-delete
- new default-off gsp_phys_work_hw_primary: the software copy is skipped
  for forwarded ops, so the physical engine authors the data; a once-only
  software copy runs before the releases on any hardware failure or when
  the request cannot be queued
- release-apply helpers extracted; exactly-once latches; channel-delete
  and teardown handling; counters and a work_guard state line
- live: deferred-release run engaged=75 applied=75 timeouts=0 with the
  full smoke green; hardware-primary run req=100 ok=100 verify_ok=100
  fallback=0 copy_skipped=119 copy_fail=0, smoke ALL OK 64/64 (guest read
  data the engine authored), with 19 fail-closed inline software
  fallbacks under single-inflight contention; a range-unregistered run
  proved the fallback (121 skipped copies, all recovered, smoke green)
- defaults off; the software path and all existing guards unchanged

1 files changed, 27 insertions, 3 deletions

diff --git a/kernel/gsp_sema_service.h b/kernel/gsp_sema_service.h
index 9638dde..35d1c0f 100644
--- a/kernel/gsp_sema_service.h
+++ b/kernel/gsp_sema_service.h
@@ -78,9 +78,11 @@ void gsp_sema_service_work_resume(void);
 
 /*
  * P6b wave-1 guest-forwarding counters for the gsp-phys-state work_guest
- * line: guest-eligible requests seen by the pump, requests that fell back to
- * the ladder copy, and verified hardware copies. Plain reads of the service
- * counters; safe from the debugfs show path.
+ * line: guest-eligible requests seen by the pump, guest requests whose
+ * hardware copy failed (wave 1: fell back to the ladder copy; with
+ * gsp_phys_work_defer_release the service applies the held releases instead),
+ * and verified hardware copies. Plain reads of the service counters; safe
+ * from the debugfs show path.
  */
 void gsp_sema_service_fwd_guest_stats(u64 *req, u64 *fallback, u64 *ok);
 
@@ -94,6 +96,28 @@ void gsp_sema_service_fwd_cand_stats(u64 *none, u64 *sys_sys, u64 *mixed,
 				     u64 *vid_vid, u64 *resolve_fail);
 
 /*
+ * P6b wave-1.5/wave-2 deferred-release and copy counters for the
+ * gsp-phys-state work_guard line: copy_skipped (the software copy was
+ * withheld because hardware-primary authorship is armed; always 0 when
+ * gsp_phys_work_hw_primary=0), copy_soft (software fallback copies the
+ * service thread performed before applying releases), copy_fail (fallback
+ * copy attempts that failed), releases deferred until the hardware copy
+ * completed, holds applied on completion (including a hold whose channel
+ * vanished after a successful copy), holds applied because the copy failed /
+ * timed out / forwarding was aborted, releases currently held (gauge),
+ * deferred records dropped by the record bound, holds engaged, holds that hit
+ * the timeout, and holds dropped unresolved after their channel vanished (no
+ * successful copy had completed). All zero while the feature is off. Plain
+ * reads, safe from the debugfs show path.
+ */
+void gsp_sema_service_fwd_hold_stats(u64 *copy_skipped, u64 *copy_soft,
+				     u64 *copy_fail, u64 *deferred,
+				     u64 *applied, u64 *fallback,
+				     u64 *pending, u64 *overflow,
+				     u64 *engaged, u64 *timeouts,
+				     u64 *canceled);
+
+/*
  * Discover and pin GR (compute) channel rings after the CE region of the
  * channel video-memory allocation. Set false to leave GR rings untranslated
  * (the pump then only runs the CE channels).