vgsp: guest GR forwarding backend and capture (gsp_phys_gr_forward, default off)gr-forwarding

Add a default-off guest GR (compute) forwarding path so a guest CUDA
launch submitted on its GR channel can be executed by the physical GR
engine on the ladder channel, with fail-closed fallback to the host CUDA
bridge and the unchanged software release path.

- New module parameter gsp_phys_gr_forward (default 0) and accessor.
- New GR execution gate: boot_stage.rmcall_gr_exec_ok, latched only after
  a real GR submission logs REAL GR SEM OK; the backend refuses while it
  is unset.
- Sema capture: a completed 64-word inline QMD on a trusted, non-replay
  GR decode becomes an svc_gr_cand (push identity, decoded release0,
  parameter/program/pointer fields, up to 8 captured inline-write
  records); the CUDA bridge stays authoritative while the parameter is
  off and is reached in order forward -> bridge -> software release.
- Backend: gsp_phys_backend_guest_gr_submit() maps pages at their own
  guest VAs through fresh Tier-1 small-PT pages/PDE installs or relocates
  inline-captured pages to ladder shadow pages (Tier 2), builds
  SET_OBJECT + the inline-QMD method run + REPORT_SEMAPHORE on the
  compute subchannel, reuses the shared GPFIFO/USERD/doorbell publish
  tail (extracted from the CE submit path, CE bytes unchanged) and polls
  the QMD release and/or module semaphore, draining the epoch-gated
  fault queue on timeout.
- Sema wiring: GR requests reuse the INFLIGHT/hold/deferral machinery so
  the guest channel release applies only on the submit's success/failure
  edge; hold shims preserve the channel engine type.
- Counters and a work_gr state line plus a stats accessor.
- Bounds are fixed: 16 pages, 8 inline records, 64 QMD words; every
  pointer/size checked, pool exhaustion and stale mappings fail closed.
- New ABI layouts pinned with static_asserts.

Compile-only change: no device test performed.

1 files changed, 11 insertions, 0 deletions

diff --git a/kernel/gsp_sema_service.h b/kernel/gsp_sema_service.h
index 22e9edc..4b04892 100644
--- a/kernel/gsp_sema_service.h
+++ b/kernel/gsp_sema_service.h
@@ -148,6 +148,17 @@ void gsp_sema_service_fwd_fault_stats(u64 *matched, u64 *unmatched,
 				      u64 *late_release);
 
 /*
+ * GR (compute) forwarding counters for the gsp-phys-state work_gr line:
+ * captures offered to the request path (req), launches the physical GR
+ * channel completed (ok), offerings that fell back to the CUDA bridge /
+ * software release (fallback), gate refusals (refused), backend timeouts
+ * (timeout) and timeouts carrying a decoded engine fault (fault). All zero
+ * while gsp_phys_gr_forward=0. Plain reads, safe from the debugfs show path.
+ */
+void gsp_sema_service_fwd_gr_stats(u64 *req, u64 *ok, u64 *fallback,
+				   u64 *refused, u64 *timeout, u64 *fault);
+
+/*
  * Discover and pin GR (compute) channel rings after the CE region of the
  * channel video-memory allocation. Set false to leave GR rings untranslated
  * (the pump then only runs the CE channels).