gsp-phys: P6c C3 - relay forwarded-engine faults to the guest (default off)

New default-off gsp_phys_fault_relay: when a relayed copy times out, the
GSP-RM status queue is drained and the first OS_ERROR_LOG (0x1006) recorded
after the submission's doorbell is decoded (NOCAT corroboration), matched
against the ladder channel's runlist/chid, and delivered to the guest by
writing its channel notifier (info32=exceptType, info16=engine type,
status=0xffff) and posting the channel event. A late-release recheck skips
delivery if the engine actually completed, one delivery per hold, and the
existing software fallback always runs first, so no release or data is lost.

- pure decode helpers extracted from the status printers; prints unchanged
- status-queue walks serialized with a mutex; unmatched 0x1006 records are
  no longer silently dropped
- fault epoch snapshot at the doorbell prevents stale records from matching
- fault counters and faults:/fault_relay: state lines
- default-off gsp_phys_fault_inject debug knob forces the next relayed
  pair's source PTE to zero to produce a real engine fault
- pump notifier latch prevents clobbering a delivered fault

Live: injection run produced a real CE0 FAULT_PTE (0x1f @0x200000), decoded
with errString, chid/runlist matched, notifier delivered, software fallback
applied (applied+fallback==engaged), clean release; the guest RM reacted to
the delivered fault by tearing down its channels (faithful semantics).
Healthy control: faults all zero, relay requested=40 ok=40, smoke
matched=64/64, clean release. Caveat: the auxiliary event post failed
(no_event=1, no MSI-X) and the guest teardown means fault injection is not
smoke-compatible; the parameter stays off.

1 files changed, 27 insertions, 0 deletions

diff --git a/kernel/gsp_phys_gpu.h b/kernel/gsp_phys_gpu.h
index 6bfe3f0..214c62d 100644
--- a/kernel/gsp_phys_gpu.h
+++ b/kernel/gsp_phys_gpu.h
@@ -139,6 +139,33 @@ bool gsp_phys_work_hw_primary_enabled(void);
 bool gsp_phys_relay_enabled(void);
 
 /*
+ * True when the gsp_phys_fault_relay module parameter is set (default 0:
+ * a relayed forwarded CE copy that times out is abandoned to the unchanged
+ * software release fallback, exactly as in P6c C0+C1). When set together
+ * with gsp_phys_relay, gsp_phys_work_forward, gsp_phys_guest_mem and
+ * gsp_phys_work_defer_release, the relay-timeout path drains the GSP-RM
+ * status queue, decodes an engine fault (function 0x1006, corroborated by a
+ * 0x1020 NOCAT record) and delivers it to the guest only when its
+ * runlistId/chid match the ladder channel: the guest channel notifier is
+ * written (info32=exceptType, info16=Nv2080 engine type, status=0xffff) and
+ * the channel event posted (MSI-X). The software releases still run; a fault
+ * is never delivered for a mismatch, a late release or a stale record.
+ * One-line rollback: gsp_phys_fault_relay=0.
+ */
+bool gsp_phys_fault_relay_enabled(void);
+
+/*
+ * True when the gsp_phys_fault_inject module parameter is set (default 0:
+ * forwarded guest CE copies program the real source PTE, byte-identical
+ * behavior). When set, the relayed pair's source PTE is written as an
+ * invalid (zero) entry on the next relay submission, so the engine faults
+ * and the C3 fault path can be exercised end to end. It only ever changes
+ * the source PTE of a relayed pair (a nonzero sem_hpa); every other copy and
+ * every non-relay submission is untouched.
+ */
+bool gsp_phys_fault_inject_enabled(void);
+
+/*
  * Translate a guest GPA to the pinned host PA (gsp_phys_guest_mem ladder
  * support). The mdev is registered with vfio_register_emulated_iommu_dev():
  * its VFIO type-1 container keeps the guest DMA mappings privately, so