gsp-phys: P6b waves 1.5 and 2 - deferred release and hardware-authored guest copies

- new default-off gsp_phys_work_defer_release: a forwarded guest push's
  releases are recorded and its GPFIFO entry is held (no consume, no
  GP_GET write, no counter publish/stall) until the work item publishes
  completion; the service thread then delivers the releases, with a
  fallback on failure/timeout (500 ms cap enforced every pump pass)/
  pause/quiesce/channel-delete
- new default-off gsp_phys_work_hw_primary: the software copy is skipped
  for forwarded ops, so the physical engine authors the data; a once-only
  software copy runs before the releases on any hardware failure or when
  the request cannot be queued
- release-apply helpers extracted; exactly-once latches; channel-delete
  and teardown handling; counters and a work_guard state line
- live: deferred-release run engaged=75 applied=75 timeouts=0 with the
  full smoke green; hardware-primary run req=100 ok=100 verify_ok=100
  fallback=0 copy_skipped=119 copy_fail=0, smoke ALL OK 64/64 (guest read
  data the engine authored), with 19 fail-closed inline software
  fallbacks under single-inflight contention; a range-unregistered run
  proved the fallback (121 skipped copies, all recovered, smoke green)
- defaults off; the software path and all existing guards unchanged

1 files changed, 62 insertions, 24 deletions

diff --git a/docs/03-architecture/11-physical-backend-guest-wiring.md b/docs/03-architecture/11-physical-backend-guest-wiring.md
index 7328800..cbb4b33 100644
--- a/docs/03-architecture/11-physical-backend-guest-wiring.md
+++ b/docs/03-architecture/11-physical-backend-guest-wiring.md
@@ -1,16 +1,22 @@
 # Physical Backend Guest Wiring: Driving the Guest-Facing Emulation from the Live GSP-RM Session
 
-> **Status: P0-P5(w1) implemented; P6a guest-page reachability and P6b wave-1
-> guest-copy forwarding proven live (2026-10-02).** The backend API skeleton
+> **Status: P0-P5(w1) implemented; P6a guest-page reachability and P6b waves
+> 1/1.5/2 (guest-copy forwarding, deferred release, hardware-primary copies)
+> proven live (2026-10-02).** The backend API skeleton
 > (P0), the mdev-owned lifecycle (P1, fail-open; the async FWSEC boot on that
 > path is an open defect off the critical path), the static-config identity
 > overlay (P2, verified live in an A/B run), the read-only control
 > forwarding whitelist (P3, default off, verified live: ok=22/fallback=2,
 > smoke `ALL OK` 64/64), the P5 wave-1 guest-submission shadow execution
 > (default off, verified live: 514 real CE copies during a green guest smoke
-> `ALL OK` 64/64) and the P6b wave-1 guest-copy forwarding through
+> `ALL OK` 64/64), the P6b wave-1 guest-copy forwarding through
 > module-built GMMU scratch mappings (default off, verified live: req=42
-> ok=42 verify_ok=42, 503 ladder shadow copies also run) are in the tree.
+> ok=42 verify_ok=42, 503 ladder shadow copies also run), the wave-1.5
+> deferred release (default off, verified live: engaged=75 applied=75,
+> releases exactly once) and the wave-2 hardware-primary copies (default off,
+> verified live: copy_skipped=119, queued req=100 ok=100 verify_ok=100
+> verify_fail=0 fallback=0 timeouts=0, smoke `matched=64/64` `ALL OK`, with
+> the software fallback for unqueued/failed copies) are in the tree.
 > The physical backend itself is live: the module-owned
 > GSP-RM completes initialization on the real TU104, serves the full
 > static-config reply, and shuts down gracefully. The host resource-manager
@@ -106,11 +112,12 @@
 
 The two planes now meet at the lifecycle seam (P0/P1, mdev-owned
 claim/release), the static-config identity overlay (P2), the read-only control
-whitelist (P3) and, for eligible CE copies, the P6b wave-1 scratch-mapped work
-path; objects, FB geometry, engine caps, real guest page tables and
-hardware-primary release are still emulated. The remaining bridge must be
-added at the reply point `gsp_rm_send()`, the inbound dispatch point
-`gsp_rm_handle_element()`, and the work-plane seams (see §2).
+whitelist (P3) and, for eligible CE copies, the P6b scratch-mapped work path
+(waves 1/1.5/2: guest operands, deferred exactly-once release,
+hardware-primary copies); objects, FB geometry, engine caps, real guest
+page tables and completion/fault relay are still emulated. The remaining
+bridge must be added at the reply point `gsp_rm_send()`, the inbound dispatch
+point `gsp_rm_handle_element()`, and the work-plane seams (see §2).
 
 ---
 
@@ -216,7 +223,7 @@ Legend: **L** = live backend, **E** = existing emulation, **H** = hybrid.
 | 6 | `GSP_RM_ALLOC` (103) | echo request + `NV_OK`; channels/events registered locally (`:2322`) | **H** - echo for emulation-only classes; forward selected classes (root/device/subdevice, then channel/memory) once a guest<->server handle map exists; always keep local side effects | `gsp_rm_handle_rm_alloc` `:2322` |
 | 7 | `GSP_RM_CONTROL` (76) | ~25 static per-cmd handlers; unknown -> zeroed success (`:2611`, default `:2807`) | **H** - whitelist of side-effect-free queries goes live with emulation fallback; vGPU-specific controls stay **E** (`COPY_SERVER_RESERVED_PDES` `:2647`, `PROMOTE_FAULT_METHOD_BUFFERS` `:2654`) | `gsp_rm_handle_rm_control` `:2611`; new forward helper before the switch |
 | 8 | `GSP_RM_FREE` (10) | echo + retire channel/events (`:2882`) | **H** - forward when the object was backend-allocated; always retire local bookkeeping | `gsp_rm_handle_free` `:2882` |
-| 9 | `SET_PAGE_DIRECTORY` (54) | recorded for translation + SUCCESS (`:2975`) | **H** - recording stays **E**; forwarding to the real MMU is P6c and guest pages are GPU-DMA-reachable on the reference host (P6a status below; page-table pre-build required). P6b wave 1 executes eligible copies without forwarding page-directory state, through module-built scratch page tables | `gsp_rm_handle_set_page_directory` `:2975` |
+| 9 | `SET_PAGE_DIRECTORY` (54) | recorded for translation + SUCCESS (`:2975`) | **H** - recording stays **E**; forwarding to the real MMU is P6c and guest pages are GPU-DMA-reachable on the reference host (P6a status below; page-table pre-build required). P6b waves 1/1.5/2 execute eligible copies without forwarding page-directory state, through module-built scratch page tables, with deferred exactly-once release and hardware-authored data | `gsp_rm_handle_set_page_directory` `:2975` |
 | 10 | `UNSET_PAGE_DIRECTORY` (79) | record remove + SUCCESS (`:3063`) | **H** - same gate as #9 (real page-directory forwarding is P6c; reachability resolved, pre-build required) | `:3063` |
 | 11 | `UPDATE_BAR_PDE` (70) | SUCCESS ack (`:3144`) | **E** now; **L** only when BAR1 is backed by real VRAM | `:3144` |
 | 12 | `UNLOADING_GUEST_DRIVER` (47) | SUCCESS ack (`:3155`) | **E** - the guest's own shutdown; the real session's shutdown is host-driven on backend stop (`gsp_phys_gpu.c:909`) | `:3155` |
@@ -235,7 +242,7 @@ Legend: **L** = live backend, **E** = existing emulation, **H** = hybrid.
 | # | Guest-visible thing | Today | Target | Change point |
 |---|---|---|---|---|
 | 17 | GSP falcon boot regs (FWSEC/Booter/RISCV/WPR2), SW interrupt tree, MMU invalidate, L2 flushes | emulated auto-complete (`gsp_emulation.c:193-452`) | **E always** - the two GSP sessions are intentionally independent; guest boot must never depend on the physical session | `gsp_emu_poll_boot` `:193` |
-| 18 | Guest VF doorbell `0x88704` | software pump (`main.c:826` -> `gsp_sema_service_doorbell` `:6369`) | **H** - pump classifies each push: pure state/release stays software; eligible CE copies are shadow-forwarded to hardware (P5 host-buffer mode, P6b wave 1 guest operands) | `svc_pump_channels` `gsp_sema_service.c:5019`, `svc_decode_stream` `:2802` |
+| 18 | Guest VF doorbell `0x88704` | software pump (`main.c:826` -> `gsp_sema_service_doorbell` `:6369`) | **H** - pump classifies each push: pure state/release stays software; eligible CE copies are forwarded to hardware (P5 host-buffer mode; P6b waves 1/1.5/2 guest operands, deferred release, hardware-primary copies with a software fallback) | `svc_pump_channels` `gsp_sema_service.c:5019`, `svc_decode_stream` `:2802` |
 | 19 | BAR0/BAR1/BAR3 shadows, PRAMIN window | host-RAM shadows (`bar_emulation.c:180`, `main.c:1205`) | **E** until real VRAM forwarding (P6) | `nv_vgpu_bar_emul_*`, `nv_vgpu_vfio_dev_mmap` |
 | 20 | legacy doorbell/shared-mem RPC class (`GSP_RPC_*`) | internal emulation client thread (`gsp_emulation.c:458-620`, `:674-694`) | **E / disable when READY** - not part of the guest physical-path protocol; gate it off in backend mode to avoid intercepting stray writes | `gsp_emu_thread_fn` `:674` |
 
@@ -486,9 +493,10 @@ go-ahead, recovery plan, pstore, never unattended
   module-owned ladder copy; the software release path stays authoritative and
   no guest memory/VA is touched. Live: guest smoke GREEN `ALL OK` 64/64 with
   514 real CE copies (194 requests coalesced/dropped while one was in flight,
-  0 refusals, 0 timeouts). The P6b wave-1 guest-copy forwarding has since
-  landed (P6 status below). Wave 2 (hardware-primary release relay) and the
-  real guest-page-table path (P6c) remain.
+  0 refusals, 0 timeouts). The P6b waves 1/1.5/2 (guest-copy forwarding,
+  deferred exactly-once release, hardware-primary copies with a software
+  fallback) have since landed (P6 status below); the real guest-page-table
+  path (P6c) remains.
 * **Scope:** the debugfs ladder's real-submission half is **done** (cycle 25:
   channel allocates, page directory commits, `REAL COPY OK` on the physical
   CE in VA mode). The remaining work: bind each guest CE/GR channel to a real
@@ -514,8 +522,9 @@ go-ahead, recovery plan, pstore, never unattended
 * **Rollback:** per-channel `forward=0`; software pump is untouched.
 
 ### P6 - Guest memory into the real MMU (end state: guest submission on hardware)
-* **Status (2026-10-02): P6a guest-page reachability and P6b wave 1
-  (guest-copy forwarding, shadow mode) proven live.** P6a: the physical GPU's
+* **Status (2026-10-02): P6a guest-page reachability and P6b waves 1/1.5/2
+  (guest-copy forwarding, deferred release, hardware-primary copies) proven
+  live.** P6a: the physical GPU's
   IOMMU group is an identity domain on the reference host and the mediated
   device registers as an emulated-IOMMU VFIO device, so its type-1 container
   keeps guest DMA mappings privately and no kernel `iommu_domain` exists for
@@ -559,11 +568,39 @@ go-ahead, recovery plan, pstore, never unattended
   already applied the same bytes, so the hard evidence is acceptance of the
   translated-HPA submission, the semaphore release and host-side
   data-consistency verification (`verify_ok`); a late hardware write can
-  theoretically race guest reuse inside the submit window. **Remaining:**
-  wave 2 (hardware-primary release relay) and P6c (real page-directory
-  shadowing, real guest channels and push forwarding with guest page tables
-  in the real MMU) - guest-submission forwarding through the real guest page
-  tables is not implemented yet.
+  theoretically race guest reuse inside the submit window. **P6b wave 1.5
+  (deferred release, verified live):** with `gsp_phys_work_defer_release=1` a
+  forwarded push's releases are recorded instead of applied and its GPFIFO
+  entry is held until the hardware copy completes; the service thread then
+  delivers the releases once. Live: `engaged=75 applied=75 timeouts=0
+  canceled=0 overflow=0 pending=0`, the CUDA driver-API smoke stayed green
+  (`ALL OK` 64/64), releases exactly once, clean release. **P6b wave 2
+  (hardware-primary, verified live):** with `gsp_phys_work_hw_primary=1`
+  (plus the forwarding, guest-memory and deferred-release parameters) the
+  software copy is skipped for forwarded operations, so the physical engine
+  authors the data; the guest read it back correctly. Live:
+  `copy_skipped=119`, queued guest copies
+  `req=100 ok=100 verify_ok=100 verify_fail=0 fallback=0 timeouts=0`, full
+  smoke `matched=64/64` `ALL OK`. Nineteen skipped copies were
+  software-authored inline fallbacks when the single forward slot was busy
+  (`copy_soft=19 copy_fail=0`), the designed fail-closed path; a separate run
+  with the guest-RAM ranges unregistered proved that fallback again
+  (`copy_skipped=121`, recovered once each by the software copy,
+  `copy_fail=0`, smoke still green), so the software fallback remains for any
+  unqueued or failed copy. **Residual (P6a trigger path):**
+  the debug-trigger path pre-builds its page-table slots once during ladder
+  stage 3; a guest page's host physical address can change between
+  translations (observed `0x13a591000` then `0x3b8318000` for the same GPA),
+  and a runtime PTE update into an already-fetched 128-byte line is invisible,
+  so the later write-direction trigger failed its data check. Forwarding is
+  unaffected (it resolves HPAs at submit time); the follow-up is to give the
+  debug triggers fresh scratch lines (the forwarding path) or to pin the
+  translated page. **Remaining:** P6c - completion/fault relay, real
+  page-directory shadowing, real guest channels and push forwarding with
+  guest page tables in the real MMU, and `vectorAdd` correct results;
+  guest-submission forwarding through the real guest page tables is not
+  implemented yet. All new parameters default off; with them off the software
+  path is unchanged.
 * **Scope:** map guest page-table pages/records so the physical GPU can DMA
   them; forward `SET_PAGE_DIRECTORY` (`gsp_rm_emulation.c:2975`) to the real
   server; allocate real memory objects for guest allocations (fn4 with real
@@ -613,9 +650,10 @@ go-ahead, recovery plan, pstore, never unattended
    the only recovery.
 4. **Wrong-page-table / wrong-doorbell writes.** No forwarding seam may
    program guest-derived page tables into the real MMU until the P6c
-   page-table population path is proven end to end; the P6b wave-1 path
-   submits engine work only through module-built scratch page tables mapping
-   already-resolved host pages and never reuses a fetch line in a boot.
+   page-table population path is proven end to end; the P6b waves 1/1.5/2
+   path submits engine work only through module-built scratch page tables
+   mapping already-resolved host pages and never reuses a fetch line in a
+   boot.
    Guest-page reachability itself is resolved on the reference host (P6a),
    so wrong-page-table writes remain the top risk. Preserve: queue-header
    validation in `gsp_phys_queue_tx_submit()` (`booter:2983`), free-space