kernel: Fix GSP emulation shadow binding, eventfd wiring, RPC livelocks + docs suite + repo reorg

CRITICAL FIXES (guest driver bring-up):
- gsp_emulation.c: emulation engine bound to the WRONG BAR0 - it polled the
  host physical BAR0 (bars[0].vaddr ioremap) for the doorbell and read RPCs
  from a local dma_alloc_wc buffer, while the guest writes land in the
  vzalloc'd bar0_shadow. The RPC path could never work; PBA writes even
  corrupted the host GPU's physical BAR0+0x3000. gsp_emu_init now binds
  exclusively to bar0_shadow (doorbell/mailbox/PBA aliases;
  shared_mem_dma==0 marks the alias so cleanup never dma_free's it).
- main.c SET_IRQS: DATA_EVENTFD eventfds were DISCARDED ('TODO' + kfree) -
  the guest never received MSI-X completions. Full wiring now:
  eventfd_ctx_fdget per vector, fd -1 clears, dev_close releases refs.
  gsp_emu_inject_completion_interrupt looks up vgpu->irqs[0].trigger LIVE
  (init-time copy was permanently NULL since QEMU wires after gsp_emu_init).
- gsp_rpc_client.c: BAR_TRAP/MIGRATE in emulation mode called
  gsp_rpc_submit_raw which rings the SAME doorbell the thread polls -
  self-triggered RPC livelock + multi-second stalls. Now fail fast with
  -ENOTSUPP + pr_warn_ratelimited (guest BAR access is direct MMIO).

MORE FIXES:
- main.c: nv_vgpu_config_write returned SIZE_MAX (while(count--) wrap) and
  was dead code while the VFIO write handler did unfiltered byte copies
  (guest could rewrite vendor ID/BARs). Return fixed; VFIO write handler
  now routes through the per-register filter.
- kernel/Makefile: M=$(PWD) used the env var which is stale when invoked
  via the top-level make (pointed at repo root -> kbuild silently built
  NOTHING; explains every 'stale .ko' incident). Now M=$(CURDIR).
- Warnings sweep: header prototypes for mdev/type functions, format casts
  in gsp_comm.c DRY-RUN logs, unused variable.

ROBUSTNESS (harness/tooling):
- vgpu_vm.py: run had no timeout (start could block forever);
  guest_shell recv could block forever on a wedged guest. Both now bounded;
  guest_shell pre-flights guest-ping and reports wedge diagnostics.
- stack_manager.sh: qemu/qemu-debug now launch via the vgpu_vm.py harness
  (workflow rule: never raw QEMU).

REPO REORGANIZATION:
- driver/ (legacy guest-side gsp-vgpu.ko) moved to legacy-driver-guest/
  (reference only, not built; README explains). 3 legacy un-buildable test
  files moved to legacy-driver-guest/kernel-tests/. Top-level Makefile
  cleaned of the dead DRIVER_DIR reference.

DOCUMENTATION SUITE (new):
- docs/GSP_EMULATION_ARCHITECTURE.md - emulator deep-dive: shadow BAR0
  model, doorbell protocol, dispatch table, MSI-X path, lifecycle, memory-
  model caveats
- docs/DEBUGGING.md - QEMU GDB stub workflow, serial console, QMP NMI,
  failure taxonomy, full bring-up validation checklist, EPT-livelock case
  study, version-matrix discipline
- docs/ONBOARDING.md - repo tour, coding standards (incl. virt_to_phys /
  dma_addr_t / get-put rules), first build & test
- docs/VERSION_UPGRADE.md - end-to-end new-driver-version procedure
  (firmware fetch, Ghidra analysis, configs, signatures, validation)
- docs/PERFORMANCE.md - bare-metal A/B methodology + targets
- README.md doc index; STATE.md refreshed; DEVELOPMENT_WORKFLOW.md corrected
  (load order: open-iov-vfio BEFORE nvidia; stock-driver blacklist note)

1 files changed, 33 insertions, 11 deletions

diff --git a/STATE.md b/STATE.md
index dc81f80..73a8654 100644
--- a/STATE.md
+++ b/STATE.md
@@ -1,6 +1,6 @@
 # Open-IOV Project State
 
-*Last updated: 2026-09-21*
+*Last updated: 2026-09-21 (PM)*
 
 ## Mission
 
@@ -21,11 +21,14 @@ proprietary GRID licenses — via host-side GSP emulation in
 | Refactor 5: Guest driver integration (mdev creation, VFIO registration) | ✅ Unblocked, in test |
 | Refactor 6-11: CUDA / graphics / benchmarks / migration / Windows / hardening | ⏳ Pending |
 
-**Right now**: host stack (nvidia.ko 610.57.04-base + open-iov-vfio.ko) loads
-cleanly, mdev device creates and removes cleanly, IOMMU group assignment
-works, `/dev/vfio/<n>` appears. Guest VM (Ubuntu 24.04) is installed; guest
-NVIDIA driver installation is in progress. Next: guest `nvidia.ko` probe
-against the emulated vGPU, then `nvidia-smi`, then CUDA validation.
+**Right now**: the **guest-probe hang root cause is found and fixed** (VFIO
+mmap garbage PFN → KVM EPT livelock — see below) plus a sweep of correctness
+fixes (mdev remove leaks, eventfd wiring, GSP shadow binding, RPC livelock).
+Guest driver **570.144 stock runfile installed** in the VM. The fixed module
+is built and pushed; it loads after the next host reboot (a leaked
+`gsp_emu` kthread from the pre-fix remove path pins the old module — rmmod
+would oops). Post-reboot: guest `nvidia.ko` probe → `nvidia-smi` → teardown
+regression → unit test → CUDA validation (`docs/DEBUGGING.md` §5 checklist).
 
 ## Recently solved bugs
 
@@ -38,24 +41,43 @@ against the emulated vGPU, then `nvidia-smi`, then CUDA validation.
 | `Unknown symbol nvidia_vgpu_vfio_probe` | NVIDIA OpenRM built without `KBUILD_EXTRA_SYMBOLS` | Build order automated in `stack_manager.sh` (kernel → nvidia → userspace) |
 | `vfio 1: No such file or directory` | VFIO device registration is NOT automatic for mdev | `vfio_register_emulated_iommu_dev()` in probe, unregister in remove (Intel kvmgt pattern) |
 | Guest-visible PCI config space wrong | BAR types/offsets, subsystem ID offsets, MSI-X table size | Config space rewritten to match physical TU104 (BAR0 16MB 32-bit non-prefetch, BAR1 256MB 64-bit prefetch VRAM, BAR3 32MB; 32 MSI-X vectors; subsystem IDs at 0x2C/0x2E) |
+| **Guest driver probe hard-hangs whole VM (580 & 570.144)** | VFIO `mmap` used `virt_to_phys()` on `vzalloc`'d BAR0 shadow + ioremap'd BAR2/3 → garbage PFN → KVM `make_spte` reserved-bit WARN → EPT-violation livelock (diagnosed via QEMU GDB stub: one vCPU frozen on a single MMIO instruction) | BAR0 mapped per-page via `vmalloc_to_pfn()`; BAR2/3/4/5 map `bars[N].gpa` directly; `nv_vgpu_mmap_virt_to_pfn()` helper |
+| **GSP emulation thread never saw guest RPCs** | Engine polled the *host physical* BAR0 doorbell + a local `dma_alloc_wc` buffer; guest writes land in the vzalloc `bar0_shadow` | `gsp_emu_init` binds exclusively to `bar0_shadow` (doorbell/mailbox/PBA aliases); also stopped PBA writes corrupting the host GPU's physical BAR0+0x3000 |
+| **Guest never received MSI-X completions** | `SET_IRQS DATA_EVENTFD` handler discarded eventfds (`TODO` + `kfree`); `msix_trigger[]` all NULL; inject used an init-time (stale) copy | Full eventfd wiring: `eventfd_ctx_fdget` per vector, `-1` clears, close releases refs; inject looks up `vgpu->irqs[0].trigger` live |
+| **BAR_TRAP/MIGRATE RPC livelock** | dispatch → `gsp_rpc_submit_raw` rings the same doorbell the thread polls → self-triggered infinite loop + 2–30 s stalls | emulation-mode guards return `-ENOTSUPP` with `pr_warn_ratelimited` |
+| **mdev remove leaked everything** | remove never `list_del`'d (available_instances stuck), never ran cleanup (leaked `gsp_emu` kthread), never unregistered VFIO (leaked `/dev/vfio/N` + iommu group), never put pdev/freed struct | kvmgt-style remove: wait fds → `vfio_unregister_group_dev` → cleanup work → `list_del` → `pci_dev_put` → `vfio_put_device` |
+| `dma_map_page` IOVA corruption | 64-bit `dma_addr_t` stored in `int ret` (truncates ≥4 GB IOVAs under IOMMU) | dedicated `dma_addr_t iova` in `gsp_rpc_alloc_emul` |
+| config-space write returned `-1` (`SIZE_MAX`) | `while(count--)` wrap + bogus return expression; function was dead code while the live path did unfiltered byte copies | return fixed; VFIO write handler now routes through the per-register filter (guest can't rewrite vendor ID/BARs) |
+| Misleading `MODULE_SOFTDEP("pre: nvidia")` | module has zero nvidia imports (it *exports* `nvidia_vgpu_vfio_*`) | removed; load order is open-iov-vfio **before** nvidia.ko |
+| Kernel unit tests didn't build | written against a removed struct-based RPC API | rewritten to drive `gsp_emu_process_rpc()` with crafted mailbox messages; `depends: open-iov-vfio` |
 
 ## Known open bugs / limitations
 
-- Guest `nvidia.ko` end-to-end probe not yet validated (in progress)
-- Kernel test modules (`kernel/tests/`) need API updates to match final
-  driver headers (low priority)
+- Guest `nvidia.ko` end-to-end probe not yet validated (fixed module loads
+  post-reboot; checklist in `docs/DEBUGGING.md` §5)
+- **Memory-model gap**: guest VRAM (BAR1) maps directly to physical BAR1 gpa,
+  but `gsp_rpc_alloc_emul` services ALLOC with host system pages — the two
+  are not unified; must be resolved before CUDA/DMA-heavy workloads
+  (`docs/GSP_EMULATION_ARCHITECTURE.md` §3)
+- BAR_TRAP / MIGRATE RPCs fail fast with `-ENOTSUPP` in emulation mode
+  (guest's normal BAR access is direct MMIO; real handlers TODO)
+- `gsp_emu_process_rpc()` doesn't bump stats counters (only the kthread
+  path does) — cosmetic
 - WPR callback registration lacks kernel eventfd support
-  (`TODO` in driver/wpr)
+  (`TODO` in driver/wpr — legacy guest-driver tree)
 - Migration WPR state save/restore incomplete
 - BAR `force_access` path not fully wired (unused-function warning)
 - Kernel `SCHEDULE` emulation returns success without real work submission
   (userspace emulation works)
 - Async `NOTIFY` events not emulated in kernel (no eventfd/poll yet)
-- `open-iov-vfio.ko` requires `nvidia.ko` loaded first (`MODULE_SOFTDEP`)
 - Single GPU validated so far: TU104 (RTX 2080 SUPER Mobile), GSP 570.144,
   kernel 6.8.0-139-generic
 - Live migration protocol implemented (4-phase) but not yet validated
 - Windows guest not yet validated (automation scaffolding present)
+- MSI-X emulation has dual stores (guest-visible `bar0_shadow` table vs
+  `bar_emul_ctx.msix_table[]` used by the currently-unused trap path) —
+  documented in `docs/GSP_EMULATION_ARCHITECTURE.md`; harmless with the
+  direct-mmap design
 
 ## Reproducibility notes