diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-02 03:24:35 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-02 03:31:06 -0400 |
| commit | d2de7bf1d6bb7d0931085f87da6c3013cd7f1aca (patch) | |
| tree | dbdb313ac24364de8b325ed96fdb75567b550832 /docs | |
| parent | 0054d3acbc3017d2a535239f8153d3c99f7abc18 (diff) | |
| download | open-iov-d2de7bf1d6bb7d0931085f87da6c3013cd7f1aca.tar.gz open-iov-d2de7bf1d6bb7d0931085f87da6c3013cd7f1aca.zip | |
gsp-phys: P6a - real copy engine reads/writes guest RAM pages
- file-backed guest RAM (launcher memory-backend-file share=on) and
in-kernel GPA->HPA translation through the shared file page cache
(read_cache_page + page_to_phys + pfn_valid), opened O_LARGEFILE;
new default-off gsp_phys_guest_mem parameter
- debugfs gsp-phys-guest-ram (path + gpa/off/size ranges) and
gsp-phys-guest-gpa (one pinned GPA per boot); rm-call triggers 6/7 run
host->guest and guest->host 4 KiB CE copies; trigger 8 is a PTE
self-test through a fresh slot
- guest page-table slots are pre-built during ladder stage 3: runtime
PTE writes into an already-walked page-table line are not observed
without a TLB invalidate
- guest RAM layout (q35 4.2, 4 GiB): below-4G GPA == file offset,
above-4G offset = GPA - 0x80000000 (2 GiB PCI hole)
- tests: tools/test/gpage_test.c guest probe; docs + STATE-CURRENT
record the live green (G2H/W2G with semaphore release, guest observed
the write, smoke ALL OK 64/64, clean latched release)
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/03-architecture/11-physical-backend-guest-wiring.md | 48 | ||||
| -rw-r--r-- | docs/04-implementation/01-known-knowns-and-unknowns.md | 15 |
2 files changed, 54 insertions, 9 deletions
diff --git a/docs/03-architecture/11-physical-backend-guest-wiring.md b/docs/03-architecture/11-physical-backend-guest-wiring.md index 8c38680..7db80dc 100644 --- a/docs/03-architecture/11-physical-backend-guest-wiring.md +++ b/docs/03-architecture/11-physical-backend-guest-wiring.md @@ -1,6 +1,7 @@ # Physical Backend Guest Wiring: Driving the Guest-Facing Emulation from the Live GSP-RM Session -> **Status: P0-P5(w1) implemented (2026-10-02).** The backend API skeleton +> **Status: P0-P5(w1) implemented; P6a guest-page reachability proven live +> (2026-10-02).** The backend API skeleton > (P0), the mdev-owned lifecycle (P1, fail-open; the async FWSEC boot on that > path is an open defect off the critical path), the static-config identity > overlay (P2, verified live in an A/B run), the read-only control @@ -211,8 +212,8 @@ Legend: **L** = live backend, **E** = existing emulation, **H** = hybrid. | 6 | `GSP_RM_ALLOC` (103) | echo request + `NV_OK`; channels/events registered locally (`:2322`) | **H** - echo for emulation-only classes; forward selected classes (root/device/subdevice, then channel/memory) once a guest<->server handle map exists; always keep local side effects | `gsp_rm_handle_rm_alloc` `:2322` | | 7 | `GSP_RM_CONTROL` (76) | ~25 static per-cmd handlers; unknown -> zeroed success (`:2611`, default `:2807`) | **H** - whitelist of side-effect-free queries goes live with emulation fallback; vGPU-specific controls stay **E** (`COPY_SERVER_RESERVED_PDES` `:2647`, `PROMOTE_FAULT_METHOD_BUFFERS` `:2654`) | `gsp_rm_handle_rm_control` `:2611`; new forward helper before the switch | | 8 | `GSP_RM_FREE` (10) | echo + retire channel/events (`:2882`) | **H** - forward when the object was backend-allocated; always retire local bookkeeping | `gsp_rm_handle_free` `:2882` | -| 9 | `SET_PAGE_DIRECTORY` (54) | recorded for translation + SUCCESS (`:2975`) | **H** - recording stays **E**; forwarding to the real MMU is P6 and gated on guest pages being GPU-DMA-reachable | `gsp_rm_handle_set_page_directory` `:2975` | -| 10 | `UNSET_PAGE_DIRECTORY` (79) | record remove + SUCCESS (`:3063`) | **H** - same gate as #9 | `:3063` | +| 9 | `SET_PAGE_DIRECTORY` (54) | recorded for translation + SUCCESS (`:2975`) | **H** - recording stays **E**; forwarding to the real MMU is P6b and guest pages are GPU-DMA-reachable on the reference host (P6a status below; page-table pre-build required) | `gsp_rm_handle_set_page_directory` `:2975` | +| 10 | `UNSET_PAGE_DIRECTORY` (79) | record remove + SUCCESS (`:3063`) | **H** - same gate as #9 (reachability resolved; pre-build required) | `:3063` | | 11 | `UPDATE_BAR_PDE` (70) | SUCCESS ack (`:3144`) | **E** now; **L** only when BAR1 is backed by real VRAM | `:3144` | | 12 | `UNLOADING_GUEST_DRIVER` (47) | SUCCESS ack (`:3155`) | **E** - the guest's own shutdown; the real session's shutdown is host-driven on backend stop (`gsp_phys_gpu.c:909`) | `:3155` | | 13 | default/unknown | generic SUCCESS (`:3169`) | **E** (fail-safe: never invent a live forward for an unknown function) | `:3169` | @@ -508,6 +509,32 @@ go-ahead, recovery plan, pstore, never unattended * **Rollback:** per-channel `forward=0`; software pump is untouched. ### P6 - Guest memory into the real MMU (end state: guest submission on hardware) +* **Status (2026-10-02): P6a - guest-page reachability proven live.** The + physical GPU's IOMMU group is an identity domain on the reference host and + the mediated device registers as an emulated-IOMMU VFIO device, so its + type-1 container keeps guest DMA mappings privately and no kernel + `iommu_domain` exists for it: `iommu_iova_to_phys` cannot translate guest + GPAs. Guest RAM is therefore backed by a QEMU shared file + (`memory-backend-file,share=yes,prealloc=on`); the module opens it and + translates GPA->HPA through the file page cache (`read_cache_page` + + `page_to_phys` + `pfn_valid`) behind the active-mdev gate, under the new + default-off `gsp_phys_guest_mem` parameter and write-only + `gsp-phys-guest-ram`/`gsp-phys-guest-gpa` debugfs controls (ladder trigger + cases 6 host->guest write, 7 guest->host read, 8 fresh-slot PTE self-test). + On the reference q35 4 GiB guest, GPA equals file offset below 4 GiB and + offset = GPA - 0x80000000 above it (2 GiB 32-bit PCI hole). **Pre-build + requirement:** channel page-table slots for the guest window are built in + ladder stage 3 before any engine access; a runtime PTE write into a line + the GMMU had already walked was accepted but never completed (no + TLB-invalidate path yet). **Live:** the physical copy engine copied 4 KiB + from a real guest RAM page into a module buffer (G2H) and back into the + same page (W2G), semaphore released; the guest process observed the write + in its own page; the driver-API smoke stayed green (`ALL OK` 64/64) and + release stayed latched (distribution driver rebound, protected region down, + exactly 5 power-domain locks intentionally held, 0 rail cycles, no reboot); + the informational PTE self-test passed. **Remaining (P6b):** page-directory + shadowing, real guest channels and push forwarding - guest-submission + forwarding is not implemented yet. * **Scope:** map guest page-table pages/records so the physical GPU can DMA them; forward `SET_PAGE_DIRECTORY` (`gsp_rm_emulation.c:2975`) to the real server; allocate real memory objects for guest allocations (fn4 with real @@ -520,10 +547,11 @@ go-ahead, recovery plan, pstore, never unattended copy with semaphore; second mdev is refused/emulated (single-session isolation until per-mdev clients exist); fault test. * **Risk:** wrong page tables -> engine writes to arbitrary host memory; - IOMMU/DMA reachability of guest pages is the project's existing - known-unknown (`docs/04-implementation/01-known-knowns-and-unknowns.md` - §2.2) and is a prerequisite, not solved by this design. Keep - `gsp_phys_seq_addr_valid()` (`booter:1800`) and all queue validations. + guest-page reachability is **resolved on the reference host** (P6a status + above; §2.2 of `docs/04-implementation/01-known-knowns-and-unknowns.md` now + carries the resolution note), so wrong-page-table writes remain the top + risk. Keep `gsp_phys_seq_addr_valid()` (`booter:1800`) and all queue + validations. * **Rollback:** per-mdev isolation not yet available; disable forwarding and fall back to P5 host-buffer mode. @@ -555,8 +583,10 @@ go-ahead, recovery plan, pstore, never unattended no object teardown (`:5123` comments); `UNLOADING_GUEST_DRIVER` + rmmod is the only recovery. 4. **Wrong-page-table / wrong-doorbell writes.** No forwarding seam may - submit engine work or program page tables until P6's reachability - prerequisite holds. Preserve: queue header validation in + submit engine work or program page tables until the P6 page-table + population path is proven end to end; guest-page reachability itself is + resolved on the reference host (P6a), so wrong-page-table writes remain + the top risk. Preserve: queue header validation in `gsp_phys_queue_tx_submit()` (`booter:2983`), free-space check (`:3032`), status-record validation (`booter:2554`), sequencer address validation (`booter:1800`), stage-4 token/bounds guards (`booter:4969-4999`). diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md index 29558b0..1a5967b 100644 --- a/docs/04-implementation/01-known-knowns-and-unknowns.md +++ b/docs/04-implementation/01-known-knowns-and-unknowns.md @@ -129,6 +129,21 @@ running its own validation paths (UVM page-tree init exercises this). **Cost:** days to weeks. +**Reachability resolution note (2026-10-02):** the related IOMMU/DMA +reachability question is closed on the reference host. The physical GPU's +IOMMU group is an identity domain (all-RAM identity) and the mediated device +registers as an emulated-IOMMU VFIO device, so its type-1 container holds +guest DMA mappings privately and no kernel `iommu_domain` exists for it: +`iommu_iova_to_phys` cannot translate guest GPAs. Guest RAM is backed by a +shared file instead, and the module translates GPA->HPA through that file's +page cache. A channel page-table line the GMMU has already walked cannot be +updated at runtime (no TLB-invalidate path exists), so channel page-table +slots for the guest window are pre-built before any engine access. Live proof: +the physical copy engine copied 4 KiB from a real guest page into a module +buffer and back, semaphore released, with the guest process observing the +write and the driver-API smoke green (`ALL OK` 64/64). Wrong-page-table writes +remain the top risk; the division-of-labour question above is unchanged. + ### 2.3 Fault semantics **Unknown:** fault buffer formats and replay/cancel semantics as the *guest* |
