diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 14:10:23 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 14:10:23 -0400 |
| commit | 0ced423a01e85b98fe0d6db7bf7c75300c900316 (patch) | |
| tree | d8d8300c5c317f04d6b95e95f758eeb0169d86f5 | |
| parent | 92e45de48d2465594d4619be11b28581020e9eea (diff) | |
| download | open-iov-0ced423a01e85b98fe0d6db7bf7c75300c900316.tar.gz open-iov-0ced423a01e85b98fe0d6db7bf7c75300c900316.zip | |
docs(review): fix GMMU-doc anchors and vEP vocabulary; index docs 13/14 in the READMEs
The GMMU doc cited pre-c33f6006 lines for the pump, doorbell and svc_sem_write,
defined vEP differently from doc 13 (which it names as the vocabulary
authority), and called Design A prototypable in the 1.0 programme while M6
classifies the same prototype as post-1.0 research. Fix all three, refresh
the status hash, and add the doc to the docs/ and wiki/ indexes with the
renumbered reading-order rows.
| -rw-r--r-- | docs/03-architecture/14-multi-tenant-gmmu-protection.md | 36 | ||||
| -rw-r--r-- | docs/README.md | 35 | ||||
| -rw-r--r-- | wiki/GMMU-Multi-Tenant-Protection.md | 6 | ||||
| -rw-r--r-- | wiki/README.md | 1 |
4 files changed, 46 insertions, 32 deletions
diff --git a/docs/03-architecture/14-multi-tenant-gmmu-protection.md b/docs/03-architecture/14-multi-tenant-gmmu-protection.md index c3f58c7..060a600 100644 --- a/docs/03-architecture/14-multi-tenant-gmmu-protection.md +++ b/docs/03-architecture/14-multi-tenant-gmmu-protection.md @@ -1,6 +1,6 @@ # Multi-Tenant GMMU Protection: Isolation Design and Paged-Attention Controls -> **Status: design (2026-10-08, master `66388560`).** This document specifies +> **Status: design (2026-10-08, master `1f571dda`).** This document specifies > how Open-IOV isolates multiple tenants — multiple mdevs *and* multiple > tenants inside one guest — at the GPU memory-management unit (GMMU), and how > a paged-attention inference stack (vLLM-style) can use those protections. @@ -119,9 +119,9 @@ The **work/completion service is not**. `gsp_sema_service.c` holds one global `static struct sema_svc svc;` (`kernel/gsp_sema_service.c:676`), and: * the service thread pumps **the first open mdev it finds** - (`kernel/gsp_sema_service.c:13296-13308`); + (`kernel/gsp_sema_service.c:13352-13364`); * the work-submission doorbell carries no owner — `gsp_sema_service_doorbell(u32 token)` - (`kernel/gsp_sema_service.c:13456`), called from the VFIO write path with + (`kernel/gsp_sema_service.c:13512`), called from the VFIO write path with no mdev argument (`kernel/main.c:865`); * channel registrations are keyed by the guest-chosen `h_object` alone (`kernel/gsp_sema_service.c:847`, table bounded by `SVC_MAX_CHANNELS 128` @@ -146,9 +146,11 @@ problem.** §2 fixes the ownership; §4 builds the GMMU policy on it. **Tenant** — isolation principal (`tenantId`): a VM/mdev at L0–L2, a logical serving principal inside a guest in §4. **Agent** — finest principal inside a tenant (`agentId`): a vLLM engine/worker, a CUDA context, a process. -**vEP** — virtual execution plane: the per-instance mediator (guest -emulation + work execution) for one mdev; see -[`13-vexecution-plane.md`](13-vexecution-plane.md). **KV block** — +**vEP** — the vGSP Execution Plane: the execution half of vGSP (guest work → +physical execution → guest-visible completion), as defined in +[`13-vexecution-plane.md`](13-vexecution-plane.md). This document's problem +is that the vEP's work-service state is currently one global context rather +than per-mdev. **KV block** — paged-attention cache page (vLLM `paged_attention_v2`: fixed-size blocks, one `block_tables` entry per logical block per sequence); a GPU allocation range, not a hardware object. @@ -171,9 +173,9 @@ What exists and what changes: | `svc.chan[]` | keyed `h_object` (`gsp_sema_service.c:847`) | key `(vgpu, h_object)`; per-mdev lifetime reset | | `svc.vas[]` | keyed `h_vaspace` (`:988`) | key `(vgpu, h_vaspace)` | | learned VA→PA map / pins | keyed GPU VA (`:113-130`) | key `(vgpu, VA)`; per-mdev epoch | -| doorbell token | ownerless (`:13456`, `main.c:865`) | `(vgpu, token)`; route to the owner's pump | +| doorbell token | ownerless (`:13512`, `main.c:865`) | `(vgpu, token)`; route to the owner's pump | | completion counters | global boot state | per mdev, reset on session/rebind | -| pump | first open mdev (`:13296-13308`) | per-mdev pump (or round-robin over all open mdevs) | +| pump | first open mdev (`:13352-13364`) | per-mdev pump (or round-robin over all open mdevs) | The GMMU half of address-space separation is the page directory: each VA space has one, published by the guest through `SET_PAGE_DIRECTORY` with its @@ -258,7 +260,7 @@ mdev's address space, and a completion event is posted only to the owning mdev's event object. Today the release path resolves the target VA through *global* tables -(`svc_resolve_va`, `svc_sem_write`; `kernel/gsp_sema_service.c:2098,1514`) +(`svc_resolve_va`, `svc_sem_write`; `kernel/gsp_sema_service.c:2098,2474`) and posts through `gsp_rm_post_channel_event()` on the emulation context (`kernel/gsp_rm_emulation.c:602`). The first is the cross-tenant hazard: a release record decoded from guest A's push buffer could resolve through a @@ -581,8 +583,10 @@ The requested chain, end to end: ### 4.4 Design A — VA-subtree partitioning (no guest KMD change) -This is the **stock-compatible** design and the one that can be prototyped -inside the 1.0 programme. It relies on one idea: **the VA space is the +This is the **stock-compatible** design and the research entry point: it is +the only design that needs no guest KMD change, but its prototype is M6 — +post-1.0, dual-gated, and it does not shape baseline 1.0 code (§7.4). It +relies on one idea: **the VA space is the security domain, and a VA is self-describing because its prefix encodes its owner.** @@ -1090,6 +1094,10 @@ starts after 1.0 goals are met, on an explicit human decision. CUDA reproducibility and per-client state issues M1 resolves; `../../debug/risc-v-gsp-rm-vm/docs/{43-GMMU-PTE-DECODER,44-GMMU-ORACLE-AND-WORKFLOW,23-RM610-READINESS}.md` — tooling and gates cited above. -* Local analysis: `LOCAL-MULTI-TENANT-GMMU.md` (adjacent; part of the - local analysis) — provenance behind §4.2 and the unredacted design - reasoning. + +## Change log + +| Date | Change | +|---|---| +| 2026-10-08 | Created: multi-mdev service-state ownership rules, isolation-layer model, VRAM oversubscription design, paged-attention tenant/agent protection stack (Design A/B, three granularities), walked example, portability, M1–M8 roadmap. Tree `66388560`. | +| 2026-10-08 | Review pass: status hash refreshed to `master @ 1f571dda`; drifted line anchors updated for the post-`66388560` code (pump loop, doorbell, `svc_sem_write`); the vEP vocabulary aligned with doc 13 (13 is the vocabulary authority, this document the GMMU-policy authority); Design A classified as the research entry point (M6), not a 1.0 programme item. | diff --git a/docs/README.md b/docs/README.md index fe39077..13f0ae5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -173,20 +173,21 @@ Read in order if you are new. Jump directly if you know what you need. | 13 | [Physical GPU integration](03-architecture/08-physical-gpu-integration.md) | developers, architects | 20 min | | 14 | [Physical backend guest wiring](03-architecture/11-physical-backend-guest-wiring.md) | developers, architects | 25 min | | 15 | [The vGSP execution plane (vEP)](03-architecture/13-vexecution-plane.md) | developers, architects | 25 min | -| 16 | [The execution frontier: the GR compute-launch chain](04-implementation/03-execution-frontier.md) | developers, debuggers | 15 min | -| 17 | [Host VF acceleration — future goal (deferred)](03-architecture/12-host-vf-acceleration-future-goal.md) | architects, planners | 10 min | -| 18 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min | -| 19 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min | -| 20 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min | -| 21 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min | -| 22 | [GSP in a RISC-V VM (risc-v-gsp-rm-debugger, experimental)](05-operations/03-risc-v-gsp-rm-vm.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min | -| 23 | [risc-v-gsp-rm-debugger user guide (experimental)](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min | -| 24 | [Defensive Operations Runbook](RUNBOOK.md) | maintainers, operators | 20 min | -| 25 | [Engineering operations](eng-ops.md) — tool admission and workflows (Part I) plus live physical-backend operations (Part II) | maintainers, developers | 20 min | -| 26 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min | -| 27 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min | -| 28 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min | -| 29 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference | +| 16 | [Multi-tenant GMMU protection](03-architecture/14-multi-tenant-gmmu-protection.md) | developers, architects, security reviewers | 25 min | +| 17 | [The execution frontier: the GR compute-launch chain](04-implementation/03-execution-frontier.md) | developers, debuggers | 15 min | +| 18 | [Host VF acceleration — future goal (deferred)](03-architecture/12-host-vf-acceleration-future-goal.md) | architects, planners | 10 min | +| 19 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min | +| 20 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min | +| 21 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min | +| 22 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min | +| 23 | [GSP in a RISC-V VM (risc-v-gsp-rm-debugger, experimental)](05-operations/03-risc-v-gsp-rm-vm.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min | +| 24 | [risc-v-gsp-rm-debugger user guide (experimental)](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min | +| 25 | [Defensive Operations Runbook](RUNBOOK.md) | maintainers, operators | 20 min | +| 26 | [Engineering operations](eng-ops.md) — tool admission and workflows (Part I) plus live physical-backend operations (Part II) | maintainers, developers | 20 min | +| 27 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min | +| 28 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min | +| 29 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min | +| 30 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference | <!-- change (2026-10-02): insert the risc-v-gsp-rm-debugger user guide and the Open-IOV Defensive Operations Runbook into the reading list; renumber rows 21-24 to @@ -202,7 +203,11 @@ additive only. --> <!-- change (2026-10-08): insert "The vGSP execution plane (vEP)" as row 15 (architecture: the work path); renumber rows 15-28 to 16-29; additive only. --> -Row 17 is a **deferred future goal**: host NVIDIA-driver acceleration over a +<!-- change (2026-10-08): insert "Multi-tenant GMMU protection" as row 16 +(architecture: isolation design, companion to the vEP doc); renumber rows +16-29 to 17-30; additive only. --> + +Row 18 is a **deferred future goal**: host NVIDIA-driver acceleration over a virtual function. The feasibility verdict and a first design sketch are recorded there for planning; it is not on the active critical path (guest CUDA execution, multi-VM, 3D, performance come first). diff --git a/wiki/GMMU-Multi-Tenant-Protection.md b/wiki/GMMU-Multi-Tenant-Protection.md index 83b00ba..6ea5b9b 100644 --- a/wiki/GMMU-Multi-Tenant-Protection.md +++ b/wiki/GMMU-Multi-Tenant-Protection.md @@ -15,7 +15,7 @@ two halves: 1. **Service-state ownership (baseline 1.0).** The work/completion service (`kernel/gsp_sema_service.c`) is one global context (`static struct sema_svc svc`, `:676`), the pump services the first open mdev - (`:13296-13308`), and channels/VA spaces/translations are keyed by + (`:13352-13364`), and channels/VA spaces/translations are keyed by guest-chosen values alone (`:847`, `:988`, `:113-130`). Two guests can pick the same handles and VAs (live: channel `0xcaf000d4`; `../debug/issue/issue-number-2.md`), so owner-keying is a prerequisite @@ -76,12 +76,12 @@ periodic owner-vs-PTE audit with canaries catches escapes. If only per-tenant separation exists, the deployable contract is **one agent per tenant, one tenant per context** (canonical §5.4). -## Key code anchors (master `66388560`) +## Key code anchors (master `1f571dda`) | What | Where | |---|---| | per-mdev emulation context | `kernel/include/nvidia-vgpu-vfio.h:133`, `kernel/gsp_emulation.h:130` | -| global service state | `kernel/gsp_sema_service.c:676`; pump `:13296`; doorbell `:13456` | +| global service state | `kernel/gsp_sema_service.c:676`; pump `:13352`; doorbell `:13512` | | channel/vaspace registries | `kernel/gsp_sema_service.c:847`, `:988` | | GMMU walk (VER2) | `kernel/gsp_sema_service.c:1174`; decode `:1093` | | page directory record | `kernel/gsp_rm_emulation.c:3320`; reserved PDEs `:2237-2262` | diff --git a/wiki/README.md b/wiki/README.md index 378ea31..d04f70f 100644 --- a/wiki/README.md +++ b/wiki/README.md @@ -10,6 +10,7 @@ ### Core Analysis - [The vGSP Execution Plane (vEP)](Execution-Plane.md) - The work/execution layer of `vgsp.ko`: guest pushes to physical execution and guest-visible completion (condensed; canonical doc [`../docs/03-architecture/13-vexecution-plane.md`](../docs/03-architecture/13-vexecution-plane.md)) +- [GMMU Multi-Tenant Protection](GMMU-Multi-Tenant-Protection.md) - Multi-mdev isolation design and paged-attention (vLLM) tenant/agent protection at the GMMU: service-state ownership, per-context page directories, VRAM oversubscription (condensed; canonical doc [`../docs/03-architecture/14-multi-tenant-gmmu-protection.md`](../docs/03-architecture/14-multi-tenant-gmmu-protection.md)) - [GSP 570.144W Firmware Analysis](GSP-570.144W-Firmware-Analysis.md) - Firmware decompilation and address record - [Address Reference](Address-Reference.md) - Firmware-side address tables - [CUDA Userspace Driver](CUDA-Userspace-Driver.md) - `libcuda.so` deep dive: layer model, guest interface, forwarding status, crash-path lesson and lookup tooling |
