summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-08 14:10:23 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-08 14:10:23 -0400
commit0ced423a01e85b98fe0d6db7bf7c75300c900316 (patch)
treed8d8300c5c317f04d6b95e95f758eeb0169d86f5
parent92e45de48d2465594d4619be11b28581020e9eea (diff)
downloadopen-iov-0ced423a01e85b98fe0d6db7bf7c75300c900316.tar.gz
open-iov-0ced423a01e85b98fe0d6db7bf7c75300c900316.zip
docs(review): fix GMMU-doc anchors and vEP vocabulary; index docs 13/14 in the READMEs
The GMMU doc cited pre-c33f6006 lines for the pump, doorbell and svc_sem_write, defined vEP differently from doc 13 (which it names as the vocabulary authority), and called Design A prototypable in the 1.0 programme while M6 classifies the same prototype as post-1.0 research. Fix all three, refresh the status hash, and add the doc to the docs/ and wiki/ indexes with the renumbered reading-order rows.
-rw-r--r--docs/03-architecture/14-multi-tenant-gmmu-protection.md36
-rw-r--r--docs/README.md35
-rw-r--r--wiki/GMMU-Multi-Tenant-Protection.md6
-rw-r--r--wiki/README.md1
4 files changed, 46 insertions, 32 deletions
diff --git a/docs/03-architecture/14-multi-tenant-gmmu-protection.md b/docs/03-architecture/14-multi-tenant-gmmu-protection.md
index c3f58c7..060a600 100644
--- a/docs/03-architecture/14-multi-tenant-gmmu-protection.md
+++ b/docs/03-architecture/14-multi-tenant-gmmu-protection.md
@@ -1,6 +1,6 @@
# Multi-Tenant GMMU Protection: Isolation Design and Paged-Attention Controls
-> **Status: design (2026-10-08, master `66388560`).** This document specifies
+> **Status: design (2026-10-08, master `1f571dda`).** This document specifies
> how Open-IOV isolates multiple tenants — multiple mdevs *and* multiple
> tenants inside one guest — at the GPU memory-management unit (GMMU), and how
> a paged-attention inference stack (vLLM-style) can use those protections.
@@ -119,9 +119,9 @@ The **work/completion service is not**. `gsp_sema_service.c` holds one
global `static struct sema_svc svc;` (`kernel/gsp_sema_service.c:676`), and:
* the service thread pumps **the first open mdev it finds**
- (`kernel/gsp_sema_service.c:13296-13308`);
+ (`kernel/gsp_sema_service.c:13352-13364`);
* the work-submission doorbell carries no owner — `gsp_sema_service_doorbell(u32 token)`
- (`kernel/gsp_sema_service.c:13456`), called from the VFIO write path with
+ (`kernel/gsp_sema_service.c:13512`), called from the VFIO write path with
no mdev argument (`kernel/main.c:865`);
* channel registrations are keyed by the guest-chosen `h_object` alone
(`kernel/gsp_sema_service.c:847`, table bounded by `SVC_MAX_CHANNELS 128`
@@ -146,9 +146,11 @@ problem.** §2 fixes the ownership; §4 builds the GMMU policy on it.
**Tenant** — isolation principal (`tenantId`): a VM/mdev at L0–L2, a logical
serving principal inside a guest in §4. **Agent** — finest principal inside
a tenant (`agentId`): a vLLM engine/worker, a CUDA context, a process.
-**vEP** — virtual execution plane: the per-instance mediator (guest
-emulation + work execution) for one mdev; see
-[`13-vexecution-plane.md`](13-vexecution-plane.md). **KV block** —
+**vEP** — the vGSP Execution Plane: the execution half of vGSP (guest work →
+physical execution → guest-visible completion), as defined in
+[`13-vexecution-plane.md`](13-vexecution-plane.md). This document's problem
+is that the vEP's work-service state is currently one global context rather
+than per-mdev. **KV block** —
paged-attention cache page (vLLM `paged_attention_v2`: fixed-size blocks,
one `block_tables` entry per logical block per sequence); a GPU allocation
range, not a hardware object.
@@ -171,9 +173,9 @@ What exists and what changes:
| `svc.chan[]` | keyed `h_object` (`gsp_sema_service.c:847`) | key `(vgpu, h_object)`; per-mdev lifetime reset |
| `svc.vas[]` | keyed `h_vaspace` (`:988`) | key `(vgpu, h_vaspace)` |
| learned VA→PA map / pins | keyed GPU VA (`:113-130`) | key `(vgpu, VA)`; per-mdev epoch |
-| doorbell token | ownerless (`:13456`, `main.c:865`) | `(vgpu, token)`; route to the owner's pump |
+| doorbell token | ownerless (`:13512`, `main.c:865`) | `(vgpu, token)`; route to the owner's pump |
| completion counters | global boot state | per mdev, reset on session/rebind |
-| pump | first open mdev (`:13296-13308`) | per-mdev pump (or round-robin over all open mdevs) |
+| pump | first open mdev (`:13352-13364`) | per-mdev pump (or round-robin over all open mdevs) |
The GMMU half of address-space separation is the page directory: each VA
space has one, published by the guest through `SET_PAGE_DIRECTORY` with its
@@ -258,7 +260,7 @@ mdev's address space, and a completion event is posted only to the owning
mdev's event object.
Today the release path resolves the target VA through *global* tables
-(`svc_resolve_va`, `svc_sem_write`; `kernel/gsp_sema_service.c:2098,1514`)
+(`svc_resolve_va`, `svc_sem_write`; `kernel/gsp_sema_service.c:2098,2474`)
and posts through `gsp_rm_post_channel_event()` on the emulation context
(`kernel/gsp_rm_emulation.c:602`). The first is the cross-tenant hazard: a
release record decoded from guest A's push buffer could resolve through a
@@ -581,8 +583,10 @@ The requested chain, end to end:
### 4.4 Design A — VA-subtree partitioning (no guest KMD change)
-This is the **stock-compatible** design and the one that can be prototyped
-inside the 1.0 programme. It relies on one idea: **the VA space is the
+This is the **stock-compatible** design and the research entry point: it is
+the only design that needs no guest KMD change, but its prototype is M6 —
+post-1.0, dual-gated, and it does not shape baseline 1.0 code (§7.4). It
+relies on one idea: **the VA space is the
security domain, and a VA is self-describing because its prefix encodes its
owner.**
@@ -1090,6 +1094,10 @@ starts after 1.0 goals are met, on an explicit human decision.
CUDA reproducibility and per-client state issues M1 resolves;
`../../debug/risc-v-gsp-rm-vm/docs/{43-GMMU-PTE-DECODER,44-GMMU-ORACLE-AND-WORKFLOW,23-RM610-READINESS}.md` —
tooling and gates cited above.
-* Local analysis: `LOCAL-MULTI-TENANT-GMMU.md` (adjacent; part of the
- local analysis) — provenance behind §4.2 and the unredacted design
- reasoning.
+
+## Change log
+
+| Date | Change |
+|---|---|
+| 2026-10-08 | Created: multi-mdev service-state ownership rules, isolation-layer model, VRAM oversubscription design, paged-attention tenant/agent protection stack (Design A/B, three granularities), walked example, portability, M1–M8 roadmap. Tree `66388560`. |
+| 2026-10-08 | Review pass: status hash refreshed to `master @ 1f571dda`; drifted line anchors updated for the post-`66388560` code (pump loop, doorbell, `svc_sem_write`); the vEP vocabulary aligned with doc 13 (13 is the vocabulary authority, this document the GMMU-policy authority); Design A classified as the research entry point (M6), not a 1.0 programme item. |
diff --git a/docs/README.md b/docs/README.md
index fe39077..13f0ae5 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -173,20 +173,21 @@ Read in order if you are new. Jump directly if you know what you need.
| 13 | [Physical GPU integration](03-architecture/08-physical-gpu-integration.md) | developers, architects | 20 min |
| 14 | [Physical backend guest wiring](03-architecture/11-physical-backend-guest-wiring.md) | developers, architects | 25 min |
| 15 | [The vGSP execution plane (vEP)](03-architecture/13-vexecution-plane.md) | developers, architects | 25 min |
-| 16 | [The execution frontier: the GR compute-launch chain](04-implementation/03-execution-frontier.md) | developers, debuggers | 15 min |
-| 17 | [Host VF acceleration — future goal (deferred)](03-architecture/12-host-vf-acceleration-future-goal.md) | architects, planners | 10 min |
-| 18 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min |
-| 19 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min |
-| 20 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min |
-| 21 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min |
-| 22 | [GSP in a RISC-V VM (risc-v-gsp-rm-debugger, experimental)](05-operations/03-risc-v-gsp-rm-vm.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
-| 23 | [risc-v-gsp-rm-debugger user guide (experimental)](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
-| 24 | [Defensive Operations Runbook](RUNBOOK.md) | maintainers, operators | 20 min |
-| 25 | [Engineering operations](eng-ops.md) — tool admission and workflows (Part I) plus live physical-backend operations (Part II) | maintainers, developers | 20 min |
-| 26 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min |
-| 27 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min |
-| 28 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min |
-| 29 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference |
+| 16 | [Multi-tenant GMMU protection](03-architecture/14-multi-tenant-gmmu-protection.md) | developers, architects, security reviewers | 25 min |
+| 17 | [The execution frontier: the GR compute-launch chain](04-implementation/03-execution-frontier.md) | developers, debuggers | 15 min |
+| 18 | [Host VF acceleration — future goal (deferred)](03-architecture/12-host-vf-acceleration-future-goal.md) | architects, planners | 10 min |
+| 19 | [Known-knowns and known-unknowns](04-implementation/01-known-knowns-and-unknowns.md) | everyone technical | 25 min |
+| 20 | [Practical challenges and adversarial interoperability](04-implementation/02-practical-challenges-and-adversarial-interop.md) | developers, stakeholders | 20 min |
+| 21 | [Performance, correctness, security](05-operations/01-performance-correctness-security.md) | developers, auditors | 25 min |
+| 22 | [Debugging and maintenance workflows](05-operations/02-debugging-and-maintenance-workflows.md) | developers, maintainers | 25 min |
+| 23 | [GSP in a RISC-V VM (risc-v-gsp-rm-debugger, experimental)](05-operations/03-risc-v-gsp-rm-vm.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
+| 24 | [risc-v-gsp-rm-debugger user guide (experimental)](05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) — implemented, documented and tested ([audit](../debug/risc-v-gsp-rm-vm/docs/27-TOOL-AUDIT.md); [value demos](../debug/risc-v-gsp-rm-vm/docs/25-VALUE-VALIDATION.md)) | developers, analysts | 15 min |
+| 25 | [Defensive Operations Runbook](RUNBOOK.md) | maintainers, operators | 20 min |
+| 26 | [Engineering operations](eng-ops.md) — tool admission and workflows (Part I) plus live physical-backend operations (Part II) | maintainers, developers | 20 min |
+| 27 | [Hardware and software matrix](06-portability/hardware-software-matrix.md) | developers, integrators | 20 min |
+| 28 | [Developer onboarding and standards](07-developers/onboarding-and-standards.md) | contributors | 20 min |
+| 29 | [HAAEEP and collaboration](08-ecosystem/haaeep-and-collaboration.md) | stakeholders, partners | 20 min |
+| 30 | [Glossary and FAQ](09-reference/glossary-and-faq.md) | everyone | reference |
<!-- change (2026-10-02): insert the risc-v-gsp-rm-debugger user guide and the Open-IOV
Defensive Operations Runbook into the reading list; renumber rows 21-24 to
@@ -202,7 +203,11 @@ additive only. -->
<!-- change (2026-10-08): insert "The vGSP execution plane (vEP)" as row 15
(architecture: the work path); renumber rows 15-28 to 16-29; additive only. -->
-Row 17 is a **deferred future goal**: host NVIDIA-driver acceleration over a
+<!-- change (2026-10-08): insert "Multi-tenant GMMU protection" as row 16
+(architecture: isolation design, companion to the vEP doc); renumber rows
+16-29 to 17-30; additive only. -->
+
+Row 18 is a **deferred future goal**: host NVIDIA-driver acceleration over a
virtual function. The feasibility verdict and a first design sketch are
recorded there for planning; it is not on the active critical path (guest CUDA
execution, multi-VM, 3D, performance come first).
diff --git a/wiki/GMMU-Multi-Tenant-Protection.md b/wiki/GMMU-Multi-Tenant-Protection.md
index 83b00ba..6ea5b9b 100644
--- a/wiki/GMMU-Multi-Tenant-Protection.md
+++ b/wiki/GMMU-Multi-Tenant-Protection.md
@@ -15,7 +15,7 @@ two halves:
1. **Service-state ownership (baseline 1.0).** The work/completion service
(`kernel/gsp_sema_service.c`) is one global context (`static struct
sema_svc svc`, `:676`), the pump services the first open mdev
- (`:13296-13308`), and channels/VA spaces/translations are keyed by
+ (`:13352-13364`), and channels/VA spaces/translations are keyed by
guest-chosen values alone (`:847`, `:988`, `:113-130`). Two guests can
pick the same handles and VAs (live: channel `0xcaf000d4`;
`../debug/issue/issue-number-2.md`), so owner-keying is a prerequisite
@@ -76,12 +76,12 @@ periodic owner-vs-PTE audit with canaries catches escapes. If only
per-tenant separation exists, the deployable contract is **one agent per
tenant, one tenant per context** (canonical §5.4).
-## Key code anchors (master `66388560`)
+## Key code anchors (master `1f571dda`)
| What | Where |
|---|---|
| per-mdev emulation context | `kernel/include/nvidia-vgpu-vfio.h:133`, `kernel/gsp_emulation.h:130` |
-| global service state | `kernel/gsp_sema_service.c:676`; pump `:13296`; doorbell `:13456` |
+| global service state | `kernel/gsp_sema_service.c:676`; pump `:13352`; doorbell `:13512` |
| channel/vaspace registries | `kernel/gsp_sema_service.c:847`, `:988` |
| GMMU walk (VER2) | `kernel/gsp_sema_service.c:1174`; decode `:1093` |
| page directory record | `kernel/gsp_rm_emulation.c:3320`; reserved PDEs `:2237-2262` |
diff --git a/wiki/README.md b/wiki/README.md
index 378ea31..d04f70f 100644
--- a/wiki/README.md
+++ b/wiki/README.md
@@ -10,6 +10,7 @@
### Core Analysis
- [The vGSP Execution Plane (vEP)](Execution-Plane.md) - The work/execution layer of `vgsp.ko`: guest pushes to physical execution and guest-visible completion (condensed; canonical doc [`../docs/03-architecture/13-vexecution-plane.md`](../docs/03-architecture/13-vexecution-plane.md))
+- [GMMU Multi-Tenant Protection](GMMU-Multi-Tenant-Protection.md) - Multi-mdev isolation design and paged-attention (vLLM) tenant/agent protection at the GMMU: service-state ownership, per-context page directories, VRAM oversubscription (condensed; canonical doc [`../docs/03-architecture/14-multi-tenant-gmmu-protection.md`](../docs/03-architecture/14-multi-tenant-gmmu-protection.md))
- [GSP 570.144W Firmware Analysis](GSP-570.144W-Firmware-Analysis.md) - Firmware decompilation and address record
- [Address Reference](Address-Reference.md) - Firmware-side address tables
- [CUDA Userspace Driver](CUDA-Userspace-Driver.md) - `libcuda.so` deep dive: layer model, guest interface, forwarding status, crash-path lesson and lookup tooling