path: root/wiki/Introduction-to-GMMU.md

# Introduction to the GMMU

> **Scope.** A from-first-principles introduction to the GPU Memory Management
> Unit (GMMU): what it translates, how its page tables work on our Turing
> reference, how the host module uses them, and how the GMMU relates to the CPU
> MMU and the platform IOMMU/SMMU. The architecture chapter is
> [`../docs/03-architecture/05-gmmu-amap-hal-gr.md`](../docs/03-architecture/05-gmmu-amap-hal-gr.md);
> the isolation policy authority is
> [`../docs/03-architecture/14-multi-tenant-gmmu-protection.md`](../docs/03-architecture/14-multi-tenant-gmmu-protection.md)
> (condensed in [`GMMU-Multi-Tenant-Protection.md`](GMMU-Multi-Tenant-Protection.md)).
>
> **Evidence discipline.** Hardware-format claims condense the tracked field
> catalogue
> [`../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md`](../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md)
> and live observations; numeric IDs, layouts and code anchors are cited, and no
> timing/cycle claim is made.
>
> Status: 2026-10-11, tree `master @ 08f39c2f`. Tags: `[live]` observed with the
> stock guest driver; `[proven]` hardware-format fact; `[hypothesis]`
> unexercised; `[design]` not built.

## 1. What the GMMU is

The **GMMU (GPU Memory Management Unit)** is the address-translation hardware
inside the GPU. Every engine — GR (shaders and CUDA kernels), CE (copy engines),
SEC2, the video engines — addresses memory through **GPU virtual addresses
(VAs)**, and the GMMU translates each VA into a physical address: an offset in
GPU local memory (**vidmem**/VRAM), an address in system memory (**sysmem**), or
a peer GPU's memory across a fabric.

It is a paged MMU, conceptually the CPU MMU's sibling:

* **Multi-level page tables** describe the mappings; the root is a **page
  directory base (PDB)**.
* A **TLB** caches translations so the walk is not paid per access.
* Translation failures raise **faults** (invalid/missing leaf, missing table),
  reported through engine fault buffers.
* **Each context has its own page directory (VA space)**, so one context cannot
  name another context's mappings — this is the hardware isolation boundary the
  project builds multi-tenancy on.

What makes it *not* just "the CPU MMU on a GPU":

* The tables live in GPU-visible memory (vidmem or sysmem), and each directory
  entry says which aperture the next level lives in.
* Entries carry GPU-specific **aperture** (which physical space) and
  **memory-kind** fields that the CPU MMU does not have.
* The "physical" addresses inside the tables are **guest-physical addresses or
  emulated-VRAM offsets** in our virtualized setting; the host module owns the
  resolution to real host addresses (§4.2). On a bare-metal GPU the same
  structures point at real vidmem/sysmem.

For this project the GMMU matters three ways: the guest driver programs it
exactly as on bare metal (no guest patches); the guest's page tables are the
ground truth for *where its memory actually is*; and the per-context page
directory is the only proven hardware isolation primitive available on the
reference generation (§4.2).

## 2. The three MMUs: CPU MMU, GMMU, IOMMU/SMMU

| | CPU MMU | GMMU | IOMMU / SMMU |
|---|---|---|---|
| Translates | process VA → host physical address | engine GPU VA → GPU physical (vidmem / sysmem / peer) | device DMA address → host physical address |
| Lives in | the CPU core | the GPU | the host platform (IOMMU: Intel VT-d / AMD-Vi) or the SoC (SMMU, ARM) |
| Programmed by | the OS kernel (per-process tables) | the GPU driver / RM (the *guest* driver here) | the hypervisor / OS (per-device or per-VM domains) |
| Table unit | per-process page tables (e.g. CR3) | per-context page directory (VA space) | per-device/per-VM translation domain |
| Isolation scope | process ↔ process | context ↔ context (engine-level) | VM/device ↔ VM/device (platform DMA) |
| In our stack | guest and host kernels | guest driver builds the tables; host walks or plants its own | hypervisor enforces it; the module resolves GPAs itself when it needs host addresses |

They are three instances of the same idea — *page tables plus a TLB give virtual
memory and isolation to an agent* — at three different points in the address
path, owned by three different software layers. In a virtualized guest a GPU
access passes through **two** of them:

```text
engine (GR / CE)
   │  GPU VA
   ▼
GMMU  (inside the GPU, driven by the guest's page tables)
   │  guest physical address (GPA) / GPU local address
   ▼
IOMMU / SMMU  (in the host platform, programmed by the hypervisor for this VM)
   │  host physical address (HPA)
   ▼
host DRAM / real VRAM
```

The two stages are independent: the **GMMU** decides *which guest page* an
engine touches; the **IOMMU/SMMU** decides *whether this device may reach that
page at all* and where it lives in host memory. GMMU isolation between guest
contexts is the GPU driver/RM's job; IOMMU isolation between VMs is the
platform's job.

**How the two meet in this project:**

* In real vGPU, the host vGPU manager uses the platform IOMMU to establish the
  GPU's view of guest memory. Here the host module does the equivalent
  resolution itself.
* On the reference host the physical GPU's IOMMU group is an **identity domain**
  (all-RAM identity) and the mediated device registers as an **emulated-IOMMU
  VFIO device**: its type-1 container holds guest DMA mappings privately and no
  kernel `iommu_domain` exists for it, so `iommu_iova_to_phys` cannot translate
  guest GPAs. Guest RAM is backed by a QEMU shared file and the module
  translates GPA→HPA through that file's page cache, behind the active-mdev gate
  (a per-mdev-gated translation, *not* itself an IOMMU guarantee).
* The project rule is explicit: **memory isolation is the platform IOMMU's job,
  and we never weaken it** — DMA only through the hypervisor-provided interface
  for the device's own VM, no global mappings, fail closed if a required
  isolation property cannot be established.
* On aarch64 hosts the **SMMU** plays the IOMMU's role; page sizes, cache
  attributes and SMMU behaviour are on the host-portability list for the
  Grace/GB10 targets.

## 3. How the GMMU works (Turing reference)

### 3.1 Page-tree geometry

The Turing page-tree path inherits the Pascal/Volta geometry and uses the
**VER2** entry encoding. Depths run root (0) to leaf (4); VA width is **49
bits**:

| Depth | HW level | VA bits | Index bits | Entry size | Role |
|---|---|---|---|---|---|
| 0 | PDE3 | 48:47 | 2 | 8 B | pointer PDE |
| 1 | PDE2 | 46:38 | 9 | 8 B | pointer PDE |
| 2 | PDE1 | 37:29 | 9 | 8 B | pointer PDE (`NO_ATS` set here) |
| 3 | PDE0 | 28:21 | 8 | 16 B (dual) / 8 B (2 MB leaf) | dual 64K/4K PDE, or a 2 MB leaf PTE |
| 4 | PTE | 20:12 (4 KB) / 20:16 (64 KB) | 9 / 5 | 8 B | leaf PTE |

Supported page sizes are **2 MB** (depth 3), **64 KB** and **4 KB** (depth 4).
A leaf entry alone does not say 4 KB vs 64 KB — the table's depth and page-size
context do — so decoders must report `4k_or_64k` rather than guess.

### 3.2 Entry encoding (VER2)

**Leaf PTE (8 bytes):**

| Field | Bits | Meaning |
|---|---|---|
| `valid` | 0 | entry is present |
| `aperture` | 2:1 | 0 video, 1 peer, 2 system coherent, 3 system non-coherent |
| `volatile` | 3 | cache/volatility bit |
| `encrypted` | 4 | encrypted memory |
| `privilege` | 5 | privileged access |
| `read_only` | 6 | write protection |
| `atomic_disable` | 7 | atomics disabled |
| `address_sys` | 53:8 | system-aperture address bits (PA 53:12) |
| `address_vid` | 32:8 | video/peer address bits (PA 36:12) |
| `peer_id` | 35:33 | peer/fabric id (peer aperture) |
| `comptagline` | 55:36 | compression tag (overloaded as high address bits for the 47-bit peer case) |
| `kind` | 63:56 | memory kind id |

**Directory PDE (8 bytes)** is the same shape with one discriminator change:
bit 0 is `is_pde_pointer` (0 = pointer to a lower table, 1 = a 2 MB leaf PTE),
and **aperture 0 means invalid at directory level** (it means video memory at
leaf level). Depth 2 sets `NO_ATS` (bit 5).

**Dual PDE0 (16 bytes)** holds two halves for mixed 64K/4K tables: BIG (low 64
bits) and SMALL (high 64 bits); bit 0 discriminates the whole slot.

Live guest note: the client's leaf entries in the format we observe are
`PA[47:12] | flags`, with `flags = 0xe` for a valid system-memory leaf. The
`kind` field carries numeric memory-kind ids (243 named values); common compute
values are `0x00 PITCH`, `0x06 GENERIC_MEMORY`, `0x07 INVALID`,
`0x0F SMSKED_MESSAGE`, `0xFE GENERIC_16BX2`.

The decoder (`sim/gmmu_pte.py`, docs/43) is decode-only: unsupported encodings
fail closed, set bits not covered by a named field are reported as reserved, and
nothing is guessed.

### 3.3 Walking, caching and faults

A walk starts at the channel's **page-directory base** and follows
`PDE3 → PDE2 → PDE1 → PDE0 → PTE`; the leaf yields the physical address, page
size, aperture and access permissions. The TLB caches results, and there is a
second, coarser cache that matters operationally:

* **PTE fetch-line rule (learned live):** the engine caches PTE fetches at
  **128-byte (16-PTE) granularity**, and the forwarding code has **no
  invalidate path**. A runtime PTE write into a line the GMMU has already walked
  is invisible to the engine. Consequences: page-table slots for a guest window
  are **pre-built before any engine access**, and each forwarded copy pair
  consumes a whole fresh line from a pre-zeroed pool (16 page-table pages,
  capacity 512 pairs per boot in the current design) — a line is never reused
  within a boot.
* **Faults:** invalid or missing leaves and missing tables raise GMMU faults;
  the driver allocates fault buffers whose sizes come from
  `INTERNAL_GMMU_GET_STATIC_INFO` (control `0x20800a59`): **64 KB replayable /
  4 KB legacy** — the values the emulation serves, because a wrong value breaks
  UVM early.

## 4. How the GMMU is used

### 4.1 Guest side (the stock driver)

* The guest driver allocates VA spaces and **builds the page tables**, then
  publishes the directory per channel/context through `SET_PAGE_DIRECTORY`
  (`hVASpace` + `chId`; the host records the directory per owner in
  `kernel/gsp_rm_emulation.c:3307-3319`).
* Memory descriptors carry an aperture: `ADDR_SYSMEM = 1`, `ADDR_FBMEM = 2`.
* **Server-owned ranges:** `COPY_SERVER_RESERVED_PDES` (control `0x90f10106`)
  tells the client that some page-table ranges are populated by the "server"
  (us). The exact division of labour for `RESERVE_ENTRIES`/`RELEASE_ENTRIES`
  and invalidations is a tracked known-unknown.
* Channels submit push buffers that reference GPU VAs; the engines walk the
  GMMU to reach the data.

### 4.2 Host side (`vgsp.ko`)

* **Truth.** The host resolves guest VAs by **walking the driver-published page
  directories itself** (the trusted GMMU walk in `kernel/gsp_sema_service.c`,
  VER2 geometry at `:1058-1072`, decode `:1093`, walk `:1174`), plus injected
  pins and content-keyed rescue when neither names a mapping. This is what lets
  the software path apply a semaphore release to the correct host page.
* **Forwarding.** For real CE copies the module builds its **own scratch page
  tables** that map already-resolved host pages (guest RAM through the shared
  file; shadow pages through the emulated-VRAM mapping), under the fetch-line
  rules of §3.3. Real guest page tables are *not yet* programmed into the real
  MMU — guest submission on hardware is the end state (P6).
* **Emulation.** Guest vidmem is a private host-RAM buffer per mdev
  (`kernel/bar_init.c:68-77`); FB-aperture accesses stay inside the owning
  mdev's buffer, and guest RAM is reached only through the hypervisor's
  per-mdev DMA interface (`vfio_dma_rw`).
* **Isolation.** The per-context page directory is the **only proven hardware
  isolation primitive** on this generation: the VER2 PTE/PDE has **no
  tenant/ACL/PASID field** (proven absence in the named field set; `peer_id` is
  a fabric peer selector, not a security domain). `read_only` (bit 6) is a
  proven *format* field, but its *enforcement* is a `[hypothesis]` until the M7
  supervised probe. The multi-tenant design therefore places decisions in
  **topology** (which directory a page appears in, which channel may name it)
  plus software validation and audit in the execution plane — see
  [`GMMU-Multi-Tenant-Protection.md`](GMMU-Multi-Tenant-Protection.md).
* **Versioning.** GMMU controls, static info and entry formats are
  version/hardware-keyed; formats are expected to differ per generation and must
  be re-derived by measurement, never guessed.

### 4.3 Tooling built around it

| Tool / doc | Question it answers |
|---|---|
| PTE decoder `sim/gmmu_pte.py` + [docs/43](../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md) | decode a captured entry field-by-field (decode-only, fail closed) |
| Walker `sim/gmmu_walk.py` + [docs/48](../debug/risc-v-gsp-rm-vm/docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md) | walk a captured PDB to the leaf mapping or the exact fault (4K/2M validated against a live captured channel PDB; 64K modelled) |
| Launch audit `harness/launch_audit.py` + docs/48 | where the engine would read/write each leg of a submission: `ok` / `pa_mismatch` / `unmapped` / `access_denied` |
| Oracle workflow [docs/44](../debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md) | compare simulated vs observed GMMU/FBIF register values, per register, with fidelity labels |
| Observe-only windows [docs/42](../debug/risc-v-gsp-rm-vm/docs/42-GMMU-FBIF-OBSERVABILITY.md) | record what firmware touches in the GMMU register windows without faking semantics |

## 5. Limits and open questions

* **`read_only` enforcement** is unproven (M7 supervised probe); **encryption
  (CCS) semantics** are unstudied. Neither can be assumed as a tenant boundary.
* **Invalidation semantics** are not fully modelled: the 128-byte fetch-line
  observation is live evidence, not a complete invalidate contract.
* The **64K dual path** is modelled from the format definitions, not yet
  validated against a capture.
* **Newer generations** (Ampere/Hopper/Blackwell, GB10) are expected to change
  the entry format; each must be re-derived. RM 610 work sits behind its own
  human approval gate.
* **No timing/cycle claims** are made anywhere here; zero hits in a run are a
  coverage fact, not proof of dead code.
* Multi-mdev isolation is first a **service-state-ownership** problem (host
  tables keyed by guest-chosen handles/VAs alone collide); GMMU policy is built
  on top of that fix, not instead of it.

## 6. Related documentation

* [`../docs/03-architecture/05-gmmu-amap-hal-gr.md`](../docs/03-architecture/05-gmmu-amap-hal-gr.md) — architecture chapter: GMMU, AMAPlib/HALs, channels, runlists, GR.
* [`GMMU-Multi-Tenant-Protection.md`](GMMU-Multi-Tenant-Protection.md) — the isolation design condensed.
* [`../docs/03-architecture/14-multi-tenant-gmmu-protection.md`](../docs/03-architecture/14-multi-tenant-gmmu-protection.md) — canonical multi-tenant GMMU policy.
* [`Execution-Plane.md`](Execution-Plane.md) — how guest work becomes physical execution; the fetch-line rule in context.
* [`../docs/03-architecture/11-physical-backend-guest-wiring.md`](../docs/03-architecture/11-physical-backend-guest-wiring.md) — page-table seams and the guest-memory reachability model.
* [`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md) — server-owned page tables and the IOMMU reachability note.
* [`../docs/05-operations/01-performance-correctness-security.md`](../docs/05-operations/01-performance-correctness-security.md) — threat model and the IOMMU rules.
* [`../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md`](../debug/risc-v-gsp-rm-vm/docs/43-GMMU-PTE-DECODER.md), [docs/44](../debug/risc-v-gsp-rm-vm/docs/44-GMMU-ORACLE-AND-WORKFLOW.md), [docs/48](../debug/risc-v-gsp-rm-vm/docs/48-GMMU-WALK-AND-LAUNCH-AUDIT.md) — the field catalogue and the offline walk/audit tooling.