1 files changed, 168 insertions, 0 deletions
diff --git a/wiki/UVM-in-the-Guest.md b/wiki/UVM-in-the-Guest.md
new file mode 100644
index 0000000..53ca755
--- /dev/null
+++ b/wiki/UVM-in-the-Guest.md
@@ -0,0 +1,168 @@
+# UVM in the Guest
+
+> **Scope.** What UVM (`nvidia_uvm.ko`) does inside the guest, why it is the
+> most demanding consumer of the virtual GPU's semantics, and how it waits.
+> Canonical source: [`../docs/03-architecture/04-uvm.md`](../docs/03-architecture/04-uvm.md);
+> where this page and the docs tree disagree, the docs tree wins.
+
+Status: 2026-10-12, tree `master @ 2f066727`. Tags: `[live]` observed with the
+stock guest driver; `[wip]` partially working, software bridge in charge;
+`[gated]` implemented default-off; `[target]` planned end state.
+
+## 1. What UVM is
+
+UVM provides unified memory and GPU-driven page-fault handling. In practice it
+is a **second GPU driver layered on the RM**, owning GPU address spaces and
+running its own memory-management work on the GPU.
+
+| Component | Role |
+|---|---|
+| VA spaces (`uvm_va_space`) | per-process GPU address spaces |
+| Channel managers | pools of GPU channels (CE, SEC2, GR as needed) for UVM's work |
+| Push buffers / GPFIFO | UVM builds and submits its own pushes |
+| Semaphores and trackers | UVM waits on GPU-written semaphores to track completion |
+| Page trees | GPU page tables for its VA spaces |
+| Fault handling | fault buffer plus notification; UVM resolves and replays/cancels faults |
+
+For CUDA, UVM is entered early: `cuInit` → driver init → `UVM_REGISTER_GPU`.
+If UVM cannot register the GPU, CUDA does not start — which is why "UVM
+advances" is the project's primary correctness metric on the way to CUDA.
+
+## 2. The registration path (traced end to end)
+
+```text
+libcuda: UVM_REGISTER_GPU ioctl (0x25)
+  uvm_api_register_gpu -> uvm_va_space_register_gpu -> uvm_gpu_retain_by_uuid
+    nvUvmInterfaceRegisterGpu / GetGpuInfo; alloc_parent_gpu / add_gpu
+      nvUvmInterfaceDeviceCreate          (RM object creation)
+      uvm_hal_init_gpu / get_gpu_caps     (CE caps)
+      uvm_ats_add_gpu / uvm_gpu_init_isr  (fault/interrupt plumbing)
+    init_channel_manager
+      create semaphore pool (sysmem)
+      create channels per CE (GPFIFO + pushbuffer + semaphores)
+      channel_init() -> first push -> semaphore release -> wait
+    uvm_page_tree_init                      (page tables for the VA space)
+```
+
+Each stage has a distinct failure signature — see
+[`Driver-Lifecycle-Field-Guide.md`](Driver-Lifecycle-Field-Guide.md).
+
+## 3. What UVM requires from the "GPU"
+
+| Requirement | Detail | Status |
+|---|---|---|
+| GPU identity | must match the UUID from RM | `[live]` |
+| CE capabilities | per-engine support bits; UVM requires `supported && !grce && sysmem` for CPU_TO_GPU / GPU_TO_CPU channels | `[live]` caps accepted; `[gated]` copies forwarded to the physical copy engine |
+| Channel creation | RM channel objects with GPFIFO/userd/instance parameters | `[live]` |
+| Pushes executed | first action on a new channel is an init push plus a semaphore-tracked wait | `[wip]` software bridge executes memsets, copies and releases; `[gated]` CE-class copies forwarded |
+| Page trees | UVM initializes page trees with its own pushes; server-owned ranges must be populated by us | `[wip]` |
+| Fault delivery | fault buffer plus notification; replay/cancel semantics | `[gated]` backend-side observation/relay; guest contract `[target]` |
+| Completion semantics | tracker values must advance exactly | `[wip]` |
+
+**Current frontier.** Context creation and the full CUDA driver-API smoke pass
+(`ALL OK`, copies matched) run end to end. CE-class copies are forwarded to the
+physical copy engine (default off) with engine-authored releases and a fault
+relay. For GR, a forwarded guest launch reaches the front-end but the inline
+launch-descriptor install has not been observed; context validity at dispatch
+is ruled out ([`Execution-Frontier.md`](Execution-Frontier.md)). The remaining
+UVM work is completion translation and the guest-side fault replay/cancel
+contract.
+
+## 4. The discovery that defines the work: how UVM waits
+
+UVM's tracker is the single most visible symptom of an unfaithful GPU. It
+spins, and after 30 seconds it dumps its state:
+
+```text
+uvm_tracker.c:285 Tracker entry for value 1 (sema VA 0x121000000)
+uvm_channel.c:4184 pending push '' started at uvm_channel.c:2535 in channel_init()
+    releasing value 1
+```
+
+Mechanically, UVM built a push ending in a **semaphore release** — a method
+sequence (CE channel class, e.g. `TURING_DMA_COPY_A`) that sets an address and
+a payload and launches a DMA operation to write the payload to memory — then
+submitted it and waits for the payload to appear. On real hardware the copy
+engine writes it; on our device nobody does unless the emulation does. The
+push we found in guest RAM decodes as:
+
+```text
+20010000 0000c5b5   set-object (TURING_DMA_COPY_A)
+2001a000 0000c076   set-object (GP100_UVM_SW)
+20038090 00000001   incrementing-method header: count 3, addr 0x240
+21000000 00000001   lower 32 bits of the semaphore VA; payload (1)
+200180c0 0000000c   LAUNCH_DMA with SEMAPHORE_TYPE=RELEASE_ONE_WORD
+```
+
+So the emulation's job for this push is: **write `payload` to the memory at
+the semaphore's GPU VA**, which requires a GPU-VA → guest-PA translation. The
+host does this through the learned VA→PA map: while the channel's GPFIFO
+has pending entries it reads each push, decodes the method stream and runs the
+work (page-table memsets, copies, semaphore releases). When a push page cannot
+be resolved — or a channel consumes entries without any release landing — it
+falls back to content: guest RAM is scanned for release groups and pending
+payloads are applied forward-only. A release is applied only when
+`payload > current`, so recycled push buffers and stale copies are harmless.
+One pitfall observed live: a semaphore-page VA can flap between two physical
+pages across map rebuilds; pinning the validated translation fixes the stalled
+wait.
+
+## 5. The memory model UVM assumes
+
+* UVM's semaphore pool is **system memory** (`UVM_APERTURE_SYS`), one 4 KiB
+  page per channel manager, addressed as 4-byte slots (1024 per page).
+* Each page has a **GPU VA** (chosen by the client's RM) and a **CPU VA** (a
+  kernel mapping). UVM polls the CPU VA; the GPU writes the GPU VA.
+* Both addresses are recorded in UVM's structures in guest memory — the key
+  that makes translation possible without extra help from the guest.
+* The CPU VA is a direct-map address, so `PA = cpu_va - page_offset_base`
+  (verified live: writing the payload there completes the push).
+* Values are per channel and monotonic (1, 2, 3 …); UVM's own check
+  (`UVM_GPU_SEMAPHORE_MAX_JUMP`) detects and reports wrong values.
+* The client's page tables for **server-owned ranges are empty chains**: the
+  client hands us the physical addresses of its page-table levels and expects
+  the server to own the lower levels ([`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md)).
+
+## 6. Page trees and faults
+
+`uvm_page_tree_init` initializes a VA space's page tables with pushes on UVM's
+own channels (memops and page-table writes; the copy engine executes them on
+real hardware). Fault handling is event-driven: UVM programs a fault buffer
+(replayable and non-replayable; sizes from `INTERNAL_GMMU_GET_STATIC_INFO`,
+answered by the emulation) and, on a fault, reads the entry, populates the
+page table and replays or cancels the fault with a `MEM_OP` push. Reconciling
+the guest's memory model with the real GPU's here is the deepest remaining work.
+
+## 7. How to observe UVM
+
+In the guest, UVM prints nothing by default:
+
+```bash
+echo 1 | sudo tee /sys/module/nvidia_uvm/parameters/uvm_debug_prints
+```
+
+Messages then appear with the `nvidia-uvm:` prefix. Known-good markers, in
+order: `uvm_gpu_retain_by_uuid` → channel manager created →
+`uvm_page_tree_init` pushes → `UVM_REGISTER_GPU` returns and CUDA proceeds.
+Failure signatures characterized live:
+
+| Message | Meaning |
+|---|---|
+| `Failed to initialize the channel manager: NV_ERR_NOT_SUPPORTED` | CE caps wrong or missing |
+| `stuck waiting` plus `Tracker entry ... channel_init()` | the push's semaphore was never written |
+| `Assert failed ... unexpected semaphore ... jump from 0x0 to 0x1010101` | a wrong payload was written; UVM caught it |
+
+## 8. Why UVM is the perfect correctness oracle
+
+UVM exercises in minutes what a driver test suite would test in hours: memory
+allocation, page tables, channel setup, push generation, semaphore semantics,
+fault handling and error recovery — all with continuous self-checks and loud
+diagnostics. When UVM registers cleanly and CUDA workloads pass, the device is
+behaving like a real GPU.
+
+## 9. Related documentation
+
+* [`../docs/03-architecture/04-uvm.md`](../docs/03-architecture/04-uvm.md) — canonical chapter (this page's source).
+* [`Guest-Stack-and-CUDASW.md`](Guest-Stack-and-CUDASW.md), [`GSP-and-GSP-RM.md`](GSP-and-GSP-RM.md) — the guest stack and the RM objects UVM consumes.
+* [`CUDA-Userspace-Driver.md`](CUDA-Userspace-Driver.md) — the userspace half of the contract; [`Execution-Plane.md`](Execution-Plane.md), [`Execution-Frontier.md`](Execution-Frontier.md) — how UVM's pushes execute, and the current wall.
+* [`../docs/04-implementation/01-known-knowns-and-unknowns.md`](../docs/04-implementation/01-known-knowns-and-unknowns.md) — server-owned page tables and the fault contract.