diff options
Diffstat (limited to 'docs/04-implementation')
| -rw-r--r-- | docs/04-implementation/01-known-knowns-and-unknowns.md | 300 | ||||
| -rw-r--r-- | docs/04-implementation/02-practical-challenges-and-adversarial-interop.md | 156 |
2 files changed, 456 insertions, 0 deletions
diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md new file mode 100644 index 0000000..0564ffc --- /dev/null +++ b/docs/04-implementation/01-known-knowns-and-unknowns.md @@ -0,0 +1,300 @@ +# Known-Knowns and Known-Unknowns + +> The most important document for planning work. It states, without +> flattery, what we know, what we know we do not know, and what we do not even +> know to ask yet — plus how each gap gets closed. + +## 0. How to read this + +| Category | Meaning | Our approach | +|---|---|---| +| **Known-knowns** | verified facts, with evidence in this repo | rely on them; keep them tested | +| **Known-unknowns** | gaps we have identified, with a concrete plan to close | schedule them, one by one | +| **Unknown-unknowns** | gaps we cannot yet name | discover them by systematic traversal + invariants + instrumentation | + +Anything listed here as a known-unknown has an owner-visible plan; anything +listed as an unknown-unknown has a *method* by which we will discover it. + +--- + +## 1. Known-knowns (verified) + +### 1.1 Device and platform + +* A stock 570.144 Linux guest driver probes and initializes against our + emulated TU104 device; `nvidia-smi` works. +* The same guest **boots unmodified**: no guest code, no guest module + parameters are required for the initialization path. +* The host is unmodified in the standalone configuration (distribution NVIDIA + driver manages the physical GPU); Open-IOV loads as an independent module. +* Multi-VM operation multiplies the same emulated device; each mdev is + independent. + +### 1.2 Protocol and firmware + +* The GSP boot state machine is emulatable by completing observable stages. +* The message queue transport is understood in detail: element header layout, + page-table translation of non-contiguous queue buffers, sequence matching, + lengths, and quirks (`length` excludes variable payloads). +* The RPC surface used during driver initialization is fully enumerated for + 570.144 (§9 of the field guide gives the ID table). +* Firmware images are analyzable: the 570.144 image is decompiled + (~130 functions of interest mapped to C) and its dispatcher/jump tables are + located. +* Version negotiation exists and is honored (the guest tells us its VGX/RPC + version; we answer with a matching profile). + +### 1.3 Memory and execution semantics + +* GPU VA → guest PA translation is recoverable **without guest cooperation** + by reading the guest's own bookkeeping (VA and CPU-VA pairs recorded in + UVM/RM structures; direct-map relationship for CPU VAs). +* Guest physical memory is readable and writable from the host module through + the hypervisor's DMA interface, from a kernel thread context. +* UVM's pushes are decodable (incrementing-method headers, engine classes, + `SET_SEMAPHORE_A/B/PAYLOAD`, `LAUNCH_DMA`), and completing their semaphore + releases advances UVM through channel manager creation and page-tree + initialization. +* The client's channel memory descriptors carry *physical* addresses we can + trust (instance/userd/ramfc/method buffers). +* The client's page-table contract for server-owned ranges is explicit + (`COPY_SERVER_RESERVED_PDES`) — and its leaf levels are *empty*, i.e. ours to + populate. + +### 1.4 Operational + +* Module reload, mdev create/destroy, VM start/stop, and guest reboot cycles + are stable when done in the documented order. +* Every layer of the emulation is instrumented and diagnosable from `dmesg` + plus a small set of module parameters. + +--- + +## 2. Known-unknowns (named gaps with plans) + +Each entry: **what we do not know**, **why it matters**, **how we will learn +it**, **rough cost**. + +### 2.1 Execution forwarding + +**Unknown:** the correct, isolated way to run a guest's engine work on the +physical GPU: which real scheduling primitive to use per engine, how to +translate guest pushes to real submissions, and how to relay completion and +fault semantics. + +**Why:** this is *the* blocker for CUDA kernel execution, and later for +graphics. + +**Plan:** evaluate the three backends (host RM/GSP proxy, direct engine +submission, hybrid) against a scorecard: correctness, isolation, latency, +maintenance cost across driver versions. Prototype the proxy first (it reuses +the unmodified host driver's knowledge), measure latency, then decide whether a +direct path is required. Deliverable: a forwarded `vectorAdd` with verified +results. + +**Cost:** weeks, not days; the largest single work item in the project. + +### 2.2 Server-owned page tables + +**Unknown:** the exact division of labour for page tables after +`COPY_SERVER_RESERVED_PDES`: which ranges the client will touch, when it +expects `RELEASE_ENTRIES`/`RESERVE_ENTRIES`, and how invalidations are +communicated. + +**Why:** mappings that only we can create must exist for the guest's own +memory to be reachable by engines. + +**Plan:** instrument all vaspace controls (add handlers that log their +parameters), then implement `RESERVE_ENTRIES`/`RELEASE_ENTRIES`/mirroring with +a test harness that maps memory and verifies the guest's expectations by +running its own validation paths (UVM page-tree init exercises this). + +**Cost:** days to weeks. + +### 2.3 Fault semantics + +**Unknown:** fault buffer formats and replay/cancel semantics as the *guest* +expects them in our virtualized setting (we know the buffer sizes and that +UVM programs them, but not yet the end-to-end contract under forwarding). + +**Why:** GPU page faults happen constantly in real CUDA workloads (UVM +oversubscription, migrations); correct handling is required for robustness. + +**Plan:** reproduce faults deliberately (map a VA to nothing, touch it from a +forwarded kernel), capture both sides, implement replay; validate with +`uvm_fault`-heavy tests (e.g. oversubscribed memory). + +**Cost:** weeks. + +### 2.4 Version matrix beyond 570.144 + +**Unknown:** which parts of the observed protocol drift between driver +families (550/560/580/610+), how much is negotiated, and how much must be +configuration. + +**Why:** durability is a core promise; users will run other versions +immediately. + +**Plan:** run the same init traces across 2–3 driver families in guests, +diff the message/control inventories, and refactor handlers to be +version-keyed where they differ. Deliverable: a capability matrix and a +documented "add a version" workflow. + +**Cost:** days per version, once the workflow exists. + +### 2.5 Windows guest specifics + +**Unknown:** the WDDM-facing differences (ROM/VBIOS expectations, MSI-X +semantics, power management, display topology) that the Windows driver +requires beyond the Linux path. + +**Why:** Windows CUDA/3D users are a first-class audience. + +**Plan:** bring up a Windows guest against the same device, capture the driver +logs (Windows ETW + our host traces), and add configuration for discovered +requirements. + +**Cost:** weeks. + +### 2.6 Non-KVM transports + +**Unknown:** what differs for Xen/Argo/OpenXT/Qubes OS Air: IRQ delivery +mechanisms, memory pinning, device-model integration points. + +**Why:** the security-focused platforms are a major part of the project's +reason for existing. + +**Plan:** abstract the transport boundary (IRQ injection, guest-memory access, +device registration) behind a small interface; implement a Xen backend against +it; test in a Xen guest. + +**Cost:** weeks. + +### 2.7 aarch64 hosts and Grace/GB10 + +**Unknown:** host-side differences (page sizes, cache attributes, SMMU +behaviour, power domains) and any firmware/ROM differences in the guest path. + +**Why:** Grace-Hopper/Grace-Blackwell/GB10 (DGX Spark, Surface-class laptops) +are explicit targets. + +**Plan:** make the emulation architecture-independent by construction +(no x86-specific assumptions in the memory model), then bring up on an +aarch64 host with the same test ladder. + +**Cost:** weeks per platform. + +### 2.8 Multi-tenant performance policy + +**Unknown:** best scheduling policy for sharing one GPU across VM tenants +(time slice, MIG partition, hybrid) and its fairness/isolation properties. + +**Why:** multi-tenancy is a core value proposition. + +**Plan:** build measurement first (per-tenant attribution), then implement +policy as configuration, then audit isolation. + +**Cost:** months (it is a research-y area). + +### 2.9 Tooling maturity + +**Unknown:** how much of the debug tooling (memory scanners, pokes, push +decoders) can be productized without exposing unsafe interfaces. + +**Why:** the tooling is currently how we make progress; it must not become a +security liability. + +**Plan:** move tooling behind a debug-only build flag/document its threat +model; keep safe, read-only diagnostics in production builds. + +**Cost:** days. + +--- + +## 3. Unknown-unknowns (and how we find them) + +Unknown-unknowns are discovered by *systematic traversal* and *invariants*, +not by luck. Our methods: + +### 3.1 Traversal by construction + +* **Enumerate every interface the guest touches.** For each: prove we answer + it or forward it. Any interface we have not seen is a discovery task, not a + bug. +* **Drive the guest further, always.** Each new guest-side milestone (probe → + nvidia-smi → cuInit → context → allocation → kernel) is a traversal step + that *forces* new interfaces to appear in the trace. +* **Differential testing across versions/hardware.** The same traversal on a + second driver version or GPU surfaces requirements that are invisible on + the first. + +### 3.2 Invariants that trip on unknowns + +We rely on the guest's own self-checks as an oracle: + +* UVM asserts on semaphore jumps, bad page tables, impossible capabilities. +* The RM validates structures and will fault on inconsistencies. +* Kernel drivers log "unexpected" states loudly. + +Every guest assert/log is a discovered unknown-unknown if we do not already +have it in this document. + +### 3.3 Structural instrumentation + +* **Trace-first policy:** no handler is written without logging the request + *and* the reply in a reconstructible form. +* **Reconstruct from dmesg:** the logs must be sufficient to rebuild the + object tree, memory map and push stream offline. This is how protocol drift + is detected early, before it becomes a crash. + +### 3.4 Fuzzing and negative tests + +* Feed the emulation malformed/partial guest actions (truncated elements, + bad sequence numbers, out-of-range handles) and require graceful + behaviour — this finds assumptions we did not know we had. +* Feed the *guest* plausible-but-wrong device behaviour in a lab profile and + verify the guest recovers (drivers do error recovery we must not break). + +### 3.5 Failure archaeology + +Every failure leaves artifacts. The discipline is: capture host `dmesg`, +guest `dmesg`, guest PCI config, and the emulation's per-RPC log; then write +the *decision tree* that leads to the failure into the debugging docs. The +tree is the artifact that survives personnel changes. + +--- + +## 4. What we need to know to declare "CUDA works" + +To make a defensible statement, we need all of: + +1. **Interfaces:** every guest→device message for the CUDA path is either + answered or forwarded, for the supported driver family. +2. **Memory:** every GPU VA used by work is translatable, and every server-owned + page-table range is populated correctly. +3. **Execution:** grid launches and copies run on real hardware with correct + results, repeatedly, with errors surfaced. +4. **Completion:** semaphores, events and faults behave as the guest expects + (values advance correctly, no lost interrupts, no stuck waits). +5. **Version:** at least two driver families pass, with a documented path for + more. +6. **Isolation:** a second tenant cannot observe or influence the first. +7. **Performance:** within an agreed factor of native for the supported cases, + measured with a published methodology. + +Until all seven hold, we say "initialization works; execution is in progress", +and we link here. + +--- + +## 5. Practical challenges (summary; details in the challenges doc) + +| Challenge | Why it is hard | Mitigation | +|---|---|---| +| Real execution | emulating GR is hopeless; forwarding must be correct and isolated | proxy-first, measure, iterate | +| Page tables both sides | guest's and hardware's expectations must agree | study AMAP/HAL definitions; build a page-table validator | +| Latency | RPC round-trips and push translation add overhead | batch, fast-path pure state in the emulation | +| Version drift | protocol and structures evolve | version-keyed handlers, config-driven profiles | +| Security | forwarding must not weaken isolation | IOMMU-only access, per-tenant contexts, audits | +| Debuggability | most bugs are in the guest's head, not ours | trace-first, reconstruct-from-logs policy | +| Test cost | real GPU time and guest boots are slow | scripted ladders, keep artifacts, parallelize |
\ No newline at end of file diff --git a/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md new file mode 100644 index 0000000..1deff02 --- /dev/null +++ b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md @@ -0,0 +1,156 @@ +# Practical Challenges and Adversarial Interoperability + +> How we intend to implement compute/CUDA support *fast and correctly*, and the +> discipline that keeps the whole stack unmodified — ours included. + +## 1. The central engineering bet + +> Emulate the interface. Pass through the work. Never patch anyone. + +Everything below follows from taking that seriously. It forces hard choices +(no "just add a flag to the guest driver"), and it pays for itself by making +the project portable, maintainable and legally clean. + +## 2. Fast and correct: the tension + +### 2.1 What "correct" means here + +Correctness is not "no crash". It is: + +* the guest observes the same **values** (capabilities, handles, table + contents) it would on real hardware, +* the guest observes the same **ordering** (completions happen after the work + they signal; faults arrive before replays), +* the guest observes the same **failure semantics** (errors are surfaced, not + swallowed), +* and the work produces the same **results** (a CUDA kernel's output matches + native execution). + +### 2.2 What "fast" means here + +Two budgets matter, and they are different: + +* **Initialization latency** — how long from VM start to CUDA ready. Dominated + by RPC round-trips and guest driver's own work. Target: within 2× of physical + device initialization in the same guest. +* **Steady-state throughput/latency** — kernel launch overhead, copy + bandwidth, and completion latency. Target: published, stable, and within an + agreed factor of native for each supported operation class. + +We will publish the methodology and the numbers together; a performance claim +without a methodology is not accepted in this project. + +### 2.3 The fast paths + +* **Pure state operations** (semaphores, membar, fences, some memops) are + satisfied in the emulation directly — they are microseconds in hardware and + would otherwise cost a round trip to the physical GPU and back. +* **Management operations** (object creation, capability queries, mappings) + are answered from cached/derived state where the hardware has no side effects + the guest can observe. +* **Real work** (grid launches, copies) is forwarded. Nothing else is allowed + on the hot path. + +## 3. Forwarding: the three options and how we choose + +``` + (A) Host RM/GSP proxy + guest push → translate → host driver's RM API → real channel → GPU + + reuses a mature driver's knowledge + − extra hop, depends on host driver interfaces + + (B) Direct engine submission + guest push → translate → our own channel/push construction → GPU + + lowest latency, no host driver dependency + − we must own page tables, scheduling, invalidation, completion + + (C) Hybrid + management via (A), hot path via (B) +``` + +**Decision procedure:** prototype (A) first to get *correct* execution, with +measurement harnesses around it. Then measure the overhead. If the overhead +violates the budgets in §2.2 for the target workloads, implement (B) for the +hot path and keep (A) for management. The architecture must not preclude +either, so the work path is abstracted behind an interface from day one. + +## 4. Hard problems we must solve, and our stance on each + +| Problem | Stance | +|---|---| +| **Page tables on both sides** | We populate server-owned ranges using the layout the client hands us and the entry encodings from AMAP/HAL references; a validator compares our tables against what the guest expects before enabling a channel. | +| **Memory coherence** | Guest sysmem is coherent by construction (same physical pages); VRAM objects must be migrated to real VRAM or DMA-mapped. Phase 2 picks per-object policy with correctness first. | +| **Scheduling between tenants** | One real GPU, N guest channels — schedule explicitly (time-slice/partition), measure fairness, audit isolation. Never let the guest's own scheduler influence another tenant. | +| **Fault handling** | Reproduce faults under forwarding, implement replay/cancel on the guest's terms, and make fault latency measurable. | +| **State explosion** | Drivers create enormous state. We do not model all of it — we model what the guest *observes*, and forward what it *uses*. Anything unobserved stays unmodeled. | +| **Version drift** | Version-keyed handlers with a configuration-driven profile per (driver, firmware) family; new versions added by measurement, never by guessing. | +| **Security of the debug tooling** | Debug parameters are development-grade; the productized build keeps only read-only diagnostics and gates destructive ones behind explicit opt-in. | + +## 5. Adversarial interoperability + +### 5.1 The rule + +**No modification is permitted in any component we do not own**: guest driver, +host driver, GPU firmware, hypervisor, device model, guest OS. + +Why so absolute: every permitted exception becomes a maintenance liability that +scales with the number of versions in the field. A patch to the guest driver +means maintaining forks for every guest OS and driver version. A patch to the +hypervisor means every hypervisor project must carry it. A firmware change is +impossible in practice. + +### 5.2 How compliance is verified + +* **Contract tests**: CI builds run against *stock* drivers obtained from + distribution repositories (or vendor runfiles), and against *unmodified* + hypervisor packages. +* **No-patch checks**: our build must not depend on any patch file for + third-party components. Where patches exist in the tree for convenience + (e.g. build glue), the contract tests must pass without them; anything that + fails this is a bug. +* **Behavioral equivalence**: where we answer from emulation, we validate the + values against traces from real hardware (or from our decompiled study notes) — + not against our own previous answers. + +### 5.3 What we *can* change + +* Our own module, userspace library, tools, scripts and docs. +* Configuration: device profiles, version profiles, capability tables. +* Tests and harnesses. + +### 5.4 Where the line gets tested + +* **Firmware signatures**: the driver verifies firmware. We satisfy the + handshake by *emulating the interface*, not by patching the image or the + driver. +* **PCI IDs and VBIOS**: the guest driver decides how to treat the device from + what it reads. We present a coherent device; we never rely on the driver + skipping checks. +* **Licensing checks**: our device is our device; we do not impersonate a + licensed product. Interoperability is the goal, not circumvention of a + vendor's own licensing for their own products. + +## 6. Why this approach is durable + +Because our artifacts are: + +* **Interface definitions** (protocol tables, structures) — survive versions. +* **Measured contracts** (what the guest observes) — survive reimplementation. +* **Configuration** (profiles per version/hardware) — survive hardware. +* **Tests** (contract ladders) — catch drift immediately. + +A new driver version or GPU is therefore *data + verification*, not a rewrite — +the difference between a hobby project and infrastructure. + +## 7. The "thrilling" part (for developers) + +If you are reading this to decide whether to work on it: this project is one of +the few places where you can: + +* reverse engineer a 28 MB RISC-V firmware and turn it into maintainable C, +* design a memory model that makes two different address-space owners agree, +* build a virtualization path for the most commercially important accelerator + on Earth, under a permissive license, +* and have your work outlive the hardware generation you started on. + +Start with [`../07-developers/onboarding-and-standards.md`](../07-developers/onboarding-and-standards.md). |