summaryrefslogtreecommitdiffstats
path: root/docs/04-implementation
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-23 12:11:10 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-23 12:11:10 -0400
commit13f46e60760cc0855484cc69de913ea34f55c24f (patch)
treedf67eb2dfc5afd2c56845e514624a4842152a1f1 /docs/04-implementation
parent21b8ebc9db1d0ae649d6406087ea22b54e3b5775 (diff)
downloadopen-iov-13f46e60760cc0855484cc69de913ea34f55c24f.tar.gz
open-iov-13f46e60760cc0855484cc69de913ea34f55c24f.zip
docs: refresh status, supersede stale material, link the CUDA analysis
* Status now matches the live state: guest driver probe and nvidia-smi work; cuInit/device queries work; cuCtxCreate returns 999 because the emulated engine list is missing the async copy engine (COPY14). The userspace libcuda segfault is recorded as solved, with its mechanism (an RM allocation reply must echo request parameters; zeroing them corrupted the caller's saved frame pointer). * Correct the emulation claims: only a subset of RM controls are special-cased; the rest return a success reply with zeroed parameters whose consumed fields are still being corrected; allocation replies are coherent but no server-side object model exists. * Add the eight-point emulation checklist and reply-fidelity ladder as explicit gates, documented in userspace/cuda and linked from the architecture, implementation and operations docs. * Fix operational errors: build/test commands, module load order, mdev type names, LFS references, debugfs claims, in-repo firmware claims, CI claims, portability matrix (guest status, config layout, userspace driver facts). * Mark historical material: one superseded banner per legacy document with a pointer to its replacement; STATE.md marked as a 2026-09-21 snapshot. * Wiki: fix the index, add status banners and in-place factual corrections (msgq transport vs legacy mailbox, BAR definitions, error statuses, migration, glossary), and note the archived 2023 imports.
Diffstat (limited to 'docs/04-implementation')
-rw-r--r--docs/04-implementation/01-known-knowns-and-unknowns.md29
-rw-r--r--docs/04-implementation/02-practical-challenges-and-adversarial-interop.md32
2 files changed, 49 insertions, 12 deletions
diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md
index 0564ffc..a085211 100644
--- a/docs/04-implementation/01-known-knowns-and-unknowns.md
+++ b/docs/04-implementation/01-known-knowns-and-unknowns.md
@@ -22,7 +22,10 @@ listed as an unknown-unknown has a *method* by which we will discover it.
### 1.1 Device and platform
* A stock 570.144 Linux guest driver probes and initializes against our
- emulated TU104 device; `nvidia-smi` works.
+ emulated TU104 device; `nvidia-smi` works and CUDA user-space starts
+ (`cuInit`, `cuDeviceGetCount`, `cuDeviceGet`). Context creation does not yet
+ complete: `cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`) because the
+ emulated engine list lacks the async copy engine (`COPY14`).
* The same guest **boots unmodified**: no guest code, no guest module
parameters are required for the initialization path.
* The host is unmodified in the standalone configuration (distribution NVIDIA
@@ -77,6 +80,15 @@ it**, **rough cost**.
### 2.1 Execution forwarding
+**Immediate blocker (2026-09-23):** the engine topology is not yet right.
+`GPU_GET_ENGINES_V2` returns 6 engines (GR + CE0..4) where a bare-metal oracle
+returns 7: the async copy engine (`COPY14`; engine descriptor `0x793ceb0e`,
+instance 14, client type `0x38`, class pair `0xc9b5`) is missing, so
+`cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`). The zeroed secondary
+engine record makes `libcuda` rewrite the channel alloc class to `0x41`
+(`NV01_ROOT_CLIENT`) with status `0x19` and roll back. Teardown of the failed
+context still oopses in the guest kernel in `uvm_pmm_devmem_page_to_va_space`.
+
**Unknown:** the correct, isolated way to run a guest's engine work on the
physical GPU: which real scheduling primitive to use per engine, how to
translate guest pushes to real submissions, and how to relay completion and
@@ -129,7 +141,8 @@ forwarded kernel), capture both sides, implement replay; validate with
### 2.4 Version matrix beyond 570.144
**Unknown:** which parts of the observed protocol drift between driver
-families (550/560/580/610+), how much is negotiated, and how much must be
+families (550/560/570/580 — the families with entries under
+`config/versions/`), how much is negotiated, and how much must be
configuration.
**Why:** durability is a core promise; users will run other versions
@@ -269,7 +282,11 @@ tree is the artifact that survives personnel changes.
To make a defensible statement, we need all of:
1. **Interfaces:** every guest→device message for the CUDA path is either
- answered or forwarded, for the supported driver family.
+ answered or forwarded, for the supported driver family — and every reply
+ whose fields the driver consumes is coherent, not just `SUCCESS`-shaped
+ (the reply-fidelity ladder: status-only → coherent zeroed params → coherent
+ values for consumed fields → exact values; see
+ [`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md)).
2. **Memory:** every GPU VA used by work is translatable, and every server-owned
page-table range is populated correctly.
3. **Execution:** grid launches and copies run on real hardware with correct
@@ -283,7 +300,11 @@ To make a defensible statement, we need all of:
measured with a published methodology.
Until all seven hold, we say "initialization works; execution is in progress",
-and we link here.
+and we link here. The CUDA-side checklist behind these gates and the current
+blocker are in
+[`../../userspace/cuda/docs/03-driver-and-gpu-interface.md` §9](../../userspace/cuda/docs/03-driver-and-gpu-interface.md);
+start at
+[`../../userspace/cuda/READ-ME-FIRST.md`](../../userspace/cuda/READ-ME-FIRST.md).
---
diff --git a/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
index 1deff02..e2769f8 100644
--- a/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
+++ b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
@@ -18,7 +18,11 @@ the project portable, maintainable and legally clean.
Correctness is not "no crash". It is:
* the guest observes the same **values** (capabilities, handles, table
- contents) it would on real hardware,
+ contents) it would on real hardware — a control whose reply fields the
+ driver consumes is only "implemented" once its reply is coherent, not just
+ `SUCCESS`-shaped (the reply-fidelity ladder: status-only → coherent zeroed
+ params → coherent values for consumed fields → exact values; see
+ [`../../userspace/cuda/docs/04-virtualization-and-emulation.md` §6](../../userspace/cuda/docs/04-virtualization-and-emulation.md)),
* the guest observes the same **ordering** (completions happen after the work
they signal; faults arrive before replays),
* the guest observes the same **failure semantics** (errors are surfaced, not
@@ -48,8 +52,10 @@ without a methodology is not accepted in this project.
* **Management operations** (object creation, capability queries, mappings)
are answered from cached/derived state where the hardware has no side effects
the guest can observe.
-* **Real work** (grid launches, copies) is forwarded. Nothing else is allowed
- on the hot path.
+* **Real work** (grid launches, copies) is meant to be forwarded to the
+ physical GPU (see §3). Today only the work the host decoder understands —
+ memsets, copies and semaphore releases — is executed in software as a
+ correctness bridge; no CUDA kernel is forwarded yet.
## 3. Forwarding: the three options and how we choose
@@ -99,15 +105,25 @@ means maintaining forks for every guest OS and driver version. A patch to the
hypervisor means every hypervisor project must carry it. A firmware change is
impossible in practice.
+In this repository, patch files exist only as build glue for the *optional*
+OpenRM submodule build (`legacy-patches/nvidia/`, applied by `install.sh`).
+The runtime and the guest stack are never patched, and the deployment path uses
+the distribution driver — so nothing patched is required to run the stack.
+
### 5.2 How compliance is verified
-* **Contract tests**: CI builds run against *stock* drivers obtained from
- distribution repositories (or vendor runfiles), and against *unmodified*
- hypervisor packages.
+* **Contract tests** (goal): CI builds run against *stock* drivers obtained
+ from distribution repositories (or vendor runfiles), and against
+ *unmodified* hypervisor packages. Today CI is push/PR-triggered only — builds,
+ kernel/unit/KUnit/mock tests, static analysis and docs checks; there is no
+ scheduled guest suite or stock-driver contract test yet, so the guest ladder
+ is still run by hand (see the operations docs).
* **No-patch checks**: our build must not depend on any patch file for
third-party components. Where patches exist in the tree for convenience
- (e.g. build glue), the contract tests must pass without them; anything that
- fails this is a bug.
+ (e.g. build glue for the optional OpenRM submodule), they are development
+ scaffolding only: the contract tests must pass without them, and the
+ deployment path uses the unpatched distribution driver; anything that fails
+ this is a bug.
* **Behavioral equivalence**: where we answer from emulation, we validate the
values against traces from real hardware (or from our decompiled study notes) —
not against our own previous answers.