summaryrefslogtreecommitdiffstats
path: root/docs/04-implementation/01-known-knowns-and-unknowns.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-29 15:29:00 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-29 15:29:00 -0400
commit1fdbf4ea8c17dab2b8fca6b5e58c2a6d3bc7596d (patch)
treecd16ecce1bbbf8e71c17091df843441965f871c1 /docs/04-implementation/01-known-knowns-and-unknowns.md
parentf4803e303e1e4bfde9f241ec3b6226399d01825d (diff)
downloadopen-iov-1fdbf4ea8c17dab2b8fca6b5e58c2a6d3bc7596d.tar.gz
open-iov-1fdbf4ea8c17dab2b8fca6b5e58c2a6d3bc7596d.zip
docs: record physical-GPU integration mechanism and refresh project status
- add docs/03-architecture/08-physical-gpu-integration.md: interfaces that exist, mechanism options A-E, chosen path (module-owned GPU driven through its real GSP firmware), sharing policy, decision record - update 06-host-module-and-hybrid-model, 02-practical-challenges, system overview, known-knowns, docs/README reading order and CUDA-side docs to match - correct stale engine-topology/COPY14 blocker claims across the tree - add STATE-CURRENT.md as the dated public status snapshot
Diffstat (limited to 'docs/04-implementation/01-known-knowns-and-unknowns.md')
-rw-r--r--docs/04-implementation/01-known-knowns-and-unknowns.md52
1 files changed, 29 insertions, 23 deletions
diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md
index a085211..29558b0 100644
--- a/docs/04-implementation/01-known-knowns-and-unknowns.md
+++ b/docs/04-implementation/01-known-knowns-and-unknowns.md
@@ -22,10 +22,12 @@ listed as an unknown-unknown has a *method* by which we will discover it.
### 1.1 Device and platform
* A stock 570.144 Linux guest driver probes and initializes against our
- emulated TU104 device; `nvidia-smi` works and CUDA user-space starts
- (`cuInit`, `cuDeviceGetCount`, `cuDeviceGet`). Context creation does not yet
- complete: `cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`) because the
- emulated engine list lacks the async copy engine (`COPY14`).
+ emulated TU104 device; `nvidia-smi -L` works and CUDA user-space starts
+ (`cuInit`, `cuDeviceGetCount`, `cuDeviceGet`). Context creation is in
+ progress: the client reaches `cuCtxCreate` and currently ends in a forced
+ `rc=999` on a completion wait; the engine inventory is complete (the async
+ copy engine is included) and the remaining work is in the completion and
+ work-execution path (see §2.1).
* The same guest **boots unmodified**: no guest code, no guest module
parameters are required for the initialization path.
* The host is unmodified in the standalone configuration (distribution NVIDIA
@@ -80,29 +82,33 @@ it**, **rough cost**.
### 2.1 Execution forwarding
-**Immediate blocker (2026-09-23):** the engine topology is not yet right.
-`GPU_GET_ENGINES_V2` returns 6 engines (GR + CE0..4) where a bare-metal oracle
-returns 7: the async copy engine (`COPY14`; engine descriptor `0x793ceb0e`,
-instance 14, client type `0x38`, class pair `0xc9b5`) is missing, so
-`cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`). The zeroed secondary
-engine record makes `libcuda` rewrite the channel alloc class to `0x41`
-(`NV01_ROOT_CLIENT`) with status `0x19` and roll back. Teardown of the failed
-context still oopses in the guest kernel in `uvm_pmm_devmem_page_to_va_space`.
-
-**Unknown:** the correct, isolated way to run a guest's engine work on the
-physical GPU: which real scheduling primitive to use per engine, how to
-translate guest pushes to real submissions, and how to relay completion and
-fault semantics.
+**Engine topology (solved):** `GPU_GET_ENGINES_V2` now reports the full engine
+inventory including the async copy engine, and the copy-engine capabilities
+match the physical device (two GRCEs). The earlier zeroed secondary engine
+record that made `libcuda` rewrite the channel alloc class and roll back is
+gone; context creation now progresses past engine selection and stalls later,
+on completion of decoded work (see §1.1).
+
+**Unknown (resolved by decision, not yet implemented):** the correct,
+isolated way to run a guest's engine work on the physical GPU. The interface
+analysis in
+[`../03-architecture/08-physical-gpu-integration.md`](../03-architecture/08-physical-gpu-integration.md)
+shows that proxying through the distribution host driver has no callable entry
+point from kernel context, so the **chosen mechanism** is for the module to own
+the physical GPU and run its real GSP firmware as the resource-manager server
+(the module becomes the host-side RM client). The emulation path remains the
+control plane.
**Why:** this is *the* blocker for CUDA kernel execution, and later for
graphics.
-**Plan:** evaluate the three backends (host RM/GSP proxy, direct engine
-submission, hybrid) against a scorecard: correctness, isolation, latency,
-maintenance cost across driver versions. Prototype the proxy first (it reuses
-the unmodified host driver's knowledge), measure latency, then decide whether a
-direct path is required. Deliverable: a forwarded `vectorAdd` with verified
-results.
+**Plan:** build the chosen mechanism in stages: device ownership + real GSP
+bring-up; host-side RM client for real objects and channels; translation of
+guest work and real submission; completion and fault relay. Measure each stage
+against a scorecard: correctness, isolation, latency, maintenance cost across
+driver versions. Deliverable: a forwarded `vectorAdd` with verified results.
+Alternatives (own scheduling without GSP; vendor vGPU or SR-IOV paths) are
+analyzed and rejected or deployment-gated in the integration doc.
**Cost:** weeks, not days; the largest single work item in the project.