diff options
Diffstat (limited to 'docs/04-implementation/01-known-knowns-and-unknowns.md')
| -rw-r--r-- | docs/04-implementation/01-known-knowns-and-unknowns.md | 52 |
1 files changed, 29 insertions, 23 deletions
diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md index a085211..29558b0 100644 --- a/docs/04-implementation/01-known-knowns-and-unknowns.md +++ b/docs/04-implementation/01-known-knowns-and-unknowns.md @@ -22,10 +22,12 @@ listed as an unknown-unknown has a *method* by which we will discover it. ### 1.1 Device and platform * A stock 570.144 Linux guest driver probes and initializes against our - emulated TU104 device; `nvidia-smi` works and CUDA user-space starts - (`cuInit`, `cuDeviceGetCount`, `cuDeviceGet`). Context creation does not yet - complete: `cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`) because the - emulated engine list lacks the async copy engine (`COPY14`). + emulated TU104 device; `nvidia-smi -L` works and CUDA user-space starts + (`cuInit`, `cuDeviceGetCount`, `cuDeviceGet`). Context creation is in + progress: the client reaches `cuCtxCreate` and currently ends in a forced + `rc=999` on a completion wait; the engine inventory is complete (the async + copy engine is included) and the remaining work is in the completion and + work-execution path (see §2.1). * The same guest **boots unmodified**: no guest code, no guest module parameters are required for the initialization path. * The host is unmodified in the standalone configuration (distribution NVIDIA @@ -80,29 +82,33 @@ it**, **rough cost**. ### 2.1 Execution forwarding -**Immediate blocker (2026-09-23):** the engine topology is not yet right. -`GPU_GET_ENGINES_V2` returns 6 engines (GR + CE0..4) where a bare-metal oracle -returns 7: the async copy engine (`COPY14`; engine descriptor `0x793ceb0e`, -instance 14, client type `0x38`, class pair `0xc9b5`) is missing, so -`cuCtxCreate` returns `rc=999` (`CUDA_ERROR_UNKNOWN`). The zeroed secondary -engine record makes `libcuda` rewrite the channel alloc class to `0x41` -(`NV01_ROOT_CLIENT`) with status `0x19` and roll back. Teardown of the failed -context still oopses in the guest kernel in `uvm_pmm_devmem_page_to_va_space`. - -**Unknown:** the correct, isolated way to run a guest's engine work on the -physical GPU: which real scheduling primitive to use per engine, how to -translate guest pushes to real submissions, and how to relay completion and -fault semantics. +**Engine topology (solved):** `GPU_GET_ENGINES_V2` now reports the full engine +inventory including the async copy engine, and the copy-engine capabilities +match the physical device (two GRCEs). The earlier zeroed secondary engine +record that made `libcuda` rewrite the channel alloc class and roll back is +gone; context creation now progresses past engine selection and stalls later, +on completion of decoded work (see §1.1). + +**Unknown (resolved by decision, not yet implemented):** the correct, +isolated way to run a guest's engine work on the physical GPU. The interface +analysis in +[`../03-architecture/08-physical-gpu-integration.md`](../03-architecture/08-physical-gpu-integration.md) +shows that proxying through the distribution host driver has no callable entry +point from kernel context, so the **chosen mechanism** is for the module to own +the physical GPU and run its real GSP firmware as the resource-manager server +(the module becomes the host-side RM client). The emulation path remains the +control plane. **Why:** this is *the* blocker for CUDA kernel execution, and later for graphics. -**Plan:** evaluate the three backends (host RM/GSP proxy, direct engine -submission, hybrid) against a scorecard: correctness, isolation, latency, -maintenance cost across driver versions. Prototype the proxy first (it reuses -the unmodified host driver's knowledge), measure latency, then decide whether a -direct path is required. Deliverable: a forwarded `vectorAdd` with verified -results. +**Plan:** build the chosen mechanism in stages: device ownership + real GSP +bring-up; host-side RM client for real objects and channels; translation of +guest work and real submission; completion and fault relay. Measure each stage +against a scorecard: correctness, isolation, latency, maintenance cost across +driver versions. Deliverable: a forwarded `vectorAdd` with verified results. +Alternatives (own scheduling without GSP; vendor vGPU or SR-IOV paths) are +analyzed and rejected or deployment-gated in the integration doc. **Cost:** weeks, not days; the largest single work item in the project. |