summaryrefslogtreecommitdiffstats
path: root/docs/eng-ops.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-03 14:05:38 -0400
commitcc79b478843f0230fe6aa8ecd822de7ba71c891e (patch)
tree5de760ddec4e45cc23d39ce7502635f5c0711217 /docs/eng-ops.md
parent40351167dd209012ab485e6164ce94f0b8965422 (diff)
downloadopen-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.tar.gz
open-iov-cc79b478843f0230fe6aa8ecd822de7ba71c891e.zip
vgsp: host CUDA bridge + sema release recovery; greeneye simulator wave
- kernel: gsp_cuda_bridge (QMD launch capture, debugfs job/out/done, param+output resolution cached from inline writes), sema drain fixes (never hold/starve a channel), GR ring autopin + counter autoreg, per-page push reads, counter-derive gate; module builds clean - tools/cuda-bridge: CUDA driver-API daemon executing mail/vadd kernels on the physical GPU, CUBB2/CBD2 v2 protocol, layout check + self-tests - tools/guest-diag + tools/cuda-ab: scanner/pin fixes, victory push-only mode - debug/risc-v-gsp-rm-vm: Phase S simulator wave (GSP device model, boot ladder, triage/re-demo harnesses, docs 30-41), GPGPU-sim scaffolding - STATE-CURRENT/docs updates
Diffstat (limited to 'docs/eng-ops.md')
-rw-r--r--docs/eng-ops.md2
1 files changed, 1 insertions, 1 deletions
diff --git a/docs/eng-ops.md b/docs/eng-ops.md
index 43103a9..35b379d 100644
--- a/docs/eng-ops.md
+++ b/docs/eng-ops.md
@@ -602,7 +602,7 @@ authoritative for the literal default.
| `gsp_phys_fault_relay` | 0 | On a relay timeout, drain the status queue for the engine fault (`0x1006`, corroborated by `0x1020`), match runlistId/chid and write the guest channel notifier + event. | Requires `gsp_phys_relay=1` and the three parameters above. |
| `gsp_phys_fault_inject` | 0 | Debug: write the relayed pair's source PTE as zero to force an engine fault on the next relay submission; only relayed pairs are affected. | Requires the relay path; default 0 programs the real source HPA. |
| `gsp_phys_rmcall_engine` | 0 | Ladder engine: 0 = COPY0 (CE copy submission), 1 = GR0 (compute object, BIND engineType 1, GR semaphore release). Snapshotted when stage 3 starts; mid-session changes are ignored. | Requires boot + init-done; CE forwarding paths refuse on a GR ladder. |
-| `gsp_phys_rmcall_gr_promote` | 0 (clamp 0..18) | GR context-promote bisection: **0 never sends `PROMOTE_CTX`** (the ladder stops after the geometry queries, context buffer/PT allocation, PDB mapping and channel/compute alloc, before BIND); 1 = single MAIN entry, initialize/no-nonmapped, size raw+subctx; 2 = two-phase PA-only then VA-only with the same PA/VA/size; 3 = single entry with the exact raw size; 4 = two-phase with the reference client's zero-field VA-only step; 5 = one control with `entryCount=2` (PA-only init + zero-field VA-only); 6 = no-entry `hVirtMemory`: allocate an `NV01_MEMORY_VIRTUAL` (class 0x70) VA-range object on the ladder VAS at the context window, then `PROMOTE_CTX` with `hVirtMemory`/`virtAddress`/`size` and `entryCount=0`; 7 = no-entry bare `(virtAddress, size)` with `entryCount=0` and no `hVirtMemory`; 8 = single MAIN entry like 1 on the KMD-managed channel variant V6 (the V5 channel plus alloc flag `SKIP_CTXBUFFER_ALLOC` bit 31); 9 = same V6 channel with the stock client's zero-field VA-only MAIN entry (`bInitialize=0`); 10 = same V6 channel, `GPU_INITIALIZE_CTX` (`0x2080012d`) installs the physical context memdesc first, then `PROMOTE_CTX` `entryCount=0` bare `(virtAddress, size)`; 11 = mode 10 with the context buffer swapped to a server FBMEM allocation (class 0x40, VIDMEM `physAttr=0x4`, aperture-0 leaf PTEs); 12 = V5 UVM/vGPU-faithful PA-only MAIN entry (`bInitialize=1 bNonmapped=1`, no VA) and continue to BIND/SCHEDULE/submit; 13 = mode 12 plus a PATCH PA-only promote first (size from engine context properties engineId 0x10); 14 = V6 KMD channel + PATCH PA-only then the single-entry MAIN init+VA (the flow that books the context VA and calls the commit); 15 = mode 14 with a server FBMEM MAIN buffer and VIDMEM `physAttr 0x4`; 16 = mode 14 plus a PM ctxsw PA-only promote first (engineId 0x09); 17 = no client promote, paired with `gsp_phys_srv_ctxbuf=1` (pre-boot registry profile with `RMSetClientRMAllocatedCtxBuffer=0`) so the GSP-RM server-manages the context buffers; 18 = mode 8 shape plus a client `FERMI_CONTEXT_SHARE_A` (0x9067) on the TSG carried as `hContextShare`, whose construct allocates the GraphicsSubcontext. Live result: modes 1, 2 step 2 and 4 all wedged GSP-RM (`0x52a8b04`, `NV_ERR_OBJECT_NOT_FOUND` 0x57 in the LOGINIT stack) while the PA-only nonmapped step of modes 2/4 is accepted every time - the VA resolve, not the entry fields, is the wall (our external VAS has no server-side VA booking). Mode 6 (hVirtMemory, V5 channel) was refused **cleanly** (`status=0x1f` `NV_ERR_INVALID_ARGUMENT`, no assert, clean release) because the channel was not KMD-managed. Modes 8-11 all completed with **clean releases** (WPR2 down, driver rebound, 5 locks, pstore empty): the V6 channel alloc is accepted (`flags=0x80800000`); modes 8 and 9 (with and without an init entry) both return a clean `status=0x40` `NV_ERR_INVALID_STATE` (mode 9 without the server error log - entry shape irrelevant); mode 10's `GPU_INITIALIZE_CTX` is accepted (`status=0`) but the bare-VA promote still returns `0x40` with the same server log anchor; mode 11 (server FBMEM context buffer) returns `0x40` with a **different** server log anchor, so FBMEM moves the failure to a different structural check. Session 2 (modes 12-17, all clean): mode 12's V5 PA-only MAIN promote is **accepted** (`status=0`) and BIND succeeds, but the GR push faults `CTXNOTVALID` (`sem=0`, `GPGet=1`) - the accepted entry does not set `ENG_CTX_VALID`; mode 13 (PATCH first) and mode 14/15/16 (V6 KMD + PATCH, +PM, FBMEM/VIDMEM) all leave the MAIN promote at the same `0x40` rec-677 site as mode 8, so PATCH/PM/buffer type are not the missing prerequisite; mode 17 (registry `RMSetClientRMAllocatedCtxBuffer=0`, no promote) also faults `CTXNOTVALID`, so even server-managed context buffers leave the channel invalid; mode 18 (client context share, share alloc accepted but `subctxId` not written back) leaves the MAIN promote at the same `0x40` - the remaining wall is the engine-context commit/RAMFC path, not buffer ownership. Offline, the `0x40` candidate is the helper at `0x5aff328` (returns `NV_ERR_INVALID_STATE` when `*(u64*)(a0+0x1b90)==NULL` or `*(u8*)(a0+0x555)==0`). **Mode 19 (the stock guest's captured 9-entry set, MAIN 2 MB PA+VA on V5) is known-wedging**: fn76 never replied, the debugfs trigger dumped core, and the release fail-safe held WPR2/GPU until an approved reboot - never run it or any V5 promote with a MAIN VA entry again. Nonzero enables can still assert GSP-RM and wedge the device. | GR ladder only; value snapshotted at stage 3; **requires confirmed reboot readiness** before enabling. |
+| `gsp_phys_rmcall_gr_promote` | 0 (clamp 0..18) | GR context-promote bisection: **0 never sends `PROMOTE_CTX`** (the ladder stops after the geometry queries, context buffer/PT allocation, PDB mapping and channel/compute alloc, before BIND); 1 = single MAIN entry, initialize/no-nonmapped, size raw+subctx; 2 = two-phase PA-only then VA-only with the same PA/VA/size; 3 = single entry with the exact raw size; 4 = two-phase with the reference client's zero-field VA-only step; 5 = one control with `entryCount=2` (PA-only init + zero-field VA-only); 6 = no-entry `hVirtMemory`: allocate an `NV01_MEMORY_VIRTUAL` (class 0x70) VA-range object on the ladder VAS at the context window, then `PROMOTE_CTX` with `hVirtMemory`/`virtAddress`/`size` and `entryCount=0`; 7 = no-entry bare `(virtAddress, size)` with `entryCount=0` and no `hVirtMemory`; 8 = single MAIN entry like 1 on the KMD-managed channel variant V6 (the V5 channel plus alloc flag `SKIP_CTXBUFFER_ALLOC` bit 31); 9 = same V6 channel with the stock client's zero-field VA-only MAIN entry (`bInitialize=0`); 10 = same V6 channel, `GPU_INITIALIZE_CTX` (`0x2080012d`) installs the physical context memdesc first, then `PROMOTE_CTX` `entryCount=0` bare `(virtAddress, size)`; 11 = mode 10 with the context buffer swapped to a server FBMEM allocation (class 0x40, VIDMEM `physAttr=0x4`, aperture-0 leaf PTEs); 12 = V5 UVM/vGPU-faithful PA-only MAIN entry (`bInitialize=1 bNonmapped=1`, no VA) and continue to BIND/SCHEDULE/submit; 13 = mode 12 plus a PATCH PA-only promote first (size from engine context properties engineId 0x10); 14 = V6 KMD channel + PATCH PA-only then the single-entry MAIN init+VA (the flow that books the context VA and calls the commit); 15 = mode 14 with a server FBMEM MAIN buffer and VIDMEM `physAttr 0x4`; 16 = mode 14 plus a PM ctxsw PA-only promote first (engineId 0x09); 17 = no client promote, paired with `gsp_phys_srv_ctxbuf=1` (pre-boot registry profile with `RMSetClientRMAllocatedCtxBuffer=0`) so the GSP-RM server-manages the context buffers; 18 = mode 8 shape plus a client `FERMI_CONTEXT_SHARE_A` (0x9067) on the TSG carried as `hContextShare`, whose construct allocates the GraphicsSubcontext. Live result: modes 1, 2 step 2 and 4 all wedged GSP-RM (`0x52a8b04`, `NV_ERR_OBJECT_NOT_FOUND` 0x57 in the LOGINIT stack) while the PA-only nonmapped step of modes 2/4 is accepted every time - the VA resolve, not the entry fields, is the wall (our external VAS has no server-side VA booking). Mode 6 (hVirtMemory, V5 channel) was refused **cleanly** (`status=0x1f` `NV_ERR_INVALID_ARGUMENT`, no assert, clean release) because the channel was not KMD-managed. Modes 8-11 all completed with **clean releases** (WPR2 down, driver rebound, 5 locks, pstore empty): the V6 channel alloc is accepted (`flags=0x80800000`); modes 8 and 9 (with and without an init entry) both return a clean `status=0x40` `NV_ERR_INVALID_STATE` (mode 9 without the server error log - entry shape irrelevant); mode 10's `GPU_INITIALIZE_CTX` is accepted (`status=0`) but the bare-VA promote still returns `0x40` with the same server log anchor; mode 11 (server FBMEM context buffer) returns `0x40` with a **different** server log anchor, so FBMEM moves the failure to a different structural check. Session 2 (modes 12-17, all clean): mode 12's V5 PA-only MAIN promote is **accepted** (`status=0`) and BIND succeeds, but the GR push faults `CTXNOTVALID` (`sem=0`, `GPGet=1`) - the accepted entry does not set `ENG_CTX_VALID`; mode 13 (PATCH first) and mode 14/15/16 (V6 KMD + PATCH, +PM, FBMEM/VIDMEM) all leave the MAIN promote at the same `0x40` rec-677 site as mode 8, so PATCH/PM/buffer type are not the missing prerequisite; mode 17 (registry `RMSetClientRMAllocatedCtxBuffer=0`, no promote) also faults `CTXNOTVALID`, so even server-managed context buffers leave the channel invalid; mode 18 (client context share, share alloc accepted but `subctxId` not written back) leaves the MAIN promote at the same `0x40` - the remaining wall is the engine-context commit/RAMFC path, not buffer ownership. Offline, the `0x40` candidate is the helper at `0x5aff328` (returns `NV_ERR_INVALID_STATE` when `*(u64*)(a0+0x1b90)==NULL` or `*(u8*)(a0+0x555)==0`). **Mode 19 (the stock guest's captured 9-entry set, MAIN 2 MB PA+VA on V5) is known-wedging**: fn76 never replied, the debugfs trigger dumped core, and the release fail-safe held WPR2/GPU until an approved reboot - never run it or any V5 promote with a MAIN VA entry again. Modes 20/21/22 (2026-10-02, all clean, no wedge): 20 = V6 KMD channel + no-entry `hVirtMemory`; 21 = same without `hVirtMemory` (bare `virtAddress`+`size`); 22 = mode 10's `GPU_INITIALIZE_CTX` (physical MAIN, accepted) followed by mode 20's `hVirtMemory` promote. All three return the same soft `NV_ERR_INVALID_STATE` (0x40) as the KMD entry modes and release cleanly; the KMD client-promote shape space is now exhausted. Nonzero enables can still assert GSP-RM and wedge the device. | GR ladder only; value snapshotted at stage 3; **requires confirmed reboot readiness** before enabling. |
| `gsp_phys_repower` | 0 (latched) | Restore the pre-claim runtime-PM/D3cold policy when releasing the GPU; with 0 the D0 locks and runtime reference are deliberately left held until reboot. | **Never enable** (rule 3); re-arming D3cold after a claim can hard-freeze the host. |
The emulation-side `sema_*` parameters (ring discovery, release recovery,