diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-05 14:02:51 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-05 14:02:51 -0400 |
| commit | 2a308ccd0786736693ef4369d387de33a0fe6447 (patch) | |
| tree | 09a81ed4f6fefe0aaa5e1304a94ea907b4a24489 /docs | |
| parent | 82a54aec2e79451ee04622e940df9bca88bf6ce3 (diff) | |
| download | open-iov-2a308ccd0786736693ef4369d387de33a0fe6447.tar.gz open-iov-2a308ccd0786736693ef4369d387de33a0fe6447.zip | |
docs: record GR rung-2 ctx-query ABI fix and the managed-booking wall
Document the corrected GR context-buffer query ABI (80-byte entries, 5136
byte params), the new gsp_phys_rmcall_srv_ctxcommit / gsp_phys_rmcall_channel
/ gsp_phys_gr_forward parameters, and the live 2026-10-05 findings: PA-only
never commits the engine context pointer; on the shared VAS the server
pre-books MAIN as managed so a client VA-only promote returns 0x40 at the
vaListSetManaged gate; EVICT_CTX does not release it; the external-VAS
VA-promote shape wedged the device and must not be repeated.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/eng-ops.md | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/docs/eng-ops.md b/docs/eng-ops.md index 35b379d..cf11265 100644 --- a/docs/eng-ops.md +++ b/docs/eng-ops.md @@ -604,12 +604,30 @@ authoritative for the literal default. | `gsp_phys_rmcall_engine` | 0 | Ladder engine: 0 = COPY0 (CE copy submission), 1 = GR0 (compute object, BIND engineType 1, GR semaphore release). Snapshotted when stage 3 starts; mid-session changes are ignored. | Requires boot + init-done; CE forwarding paths refuse on a GR ladder. | | `gsp_phys_rmcall_gr_promote` | 0 (clamp 0..18) | GR context-promote bisection: **0 never sends `PROMOTE_CTX`** (the ladder stops after the geometry queries, context buffer/PT allocation, PDB mapping and channel/compute alloc, before BIND); 1 = single MAIN entry, initialize/no-nonmapped, size raw+subctx; 2 = two-phase PA-only then VA-only with the same PA/VA/size; 3 = single entry with the exact raw size; 4 = two-phase with the reference client's zero-field VA-only step; 5 = one control with `entryCount=2` (PA-only init + zero-field VA-only); 6 = no-entry `hVirtMemory`: allocate an `NV01_MEMORY_VIRTUAL` (class 0x70) VA-range object on the ladder VAS at the context window, then `PROMOTE_CTX` with `hVirtMemory`/`virtAddress`/`size` and `entryCount=0`; 7 = no-entry bare `(virtAddress, size)` with `entryCount=0` and no `hVirtMemory`; 8 = single MAIN entry like 1 on the KMD-managed channel variant V6 (the V5 channel plus alloc flag `SKIP_CTXBUFFER_ALLOC` bit 31); 9 = same V6 channel with the stock client's zero-field VA-only MAIN entry (`bInitialize=0`); 10 = same V6 channel, `GPU_INITIALIZE_CTX` (`0x2080012d`) installs the physical context memdesc first, then `PROMOTE_CTX` `entryCount=0` bare `(virtAddress, size)`; 11 = mode 10 with the context buffer swapped to a server FBMEM allocation (class 0x40, VIDMEM `physAttr=0x4`, aperture-0 leaf PTEs); 12 = V5 UVM/vGPU-faithful PA-only MAIN entry (`bInitialize=1 bNonmapped=1`, no VA) and continue to BIND/SCHEDULE/submit; 13 = mode 12 plus a PATCH PA-only promote first (size from engine context properties engineId 0x10); 14 = V6 KMD channel + PATCH PA-only then the single-entry MAIN init+VA (the flow that books the context VA and calls the commit); 15 = mode 14 with a server FBMEM MAIN buffer and VIDMEM `physAttr 0x4`; 16 = mode 14 plus a PM ctxsw PA-only promote first (engineId 0x09); 17 = no client promote, paired with `gsp_phys_srv_ctxbuf=1` (pre-boot registry profile with `RMSetClientRMAllocatedCtxBuffer=0`) so the GSP-RM server-manages the context buffers; 18 = mode 8 shape plus a client `FERMI_CONTEXT_SHARE_A` (0x9067) on the TSG carried as `hContextShare`, whose construct allocates the GraphicsSubcontext. Live result: modes 1, 2 step 2 and 4 all wedged GSP-RM (`0x52a8b04`, `NV_ERR_OBJECT_NOT_FOUND` 0x57 in the LOGINIT stack) while the PA-only nonmapped step of modes 2/4 is accepted every time - the VA resolve, not the entry fields, is the wall (our external VAS has no server-side VA booking). Mode 6 (hVirtMemory, V5 channel) was refused **cleanly** (`status=0x1f` `NV_ERR_INVALID_ARGUMENT`, no assert, clean release) because the channel was not KMD-managed. Modes 8-11 all completed with **clean releases** (WPR2 down, driver rebound, 5 locks, pstore empty): the V6 channel alloc is accepted (`flags=0x80800000`); modes 8 and 9 (with and without an init entry) both return a clean `status=0x40` `NV_ERR_INVALID_STATE` (mode 9 without the server error log - entry shape irrelevant); mode 10's `GPU_INITIALIZE_CTX` is accepted (`status=0`) but the bare-VA promote still returns `0x40` with the same server log anchor; mode 11 (server FBMEM context buffer) returns `0x40` with a **different** server log anchor, so FBMEM moves the failure to a different structural check. Session 2 (modes 12-17, all clean): mode 12's V5 PA-only MAIN promote is **accepted** (`status=0`) and BIND succeeds, but the GR push faults `CTXNOTVALID` (`sem=0`, `GPGet=1`) - the accepted entry does not set `ENG_CTX_VALID`; mode 13 (PATCH first) and mode 14/15/16 (V6 KMD + PATCH, +PM, FBMEM/VIDMEM) all leave the MAIN promote at the same `0x40` rec-677 site as mode 8, so PATCH/PM/buffer type are not the missing prerequisite; mode 17 (registry `RMSetClientRMAllocatedCtxBuffer=0`, no promote) also faults `CTXNOTVALID`, so even server-managed context buffers leave the channel invalid; mode 18 (client context share, share alloc accepted but `subctxId` not written back) leaves the MAIN promote at the same `0x40` - the remaining wall is the engine-context commit/RAMFC path, not buffer ownership. Offline, the `0x40` candidate is the helper at `0x5aff328` (returns `NV_ERR_INVALID_STATE` when `*(u64*)(a0+0x1b90)==NULL` or `*(u8*)(a0+0x555)==0`). **Mode 19 (the stock guest's captured 9-entry set, MAIN 2 MB PA+VA on V5) is known-wedging**: fn76 never replied, the debugfs trigger dumped core, and the release fail-safe held WPR2/GPU until an approved reboot - never run it or any V5 promote with a MAIN VA entry again. Modes 20/21/22 (2026-10-02, all clean, no wedge): 20 = V6 KMD channel + no-entry `hVirtMemory`; 21 = same without `hVirtMemory` (bare `virtAddress`+`size`); 22 = mode 10's `GPU_INITIALIZE_CTX` (physical MAIN, accepted) followed by mode 20's `hVirtMemory` promote. All three return the same soft `NV_ERR_INVALID_STATE` (0x40) as the KMD entry modes and release cleanly; the KMD client-promote shape space is now exhausted. Nonzero enables can still assert GSP-RM and wedge the device. | GR ladder only; value snapshotted at stage 3; **requires confirmed reboot readiness** before enabling. | | `gsp_phys_repower` | 0 (latched) | Restore the pre-claim runtime-PM/D3cold policy when releasing the GPU; with 0 the D0 locks and runtime reference are deliberately left held until reboot. | **Never enable** (rule 3); re-arming D3cold after a claim can hard-freeze the host. | +| `gsp_phys_rmcall_srv_ctxcommit` | 1 | On the server-managed ctx path, send the client commit for the server-reported MAIN: map it at the ladder ctx VA with FBMEM PTEs, `GPU_INITIALIZE_CTX`, then the UVM-shape VA-only `PROMOTE_CTX` (bInitialize=0/bNonmapped=0). 0 = pure server-managed, no client promote. | Requires `gsp_phys_rmcall_gr_promote=17` + `gsp_phys_srv_ctxbuf=1`. Live 2026-10-05: the corrected ctx query reports all server buffers (MAIN/PATCH FBMEM); on the shared VAS the VA promote returns `0x40` because the server pre-booked MAIN as managed (`vaListSetManaged(FALSE)` gate) and `EVICT_CTX` does not release it; PA-only is accepted but never commits the engine context pointer. The external-VAS VA-promote variant wedged GSP-RM and must never be repeated. | +| `gsp_phys_rmcall_channel` | 0 | Force one GR channel-alloc variant id (1/3/4/5/6) instead of the promote-mode-selected walk. 6 = `SKIP_CTXBUFFER_ALLOC` KMD virtual context: the server neither allocates nor books the ctx buffer, so the client-owned mapping + VA commit path runs. | GR ladder only; 0 keeps the historical walk byte-identical. | +| `gsp_phys_gr_forward` | 0 | Forward a captured guest GR launch (inline QMD + releases) to the physical ladder channel as a real GR method stream, with fail-closed fallback to the host bridge / software release; the guest-visible release is held until the submission edge. | Requires a proven ladder GR release (`REAL GR SEM OK`) and `gsp_phys_guest_mem=1`; default off. | The emulation-side `sema_*` parameters (ring discovery, release recovery, diagnostic watches, grace windows) are not physical-backend controls; their canonical description is [`05-operations/02`](05-operations/02-debugging-and-maintenance-workflows.md) ยง3.2 and [`03-architecture/06`](03-architecture/06-host-module-and-hybrid-model.md). +**GR rung-2 update (2026-10-05, public behaviour).** The +`NV2080_CTRL_CMD_GR_GET_CTX_BUFFER_INFO` ABI was corrected: the +context-buffer entry is 80 bytes (one-byte booleans, `uuid` at 60) and the +params structure is 5136 bytes, not 5648. The old shape always returned +`NV_ERR_INVALID_ARGUMENT`; with the fix, the server-managed path +(`gr_promote=17` + `gsp_phys_srv_ctxbuf=1`) reports the full eight-buffer +geometry (MAIN and PATCH in FBMEM). The engine context pointer is programmed +only by the per-buffer VA commit branch of `PROMOTE_CTX`; a PA-only entry is +accepted but never takes it, and on the shared VAS the server pre-books MAIN +in the engine-context VA list as *managed*, so a client VA-only entry returns +`NV_ERR_INVALID_STATE` at the `vaListSetManaged(FALSE)` gate (`EVICT_CTX` +does not release the booking). The V6 KMD virtual-context channel skips that +booking and is the next bisection shape. The ladder's own GR semaphore push +has not yet executed on the physical engine. + ### 10.3 The routine supervised cycle One cycle is one attempt; there is no second attempt in the same boot after a |
