From ab48e44832db55fdedadcf44f55fa9904ca36f2f Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Fri, 2 Oct 2026 09:05:28 -0400 Subject: STATE-CURRENT: 09:25 check-back - P6c relay (C0/C1/C3), H1 host GR ladder wall, green-eye tooling --- STATE-CURRENT.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 64 insertions(+), 7 deletions(-) (limited to 'STATE-CURRENT.md') diff --git a/STATE-CURRENT.md b/STATE-CURRENT.md index b53145b..ae7fb95 100644 --- a/STATE-CURRENT.md +++ b/STATE-CURRENT.md @@ -1,6 +1,6 @@ # Open-IOV Current State -> **Snapshot: 2026-10-02 03:07.** This is a dated public status page, not live +> **Snapshot: 2026-10-02 09:25.** This is a dated public status page, not live > state; it is refreshed on every progress check. Live working notes are > maintained outside this repository; the public known-knowns/unknowns document > is authoritative here: @@ -81,6 +81,46 @@ chain succeeds. ## Right now (frontier) +- **Guest-submission relay and fault replay landed, default off (P6c + C0/C1/C3, 2026-10-02 07:43-08:07).** With `gsp_phys_relay=1` the physical + engine authors the guest CE copy and the release semaphore: the + engine-authored release was observed in the guest page (`relays=64`), + releases are delivered exactly once (deferred hold), and the relay-on run + stayed green (`requested=68 ok=67 fallback=9`, smoke `ALL OK` 64/64). A + forced-unresolvable control proved the software fallback recovers every + copy exactly once. One relayed copy released its semaphore but failed the + host-side data compare, so the parameter stays default-off pending triage; + verification hardening now logs the mismatch (sem target/payload/poll, + first differing byte, live/snapshot dumps) and falls back to the software + apply. `gsp_phys_fault_relay=1` relays forwarded-engine faults to the guest + channel notifier/event path (live green, default off). The software path is + unchanged with all parameters off. +- **Host GR ladder (H1) opened and hit the next wall (2026-10-02 08:29).** + The host resource-manager ladder is now selectable to the compute engine + (`gsp_phys_rmcall_engine=1`, default off) and allocates a GR0 channel with + a compute-class object on the live GSP-RM (`engine=GR0 hGr=0xcaf0002c`), but + the first host GR semaphore push did not complete: the submit timed out + (`-110`) with no release. The copy-engine control run stayed green + (`REAL COPY OK`), and both runs released latched and clean (protected + region down, distribution driver rebound, exactly 5 power-domain locks, + 0 rail cycles, same boot). Current wall: make the GR engine execute the + submitted compute block - bisect GR allocation parameters, the release + method encoding and the method order/context init, using the drained GR + fault records. Next: host `vectorAdd`, then guest GR forwarding. +- **Experimental tooling: green-eye (2026-10-02 08:36, in development).** + `debug/risc-v-gsp-rm-vm/` boots GSP/uproc firmware in a RISC-V QEMU VM + with a GDB stub and adds Frida-like runtime inspection, a scripting/plugin + API and a Ghidra bridge for GSP-RM runtime-memory analysis and + emulation-accuracy measurement. It is debug tooling only (no product code + changes) and is explicitly marked `# TOOL IN DEVELOPMENT - NOT READY YET!`; + see + [`docs/05-operations/03-risc-v-gsp-rm-vm.md`](docs/05-operations/03-risc-v-gsp-rm-vm.md). +- **P6a trigger-path fix landed (2026-10-02 07:10).** The debug copy + triggers now resolve the guest page host-physical address at trigger time + through the fresh-scratch-line path, retiring the stale-PTE data-check + failure (live: `GUEST G2H COPY OK`, `GUEST W2G COPY OK verify_ok=1`, PTE + self-test and ladder `REAL COPY OK`, clean release). Forwarding was never + affected (HPAs are resolved at submit time). - **Guest CE copies are hardware-authored with verified data, with deferred exactly-once release (P6b waves 1.5 and 2, 2026-10-02, run p6w2d).** Wave 1.5 (run p6w15), with `gsp_phys_work_defer_release=1` (plus @@ -115,6 +155,9 @@ chain succeeds. Module sha256 `5fb5bfbe...`. **Next: P6c** (completion/fault relay, real guest page tables/channels, `vectorAdd`) plus the trigger-page pinning follow-up; the wave-1 forwarding bullet below is subsumed by this result. + The trigger-path fix, the P6c relay/fault relay and the host GR ladder have + since landed (bullets above); the relay data-mismatch triage and the GR + execution wall are the open items. - **Guest CE copies run on the physical engine with verified data (P6b wave 1, 2026-10-02, run p6run13; superseded by the P6b wave-2 result above).** With `gsp_phys_work_forward=1 gsp_phys_guest_mem=1` during a full CUDA @@ -665,10 +708,13 @@ Work execution (host service vs client synchronization): engine block runs on the physical CE - 4 KiB copied, semaphore released, destination equals source [partial] guest-submitted work relayed to the physical engine: eligible guest - CE copies execute on hardware through module-built scratch mappings - with hardware-authored data, host-side verification and deferred, - exactly-once release (software fail-closed fallback remains); real - guest page tables/channels (P6c) remain + CE copies execute on hardware with hardware-authored data, deferred + exactly-once release, engine-authored release semaphores and + forwarded-fault relay (all default off; one relay data-mismatch + under triage; software fail-closed fallback remains); a host GR + ladder allocates a GR channel + compute object but its first GR + submit timed out - real guest page tables/channels and GR + execution remain [not yet] correct results (vectorAdd) ``` @@ -695,7 +741,10 @@ Recent chain of walls removed (2026-09-28 to 2026-09-30): -> DtoH copy-engine-on-compute allowed -> payload roundtrip 64/64 -> runtime client initializes rc=0 (legacy clock-domain fix) -> kernel launch queued -> real-release chain rc=0 (pump bookkeeping fixes) -> execution - backend bring-up is the next wall + backend bring-up -> real GSP-RM session -> host RM client -> real CE copy + on the physical engine -> guest CE copies hardware-authored with + engine-authored releases -> forwarded-fault relay -> host GR ladder + -> GR execution is the next wall ``` ## Tools, code paths and fixes behind this progress @@ -724,9 +773,16 @@ Recent chain of walls removed (2026-09-28 to 2026-09-30): completion counter table (per channel) sema_map_add, chan_dump, shadow_dump semaphore pool page (GPU VA 0x121000000) QMP gva2gpa + guest gdbstub reads push buffer allocation (GPU VA 0x120000000) guest kprobes; /proc//pagemap - client trackers / RM error record chain marker-based /dev/kmsg parsing + client trackers / RM error record chain marker-based /dev/kmsg parsing ``` +Experimental developer tooling (not part of the product path): **green-eye** +(`debug/risc-v-gsp-rm-vm/`) boots GSP/uproc firmware in a RISC-V QEMU VM with +a GDB stub and adds Frida-like runtime inspection, a scripting/plugin API and +a Ghidra bridge for GSP-RM runtime-memory analysis and emulation-accuracy +measurement. It is marked `# TOOL IN DEVELOPMENT - NOT READY YET!`; see +[`docs/05-operations/03-risc-v-gsp-rm-vm.md`](docs/05-operations/03-risc-v-gsp-rm-vm.md). + | Wall | Fix | |---|---| | Host panic: oversized stack frame in a channel snapshot path | static buffers; frame-size audit on every build | @@ -813,6 +869,7 @@ device. - [`docs/03-architecture/08-physical-gpu-integration.md`](docs/03-architecture/08-physical-gpu-integration.md) - chosen execution backend (module-owned GPU + real GSP session) - [`docs/03-architecture/09-full-system-picture.md`](docs/03-architecture/09-full-system-picture.md) - visual end-to-end system map with status tags - [`docs/03-architecture/10-interaction-and-scheduling.md`](docs/03-architecture/10-interaction-and-scheduling.md) - cross-component interaction, queues, interrupts, scheduling detail +- [`docs/05-operations/03-risc-v-gsp-rm-vm.md`](docs/05-operations/03-risc-v-gsp-rm-vm.md) - green-eye GSP RISC-V VM runtime analysis (`# TOOL IN DEVELOPMENT - NOT READY YET!`) --- -- cgit v1.2.3