summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-05 10:39:31 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-05 10:39:31 -0400
commitd350fb6eb56c67fd8a435f4f41f7b81d7c62c7cb (patch)
tree7a633cc3cf95b182ad66b568a633d0a741384c3c
parente8e31431469f347b9dae8c0e33db23faafa51965 (diff)
downloadopen-iov-d350fb6eb56c67fd8a435f4f41f7b81d7c62c7cb.tar.gz
open-iov-d350fb6eb56c67fd8a435f4f41f7b81d7c62c7cb.zip
STATE-CURRENT: 10:38 check-back snapshot (campaign 239-242 green; 243b live; committed experiment e7ff3bf7)
-rw-r--r--STATE-CURRENT.md26
1 files changed, 14 insertions, 12 deletions
diff --git a/STATE-CURRENT.md b/STATE-CURRENT.md
index 7a52b80..5bd6cad 100644
--- a/STATE-CURRENT.md
+++ b/STATE-CURRENT.md
@@ -1,7 +1,7 @@
# Open-IOV Current State
> **Snapshot: 2026-10-05 (reorganized; simulator status current 2026-10-04,
-> physical/guest campaign data as of 2026-10-05 ~10:12; debugging-options
+> physical/guest campaign data as of 2026-10-05 ~10:38; debugging-options
> catalogue added 2026-10-05).** This is a dated public
> status page, not live state; it is refreshed on every progress check. Live
> working notes are maintained outside this repository; the public
@@ -48,8 +48,9 @@ isolation, performance, portability).
and fakes/run fell 3.39 -> 0.61. A further experiment (hold every unreadable
CE entry for the bounded grace, delete the silent pre-rescue fake, grace
25 s) is now committed (`4115f1dd`, module `e7ff3bf7`) and under live
- validation: campaign `239..262` is running with `237-240` green on the
- deterministic acceptance and only a harness push failure (`241a`, retried).
+ validation: campaign `239..262` is running with `239-242` green on the
+ deterministic acceptance (`241a`/`243a` harness push failures retried) and
+ `243b` in progress at the 10:38 check.
Residual classes: late-decode corruption (~3-5%), an early UVM
channel-manager stall (~5-10%), and a guest init panic (~1/10, retried).
The vadd executes via the host CUDA bridge daemon; forwarding guest engine
@@ -59,8 +60,9 @@ isolation, performance, portability).
host driver owned the device and triggered an uncorrectable ECC Xid, after
which `RmInitAdapter` failed persistently even after a full driver reload.
The host recovered after a reboot (~20:11); `nvidia-smi -L` is green and the
- device is healthy. The fix (shadow BAR2 in emulation-only mode) is in flight
- and gates any further no-claim run.
+ device is healthy. The fix (shadow BAR2 in emulation-only mode) is committed
+ (in `cf6ff542`) and has held across the acceptance campaigns since - no Xid
+ observed.
- **Simulator workstream:** see the section below (separate, development-only).
**Blockers (current walls).**
@@ -102,11 +104,10 @@ isolation, performance, portability).
4. **Known hazards, mitigated, never unattended:** P1 async FWSEC boot defect
(`staging not visible before kick`; fail-open, quarantined off the critical
path); the D0/D3cold power-gating freeze hazard (default-latched locks,
- exactly 5 power-domain locks per release); and the newly found **no-claim
- BAR2 exposure** (real GPU MMIO mapped to the guest in emulation-only mode
- can drive the host-owned device to an ECC Xid; shadow BAR2 before any
- further no-claim run); the shadow-BAR2 fix is now in the working tree and in
- the running module, pending commit. Recurring and separate: guest-kernel boot
+ exactly 5 power-domain locks per release); and the **no-claim BAR2
+ exposure** (real GPU MMIO mapped to the guest in emulation-only mode
+ can drive the host-owned device to an ECC Xid; shadowed BAR2 prevents it -
+ committed and holding, no Xid since). Recurring and separate: guest-kernel boot
panics (a NULL-in-interrupt/swap class) can kill a no-claim run at startup -
guest-side only, host stays healthy.
@@ -275,7 +276,7 @@ chain succeeds.
| ID | Milestone | When | State |
|---|---|---|---|
-| MS-20261005-01 | Hardening the accepted no-claim path (hazard classes closed; rates up) | 2026-10-05 overnight | per-op once bitmap + epoch/content-hash gate + grace 15 s (`b645cf6b`, `3b094c2d`, `c3cd4631`); first-attempt 86% -> ~94-95%; fakes/run 3.39 -> 0.61; hold-all-unreadable/25 s experiment committed (`4115f1dd`, module `e7ff3bf7`), campaign `239..262` validating (`237-240` green) |
+| MS-20261005-01 | Hardening the accepted no-claim path (hazard classes closed; rates up) | 2026-10-05 overnight | per-op once bitmap + epoch/content-hash gate + grace 15 s (`b645cf6b`, `3b094c2d`, `c3cd4631`); first-attempt 86% -> ~94-95%; fakes/run 3.39 -> 0.61; hold-all-unreadable/25 s experiment committed (`4115f1dd`, module `e7ff3bf7`), campaign `239..262` validating (`239-242` green; `241a`/`243a` infra retries; `243b` live) |
| MS-20261004-04 | **First green end-to-end guest CUDA acceptance** | 2026-10-04 ~16:20 | `noclaim38/39/40` `MAIL` + `VADD n=1048576 bad=0 checksum=392344090.500000` + `STATUS OK`, deterministic, committed `cf6ff542`; 14/15 green in the repeat campaign |
| MS-20261004-01 | No-claim bridge reaches the guest `vectorAdd` launch | 2026-10-04 ~14:45 | guest-truth pins + DtoH mail read-back fix + FB 512 MiB; `cudaMalloc` unblocked; no `STATUS OK` yet |
| MS-20261004-02 | Latent bridge pending-slot oops found, fixed; reboot | 2026-10-04 ~14:45-15:00 | `cubb_pend_release` slot reuse -> NULL deref; fixed `85539844...`; approved reboot restored the GPU |
@@ -321,7 +322,8 @@ chain succeeds.
defer-the-release branch missed the late-first-decode path); a third (hold
every unreadable CE entry, delete the silent pre-rescue fake, 25 s grace) is
now committed (`4115f1dd`, module `e7ff3bf7`) and under live validation
- (campaign `239..262`; `237-240` green, only an infra push failure at `241a`).
+ (campaign `239..262`; `239-242` green, the only failures being harness pushes
+ at `241a`/`243a`, retried; `243b` live at the 10:38 check).
Residual classes: late-decode corruption (~3-5%), early UVM channel-manager
stall (~5-10%), guest init panic (~1/10, retried).
- `MS-20261004-04` **First green end-to-end guest CUDA acceptance (2026-10-04