diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-04 16:19:31 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-04 16:19:31 -0400 |
| commit | aa3520f252182aaf64ea0b8856a3ac5e4466ce0e (patch) | |
| tree | f5382da924162e6b0705c2eeff60361cf3911cda /docs/03-architecture | |
| parent | 5d305e94d028f14edffa042f006770494fda1d86 (diff) | |
| download | open-iov-aa3520f252182aaf64ea0b8856a3ac5e4466ce0e.tar.gz open-iov-aa3520f252182aaf64ea0b8856a3ac5e4466ce0e.zip | |
vgsp: end-to-end CUDA acceptance in the guest
The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.
Two data-path correctness fixes make the transfers sound:
- Once-only op execution per GPFIFO entry. The pump replays recently
consumed ring entries when translation batches are injected so missed
releases can land; re-executing a CE copy is not idempotent because
its source is a client staging buffer that may have been refilled.
Slots now record their raw entry words and a replayed entry matching
that identity suppresses operations while still processing releases.
- Per-page translation of virtual operands. svc_copy resolved only the
first page of a VA range and walked the PA linearly; client staging
ranges are frequently split (observed 195 + 61 pages), so the tail of
alternating chunks was read from unrelated memory. Multi-page copies
now re-resolve each 4 KB page through the trusted resolver.
Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).
Diffstat (limited to 'docs/03-architecture')
| -rw-r--r-- | docs/03-architecture/06-host-module-and-hybrid-model.md | 33 |
1 files changed, 33 insertions, 0 deletions
diff --git a/docs/03-architecture/06-host-module-and-hybrid-model.md b/docs/03-architecture/06-host-module-and-hybrid-model.md index 7f670d7..dbf0c00 100644 --- a/docs/03-architecture/06-host-module-and-hybrid-model.md +++ b/docs/03-architecture/06-host-module-and-hybrid-model.md @@ -224,6 +224,38 @@ and the kernel-side bookkeeping records physical pages without a GPU VA. Until a durable join is implemented, a session can feed verified pairs with the `sema_map_add` debug parameter. +### 4.4 Copy execution rules (correctness invariants) + +Two rules are load-bearing for any client-visible data transfer; both were +established by live differential runs of a device-to-host/host-to-device +pipeline: + +1. **A decoded copy runs at most once per GPFIFO entry.** The service pump + intentionally replays recently consumed ring entries when a new batch of + translations is injected (so releases that previously missed a + translation can land). Release application is forward-progress guarded, + but *copy execution is not idempotent*: a CE copy may read a client + staging buffer that the client has since refilled for the next chunk, so + re-running a consumed copy overwrites finished data with the next chunk's + bytes (observed as a deterministic corruption pattern in the transferred + buffer). The pump therefore records the raw GPFIFO entry words + (`e0 | e1 << 32`) per slot and, when a replayed entry matches the recorded + identity, suppresses its operations while still processing releases. A + slot whose entry words change (a recycled slot in a later epoch) executes + normally. +2. **A virtual operand is translated per 4 KB page.** A GPU VA range is a + list of independent page mappings; resolving only the first page and + walking the physical address linearly is correct only while the mapping + is physically contiguous. Client staging buffers are frequently split + (a 1 MB range was observed as 195 contiguous pages plus 61 pages + elsewhere), so a linear walk reads unrelated memory past the split. Every + multi-page copy translates each page through the same trusted resolver + (pins, learned map, published page directories, targeted resolve) before + moving its 256-byte bounce. + +The same page-granular discipline already applies to the host-side bridge +read/write helpers and to semaphore I/O. + ## 5. The forwarding path (design) ``` @@ -332,6 +364,7 @@ in `kernel-open` first). | Message queues + RPC + all init controls observed from stock 570.144 | implemented | | Guest sysmem access + VA↔PA discovery | implemented (debug-grade, to be productized) | | Push decoding + semaphore completion | demonstrated live; being productized | +| Emulation-mode CUDA end-to-end (host bridge executes client kernels on the physical GPU) | verified live and repeatable: mail kernel bytes authored by GPU execution and read back, `vectorAdd` over 1 M elements returning `bad=0` and the reference checksum, deterministic across consecutive supervised cycles (see `docs/04-implementation/01-known-knowns-and-unknowns.md`) | | Guest-RAM physical path | implemented default-off: guest RAM is file-backed shared memory and GPA→HPA goes through its page cache, verified live on the CE | | Page-table population for server-owned ranges | implemented default-off: forwarded CE copies map resolved host pages through module-built GMMU scratch page tables; real guest page tables remain | | GPU work forwarding (GR/CE execution) | CE forwarding, engine-authored release relay and fault relay implemented default-off and verified live; GR execution open | |
