summaryrefslogtreecommitdiffstats
path: root/docs/03-architecture
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-04 16:19:31 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-04 16:19:31 -0400
commitaa3520f252182aaf64ea0b8856a3ac5e4466ce0e (patch)
treef5382da924162e6b0705c2eeff60361cf3911cda /docs/03-architecture
parent5d305e94d028f14edffa042f006770494fda1d86 (diff)
downloadopen-iov-aa3520f252182aaf64ea0b8856a3ac5e4466ce0e.tar.gz
open-iov-aa3520f252182aaf64ea0b8856a3ac5e4466ce0e.zip
vgsp: end-to-end CUDA acceptance in the guest
The no-claim emulation path now runs a stock CUDA program end to end: device properties, cudaMalloc, a GPU-authored string kernel read back byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd executed on the physical GPU through the host bridge, and the device-to-host copy-back verified element by element. Three consecutive supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 + STATUS OK with identical input/output hashes. Two data-path correctness fixes make the transfers sound: - Once-only op execution per GPFIFO entry. The pump replays recently consumed ring entries when translation batches are injected so missed releases can land; re-executing a CE copy is not idempotent because its source is a client staging buffer that may have been refilled. Slots now record their raw entry words and a replayed entry matching that identity suppresses operations while still processing releases. - Per-page translation of virtual operands. svc_copy resolved only the first page of a VA range and walked the PA linearly; client staging ranges are frequently split (observed 195 + 61 pages), so the tail of alternating chunks was read from unrelated memory. Multi-page copies now re-resolve each 4 KB page through the trusted resolver. Also carries the earlier uncommitted emulation fixes that got the client to the launch (QMD address-window gate, counter registration persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon INCHECK/OUTCHECK/dumps).
Diffstat (limited to 'docs/03-architecture')
-rw-r--r--docs/03-architecture/06-host-module-and-hybrid-model.md33
1 files changed, 33 insertions, 0 deletions
diff --git a/docs/03-architecture/06-host-module-and-hybrid-model.md b/docs/03-architecture/06-host-module-and-hybrid-model.md
index 7f670d7..dbf0c00 100644
--- a/docs/03-architecture/06-host-module-and-hybrid-model.md
+++ b/docs/03-architecture/06-host-module-and-hybrid-model.md
@@ -224,6 +224,38 @@ and the kernel-side bookkeeping records physical pages without a GPU VA.
Until a durable join is implemented, a session can feed verified pairs with
the `sema_map_add` debug parameter.
+### 4.4 Copy execution rules (correctness invariants)
+
+Two rules are load-bearing for any client-visible data transfer; both were
+established by live differential runs of a device-to-host/host-to-device
+pipeline:
+
+1. **A decoded copy runs at most once per GPFIFO entry.** The service pump
+ intentionally replays recently consumed ring entries when a new batch of
+ translations is injected (so releases that previously missed a
+ translation can land). Release application is forward-progress guarded,
+ but *copy execution is not idempotent*: a CE copy may read a client
+ staging buffer that the client has since refilled for the next chunk, so
+ re-running a consumed copy overwrites finished data with the next chunk's
+ bytes (observed as a deterministic corruption pattern in the transferred
+ buffer). The pump therefore records the raw GPFIFO entry words
+ (`e0 | e1 << 32`) per slot and, when a replayed entry matches the recorded
+ identity, suppresses its operations while still processing releases. A
+ slot whose entry words change (a recycled slot in a later epoch) executes
+ normally.
+2. **A virtual operand is translated per 4 KB page.** A GPU VA range is a
+ list of independent page mappings; resolving only the first page and
+ walking the physical address linearly is correct only while the mapping
+ is physically contiguous. Client staging buffers are frequently split
+ (a 1 MB range was observed as 195 contiguous pages plus 61 pages
+ elsewhere), so a linear walk reads unrelated memory past the split. Every
+ multi-page copy translates each page through the same trusted resolver
+ (pins, learned map, published page directories, targeted resolve) before
+ moving its 256-byte bounce.
+
+The same page-granular discipline already applies to the host-side bridge
+read/write helpers and to semaphore I/O.
+
## 5. The forwarding path (design)
```
@@ -332,6 +364,7 @@ in `kernel-open` first).
| Message queues + RPC + all init controls observed from stock 570.144 | implemented |
| Guest sysmem access + VA↔PA discovery | implemented (debug-grade, to be productized) |
| Push decoding + semaphore completion | demonstrated live; being productized |
+| Emulation-mode CUDA end-to-end (host bridge executes client kernels on the physical GPU) | verified live and repeatable: mail kernel bytes authored by GPU execution and read back, `vectorAdd` over 1 M elements returning `bad=0` and the reference checksum, deterministic across consecutive supervised cycles (see `docs/04-implementation/01-known-knowns-and-unknowns.md`) |
| Guest-RAM physical path | implemented default-off: guest RAM is file-backed shared memory and GPA→HPA goes through its page cache, verified live on the CE |
| Page-table population for server-owned ranges | implemented default-off: forwarded CE copies map resolved host pages through module-built GMMU scratch page tables; real guest page tables remain |
| GPU work forwarding (GR/CE execution) | CE forwarding, engine-authored release relay and fault relay implemented default-off and verified live; GR execution open |