summaryrefslogtreecommitdiffstats
path: root/docs/04-implementation
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-06 15:18:39 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-10-06 15:18:39 -0400
commit6b0673a21b1cb5493554c898181fe289355cadc0 (patch)
treebc684bb7e4a9897de2530304f3946c9da00ee941 /docs/04-implementation
parentaf5489f9473cedb146d5a9269d19195e03396d02 (diff)
downloadopen-iov-6b0673a21b1cb5493554c898181fe289355cadc0.tar.gz
open-iov-6b0673a21b1cb5493554c898181fe289355cadc0.zip
docs: add the GR compute-launch frontier page and align execution status across the tree
New page docs/04-implementation/03-execution-frontier.md describes the chain that must fire for a guest CUDA launch (guest push -> GPFIFO/front-end -> inline launch-descriptor install -> engine read/dispatch -> completion), where the leading edge is (the descriptor install path), how we know (front-end vs engine-authored observables and their limits), what is exonerated (context validity at dispatch, destination window mapping, fetch alignment, shared-buffer rule, parenting), the ranked issues (verbatim prelude targets, missing descriptor-virtualization control, sentinel hygiene), live investigation lines and labelled theories. Align every page that still said GR execution is blocked on engine-context validity to the current frontier, and add the new page to the README reading order (row 15, later rows renumbered). Docs only; no code.
Diffstat (limited to 'docs/04-implementation')
-rw-r--r--docs/04-implementation/01-known-knowns-and-unknowns.md21
-rw-r--r--docs/04-implementation/02-practical-challenges-and-adversarial-interop.md7
-rw-r--r--docs/04-implementation/03-execution-frontier.md220
3 files changed, 239 insertions, 9 deletions
diff --git a/docs/04-implementation/01-known-knowns-and-unknowns.md b/docs/04-implementation/01-known-knowns-and-unknowns.md
index 084afc8..881d608 100644
--- a/docs/04-implementation/01-known-knowns-and-unknowns.md
+++ b/docs/04-implementation/01-known-knowns-and-unknowns.md
@@ -28,8 +28,10 @@ listed as an unknown-unknown has a *method* by which we will discover it.
copies are verified byte-for-byte. The execution backend is live: CE-class copies are
forwarded to the physical copy engine (default off) with engine-authored
semaphore releases and a fault relay; the remaining frontier is engine
- execution — GR kernel execution is blocked on GR engine-context validity
- (see §2.1).
+ execution — a forwarded guest launch now reaches the front-end (entry
+ fetched, methods run) but the inline launch-descriptor install has not been
+ observed and no grid has dispatched; context validity at dispatch is ruled
+ out (see §2.1 and [`03-execution-frontier.md`](03-execution-frontier.md)).
* The same guest **boots unmodified**: no guest code, no guest module
parameters are required for the initialization path.
* The host is unmodified in the standalone configuration (distribution NVIDIA
@@ -111,15 +113,20 @@ the physical GPU and run its real GSP firmware as the resource-manager server
(the module becomes the host-side RM client). That backend is now live: CE-class
copies and their releases are forwarded to the physical copy engine (default
off) with a fault relay, while the emulation path remains the control plane.
-What is still open is **GR (kernel) execution**, blocked on GR engine-context
-validity.
+What is still open is **GR (kernel) execution**: a forwarded guest launch now
+reaches the front-end (ring entry fetched, methods run) but the inline
+launch-descriptor install has not been observed and no grid has dispatched —
+context validity at dispatch is ruled out. The chain, evidence and live
+investigation lines are in
+[`03-execution-frontier.md`](03-execution-frontier.md).
**Why:** GR execution is *the* blocker for CUDA kernels, and later for
graphics.
-**Plan:** finish the backend in stages: GR engine-context validity and kernel
-submission; translation of the remaining guest work; completion and fault relay
-for every engine class. Measure each stage against a scorecard: correctness,
+**Plan:** finish the backend in stages: the launch-descriptor install path and
+GR kernel submission (see
+[`03-execution-frontier.md`](03-execution-frontier.md)); translation of the
+remaining guest work; completion and fault relay for every engine class. Measure each stage against a scorecard: correctness,
isolation, latency, maintenance cost across driver versions. Deliverable: a
forwarded `vectorAdd` with verified results. Alternatives (own scheduling
without GSP; vendor vGPU or SR-IOV paths) are analyzed and rejected or
diff --git a/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
index 4b886b3..93245c4 100644
--- a/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
+++ b/docs/04-implementation/02-practical-challenges-and-adversarial-interop.md
@@ -56,8 +56,11 @@ without a methodology is not accepted in this project.
physical GPU (see §3). CE-class copies and their semaphore releases are now
forwarded to the physical copy engine (default off) with a host-side fault
relay, and memsets/copies/releases the host decoder understands still run in
- software as a fail-closed correctness bridge. No CUDA kernel is forwarded
- yet: GR kernel execution is blocked on GR engine-context validity.
+ software as a fail-closed correctness bridge. GR (kernel) execution is the
+ open frontier: a forwarded guest launch reaches the front-end but the inline
+ launch-descriptor install has not been observed and no grid has dispatched
+ (context validity at dispatch is ruled out — see
+ [`03-execution-frontier.md`](03-execution-frontier.md)).
## 3. Forwarding: the options and the chosen mechanism
diff --git a/docs/04-implementation/03-execution-frontier.md b/docs/04-implementation/03-execution-frontier.md
new file mode 100644
index 0000000..f5546ca
--- /dev/null
+++ b/docs/04-implementation/03-execution-frontier.md
@@ -0,0 +1,220 @@
+# The Execution Frontier: The GR Compute-Launch Chain
+
+> **Status: 2026-10-06.** This page describes the current leading edge of
+> guest-facing execution: the chain that must fire for a stock guest CUDA
+> kernel to run on the physical GPU, what we have proven about it, where it
+> currently stops, how we know, and the open lines of investigation and
+> theories. It is evidence-led: every statement carries either a live
+> observation, a decompilation finding, or a clearly labelled theory.
+>
+> Evidence sources for this page: supervised physical cycles (module-owned
+> GSP-RM on the reference TU104), offline firmware emulation/probing, the host
+> client's driver sources for the supported version, and decompilation of the
+> shipped firmware. Fidelity rules apply: emulator output is
+> instruction/memory-level (L0/L1); no timing or device-environment claims are
+> made from it.
+
+---
+
+## 1. The chain that should fire
+
+A guest CUDA launch becomes a **method stream** in a push buffer, referenced by
+a **GPFIFO ring entry**, announced by a **doorbell**. The host emulation decodes
+it and forwards an equivalent method stream to a real channel owned by the
+module's resource-manager session. On the hardware side the **front-end**
+(PBDMA) fetches and executes the methods, the launch descriptor (**QMD**) is
+materialised and read by the **GR engine**, the grid is dispatched to the SMs,
+and completion is published as **semaphores** the guest waits on.
+
+```mermaid
+flowchart TB
+ A["Guest: CUDA builds launch<br/>program, parameters, QMD, method stream<br/>GPFIFO entry + doorbell"] --> B
+ B["Host: doorbell trap, decode,<br/>rebuild forward stream,<br/>prewrite QMD, ring/USERD, doorbell"] --> C
+ C["Front-end: fetch GPFIFO entry<br/>(ring cursor advances)"] --> D
+ D["Front-end: execute methods<br/>object bind, cache ops, report sem"] --> E
+ E["Front-end: install inline QMD<br/>into the descriptor buffer"] --> F
+ F["GR engine: read QMD through<br/>the channel's page tables"] --> G
+ G["Context switch loads channel context<br/>(FECS/ucode)"] --> H
+ H["GR: dispatch grid to SMs"] --> I
+ I["Engine authors release semaphore<br/>+ channel semaphore"] --> J
+ J["Host polls release, relays to guest;<br/>CUDA wakes"]
+```
+
+| # | Stage | State (2026-10-06) |
+|---|---|---|
+| 1 | Guest builds launch + method stream | ✅ observed (stock driver, unmodified) |
+| 2 | GPFIFO entry + doorbell | ✅ observed |
+| 3 | Host decode + forward stream + QMD prewrite + ring/USERD | ✅ observed (forwarding path) |
+| 4 | Front-end fetches the ring entry | ✅ observed (ring cursor advances) |
+| 5 | Front-end executes methods | ✅ observed on the ladder channel (object bind, cache ops, report semaphore) |
+| 6 | **Inline-QMD install** (descriptor materialised into the QMD buffer) | ✗ **not observed** |
+| 7 | **Engine reads the QMD** | ✗ **not observed** |
+| 8 | Context switch loads the channel context | ✅ context validity at dispatch is ruled in (instance flag set) — see §4 |
+| 9 | GR dispatches the grid to the SMs | ✗ never observed on this stack |
+| 10 | Engine authors release + channel semaphore | ✗ absent for forwarded launches |
+| 11 | Host relays completion; guest wakes | ✗ guest waits and times out |
+
+**One-line summary:** the guest, the emulation, the forwarding path and the
+front-end are live; the **launch-descriptor install path** (stages 6–7) is the
+leading edge, and with it stages 9–11.
+
+---
+
+## 2. Where the leading edge is
+
+A stock guest program now reaches its kernel launch and submits it through the
+forwarding path. On the wire we observe the ring entry consumed and the
+front-end methods run — but the launch descriptor is never materialised, the
+engine never reads a QMD, no engine-authored completion appears, and the guest
+blocks in its synchronisation wait.
+
+This is **not** a context-validity wall (that was the previous frontier and is
+now ruled out at dispatch, §4), and it is **not** a window-mapping wall for the
+descriptor destination (that install is accepted). The wall is the **form of
+the launch method stream**: what the front-end requires in order to install the
+inline descriptor, and what the engine requires in order to read and dispatch
+it.
+
+---
+
+## 3. How we know (observables and their limits)
+
+| Observable | What it proves | What it does **not** prove |
+|---|---|---|
+| Ring entry consumed; ring cursor advances | The front-end is alive and fetching | Nothing about the engine |
+| Method execution on the ladder channel (report semaphore, channel semaphore) | The front-end can execute the method vocabulary | **Not** engine execution — these signals can be front-end-authored |
+| Descriptor buffer prewritten by the host with a detectable sentinel | Whether the descriptor was materialised/modified after submission | A read does not modify memory; the sentinel detects installs, not reads |
+| Install sentinel unchanged after submit | The inline-install method did not run | It does not say why (mapping, required control method, or ordering) |
+| Engine-authored release word absent; completion poll times out | The engine did not complete the launch | It does not say whether the engine started |
+| No GR fault recorded | The engine did not raise an error | Silence is not health: a skipped or never-reached launch is silent by construction |
+| Channel instance context flag set | The channel is context-valid at dispatch | Nothing about the descriptor path |
+
+**Consequence:** progress on this frontier must be measured by an
+**engine-authored** signal (the release word advancing) — not by front-end
+signals, which we have already learned to distinguish.
+
+---
+
+## 4. What is exonerated (do not relitigate)
+
+* **Context validity at dispatch** — the channel's context is valid when the
+ launch is submitted (instance flag set). The earlier "engine context never
+ committed" wall is a historical stage.
+* **The descriptor destination window mapping** — the destination window is
+ installed in the engine's table view at submit (`status=0`).
+* **Front-end fetch and control-block alignment** — the entry is consumed with
+ the host control block at offset 0; index semantics are not applied to a
+ host-backed userd descriptor.
+* **The shared context-buffer one-shot rule** — the refusal seen on one
+ channel family was traced to a shared per-GPU global buffer already
+ initialised in the same cycle; it is not the dispatch delta.
+* **Parenting/TSG for the fetch** — both parenting variants fetch identically.
+* **The copy-engine path** — CE-class work, its releases and its fault relay
+ are live (default off); the GR path is a separate problem.
+
+---
+
+## 5. The clear issues (ranked)
+
+**Issue A — the forwarded stream carries a verbatim guest prelude that cannot
+complete.**
+The forwarded stream includes a verbatim copy of the guest's prelude whose
+inline-data writes target addresses that are not present in the engine's view
+of the channel's page tables, and the referenced data is not relocated. The
+shorter, proven stream shape fires its tail (channel semaphore) but still does
+not schedule the descriptor.
+*How we know:* the prelude stalls in the forwarded form; the shorter shape's
+tail fires; the descriptor-install sentinel never changes in either case.
+*If fixed:* the install path is reached and we can observe stage 6 directly.
+
+**Issue B — the descriptor-virtualization control method is absent.**
+The method that arms the inline-descriptor install path (the QMD
+virtualization control) is not present in the forwarded stream.
+*How we know:* the method does not appear in the submitted stream, and the
+install sentinel never changes.
+*If fixed:* the front-end should materialise the descriptor — falsifiable in
+one cycle by watching the sentinel and the engine-read signal.
+
+**Issue C — diagnostic hygiene.**
+The descriptor sentinel placed at word 0 overlaps real descriptor fields; it
+must be moved to an unused word so diagnostics do not perturb what they
+measure.
+
+---
+
+## 6. Lines of investigation (live)
+
+1. **Restore the proven launch shape** as the baseline (the shorter stream that
+ fires its tail) and **re-add the virtualization control method** — one
+ variable at a time, with the install sentinel and the engine-read signal as
+ the readout.
+2. **Relocate or drop the verbatim prelude** so the inline-data writes target
+ addresses present in the engine's view, or replace it with the equivalent
+ host-side staging the proven shape already uses.
+3. **Move the sentinel** and re-run to confirm the readout is clean.
+4. **Diff against a known-good reference.** Two ways to obtain one: (a) make
+ our own ladder dispatch a descriptor built from the same source material
+ (in-house reference), or (b) capture a working launch of the stock stack on
+ the same GPU (real method stream + descriptor + engine-visible channel
+ state) and compare field by field. Either reference converts the remaining
+ work from inference to differential.
+5. **Offline probing** of any rule the front-end or engine is believed to
+ require, using the firmware emulator/probe tooling before spending a
+ supervised cycle.
+
+---
+
+## 7. Theories (labelled; not yet proven)
+
+* **T1 — the virtualization control is the gate.** The front-end will not
+ materialise an inline descriptor until the virtualization-control method
+ enables the install path; its absence alone explains the inert descriptor.
+* **T2 — materialise-then-read.** The engine only schedules a descriptor that
+ exists in its own view; an install targeting unmapped destinations aborts
+ silently rather than faulting.
+* **T3 — the verbatim prelude poisons the launch.** Even where the tail shape
+ works, the prelude's refused writes leave the launch path in a state that
+ prevents the descriptor from being scheduled.
+* **T4 — relocation is the missing transform.** The guest's inline descriptor
+ is valid, but it is built for the guest's view of memory; the forwarding path
+ must relocate referenced data (and any embedded pointers) into the channel's
+ view before install.
+
+Each theory has a one-cycle falsifier: the install sentinel, the engine-read
+signal, and the engine-authored release, in that order.
+
+---
+
+## 8. Claim discipline
+
+* **Engine-authored release is the only proof of execution.** Front-end
+ signals (ring cursor, method execution, semaphores that the front-end can
+ author) are necessary but not sufficient.
+* **Silence is not health.** No fault recorded does not mean no fault
+ occurred; the current GR path runs without an error-reporting object, which
+ is itself a known observability gap.
+* **Emulator verdicts are L0/L1.** They bound behaviour, never timing or
+ device environment.
+* **A wall is closed only by an acceptance signal**, and the acceptance signal
+ for this frontier is named before the experiment.
+
+---
+
+## 9. Where the rest of the knowledge lives
+
+| Topic | Document |
+|---|---|
+| Honest known-knowns / known-unknowns (the execution gate) | [`01-known-knowns-and-unknowns.md`](01-known-knowns-and-unknowns.md) |
+| The physical backend lifecycle and guest wiring (init → work → completion → teardown) | [`../03-architecture/11-physical-backend-guest-wiring.md`](../03-architecture/11-physical-backend-guest-wiring.md) |
+| Channels, GPFIFO, USERD, runlist, GR and QMD concepts | [`../03-architecture/05-gmmu-amap-hal-gr.md`](../03-architecture/05-gmmu-amap-hal-gr.md) |
+| Interaction and scheduling model (guest ↔ host ↔ engine) | [`../03-architecture/10-interaction-and-scheduling.md`](../03-architecture/10-interaction-and-scheduling.md) |
+| Supervised-cycle mechanics and hazard discipline | [`../eng-ops.md`](../eng-ops.md), [`../RUNBOOK.md`](../RUNBOOK.md) |
+| Offline firmware emulation/debugger | [`../05-operations/03-risc-v-gsp-rm-vm.md`](../05-operations/03-risc-v-gsp-rm-vm.md), [`../05-operations/04-risc-v-gsp-rm-debugger-user-guide.md`](../05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) |
+
+---
+
+## 10. Change log
+
+| Date | Change |
+|---|---|
+| 2026-10-06 | Created: the GR compute-launch chain, the leading edge (inline-descriptor install), evidence limits, exonerated walls, ranked issues, live investigation lines and theories. |