diff options
Diffstat (limited to 'docs/04-implementation/03-execution-frontier.md')
| -rw-r--r-- | docs/04-implementation/03-execution-frontier.md | 220 |
1 files changed, 220 insertions, 0 deletions
diff --git a/docs/04-implementation/03-execution-frontier.md b/docs/04-implementation/03-execution-frontier.md new file mode 100644 index 0000000..f5546ca --- /dev/null +++ b/docs/04-implementation/03-execution-frontier.md @@ -0,0 +1,220 @@ +# The Execution Frontier: The GR Compute-Launch Chain + +> **Status: 2026-10-06.** This page describes the current leading edge of +> guest-facing execution: the chain that must fire for a stock guest CUDA +> kernel to run on the physical GPU, what we have proven about it, where it +> currently stops, how we know, and the open lines of investigation and +> theories. It is evidence-led: every statement carries either a live +> observation, a decompilation finding, or a clearly labelled theory. +> +> Evidence sources for this page: supervised physical cycles (module-owned +> GSP-RM on the reference TU104), offline firmware emulation/probing, the host +> client's driver sources for the supported version, and decompilation of the +> shipped firmware. Fidelity rules apply: emulator output is +> instruction/memory-level (L0/L1); no timing or device-environment claims are +> made from it. + +--- + +## 1. The chain that should fire + +A guest CUDA launch becomes a **method stream** in a push buffer, referenced by +a **GPFIFO ring entry**, announced by a **doorbell**. The host emulation decodes +it and forwards an equivalent method stream to a real channel owned by the +module's resource-manager session. On the hardware side the **front-end** +(PBDMA) fetches and executes the methods, the launch descriptor (**QMD**) is +materialised and read by the **GR engine**, the grid is dispatched to the SMs, +and completion is published as **semaphores** the guest waits on. + +```mermaid +flowchart TB + A["Guest: CUDA builds launch<br/>program, parameters, QMD, method stream<br/>GPFIFO entry + doorbell"] --> B + B["Host: doorbell trap, decode,<br/>rebuild forward stream,<br/>prewrite QMD, ring/USERD, doorbell"] --> C + C["Front-end: fetch GPFIFO entry<br/>(ring cursor advances)"] --> D + D["Front-end: execute methods<br/>object bind, cache ops, report sem"] --> E + E["Front-end: install inline QMD<br/>into the descriptor buffer"] --> F + F["GR engine: read QMD through<br/>the channel's page tables"] --> G + G["Context switch loads channel context<br/>(FECS/ucode)"] --> H + H["GR: dispatch grid to SMs"] --> I + I["Engine authors release semaphore<br/>+ channel semaphore"] --> J + J["Host polls release, relays to guest;<br/>CUDA wakes"] +``` + +| # | Stage | State (2026-10-06) | +|---|---|---| +| 1 | Guest builds launch + method stream | ✅ observed (stock driver, unmodified) | +| 2 | GPFIFO entry + doorbell | ✅ observed | +| 3 | Host decode + forward stream + QMD prewrite + ring/USERD | ✅ observed (forwarding path) | +| 4 | Front-end fetches the ring entry | ✅ observed (ring cursor advances) | +| 5 | Front-end executes methods | ✅ observed on the ladder channel (object bind, cache ops, report semaphore) | +| 6 | **Inline-QMD install** (descriptor materialised into the QMD buffer) | ✗ **not observed** | +| 7 | **Engine reads the QMD** | ✗ **not observed** | +| 8 | Context switch loads the channel context | ✅ context validity at dispatch is ruled in (instance flag set) — see §4 | +| 9 | GR dispatches the grid to the SMs | ✗ never observed on this stack | +| 10 | Engine authors release + channel semaphore | ✗ absent for forwarded launches | +| 11 | Host relays completion; guest wakes | ✗ guest waits and times out | + +**One-line summary:** the guest, the emulation, the forwarding path and the +front-end are live; the **launch-descriptor install path** (stages 6–7) is the +leading edge, and with it stages 9–11. + +--- + +## 2. Where the leading edge is + +A stock guest program now reaches its kernel launch and submits it through the +forwarding path. On the wire we observe the ring entry consumed and the +front-end methods run — but the launch descriptor is never materialised, the +engine never reads a QMD, no engine-authored completion appears, and the guest +blocks in its synchronisation wait. + +This is **not** a context-validity wall (that was the previous frontier and is +now ruled out at dispatch, §4), and it is **not** a window-mapping wall for the +descriptor destination (that install is accepted). The wall is the **form of +the launch method stream**: what the front-end requires in order to install the +inline descriptor, and what the engine requires in order to read and dispatch +it. + +--- + +## 3. How we know (observables and their limits) + +| Observable | What it proves | What it does **not** prove | +|---|---|---| +| Ring entry consumed; ring cursor advances | The front-end is alive and fetching | Nothing about the engine | +| Method execution on the ladder channel (report semaphore, channel semaphore) | The front-end can execute the method vocabulary | **Not** engine execution — these signals can be front-end-authored | +| Descriptor buffer prewritten by the host with a detectable sentinel | Whether the descriptor was materialised/modified after submission | A read does not modify memory; the sentinel detects installs, not reads | +| Install sentinel unchanged after submit | The inline-install method did not run | It does not say why (mapping, required control method, or ordering) | +| Engine-authored release word absent; completion poll times out | The engine did not complete the launch | It does not say whether the engine started | +| No GR fault recorded | The engine did not raise an error | Silence is not health: a skipped or never-reached launch is silent by construction | +| Channel instance context flag set | The channel is context-valid at dispatch | Nothing about the descriptor path | + +**Consequence:** progress on this frontier must be measured by an +**engine-authored** signal (the release word advancing) — not by front-end +signals, which we have already learned to distinguish. + +--- + +## 4. What is exonerated (do not relitigate) + +* **Context validity at dispatch** — the channel's context is valid when the + launch is submitted (instance flag set). The earlier "engine context never + committed" wall is a historical stage. +* **The descriptor destination window mapping** — the destination window is + installed in the engine's table view at submit (`status=0`). +* **Front-end fetch and control-block alignment** — the entry is consumed with + the host control block at offset 0; index semantics are not applied to a + host-backed userd descriptor. +* **The shared context-buffer one-shot rule** — the refusal seen on one + channel family was traced to a shared per-GPU global buffer already + initialised in the same cycle; it is not the dispatch delta. +* **Parenting/TSG for the fetch** — both parenting variants fetch identically. +* **The copy-engine path** — CE-class work, its releases and its fault relay + are live (default off); the GR path is a separate problem. + +--- + +## 5. The clear issues (ranked) + +**Issue A — the forwarded stream carries a verbatim guest prelude that cannot +complete.** +The forwarded stream includes a verbatim copy of the guest's prelude whose +inline-data writes target addresses that are not present in the engine's view +of the channel's page tables, and the referenced data is not relocated. The +shorter, proven stream shape fires its tail (channel semaphore) but still does +not schedule the descriptor. +*How we know:* the prelude stalls in the forwarded form; the shorter shape's +tail fires; the descriptor-install sentinel never changes in either case. +*If fixed:* the install path is reached and we can observe stage 6 directly. + +**Issue B — the descriptor-virtualization control method is absent.** +The method that arms the inline-descriptor install path (the QMD +virtualization control) is not present in the forwarded stream. +*How we know:* the method does not appear in the submitted stream, and the +install sentinel never changes. +*If fixed:* the front-end should materialise the descriptor — falsifiable in +one cycle by watching the sentinel and the engine-read signal. + +**Issue C — diagnostic hygiene.** +The descriptor sentinel placed at word 0 overlaps real descriptor fields; it +must be moved to an unused word so diagnostics do not perturb what they +measure. + +--- + +## 6. Lines of investigation (live) + +1. **Restore the proven launch shape** as the baseline (the shorter stream that + fires its tail) and **re-add the virtualization control method** — one + variable at a time, with the install sentinel and the engine-read signal as + the readout. +2. **Relocate or drop the verbatim prelude** so the inline-data writes target + addresses present in the engine's view, or replace it with the equivalent + host-side staging the proven shape already uses. +3. **Move the sentinel** and re-run to confirm the readout is clean. +4. **Diff against a known-good reference.** Two ways to obtain one: (a) make + our own ladder dispatch a descriptor built from the same source material + (in-house reference), or (b) capture a working launch of the stock stack on + the same GPU (real method stream + descriptor + engine-visible channel + state) and compare field by field. Either reference converts the remaining + work from inference to differential. +5. **Offline probing** of any rule the front-end or engine is believed to + require, using the firmware emulator/probe tooling before spending a + supervised cycle. + +--- + +## 7. Theories (labelled; not yet proven) + +* **T1 — the virtualization control is the gate.** The front-end will not + materialise an inline descriptor until the virtualization-control method + enables the install path; its absence alone explains the inert descriptor. +* **T2 — materialise-then-read.** The engine only schedules a descriptor that + exists in its own view; an install targeting unmapped destinations aborts + silently rather than faulting. +* **T3 — the verbatim prelude poisons the launch.** Even where the tail shape + works, the prelude's refused writes leave the launch path in a state that + prevents the descriptor from being scheduled. +* **T4 — relocation is the missing transform.** The guest's inline descriptor + is valid, but it is built for the guest's view of memory; the forwarding path + must relocate referenced data (and any embedded pointers) into the channel's + view before install. + +Each theory has a one-cycle falsifier: the install sentinel, the engine-read +signal, and the engine-authored release, in that order. + +--- + +## 8. Claim discipline + +* **Engine-authored release is the only proof of execution.** Front-end + signals (ring cursor, method execution, semaphores that the front-end can + author) are necessary but not sufficient. +* **Silence is not health.** No fault recorded does not mean no fault + occurred; the current GR path runs without an error-reporting object, which + is itself a known observability gap. +* **Emulator verdicts are L0/L1.** They bound behaviour, never timing or + device environment. +* **A wall is closed only by an acceptance signal**, and the acceptance signal + for this frontier is named before the experiment. + +--- + +## 9. Where the rest of the knowledge lives + +| Topic | Document | +|---|---| +| Honest known-knowns / known-unknowns (the execution gate) | [`01-known-knowns-and-unknowns.md`](01-known-knowns-and-unknowns.md) | +| The physical backend lifecycle and guest wiring (init → work → completion → teardown) | [`../03-architecture/11-physical-backend-guest-wiring.md`](../03-architecture/11-physical-backend-guest-wiring.md) | +| Channels, GPFIFO, USERD, runlist, GR and QMD concepts | [`../03-architecture/05-gmmu-amap-hal-gr.md`](../03-architecture/05-gmmu-amap-hal-gr.md) | +| Interaction and scheduling model (guest ↔ host ↔ engine) | [`../03-architecture/10-interaction-and-scheduling.md`](../03-architecture/10-interaction-and-scheduling.md) | +| Supervised-cycle mechanics and hazard discipline | [`../eng-ops.md`](../eng-ops.md), [`../RUNBOOK.md`](../RUNBOOK.md) | +| Offline firmware emulation/debugger | [`../05-operations/03-risc-v-gsp-rm-vm.md`](../05-operations/03-risc-v-gsp-rm-vm.md), [`../05-operations/04-risc-v-gsp-rm-debugger-user-guide.md`](../05-operations/04-risc-v-gsp-rm-debugger-user-guide.md) | + +--- + +## 10. Change log + +| Date | Change | +|---|---| +| 2026-10-06 | Created: the GR compute-launch chain, the leading edge (inline-descriptor install), evidence limits, exonerated walls, ranked issues, live investigation lines and theories. | |