diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/40-NEW-VERSION-TRIAGE.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/40-NEW-VERSION-TRIAGE.md | 296 |
1 files changed, 296 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/40-NEW-VERSION-TRIAGE.md b/debug/risc-v-gsp-rm-vm/docs/40-NEW-VERSION-TRIAGE.md new file mode 100644 index 0000000..613e966 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/40-NEW-VERSION-TRIAGE.md @@ -0,0 +1,296 @@ +# 40 — New-version triage (one command) + +> **Status: in development (2026-10-03).** Implemented, tested and measured on +> the TU104 570.144 image; the measured value proof is section 5. Interfaces +> and report fields may still change. Simulator-only: never touches hardware, +> GPU, mdev, a kernel module or a guest; nothing is downloaded. +> +> **Audience:** agents and maintainers of risc-v-gsp-rm-debugger. +> +> This document is public-safe: it describes components, behaviour and +> measured numbers only. + +--- + +## 1. Why this exists (U46) + +The program has one goal: cut the time from a new NVIDIA RM driver / GSP +firmware release to supported in `vgsp`. Today a new image is triaged by hand +by reading docs/07 P1 and invoking four different tools in sequence (probe, +native boot, A/B, version diff), then collecting session paths and writing a +summary. + +`harness/triage.py` turns that into **one bounded command** that reuses the +existing tools - it does not reimplement any of them: + +| Step | Reuses | Answers | +|---|---|---| +| `identity` | `greeneye.elf`, `sha256` | what is the image, which version, which baseline | +| `probe` | `harness/sim_qemu_line.py:run_qemu_baseline` (the v0.1 probe path) | where does it first fault / first touch under QEMU | +| `native_boot` | `python3 -m sim.gsp_boot` (S1 device model + boot ladder) | how far does the native ladder get (B0–B4) | +| `ab_eval` | `harness/sim_qemu_line.py:run_harness` | QEMU vs native A/B, determinism, frozen thresholds | +| `version_diff` | `harness/version_diff.py` (only with `--baseline`) | image/symbol/corpus/probe/behavior delta, RM 610 gate respected | +| `features` | none (introspection) | hooks/watch/memory/session API, native sim, TCG plugin, MCP, demo | +| `support_readiness` | derived | blockers and ordered next actions for `vgsp` support work | + +Every step is wrapped: a failing step is recorded as `status: error`, the +remaining steps still run, and the report is still written. Every session a +step used is listed with its sha256 hashes. + +--- + +## 2. Usage + +Run from the tool root (`open-iov/debug/risc-v-gsp-rm-vm`): + +```bash +# Plan only (no targets opened, no --out created) +python3 harness/triage.py \ + --firmware ../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --version 570.144 --out dumps/triage-570.144 --dry-run + +# Single new image: identity + probe + bootstrap ladder + A/B + features +python3 harness/triage.py \ + --firmware <new.fwimage.bin> --version <label> --out dumps/triage-<label> + +# With a baseline: adds the full version diff (docs/21) +python3 harness/triage.py \ + --firmware <new.fwimage.bin> --version <new-label> \ + --baseline <old.fwimage.bin> --baseline-label <old-label> \ + --out dumps/triage-<old>-<new> + +# One command via the launcher or make +./risc-v-gsp-rm-debugger.sh triage --firmware <new> --out dumps/triage-new +make triage FW=<new> OUT=dumps/triage-new TRIAGE_ARGS="--baseline <old>" +``` + +Flags: `--firmware` (required), `--baseline`, `--version`, `--baseline-label`, +`--out` (required unless `--dry-run`), `--budget` (native instructions per +attempt, default 200000, hard cap 2000000), `--qemu-duration` (default 0.5s), +`--step-timeout` (per-step wall-clock net, default 300s), `--release`, +`--approval-token`, `--dry-run`. + +Safety: `--dry-run` opens no target and creates no `--out`. The RM 610 gate +runs first in every mode (docs/23) and refuses with exit `3`; labels checked +are `--release`, `--version`, `--baseline-label` and both file basenames. +Exit codes: `0` report written (skipped steps included), `1` at least one step +errored, `2` tool/input error (or firmware absent), `3` RM 610 refusal. + +Output tree under `--out`: + +```text +<out>/ + triage-report.json stable machine-readable report + triage-report.md rendered report + sessions/qemu-probe/ QEMU probe session + sessions/native-boot/ S1 boot-ladder session + ab-eval/ docs/37 A/B report + sim-qemu-line.json|.md + sessions/ + version-diff/ only with --baseline (docs/21 tree) +``` + +--- + +## 3. Report schema (`greeneye-new-version-triage/1`) + +```json +{ + "format": "greeneye-new-version-triage/1", + "tool": "harness/triage.py", + "generated_at": "2026-10-03T04:44:24Z", + "rm610_gate": {"status": "passed", "labels": ["570.144"]}, + "input": { + "firmware": {"path": "...", "sha256": "...", "size": 28528288, + "version": "570.144W", "version_source": "gsp_version.txt sidecar"}, + "baseline": null, "baseline_supplied": false, + "budget": 200000, "qemu_duration": 0.5, "triage_command": "..." + }, + "thresholds": {"frozen_on": "2026-10-03", "manual_baseline_s": 1800.0, + "triage_wall_s_max": 600.0}, + "steps": [ + {"name": "identity", "status": "ok", "wall_s": 0.099, "firmware": {...}, "elf": {...}}, + {"name": "probe", "status": "ok", "rung": "pre-B0", "first_fault": {...}, + "stopped_pc": 0, "sessions": [...]}, + {"name": "native_boot", "status": "ok", "rung": "B2", + "instructions_executed": 14682, "stop": {...}}, + {"name": "ab_eval", "status": "ok", "native_rung": "B2", + "native_identical": true, "above_qemu_line": true, "verdict": "above"}, + {"name": "version_diff", "status": "skipped", "reason": "no --baseline supplied"}, + {"name": "features", "status": "ok", "checks": {...}, "unavailable": []} + ], + "sessions": [{"step": "probe", "name": "qemu-probe", "dir": "sessions/qemu-probe", + "present": true, "hashes": {"session.json": "...", "events.jsonl": "...", "summary.md": "..."}}], + "support_readiness": {"verdict": "sim-observable", "baseline_compared": false, + "observable_rung": "B2", "blockers": [], "next_actions": ["..."]}, + "threshold_checks": {"triage_wall_s": 2.781, "triage_wall_s_max": 600.0, + "wall_within_max": true, "manual_baseline_s": 1800.0, + "speedup_x": 647.2, "beats_manual_baseline": true, + "required_steps_missing": []}, + "wall_s": 2.781, + "artifacts": {"report_json": "<out>/triage-report.json", "report_md": "<out>/triage-report.md"} +} +``` + +The `wall_s`/`threshold_checks` values above are the **measured 2026-10-03 +sample** and are host-load dependent, not fixed constants. + +Key fields: + +* `steps[].status` is `ok | skipped | error`; `skipped` carries a reason and + never fails the run. +* `support_readiness.verdict` is `blocked` (identity/step errors), + `first-touch-only` (no runtime observation yet) or `sim-observable` (native + ladder reached `B2`+, the point where the firmware runs past boot and can be + instrumented). +* `support_readiness.next_actions` is the ordered work list for `vgsp` support + (device-model gaps, symbol DBs, A/B above-the-line work, missing inputs). +* `sessions[].hashes` is the docs/07 evidence-checklist hash list; attach it + with the recorded sessions. + +--- + +## 4. Worked example — 570.144 (measured 2026-10-03) + +Commands (real, read-only; `/usr/bin/time` around the wrapper): + +```bash +./risc-v-gsp-rm-debugger.sh triage \ + --firmware ../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --version 570.144 --out dumps/triage-570.144 +``` + +Measured output (trimmed): + +```text +[triage] identity: status=ok +[triage] probe: status=ok reason=ok +[triage] native_boot: status=ok +[triage] ab_eval: status=ok +[triage] version_diff: status=skipped reason=no --baseline supplied +[triage] features: status=ok +[triage] readiness: sim-observable (baseline_compared=False) +[triage] next: native ladder is at B2 (runtime-observable): inspect the milestone/stop with hooks ... +[triage] next: native A/B is not yet above the QEMU line (verdict=above_lenient_only): device model ... +[triage] next: re-run with --baseline <previous.fwimage> for the symbol/behavior delta ... +[triage] wall: 2.781s vs manual baseline 1800.0s (speedup 647.2x) +``` + +The `2.781s`/`647.2x` line is the **measured 2026-10-03 sample** (host-load +dependent, not a fixed constant). + +What the report recorded: + +| Step | Measured | Meaning | +|---|---|---| +| identity | `570.144W` via `gsp_version.txt`, sha256 + 28,528,288 bytes | image paired and hashed before any run | +| probe | QEMU first fault `pc_fw 0x4009014`, load access at `0x2000000000000A00` | matches the docs/37 QEMU reference first touch | +| native_boot | rung **B2**, 14,682 instructions, stop fault `pc_fw 0x400EA0C`, 0 unknown MMIO | S1 ladder is two rungs past the QEMU fault; boot shim consumed, relocation milestone hit | +| ab_eval | native `B2` (14,682 instructions, 0 unknown MMIO), identical across the repeat, `above_qemu_line: true`, verdict `above` | post-S1 re-run (2026-10-03, later): the discovered `sim/profiles` model + shim now satisfy the docs/37 line contract; the pre-S1 run was `B0`/`above_lenient_only` | +| features | greeneye API, native backend, boot ladder, A/B, version_diff, TCG plugin, MCP, demo, QEMU all present | capability surface for the follow-up work | + +Post-S1 re-run (2026-10-03, later; same command, after S1 and the harness +wiring landed): `ab_eval` reports native rung **B2**, all five frozen +thresholds true and `above_qemu_line: true`; the "native A/B is not yet above +the QEMU line" next-action line no longer appears. Wall ≈2.7 s +(host-load dependent). The pre-S1 trace above is kept as the recorded +before/after evidence. + +Baseline variant (same image both sides, an unchanged-version rehearsal with +the docs/21 path): + +```bash +./risc-v-gsp-rm-debugger.sh triage \ + --firmware ../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --baseline ../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --version 570.144 --baseline-label 570.144 --out dumps/triage-570.144-baseline +``` + +Measured (2026-10-03, host-load dependent): **11.388s** internal wall +(11.64s including interpreter startup); +`version_diff` = 8.526s, sections `image/probe/behavior: ok`, +`symbols/corpus: skipped` with reasons; image self-diff `same: true`; 7 session +directories hashed (probe, native boot, 2 A/B native, A/B QEMU, version-diff +probe + behavior). Readiness stayed `sim-observable`, `baseline_compared: +true`, with next actions for the missing symbol/corpus inputs. + +Frozen thresholds (recorded in `harness/triage.py` on 2026-10-03 **before** +the runs, per handoff §14.5): manual baseline `1800s`, triage wall max +`600s`. Result: `wall_within_max: true`, `beats_manual_baseline: true`, +speedup `647x` (single image) / `158x` (with baseline) — both wall times are +**measured 2026-10-03 samples (host-load dependent)**, not fixed constants. The +manual baseline is +a conservative understatement of the steps replaced (read docs/07 P1, choose +budgets, run probe + boot ladder + A/B, run version diff, collect session +hashes and write the summary for one image). + +--- + +## 5. Value proof and its limits + +* The frozen contract is in the report itself (`thresholds`, + `threshold_checks`); the same numbers are restated in the module docstring + and here. Never move them silently. +* The measured numbers above are from this checkout on 2026-10-03; reproduce + with the exact commands in section 4. +* **Conservative baseline:** 1800s is an *estimate of manual effort*, not a + timed human study. The speedup numbers are reported against it, and the + report flags them as such. +* **What the run proves:** one bounded command produces the identity, probe, + boot-ladder, A/B and capability evidence with hashed sessions in seconds - + the mechanical part of docs/07 P1. +* **What it does not prove:** that the remaining human work (interpreting the + deltas, extending the device model, the accuracy loop, the `vgsp` change) is + automated; it is not, and the next-actions list is where it stays explicit. + +--- + +## 6. Release-to-support mapping + +| Triage output | Next workflow (docs) | +|---|---| +| identity + hash + version | docs/07 P1 step 1, docs/22 pairing checks, docs/28 sweep input | +| probe first fault / first touch | docs/07 P1 step 2, docs/24 device map harvest | +| native boot rung + stop | docs/35 §3.3 ladder; the `vgsp` support scope starts at `B2`+ | +| A/B verdict | docs/37 line contract (U50) | +| version_diff sections + symbol/corpus skips | docs/21, docs/14 §6 evidence pack | +| readiness next actions | docs/07 P1.7 acceptance, the actual `vgsp` support work items | +| RM 610 gate | docs/23 (never bypassed) | + +--- + +## 7. Not verified (explicit) + +* **No real 570.211+ new-version pair exists locally**; the cross-version + delta path is exercised only by the self-diff rehearsal in section 4. +* **Symbols and corpus are skipped for raw `.fwimage` targets** (no `.symtab`, + no Ghidra exports supplied): the handler-inventory delta is not measured + until those inputs are supplied. +* **The S4 A/B verdict is `above` on the frozen checks after S1 landed** + (B2, documented device model + shim, deterministic, within the wall + bound); B3/B4 are not reached and triage makes no "beats QEMU" claim beyond + the documented rung contract (`above_lenient_only` was the pre-S1 result). +* **Fidelity:** L0/L1 execution facts only; no L2 timing and no full L3 + device-environment claims (docs/06, docs/36). Cycles are modeled; the boot + shim values are declared assumptions. +* **No physical backend, `/dev`, module, mdev, guest or network access** was + used or is performed by the tool. + +--- + +## 8. Tests + +```bash +python3 -m unittest tests.test_triage -v # 14 tests +make test # full suite +``` + +`tests/test_triage.py` covers the version-label extraction chain, the report +shape and artifact writes, session hashing, partial-failure handling +(injected step failure still yields a report), missing-firmware behaviour +(reported, not raised), budget bounds, RM 610 refusal (exit `3`), dry-run +side-effect freedom, and a tiny-budget integration run against the real +570.144 image that skips cleanly when the image or QEMU is absent. + +<!-- change (2026-10-03): new document recording the one-command new-version +triage workflow (U46), its measured 570.144 value proof and its limits; +scope: docs/40 only; benefit: the release-to-support loop has one documented +entry point and one attachable evidence bundle. --> |