diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/21-CI-RELEASE-PIPELINE.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/21-CI-RELEASE-PIPELINE.md | 320 |
1 files changed, 320 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/21-CI-RELEASE-PIPELINE.md b/debug/risc-v-gsp-rm-vm/docs/21-CI-RELEASE-PIPELINE.md new file mode 100644 index 0000000..d687cca --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/21-CI-RELEASE-PIPELINE.md @@ -0,0 +1,320 @@ +# 21 — CI and release pipeline (version diff + evidence bundle) + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status. + +## 1. Purpose + +This is the release-triage path for a new GSP firmware image (roadmap R6 / +H4.1, docs/23 C5/C7). Two stdlib-only tools turn the manual docs/07 P1/P2 +playbook into one repeatable run and one attachable bundle: + +| Tool | What it does | +|---|---| +| `harness/version_diff.py` | old vs new target, section by section: image/shape, symbol inventory, decompiled corpus hints, bounded QEMU probe, shared-scenario behavior, M1–M6 skeleton | +| `harness/ci_release.py` | runs version_diff → portability matrix → mutation smoke and hashes every artifact into `bundle-manifest.json` | + +`.github/workflows/greeneye.yml` runs the tool's test suite on pushes/PRs that +touch `debug/risc-v-gsp-rm-vm/**` or `tools/rm*/**`. The repository-level +`.github/workflows/ci.yml` is independent and unchanged. + +Everything here is local: the tools never download firmware, never attach to +hardware, a guest or a kernel module, and use their own QEMU processes. + +--- + +## 2. Version diff + +### 2.1 Usage + +Self-diff of the demo (a determinism/regression check; must be all-zero): + +```bash +python3 harness/version_diff.py \ + --old firmware/uproc_demo/build/uproc_demo.elf \ + --new firmware/uproc_demo/build/uproc_demo.elf \ + --out dumps/vdiff-selfcheck \ + --probe-instructions 200 --behavior-duration 0.2 +``` + +Old vs new firmware with per-version symbol DBs and Ghidra exports: + +```bash +python3 harness/version_diff.py \ + --old firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --new firmware/57X-GSP/570.211/gsp-570.211.fwimage.bin \ + --old-label 570.144 --new-label 570.211 \ + --old-ghidra-exports analysis/570.144/symbols.tsv \ + --new-ghidra-exports analysis/570.211/symbols.tsv \ + --corpus-diff analysis/570.144/decompiled_index.tsv analysis/570.211/decompiled_index.tsv \ + --out dumps/vdiff-570 +``` + +Attaching a fixed stimulus and a pre-produced corpus report: + +```bash +python3 harness/version_diff.py --old <rehosted-old.elf> --new <rehosted-new.elf> \ + --stimulus examples/scripts/trace_rpc.py \ + --corpus-report run/corpus-delta-570.144-570.211.txt \ + --out dumps/vdiff-rehosted +``` + +Targets are RISC-V ELF files or raw images (`.fwimage`). A raw target is +relocated to `0x80000000` as one implicit load; its firmware-vaddr translation +is `runtime - 0x80000000`. Symbols may be absent for raw images; the symbol +section then reports `skipped` with a reason unless Ghidra/JSON sources are +supplied. + +Flags: `--old/--new` (required), `--out` (required), `--old-label/--new-label/ +--release`, `--old-symbols/--new-symbols` (green-eye JSON), `--old-ghidra-names`, +`--new-ghidra-names`, `--old-ghidra-index`, `--new-ghidra-index`, +`--old-ghidra-exports`, `--new-ghidra-exports` (`--old-ghidra-bias`/ +`--new-ghidra-bias`), `--corpus-diff OLD NEW`, `--corpus-report FILE`, +`--stimulus`, `--probe-instructions` (`0` = `--probe-duration` wall clock), +`--behavior-duration`, `--max-stops`, `--max-rows`, `--approval-token`. + +### 2.2 Sections + +Every section reports `status: ok | skipped | error`, and a skipped section +never aborts the others. A failing QEMU probe or scenario is recorded as +`error` for that section only. + +1. **Image / shape.** Entry, span, file size/hash, PT_LOAD layout (vaddr, + filesz/memsz, flags), and the ELF section table (name/type/addr/size). + Raw images have no section table; that is stated, not guessed. +2. **Symbol inventory.** ELF `.symtab` plus Ghidra `names.tsv`, + `decompiled_index.tsv`/bucket TSVs, 4-column `exports.tsv` and the + 3-column `ExportForGreenEye` form (`addr<TAB>name<TAB>size`), plus + green-eye `greeneye-symbols/1` JSON DBs. Matching follows + `userspace/cuda/tools/diff_corpus.py`: named functions match by name + (duplicates in address order), the address shift is the median of named + deltas, and `FUN_*` functions match at the shifted address. The report + lists added/removed/moved/resized (named and anonymous), the shift and its + dominance, match rates and the named-transfer rate. Merged per-side DBs are + saved under `<out>/symbols/`. +3. **Decompiled-change hints.** `--corpus-diff A B` runs the same matching + discipline over two corpus indexes and writes `<out>/corpus/corpus-diff.json`; + `--corpus-report FILE` attaches a previously produced report (hash + + preview). With neither flag the section is `skipped` with reason + `not supplied: ...`. +4. **Probe first-touch.** Both targets are flattened, relocated to + `0x80000000`, stepped for the bounded budget, and their QEMU logs parsed + with `greeneye.analyze.parse_qemu_faults`. The first fault is reported with + cause/desc, `epc` **and** `epc_firmware`, the access address **and** + `access_firmware` (translated only when the access address falls inside the + image). The diff says whether the stopped PC/first fault changed. +5. **Behavior.** The shared demo scenario from `portability_matrix` (hooks on + every resolvable `rpc_*`, a write watchpoint on `gsp_ctrl.rpc_count`, the + self-test, one deterministic PING) or a `--stimulus` script is run on both + targets and recorded as normal `greeneye-session/1` directories. The report + includes `analyze.compare_sessions` output under `<out>/compare/`, explicit + hit/watch/written-page deltas, `all_zero`, and the docs/06 M1–M6 skeleton + with every agreement metric marked `not measured` (a reference capture is + required, docs/06 §5). + +### 2.3 Output + +```text +<out>/ + version-diff.json stable machine-readable report + version-diff.md rendered report + symbols/<target>.json merged symbol DB per side (when non-empty) + sessions/<target>-probe/ recorded probe session + sessions/<target>-behavior/ recorded scenario session + compare/<old>--<new>.md compare_sessions report + corpus/corpus-diff.json when --corpus-diff is used +``` + +Exit status: `0` for a normal report (skips included), `2` for a tool error, +`3` for an RM 610 gate refusal. + +--- + +## 3. RM 610 gate (hard, testable) + +The gate is enforced by `version_diff` and `ci_release` **before any target +file is opened** and before `--out` is created: + +* a target/release label matching `rm610` / `RM-610` / `rm_610` etc. (the + `--release`, `--old-label`, `--new-label` values and the target basenames) + is refused unless + * `--approval-token RM610-APPROVED` is passed, or + * `GREENEYE_RM610_APPROVED=1` is set; +* the refusal text points at [`docs/23-RM610-READINESS.md`](23-RM610-READINESS.md) + and states that nothing was downloaded and no target file was opened; +* the same check runs in `--dry-run` mode. + +Nothing in either tool performs network access, and `--dry-run` never touches +a target. Current-version rehearsal runs use `570.144`-labelled inputs; 610 +inputs require the §6 approval message in docs/23. + +--- + +## 4. Release pipeline + +### 4.1 Dry run first + +```bash +python3 harness/ci_release.py --old <old> --new <new> --out dumps/release-570 \ + --old-label 570.144 --new-label 570.211 --dry-run +``` + +`--dry-run` prints a deterministic JSON plan (the three step commands and the +artifact paths) and exits `0` without creating `--out` or opening a target. +This is the docs/23 §4 step 6 rehearsal. + +### 4.2 Real run + +```bash +python3 harness/ci_release.py --old <old> --new <new> --out dumps/release-570 \ + --old-label 570.144 --new-label 570.211 \ + --old-symbols symbols-570.144.json --new-symbols symbols-570.211.json \ + --stimulus examples/scripts/trace_rpc.py \ + --mutation-kernel firmware/uproc_demo/build/uproc_demo.elf \ + --mutation-iterations 8 --mutation-seed 7 +``` + +Steps, in order: + +1. **version_diff** — writes `<out>/version-diff/` (section flags above); +2. **portability_matrix** — old and new as two targets, same budgets, writes + `<out>/portability-matrix/portability-report.json|.md` and per-pair compare + reports; the self-comparison must show `all_zero`; +3. **mutation_smoke** — `mutation_harness` on the demo (or `--mutation-kernel`, + with `--allow-foreign-mutation` for a re-hosted ELF) with a small budget; + writes `<out>/mutation-smoke/findings.jsonl|summary.md`. + +A step that raises is recorded as `status: error` and the later steps still +run; the process exits `1` when any step errored, `0` otherwise (`skipped` +steps do not fail the run, but their reasons are in the report). Exit `3` is +the RM 610 refusal. + +`ci_release` flags: the `version_diff` inputs above plus `--matrix-symbols` +(one JSON DB applied to both matrix targets), `--mutation-kernel` (default: +the bundled demo), `--mutation-iterations`/`--mutation-seed`/ +`--mutation-timeout`/`--mutation-settle`, and `--allow-foreign-mutation` for a +non-demo re-hosted ELF. + +### 4.3 Evidence bundle format + +```text +<out>/ + release-report.json pipeline summary (steps, gate, labels) + bundle-manifest.json sha256 + byte size of every other file + version-diff/ docs/21 §2.3 tree + portability-matrix/ portability-report.json|.md, symbols/, sessions/, compare/ + mutation-smoke/ findings.jsonl, summary.md, sessions/ +``` + +`bundle-manifest.json`: + +```json +{ + "format": "greeneye-release-bundle/1", + "tool": "harness/ci_release.py", + "files": [{"path": "version-diff/version-diff.json", "bytes": 12345, "sha256": "..."}] +} +``` + +`release-report.json` shape: + +```json +{ + "format": "greeneye-ci-release/1", + "dry_run": false, + "rm610_gate": {"status": "passed", "labels": ["570.144", "570.211"]}, + "old": {"label": "570.144", "path": "gsp-570.144.fwimage.bin"}, + "new": {"label": "570.211", "path": "gsp-570.211.fwimage.bin"}, + "steps": [{"name": "version_diff", "status": "ok", + "sections": {"image": "ok", "symbols": "ok", "corpus": "ok", + "probe": "ok", "behavior": "ok"}}], + "errors": [], + "manifest": "bundle-manifest.json" +} +``` + +The manifest is written last, so its hashes cover the final release report. +Attach the directory (or its manifest) to the docs/07 evidence checklist and +the docs/14 §6 pack; the docs/23 §4 step 7 expected-artifact list maps 1:1. + +--- + +## 5. CI workflow + +`.github/workflows/greeneye.yml` (new; `ci.yml` is untouched): + +* triggers on push/PR paths `debug/risc-v-gsp-rm-vm/**` and `tools/rm*/**`; +* `ubuntu-latest`, no secrets, no hardware; +* installs `qemu-system-misc clang lld-18`; +* builds `make -C debug/risc-v-gsp-rm-vm sdk demo`; +* runs `python3 -m unittest discover -s tests -t .` from the tool root (QEMU + integration tests run; if QEMU were absent they skip cleanly). + +The intended release gates this workflow covers are the fast inventory smoke +from docs/19 §6 (`control_discovery --max-id 32 --patterns zero`), the harness +suites, and the docs/21 dry-run. A full `ci_release` run against real firmware +is driven manually/agentically because it needs the versioned inputs and +symbol DBs; CI-adjacent automation should call it with `--dry-run` first. + +--- + +## 6. MCP monitoring + +`plugins/mcp_server.py` (docs/20) exposes the same machinery to an agent: +`probe_firmware`, `portability_matrix_run`, `mutation_run`, `compare_sessions` +and `analyze_current`. The agent pattern for a release: + +1. `portability_matrix_run` with the old/new targets and fixed budgets; +2. `mutation_run` on the demo with the pinned seed and budget; +3. `compare_sessions` for the recorded evidence pair; +4. read `version-diff.json` / `release-report.json` / `bundle-manifest.json` + from the run directory (agent reads the bundle, it does not recompute it). + +The MCP server remains read/observe by default and simulator-only (docs/20 §7). + +--- + +## 7. Determinism, limits and safety + +* Fixed probe budgets, fixed scenario phases and the same stimulus make the + self-diff zero: an unchanged target reports zero symbol/probe/behavior + deltas (verified by `tests/test_version_diff.py`). +* Agreement metrics M1–M6 are **not measured** without a reference capture + (docs/06 §5); the skeleton carries local `observed` facts only. L2/L3 claims + (timing, interrupts, device model) are out of scope. +* Raw `.fwimage` targets execute as flat blobs; a firmware linked differently + may fault earlier than on silicon — that is first-touch data, not a boot + (docs/06 §3). +* Only temporary work directories and `<out>/` are written; QEMU is always + launched and killed by the tools; log reads are bounded + (`--log-bytes`). +* No downloads, no `/dev`, no kernel module, no existing gdbstub, no physical + GPU. The 610 gate is part of both CLIs, not documentation-only. + +--- + +## 8. Tests + +```bash +python3 -m unittest tests.test_version_diff -v +make test # includes these with the rest of the suite +``` + +`tests/test_version_diff.py` covers the gate (default refusal, token/env +approval, gate-before-open), the `diff_corpus.py` matching discipline +(moved/resized/added/removed, anonymous shift), the 3-column export parser, +missing/incompatible/raw target handling, corpus hints, `ci_release --dry-run` +side-effect freedom, clean `--help`, and a demo self-diff asserting zero +inventory and behavior deltas. QEMU integration tests skip cleanly when QEMU +or the demo ELF is absent. + +--- + +## 9. Roadmap mapping + +| Item | Where this doc/tool contributes | +|---|---| +| **R6 — multi-version orchestration** | `version_diff` + `ci_release` prefix the matrix with a version-scoped diff | +| **H4.1 — semantic diffing** | symbol/corpus inventory deltas and behavior profiles for two firmware versions | +| **V1 — metrics/golden data** | deterministic self-diff and stable JSON reports support golden comparisons | +| **docs/23 C5/C7** | release pipeline implemented and exercisable; dry-run rehearsal path | +| **H7 — agentic harness** | self-describing bundle + docs/20 MCP tools for monitored runs | |