diff options
Diffstat (limited to 'debug/risc-v-gsp-rm-vm/docs/28-PORTING-AUTOMATION.md')
| -rw-r--r-- | debug/risc-v-gsp-rm-vm/docs/28-PORTING-AUTOMATION.md | 319 |
1 files changed, 319 insertions, 0 deletions
diff --git a/debug/risc-v-gsp-rm-vm/docs/28-PORTING-AUTOMATION.md b/debug/risc-v-gsp-rm-vm/docs/28-PORTING-AUTOMATION.md new file mode 100644 index 0000000..4669f93 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/28-PORTING-AUTOMATION.md @@ -0,0 +1,319 @@ +# 28 — Porting automation: driver-version sweep and new-device scaffold + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status and +> [`GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md) H4/R6 for the program. + +Two stdlib-only orchestrators turn the docs/07 P1/P2 playbooks and the +docs/22 porting guide into one repeatable chain: a **driver-version sweep** +(`harness/port_driver_versions.py`) and a **device-port scaffold** +(`harness/port_new_device.py`). They reuse the version-diff, portability-matrix +and mutation machinery already documented in docs/18 and docs/21, add the +RM 610 gate verbatim from `version_diff.py`, and never download firmware, +never touch hardware, and write only under `--out`. Both are **unadmitted +pilots** under `eng-ops.md` §2 — the keep/retire evaluation is F2/F3 in +[`future-eng-ops.md`](../../../docs/future-eng-ops.md), and a sweep is not +routine release tooling until that record exists. + +| Tool | Roadmap | What it does | +|---|---|---| +| `harness/port_driver_versions.py` | **R6**, H4.1 | Finds each driver/firmware version's local image, runs version_diff vs a baseline, the portability matrix and a mutation smoke, and assembles per-version evidence bundles plus a cross-version summary with the docs/06 M1–M6 skeleton. | +| `harness/port_new_device.py` | **R6**, H4.1, docs/07 P2 | Creates/uses a user-supplied device profile, probes the device firmware's first touch, builds a symbol inventory, runs the matrix against a reference, and emits a port checklist with a go/no-go recommendation. | + +Both are dry-run by default, resume-safe (sweep), bounded by QEMU budgets plus +a per-step wall-clock net, and simulator-only (no `/dev`, kernel module, mdev, +guest or physical GPU). + +--- + +## 1. The chain + +```text +new driver / GSP image lands on disk (runfile, .fwimage, re-hosted ELF) + | + +-- port_driver_versions --dry-run plan: inputs found/missing, steps, artifacts + | | + | +-- --run per version: + | 1. target discovery/identification (never downloads) + | 2. version_diff vs the chosen baseline + | 3. portability_matrix (baseline + candidate) + | 4. mutation_smoke (bounded, seeded) + | 5. per-version bundle-manifest.json + | +-- sweep-summary.json/.md (M1-M6 skeleton, pass/fail interpretation) + | + +-- port_new_device --dry-run plan: profile fields, firmware, steps + | + +-- --run profile + probe-first-touch + symbols + matrix + checklist +``` + +The chain is the automation of docs/07 P1 (new GSP version), P2 (same version, +new device) and the docs/22 port checklists; the metric contract stays +docs/06 §5. Evidence is always attachable to the docs/07 checklist and the +docs/14 §6 pack. + +--- + +## 2. Driver-version sweep + +### 2.1 Known branch cadence + +The default version list documents the branch cadence and where to edit it — +the tool never invents an image or downloads one: + +| branch | expected image | default sweep | +|---|---|---| +| 540 | earlier branch | yes | +| 550 | earlier branch | yes | +| 560 | earlier branch | yes | +| 570 | current anchor (570.144 and patches) | yes | +| 580 | expected cadence, no local image assumed | no | +| 610 | RM 610 gate, refused without approval (docs/23) | no | + +Edit `DEFAULT_VERSIONS` in `harness/port_driver_versions.py`, or override on +the command line / in a config file. A branch with no local image is reported +`no-input`; it is not an error and nothing is fetched. + +### 2.2 Invocation + +Dry-run first (the default; reads input directories only): + +```bash +cd debug/risc-v-gsp-rm-vm +python3 harness/port_driver_versions.py --input-dir ../../firmware/57X-GSP \ + --versions 540,550,560,570 +``` + +Execute, resume-safe, with explicit targets and per-version symbol DBs: + +```bash +python3 harness/port_driver_versions.py --run \ + --versions 570.144 \ + --target 570.144=../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --baseline 570.144 \ + --symbols 570.144=symbols-570.144.json \ + --out dumps/sweep-570.144 +``` + +Cron (weekly, quiet, resume-safe; completed versions are skipped): + +```cron +17 3 * * 1 cd /path/to/open-iov/debug/risc-v-gsp-rm-vm && \ + python3 harness/port_driver_versions.py --run \ + --input-dir ../../firmware/57X-GSP --versions 540,550,560,570 \ + --out dumps/sweep-570 >> dumps/sweep-570/cron.log 2>&1 +``` + +Agent (docs/20): run the dry-run to get the plan, use +`portability_matrix_run`/`mutation_run` for single steps if the sweep itself is +not exposed by MCP yet, then read `sweep-summary.json` and the per-version +`bundle-manifest.json` from the bundle instead of recomputing. The dry-run plan +is machine-readable (`greeneye-port-sweep-plan/1`) and side-effect free. + +Flags that matter: + +* `--versions` / `--config` — explicit list or JSON config (below); +* `--input-dir` — recursive discovery of local `.fwimage`, `.fwimage.bin`, + `.elf`, `.bin` and `.run` files; version matching uses file and directory + names (`570` matches `570.144`); +* `--target VERSION=PATH`, `--symbols VERSION=PATH`, + `--ghidra-exports VERSION=PATH` — explicit inputs, no discovery; +* `--baseline` — version diffed against (default: earliest version found); +* `--extract-runfiles` — opt in to local `sh <runfile> --extract-only` + extraction when only a driver runfile is present (still never downloads); +* `--force` — re-run versions whose `version.json` reports `ok`/`partial`; +* `--step-timeout` — wall-clock safety net per step (`0` disables; QEMU + budgets remain the first bound); +* the usual `version_diff`/`matrix`/`mutation` budgets. + +### 2.3 Configuration file + +```json +{ + "format": "greeneye-port-sweep-config/1", + "versions": ["540", "550", "560", "570.144"], + "baseline": "570.144", + "input_dir": "../../firmware/57X-GSP", + "targets": {"570.144": ["../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin"]}, + "symbols": {"570.144": "symbols-570.144.json"} +} +``` + +Unknown keys are refused so a typo cannot silently change the sweep. CLI flags +override config values. + +### 2.4 Output and evidence bundle + +```text +<out>/ + sweep-summary.json cross-version report (also .md) + bundle-manifest.json sha256 + size of every artifact below + <version>/ + version.json status, inputs, steps, M1-M6, sha256 + version.md rendered per-version report + bundle-manifest.json per-version evidence bundle + version-diff/ docs/21 §2.3 tree + portability-matrix/ portability-report.json|.md, symbols/, sessions/, compare/ + mutation-smoke/ findings.jsonl, summary.md, sessions/ +``` + +`sweep-summary.json` carries per-version status, the cross-version M1–M6 +skeleton (every agreement metric `not measured` without a reference capture, +docs/06 §5), the RM 610 gate record, and an explicit `interpretation` string: +`pass`, `fail` (with the failing versions) or `not-run`. + +### 2.5 Reading failures + +| Status | Meaning | Action | +|---|---|---| +| `ok` | all steps ran, none errored | attach the bundle to the docs/07 checklist | +| `partial` | a step skipped (e.g. no QEMU, no mutation kernel) | explain the skip; re-run where the capability exists | +| `error` | a step raised or timed out | inspect `<version>/version.json`, then the step report (`version-diff.json`, `portability-report.json`, `findings.jsonl`) | +| `no-input` | no local image found for the version | add the image under `--input-dir`, pass `--target`, or drop the version from the list | +| resumed | `version.json` already reported `ok`/`partial` | use `--force` to re-run | + +`--run` exits non-zero when any executed version errored or when nothing ran +at all; skipped steps alone do not fail the sweep, but they are always listed. +An unexplained delta blocks the docs/07 acceptance: an empty +unexplained-delta list is a claim, not a default. + +### 2.6 RM 610 gate (hard, testable) + +Versions numerically `>= 610` or labelled `rm610`/`RM-610` are refused before +any target file is opened — in dry-run too — unless `--approval-token +RM610-APPROVED` is passed or `GREENEYE_RM610_APPROVED=1` is set. The refusal +text points at [`docs/23-RM610-READINESS.md`](23-RM610-READINESS.md) and +states that nothing was downloaded and no target was opened. Do not put a 610 +input in a default list or a cron line before the §6 approval message exists. + +--- + +## 3. New-device scaffold + +### 3.1 Profile + +`--device <name>` uses `--profile FILE` or `<out>/profiles/<name>.json`. A +missing profile is created as a scaffold (`greeneye-device-profile/1`) with the +user-supplied sections and a `where_to_fill` map: + +* `pci` — vendor/device/subsystem IDs (`lspci`, vendor docs); +* `bars` — BAR layout and sizes (guest `lspci -vvv`, host sysfs); +* `fb` — VRAM/BAR1 geometry (host oracle `rmtool mem stats --format=json`); +* `engines` — engine inventory (host oracle `rmtool engine list --format=json`); +* `interrupts` — interrupt topology (MSI-X); +* `rom` — ROM/VBIOS notes and hash; +* `firmware` / `reference` — local image paths (or `--firmware`/`--reference`); +* `firmware_family` / `firmware_version` — the pairing record (docs/07 P2.2); +* `physical_validation` — supervised physical evidence, recorded by the human, + never by this tool. + +Profile data is user-supplied; the tool never guesses a device ID, BAR or +engine list. `config/hardware/*.yaml` remains a design target (see +hardware-software-matrix §6) — do not treat it as live configuration. + +### 3.2 Invocation + +```bash +# dry-run: check profile fields and inputs, create nothing +python3 harness/port_new_device.py --device rtx4090 --profile profiles/rtx4090.json + +# execute: scaffold profile (if absent), probe, symbols, matrix, checklist +python3 harness/port_new_device.py --run --device rtx4090 \ + --firmware ../../firmware/57X-GSP/570.144/gsp-570.144.fwimage.bin \ + --reference firmware/uproc_demo/build/uproc_demo.elf \ + --out dumps/port-rtx4090 +``` + +Output tree: + +```text +<out>/<device>/ + device-port.json / device-port.md + port-checklist.json / port-checklist.md + symbols/<device>.json merged ELF/Ghidra/green-eye DB + provenance + sessions/<firmware>-probe/ recorded first-touch session + portability-matrix/ matrix report + sessions + compare + bundle-manifest.json sha256 of every artifact +<out>/profiles/<device>.json scaffolded profile (when absent) +``` + +The checklist items are the docs/07 evidence checklist plus the docs/22 §3.1 +profile fields; the recommendation is `hold` until the simulator-side items +are complete, `simulator-evidence-complete` while physical validation is +pending, and `go` only when `physical_validation.status` is recorded. The last +row is deliberately outside this tool's reach: simulator-only evidence never +justifies a support claim (docs/22 §3.5, hardware-software-matrix §5.1). + +--- + +## 4. Tests, CI and MCP ties + +```bash +python3 -m unittest tests.test_porting_automation -v +make test # includes these with the rest of the suite +``` + +`tests/test_porting_automation.py` is deterministic: `--help` clean, +dry-runs side-effect free, missing-input handling without tracebacks, resume +detection, config validation, the RM 610 gate (refusal, token, env), and a +tiny one-version QEMU sweep against the demo ELF that must produce the +evidence bundle; QEMU-dependent tests skip cleanly when QEMU or the demo ELF +is absent. The repository CI workflow (docs/21 §5) runs the suite on pushes +and PRs touching the tool tree; a full sweep against real firmware is driven +manually/agentically, dry-run first, because it needs versioned inputs and +symbol DBs — the same policy as `ci_release` (docs/21 §4.1). + +MCP (docs/20) exposes the step-level tools (`probe_firmware`, +`portability_matrix_run`, `mutation_run`); the sweep is the CLI/agent entry +point that sequences them and owns the bundle. An agent should read the dry-run +plan, run, then read `sweep-summary.json`/`device-port.json` and the manifests +rather than recompute hashes. + +--- + +## 5. When to extend + +* **New branch cadence** — add the version to `--versions`/config; if it should + be swept by default, edit `DEFAULT_VERSIONS` and the cadence table above. + Never add an image you do not have; never add 580/610 to a default. +* **New per-version artifact** — add a step function in + `harness/port_driver_versions.py`, register it in `run_version`, and list the + path in `_expected_artifacts`. +* **New device profile field** — add it to `scaffold_profile`, `PROFILE_HELP` + and `profile_missing_fields`, and surface it in the checklist. +* **New architecture (GB10-class, aarch64)** — follow docs/07 P3 first; the + sweep handles the ISA/image shape and the matrix records the first-touch + deltas, but PMU/SOE and partition-model work belongs to the roadmap. +* **New product behaviour** — out of scope: these tools produce evidence, they + never edit `open-iov-vfio`. Product changes need the docs/07 P4 escalation + and a physical reference. + +--- + +## 6. Safety and limits + +* No downloads: discovery and `--extract-runfiles` only read files already on + disk; a driver runfile is never fetched. +* Simulator-only: own QEMU processes on demo/re-hosted ELF or raw firmware; + never a guest, kernel module, mdev, existing gdbstub or physical GPU. +* Bounded: QEMU probe/scenario/mutation budgets plus `--step-timeout`; the + mutation harness refuses non-demo ELFs unless explicitly allowed. +* No support claims: the sweep records what changed; the device checklist + holds until physical validation; the matrix and RUNBOOK process own the + "Supported" label (hardware-software-matrix §1). +* Fidelity limits are docs/06: L0/L1 observations only, L2/L3 (timing, + interrupts, device model) explicitly not claimed. + +## 7. Roadmap mapping + +| Item | Where this doc/tool contributes | +|---|---| +| **R6 — multi-version orchestration** | One sweep over 540→current with version_diff + matrix + mutation and a cross-version report | +| **H4.1 — semantic diffing** | Per-version symbol/corpus/behaviour deltas and recorded sessions | +| **H4.3 — mutation/fuzzing** | Seeded mutation smoke per version | +| **docs/07 P1/P2** | The playbooks become dry-run plans and evidence bundles | +| **docs/23 C5/C7** | Release pipeline extended by a sweep-scale orchestrator; 610 gate reused | +| **H7 — agentic platform** | Machine-readable plans, summaries and bundles for monitored agent runs | + +<!-- change (2026-10-02): state the unadmitted-pilot status and link the +future-eng-ops F2/F3 evaluation from the document that ships the scaffolds; +scope: intro only; benefit: no reader mistakes the sweep for admitted release +tooling. --> |