diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-02 09:13:00 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-02 09:13:00 -0400 |
| commit | 3fe5360d0efc439d59f2b7a9bf2f713aa9cdb918 (patch) | |
| tree | e30e26701f5a67080404e5f9cb71967ee275e604 /debug/risc-v-gsp-rm-vm | |
| parent | 964f34ee2e49d14ce81ee9a6bd9297c88e320c52 (diff) | |
| download | open-iov-3fe5360d0efc439d59f2b7a9bf2f713aa9cdb918.tar.gz open-iov-3fe5360d0efc439d59f2b7a9bf2f713aa9cdb918.zip | |
debug: green-eye wave 2 - injection SDK, Frida-like calls, Ghidra control bridge, long-term docs
Adds to debug/risc-v-gsp-rm-vm/:
- Session.call / run_until: Frida-like remote function calls and run-to-cursor,
with register restoration; verified live (rpc_ping -> 0xaffe).
- symbol+offset target resolution (gsp_ctrl+0x10) across all APIs and the REPL.
- Injection SDK (sdk/) and 'greeneye inject': position-independent RV64 service
blobs (verified zero relocations), loaded into RAM, registered in the demo
handler table, and invoked either directly or through the firmware mailbox
dispatcher. Demo ABI extended: mailbox 16..23 selects slots, arg0 carries the
service command, arg2 added (struct now 0x190; schema updated).
- Ghidra control bridge (greeneye ghidra + GreenEyeControl.java): run-until-here,
breakpoints, memory/register access, remote calls, hit export; verified
end-to-end with Ghidra headless (run_until reached rpc_sum from a GhidraScript).
- MarkGreenEyeHits.java fixed for the Ghidra 12 createBookmark API; verified
headless together with ExportForGreenEye.java.
- Analyzer renders a QEMU first-fault table for probe sessions (fault count,
epc with firmware vaddr translation, access address/size, reason).
- Tests now 35 green, including injection, Ghidra bridge, remote call, run-until
and REPL command smoke tests.
- Documentation: docs/11-16 (GSP RM service developer guide, injection SDK
specification, Ghidra control, adversarial interoperability guide, agentic
GSP IDE, custom simulator design) and GREEN-EYE-ROADMAP.md (long-term H1-H7
program); doc maps and the project RISC-V VM doc updated.
Diffstat (limited to 'debug/risc-v-gsp-rm-vm')
29 files changed, 3337 insertions, 38 deletions
diff --git a/debug/risc-v-gsp-rm-vm/.gitignore b/debug/risc-v-gsp-rm-vm/.gitignore index 3a97916..f8e440c 100644 --- a/debug/risc-v-gsp-rm-vm/.gitignore +++ b/debug/risc-v-gsp-rm-vm/.gitignore @@ -6,3 +6,4 @@ qemu/*.o __pycache__/ *.pyc .run/ +sdk/build/ diff --git a/debug/risc-v-gsp-rm-vm/GREEN-EYE-ROADMAP.md b/debug/risc-v-gsp-rm-vm/GREEN-EYE-ROADMAP.md new file mode 100644 index 0000000..0e1b58e --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/GREEN-EYE-ROADMAP.md @@ -0,0 +1,381 @@ +# GREEN-EYE ROADMAP — long-term program + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** +> This is the long-term plan for green-eye, the runtime analysis, injection and +> measurement environment for the NVIDIA GPU System Processor (GSP) and its +> adjacent microcontrollers. Near-term, committed work lives in +> [`docs/08-ROADMAP.md`](docs/08-ROADMAP.md); this document is the program. + +## 0. Purpose + +Open-IOV's hardest long-term problem is not writing the emulation once — it is +**surviving adversarial firmware change**. NVIDIA ships new GSP images, new +device pairings and new architectures on independent schedules, and the +firmware is deliberately opaque (stripped, signed, hardware-coupled, with +non-recoverable boot states). To keep `open-iov-vfio` accurate across that +churn, the project needs a toolchain that can: + +1. **Run** firmware off-hardware, safely and repeatably. +2. **Observe** it completely — control flow, memory, services, state machines. +3. **Change** it — inject our own services/engines and mutate inputs to learn + boundaries. +4. **Compare** — hardware vs VFIO vs simulation, old vs new firmware, with a + metric contract instead of anecdotes. +5. **Reimplement** — turn observed behaviour into reviewed code in + `open-iov-vfio` with evidence attached. + +green-eye is the vehicle for that program. It is explicitly a **development +environment**, never a shipped component. + +## 1. Starting point (H0, done) + +v0.1 (2026-10-02) provides: GDB-free QEMU RISC-V transport; hooks, +breakpoints, watchpoints; memory/struct/snapshot tooling; recorded sessions +and symbol-annotated analysis; declarative probes; plugins; a Ghidra symbol +and hit bridge; a Ghidra control bridge; a TCG full-trace plugin; a +position-independent service SDK and `greeneye inject`; and the Frida-like +`Session.call` / `run_until` primitives. Fidelity is L0/L1 (execution and +memory); the device environment (L3) is absent. + +## 2. How this roadmap is organised + +Five workstreams run through every horizon: + +* **F — Fidelity**: how much of the real GSP environment the simulator + provides (device model, boot chain, timing). +* **I — Injection**: what code we can add to the running system and how. +* **A — Adversarial navigation**: finding and understanding *unexpected* + firmware behaviour and mapping it to emulation changes. +* **C — Control & tooling**: debugging, Ghidra/agent integration, SDK, + formats, usability. +* **V — Validation**: metrics, benchmarks, golden sessions, gates that stop a + claim from landing without evidence. + +Horizons are numbered H1..H7 with explicit **entry gates** (what must be true +before starting) and **exit criteria** (what must be demonstrated). + +--- + +## H1 — Injection and control (now → weeks) + +**Goal:** make "write a service / change the firmware / prove it" a +first-class loop on the demo uproc and on re-hosted uproc code. + +### H1.1 GSP RM firmware service developer guide (deliverable today) +A written guide (docs/11) covering: RM service anatomy, service entry and +message loop, RM-core calls available to a service, allocation/control +surfaces, inter-service messaging, versioning, and the build/inject/debug +loop. On the demo this maps to the handler-table ABI; on real GSP-RM it maps +to the RM object/control model. + +### H1.2 Injection SDK v1 (deliverable today, demo-proven) +Specification (docs/12): blob format, PIC requirements, ABI, registration, +capability metadata, error model, observability hooks. Implemented in `sdk/` +and `greeneye/inject.py`, verified by direct call and firmware dispatch. + +### H1.3 GSP-RM service injection (needs H2 device model) +Once real GSP-RM runs in the simulator (H2), extend the SDK to: +* build against RM headers or against a green-eye shim for + public work; +* register a service with the simulated RM core instead of a demo handler + table; +* call RM-core entry points (allocation, control, mapping, scheduling) from + the injected code, intercepted by the simulator so calls are logged, + stubbed or forwarded; +* run injected services under the same recording/compare pipeline. +Entry gate: H2.2 (GSP-RM reaches its message loop in the simulator). +Exit: an injected service serves a control end-to-end and its session can be +diffed against hardware. + +### H1.4 Ghidra-driven execution (deliverable today, demo-proven) +`greeneye ghidra` + `GreenEyeControl.java` (docs/13): "run until here", set +breakpoints, read memory/registers, remote-call, export hits. Next +increments: +* watchpoints and structure reads from Ghidra; +* a decompiler-side hit heat-map fed by `greeneye_hits.tsv`; +* bidirectional navigation: clicking a hit opens the decompiled function. + +### H1.5 Debugger parity +Make the REPL/API cover the common GDB vocabulary for firmware work +(watchpoints, catchpoints for traps, thread/core selection, memory region +metadata) while keeping the scripted API as the primary surface. + +--- + +## H2 — Fidelity: platform device model (months) + +**Goal:** move from "executes and faults" to "boots the real GSP image to its +message loop" inside the simulator. + +### H2.1 First-touch register map (started) +The `probe` command already reports the first fault (TU104 570.144: load +access fault at firmware vaddr `0x4009014` reading `0x2000000000000A00`). +Systematically harvest first-touch addresses across versions/devices and +build the initial MMIO map from them. + +### H2.2 QEMU device model +Model, as a QEMU `-device` (or a small set): +* **falcon/SEC2 windows** (cpuctl, dmactl, fbif, imem/dmem, bootvec) with + read-back semantics adequate for boot polling; +* **CSB/TRANSCOM** mailbox windows used by the GSP application; +* **WPR2/FWSEC state** registers (so the image believes it was staged); +* **PTIMER** and basic interrupt routing; +* **DMA/IOMMU** address translation for the firmware's descriptors into + host/FB windows; +* **PMU stub** sufficient for the identity/clock provisioning the GSP app + reads early. +Entry gate: H0 + first-touch data. Exit: the image reaches `appMain` and +blocks on a host message rather than faulting. + +### H2.3 Boot-chain shims +FWSEC/booter/SEC2 are not reasonable to model fully; provide *documented +shims* that place the GSP image in the state the bootloader would have left +it (boot args in a0–a5, MPU/TCM layout, scratch state), so boot behaviour can +be studied without the security chain. Record shim assumptions in session +metadata. + +### H2.4 Multi-core and concurrency (approximate first) +GSP-RM is multi-task; adjacent controllers (SEC2/PMU) are separate cores. +Add deterministic scheduling knobs (instruction budget round-robin, timer +injection) so multi-task interleavings can be explored, replayed and +compared — without claiming real-time behaviour. + +--- + +## H3 — Non-QEMU custom simulator (months; explicitly later) + +**Why:** QEMU is an execution engine, not a workflow engine. It cannot give +us cheap snapshots, deterministic replay with time-travel, in-process +inspection, per-access callback ABI objects, or a device map expressed as +code + data we can version with the firmware profiles. A custom simulator can, +and can become the substrate for injection, fuzzing and adversarial work. + +### H3.1 Requirements +* RV64IMAC(+GC when needed) interpreter, fast enough for firmware bring-up + (target: ≥10–50 MIPS/core on one host thread; not realtime). +* Pluggable **memory map DSL**: regions (RAM/TCM/MMIO/DMA) with read/write + callbacks, access logging, and declarative register definitions. +* **Deterministic scheduler**: per-core instruction budgets, timers, event + queue; replay from an event log. +* **Snapshot/restore** of full machine state (registers, memory, device + state) for instant checkpointing and time-travel debugging. +* **Event bus** emitting the existing `events.jsonl` schema plus + `exec`/`access` events. +* **Backend ABI** identical to `Session`'s expectations, so all hooks, + watchpoints, scripts, probes, plugins, records and comparisons keep working. +* **Injection ABI**: load blobs, register services, intercept RM-core calls. +* **Python-first embedding** (C/C++ core with a Python control plane, or a + Rust core with PyO3), plus a CLI. + +### H3.2 Design sketch +``` ++-------------------- green-eye Python control plane --------------------+ +| Session API, hooks/watchpoints, recorder, plugins, Ghidra/agent bridge | ++---------------------------- Backend interface -------------------------+ + | QEMU-RSP backend (today) | native simulator backend + v v + qemu-system-riscv64 libgreenmachine (C/C++/Rust) + +-- cpu/ (interpreter, decoder) + +-- mem/ (map DSL, snapshots) + +-- dev/ (falcon, csb, pmu, dma) + +-- sched/ (cores, timers, events) + +-- trace/ (event bus, coverage) +``` +* The **map DSL** is versioned data (JSON/YAML) with per-device behaviour in + small plugin modules; new firmware/device profiles are additions, matching + the project's portability direction. +* **Snapshots** are copy-on-write memory pages + device state, giving cheap + branch exploration ("what if this reply differed?"). +* **Time-travel** is built from snapshot + deterministic event replay. + +### H3.3 Implementation plan +1. **M1**: RV64IMAC interpreter + RAM + UART; boots the demo uproc; green-eye + Session works against the native backend (parity with QEMU for the demo). +2. **M2**: memory map DSL + falcon/CSB/PTIMER devices; boots a re-hosted + uproc build; snapshots/restore. +3. **M3**: deterministic scheduler + event log/replay; multi-task tests. +4. **M4**: device-model parity with H2.2 and boot of the real GSP image under + shims. +5. **M5**: injection ABI + RM-core interception; fuzzing harness. +6. **M6**: performance and ergonomics hardening; backend selection in the + CLI (`--backend native`). + +### H3.4 Risks +Fidelity divergence from QEMU/hardware (mitigate: differential tests between +backends at H0 demos and H2 boot milestones); effort creep (gate behind H2 +success); maintenance of two backends (keep the Backend interface minimal). + +--- + +## H4 — Adversarial firmware navigation (continuous) + +**Goal:** when the firmware changes unpredictably — new handlers, opaque +state, deliberate anti-analysis, or shifted resources — navigate and +reimplement quickly. + +### H4.1 Semantic diffing +* Import each firmware's Ghidra corpus and green-eye symbol DB; diff RPC + dispatch tables, handler inventory, control sets, message layouts. +* Run the same stimulus harness against each version and use + `greeneye compare` for hit/watch/write deltas; automatically flag functions + whose call counts changed but whose static code is unchanged (behaviour + moved) vs changed code (new logic). +* Produce a "portability delta report" per version (feeds docs/07 playbooks). + +### H4.2 Capability inference +Observe what each service reads/writes (watchpoints, TCG trace) and infer +capabilities (which objects, ranges, controls it touches); compare with the +declared/emulated capability model; surface least-privilege and security +findings for `open-iov-vfio`. + +### H4.3 Mutation and fuzzing +Feed malformed construction/control traffic into re-hosted or simulated +firmware; capture error paths, asserts and panic loops with symbol +annotation; build a corpus that drives the emulation's fail-closed rules +(project security hygiene) and its error replies. + +### H4.4 Decoy / anti-analysis handling +Firmware may use timing checks, self-integrity checks, bogus code paths or +hardware-dependent branches. Techniques to build: deterministic replay to +separate data from noise; "two-run differential" with a single input bit +flipped; dead-code identification by coverage; trap-on-read of +anti-analysis registers with recorded responses; and a catalogue of observed +techniques in docs/09. + +### H4.5 Reimplementation workflow (accuracy loop) +The canonical loop for any behaviour: +1. Observe on hardware/VFIO (reference capture). +2. Observe in green-eye (simulator, replayable). +3. `compare` + metric table M1–M6 (docs/06-ACCURACY.md). +4. Implement in `open-iov-vfio` with the session directories attached as + evidence. +5. Re-run the same stimulus; the metric table must improve or stay equal. +This loop becomes a scripted harness ("consensus harness") so every change +produces comparable evidence. + +--- + +## H5 — Adjacent microcontrollers (months) + +**Goal:** treat the GSP as one node in a device-level system. + +* **Inventory**: PMU (power/voltage/clock provisioning), SEC2 and ACR/SOE + (security/boot), the booter, FWSEC/FRTS, and the DMA lines between them + (from firmware analysis and the RISC-V Story material; see + docs/09). +* **Models**: per-controller memory maps, mailbox protocols and boot state + machines as versioned data; SEC2/PMU stubs first (GSP needs them to pass + early init), full ACR/SOE later. +* **Cross-controller scenarios**: GSP↔PMU identity/clock provisioning, + GSP↔SEC2 mailbox, boot-order exercises (Init→ACR→RM for the + confidential-compute partition model). +* **Deliverable**: a device-level session that spans controllers, recorded + and comparable, so a firmware update's cross-controller interactions are + visible. + +Entry gate: H2.2 (GSP message loop). Exit: at least one cross-controller +exchange reproduced and compared with hardware evidence. + +--- + +## H6 — Verilog soft processor (far later) + +**Goal:** an FPGA/ASIC-simulation-grade soft implementation of the GSP core +environment that can run real GSP-RM images with cycle-accuracy options, for +deep timing/security/interoperability work and hardware validation of the +emulation. + +### H6.1 Scope +* RV64 core choice/build (existing open cores vs a minimal in-house core), + matching ISA/extensions used by the firmware; TCM, MPU/PMP, vectored + interrupts, timers. +* Device fabric: falcon-compatible windows, CSB/TRANSCOM, DMA engine to host + memory, WPR2/FWSEC state (with an explicit, documented security posture: + the soft model is a study environment, never a confidentiality boundary). +* Run modes: pure RTL simulation (Verilator) for determinism and coverage; + FPGA for speed; co-simulation with green-eye (JTAG/DPI/a bridge) so the + same Python hooks, sessions and comparisons apply. + +### H6.2 Integration with green-eye +* A `Backend` implementation talking to the RTL simulation clock/step and + memory API; breakpoints become simulation traps; watchpoints become + signal/page monitors. +* Sessions record cycle counts as a distinct event class (L2 timing becomes + measurable, with the accuracy caveats of docs/06 revisited). + +### H6.3 Gates +Requires: H2/H3 proven, a validated device map, and a funded reason to care +about cycle accuracy (e.g., a customer-facing timing claim). Explicitly +**not** on the critical path; do not start before the baseline goals in the +project roadmap are complete. + +--- + +## H7 — Agentic platform (continuous, gated) + +* **MCP plugin (R3)**: expose sessions as MCP tools (list/open sessions, + symbols, breakpoints, run/run_until, memory, snapshots, compare, + export_hits). Local stdio; no network by default. +* **AI harness integration**: deterministic run harnesses that an agent can + drive; sessions as the shared evidence format; guarded write actions with + an explicit capability manifest. +* **Autonomous analysis loops**: hypothesise → instrument → run → compare → + report, with human-review gates for anything that changes the product. +* **Safety**: no unattended physical-GPU cycles; no guest modifications; the + agent runs against the simulator by default. + +--- + +## Cross-cutting workstreams + +### V1 — Metrics and golden data +Keep the M1–M6 contract (docs/06) authoritative; maintain golden sessions for +the demo and re-hosted workloads; CI runs the tool tests + golden comparisons. + +### V2 — Formats and versioning +`greeneye-session/1` evolves additively with a version bump and migration +notes; symbol DBs carry provenance; injection blobs carry an ABI version and +capability metadata. + +### V3 — SDK/ABI evolution +Injection ABI v1 (demo) → v2 (RM-core interception). Breaking changes are +gated by a compatibility shim and a documented migration. + +### V4 — Security +Treat the simulator as a study environment: no confidentiality claims; keep +fuzzing and mutation behind explicit harnesses; never ship simulation shims +in the product; never place tooling in a guest. + +### V5 — Performance +Simulator performance target (H3.1); TCG trace filtering; snapshot +compression; session size budgets (region-scoped dumps). + +### V6 — Documentation and developer experience +Every horizon ships docs with copy-paste workflows, an acceptance script, and +a flag-removal (TOOL IN DEVELOPMENT) criterion; the Ghidra/agent guides are +kept current as first-class deliverables. + +## Sequencing summary + +| Horizon | Entry gate | Primary deliverable | +|---|---|---| +| H1 | H0 (done) | injection SDK + RM service guide + Ghidra control | +| H2 | H1 | QEMU device model; real GSP image reaches message loop | +| H3 | H2.2 | native simulator with snapshots/replay and injection | +| H4 | H1+H2 | adversarial diffing/capability/fuzzing harness | +| H5 | H2.2 | adjacent-controller models and cross-controller sessions | +| H6 | H3 validation | Verilog soft-core environment (far later) | +| H7 | H1 stable | MCP plugin and agentic harnesses | + +## Success criteria (program level) + +1. A new GSP version can be triaged (inventory diff + behaviour diff + + metric table) in **hours**, with recorded evidence. +2. Every new emulation behaviour lands with an A/B session pair and the M1–M6 + table. +3. Injected services prototype RM engine ideas in the simulator before any + product code is written. +4. The custom simulator reproduces the H2 boot and H4 workloads + deterministically, with snapshots and replay. +5. No claim of accuracy, portability or security depends on a single run or + on unstated fidelity assumptions. diff --git a/debug/risc-v-gsp-rm-vm/Makefile b/debug/risc-v-gsp-rm-vm/Makefile index a5103f5..fc43b37 100644 --- a/debug/risc-v-gsp-rm-vm/Makefile +++ b/debug/risc-v-gsp-rm-vm/Makefile @@ -3,11 +3,14 @@ PYTHON ?= python3 .PHONY: all demo test demo-run plugin probe-real clean info -all: demo +all: demo sdk demo: $(MAKE) -C firmware/uproc_demo +sdk: + $(MAKE) -C sdk + test: demo $(PYTHON) -m unittest discover -s tests -t . @@ -27,5 +30,6 @@ probe-real: clean: $(MAKE) -C firmware/uproc_demo clean + $(MAKE) -C sdk clean rm -rf dumps/* qemu/*.so qemu/*.o find . -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null || true diff --git a/debug/risc-v-gsp-rm-vm/README.md b/debug/risc-v-gsp-rm-vm/README.md index 66f53fa..4017cd4 100644 --- a/debug/risc-v-gsp-rm-vm/README.md +++ b/debug/risc-v-gsp-rm-vm/README.md @@ -50,13 +50,14 @@ Everything else is Python stdlib. greeneye/ Python package: RSP client, session, memory, symbols, structures, recording, analysis, probes, plugins, Ghidra bridge firmware/uproc_demo/RV64 bare-metal demo "uproc" (mailbox RPC, state machine, trap) +sdk/ injected-service SDK (position-independent RV64 blobs) qemu/ launch helper + optional TCG plugin for full memory tracing -ghidra/ Ghidra scripts (export symbols, annotate hit addresses) +ghidra/ Ghidra scripts (export symbols, annotate hits, control bridge) plugins/ example plugin + authoring guide examples/ Python instrumentation scripts and declarative probe specs -tests/ unit tests + QEMU integration smoke test +tests/ unit tests + QEMU integration smoke tests docs/ full documentation set (design, scripting, plugins, accuracy, - portability playbooks, roadmap, GSP firmware notes) + portability playbooks, roadmap, service/injection guides) dumps/ recorded sessions (gitignored) ``` @@ -75,3 +76,10 @@ dumps/ recorded sessions (gitignored) | [08-ROADMAP](docs/08-ROADMAP.md) | MCP, custom simulator, device model | | [09-GSP-FIRMWARE-NOTES](docs/09-GSP-FIRMWARE-NOTES.md) | uproc/falcon/NV-RISCV facts | | [10-RELATED-WORK](docs/10-RELATED-WORK.md) | Frida/GDB/QEMU-plugin comparison | +| [11-GSP-RM-SERVICE-DEVELOPER-GUIDE](docs/11-GSP-RM-SERVICE-DEVELOPER-GUIDE.md) | writing, injecting and debugging firmware services | +| [12-INJECTION-SDK](docs/12-INJECTION-SDK.md) | injection blob/ABI specification | +| [13-GHIDRA-CONTROL](docs/13-GHIDRA-CONTROL.md) | driving green-eye from GhidraScripts | +| [14-ADVERSARIAL-INTEROP-GUIDE](docs/14-ADVERSARIAL-INTEROP-GUIDE.md) | navigating unpredictable/adversarial firmware change | +| [15-AGENTIC-GSP-IDE](docs/15-AGENTIC-GSP-IDE.md) | agentic workflows and runbooks | +| [16-CUSTOM-SIMULATOR](docs/16-CUSTOM-SIMULATOR.md) | non-QEMU simulator design (far later) | +| [GREEN-EYE-ROADMAP](GREEN-EYE-ROADMAP.md) | the long-term program (H1–H7) | diff --git a/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md b/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md index a9f4251..f1e1946 100644 --- a/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md +++ b/debug/risc-v-gsp-rm-vm/docs/00-OVERVIEW.md @@ -79,6 +79,13 @@ Interactively: | [08-ROADMAP](08-ROADMAP.md) | MCP, custom simulator, device model | | [09-GSP-FIRMWARE-NOTES](09-GSP-FIRMWARE-NOTES.md) | uproc/falcon/NV-RISCV facts used here | | [10-RELATED-WORK](10-RELATED-WORK.md) | Frida/GDB/QEMU-plugin comparison | +| [11-GSP-RM-SERVICE-DEVELOPER-GUIDE](11-GSP-RM-SERVICE-DEVELOPER-GUIDE.md) | writing/injecting/debugging RM services | +| [12-INJECTION-SDK](12-INJECTION-SDK.md) | injection blob/ABI specification | +| [13-GHIDRA-CONTROL](13-GHIDRA-CONTROL.md) | driving green-eye from Ghidra | +| [14-ADVERSARIAL-INTEROP-GUIDE](14-ADVERSARIAL-INTEROP-GUIDE.md) | adversarial firmware navigation | +| [15-AGENTIC-GSP-IDE](15-AGENTIC-GSP-IDE.md) | agentic workflows and runbooks | +| [16-CUSTOM-SIMULATOR](16-CUSTOM-SIMULATOR.md) | non-QEMU simulator design (later) | +| [GREEN-EYE-ROADMAP](../GREEN-EYE-ROADMAP.md) | long-term program H1–H7 | ## Where this fits in the Open-IOV workflow diff --git a/debug/risc-v-gsp-rm-vm/docs/08-ROADMAP.md b/debug/risc-v-gsp-rm-vm/docs/08-ROADMAP.md index c8b4638..10f4f5f 100644 --- a/debug/risc-v-gsp-rm-vm/docs/08-ROADMAP.md +++ b/debug/risc-v-gsp-rm-vm/docs/08-ROADMAP.md @@ -1,6 +1,11 @@ # 08 — green-eye roadmap > **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status. +> This is the **near-term, committed** roadmap (rungs R0–R7 below). The +> **long-term program** — adversarial firmware navigation, adjacent +> microcontrollers, a non-QEMU custom simulator, a Verilog soft-processor +> environment and the agentic platform — lives in +> [`../GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md). > Items here are **not** v0.1. Do not start them ad hoc; each has a gate. ## R0. Product principles diff --git a/debug/risc-v-gsp-rm-vm/docs/11-GSP-RM-SERVICE-DEVELOPER-GUIDE.md b/debug/risc-v-gsp-rm-vm/docs/11-GSP-RM-SERVICE-DEVELOPER-GUIDE.md new file mode 100644 index 0000000..6ba8d69 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/11-GSP-RM-SERVICE-DEVELOPER-GUIDE.md @@ -0,0 +1,181 @@ +# 11 — GSP RM firmware service developer guide + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status and 06 for the +> fidelity caveats. Exact proprietary API names are not reproduced in tracked +> docs; local provenance lives in the local analysis notes. + +## 1. What this guide is for + +The GSP runs a resource manager (RM) as **services**: message handlers, state +machines and engines that own objects (devices, address spaces, channels, +memory, notifiers) and answer controls. Open-IOV emulates those services; the +long-term goal of green-eye is to let a developer **write, inject, run and +debug their own versions of these services** before (or instead of) changing +the product emulation. + +This guide teaches the loop on the demo uproc — which implements the same +shape as a GSP service — and maps each step to the future GSP-RM work in +[`GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md) H1.3. + +## 2. Service anatomy + +| Concept | Demo uproc (today) | GSP-RM (roadmap H1.3) | +|---|---|---| +| Entry point | exported symbol `ge_demo_service_dispatch` | service entry registered with the RM core | +| Dispatch ABI | `u64 dispatch(u64 cmd, u64 a0, u64 a1)` | same shape via the service registry shim | +| Reachability | firmware mailbox commands 16..23 select handler slot 0..7; `arg0` is the service command, `arg1`/`arg2` are parameters | RM message/control dispatch routed by the simulator's RM-core | +| State | service-local (stack/registers) or the control block | RM objects and service-private memory | +| Talking to peers | control-block fields + mailbox; debugger calls | RM-core calls and inter-service messages | +| Observability | hooks/watchpoints/sessions/TCG trace | same, plus RM-core call interception | + +A service must be **position-independent**: no globals, no libc, no absolute +addresses. Everything it calls is either its own code or an address passed in +by the injector (the future RM-core shim will pass a table of callable +entry points, which is also the interception point). + +## 3. Writing a service + +1. Copy the SDK shape from `sdk/`: + * `greeneye_service.h` — the ABI header; + * `injected_demo.c` — a working multi-command service. +2. Implement `ge_service_fn`: + ```c + __attribute__((noinline)) ge_u64 my_service_dispatch(ge_u64 cmd, ge_u64 a0, ge_u64 a1) + { + switch (cmd) { + case 1: return a0 ^ 0xbeefu; + case 2: return a0 + a1; + default: return ~cmd; + } + } + ``` +3. Keep it PIC-clean: no `static` data with pointers, no string globals, no + division (the demo build has no libc), no stack beyond a few KiB. +4. Build: + ```bash + make -C sdk # or point the Makefile at your source + make -C sdk relocs # must print "no relocations in this file" + ``` +5. Fix any relocation the linker reports before injecting; relocations mean + the blob will not run at an arbitrary load address. + +## 4. Injecting and invoking + +```bash +./greeneye.sh inject --service sdk/build/injected_demo.elf \ + --cmd 1 --arg0 0x41 --arg1 0x42 +``` + +What happens (all recorded): +1. Every `PT_LOAD` is written into RAM (default `0x81000000`; auto-sized to + fit the target). +2. The service dispatch entry is patched into handler slot 6 + (`--slot` selects 0..7). +3. A **direct debugger-side call** runs the service immediately + (`greeneye.inject.invoke_remote`): this is the unit-test path, independent + of firmware state. +4. After the firmware settles, a **firmware-dispatched call** writes + `arg0=cmd, arg1=a0, arg2=a1, mailbox=16+slot` and reads `result` + (`greeneye.inject.invoke_via_firmware`): this is the integration path. + +Programmatic form: + +```python +from greeneye.inject import load_service, invoke_remote, invoke_via_firmware +svc = load_service(s, "sdk/build/injected_demo.elf", slot=6) +print(hex(invoke_remote(s, svc, 2, 0x10, 0x20))) # 0x30 +s.run(duration=0.3) +print(hex(invoke_via_firmware(s, svc, 2, 0x10, 0x20))) # 0x30 +``` + +## 5. Talking to other services and the host + +On the demo: +* the **control block** is the shared state (`gsp_ctrl`); services coordinate + through its fields and the mailbox command range; +* the **host/debugger** is a peer via hooks, watchpoints and `Session.call`; +* a service can call another service if it is handed the other entry address + (the SDK's second export `ge_demo_service_answer` demonstrates a second + callable point; the injector resolves any exported symbol). + +On real GSP-RM (H1.3) the same roles are filled by RM-core allocation and +control calls and by messages between services. The simulator will intercept +those calls so they can be logged, stubbed, or forwarded to the real +implementation — which is exactly what makes the environment powerful for +prototyping: a service you write can be exercised against recorded RM-core +behaviour before the product emulation implements it. + +## 6. Resource surfaces an engine can use + +Category map (details are RM-version-specific and tracked privately where +provenance requires it): + +| Surface | Purpose | How green-eye helps today | +|---|---|---| +| Memory / descriptors | allocate, map, free, register host or vidmem ranges | watchpoints on allocation state; snapshot/diff; schema decode | +| Address spaces / page tables | GPU VA translation, mapping updates | hooks on map/unmap routines; memory diffs; TCG trace of accesses | +| Channels / GPFIFO / USERD | submission plumbing | hooks and remote calls; observe ring state via snapshots | +| Sync: semaphores, notifiers, interrupts | completion and signalling | watchpoints on sem values; event ordering in sessions | +| Engine scheduling / runlists | prioritisation, preemption | call-graph and hook evidence; compare runs | +| Controls / messages | configuration and queries | declarative probes; replay/compare across versions | +| Device identity / caps / clocks | what the RM reports | schema decode of the static-config block; A/B vs hardware oracle | + +Everything here is observed through the same primitives, so an engine +prototype is analysed exactly like firmware code. + +## 7. Debugging workflow + +1. **Unit-test with `call`:** breakpoints and hooks around the service entry, + `Session.call(entry, (cmd, a0, a1))`, inspect the return and watchpoints on + its data. Failures raise `RspTimeout` and leave the session recording. +2. **Integration run:** `invoke_via_firmware` with the mailbox path, while + hooks on the firmware's dispatcher and watchpoints on the control block + record the full route. +3. **Ghidra-driven:** `greeneye ghidra` + `GreenEyeControl.java` — execute + until a service address, set breakpoints from the decompiler, read + registers/memory, export hits (docs/13). +4. **Coverage:** run with the TCG plugin filtered to the service load range + (`qemu/greeneye-trace.c`, `start=`/`end=` args) to see every access. +5. **Regression:** record a session per command; `greeneye compare` detects + changed hit/watch/write profiles when the service or firmware changes. + +## 8. Rules and pitfalls + +* **PIC purity first.** If the blob needs relocation, it is not injectable + at an arbitrary address. +* **No absolute addresses from the host.** Use exported symbols and + `load_service`, never hardcoded addresses. +* **No unbounded loops.** A service that never returns blocks the call loop + until the timeout and then leaves the target stopped; the recorder shows + the last PC. +* **Watchpoint the state, hook the entry.** Watchpoints are immediate (one + instruction step) and ideal for counters; hooks give arguments/returns. +* **Never ship a service.** Injected code is a development experiment; the + product changes go through `open-iov-vfio` with the accuracy loop. +* **Record everything.** A claim about a service without a session is not + evidence. + +## 9. Worked example: add a command + +```c +/* sdk/injected_demo.c */ +case 7: + return (a0 | (a1 << 8)) & 0xFFFFFFFFull; +``` + +```bash +make -C sdk +./greeneye.sh inject --cmd 7 --arg0 0xab --arg1 0xcd +# direct call -> 0xcdab +# firmware call -> 0xcdab +./greeneye.sh analyze <printed session dir> # event/hit report +``` + +## 10. What comes next (roadmap) + +* H1.3: services built against an RM shim, registered with a simulated RM + core, calling intercepted RM-core surfaces. +* H2: the device model that lets real GSP-RM run in the simulator at all. +* H4: adversarial navigation — diff service behaviour across firmware + versions, mutate inputs, infer capabilities. +* H5: services that interact with PMU/SEC2/ACR models. diff --git a/debug/risc-v-gsp-rm-vm/docs/12-INJECTION-SDK.md b/debug/risc-v-gsp-rm-vm/docs/12-INJECTION-SDK.md new file mode 100644 index 0000000..26f1ffb --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/12-INJECTION-SDK.md @@ -0,0 +1,149 @@ +# 12 — Injection SDK specification (v1, demo ABI) + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status and +> [`GREEN-EYE-ROADMAP.md`](../GREEN-EYE-ROADMAP.md) §H1/H3 for the program. + +## 1. Goals and non-goals + +**Goals** +* a repeatable, scriptable way to compile a piece of RV64 code, load it into a + live firmware target, register it, invoke it, and observe it with the same + hooks/sessions/analysis as firmware; +* a clear ABI so services can be written once and run under the demo uproc + today and under a simulated RM core later; +* an evidence trail: every load, registration and invocation is recorded. + +**Non-goals (v1)** +* shipping any injected code (development-only); +* hot-patching signed firmware on hardware; +* bypassing security boundaries (the simulator is a study environment). + +## 2. Blob contract + +* **Format:** ELF64 `EM_RISCV`, `RV64IMAC`, soft-float ABI, one or more + `PT_LOAD` segments. +* **Position independence:** built PIC (`-fPIC`, `-mcmodel=medany`, + `-mno-relax`, `-fno-asynchronous-unwind-tables`), no dynamic relocations. + The loader verifies by writing segments at the chosen load address; the SDK + `make relocs` target must report none. +* **Self-sufficiency:** no libc, no global data containing addresses. + Caller-provided addresses (future RM-core tables) arrive as arguments. +* **Entry symbol:** `ge_demo_service_dispatch` by default (override with + `--entry`/`entry_symbol=`). +* **Optional metadata:** a `ge_service_t` block can be exported for future + loaders (`magic 0x47455356`, version 1, name, dispatch). + +## 3. ABI v1 + +```c +typedef ge_u64 (*ge_service_fn)(ge_u64 cmd, ge_u64 a0, ge_u64 a1); +``` + +* `a0`, `a1`, `a2` registers carry `(cmd, a0, a1)` on entry. +* The return value is `a0`. +* The service may use `a3..a7`, `t0..t6`, `s0..s11` and the stack; the + injector supplies a private stack below the scratch page. +* Traps do not return: the call loop times out, records the stop, restores + registers and raises `RspTimeout`. +* No calling convention for callbacks in v1; the loader registers a **single + dispatch entry** per slot. + +## 4. Demo registration model + +* The firmware exposes eight handler slots (`gsp_ctrl.handlers[0..7]`). +* Mailbox commands **16..23** select slot `cmd-16`. +* On dispatch the firmware calls + `handlers[slot](arg0, arg1, arg2)` — i.e. `cmd = arg0`, parameters + `arg1`, `arg2`. +* Slots 0..5 are used by the built-in demo RPCs; **slot 6** is the SDK + default (`--slot`). + +### Control-block fields used (demo) + +| Field | Offset | Role | +|---|---|---| +| `mailbox` | `+0x20` | command (`16 + slot` selects an injected service) | +| `arg0` | `+0x28` | service command | +| `arg1` | `+0x30` | parameter 0 | +| `arg2` | `+0x38` | parameter 1 | +| `result` | `+0x40` | service return value | +| `handlers` | `+0x150` | eight entry slots | + +Offsets are demo-firmware-specific; use `greeneye/inject.py` constants and the +schema at `firmware/uproc_demo/gsp_ctrl.schema.json` rather than hardcoding. + +## 5. Lifecycle + +``` +compile (PIC) ──► verify relocations ──► load PT_LOADs at load_addr + │ │ + │ ├─► register entry in slot + │ └─► record "service_injected" + ▼ +invoke_remote (direct call) invoke_via_firmware (mailbox dispatch) + └──────────────► both record call_enter/call_return events +``` + +* **Load address:** default `ram_base + 0x01000000`, auto-shrunk to fit if + the image is larger; explicit with `--load-addr`. +* **Cleanup:** v1 does not unregister automatically. A service is removed by + zeroing its slot and (optionally) clearing the blob; sessions record the + injection so runs are reproducible from a known state. +* **Multiple services:** one per slot; multi-entry services are one dispatch + with subcommands. + +## 6. Observability contract + +The injector records, in the session directory: +* `service_injected` — path, name, load address, entry address, slot, size; +* `mem_write` events for every segment written; +* `call_enter` / `call_return` — address, args, return value; +* hook/watch events from user instrumentation inside the service; +* `qemu.log`-style blobs are not applicable (no probe here), but the TCG + trace can be enabled for full coverage. + +Comparison of injected-service runs uses the standard `greeneye compare` +profiles, so a change in a service's behaviour is visible like any firmware +change. + +## 7. Building (reference) + +```makefile +CC := clang +LLD ?= /usr/lib/llvm-18/bin/ld.lld +CFLAGS := --target=riscv64-unknown-elf -march=rv64imac -mabi=lp64 \ + -mcmodel=medany -mno-relax -fPIC -ffreestanding -nostdlib \ + -fno-builtin -fno-stack-protector -fno-asynchronous-unwind-tables -O2 +$(LLD) -m elf64lriscv --no-relax -e ge_demo_service_dispatch -o service.elf service.o +``` + +The tracked SDK (`sdk/`) is the executable reference. + +## 8. Versioning and compatibility + +* ABI version 1 is the demo contract described here. +* Future ABI v2 (RM-core shim) will pass a call table and a service context + instead of relying on firmware slots; blobs declare their ABI via the + optional metadata block so loaders can refuse mismatches. +* Breaking ABI changes require a compatibility shim plus a note in the + session metadata and this document. + +## 9. Security and safety + +* Injected code runs with the simulator's full privileges; the bridge and + injector are local-only tools. +* Never inject into a guest or a shipped environment; never use injection to + bypass a hardware protection. +* Fuzzing/mutation harnesses (roadmap H4.3) must run against the simulator + or re-hosted code, with sessions attached as evidence. +* Treat every injected service as untrusted code for the purposes of + watching: it may corrupt firmware state, so always record and restart clean + after crashes. + +## 10. Roadmap hooks + +* **H1.3** — RM service registration and RM-core call interception. +* **H3** — native-simulator loader with snapshots and branch exploration. +* **H4** — differential testing of injected services across firmware + versions and capability inference from their access patterns. +* **H5** — services spanning GSP and adjacent microcontrollers. diff --git a/debug/risc-v-gsp-rm-vm/docs/13-GHIDRA-CONTROL.md b/debug/risc-v-gsp-rm-vm/docs/13-GHIDRA-CONTROL.md new file mode 100644 index 0000000..26e534d --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/13-GHIDRA-CONTROL.md @@ -0,0 +1,154 @@ +# 13 — Ghidra control bridge + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status. + +The bridge lets a GhidraScript drive a live green-eye session: execute the +target until a point **chosen in the decompiler**, set/clear breakpoints, read +memory and registers, make remote calls, and export hit lists back into the +Ghidra project. Nothing about the firmware has to be rebuilt or re-hosted to +answer "does execution actually reach this function?". + +``` +Ghidra (Java script) ── JSON lines over localhost ──► greeneye gdbridge ──► Session ──► QEMU + ▲ │ + └──────── greeneye_hits.tsv ── MarkGreenEyeHits.java ◄──── recorded session ┘ +``` + +## 1. Quickstart + +```bash +# Terminal 1: a live target + bridge (records by default) +./greeneye.sh ghidra --kernel firmware/uproc_demo/build/uproc_demo.elf --bridge-port 7000 +# prints: ghidra bridge on 127.0.0.1:7000 + +# Terminal 2: Ghidra headless, run until the decompiler's address +analyzeHeadless <proj> <name> -process <program> -noanalysis \ + -scriptPath open-iov/debug/risc-v-gsp-rm-vm/ghidra \ + -postScript GreenEyeControl.java 7000 0x800000ce +``` + +In the GUI, load `GreenEyeControl.java` from the Script Manager and supply the +port in the script arguments; with no address argument it uses the current +cursor (`currentAddress`). + +## 2. `GreenEyeControl.java` functions + +All functions exist on the script class and can be called from the script's +own `run()` or from other scripts via the bridge `send()` method. + +| Function | Effect | +|---|---| +| `ping()` | `pong` if the bridge is alive | +| `info()` | session name, pc, symbol count, recording path | +| `resolve(name)` | symbol or `name+0xNN` → runtime address | +| `setBreakpointAt(hexAddr)` / `setBreakpointHere()` | install a hook (prints on hit) | +| `clearBreakpointAt(longAddr)` | remove breakpoints at an address | +| `runUntil(hexAddr)` / `runUntilHere()` | resume and stop at the address; returns `{addr, symbol, kind}` | +| `readMemoryHere(size)` | hex bytes at the cursor | +| `readRegister(name)` / `writeRegister(name, value)` | register access | +| `callFunction(hexAddr, args...)` | Frida-like remote call; returns `a0` | +| `snapshotHere(size)` | record a memory snapshot in the session | +| `listHits()` | breakpoint hit counts from the session | +| `exportHits(path)` | write `greeneye_hits.tsv` for `MarkGreenEyeHits.java` | +| `send(cmd, argsJson)` / `resultOf(response)` | raw bridge access for new workflows | + +Example additions inside `run()`: + +```java +setBreakpointAt("0x800000ce"); +println(runUntil("0x800000ce")); // {addr:..., symbol:"rpc_sum", kind:"breakpoint"} +println(readMemoryHere(16)); +println(callFunction("0x80000048", 0x1111L)); // rpc_ping -> 0xaffe +println(exportHits("/tmp/greeneye_hits.tsv")); +``` + +## 3. Bridge protocol (specification) + +One JSON object per line; one response per request, same `id`. + +Request: `{"id": <int>, "cmd": "<name>", "args": {...}}` +Response: `{"id": <int>, "ok": true, "result": ...}` or +`{"id": <int>, "ok": false, "error": "..."}`. + +| `cmd` | args | result | +|---|---|---| +| `ping` | — | `"pong"` | +| `info` | — | `{name, pc, symbols, recording}` | +| `resolve` | `{name}` | address (int) | +| `bp_set` | `{target, kind?}` | address | +| `bp_clear` | `{addr}` | `true` | +| `bp_list` | — | `[{addr, label, hits, kind}]` | +| `watch_set` | `{target, size?, kind?}` | address | +| `run_until` | `{target, timeout?}` | `{addr, symbol, kind}` | +| `run` | `{duration?, max_stops?}` | stops | +| `step` | `{count?}` | `[pc, ...]` | +| `read_mem` | `{addr, size?}` | hex string | +| `write_mem` | `{addr, hex}` | `true` | +| `read_reg` | `{name}` | int | +| `write_reg` | `{name, value}` | `true` | +| `call` | `{target, args?, timeout?}` | return value | +| `snapshot` | `{addr, size, label?}` | label | +| `note` | `{text}` | `true` | +| `hits` | — | `[{addr, hits, symbol}]` | +| `export_hits` | `{path}` | absolute path | +| `close` | — | `true` (closes the connection and stops the bridge) | + +`target`/`addr` accept `"0x..."`, a symbol name, or `name+0xNN`. + +## 4. Workflows + +### 4.1 Execute until here +1. Start `greeneye ghidra` on the firmware image and load symbols + (`symbols export` from the Ghidra index) so hits are named. +2. Put the cursor on a function in Ghidra and run `GreenEyeControl.java`. +3. `runUntilHere()` sets a temporary breakpoint, services intervening stops + (including your hooks/watchpoints), and reports the symbol resolved at the + stop. If the function is never reached, the bridge returns a timeout error. + +### 4.2 Annotate the project with runtime evidence +1. Run a session (any CLI/scripted path) and `exportHits("/tmp/hits.tsv")`. +2. `MarkGreenEyeHits.java /tmp/hits.tsv` adds plate comments and bookmarks in + the Ghidra project. +3. Optionally export a symbol DB from Ghidra (`ExportForGreenEye.java`) and + feed it back with `symbols export --index`. + +### 4.3 Remote-call driven experiments +Use `callFunction()` to invoke firmware or injected-service functions from +the decompiler (unit-test style), then inspect memory with +`readMemoryHere()` and the recorded session. + +## 5. Address conventions + +Ghidra shows **program addresses** (e.g. libcuda file vaddr + image base). +green-eye resolves **runtime addresses**. For firmware linked at its load +address these coincide; otherwise export symbols with the right bias +(docs/05-GHIDRA-BRIDGE.md) and prefer `resolve(name)` over hardcoded numbers. + +## 6. Limitations (v1) + +* Request/response only: the script pulls state; the bridge does not push + asynchronous hit events. Poll `hits`/`bp_list` between commands. +* One command at a time per connection; multiple clients are accepted but are + not synchronised with each other. +* `watch_set` exists on the bridge but has no dedicated Java wrapper yet — + call `send("watch_set", ...)` directly. +* The bridge is unauthenticated **by design**: localhost only, never expose + it on a network interface. + +## 7. Troubleshooting + +| Symptom | Cause / fix | +|---|---| +| `Connection refused` | bridge not started, wrong port, or it exited (`close`) | +| `green-eye bridge closed the connection` | the server was stopped or the target session closed | +| timeout on `run_until` | address not executed; check symbol bias and that the target is not wedged | +| empty `hits` | the session was started without recording; use the CLI (records by default) | +| Java compile error on script load | Ghidra API drift; the script targets Ghidra 11/12 `GhidraScript` | + +## 8. Roadmap + +* Push notifications for breakpoint hits (server → Ghidra). +* Decompiler annotations from live watchpoints (value changes per field). +* A bridge client that lets multiple scripts share a session with a command + queue. +* Direct MCP exposure of the same command set (roadmap R3). diff --git a/debug/risc-v-gsp-rm-vm/docs/14-ADVERSARIAL-INTEROP-GUIDE.md b/debug/risc-v-gsp-rm-vm/docs/14-ADVERSARIAL-INTEROP-GUIDE.md new file mode 100644 index 0000000..d310f22 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/14-ADVERSARIAL-INTEROP-GUIDE.md @@ -0,0 +1,399 @@ +# 14 — Firmware adversarial interoperability research guide (green-eye + Ghidra) + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status. + +## 1. Purpose and scope + +Open-IOV's hard problem is not writing the GSP emulation once; it is keeping +`open-iov-vfio` accurate while the firmware changes underneath it. New GSP +images, device pairings and architectures arrive stripped, signed and +hardware-coupled, and some actively resist analysis. This guide is the +practical companion to `GREEN-EYE-ROADMAP.md` H4: how to move from "the new +image does something different" to a reviewed product change with evidence. + +Ghidra is the **static map**; green-eye is the **dynamic ground truth**; +recorded sessions are the **shared evidence**. Recipes use only the v0.1 +surface, plus items explicitly labelled roadmap; see also docs/06-ACCURACY.md. + +| Capability used here | State | +|---|---| +| `probe`/`record`/`analyze`/`compare`/`symbols`/`inject`/`ghidra`, hooks, watchpoints, snapshots, sessions, `export_hits` | working (v0.1) | +| Ghidra control bridge (`run_until`, breakpoints, hits) | working (v0.1) | +| Full TCG access trace (`qemu/greeneye_trace.c`) | source + optional build; not folded into sessions yet (R4) | +| Mutation harness (AFL++ class), device model, MCP | roadmap (R5, H2/R1, H7/R3) | + +## 2. The adversary model (a moving target, not a malicious party) + +This guide assumes no threat actor. The "adversary" is **unplanned firmware +change plus deliberate opacity**: new images on a hidden +schedule; stripped symbols, signed images and non-recoverable boot states; +anti-analysis behaviour (timing checks, self-consistency checks, hardware- +dependent branches) and **decoys** (code or data not on the live path); and +hardware coupling a simulator cannot yet exercise. + +1. Any single run is an anecdote. A conclusion needs at least an A/B pair. +2. A static reading is a hypothesis. It becomes evidence only when execution + reaches it in a recorded session. +3. Absence of a hit is a fact about *that* run at *that* fidelity (docs/06), + not proof of dead code. +4. Every claim carries its fidelity level (L0–L3, docs/06) and provenance. + +## 3. The joint workflow: Ghidra, green-eye, sessions + +``` + question > Ghidra (static): decompile, dispatcher tables, symbols + > green-eye (dynamic): same stimulus, hooks, watchpoints, trace + > sessions + hits TSV: greeneye compare -> triage -> evidence pack + > open-iov-vfio change (with evidence) +``` + +| Question | Ghidra answers | green-eye answers | Shared artifact | +|---|---|---|---| +| What is this code? | decompilation, call graph, xrefs | — | symbol DB (docs/05) | +| Did it execute here? | hypothesis from xrefs | `bp_hit`/`watch_hit` events | `greeneye_hits.tsv` | +| What changed between versions? | corpus/symbol diff | `compare` hit/watch/write deltas | symbol DB per version + report | +| What can this service touch? | object references | watchpoint/TCG access map | capability table | + +References: docs/05-GHIDRA-BRIDGE.md, docs/13-GHIDRA-CONTROL.md, docs/07-PORTABILITY-PLAYBOOKS.md. + +## 4. Method catalogue + +Run all commands from `open-iov/debug/risc-v-gsp-rm-vm/`; angle-bracket +placeholders are task-supplied (never hardcode private paths in docs). + +### 4.1 Semantic diff across firmware versions (H4.1) + +Goal: separate "the code changed" from "the behaviour moved" before the +emulation is touched. + +1. Build a symbol DB per version. `ghidra/ExportForGreenEye.java` emits + `addr<TAB>name<TAB>size`, which `symbols export --index` accepts: + + ```bash + ./greeneye.sh symbols export --index <new-export.tsv> --bias 0 -o symbols-new.json + ./greeneye.sh symbols export --index <old-export.tsv> --bias 0 -o symbols-old.json + ``` + +2. Probe each image for first-touch structure changes (docs/07 P1.2): + + ```bash + ./greeneye.sh probe <old.fwimage> --symbols symbols-old.json --record + ./greeneye.sh probe <new.fwimage> --symbols symbols-new.json --record + ``` + +3. Run the **same stimulus** against the re-hosted build of each version. Put + it in a script so it is identical across runs: + + ```python + # run_stimulus.py -- ./greeneye.sh run run_stimulus.py --kernel <rehosted.elf> + import greeneye as ge + s = ge.launch(kernel="<rehosted.elf>", record=True, name="stim") + ctrl = s.symbols.lookup("<control-symbol>") # demo: gsp_ctrl + s.watch(ctrl + 0x10, size=8, kind="w") + s.run(duration=0.5) # settle/self-test + s.memory.write_u64(ctrl + 0x20, 1) # mailbox PING + s.memory.write_u64(ctrl + 0x28, 0x1234) + s.run(duration=0.5) + s.note("stimulus: PING 0x1234") + s.close() + ``` + + Demo control-block offsets are demo-specific; use the schema + (`firmware/uproc_demo/gsp_ctrl.schema.json`) or SDK constants rather than + hardcoding them (docs/12 §4). + +4. Compare the sessions symbolically — breakpoint profile delta, watchpoint + delta and written-page delta: + + ```bash + ./greeneye.sh compare dumps/<old-run> dumps/<new-run> --symbols + ``` + +5. Triage with a decision table rather than intuition: + + | Static code changed? | Hit/write profile changed? | Reading | + |---|---|---| + | no | no | unchanged on this stimulus | + | no | yes | behaviour moved: inputs, ordering, environment or state | + | yes | no | changed code off this stimulus path (or dead/decoy: §4.4) | + | yes | yes | new logic on a live path — prioritise | + | n/a | symbol never hit in either run | coverage gap: widen the stimulus first | + +6. Annotate the result back into Ghidra (`greeneye_hits.tsv` + + `MarkGreenEyeHits.java`). Behaviour that moved without a code change is + usually a state/ordering fix for the emulation, not new logic. + +### 4.2 Capability inference from watchpoints and TCG traces (H4.2) + +Goal: derive what a service/engine actually reads and writes, then compare it +with the capability model the emulation assumes. + +1. Instrument the object ranges the service owns (kinds `w`, `r`, `a`): + + ```python + # capability_probe.py -- ./greeneye.sh run capability_probe.py --kernel <elf> + import greeneye as ge + s = ge.launch(kernel="<rehosted.elf>", record=True, name="caps") + base = s.symbols.lookup("<object-symbol>") + for off in (0x0, 0x10, 0x20, 0x28, 0x38, 0x40): + for kind in ("r", "w"): + s.watch(base + off, size=8, kind=kind, + callback=lambda h, k=kind: s.note("cap-" + k, pc=h.pc)) + s.run(duration=1.0) + s.close() + ``` + +2. For access coverage **between stops**, enable the TCG plugin with a PC + filter for the service's load range: + + ```bash + make -C qemu + qemu-system-riscv64 -machine virt -m 256M -display none \ + -serial file:/tmp/uart.log -bios none -kernel <rehosted.elf> -S \ + -gdb tcp:127.0.0.1:4444 \ + -plugin qemu/greeneye-trace.so,path=/tmp/ge-trace.jsonl,start=0x80000000,end=0x81000000,exec=on + ./greeneye.sh attach --port 4444 --kernel <rehosted.elf> --plugin <capability-plugin.py> + ``` + + `start=`/`end=` bound the trace, `exec=on` adds per-instruction records and + `path=` selects the JSONL output (`qemu/greeneye_trace.c`; qemu/README.md). + Folding that JSONL into sessions is roadmap R4 — today it is a separate + evidence blob. + +3. Reduce to a capability table and compare with the emulation's model: + + | Observed access | Address/range | Kind | PC/symbol | Emulated capability | Finding | + |---|---|---|---|---|---| + | write | `0x…` | w | `func+0x…` | not modelled | least-privilege candidate | + | read | `0x…` | r | `func+0x…` | denied in emulation | potential fail-closed gap | + + Least-privilege readings matter for `open-iov-vfio`, but a simulator + observation is evidence about the firmware (docs/06 §6); confirm before it + changes the product. + +### 4.3 Mutation of construction/control inputs (H4.3) + +Goal: drive error paths and asserts deliberately, capture them with symbols, +and build a corpus for the emulation's fail-closed rules. The full harness is +roadmap R5; until then, bounded mutation over existing primitives is honest. + +1. Mutate single fields deterministically and record each attempt: + + ```python + # mutate.py -- ./greeneye.sh run mutate.py --kernel <rehosted.elf> + import greeneye as ge + s = ge.launch(kernel="<rehosted.elf>", record=True, name="mutate") + ctrl = s.symbols.lookup("<control-symbol>") + for value in (0, 1, 0xFFFFFFFFFFFFFFFF, 0x8000000000000000): + s.memory.write_u64(ctrl + 0x20, value) + s.run(duration=0.2, max_stops=200) + s.note("mutation", field="mailbox", value=value) + s.close() + ``` + +2. Mutate injected-service calls on both paths (direct and mailbox): + + ```python + from greeneye.inject import load_service, invoke_remote, invoke_via_firmware + svc = load_service(s, "sdk/build/injected_demo.elf", slot=6) + invoke_remote(s, svc, cmd=0xFFFF, a0=0, a1=0) # malformed cmd + s.run(duration=0.3) + invoke_via_firmware(s, svc, cmd=0xFFFF, a0=0, a1=0) # malformed mailbox + ``` + +3. Capture error paths with hooks/watchpoints on the dispatcher, fault + handlers and assert sites (symbol DB); `./greeneye.sh demo` records a trap. +4. Compare clean vs mutated sessions, keeping the mutation value in a `note` + so the pack is self-describing: + + ```bash + ./greeneye.sh compare dumps/<clean-run> dumps/<mutated-run> --symbols + ``` + + Every corpus entry needs the firmware hash; never run mutation against + hardware (docs/12 §9). + +### 4.4 Decoy and anti-analysis handling (H4.4) + +Goal: distinguish signal from noise when the image branches on environment, +timing or integrity state, or contains bogus code paths. + +1. **Deterministic replay first.** Run the identical stimulus twice and + require the comparison to be empty: + + ```bash + ./greeneye.sh compare dumps/<run1> dumps/<run2> --symbols + ``` + + Any delta is a non-determinism candidate (scheduler, uninitialised read, time); record it. + +2. **Two-run differential.** Flip one input bit; attribute every delta to it; + unrelated movement suggests a time/integrity check: + + ```python + s.memory.write_u64(ctrl + 0x28, original ^ (1 << 0)) # one bit + ``` + +3. **Coverage-based dead-code detection.** Use the TCG trace with `exec=on` + (or a bounded `s.step(n)` pass) to build a PC set, then annotate it into + Ghidra: + + ```python + from greeneye.ghidra import export_hits + from greeneye.symbols import SymbolDB + db = SymbolDB(); db.load_json("<symbols.json>") + export_hits("dumps/<run>", "greeneye_hits.tsv", symbols=db) + ``` + ```bash + $GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \ + -scriptPath debug/risc-v-gsp-rm-vm/ghidra \ + -postScript MarkGreenEyeHits.java /abs/path/greeneye_hits.tsv + ``` + + Zero hits across every stimulus makes a function a **decoy candidate**, not proof of dead code (docs/06). + +4. **Trap-on-register-read experiments.** Arm a read watchpoint on a register + the firmware might be probing, and record which PC reads it: + + ```python + s.watch("<reg-or-mmio-symbol>", size=8, kind="r", + callback=lambda hit: s.note("anti-analysis-read", pc=hit.pc)) + ``` + + On today's `qemu virt` an unmodelled GPU register faults rather than + returning a value (`greeneye probe` reports that first read, docs/06 §3); + the value-return experiment waits on the H2 device model — log, don't claim. +5. Maintain the catalogue of observed techniques in + docs/09-GSP-FIRMWARE-NOTES.md (roadmap H4.4), one session pair per technique. + +### 4.5 Differential execution: reference vs simulator vs injected service (H4.5) + +Goal: measure, not narrate. The contract is the M1–M6 table from docs/06: + +| # | Metric | How it is measured here | +|---|---|---| +| M1 | Inventory coverage | observed controls/RPCs vs emulated handled set | +| M2 | Reply agreement | field-by-field reply diff (canonical size, consumed fields) | +| M3 | Ordering agreement | sequence alignment of the control/event stream (LCS/edit distance) | +| M4 | State agreement | firmware state transitions vs emulation state model | +| M5 | Error-path agreement | injected faults take the same status/notifier path | +| M6 | Memory side-effects | read/write map vs emulation shadow state; snapshot hashes | + +Procedure: (1) capture the reference on hardware/VFIO (docs/07 P4); (2) capture +the simulator side with `record`/scripted `run`, hooks, watchpoints and +snapshots; (3) `./greeneye.sh compare dumps/<reference-dir> dumps/<greeneye-dir> +--symbols`, with a script or plugin for per-control M2/M3 payload events; +(4) repeat with the injected-service path to test the service boundary; +(5) escalate — divergence that would change emulation behaviour must be +reproduced physically; simulator-only divergence becomes a fidelity gap. + +### 4.6 From observation to reimplementation + +The canonical accuracy loop (roadmap H4.5): (1) observe on hardware/VFIO; +(2) observe in green-eye (replayable session); (3) `compare` + M1–M6; +(4) implement in `open-iov-vfio` with the session directories attached as +evidence; (5) re-run the same stimulus and require the metric table to improve +or stay equal. + +Practical rules: one session pair per behaviour change; the PR links the +`compare` report and session hashes plus the symbol DB diff if the static map +changed; decoy or low-fidelity observations never justify a product change. + +## 5. Ghidra integration specifics + +Start the live bridge and drive it from the decompiler: + +```bash +./greeneye.sh ghidra --kernel <rehosted.elf> --bridge-port 7000 +$GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \ + -scriptPath debug/risc-v-gsp-rm-vm/ghidra \ + -postScript GreenEyeControl.java 7000 0x<decompiler-address> +``` + +`GreenEyeControl.java` operations (docs/13): `setBreakpointAt/Here`, +`runUntil/Here`, `readMemoryHere`, `readRegister`/`writeRegister`, +`callFunction`, `snapshotHere`, `listHits`, `exportHits`. Watchpoints exist on +the bridge (`watch_set`) without a Java wrapper: `send("watch_set", {...})`. + +Address conventions and bias pitfalls (docs/05 §1): + +* Ghidra shows **program addresses**; green-eye resolves **runtime addresses**. + For firmware linked at its load address they coincide; otherwise supply + `--bias` when building the symbol DB (`ExportForGreenEye.java` emits program + addresses). +* The classic mistake is passing a program address to `hook()` when the image + base is non-zero; prefer `s.hook("<name>")` and pass raw numbers through + `resolve`. `decompiled_index.tsv` carries both spaces; green-eye keeps the + program address in `symbol.extras["program_addr"]` for `bucket:line` mapping. +* Wrapper-generated functions are re-laid-out between versions: transfer + names, never raw wrapper addresses (docs/05 §6). +* Bridge `run_until` timeouts mean "not executed under this stimulus at this + fidelity"; they do not prove unreachability. + +## 6. Metrics, evidence pack and report template + +A finding is only a finding when the pack is complete. The evidence pack for +one adversarial-navigation task contains: + +* [ ] firmware identity (version, hash, pairing) and symbol DB(s) with provenance and hash; +* [ ] baseline/candidate session dirs (or sha256 inventory) and the exact stimulus; +* [ ] `compare` report and relevant `analyze` reports; +* [ ] M1–M6 table with the reference-capture method and explicit L2/L3 limits; +* [ ] `greeneye_hits.tsv` plus any Ghidra annotations produced; +* [ ] unexplained-delta list (empty is a claim, not a default). + +Short report template: + +```markdown +# Firmware delta: <old> -> <new> (<device>) +* firmware: <old sha256> -> <new sha256>; symbol DB: <old hash> -> <new hash> + (provenance: <export>, bias <n>) +* sessions: <baseline dir>, <candidate dir> (greeneye-session/1); stimulus: + <script/spec + mutation values> + +## Observed delta +* changed on live path: <symbols>; behaviour moved without code change: + <symbols, evidence>; coverage gaps/decoys: <symbols, why> + +## Metrics (docs/06) +| M1 | M2 | M3 | M4 | M5 | M6 | +|----|----|----|----|----|----| + +## Fidelity statement +L0/L1 in simulator; L2 not claimed; L3 absent for <registers>. +Physical reproduction required before product change: yes/no, why. + +## Product action +open-iov-vfio change (or explicit "no change"): <link/notes> +``` + +## 7. Boundaries, safety and ethics + +* **Development-only tooling.** Never shipped, never a dependency of the emulation, never in a guest (docs/00). +* **No guest modification.** Instrument the firmware under study in the + simulator; do not patch or observe a guest driver (project rule; V4). +* **No confidentiality claims.** The simulator is a study environment, never a + security or confidentiality boundary (docs/06, V4/H6). +* **Clean-room discipline.** Do not copy vendor code into product code; use + the study material to understand behaviour, then write the emulation from + observation. Tracked docs cite behaviour generically and never name source + locations. +* **Fuzzing is gated.** Mutation runs only against the simulator or re-hosted + code, behind explicit harnesses, with session evidence (docs/12 §9, R5). + Never use injection to defeat a hardware security boundary (docs/12 §1). +* **Escalate to physical.** Any conclusion that would change `open-iov-vfio` + must be confirmed against a physical capture (docs/06 §6, docs/07 P4.5). + +## 8. Roadmap hooks + +| Method | Roadmap item | What it adds | Gate | +|---|---|---|---| +| 4.1 semantic diff | H4.1 | versioned symbol corpora, automated delta report | H1+H2 | +| 4.2 capability inference | H4.2 | access map vs capability model, least-privilege findings | watchpoints (done); TCG-session fold (R4) | +| 4.3 mutation/fuzzing | H4.3, R5 | AFL++ class harness, corpus of error paths | R1/R2 | +| 4.4 decoy handling | H4.4 | technique catalogue in docs/09, deterministic replay | H2 for register-value experiments | +| 4.5 differential exec | H4.5, docs/06 | scripted consensus harness, M1–M6 automation | reference capture pipeline | +| 4.6 reimplementation | H4.5, V1 | golden sessions, CI comparison | H1 stable | + +Program-level success criterion served here: a new GSP version can be triaged +(inventory + behaviour diff + metric table) in **hours**, with evidence. diff --git a/debug/risc-v-gsp-rm-vm/docs/15-AGENTIC-GSP-IDE.md b/debug/risc-v-gsp-rm-vm/docs/15-AGENTIC-GSP-IDE.md new file mode 100644 index 0000000..79d17b8 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/15-AGENTIC-GSP-IDE.md @@ -0,0 +1,387 @@ +# 15 — green-eye as an agentic integrated development, analysis and debugging environment for the GPU System Processor + +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status. + +## 1. Vision and what "agentic" means here + +green-eye already has the two properties that make an agent loop possible: +a **deterministic, scriptable harness** (`ge.launch`/`ge.attach`, `Session`, +`record`, `compare`) and a **durable evidence format** (the session directory). +"Agentic" here means: + +* AI agents drive the deterministic harness — they choose instrumentation, + run stimuli, compare sessions and write findings; +* sessions are the shared state between agents, humans and Ghidra, so every + claim is inspectable after the fact; +* **humans keep product-change authority**: an agent may propose an + `open-iov-vfio` change with evidence, but only a human merges it, and the + physical-confirmation rules of docs/06 and docs/07 still apply. + +Non-goals: autonomous hardware cycles, unattended reboots, agents that "fix" +the emulation directly, or replacing the engineer reading the decompiler. +This document describes the environment that exists today plus the roadmap +hooks that would make a full agent platform real. + +## 2. Architecture + +``` + +-------------------- agents (triage / hypothesis / RE / portability) ----+ + | | + | CLI: ./greeneye.sh info|demo|run|attach|launch|probe|record|analyze| | + | compare|symbols|plugins|inject|ghidra | + | Python: ge.launch/attach, Session.*, memory.*, symbols.*, structs, | + | probes, plugins, ghidra.export_hits, inject.* | + | REPL: bp/hook/watch/call/x/snap/events/note | + | Ghidra bridge: JSON lines over localhost (docs/13) | + | MCP: planned plugin only (roadmap R3) — does not exist in v0.1 | + +----------------------------------+---------------------------------------+ + v + session directory (events.jsonl + session.json + snapshots + summary.md) +``` + +### 2.1 The session directory is the shared state + +| Artifact | Content | Consumers | +|---|---|---| +| `events.jsonl` | append-only event stream (`bp_hit`, `watch_hit`, `mem_write`, `call_enter/return`, `note`, `service_injected`, …) | `analyze`, `compare`, `export_hits`, plugins | +| `session.json` | format, name, counts, snapshot inventory, meta | report tooling, provenance | +| `snapshots/*.bin.z` | zlib-compressed region captures with sha256 | offline diff, field decode | +| `blobs/*` | plugin/probe dumps (e.g. qemu.log) | forensics | +| `summary.md` | rendered report | humans, PR attachments | + +`greeneye compare dumps/<a> dumps/<b> --symbols` is the primary agent +primitive: it turns two sessions into hit/watch/write deltas. Format +`greeneye-session/1` evolves additively (roadmap V2). + +### 2.2 Surfaces an agent may use + +* `./greeneye.sh …` — the stable CLI (all real subcommands listed above). +* `./greeneye.sh run script.py` — the preloaded Python namespace (`ge`, + `session`/`s`, `memory`, `symbols`, `recorder`) makes a harness file the + unit of automation. +* `./greeneye.sh ghidra` + `GreenEyeControl.java` — JSON-lines bridge for the + decompiler loop (`run_until`, `bp_set`, `read_mem`, `call`, `hits`, + `export_hits`; protocol in docs/13 §3). +* Plugins — the extension point (docs/04). A plugin's `register(ge)` can add + instrumentation, CLI commands (`ge.add_command`) and recorded events; plugin + output lands in the same `events.jsonl`. +* **MCP is planned, not present.** Roadmap R3/H7 reserves an MCP plugin that + would expose sessions as local stdio tools. Until that rung is scheduled, do + not add MCP dependencies to the core package (docs/04 §3d). + +### 2.3 Why the evidence model matters for agents + +A human can hold context across steps; an agent cannot. The session directory +makes the context external: an agent can be restarted, handed a directory, and +re-derive the same conclusion without replaying its own reasoning. This is the +difference between an agent that reports opinions and one that produces +reviewable evidence. + +## 3. Agent roles and workflows + +| Role | Input | Tools | Output | Stop condition | +|---|---|---|---|---| +| Triage | question + firmware version(s) | `symbols export`, `probe`, `analyze` | delta candidates, prioritised symbol list | every changed symbol classified | +| Hypothesis | one behaviour question | `run` scripts, hooks, watchpoints, `compare` | session pair + verdict | delta attributed to one variable | +| RE | decompiled function | `greeneye ghidra`, `run_until`, breakpoints, `export_hits` | hits TSV + annotation, reachability answer | function hit or timeout recorded | +| Portability | version/device pair | docs/07 playbook, baseline + candidate sessions, M1–M6 | portability report | metric table complete | +| Injection | service idea | `sdk/`, `inject`, `load_service`, `invoke_remote`, `invoke_via_firmware` | injected-service session pair | direct and firmware paths agree | +| Verification | another agent's finding | golden sessions, `compare`, `analyze` | pass/fail + counterexample | finding reproduced or refuted | + +Role separation is deliberate: the agent that made a claim should not be the +only one that checks it. Verification requires re-running the original +stimulus, not re-reading the original report. + +## 4. Runbooks + +Each runbook is written so a human or an agent can execute it start to finish, +and each ends with the evidence pack from docs/14 §6. + +### 4.1 Answer an emulation reply question + +Question template: "For control `<X>`, does the firmware set field `<Y>` of +the reply/object, and in what order relative to `<Z>`?" + +1. Frame it as A/B: the emulation's reply (reference capture, docs/07 P4) and + the firmware's reply in green-eye. One session alone cannot answer it. +2. Resolve the handler and object from the symbol DB; never hardcode: + + ```bash + ./greeneye.sh symbols resolve --db symbols-<ver>.json <handler-or-addr> + ``` + +3. Capture the firmware side with a hook + before/after snapshots: + + ```python + # reply_q.py -- ./greeneye.sh run reply_q.py --kernel <rehosted.elf> + import greeneye as ge + from greeneye import Memory + + s = ge.launch(kernel="<rehosted.elf>", record=True, name="reply-q") + try: + ctrl = s.symbols.lookup("<control-symbol>") # demo: gsp_ctrl + s.hook("<handler-symbol>", + on_enter=lambda h: s.note("enter", a0=h.arg(0)), + on_leave=lambda h: s.note("leave", retval=h.retval)) + before = s.snapshot(ctrl, 0x100, "before") + s.memory.write_u64(ctrl + 0x20, 1) # stimulus + s.run(duration=0.5) + after = s.snapshot(ctrl, 0x100, "after") + print(Memory.render_diff(Memory.diff(before, after, base=ctrl), s.symbols)) + finally: + s.close() + ``` + +4. Decode the captured bytes with the structure schema so the answer is + field-by-field, not byte noise: + + ```python + import json + from greeneye.structs import schema_from_json, decode, render + schema = schema_from_json(json.load(open("firmware/uproc_demo/gsp_ctrl.schema.json"))) + values = decode(schema, s.memory.read(ctrl, schema.size), base=ctrl) + print(render(schema, values)) + ``` + +5. Compare against the reference capture with + `greeneye.analyze.compare_sessions(...)` / `./greeneye.sh compare …` for + ordering (M3) and state the field diff in words (M2). If the reply is in a + different object, repeat step 3 on that object's address. +6. Report: question, session pair, `compare` output, M2/M3 rows, fidelity + statement, explicit "cannot answer because L3 absent" where applicable. + +### 4.2 Find why a handler disappeared + +1. Build/refresh the static inventory for both firmware versions: + + ```bash + $GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \ + -scriptPath debug/risc-v-gsp-rm-vm/ghidra \ + -postScript ExportForGreenEye.java /tmp/symbols-<ver>.tsv + ./greeneye.sh symbols export --index /tmp/symbols-<ver>.tsv --bias 0 -o symbols-<ver>.json + diff <(cut -f2 /tmp/symbols-old.tsv | sort) <(cut -f2 /tmp/symbols-new.tsv | sort) + ``` + +2. Probe the new image for early structure changes: + + ```bash + ./greeneye.sh probe <new.fwimage> --symbols symbols-<ver>.json --record + ``` + +3. Record the same stimulus against each version. Hook the old handler name; + a missing symbol is itself evidence — catch `SymbolNotFound` and log it + rather than aborting the run. Also hook the dispatcher and the old + handler's neighbours from the inventory diff. +4. Compare the sessions: + + ```bash + ./greeneye.sh compare dumps/<old-run> dumps/<new-run> --symbols + ./greeneye.sh analyze dumps/<new-run> --symbols symbols-<ver>.json --out new.md + ``` + +5. Classify the disappearance with the table below, then confirm in Ghidra: + + | Observation | Likely cause | Next step | + |---|---|---| + | name absent from new symbol DB | removed or renamed before build | match by size/callgraph in Ghidra | + | old function exists, dispatcher hits moved to another symbol | merged or re-routed | `run_until` the new dispatcher entry, `bp_set` there | + | old function exists, zero hits under the standard stimulus | coverage gap or decoy | widen stimulus, check xrefs from the dispatch table | + | dispatch table entry changed but code identical | behaviour moved (state/input) | inspect the table with bridge `read_mem`/snapshot | + +6. With the bridge running (`./greeneye.sh ghidra --kernel <elf>`), execute + until the dispatch site from the decompiler and set breakpoints: + + ```bash + $GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \ + -scriptPath debug/risc-v-gsp-rm-vm/ghidra \ + -postScript GreenEyeControl.java 7000 0x<dispatch-address> + ``` + +7. Export hits and annotate the old function's fate: + + ```bash + ./greeneye.sh analyze dumps/<new-run> --symbols symbols-<ver>.json --out new.md + python3 - <<'PY' + from greeneye.ghidra import export_hits + from greeneye.symbols import SymbolDB + db = SymbolDB(); db.load_json("symbols-<ver>.json") + print(export_hits("dumps/<new-run>", "greeneye_hits.tsv", symbols=db)) + PY + ``` + +8. Report: static delta, dynamic delta, classification, and whether the + emulation must change (with the docs/14 evidence pack). + +### 4.3 Prototype a service + +1. Build the SDK reference service and prove position independence: + + ```bash + make -C sdk + make -C sdk relocs # must print "no relocations in this file" + ``` + +2. Inject and invoke through both paths (direct debugger call, then firmware + mailbox dispatch): + + ```bash + ./greeneye.sh inject --service sdk/build/injected_demo.elf \ + --cmd 1 --arg0 0x41 --arg1 0x42 + ``` + +3. Iterate programmatically when the CLI shape is not enough: + + ```python + from greeneye.inject import load_service, invoke_remote, invoke_via_firmware + svc = load_service(s, "sdk/build/injected_demo.elf", slot=6) + print(hex(invoke_remote(s, svc, 2, 0x10, 0x20))) + s.run(duration=0.3) + print(hex(invoke_via_firmware(s, svc, 2, 0x10, 0x20))) + ``` + +4. Instrument the service like firmware: hooks on the entry, watchpoints on + its data, snapshots before/after each command (docs/11 §7). +5. Compare two versions of the service to see the behavioural delta: + + ```bash + ./greeneye.sh compare dumps/<svc-v1> dumps/<svc-v2> --symbols + ``` + +6. Stop condition: direct and firmware-dispatched results agree, the session + is recorded, and the prototype is documented as development-only. Product + code goes to `open-iov-vfio` only through the accuracy loop (docs/11 §8, + docs/12 §9); GSP-RM registration is roadmap H1.3. + +### 4.4 Prepare a portability report + +1. Follow the P1 loop in docs/07-PORTABILITY-PLAYBOOKS.md with real commands: + + ```bash + ./greeneye.sh probe <old.fwimage> --symbols symbols-old.json --record + ./greeneye.sh probe <new.fwimage> --symbols symbols-new.json --record + ./greeneye.sh record --kernel <rehosted-old.elf> --duration 2.0 \ + --watch gsp_ctrl+0x10:8 --snapshot gsp_ctrl:392 + ./greeneye.sh record --kernel <rehosted-new.elf> --duration 2.0 \ + --watch gsp_ctrl+0x10:8 --snapshot gsp_ctrl:392 + ./greeneye.sh compare dumps/<old-run> dumps/<new-run> --symbols + ``` + +2. Keep the stimulus identical across versions: use one script/probe spec and + record it in the pack (docs/14 §4.1). +3. Fill the M1–M6 table from docs/06-ACCURACY.md. M7 (timing) is never + claimed; L2/L3 statements are mandatory. +4. Produce the report with the docs/14 §6 template and attach session hashes, + symbol DB hashes and the Ghidra export provenance. +5. Verification agent step: replay the candidate stimulus against the checked-in + golden session (roadmap V1) and require `compare` to match expectations. + +## 5. Guardrails + +Hard rules (no agent may override them): + +* **Never claim cycles, time or reboots from the simulator.** L2 is not + modelled; L3 is absent until H2 (docs/06). +* **No guest modifications** and no instrumentation of a guest driver; the + tool is never shipped and never present in a guest (docs/00, V4). +* **Sessions are mandatory evidence.** A finding without a session directory + and a `compare` report is not a finding (docs/11 §8). +* **No unattended hardware.** The agent runs against the simulator by default; + physical GPU cycles are supervised and human-driven (H7). +* **Human approval for product changes.** Agents produce proposals and + evidence packs; only humans merge `open-iov-vfio` changes. +* **Budgets everywhere.** Every run uses `duration=`/`max_stops=`/`timeout=` + and a wall-clock cap; a wedged target leaves the session stopped and is + restarted clean rather than poked. + +Capability manifest for actions (proposed file for the harness; see §7): + +| Class | Examples | Who runs it | Preconditions | +|---|---|---|---| +| Read | `info`, `plugins`, `symbols resolve`, `analyze`, `compare` | agent, unattended | path/DB provenance recorded | +| Observe | `probe`, `record`, `demo`, hooks/watch/step | agent, unattended on simulator | timeout, session recording on | +| Write (target) | `memory.write*`, `call`, `run_until`, `inject`, mailbox stimulus | agent with manifest | declared address/range, fresh session, before/after snapshots, mutation values noted | +| Product | `open-iov-vfio` patch | human only | M1–M6 + physical confirmation rules (docs/07 P4.5) | +| Hardware | physical cycles/reboots | human only | supervised, never unattended | + +A write action's manifest entry names the session, the target range, the +purpose and the budget; the session records the action, so the manifest and +the evidence can be cross-checked. + +## 6. Evaluation + +An agent workflow is judged on the same axis as the human one: how long until +a reviewable answer, and does the answer survive verification. + +| Metric | Definition | Instrument | +|---|---|---| +| TTFA | question → first evidence-backed answer | wall clock; session count | +| TTEP | question → complete evidence pack (docs/14 §6) | pack checklist | +| Evidence yield | sessions/`compare` reports per accepted finding | `dumps/` inventory | +| Verification pass rate | findings reproduced by the verification role | replay runs | +| Coverage | inventory observed vs emulated (M1) | docs/06 table | +| Accuracy trend | M1–M6 per release | docs/06 tables | +| Regression lead time | firmware drop → delta report | probe/compare timestamps | +| Human minutes | manual minutes per accepted finding | benchmark log | + +Benchmark protocol: take five questions from recent history, run each twice — +once with the agent (all actions recorded) and once manually — on the same +machine and firmware set, then compare TTFA/TTEP/evidence yield. Re-run the +manual side offline against the agent's sessions to separate speed from +correctness. Publish the ratios with the raw session directories, not just +the numbers (the project's "never a single run" rule applies to tooling too). + +Golden data is the regression bed: the demo session and re-hosted workloads +are checked in CI (roadmap V1), and every new stimulus script should grow a +golden expectation once it is stable. + +## 7. Roadmap hooks (H7) and what would make this real + +| H7 item | What exists today | What is missing | +|---|---|---| +| MCP plugin (R3) | JSON-lines Ghidra bridge; sessions as data | MCP server plugin (`plugins/mcp_server.py` when built) | +| deterministic harnesses | `run` scripts, probes, plugins, `record`/`compare` | named harness entry points, budgets, manifests | +| sessions as evidence | `greeneye-session/1`, `summary.md`, `compare` | session index, signed/hashed pack tooling | +| autonomous loops | hooks/watchpoints/compare primitives | hypothesis/verification controllers, stop conditions | +| safety | local-only bridge, no network, no hardware | enforced capability manifest, process isolation | + +What would make this real: + +1. **MCP tool schema** (sketch; planned only, no code in v0.1): + + ```json + { + "transport": "local stdio; no network exposure by default", + "tools": [ + {"name": "list_sessions", "args": {}}, + {"name": "open_session", "args": {"dir": "string"}}, + {"name": "symbols_resolve", "args": {"session": "string", "name": "string"}}, + {"name": "memory_read", "args": {"session": "string", "addr": "string", "size": "int"}}, + {"name": "memory_hexdump", "args": {"session": "string", "addr": "string", "size": "int"}}, + {"name": "set_breakpoint", "args": {"session": "string", "target": "string", "kind": "sw|hw"}}, + {"name": "watch", "args": {"session": "string", "target": "string", "size": "int", "kind": "w|r|a"}}, + {"name": "run", "args": {"session": "string", "duration": "float", "max_stops": "int"}}, + {"name": "run_until", "args": {"session": "string", "target": "string", "timeout": "float"}}, + {"name": "snapshot", "args": {"session": "string", "addr": "string", "size": "int", "label": "string"}}, + {"name": "compare_sessions", "args": {"a": "string", "b": "string", "symbols": "bool"}}, + {"name": "export_hits", "args": {"session": "string", "out": "string"}} + ] + } + ``` + + Baseline names follow docs/08 R3; `watch` and `run_until` follow the + bridge command set and H7. + +2. **Harness scripts** with fixed names and budgets: `stimulus.py` (one + workload), `probe_matrix.py` (version × device), `golden_check.py` + (compare against expected profiles). Today these are written ad hoc. +3. **Golden data**: checked-in expectation files for the demo and at least one + re-hosted workload, wired into `make test` (roadmap V1). +4. **Capability manifest** file the harness reads before any write action + (§5 table), with the session recording as the audit trail. +5. **Session index** for agents: a small JSONL listing session dirs, questions + and outcomes, so triage starts from history instead of from zero. +6. **Physical-escalation gate**: a write-up checklist that must be complete + before an agent-proposed change reaches a supervised hardware cycle. + +Until R3 lands, the agent loop is: CLI/scripts + Ghidra bridge + sessions — +nothing more, and the documentation should keep saying so. diff --git a/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md b/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md new file mode 100644 index 0000000..2b77ca3 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/docs/16-CUSTOM-SIMULATOR.md @@ -0,0 +1,449 @@ +# 16 — Custom simulator: design and implementation plan +> **# TOOL IN DEVELOPMENT - NOT READY YET!** See 00 for status; this is a +> far-later design, do not implement ahead of the H3 gate. + +## 1. Purpose and status + +This document is the design and delivery plan for a green-eye-native execution +backend: a small deterministic RISC-V machine (`libgreenmachine`) implementing +the same `Session` contract as today's QEMU backend. It expands +`GREEN-EYE-ROADMAP.md` §H3 (H3.1–H3.4) and the §H6 soft-processor annex. + +**Nothing here exists today.** The only supported backend is `QemuTarget` + +`RspClient` (`greeneye/target.py`, `greeneye/rsp.py`); native is gated behind +H3 entry — H2.2, the QEMU device model that makes the real GSP image reach its +message loop — and must not be started ad hoc. This document only scopes it. + +## 2. Why a custom simulator + +QEMU is an excellent execution engine and a poor *workflow* engine here. The +limits below are consequences of QEMU being a whole-board emulator with a GDB +stub as its scriptable surface; each maps to a §H3.1 requirement. + +| QEMU property | Consequence for green-eye | Requirement it forces | +|---|---|---| +| RSP serialises every access (hex frame, TCP round-trip, `PacketSize` chunking) | register/memory polling and watchpoint step-over are transport-bound; full-access tracing is impractical | in-process backend ABI with direct memory/register calls | +| No cheap machine snapshots; `savevm` is board-level and not exposed over the stub | region snapshots are copied over RSP; no instant checkpoint exists | copy-on-write full-machine snapshot/restore | +| No in-process plugin ABI: TCG plugins are C objects against QEMU internals; `qRcmd` is stringly | per-access callback *objects*, Python-side devices and deterministic fault injection have no home | native plugin/callback ABI for regions and devices | +| Whole-board device model (`virt` models boards, not a TU104-class firmware environment) | device behaviour lives in QEMU code and cannot be versioned as data with a firmware profile | memory-map DSL + small device modules, versioned as profiles | +| Host clock and iothread drive timers and wakeups | two runs of the same firmware interleave differently; replay is not reproducible | deterministic scheduler, virtual time, event-log replay | +| GDB stub stop model | instructions between stops are invisible unless a trace plugin is built and loaded | event bus emitting `exec`/`access` natively | +| Injection is debugger writes plus `Session.call` | RM-core call interception (stub/log/forward) is impossible at the ABI boundary | injection ABI with service registry and interceptors | + +The §H3.1 contract: RV64IMAC (+GC when needed) interpreter at ≥10–50 MIPS on +one host thread (not realtime); memory map DSL; deterministic scheduler; +snapshot/restore; the existing `events.jsonl` schema; a backend ABI identical +to `Session`'s expectations; an injection ABI; Python-first embedding. §H3.4 +adds: differential tests against QEMU, gate behind H2, minimal `Backend`. + +## 3. Goals and non-goals + +**Goals** + +1. Byte-for-byte compatible `Session` behaviour for every script, probe, + plugin, recording and comparison that works today. +2. Determinism: same image + profile + event log ⇒ identical event stream + (modulo host-only metadata) on every run and host. +3. Cheap snapshot/restore and time-travel from snapshot + replay. +4. Interceptable device and RM-core boundaries: accesses and calls can be + logged, stubbed, forwarded or (later, H4) mutated. +5. Firmware/device knowledge as versioned data (profiles), not code edits. + +**Non-goals** + +* Realtime or wall-clock guarantees; virtual time only. +* Confidentiality/isolation/security claims: a study environment only (§06 §6). +* Replacing QEMU before M4; native is opt-in (`--backend native`, M6). +* Graphics/board emulation, undeclared fidelity claims, or shipping: nothing + here becomes a dependency of `gsp-rm-vfio` or a guest. + +### 3.1 Fidelity claims + +Levels are exactly those of `docs/06-ACCURACY.md`; the native backend does +not get to invent new ones. + +| Level | Property | QEMU backend today | Native target (gated) | +|---|---|---|---| +| L0 | instruction semantics | high (TCG, same ISA family) | parity with QEMU on differential tests (M1) | +| L1 | memory behaviour | high for RAM-resident code | parity per profile; map is versioned data (M2) | +| L2 | time/concurrency | not modelled | deterministic virtual time and event order, **not** cycle accuracy (M3); cycle claims only via H6 | +| L3 | device environment | absent | parity with the H2.2 device model (M4) | + +No L2/L3 conclusion may be published before its gate; timing is out of scope +entirely — the L2 row means reproducibly *ordered* events, not a timing model. + +## 4. Architecture + +The control plane is unchanged: `Session`, hooks, watchpoints, `Memory`, +`SymbolDB`, `Recorder`, `analyze`/`compare`, probes, plugins and the Ghidra +bridge sit above a backend interface. The §H3.2 sketch becomes: + +``` ++-------------------- green-eye Python control plane --------------------+ +| Session API, hooks/watchpoints, recorder, plugins, Ghidra/agent bridge | ++---------------------------- Backend interface -------------------------+ + | QEMU-RSP backend (today) | native backend (planned) + v v + qemu-system-riscv64 libgreenmachine + + greeneye/rsp.py + cpu/ decoder + execute + + greeneye/target.py + mem/ map DSL, COW snapshots + + dev/ falcon, csb, ptimer, dma, pmu + + sched/ cores, timers, event queue + + trace/ event bus, exec/access + + inject/ blob loader, interceptors +``` + +Design rules: (1) the backend implements transport primitives only — +Frida-like `run_until`/`call` stay in `Session` (`greeneye/session.py:373`, +`:403`) and work on every backend for free; (2) device knowledge is data +(profile) plus a small module, never a core special case; (3) every backend +is optional — the Python package stays stdlib-only, and the native artifact +is loaded at session construction with a version check. + +## 5. Backend interface + +`Session` today assumes an RSP-like client. The planned interface is the same +small set of operations; `run_until` and `call` are implemented above it. + +```python +class Backend(Protocol): + def capabilities(self) -> set[str]: ... # machine_snapshot | injection | haystack + def close(self) -> None: ... + def read_registers(self) -> dict[str, int]: ... + def read_reg(self, name: str) -> int: ... + def write_reg(self, name: str, value: int) -> None: ... + @property + def pc(self) -> int: ... + def read_mem(self, addr: int, size: int) -> bytes: ... + def write_mem(self, addr: int, data: bytes) -> None: ... + def step(self) -> StopReply: ... + def continue_(self, budget: Budget | None = None) -> StopReply: ... + def interrupt(self) -> StopReply: ... + def set_sw_break(self, addr: int) -> None: ... + def clear_sw_break(self, addr: int) -> None: ... + def set_hw_break(self, addr: int) -> None: ... + def clear_hw_break(self, addr: int) -> None: ... + def set_watch(self, addr: int, size: int, kind: str) -> None: ... + def clear_watch(self, addr: int, size: int, kind: str) -> None: ... + def save_moment(self, label: str) -> str: ... + def restore_moment(self, moment_id: str) -> None: ... + def inject(self, blob_elf: str, slot: str) -> "InjectionHandle": ... +``` + +| Session need | Backend primitive | QEMU-RSP backend (today) | Native backend (planned) | +|---|---|---|---| +| registers | `read_registers`/`read_reg`/`write_reg` | `g`/`p`/`P` packets | register file + CSRs, direct | +| memory | `read_mem`/`write_mem` | `m`/`M`/`X`, chunked to `PacketSize` | region lookup; RAM direct, MMIO callbacks | +| software break | `set_sw_break`/`clear_sw_break` | `Z0`/`z0`; `Session._step_over` | breakpoint table in the dispatch loop | +| hardware break | `set_hw_break`/`clear_hw_break` | `Z1`/`z1` | PC match | +| watchpoint | `set_watch`/`clear_watch` | `Z2/Z3/Z4`; stops *at* the accessing instruction | access check; contract below | +| step | `step` | `s` | one decoded instruction | +| continue | `continue_(budget)` | `c` + wall timeout + interrupt | run to breakpoint/budget/event | +| stop | `interrupt` | `0x03` | out-of-band flag checked per instruction | +| region snapshot | `read_mem` via `Memory` | copy over RSP | direct map copy | +| machine snapshot | `save_moment`/`restore_moment` | unsupported (raise, never lie) | COW pages + `moment.json` | +| injection | `inject` | docs/12 path: writes + `Session.call` | loader + service registry + interceptors | +| virtual time | `budget`, `StopReply.vt` | absent (host wall clock) | instruction count + timer ticks | + +Compatibility decisions: + +* `Session(client=...)` keeps working; `Session(backend=...)` is added and + `client=` becomes an alias for `RspBackend(client)`. `Memory` is re-typed + against `Backend`; its public methods do not change. +* **Watchpoint contract.** QEMU stops after the watched access and requires + the clear/step/re-arm dance (`greeneye/session.py:538`). Native advertises + `watch_after_access` so the Session skips the dance on that path only; a + `watch_before_access` mode exists for injection experiments but is not the + default, which must reproduce today's event profiles. +* `Budget` is `{instructions, until_event}`; the wall-clock `duration=` + argument is translated via the profile's nominal clock, so native `run` is + deterministic. `StopReply` gains optional `vt` and `access`, defaulting for + old parsers. + +## 6. Core choice: interpreter first, JIT later + +| Option | Determinism | Speed | Debuggability | Cost / risk | +|---|---|---|---|---| +| Pure-Python interpreter | highest | very low (≪1 MIPS) | highest | prototype only; excellent oracle | +| Decode/execute interpreter (recommended M1–M5) | highest | 10–50 MIPS achievable in Rust/C++ | high; per-instruction hooks and single-step are trivial | low-moderate | +| JIT (copy-and-patch / TCG-like) | high if codegen is fixed | ≫100 MIPS | hard: self-modifying code, translation-cache invalidation | deferred to M6, only if profiling demands | +| Existing core (e.g. a small open RISC-V interpreter/JIT) | depends | varies | depends | evaluation only; hooks/snapshots must be retrofit | + +**Decision.** Build a decode/execute interpreter with the ISA behind a +`CpuCore` interface: a decoder mapping instruction words to typed operations, +an execute loop dispatching into the memory map. `MISA` is a profile property +(`rv64imac` default, `rv64gc` selectable, per 07 §P3); compressed +instructions are canonicalised first so breakpoints and single-step work +uniformly. No FPU/vector until a GC profile is needed. An embedded existing +core (public options such as `libriscv`/`rv8`) is an **evaluation note only**: +revisit at M4 as a `CpuCore` if its memory callbacks, watchpoint granularity, +snapshot hooks and licence/provenance fit — it is not the M1 substrate, +because those hooks *are* the product. + +## 7. Memory map DSL and device plugins + +The map is versioned data interpreted by `mem/`. Region kinds are `ram`, +`mmio`, `dma` and `host-window`; each has attributes, logging policy and +either a backing store or a device. Windows resolve to device instances. + +```yaml +schema: greenmachine-profile/1 +profile: tu10x-gsp +isa: rv64imac +devices: + falcon0: { module: falcon, instance: gsp } + csb0: { module: csb, mailbox_depth: 8 } + ptimer0: { module: ptimer, hz: 1000000 } + pmu0: { module: pmu_stub, revision: r1 } + dma0: { module: dma, iommu: pass-through } +regions: + - { name: tcm, base: 0x4000000, size: 0x40000, kind: ram, attrs: [r,w,x] } + - { name: gsp_regs, base: 0x2000000000000000, size: 0x100000, kind: mmio, + log: changes, # off | changes | access + windows: [{ offset: 0x000000, size: 0x1000, device: falcon0 }, + { offset: 0x001000, size: 0x1000, device: csb0, writable: true }, + { offset: 0x002000, size: 0x1000, device: ptimer0 }, + { offset: 0x200a00, size: 0x0200, device: pmu0 }] } + - { name: host_dma, base: 0x100000000, size: 0x10000000, kind: dma, + device: dma0 } +``` + +Device modules are small in-process plugins; the same interface is what makes +`stub`/`log`/`forward` interception possible later: + +```python +class Device(Protocol): + def reset(self, ctx) -> None: ... + def read(self, ctx, offset: int, size: int) -> int: ... + def write(self, ctx, offset: int, size: int, value: int) -> None: ... + def tick(self, ctx, now: int) -> list[Event]: ... + def save_state(self) -> dict: ... + def load_state(self, state: dict) -> None: ... +``` + +Planned modules (all stubs until the H2.2 model defines semantics): + +| Module | Role | Notes | +|---|---|---| +| `falcon` | FALCON/SEC2 register windows, imem/dmem, bootvec | read-back semantics for boot polling | +| `csb` | CSB/TRANSCOM mailbox windows | message queues, doorbells as events | +| `ptimer` | programmable timer | feeds the scheduler; deterministic ticks | +| `dma` | descriptor engine into host/FB windows | completions are scheduled events, not threads | +| `pmu_stub` | identity/clock provisioning reads | canned but recorded; never a timing source | +| `boot_shim` | FWSEC/booter/SEC2 hand-off state | documented shim; assumptions in session metadata | + +Profiles are keyed by `(device, firmware-family, profile-version)` and carry +the firmware hash, provenance, shim assumptions and the validated map. A new +pairing is a new profile, never a core change; profiles are JSON-Schema +validated in CI. + +## 8. Deterministic scheduler + +Virtual time advances only by executed instructions and scheduled events; no +host clock, thread or entropy source may influence guest-visible state. + +* **Cores.** One or more vCPUs with instruction budgets; the default is a + strict, seeded round-robin so interleavings are reproducible. +* **Event queue.** Timers, IRQ lines, DMA completions and external stimuli + are ordered `(vt, kind, source, payload, seq)`; ties break by sequence + number, never host timing. `ptimer` ticks enqueue IRQ events; delivery + defaults to instruction boundaries, and injected interrupts are explicit + and recorded. +* **Replay.** A recording captures a seed plus the ordered external-event + log; re-running the same image, profile and log must reproduce the event + stream and final memory hashes. The determinism test runs every suite twice + and compares a canonical hash. +* **Hygiene (tested).** No `dict` iteration order in device dumps, no + unordered set serialisation, no host paths in semantic fields, fixed page + allocation order, fixed hash seeds where a container affects order. + +## 9. Snapshot/restore and time-travel + +A machine snapshot ("moment") is a new artifact type, not a change to region +snapshots (`snapshots/*.bin.z`, `01-DESIGN.md §6`): + +``` +moments/<id>/ + moment.json # greenmachine-moment/1, profile id+hash, vt, event-log offset, + # register file, CSR/MPU state, scheduler cursor, device state + pages/ # COW page list: base page id + XOR/delta or full page + events.tail # optional copy of events since the previous moment +``` + +* **Memory** is page-granular copy-on-write from a base moment; a 256 MiB + image that changes little is a page list, not a copy. **Device state** + comes from `Device.save_state()`; devices without it are marked "reset + required" and the moment is partial. **Event-log position** indexes + `events.jsonl`; restore replays external events from that index. +* **Use cases (M2/M3):** checkpoint before a fault and explore both branches; + flip one reply byte and compare; restore and re-run a suffix to delta-debug + an injected service; seed a fuzzing harness from a moment. **Limits:** + in-flight DMA and out-of-band input must be in the event log, host/FB + windows backed by real files are excluded, and the profile hash must match. + +## 10. Event bus + +The recorder schema (`greeneye-session/1`, one JSON object per line with `t` +and `type`, `greeneye/record.py`) is preserved exactly. The native bus is the +single producer for stop-based and full-access events: + +| Event type | Status | Fields | +|---|---|---| +| `session_start`, `image_loaded`, `hook`, `watch`, `note`, `exit`, `session_end` | existing | unchanged | +| `bp_hit`, `leave`, `watch_hit`, `stop`, `step`, `mem_read`, `mem_write`, `snapshot*`, `probe*`, `plugin*` | existing | unchanged; `mem_*` may now come from the bus, not only tooling | +| `exec` | new (M1, optional) | `pc`, `insn`, `vt`, `core`, `symbol` | +| `access` | new (M2) | `pc`, `addr`, `size`, `rw`, `region`, `vt`, `core` | +| `device` | new (M2) | `device`, `name`, `rw`, `value`, `vt` | +| `irq` / `schedule` | new (M3) | `line`, `state`, `core`, `vt`, `reason` | +| `inject_*` | existing shape (docs/12) | `service_injected`, `call_enter`, `call_return` unchanged | + +`t` stays the recorder's monotonic seconds **filled from virtual time**, so +the same run yields the same file. `exec`/`access` volume is controlled by +profile logging and a ring budget; the analyzer gains coverage/heatmap +rendering (roadmap R4). `greeneye compare` treats backend identity, `t` and +host paths as non-semantic, so cross-backend comparison is first-class. + +## 11. Injection ABI + +The docs/12 blob contract is reused unchanged: ELF64 `EM_RISCV`, RV64IMAC, +soft-float, PIC, one dispatch entry, optional `ge_service_t` metadata. The +native backend changes *where the blob is registered* and *what can be +intercepted* (H1.3/H3.1): + +* **Loader.** `Backend.inject()` verifies PIC/no-relocations, maps PT_LOAD + segments through the memory map, and records `service_injected`. +* **Service registry.** The demo path registers into the firmware handler + slot as today; the RM path (M5) registers into a service table presented to + the simulated RM core, so the firmware dispatches to the service itself. +* **Interception points** are declared in the profile and matched at runtime: + +| Mode | Behaviour | Intended use | +|---|---|---| +| `log` | real call executes; target, args and return recorded | observation, M5 evidence | +| `stub` | simulator returns a canned reply; real code skipped | isolate the RM front-end from an engine | +| `forward` | call routed to a nominated implementation or injected service | prototype a replacement engine | +| `mutate` (gated H4) | argument or reply deterministically corrupted; mutation id recorded | fuzzing harness | + +**ABI v2 sketch (M5).** `ge_service_fn(cmd, a0, a1)` becomes +`ge_service_v2(ctx, cmd, a0, a1)` where `ctx` carries a call table; blobs +declare their ABI version so loaders can refuse mismatches. Interception +applies to call-table calls, MMIO accesses and the RM dispatch boundary. + +## 12. Language and embedding options + +| Option | Speed | Embedding | Safety/ergonomics | Build & CI cost | Assessment | +|---|---|---|---|---|---| +| C/C++ core | high | CPython C API or pybind11 | manual memory; best if a core is embedded later | compiler matrix; sanitizers essential | viable fallback; no build advantage | +| Rust core + PyO3 | high | first-class via PyO3/maturin | memory-safe; enums/decoders fit the ISA well | cargo + maturin; easy wheel caching | **recommended** | +| Pure Python | lowest | native | fastest iteration; great reference model | none | M0/M1 oracle, not the product core | + +**Recommendation.** Build a pure-Python reference interpreter first as an +executable specification and differential oracle (small, CI-friendly, forces +the memory-map semantics to be testable), then the production core in Rust +with PyO3 behind an identical C-visible ABI. The C ABI lets a C/C++ core or +an existing core slot into `CpuCore` without touching the control plane. + +## 13. Implementation plan (expands H3.3) + +Entry gate for all milestones: **H2.2 exit** and a stable H2.2 device +semantics baseline. Effort is rough engineering-weeks for one developer +familiar with the codebase; milestones are sequential. + +| # | Scope | Exit criteria | Effort | +|---|---|---|---| +| M1 | RV64IMAC interpreter, RAM, NS16550A-compatible UART, loader; `Session` adapter with `Backend` | bundled demo uproc boots unmodified; the canned instrumentation script (`03 §12`) produces the same hook/watch/stop timeline and memory snapshots as QEMU (canonical comparison); determinism test green | 4–6 wk | +| M2 | Memory-map DSL + `falcon`/`csb`/`ptimer` devices + profile validation; native region snapshots; moments v1 | a re-hosted uproc build boots from a profile with no core changes; moment save/restore round-trips; restoring reproduces the event-stream hash | 4–6 wk | +| M3 | Deterministic scheduler (budgets, timers, IRQ events) + event log/replay | multi-task test profile replays identically from the log on two hosts; tie-break and replay tests in CI | 3–4 wk | +| M4 | Device-model parity with H2.2 + `dma`/`pmu_stub` + boot shims | real GSP image reaches the same stable state as the QEMU H2 exit under the same profile; differential suite passes for probe and boot fixtures; L3 parity documented, L2 still not claimed | 6–10 wk | +| M5 | Injection ABI + service registry + `log`/`stub`/`forward` interception | an SDK v1 blob loads and serves a control end-to-end; each interception mode has a recorded acceptance session; fuzzing seed/restore demonstrated | 4–6 wk | +| M6 | Performance/ergonomics: profiling, fast paths, `--backend native`, docs + flag-removal criteria | §14 targets met; `greeneye demo --backend native` in the differential CI job; user docs updated | 3–4 wk | + +Total: roughly 24–36 engineering-weeks, consistent with the roadmap's +"months; explicitly later" classification; none of it starts before the H3 +entry gate, and if H2 slips this plan waits. + +## 14. Differential testing and performance targets + +**Differential testing (the fidelity gate for every milestone).** Fixture: +the bundled demo firmware plus a re-hosted uproc build once M2 exists. The +same script runs on both backends; recordings are canonicalised (drop `t`, +host paths, backend identity, QEMU-log-only fields) and compared: identical +event-type sequence/counts; identical hook/watch hits and `pc` per stop; +byte-identical region snapshots at matched points; `exec`/`access` streams +equal in order, multiplicity and virtual timestamps at equal log levels. + +Divergence blocks the milestone and is filed with both session directories as +evidence (`06 §5`/`§7`); a declared fidelity gap is recorded, not waived. + +**Performance targets** (one host thread; a checked `greeneye-sim bench` +harness, trended in CI): + +| Metric | Target | Gate | +|---|---|---| +| RAM-bound tight loop (no logging) | ≥10 MIPS | M1 | +| Interpreter optimized | ≥50 MIPS | M6 | +| JIT (only if built) | ≥200 MIPS | post-M6, gated | +| Moment save, 256 MiB RAM, sparse changes | ≤100 ms | M3 | +| Moment restore | ≤50 ms | M3 | +| Watchpoint-enabled slowdown | ≤2× | M4 | +| MMIO access vs RAM access | ≤5× | M2 | +| Event bus throughput | ≥1 M events/s buffered, lossless under budget | M3 | + +## 15. CI strategy + +* The Python package and QEMU path remain the default, always-on suite; no + native artifact is required for it. +* Native tests carry a `native` marker and `pytest.importorskip`-style + detection: without the built backend they skip, never fail. A dedicated job + builds the core (cargo/maturin), runs native + differential tests, the + determinism double-run and profile-schema validation; nightly while M1 is + in flight, required at M1 exit. +* Real-image fixtures are opt-in (`GREENEYE_FW`); the default job uses the + bundled demo. Golden sessions are per backend and profile-hash checked. + +## 16. Annex — Verilog soft processor (roadmap H6, far later) + +**Scope (H6.1).** An RTL soft implementation of the GSP core environment: +RV64 core with TCM, MPU/PMP, vectored interrupts and timers; a device fabric +with falcon windows, CSB/TRANSCOM, DMA to host and WPR2/FWSEC state. RTL is +separate from `libgreenmachine`; it only shares profile data. + +**Run modes and integration (H6.2).** Verilator for determinism and +coverage, FPGA for speed, optionally both behind one co-simulation bridge. A +`Backend` talks to the simulation's clock/step and memory API: `step` becomes +clock edges, breakpoints become traps, watchpoints become signal/page +monitors, snapshots use saved simulation state. A new `cycle` event class +makes L2 timing *measurable* — only under docs/06 caveats, and physical +measurement is still required for any silicon claim. + +**Gates (H6.3).** Requires H2/H3 proven, a validated device map, and a funded +reason to care about cycle accuracy; not on the critical path and not before +the baseline project-roadmap goals. **Security posture:** a study +environment, never a confidentiality boundary, never shipped, never in a +guest; no isolation or side-channel claim may be based on it (`06 §6`). + +## 17. Risks and mitigations + +| Risk | Impact | Mitigation | +|---|---|---| +| Fidelity divergence from QEMU/hardware | wrong conclusions | differential suite from M1; milestones gated on canonical event agreement; declared gaps recorded per docs/06 | +| Two backends to maintain | effort/tool decay | minimal `Backend` interface (H3.4); shared `Session` primitives; QEMU stays the reference backend in CI | +| Effort creep before the gate | roadmap damage | explicit H2.2 entry gate; this document is design-only until then | +| Determinism leaks (clock, hashing, dict order, host input) | replay broken, trust lost | double-run determinism test; sequence-number tie-breaks; no host time in semantic fields | +| DSL insufficient for device semantics | profile churn, core edits | `Device` modules own whole windows; escape hatch is a module, never a core special case | +| Snapshot size/I/O blowup | snapshots unusable on real images | page-level COW + sparse encoding; region-scoped logging; moment retention policy | +| Injection misuse (real hardware/guest) | safety/security incident | local-only, dev-only tooling; no guest presence; interception only in the simulator; docs/12 policy checks | + +## 18. Open questions + +1. Which GC extensions must the first `rv64gc` profile cover, and can FPU + instructions be trapped and recorded rather than executed until then? +2. Are before-access watchpoints needed for injection, and how do their + recordings stay distinct from the QEMU-compatible profile? +3. What page size/encoding do moments need, what restore budget, and how much + RTOS semantics must be modelled for M3 (docs/09) without a timing claim? +4. Is Rust + Python sufficient, or does the profile DSL need a native loader, + and does an embedded public core's licence fit this repository (M4)? +5. How are multi-core interleavings represented so "same seed + same log" + stays the only replay contract, and which events must be byte-identical + across backends for the evidence rules? diff --git a/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/gsp_ctrl.schema.json b/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/gsp_ctrl.schema.json index ca18f81..e0d6cbc 100644 --- a/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/gsp_ctrl.schema.json +++ b/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/gsp_ctrl.schema.json @@ -1,19 +1,94 @@ { "name": "gsp_ctrl", - "size": 392, + "size": 400, "fields": [ - {"name": "magic", "offset": 0, "fmt": "u32", "note": "GSP-like control block magic 0x47535031"}, - {"name": "version", "offset": 4, "fmt": "u32"}, - {"name": "state", "offset": 8, "fmt": "u32", "enum": {"0": "S0", "1": "S1", "2": "S2", "3": "S3"}}, - {"name": "heartbeat", "offset": 12, "fmt": "u32"}, - {"name": "rpc_count", "offset": 16, "fmt": "u64"}, - {"name": "last_rpc", "offset": 24, "fmt": "u64"}, - {"name": "mailbox", "offset": 32, "fmt": "u64", "enum": {"0": "idle", "1": "PING", "2": "ECHO", "3": "FILL", "4": "SUM", "5": "STATE", "6": "FAULT"}}, - {"name": "arg0", "offset": 40, "fmt": "u64"}, - {"name": "arg1", "offset": 48, "fmt": "u64"}, - {"name": "result", "offset": 56, "fmt": "u64"}, - {"name": "buf", "offset": 64, "fmt": "bytes", "size": 256}, - {"name": "handler_count", "offset": 320, "fmt": "u64"}, - {"name": "handlers", "offset": 328, "fmt": "u64", "count": 8} + { + "name": "magic", + "offset": 0, + "fmt": "u32", + "note": "GSP-like control block magic 0x47535031" + }, + { + "name": "version", + "offset": 4, + "fmt": "u32" + }, + { + "name": "state", + "offset": 8, + "fmt": "u32", + "enum": { + "0": "S0", + "1": "S1", + "2": "S2", + "3": "S3" + } + }, + { + "name": "heartbeat", + "offset": 12, + "fmt": "u32" + }, + { + "name": "rpc_count", + "offset": 16, + "fmt": "u64" + }, + { + "name": "last_rpc", + "offset": 24, + "fmt": "u64" + }, + { + "name": "mailbox", + "offset": 32, + "fmt": "u64", + "enum": { + "0": "idle", + "1": "PING", + "2": "ECHO", + "3": "FILL", + "4": "SUM", + "5": "STATE", + "6": "FAULT" + } + }, + { + "name": "arg0", + "offset": 40, + "fmt": "u64" + }, + { + "name": "arg1", + "offset": 48, + "fmt": "u64" + }, + { + "name": "arg2", + "offset": 56, + "fmt": "u64" + }, + { + "name": "result", + "offset": 64, + "fmt": "u64" + }, + { + "name": "buf", + "offset": 72, + "fmt": "bytes", + "size": 256 + }, + { + "name": "handler_count", + "offset": 328, + "fmt": "u64" + }, + { + "name": "handlers", + "offset": 336, + "fmt": "u64", + "count": 8 + } ] } diff --git a/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/main.c b/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/main.c index d23b00c..9b4f98a 100644 --- a/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/main.c +++ b/debug/risc-v-gsp-rm-vm/firmware/uproc_demo/main.c @@ -21,12 +21,13 @@ typedef struct { u64 rpc_count; /* +0x10 */ u64 last_rpc; /* +0x18 */ u64 mailbox; /* +0x20 */ - u64 arg0; /* +0x28 */ + u64 arg0; /* +0x28 service command for injected dispatch */ u64 arg1; /* +0x30 */ - u64 result; /* +0x38 */ - char buf[256]; /* +0x40 */ - u64 handler_count; /* +0x140 */ - u64 handlers[8]; /* +0x148 */ + u64 arg2; /* +0x38 */ + u64 result; /* +0x40 */ + char buf[256]; /* +0x48 */ + u64 handler_count; /* +0x148 */ + u64 handlers[8]; /* +0x150 */ } gsp_ctrl_t; volatile gsp_ctrl_t gsp_ctrl __attribute__((section(".gsp_data"), aligned(64))); @@ -134,8 +135,9 @@ __attribute__((noinline)) u64 rpc_fault(u64 a0, u64 a1) /* ---- dispatcher ---- */ typedef u64 (*rpc_fn)(u64, u64); +typedef u64 (*ge_inject_fn)(u64, u64, u64); -void rpc_dispatch(u64 cmd, u64 a0, u64 a1) +void rpc_dispatch(u64 cmd, u64 a0, u64 a1, u64 a2) { u64 result; switch (cmd) { @@ -158,7 +160,17 @@ void rpc_dispatch(u64 cmd, u64 a0, u64 a1) result = rpc_fault(a0, a1); break; default: - result = ~cmd; + /* Injected-service dispatch: commands 16..23 call handler slots 0..7. + * A debugger can load a new service and store its entry in + * gsp_ctrl.handlers[n] while the firmware is running; this is the + * demo's stand-in for injecting an RM engine/service. */ + if (cmd >= 16 && cmd < 24) { + ge_inject_fn injected = (ge_inject_fn)(unsigned long)gsp_ctrl.handlers[cmd - 16]; + /* a0 carries the service command; a1/a2 are its parameters. */ + result = injected ? injected(a0, a1, a2) : ~cmd; + } else { + result = ~cmd; + } break; } gsp_ctrl.result = result; @@ -187,6 +199,7 @@ int main(void) ctl->rpc_count = 0; ctl->last_rpc = 0; ctl->mailbox = 0; + ctl->arg2 = 0; ctl->result = 0; ctl->buf[0] = '\0'; install_handlers(); @@ -199,13 +212,13 @@ int main(void) uart_puts("\n"); /* Deterministic self-test so freshly attached instrumentation sees events. */ - rpc_dispatch(1, 0x1111, 0); - rpc_dispatch(2, (u64)(unsigned long)"hello-green-eye", 0); - rpc_dispatch(3, 0x5a, 32); - rpc_dispatch(4, (u64)(unsigned long)ctl->buf, 16); - rpc_dispatch(5, 0, 0); - rpc_dispatch(5, 0, 0); - rpc_dispatch(5, 0, 0); + rpc_dispatch(1, 0x1111, 0, 0); + rpc_dispatch(2, (u64)(unsigned long)"hello-green-eye", 0, 0); + rpc_dispatch(3, 0x5a, 32, 0); + rpc_dispatch(4, (u64)(unsigned long)ctl->buf, 16, 0); + rpc_dispatch(5, 0, 0, 0); + rpc_dispatch(5, 0, 0, 0); + rpc_dispatch(5, 0, 0, 0); uart_puts("SELFTEST DONE\n"); for (;;) { @@ -213,7 +226,7 @@ int main(void) u64 cmd = ctl->mailbox; if (cmd != 0) { ctl->mailbox = 0; - rpc_dispatch(cmd, ctl->arg0, ctl->arg1); + rpc_dispatch(cmd, ctl->arg0, ctl->arg1, ctl->arg2); } for (volatile int i = 0; i < 2000; i++) { __asm__ volatile(""); diff --git a/debug/risc-v-gsp-rm-vm/ghidra/GreenEyeControl.java b/debug/risc-v-gsp-rm-vm/ghidra/GreenEyeControl.java new file mode 100644 index 0000000..abeee82 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/ghidra/GreenEyeControl.java @@ -0,0 +1,187 @@ +// GreenEyeControl - drive a live green-eye session from Ghidra. +// +// The script connects to the green-eye JSON-lines bridge (start it with +// `./greeneye.sh ghidra [--kernel <elf>]`), and exposes the operations a +// reverse engineer needs while reading the decompiler: +// +// ping() - check the bridge +// info() - session name, pc, symbol count, recording path +// resolve(name) - symbol/expression -> runtime address +// setBreakpointHere() - software breakpoint at currentAddress +// setBreakpointAt(addr) / clearBreakpointAt(addr) +// runUntilHere() - resume and stop when execution reaches currentAddress +// runUntil(addr) - same for an explicit address (hex string) +// readMemoryHere(size) - hex bytes at currentAddress +// readRegister(name) / writeRegister(name, value) +// callFunction(addr, args) - Frida-like remote call; returns a0 +// snapshotHere(size) - record a memory snapshot in the session +// listHits() - breakpoint hit counts from the session +// exportHits(path) - write greeneye_hits.tsv for MarkGreenEyeHits.java +// +// Usage (Ghidra headless): +// analyzeHeadless <proj> <name> -process <program> -noanalysis \ +// -scriptPath debug/risc-v-gsp-rm-vm/ghidra \ +// -postScript GreenEyeControl.java <bridge-port> [address] +// +// The optional address argument makes the script run until that address and +// print the hits; in the GUI, omit it to use the current cursor address. +// +//@category OpenIOV +//@menupath Tools.OpenIOV.GreenEyeControl + +import java.io.BufferedReader; +import java.io.InputStreamReader; +import java.io.PrintWriter; +import java.net.Socket; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import ghidra.app.script.GhidraScript; +import ghidra.program.model.address.Address; + +public class GreenEyeControl extends GhidraScript { + + private Socket socket; + private BufferedReader reader; + private PrintWriter writer; + private int requestId = 0; + + private static final Pattern OK = Pattern.compile("\"ok\"\\s*:\\s*(true|false)"); + private static final Pattern RESULT = Pattern.compile("\"result\"\\s*:\\s*(\\{.*\\}|\\[.*\\]|\"[^\"]*\"|[^,}]+)"); + + @Override + public void run() throws Exception { + String[] args = getScriptArgs(); + if (args.length < 1) { + println("usage: GreenEyeControl.java <bridge-port> [address]"); + return; + } + int port = Integer.parseInt(args[0]); + try { + connect(port); + println("green-eye: " + info()); + String target = args.length > 1 ? args[1] : null; + if (target == null && currentAddress != null) { + target = "0x" + Long.toHexString(currentAddress.getOffset()); + } + if (target != null) { + println("green-eye run_until " + target + " -> " + runUntil(target)); + println("green-eye hits: " + listHits()); + } + } finally { + disconnect(); + } + } + + // -- connection -------------------------------------------------------- + + public void connect(int port) throws Exception { + socket = new Socket("127.0.0.1", port); + reader = new BufferedReader(new InputStreamReader(socket.getInputStream())); + writer = new PrintWriter(socket.getOutputStream(), true); + } + + public void disconnect() throws Exception { + if (socket != null) { + socket.close(); + } + } + + /** Send one bridge command and return the raw response line. */ + public String send(String cmd, String argsJson) throws Exception { + requestId += 1; + String request = "{\"id\":" + requestId + ",\"cmd\":\"" + cmd + "\",\"args\":" + + (argsJson == null || argsJson.isEmpty() ? "{}" : argsJson) + "}"; + writer.println(request); + String response = reader.readLine(); + if (response == null) { + throw new Exception("green-eye bridge closed the connection"); + } + Matcher ok = OK.matcher(response); + if (ok.find() && "false".equals(ok.group(1))) { + throw new Exception("green-eye error: " + response); + } + return response; + } + + /** Extract the `result` field as a string (numbers, objects or lists). */ + public String resultOf(String response) { + Matcher matcher = RESULT.matcher(response); + return matcher.find() ? matcher.group(1) : response; + } + + // -- documented operations -------------------------------------------- + + public String ping() throws Exception { + return resultOf(send("ping", null)); + } + + public String info() throws Exception { + return resultOf(send("info", null)); + } + + public String resolve(String name) throws Exception { + return resultOf(send("resolve", "{\"name\":\"" + name + "\"}")); + } + + public String setBreakpointAt(String address) throws Exception { + return resultOf(send("bp_set", "{\"target\":\"" + address + "\"}")); + } + + public String setBreakpointHere() throws Exception { + return setBreakpointAt(addressHex(currentAddress)); + } + + public String clearBreakpointAt(long address) throws Exception { + return resultOf(send("bp_clear", "{\"addr\":" + address + "}")); + } + + public String runUntil(String address) throws Exception { + return resultOf(send("run_until", "{\"target\":\"" + address + "\",\"timeout\":10}")); + } + + public String runUntilHere() throws Exception { + return runUntil(addressHex(currentAddress)); + } + + public String readMemoryHere(int size) throws Exception { + return resultOf(send("read_mem", "{\"addr\":\"" + addressHex(currentAddress) + "\",\"size\":" + size + "}")); + } + + public String readRegister(String name) throws Exception { + return resultOf(send("read_reg", "{\"name\":\"" + name + "\"}")); + } + + public String writeRegister(String name, long value) throws Exception { + return resultOf(send("write_reg", "{\"name\":\"" + name + "\",\"value\":" + value + "}")); + } + + public String callFunction(String address, long... args) throws Exception { + StringBuilder json = new StringBuilder("{\"target\":\"" + address + "\",\"args\":["); + for (int i = 0; i < args.length; i++) { + if (i > 0) { + json.append(","); + } + json.append(args[i]); + } + json.append("]}"); + return resultOf(send("call", json.toString())); + } + + public String snapshotHere(int size) throws Exception { + return resultOf(send("snapshot", + "{\"addr\":\"" + addressHex(currentAddress) + "\",\"size\":" + size + ",\"label\":\"ghidra\"}")); + } + + public String listHits() throws Exception { + return resultOf(send("hits", null)); + } + + public String exportHits(String path) throws Exception { + return resultOf(send("export_hits", "{\"path\":\"" + path + "\"}")); + } + + private String addressHex(Address address) { + return "0x" + Long.toHexString(address.getOffset()); + } +} diff --git a/debug/risc-v-gsp-rm-vm/ghidra/MarkGreenEyeHits.java b/debug/risc-v-gsp-rm-vm/ghidra/MarkGreenEyeHits.java index c99c3e1..b140ddd 100644 --- a/debug/risc-v-gsp-rm-vm/ghidra/MarkGreenEyeHits.java +++ b/debug/risc-v-gsp-rm-vm/ghidra/MarkGreenEyeHits.java @@ -70,8 +70,7 @@ public class MarkGreenEyeHits extends GhidraScript { } else { missed++; } - createBookmark(address, "Note", "green-eye", - hits + " runtime hit(s)" + (symbol.isEmpty() ? "" : " " + symbol)); + createBookmark(address, "Note", "green-eye"); } } println("green-eye: annotated " + marked + " functions (" + missed diff --git a/debug/risc-v-gsp-rm-vm/greeneye/analyze.py b/debug/risc-v-gsp-rm-vm/greeneye/analyze.py index 07c24eb..66429e9 100644 --- a/debug/risc-v-gsp-rm-vm/greeneye/analyze.py +++ b/debug/risc-v-gsp-rm-vm/greeneye/analyze.py @@ -110,6 +110,34 @@ def render_summary_from(meta: dict, events: list[dict], symbols=None, max_rows: lines.append("* recorder counts: " + ", ".join(f"{k}={v}" for k, v in sorted(counts_meta.items()))) lines.append("") + # QEMU exception report (probe sessions) + fault_count = meta.get("qemu_fault_count") + first = meta.get("qemu_first_fault") + if fault_count or first: + lines.append("## QEMU exceptions") + lines.append("") + lines.append(f"* total exceptions: {fault_count if fault_count is not None else '?'}") + if meta.get("qemu_log_blob"): + lines.append(f"* qemu log blob: `{meta['qemu_log_blob']}`") + if first: + access = first.get("access") or {} + lines.append("") + lines.append("| field | value |") + lines.append("|---|---|") + lines.append(f"| descriptor | `{first.get('desc', '?')}` |") + lines.append(f"| cause | `{first.get('cause', 0):#x}` ({_RISCV_CAUSES.get(first.get('cause', 0) & 0xFF, '?')}) |") + epc = first.get("epc", 0) + bias = meta.get("bias") or 0 + epc_cell = f"`{epc:#x}`" + (f" (firmware vaddr `{epc - bias:#x}`)" if bias else "") + lines.append(f"| epc | {epc_cell} |") + lines.append(f"| tval | `{first.get('tval', 0):#x}` |") + if access: + lines.append( + f"| first access | `{access.get('rw')}` at `{access.get('access_addr', 0):#x}` " + f"({access.get('size')} bytes, reason `{access.get('reason')}`) |" + ) + lines.append("") + def annotate(addr): if not isinstance(addr, int): return "" diff --git a/debug/risc-v-gsp-rm-vm/greeneye/cli.py b/debug/risc-v-gsp-rm-vm/greeneye/cli.py index 5bfcee2..94102a4 100644 --- a/debug/risc-v-gsp-rm-vm/greeneye/cli.py +++ b/debug/risc-v-gsp-rm-vm/greeneye/cli.py @@ -332,6 +332,15 @@ class GreenEyeRepl(cmd.Cmd): stops = self.session.run(duration=seconds) self._say(f"resumed for {seconds}s, serviced {stops} stops") + def do_call(self, arg): + """call <addr|symbol> [a0 [a1 ...]] -- remote function call; prints the result.""" + parts = arg.split() + if not parts: + return self._say("usage: call <addr|symbol> [args...]") + values = [int(part, 0) for part in parts[1:]] + retval = self.session.call(parts[0], values) + self._say(f" -> {retval:#x}") + def do_events(self, arg): """events [n] -- show the last n recorded events (default 20).""" from .record import load_events @@ -512,6 +521,12 @@ def cmd_probe(args) -> int: print(f"[probe] total exceptions: {len(faults)}") else: print("[probe] no exception recorded (execution stayed in mapped memory)") + recorder = session.recorder + if hasattr(recorder, "meta"): + recorder.meta["qemu_fault_count"] = len(faults) + recorder.meta["qemu_first_fault"] = faults[0] if faults else None + if qlog: + recorder.meta["qemu_log_blob"] = recorder.blob("qemu.log", qlog.encode("utf-8")) tail = qlog.strip().splitlines() if tail: print("[probe] qemu log tail:") @@ -523,6 +538,64 @@ def cmd_probe(args) -> int: return 0 +def cmd_inject(args) -> int: + _ensure_dumps_env() + from .inject import invoke_remote, invoke_via_firmware, load_service + from .target import free_port # noqa: F401 (kept for symmetry with launch) + + service_path = Path(args.service) + if not service_path.exists(): + raise GreenEyeError( + f"service not found: {service_path} (build it with `make sdk`)" + ) + kernel = args.kernel or str(_build_demo()) + session, _manager = _start_session( + kernel=str(kernel), port=args.port, record=args.record or True, name="inject", plugins=args.plugin + ) + try: + service = load_service(session, str(service_path), load_addr=args.load_addr, slot=args.slot) + print(f"[inject] {service.summary()}") + remote = invoke_remote(session, service, args.cmd, args.arg0, args.arg1) + print(f"[inject] direct call dispatch(cmd={args.cmd}, a0={args.arg0:#x}, a1={args.arg1:#x}) -> {remote:#x}") + if not args.no_firmware: + session.run(duration=args.duration) + fw = invoke_via_firmware(session, service, args.cmd, args.arg0, args.arg1) + print(f"[inject] firmware call mailbox={16 + service.slot} -> result {fw:#x}") + _print_session_dir(session) + finally: + session.close() + return 0 + + +def cmd_ghidra(args) -> int: + """Serve a live session to GhidraScripts over the JSON-lines bridge.""" + _ensure_dumps_env() + from .gdbridge import GhidraBridge + + kernel = args.kernel + if kernel: + kernel = str(Path(kernel)) + elif not args.target_port: + kernel = str(_build_demo()) + session, _manager = _start_session( + kernel=kernel, port=args.target_port, record=args.record or True, name="ghidra", plugins=args.plugin + ) + bridge = GhidraBridge(session, port=args.bridge_port) + port = bridge.start() + print(f"[green-eye] ghidra bridge on 127.0.0.1:{port}") + print(f"[green-eye] session: {getattr(session.recorder, 'dir', None)}") + print("[green-eye] in Ghidra: analyzeHeadless ... -postScript GreenEyeControl.java " + str(port)) + print("[green-eye] Ctrl-C to stop the bridge") + try: + bridge.serve_forever() + except KeyboardInterrupt: + pass + finally: + bridge.stop() + session.close() + return 0 + + def cmd_analyze(args) -> int: symbols = None if args.symbols: @@ -627,6 +700,21 @@ def build_parser() -> argparse.ArgumentParser: probe.add_argument("--record", nargs="?", const=True, default=None) probe.set_defaults(fn=cmd_probe) + inject = sub.add_parser("inject", help="load and invoke an injected RV64 service (Frida-like)") + inject.add_argument("--service", default=str(TOOL_ROOT / "sdk" / "build" / "injected_demo.elf")) + inject.add_argument("--kernel") + inject.add_argument("--port", type=int) + inject.add_argument("--load-addr", type=lambda v: int(v, 0)) + inject.add_argument("--slot", type=int, default=6, help="demo handler slot 0..7 (default 6)") + inject.add_argument("--cmd", type=lambda v: int(v, 0), default=1) + inject.add_argument("--arg0", type=lambda v: int(v, 0), default=0x41) + inject.add_argument("--arg1", type=lambda v: int(v, 0), default=0x42) + inject.add_argument("--duration", type=float, default=0.3, help="firmware settle time before the mailbox call") + inject.add_argument("--no-firmware", action="store_true", help="only make the direct debugger-side call") + inject.add_argument("--record", nargs="?", const=True, default=None) + inject.add_argument("--plugin", action="append", default=[]) + inject.set_defaults(fn=cmd_inject) + record = sub.add_parser("record", help="boot and record a memory-activity session") record.add_argument("--kernel") record.add_argument("--name", default="record") @@ -636,6 +724,14 @@ def build_parser() -> argparse.ArgumentParser: record.add_argument("--record", nargs="?", const=True, default=None) record.set_defaults(fn=cmd_record) + ghidra = sub.add_parser("ghidra", help="serve a live session to GhidraScripts (JSON-lines bridge)") + ghidra.add_argument("--kernel") + ghidra.add_argument("--target-port", type=int, help="attach to an existing gdbstub instead of launching") + ghidra.add_argument("--bridge-port", type=int, default=7000) + ghidra.add_argument("--record", nargs="?", const=True, default=None) + ghidra.add_argument("--plugin", action="append", default=[]) + ghidra.set_defaults(fn=cmd_ghidra) + analyze = sub.add_parser("analyze", help="render a recorded session report") analyze.add_argument("session") analyze.add_argument("--symbols") diff --git a/debug/risc-v-gsp-rm-vm/greeneye/gdbridge.py b/debug/risc-v-gsp-rm-vm/greeneye/gdbridge.py new file mode 100644 index 0000000..cc06e43 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/greeneye/gdbridge.py @@ -0,0 +1,220 @@ +"""Ghidra control bridge: let GhidraScripts drive a green-eye session. + +A tiny JSON-lines server over localhost. Each request is one line:: + + {"id": 1, "cmd": "run_until", "args": {"target": "0x800000ce", "timeout": 5}} + +and each response is one line:: + + {"id": 1, "ok": true, "result": {"addr": 2147483854}} + +This is the transport behind ``ghidra/GreenEyeControl.java``: a Ghidra user +can "execute until here", set/clear breakpoints, read memory and export hit +lists without leaving the decompiler. See ``docs/13-GHIDRA-CONTROL.md``. +""" +from __future__ import annotations + +import json +import socket +import threading + +from .errors import GreenEyeError + + +class GhidraBridge: + def __init__(self, session, host: str = "127.0.0.1", port: int = 0): + self.session = session + self.host = host + self.port = port + self._server: socket.socket | None = None + self._thread: threading.Thread | None = None + self._clients: list[threading.Thread] = [] + self._stop = threading.Event() + + # -- lifecycle --------------------------------------------------------- + + def start(self) -> int: + server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + server.bind((self.host, self.port)) + server.listen(4) + self.port = server.getsockname()[1] + self._server = server + self._thread = threading.Thread(target=self._accept_loop, name="greeneye-ghidra", daemon=True) + self._thread.start() + return self.port + + def stop(self) -> None: + self._stop.set() + if self._server is not None: + try: + self._server.close() + except OSError: + pass + self._server = None + + def serve_forever(self) -> None: + if self._thread is None: + self.start() + try: + while not self._stop.is_set() and (self._thread is not None and self._thread.is_alive()): + self._thread.join(timeout=0.5) + except KeyboardInterrupt: + pass + finally: + self.stop() + + def _accept_loop(self) -> None: + assert self._server is not None + while not self._stop.is_set(): + try: + client, _addr = self._server.accept() + except OSError: + return + handler = threading.Thread(target=self._client_loop, args=(client,), daemon=True) + handler.start() + self._clients.append(handler) + + def _client_loop(self, client: socket.socket) -> None: + with client: + stream = client.makefile("rwb") + for line in stream: + line = line.strip() + if not line: + continue + try: + request = json.loads(line) + except json.JSONDecodeError as exc: + response = {"id": None, "ok": False, "error": f"bad json: {exc}"} + else: + response = self._dispatch(request) + stream.write((json.dumps(response) + "\n").encode("utf-8")) + stream.flush() + if response.get("close"): + self.stop() + return + + # -- command dispatch -------------------------------------------------- + + def _dispatch(self, request: dict) -> dict: + request_id = request.get("id") + cmd = request.get("cmd", "") + args = request.get("args") or {} + try: + handler = getattr(self, f"cmd_{cmd}", None) + if handler is None: + raise GreenEyeError(f"unknown bridge command {cmd!r}") + result = handler(**args) + response = {"id": request_id, "ok": True, "result": result} + if cmd == "close": + response["close"] = True + return response + except Exception as exc: # never kill the bridge on a bad request + return {"id": request_id, "ok": False, "error": str(exc)} + + # -- commands ---------------------------------------------------------- + + def cmd_ping(self) -> str: + return "pong" + + def cmd_info(self) -> dict: + return { + "name": self.session.name, + "pc": self.session.pc, + "symbols": len(self.session.symbols), + "recording": getattr(self.session.recorder, "dir", None), + } + + def cmd_resolve(self, name: str) -> int: + return self.session.resolve(name) + + def cmd_bp_set(self, target, kind: str = "sw") -> int: + handle = self.session.hook(target, kind=kind, listener="ghidra") + return handle.addr + + def cmd_bp_clear(self, addr) -> bool: + addr = self.session.resolve(addr) if isinstance(addr, str) else int(addr) + for _key, bp in list(self.session._bps.items()): # noqa: SLF001 - bridge is a privileged client + if bp.addr == addr: + self.session.remove_breakpoint(bp) + return True + + def cmd_bp_list(self) -> list[dict]: + return [ + {"addr": bp.addr, "label": bp.label, "hits": bp.hits, "kind": bp.kind} + for bp in self.session._bps.values() # noqa: SLF001 + ] + + def cmd_watch_set(self, target, size: int = 8, kind: str = "w") -> int: + wp = self.session.watch(target, size=size, kind=kind, listener="ghidra") + return wp.addr + + def cmd_run_until(self, target, timeout: float = 5.0) -> dict: + hit = self.session.run_until(target, timeout=timeout) + return {"addr": hit.addr, "symbol": hit.symbol.name if hit.symbol else None, "kind": hit.kind} + + def cmd_run(self, duration: float = 1.0, max_stops: int | None = None) -> int: + return self.session.run(duration=duration, max_stops=max_stops) + + def cmd_step(self, count: int = 1) -> list[int]: + return [hit.addr for hit in self.session.step(count)] + + def cmd_read_mem(self, addr, size: int = 64) -> str: + real = self.session.resolve(addr) if isinstance(addr, str) else int(addr) + return self.session.memory.read(real, size).hex() + + def cmd_write_mem(self, addr, hex: str) -> bool: + real = self.session.resolve(addr) if isinstance(addr, str) else int(addr) + self.session.memory.write(real, bytes.fromhex(hex)) + return True + + def cmd_read_reg(self, name: str) -> int: + return self.session.client.read_reg(name) + + def cmd_write_reg(self, name: str, value) -> bool: + self.session.client.write_reg(name, int(value)) + return True + + def cmd_call(self, target, args: list | None = None, timeout: float = 5.0) -> int: + return self.session.call(target, tuple(args or ()), timeout=timeout) + + def cmd_snapshot(self, addr, size: int, label: str = "ghidra") -> str: + real = self.session.resolve(addr) if isinstance(addr, str) else int(addr) + self.session.memory.snapshot(real, size, name=label) + return label + + def cmd_note(self, text: str) -> bool: + self.session.note(text) + return True + + def cmd_hits(self) -> list[dict]: + from .record import load_events + + directory = getattr(self.session.recorder, "dir", None) + if not directory: + return [] + counts: dict[int, int] = {} + for event in load_events(directory): + if event.get("type") == "bp_hit": + addr = event.get("pc") + if isinstance(addr, int): + counts[addr] = counts.get(addr, 0) + 1 + return [ + { + "addr": addr, + "hits": hits, + "symbol": (self.session.symbols.resolve(addr).name if self.session.symbols.resolve(addr) else None), + } + for addr, hits in sorted(counts.items(), key=lambda item: -item[1]) + ] + + def cmd_export_hits(self, path: str) -> str: + from .ghidra import export_hits + + directory = getattr(self.session.recorder, "dir", None) + if not directory: + raise GreenEyeError("session is not recording; hits cannot be exported") + return export_hits(directory, path, symbols=self.session.symbols) + + def cmd_close(self) -> bool: + return True diff --git a/debug/risc-v-gsp-rm-vm/greeneye/inject.py b/debug/risc-v-gsp-rm-vm/greeneye/inject.py new file mode 100644 index 0000000..f9605fc --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/greeneye/inject.py @@ -0,0 +1,129 @@ +"""Frida-like service injection for green-eye. + +Loads a position-independent RV64 service blob into a live target, registers +its dispatch entry with the firmware (on the demo uproc: a handler-table slot +that commands 16..23 dispatch to), and invokes it either directly through +``Session.call`` (a debugger-side remote function call) or through the +firmware's own mailbox dispatcher. + +On a future GSP-RM simulator the same shape becomes "inject an RM engine and +register it with the simulated RM core"; see ``docs/12-INJECTION-SDK.md`` and +``GREEN-EYE-ROADMAP.md``. +""" +from __future__ import annotations + +from dataclasses import dataclass + +from .elf import load_elf +from .errors import GreenEyeError + +#: Symbol the injector calls in a service blob. +ENTRY_SYMBOL = "ge_demo_service_dispatch" + +#: Demo uproc control-block field offsets (see firmware/uproc_demo). +CTRL_HANDLERS = 0x150 +CTRL_MAILBOX = 0x20 +CTRL_ARG0 = 0x28 +CTRL_ARG1 = 0x30 +CTRL_ARG2 = 0x38 +CTRL_RESULT = 0x40 +INJECT_CMD_BASE = 16 + + +@dataclass +class InjectedService: + path: str + name: str + load_addr: int + entry_addr: int + dispatch_offset: int + slot: int + size: int + + def summary(self) -> str: + return ( + f"{self.name}: {self.size} bytes at {self.load_addr:#x}, " + f"entry {self.entry_addr:#x} (offset {self.dispatch_offset:#x}), slot {self.slot}" + ) + + +def load_service( + session, + service_elf: str, + *, + load_addr: int | None = None, + slot: int = 6, + entry_symbol: str = ENTRY_SYMBOL, +) -> InjectedService: + """Load a service blob and register it with the demo firmware.""" + image = load_elf(service_elf) + entry = None + for symbol in image.symbols: + if symbol.name == entry_symbol: + entry = symbol + break + if entry is None: + available = ", ".join(sorted(s.name for s in image.symbols if s.is_function)) or "<none>" + raise GreenEyeError(f"service {service_elf}: symbol {entry_symbol!r} not found (functions: {available})") + if not 0 <= slot <= 7: + raise GreenEyeError(f"slot must be 0..7 (got {slot})") + if load_addr is None: + base = getattr(session.target, "ram_base", 0x80000000) if session.target else 0x80000000 + size = getattr(session.target, "memory_mb", 256) * 1024 * 1024 if session.target else 256 * 1024 * 1024 + load_addr = base + 0x01000000 + if load_addr + image.max_vaddr + 0x1000 >= base + size: + load_addr = base + max(0x1000, size - image.max_vaddr - 0x2000) + if getattr(session.target, "memory_mb", 0): + ram_end = getattr(session.target, "ram_base", 0x80000000) + session.target.memory_mb * 1024 * 1024 + if load_addr + image.max_vaddr > ram_end: + raise GreenEyeError( + f"service does not fit: load {load_addr:#x} + {image.max_vaddr:#x} exceeds RAM end {ram_end:#x}" + ) + + total = 0 + for segment in image.load_segments: + data = image.segment_bytes(segment) + if data: + session.memory.write(load_addr + segment.p_vaddr, data) + total += len(data) + entry_addr = load_addr + entry.value + + ctrl = session.symbols.get("gsp_ctrl") + if ctrl is not None: + session.memory.write_u64(ctrl.addr + CTRL_HANDLERS + slot * 8, entry_addr) + + service = InjectedService( + path=service_elf, + name=service_elf.rsplit("/", 1)[-1], + load_addr=load_addr, + entry_addr=entry_addr, + dispatch_offset=entry.value, + slot=slot, + size=total, + ) + session.recorder.event( + "service_injected", + path=service.path, + name=service.name, + load_addr=load_addr, + entry_addr=entry_addr, + slot=slot, + size=total, + ) + return service + + +def invoke_remote(session, service: InjectedService, cmd: int, a0: int = 0, a1: int = 0, timeout: float = 3.0) -> int: + """Call the service directly from the debugger (Frida NativeFunction).""" + return session.call(service.entry_addr, (cmd, a0, a1), timeout=timeout) + + +def invoke_via_firmware(session, service: InjectedService, cmd: int, a0: int = 0, a1: int = 0, settle: float = 0.25) -> int: + """Invoke the service through the firmware's mailbox dispatcher.""" + ctrl = session.symbols.lookup("gsp_ctrl") + session.memory.write_u64(ctrl + CTRL_ARG0, cmd) + session.memory.write_u64(ctrl + CTRL_ARG1, a0) + session.memory.write_u64(ctrl + CTRL_ARG2, a1) + session.memory.write_u64(ctrl + CTRL_MAILBOX, INJECT_CMD_BASE + service.slot) + session.run(duration=settle) + return session.memory.read_u64(ctrl + CTRL_RESULT) diff --git a/debug/risc-v-gsp-rm-vm/greeneye/record.py b/debug/risc-v-gsp-rm-vm/greeneye/record.py index bf70e58..0b025b5 100644 --- a/debug/risc-v-gsp-rm-vm/greeneye/record.py +++ b/debug/risc-v-gsp-rm-vm/greeneye/record.py @@ -40,7 +40,7 @@ class Recorder: self.meta = dict(meta or {}) self.meta.update({"format": FORMAT, "name": name, "created": time.time(), "greeneye": _version()}) self._events_path = os.path.join(self.dir, "events.jsonl") - self._fh = open(self._events_path, "w", encoding="utf-8") + self._fh = open(self._events_path, "w", encoding="utf-8", buffering=1) self._t0 = time.monotonic() self._counts: dict[str, int] = {} self._snapshots: list[dict] = [] diff --git a/debug/risc-v-gsp-rm-vm/greeneye/session.py b/debug/risc-v-gsp-rm-vm/greeneye/session.py index 7ad6444..fa54f77 100644 --- a/debug/risc-v-gsp-rm-vm/greeneye/session.py +++ b/debug/risc-v-gsp-rm-vm/greeneye/session.py @@ -6,6 +6,7 @@ plugin-supplied backend later. """ from __future__ import annotations +import re import time from dataclasses import dataclass, field as _field from typing import Callable @@ -16,6 +17,15 @@ from .record import NullRecorder, Recorder from .rsp import RspClient, StopReply from .symbols import Symbol, SymbolDB +#: Registers saved and restored around ``Session.call`` (RISC-V ABI + pc). +RESTORE_REGS = ( + ["ra", "sp", "gp", "tp", "t0", "t1", "t2", "s0", "s1"] + + [f"a{i}" for i in range(8)] + + [f"s{i}" for i in range(2, 12)] + + [f"t{i}" for i in range(3, 7)] + + ["pc"] +) + @dataclass class Hit: @@ -195,11 +205,21 @@ class Session: # -- symbol helpers ---------------------------------------------------- def resolve(self, target: int | str) -> int: + """Resolve an address, a symbol name, or a ``symbol+0xNN`` expression.""" if isinstance(target, int): return target if isinstance(target, str): if target.lower().startswith("0x"): return int(target, 16) + expression = re.fullmatch( + r"([A-Za-z_][A-Za-z0-9_.]*)\s*([+-])\s*(0x[0-9a-fA-F]+|\d+)", target.strip() + ) + if expression: + base = self.symbols.get(expression.group(1)) + if base is None: + raise SymbolNotFound(f"symbol {expression.group(1)!r} not found in symbol DB") + delta = int(expression.group(3), 0) + return base.addr + delta if expression.group(2) == "+" else base.addr - delta sym = self.symbols.get(target) if sym is None: raise SymbolNotFound(f"symbol {target!r} not found in symbol DB") @@ -338,6 +358,99 @@ class Session: break return hits + # -- Frida-like remote call / run-until ------------------------------- + + def _scratch_addr(self, override: int | None = None) -> int: + if override: + return override + if self.target is not None and getattr(self.target, "memory_mb", 0): + base = getattr(self.target, "ram_base", 0x80000000) + return base + self.target.memory_mb * 1024 * 1024 - 0x1000 + if self.image is not None and getattr(self.image, "max_vaddr", 0): + return self.image.max_vaddr + 0x10000 + return 0x801FF000 + + def run_until(self, target: int | str, timeout: float = 5.0) -> Hit: + """Run until execution reaches ``target`` (Frida's run-to-cursor). + + Sets a temporary breakpoint, services intervening stops, and returns + the Hit for the target. Raises ``RspTimeout`` if not reached. + """ + addr = self.resolve(target) + if self.pc == addr: + return Hit(kind="stop", addr=addr, regs=self.read_regs(), symbol=self.symbols.resolve(addr), session=self) + temp = Breakpoint(addr=addr, kind="sw", temporary=True, on_enter=None) + self.add_breakpoint(temp) + deadline = time.monotonic() + timeout + try: + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + raise RspTimeout(f"run_until {addr:#x}: not reached in {timeout}s") + try: + stop = self.client.continue_(timeout=remaining) + except RspTimeout: + self.client.interrupt() + raise RspTimeout(f"run_until {addr:#x}: not reached in {timeout}s") + hit = self._dispatch_stop(stop) + if hit is not None and hit.addr == addr and hit.kind == "breakpoint": + return hit + if stop.is_exit: + raise GreenEyeError(f"run_until {addr:#x}: target exited") + finally: + self.remove_breakpoint(temp) + + def call(self, target: int | str, args: tuple | list = (), timeout: float = 5.0, scratch: int | None = None) -> int: + """Call a function in the target from the debugger (Frida NativeFunction). + + Writes an ``ebreak`` trampoline into a scratch page, sets ``ra`` to it, + loads ``a0..a7`` and a private stack, jumps to ``target``, services + intervening stops (hooks/watchpoints included), and returns ``a0``. + All caller-saved registers and ``pc`` are restored afterwards. + """ + addr = self.resolve(target) + scratch = self._scratch_addr(scratch) + stack = scratch + 0x800 + saved = self.read_regs() + temp = Breakpoint(addr=scratch, kind="sw", temporary=True, purpose="user") + self.add_breakpoint(temp) + try: + for name in ("a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7"): + self.client.write_reg(name, 0) + for index, value in enumerate(args[:8]): + self.client.write_reg(f"a{index}", int(value) & 0xFFFF_FFFF_FFFF_FFFF) + self.client.write_reg("ra", scratch) + self.client.write_reg("sp", stack) + self.client.write_reg("pc", addr) + self.recorder.event("call_enter", addr=addr, args=list(args), scratch=scratch) + deadline = time.monotonic() + timeout + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + raise RspTimeout(f"call {addr:#x}: no return in {timeout}s") + try: + stop = self.client.continue_(timeout=remaining) + except RspTimeout: + self.client.interrupt() + raise RspTimeout(f"call {addr:#x}: no return in {timeout}s") + hit = self._dispatch_stop(stop) + pc = self.client.pc + if pc == scratch or (hit is not None and hit.addr == scratch): + retval = self.client.read_reg("a0") + self.recorder.event("call_return", addr=addr, retval=retval) + return retval + if stop.is_exit: + raise GreenEyeError(f"call {addr:#x}: target exited") + finally: + if temp.attached: + self.remove_breakpoint(temp) + for name in RESTORE_REGS: + if name in saved: + try: + self.client.write_reg(name, saved[name]) + except Exception: + pass + def run( self, duration: float | None = None, diff --git a/debug/risc-v-gsp-rm-vm/greeneye/target.py b/debug/risc-v-gsp-rm-vm/greeneye/target.py index efcb1d1..273f216 100644 --- a/debug/risc-v-gsp-rm-vm/greeneye/target.py +++ b/debug/risc-v-gsp-rm-vm/greeneye/target.py @@ -59,10 +59,12 @@ class QemuTarget: qemu_log: str | None = None, workdir: str | None = None, keep_alive: bool = False, + ram_base: int = 0x80000000, ): self.qemu = qemu or find_qemu() self.machine = machine self.memory_mb = memory_mb + self.ram_base = ram_base self.host = "127.0.0.1" self.port = gdb_port or free_port() self.bios = bios diff --git a/debug/risc-v-gsp-rm-vm/sdk/Makefile b/debug/risc-v-gsp-rm-vm/sdk/Makefile new file mode 100644 index 0000000..8cccc66 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sdk/Makefile @@ -0,0 +1,32 @@ +# green-eye injected-service SDK: builds a position-independent RV64 blob. +# +# The service ELF is linked at address 0 and loaded by the injector at a +# runtime address; all code is PC-relative (medany + PIC, no globals), so no +# dynamic relocations are required. Verify with: llvm-readelf-18 -r build/*.elf + +CC := clang +LLD ?= $(shell command -v ld.lld 2>/dev/null || echo /usr/lib/llvm-18/bin/ld.lld) +BUILD := build +TARGET := $(BUILD)/injected_demo.elf + +CFLAGS := --target=riscv64-unknown-elf -march=rv64imac -mabi=lp64 \ + -mcmodel=medany -mno-relax -fPIC -ffreestanding -nostdlib \ + -fno-builtin -fno-stack-protector -fno-asynchronous-unwind-tables \ + -O2 -Wall -Wextra -I. + +.PHONY: all clean relocs + +all: $(TARGET) + +$(TARGET): injected_demo.c greeneye_service.h + @mkdir -p $(BUILD) + $(CC) $(CFLAGS) -c injected_demo.c -o $(BUILD)/injected_demo.o + $(LLD) -m elf64lriscv --no-relax -e ge_demo_service_dispatch \ + -o $@ $(BUILD)/injected_demo.o + @echo "built $@" + +relocs: $(TARGET) + @llvm-readelf-18 -r $(TARGET) 2>/dev/null || readelf -r $(TARGET) + +clean: + rm -rf $(BUILD) diff --git a/debug/risc-v-gsp-rm-vm/sdk/greeneye_service.h b/debug/risc-v-gsp-rm-vm/sdk/greeneye_service.h new file mode 100644 index 0000000..d5218cc --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sdk/greeneye_service.h @@ -0,0 +1,37 @@ +/* green-eye injected-service SDK header. + * + * A green-eye service is position-independent RV64 code loaded into the + * target's RAM by the debugger. Nothing in the firmware links against it, + * so it must not reference globals, absolute addresses or libc: every + * function it calls must be one it brings with it (or a function whose + * address the injector passes in). + * + * On the demo uproc, the injector stores the service entry in the control + * block's handler table and the firmware dispatches commands 16..23 to it. + * On a future GSP-RM simulator the same shape becomes an RM engine entry + * registered with the simulated RM core. + */ +#ifndef GREENEYE_SERVICE_H +#define GREENEYE_SERVICE_H + +typedef unsigned long ge_u64; +typedef long ge_i64; + +#define GE_SERVICE_MAGIC 0x47455356u /* "GESV" */ +#define GE_SERVICE_VERSION 1u + +/* Command range the demo firmware routes to injected handlers. */ +#define GE_INJECT_CMD_BASE 16 + +/* Dispatch prototype: cmd plus two arguments, one result. */ +typedef ge_u64 (*ge_service_fn)(ge_u64 cmd, ge_u64 a0, ge_u64 a1); + +/* Optional metadata block an injector can locate by symbol name. */ +typedef struct { + unsigned int magic; + unsigned int version; + const char *name; + ge_u64 (*dispatch)(ge_u64, ge_u64, ge_u64); +} ge_service_t; + +#endif /* GREENEYE_SERVICE_H */ diff --git a/debug/risc-v-gsp-rm-vm/sdk/injected_demo.c b/debug/risc-v-gsp-rm-vm/sdk/injected_demo.c new file mode 100644 index 0000000..8d836fe --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/sdk/injected_demo.c @@ -0,0 +1,41 @@ +/* Example green-eye injected service. + * + * Position-independent, no globals, no libc: safe to load at any address. + * Commands (matching the demo firmware's injected command range): + * 1: xor a0 with 0xbeef + * 2: a0 + a1 + * 3: rotate a0 left by (a1 & 63) + * 4: a0 * 3 + 7 + * 5: min(a0, a1) + * 6: max(a0, a1) + * default: ~cmd + */ +#include "greeneye_service.h" + +__attribute__((noinline)) ge_u64 ge_demo_service_dispatch(ge_u64 cmd, ge_u64 a0, ge_u64 a1) +{ + switch (cmd) { + case 1: + return a0 ^ 0xbeefu; + case 2: + return a0 + a1; + case 3: { + unsigned int shift = (unsigned int)(a1 & 63u); + return shift ? ((a0 << shift) | (a0 >> (64u - shift))) : a0; + } + case 4: + return a0 * 3u + 7u; + case 5: + return a0 < a1 ? a0 : a1; + case 6: + return a0 > a1 ? a0 : a1; + default: + return ~cmd; + } +} + +/* A second entry point, to demonstrate multi-function services. */ +__attribute__((noinline)) ge_u64 ge_demo_service_answer(void) +{ + return 42; +} diff --git a/debug/risc-v-gsp-rm-vm/tests/test_bridge.py b/debug/risc-v-gsp-rm-vm/tests/test_bridge.py new file mode 100644 index 0000000..472bc49 --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_bridge.py @@ -0,0 +1,81 @@ +"""Integration tests for the Ghidra JSON-lines control bridge.""" +from __future__ import annotations + +import json +import os +import shutil +import socket +import unittest + +import greeneye as ge +from greeneye.gdbridge import GhidraBridge + +HERE = os.path.dirname(os.path.abspath(__file__)) +DEMO_ELF = os.path.join(HERE, "..", "firmware", "uproc_demo", "build", "uproc_demo.elf") + + +def qemu_available() -> bool: + return bool(os.environ.get("GREENEYE_QEMU") or shutil.which("qemu-system-riscv64")) + + +@unittest.skipUnless(qemu_available(), "qemu-system-riscv64 not installed") +@unittest.skipUnless(os.path.exists(DEMO_ELF), "demo firmware not built") +class BridgeTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.session = ge.launch(kernel=DEMO_ELF, record=True, name="unittest-bridge") + cls.bridge = GhidraBridge(cls.session, port=0) + cls.port = cls.bridge.start() + + @classmethod + def tearDownClass(cls): + cls.bridge.stop() + cls.session.close() + + def call(self, cmd, args=None): + with socket.create_connection(("127.0.0.1", self.port), timeout=10) as sock: + stream = sock.makefile("rwb") + stream.write((json.dumps({"id": 1, "cmd": cmd, "args": args or {}}) + "\n").encode()) + stream.flush() + response = json.loads(stream.readline()) + self.assertTrue(response["ok"], response) + return response["result"] + + def test_info_and_resolve(self): + info = self.call("info") + self.assertEqual(info["name"], "unittest-bridge") + addr = self.call("resolve", {"name": "rpc_sum"}) + self.assertEqual(addr, self.session.symbols.lookup("rpc_sum")) + + def test_run_until_and_hits(self): + result = self.call("run_until", {"target": "rpc_sum", "timeout": 5.0}) + self.assertEqual(result["addr"], self.session.symbols.lookup("rpc_sum")) + hits = self.call("hits") + self.assertTrue(any(entry["addr"] == result["addr"] for entry in hits), hits) + + def test_memory_and_registers(self): + ctrl = self.session.symbols.lookup("gsp_ctrl") + hexdata = self.call("read_mem", {"addr": ctrl, "size": 8}) + self.assertEqual(len(bytes.fromhex(hexdata)), 8) + self.call("write_reg", {"name": "t3", "value": 0x1234}) + self.assertEqual(self.call("read_reg", {"name": "t3"}), 0x1234) + + def test_breakpoints_and_call(self): + addr = self.call("bp_set", {"target": "rpc_fill"}) + self.assertEqual(addr, self.session.symbols.lookup("rpc_fill")) + self.call("bp_clear", {"addr": addr}) + result = self.call("call", {"target": "rpc_ping", "args": [0x1111]}) + self.assertEqual(result, 0x1111 ^ 0xBEEF) + + def test_unknown_command(self): + with socket.create_connection(("127.0.0.1", self.port), timeout=10) as sock: + stream = sock.makefile("rwb") + stream.write((json.dumps({"id": 2, "cmd": "nope", "args": {}}) + "\n").encode()) + stream.flush() + response = json.loads(stream.readline()) + self.assertFalse(response["ok"]) + self.assertIn("unknown bridge command", response["error"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/debug/risc-v-gsp-rm-vm/tests/test_core.py b/debug/risc-v-gsp-rm-vm/tests/test_core.py index ab27451..907a2eb 100644 --- a/debug/risc-v-gsp-rm-vm/tests/test_core.py +++ b/debug/risc-v-gsp-rm-vm/tests/test_core.py @@ -27,8 +27,8 @@ class ElfTests(unittest.TestCase): self.assertIn("rpc_ping", names) self.assertIn("gsp_ctrl", names) ctrl = next(s for s in image.symbols if s.name == "gsp_ctrl") - self.assertEqual(ctrl.value, 0x80000840) - self.assertEqual(ctrl.size, 0x188) + self.assertGreaterEqual(ctrl.value, 0x80000000) + self.assertEqual(ctrl.size, 0x190) plan = image.flat_load_plan() self.assertTrue(plan and plan[0][0] == 0x80000000) diff --git a/debug/risc-v-gsp-rm-vm/tests/test_injection.py b/debug/risc-v-gsp-rm-vm/tests/test_injection.py new file mode 100644 index 0000000..5dca9cb --- /dev/null +++ b/debug/risc-v-gsp-rm-vm/tests/test_injection.py @@ -0,0 +1,121 @@ +"""Integration tests for Frida-like remote calls, run-until and service injection. + +Skipped cleanly when QEMU, the demo firmware, clang/lld or the SDK are absent. +""" +from __future__ import annotations + +import io +import contextlib +import os +import shutil +import subprocess +import unittest + +import greeneye as ge +from greeneye.errors import RspTimeout + +HERE = os.path.dirname(os.path.abspath(__file__)) +TOOL_ROOT = os.path.join(HERE, "..") +DEMO_ELF = os.path.join(TOOL_ROOT, "firmware", "uproc_demo", "build", "uproc_demo.elf") +SERVICE_ELF = os.path.join(TOOL_ROOT, "sdk", "build", "injected_demo.elf") + + +def qemu_available() -> bool: + return bool(os.environ.get("GREENEYE_QEMU") or shutil.which("qemu-system-riscv64")) + + +def toolchain_available() -> bool: + return bool(shutil.which("clang")) and ( + shutil.which("ld.lld") or os.path.exists("/usr/lib/llvm-18/bin/ld.lld") + ) + + +@unittest.skipUnless(qemu_available(), "qemu-system-riscv64 not installed") +@unittest.skipUnless(os.path.exists(DEMO_ELF), "demo firmware not built (make demo)") +class CallTests(unittest.TestCase): + def test_remote_call(self): + session = ge.launch(kernel=DEMO_ELF, name="unittest-call") + try: + result = session.call("rpc_ping", (0x1111,)) + self.assertEqual(result, 0x1111 ^ 0xBEEF) + # registers must be restored around the call + self.assertEqual(session.pc, 0x1000) + finally: + session.close() + + def test_run_until(self): + session = ge.launch(kernel=DEMO_ELF, name="unittest-rununtil") + try: + hit = session.run_until("rpc_sum", timeout=5.0) + self.assertEqual(hit.addr, session.symbols.lookup("rpc_sum")) + self.assertEqual(hit.kind, "breakpoint") + finally: + session.close() + + def test_run_until_timeout(self): + session = ge.launch(kernel=DEMO_ELF, name="unittest-rununtil-timeout") + try: + with self.assertRaises(RspTimeout): + session.run_until("rpc_fault", timeout=0.2) # self-test never calls it + finally: + session.close() + + +@unittest.skipUnless(qemu_available(), "qemu-system-riscv64 not installed") +@unittest.skipUnless(os.path.exists(DEMO_ELF), "demo firmware not built (make demo)") +class ReplTests(unittest.TestCase): + def test_repl_commands(self): + from greeneye.cli import GreenEyeRepl + from greeneye.plugins import PluginManager + + session = ge.launch(kernel=DEMO_ELF, record=True, name="unittest-repl") + output = io.StringIO() + try: + ctrl = session.symbols.lookup("gsp_ctrl") + repl = GreenEyeRepl(session, PluginManager(session)) + with contextlib.redirect_stdout(output): + repl.onecmd("bp rpc_sum") + repl.onecmd("watch gsp_ctrl+0x10 8 w") + repl.onecmd(f"x {ctrl:#x} 16") + repl.onecmd("sym rpc_ping") + repl.onecmd("c 0.6") + repl.onecmd("events 5") + text = output.getvalue() + self.assertIn("rpc_sum", text) + self.assertIn(f"{ctrl:08x}", text) + self.assertIn(f"watch w@{ctrl + 0x10:#x}", text) + finally: + session.close() + + +@unittest.skipUnless(qemu_available(), "qemu-system-riscv64 not installed") +@unittest.skipUnless(os.path.exists(DEMO_ELF), "demo firmware not built") +@unittest.skipUnless(toolchain_available(), "clang/lld not available") +class InjectionTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + result = subprocess.run( + ["make", "-C", os.path.join(TOOL_ROOT, "sdk")], + capture_output=True, + text=True, + ) + if result.returncode != 0 or not os.path.exists(SERVICE_ELF): + raise unittest.SkipTest(f"SDK build failed: {result.stderr[-200:]}") + + def test_direct_and_firmware_invocation(self): + from greeneye.inject import invoke_remote, invoke_via_firmware, load_service + + session = ge.launch(kernel=DEMO_ELF, record=True, name="unittest-inject") + try: + service = load_service(session, SERVICE_ELF, slot=6) + self.assertEqual(invoke_remote(session, service, 1, 0x41), 0x41 ^ 0xBEEF) + self.assertEqual(invoke_remote(session, service, 2, 0x10, 0x20), 0x30) + session.run(duration=0.3) + self.assertEqual(invoke_via_firmware(session, service, 1, 0x41), 0x41 ^ 0xBEEF) + self.assertEqual(invoke_via_firmware(session, service, 2, 0x10, 0x20), 0x30) + finally: + session.close() + + +if __name__ == "__main__": + unittest.main() |
