path: root/debug/risc-v-gsp-rm-vm/docs/15-AGENTIC-GSP-IDE.md

# 15 — risc-v-gsp-rm-debugger as an agentic integrated development, analysis and debugging environment for the GPU System Processor

> Status: implemented, documented and tested (audit: docs/27-TOOL-AUDIT.md; value demos: docs/25-VALUE-VALIDATION.md).

## 1. Vision and what "agentic" means here

risc-v-gsp-rm-debugger already has the two properties that make an agent loop possible:
a **deterministic, scriptable harness** (`ge.launch`/`ge.attach`, `Session`,
`record`, `compare`) and a **durable evidence format** (the session directory).
"Agentic" here means:

* AI agents drive the deterministic harness — they choose instrumentation,
  run stimuli, compare sessions and write findings;
* sessions are the shared state between agents, humans and Ghidra, so every
  claim is inspectable after the fact;
* **humans keep product-change authority**: an agent may propose an
  `open-iov-vfio` change with evidence, but only a human merges it, and the
  physical-confirmation rules of docs/06 and docs/07 still apply.

Non-goals: autonomous hardware cycles, unattended reboots, agents that "fix"
the emulation directly, or replacing the engineer reading the decompiler.
This document describes the environment that exists today plus the roadmap
hooks that would make a full agent platform real.

## 2. Architecture

```
      +-------------------- agents (triage / hypothesis / RE / portability) ----+
      |                                                                          |
      |   CLI: ./risc-v-gsp-rm-debugger.sh info|demo|run|attach|launch|probe|record|analyze|   |
      |        compare|symbols|plugins|inject|ghidra                             |
      |   Python: ge.launch/attach, Session.*, memory.*, symbols.*, structs,     |
      |           probes, plugins, ghidra.export_hits, inject.*                  |
      |   REPL: bp/hook/watch/call/x/snap/events/note                            |
      |   Ghidra bridge: JSON lines over localhost (docs/13)                     |
      |   MCP: planned plugin only (roadmap R3) — does not exist in v0.1          |
      +----------------------------------+---------------------------------------+
                                         v
              session directory (events.jsonl + session.json + snapshots + summary.md)
```

### 2.1 The session directory is the shared state

| Artifact | Content | Consumers |
|---|---|---|
| `events.jsonl` | append-only event stream (`bp_hit`, `watch_hit`, `mem_write`, `call_enter/return`, `note`, `service_injected`, …) | `analyze`, `compare`, `export_hits`, plugins |
| `session.json` | format, name, counts, snapshot inventory, meta | report tooling, provenance |
| `snapshots/*.bin.z` | zlib-compressed region captures with sha256 | offline diff, field decode |
| `blobs/*` | plugin/probe dumps (e.g. qemu.log) | forensics |
| `summary.md` | rendered report | humans, PR attachments |

`./risc-v-gsp-rm-debugger.sh compare dumps/<a> dumps/<b> --symbols` is the primary agent
primitive: it turns two sessions into hit/watch/write deltas. Format
`greeneye-session/1` evolves additively (roadmap V2).

### 2.2 Surfaces an agent may use

* `./risc-v-gsp-rm-debugger.sh …` — the stable CLI (all real subcommands listed above).
* `./risc-v-gsp-rm-debugger.sh run script.py` — the preloaded Python namespace (`ge`,
  `session`/`s`, `memory`, `symbols`, `recorder`) makes a harness file the
  unit of automation.
* `./risc-v-gsp-rm-debugger.sh ghidra` + `GreenEyeControl.java` — JSON-lines bridge for the
  decompiler loop (`run_until`, `bp_set`, `read_mem`, `call`, `hits`,
  `export_hits`; protocol in docs/13 §3).
* Plugins — the extension point (docs/04). A plugin's `register(ge)` can add
  instrumentation, CLI commands (`ge.add_command`) and recorded events; plugin
  output lands in the same `events.jsonl`.
* **MCP is planned, not present.** Roadmap R3/H7 reserves an MCP plugin that
  would expose sessions as local stdio tools. Until that rung is scheduled, do
  not add MCP dependencies to the core package (docs/04 §3d).

### 2.3 Why the evidence model matters for agents

A human can hold context across steps; an agent cannot. The session directory
makes the context external: an agent can be restarted, handed a directory, and
re-derive the same conclusion without replaying its own reasoning. This is the
difference between an agent that reports opinions and one that produces
reviewable evidence.

## 3. Agent roles and workflows

| Role | Input | Tools | Output | Stop condition |
|---|---|---|---|---|
| Triage | question + firmware version(s) | `symbols export`, `probe`, `analyze` | delta candidates, prioritised symbol list | every changed symbol classified |
| Hypothesis | one behaviour question | `run` scripts, hooks, watchpoints, `compare` | session pair + verdict | delta attributed to one variable |
| RE | decompiled function | `./risc-v-gsp-rm-debugger.sh ghidra`, `run_until`, breakpoints, `export_hits` | hits TSV + annotation, reachability answer | function hit or timeout recorded |
| Portability | version/device pair | docs/07 playbook, baseline + candidate sessions, M1–M6 | portability report | metric table complete |
| Injection | service idea | `sdk/`, `inject`, `load_service`, `invoke_remote`, `invoke_via_firmware` | injected-service session pair | direct and firmware paths agree |
| Verification | another agent's finding | golden sessions, `compare`, `analyze` | pass/fail + counterexample | finding reproduced or refuted |

Role separation is deliberate: the agent that made a claim should not be the
only one that checks it. Verification requires re-running the original
stimulus, not re-reading the original report.

## 4. Runbooks

Each runbook is written so a human or an agent can execute it start to finish,
and each ends with the evidence pack from docs/14 §6.

### 4.1 Answer an emulation reply question

Question template: "For control `<X>`, does the firmware set field `<Y>` of
the reply/object, and in what order relative to `<Z>`?"

1. Frame it as A/B: the emulation's reply (reference capture, docs/07 P4) and
   the firmware's reply in risc-v-gsp-rm-debugger. One session alone cannot answer it.
2. Resolve the handler and object from the symbol DB; never hardcode:

   ```bash
   ./risc-v-gsp-rm-debugger.sh symbols resolve --db symbols-<ver>.json <handler-or-addr>
   ```

3. Capture the firmware side with a hook + before/after snapshots:

   ```python
   # reply_q.py -- ./risc-v-gsp-rm-debugger.sh run reply_q.py --kernel <rehosted.elf>
   import greeneye as ge
   from greeneye import Memory

   s = ge.launch(kernel="<rehosted.elf>", record=True, name="reply-q")
   try:
       ctrl = s.symbols.lookup("<control-symbol>")     # demo: gsp_ctrl
       s.hook("<handler-symbol>",
              on_enter=lambda h: s.note("enter", a0=h.arg(0)),
              on_leave=lambda h: s.note("leave", retval=h.retval))
       before = s.snapshot(ctrl, 0x100, "before")
       s.memory.write_u64(ctrl + 0x20, 1)              # stimulus
       s.run(duration=0.5)
       after = s.snapshot(ctrl, 0x100, "after")
       print(Memory.render_diff(Memory.diff(before, after, base=ctrl), s.symbols))
   finally:
       s.close()
   ```

4. Decode the captured bytes with the structure schema so the answer is
   field-by-field, not byte noise:

   ```python
   import json
   from greeneye.structs import schema_from_json, decode, render
   schema = schema_from_json(json.load(open("firmware/uproc_demo/gsp_ctrl.schema.json")))
   values = decode(schema, s.memory.read(ctrl, schema.size), base=ctrl)
   print(render(schema, values))
   ```

5. Compare against the reference capture with
   `greeneye.analyze.compare_sessions(...)` / `./risc-v-gsp-rm-debugger.sh compare …` for
   ordering (M3) and state the field diff in words (M2). If the reply is in a
   different object, repeat step 3 on that object's address.
6. Report: question, session pair, `compare` output, M2/M3 rows, fidelity
   statement, explicit "cannot answer because L3 absent" where applicable.

### 4.2 Find why a handler disappeared

1. Build/refresh the static inventory for both firmware versions:

   ```bash
   $GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \
       -scriptPath debug/risc-v-gsp-rm-vm/ghidra \
       -postScript ExportForGreenEye.java /tmp/symbols-<ver>.tsv
   ./risc-v-gsp-rm-debugger.sh symbols export --index /tmp/symbols-<ver>.tsv --bias 0 -o symbols-<ver>.json
   diff <(cut -f2 /tmp/symbols-old.tsv | sort) <(cut -f2 /tmp/symbols-new.tsv | sort)
   ```

2. Probe the new image for early structure changes:

   ```bash
   ./risc-v-gsp-rm-debugger.sh probe <new.fwimage> --symbols symbols-<ver>.json --record
   ```

3. Record the same stimulus against each version. Hook the old handler name;
   a missing symbol is itself evidence — catch `SymbolNotFound` and log it
   rather than aborting the run. Also hook the dispatcher and the old
   handler's neighbours from the inventory diff.
4. Compare the sessions:

   ```bash
   ./risc-v-gsp-rm-debugger.sh compare dumps/<old-run> dumps/<new-run> --symbols
   ./risc-v-gsp-rm-debugger.sh analyze dumps/<new-run> --symbols symbols-<ver>.json --out new.md
   ```

5. Classify the disappearance with the table below, then confirm in Ghidra:

   | Observation | Likely cause | Next step |
   |---|---|---|
   | name absent from new symbol DB | removed or renamed before build | match by size/callgraph in Ghidra |
   | old function exists, dispatcher hits moved to another symbol | merged or re-routed | `run_until` the new dispatcher entry, `bp_set` there |
   | old function exists, zero hits under the standard stimulus | coverage gap or decoy | widen stimulus, check xrefs from the dispatch table |
   | dispatch table entry changed but code identical | behaviour moved (state/input) | inspect the table with bridge `read_mem`/snapshot |

6. With the bridge running (`./risc-v-gsp-rm-debugger.sh ghidra --kernel <elf>`), execute
   until the dispatch site from the decompiler and set breakpoints:

   ```bash
   $GHIDRA/support/analyzeHeadless <proj> <name> -process <program> -noanalysis \
       -scriptPath debug/risc-v-gsp-rm-vm/ghidra \
       -postScript GreenEyeControl.java 7000 0x<dispatch-address>
   ```

7. Export hits and annotate the old function's fate:

   ```bash
   ./risc-v-gsp-rm-debugger.sh analyze dumps/<new-run> --symbols symbols-<ver>.json --out new.md
   python3 - <<'PY'
   from greeneye.ghidra import export_hits
   from greeneye.symbols import SymbolDB
   db = SymbolDB(); db.load_json("symbols-<ver>.json")
   print(export_hits("dumps/<new-run>", "greeneye_hits.tsv", symbols=db))
   PY
   ```

8. Report: static delta, dynamic delta, classification, and whether the
   emulation must change (with the docs/14 evidence pack).

### 4.3 Prototype a service

1. Build the SDK reference service and prove position independence:

   ```bash
   make -C sdk
   make -C sdk relocs          # must print "no relocations in this file"
   ```

2. Inject and invoke through both paths (direct debugger call, then firmware
   mailbox dispatch):

   ```bash
   ./risc-v-gsp-rm-debugger.sh inject --service sdk/build/injected_demo.elf \
       --cmd 1 --arg0 0x41 --arg1 0x42
   ```

3. Iterate programmatically when the CLI shape is not enough:

   ```python
   from greeneye.inject import load_service, invoke_remote, invoke_via_firmware
   svc = load_service(s, "sdk/build/injected_demo.elf", slot=6)
   print(hex(invoke_remote(s, svc, 2, 0x10, 0x20)))
   s.run(duration=0.3)
   print(hex(invoke_via_firmware(s, svc, 2, 0x10, 0x20)))
   ```

4. Instrument the service like firmware: hooks on the entry, watchpoints on
   its data, snapshots before/after each command (docs/11 §7).
5. Compare two versions of the service to see the behavioural delta:

   ```bash
   ./risc-v-gsp-rm-debugger.sh compare dumps/<svc-v1> dumps/<svc-v2> --symbols
   ```

6. Stop condition: direct and firmware-dispatched results agree, the session
   is recorded, and the prototype is documented as development-only. Product
   code goes to `open-iov-vfio` only through the accuracy loop (docs/11 §8,
   docs/12 §9); GSP-RM registration is roadmap H1.3.

### 4.4 Prepare a portability report

1. Follow the P1 loop in docs/07-PORTABILITY-PLAYBOOKS.md with real commands:

   ```bash
   ./risc-v-gsp-rm-debugger.sh probe <old.fwimage> --symbols symbols-old.json --record
   ./risc-v-gsp-rm-debugger.sh probe <new.fwimage> --symbols symbols-new.json --record
   ./risc-v-gsp-rm-debugger.sh record --kernel <rehosted-old.elf> --duration 2.0 \
       --watch gsp_ctrl+0x10:8 --snapshot gsp_ctrl:392
   ./risc-v-gsp-rm-debugger.sh record --kernel <rehosted-new.elf> --duration 2.0 \
       --watch gsp_ctrl+0x10:8 --snapshot gsp_ctrl:392
   ./risc-v-gsp-rm-debugger.sh compare dumps/<old-run> dumps/<new-run> --symbols
   ```

2. Keep the stimulus identical across versions: use one script/probe spec and
   record it in the pack (docs/14 §4.1).
3. Fill the M1–M6 table from docs/06-ACCURACY.md. M7 (timing) is never
   claimed; L2/L3 statements are mandatory.
4. Produce the report with the docs/14 §6 template and attach session hashes,
   symbol DB hashes and the Ghidra export provenance.
5. Verification agent step: replay the candidate stimulus against the checked-in
   golden session (roadmap V1) and require `compare` to match expectations.

## 5. Guardrails

Hard rules (no agent may override them):

* **Never claim cycles, time or reboots from the simulator.** L2 is not
  modelled; L3 is absent until H2 (docs/06).
* **No guest modifications** and no instrumentation of a guest driver; the
  tool is never shipped and never present in a guest (docs/00, V4).
* **Sessions are mandatory evidence.** A finding without a session directory
  and a `compare` report is not a finding (docs/11 §8).
* **No unattended hardware.** The agent runs against the simulator by default;
  physical GPU cycles are supervised and human-driven (H7).
* **Human approval for product changes.** Agents produce proposals and
  evidence packs; only humans merge `open-iov-vfio` changes.
* **Budgets everywhere.** Every run uses `duration=`/`max_stops=`/`timeout=`
  and a wall-clock cap; a wedged target leaves the session stopped and is
  restarted clean rather than poked.

Capability manifest for actions (proposed file for the harness; see §7):

| Class | Examples | Who runs it | Preconditions |
|---|---|---|---|
| Read | `info`, `plugins`, `symbols resolve`, `analyze`, `compare` | agent, unattended | path/DB provenance recorded |
| Observe | `probe`, `record`, `demo`, hooks/watch/step | agent, unattended on simulator | timeout, session recording on |
| Write (target) | `memory.write*`, `call`, `run_until`, `inject`, mailbox stimulus | agent with manifest | declared address/range, fresh session, before/after snapshots, mutation values noted |
| Product | `open-iov-vfio` patch | human only | M1–M6 + physical confirmation rules (docs/07 P4.5) |
| Hardware | physical cycles/reboots | human only | supervised, never unattended |

A write action's manifest entry names the session, the target range, the
purpose and the budget; the session records the action, so the manifest and
the evidence can be cross-checked.

## 6. Evaluation

An agent workflow is judged on the same axis as the human one: how long until
a reviewable answer, and does the answer survive verification.

| Metric | Definition | Instrument |
|---|---|---|
| TTFA | question → first evidence-backed answer | wall clock; session count |
| TTEP | question → complete evidence pack (docs/14 §6) | pack checklist |
| Evidence yield | sessions/`compare` reports per accepted finding | `dumps/` inventory |
| Verification pass rate | findings reproduced by the verification role | replay runs |
| Coverage | inventory observed vs emulated (M1) | docs/06 table |
| Accuracy trend | M1–M6 per release | docs/06 tables |
| Regression lead time | firmware drop → delta report | probe/compare timestamps |
| Human minutes | manual minutes per accepted finding | benchmark log |

Benchmark protocol: take five questions from recent history, run each twice —
once with the agent (all actions recorded) and once manually — on the same
machine and firmware set, then compare TTFA/TTEP/evidence yield. Re-run the
manual side offline against the agent's sessions to separate speed from
correctness. Publish the ratios with the raw session directories, not just
the numbers (the project's "never a single run" rule applies to tooling too).

Golden data is the regression bed: the demo session and re-hosted workloads
are checked in CI (roadmap V1), and every new stimulus script should grow a
golden expectation once it is stable.

## 7. Roadmap hooks (H7) and what would make this real

| H7 item | What exists today | What is missing |
|---|---|---|
| MCP plugin (R3) | JSON-lines Ghidra bridge; sessions as data | MCP server plugin (`plugins/mcp_server.py` when built) |
| deterministic harnesses | `run` scripts, probes, plugins, `record`/`compare` | named harness entry points, budgets, manifests |
| sessions as evidence | `greeneye-session/1`, `summary.md`, `compare` | session index, signed/hashed pack tooling |
| autonomous loops | hooks/watchpoints/compare primitives | hypothesis/verification controllers, stop conditions |
| safety | local-only bridge, no network, no hardware | enforced capability manifest, process isolation |

What would make this real:

1. **MCP tool schema** (sketch; planned only, no code in v0.1):

   ```json
   {
     "transport": "local stdio; no network exposure by default",
     "tools": [
       {"name": "list_sessions", "args": {}},
       {"name": "open_session", "args": {"dir": "string"}},
       {"name": "symbols_resolve", "args": {"session": "string", "name": "string"}},
       {"name": "memory_read", "args": {"session": "string", "addr": "string", "size": "int"}},
       {"name": "memory_hexdump", "args": {"session": "string", "addr": "string", "size": "int"}},
       {"name": "set_breakpoint", "args": {"session": "string", "target": "string", "kind": "sw|hw"}},
       {"name": "watch", "args": {"session": "string", "target": "string", "size": "int", "kind": "w|r|a"}},
       {"name": "run", "args": {"session": "string", "duration": "float", "max_stops": "int"}},
       {"name": "run_until", "args": {"session": "string", "target": "string", "timeout": "float"}},
       {"name": "snapshot", "args": {"session": "string", "addr": "string", "size": "int", "label": "string"}},
       {"name": "compare_sessions", "args": {"a": "string", "b": "string", "symbols": "bool"}},
       {"name": "export_hits", "args": {"session": "string", "out": "string"}}
     ]
   }
   ```

   Baseline names follow docs/08 R3; `watch` and `run_until` follow the
   bridge command set and H7.

2. **Harness scripts** with fixed names and budgets: `stimulus.py` (one
   workload), `probe_matrix.py` (version × device), `golden_check.py`
   (compare against expected profiles). Today these are written ad hoc.
3. **Golden data**: checked-in expectation files for the demo and at least one
   re-hosted workload, wired into `make test` (roadmap V1).
4. **Capability manifest** file the harness reads before any write action
   (§5 table), with the session recording as the audit trail.
5. **Session index** for agents: a small JSONL listing session dirs, questions
   and outcomes, so triage starts from history instead of from zero.
6. **Physical-escalation gate**: a write-up checklist that must be complete
   before an agent-proposed change reaches a supervised hardware cycle.

Until R3 lands, the agent loop is: CLI/scripts + Ghidra bridge + sessions —
nothing more, and the documentation should keep saying so.