diff options
| author | Open-IOV <dev@open-iov.org> | 2026-09-23 05:44:08 -0400 |
|---|---|---|
| committer | Open-IOV <dev@open-iov.org> | 2026-09-23 05:46:43 -0400 |
| commit | 68cdd7728a7cc6715bbbff7decfe2b11c3bdf558 (patch) | |
| tree | 3dc8ce2e6af313a97ec6852f5c663a9a93665d2e /docs/08-ecosystem | |
| download | open-iov-68cdd7728a7cc6715bbbff7decfe2b11c3bdf558.tar.gz open-iov-68cdd7728a7cc6715bbbff7decfe2b11c3bdf558.zip | |
Open-IOV: initial public release
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
Diffstat (limited to 'docs/08-ecosystem')
| -rw-r--r-- | docs/08-ecosystem/haaeep-and-collaboration.md | 154 |
1 files changed, 154 insertions, 0 deletions
diff --git a/docs/08-ecosystem/haaeep-and-collaboration.md b/docs/08-ecosystem/haaeep-and-collaboration.md new file mode 100644 index 0000000..c4fb352 --- /dev/null +++ b/docs/08-ecosystem/haaeep-and-collaboration.md @@ -0,0 +1,154 @@ +# HAAEEP and Collaboration + +> The High Assurance Agent Edge Environment Platform: a concrete, buildable +> use-case architecture that Open-IOV exists to enable — and how we want to +> build it with you. + +## 1. Motivation + +An AI agent harness (Claude Code, OpenCode, or any future equivalent) is an +untrusted, capability-constrained workload that runs generated code, holds +credentials, and talks to a model. In a *cloud* deployment, the agent's requests +leave the device. In a *secure edge* deployment — a workstation, a kiosk, a +vehicle, a medical or industrial system — they must not. + +We want an edge platform where: + +* inference runs **locally**, in its own VM, with the GPU; +* agents run in **their own VMs**, without the GPU and without the weights; +* other functions (desktop, networking, storage) run in **other VMs**, each + isolated; +* the whole stack is **attestable**, so remote parties can verify it; +* and nothing in this design requires the model server, the agent, or the GPU + driver to be modified. + +That platform is HAAEEP. Open-IOV is its GPU component. + +## 2. Architecture + +``` +┌──────────────────────────────────────────────────────────────────────────┐ +│ EDGE DEVICE — one GPU, local inference only │ +│ │ +│ ┌─────────────────┐ ┌──────────────────┐ ┌───────────────────┐ │ +│ │ Agent VM │ │ Inference VM │ │ Infrastructure │ │ +│ │ capability- │────▶│ vLLM / SGLang │ │ desktop, net, │ │ +│ │ constrained │ API │ owns GPU via │ │ storage VMs │ │ +│ │ no GPU │ │ Open-IOV mdev or │ │ optional GPU accel│ │ +│ │ │ │ xen-pci + relaying│ │ via Open-IOV │ │ +│ └─────────────────┘ └────────┬─────────┘ └───────────────────┘ │ +│ │ │ +│ ┌────────────────────────────────┴───────────────────────────────────┐ │ +│ │ Hypervisor: KVM / Xen + Argo/OpenXT / Qubes OS Air │ │ +│ │ • Flask-XSM / HMX mandatory access control between VMs │ │ +│ │ • TrenchBoot measured boot: firmware → hypervisor → OS │ │ +│ │ • Open-IOV: per-VM virtual GPUs, IOMMU-enforced isolation │ │ +│ └────────────────────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────────┘ +``` + +### 2.1 Roles and trust + +| Component | Trust | Why | +|---|---|---| +| Agent VM | untrusted | runs generated code, may be compromised | +| Inference VM | high (holds weights) | isolated from agents by MAC and by device virtualization | +| GPU-owning VM (optional) | high | owns the physical device; can serve others through Open-IOV | +| Infrastructure VMs | medium | GPU acceleration optional, no access to weights | +| Hypervisor | TCB | enforced by the platform (Argo/OpenXT/Qubes) | +| Open-IOV device | TCB-adjacent | must fail closed; audited (see the security doc) | + +### 2.2 Data flow + +1. An agent VM issues an API call to the inference VM over an internal, + MAC-controlled channel. +2. The inference VM's vLLM/SGLang (unchanged) serves the request with the GPU, + using its **KV-cache plugins** (also unchanged). +3. No request ever leaves the device: the platform's network policy forbids + external inference endpoints by construction. + +### 2.3 Optional topology: GPU relay domain + +Where the platform prefers it (Xen with `xen-pci`), a single domain owns the +physical GPU and **relays GPU acceleration to other domains via Open-IOV**. This +gives a stronger separation: the process running the model is itself isolated +from the domain that owns the silicon, and agents are isolated from both. + +## 3. Why this needs Open-IOV (and not something else) + +* **Discrete passthrough** gives the GPU to exactly one VM — but HAAEEP needs + the GPU in the inference VM *and* optional acceleration elsewhere. +* **Process-level sharing** (host CUDA with containers) does not give + VM-level isolation, which is the whole point of the platform. +* **Proprietary vGPU** could do much of this — but you cannot build a + sovereign, auditable, redistributable edge platform on a licensed host stack + you cannot inspect or modify. +* **Software GPU emulation** cannot run models usefully. + +Open-IOV is the only component in this design that *must* be open, permissively +licensed and version-durable — so it is the component we are building. + +## 4. Roadmap for HAAEEP + +| Milestone | Depends on | +|---|---| +| CUDA execution in a guest (rung 3) | Open-IOV forwarding work | +| Two-VM coexistence (inference + agent) with isolation | Open-IOV multi-tenancy + platform MAC | +| Xen/OpenXT or Qubes Air variant | Open-IOV transport backends | +| Attested boot-to-inference chain | TrenchBoot integration | +| Published reference platform (scripts, docs) | all of the above | + +We will document each milestone as it lands, with the same evidence discipline +as the rest of the project. + +## 5. Collaboration models + +### 5.1 Open source projects + +Adopt the code under BSD-3-Clause (kernel modules are GPL-2.0). We will: + +* treat your interface requirements as design inputs (transports, IRQ models, + memory interfaces), +* provide the transport abstraction you need rather than asking you to fork, +* test against your platform in our matrix. + +### 5.2 Commercial partners + +Welcome, with no license fee: + +* ship Open-IOV in products; differentiate above it, +* sponsor support for specific hardware/driver versions, +* co-develop under the same permissive terms. + +### 5.3 Hardware vendors + +Nothing is required from you. Our goal is that new hardware works through +configuration and measurement. If you want to accelerate support, share +hardware and traces; we publish what we learn (within the bounds of what can be +published). + +### 5.4 Security researchers + +The emulation's attack surface is small and documented. Break it. We will +publish findings and fixes, and we will credit you. + +## 6. Cross-project notes + +| Project | Relationship | +|---|---| +| **Qubes OS Air** | provides compartmentalization on modern hardware; Open-IOV makes GPU acceleration usable per compartment (especially AI workloads) | +| **TrenchBoot** | provides measured/attested boot; combined, the platform can attest the inference path | +| **Argo / OpenXT** | provide hypervisor-mediated exchange (HMX) and Flask-XSM MAC; Open-IOV respects and complements that policy | +| **KVM/VFIO gaming** | established the fast-passthrough pattern; Open-IOV serves the users that need *multiple* VMs or cannot dedicate the GPU | +| **vLLM / SGLang** | remain the inference answer; Open-IOV isolates them without changing them; KV-cache plugins keep working | + +## 7. How to help + +* **Platform builders**: bring your hypervisor, tell us the transport you need. +* **Security engineers**: review the isolation model and the debug surface. +* **AI-infrastructure engineers**: define the reference HAAEEP deployment with + us (model server, KV cache, agent harness). +* **Everyone**: run the ladder on your hardware and send the evidence. + +Contact and participation channels are listed in the repository README. We +would rather have a hundred partners than a hundred forks. |
