1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
|
# HAAEEP and Collaboration
> The High Assurance Agent Edge Environment Platform: a concrete, buildable
> use-case architecture that Open-IOV exists to enable — and how we want to
> build it with you.
## 1. Motivation
An AI agent harness (Claude Code, OpenCode, or any future equivalent) is an
untrusted, capability-constrained workload that runs generated code, holds
credentials, and talks to a model. In a *cloud* deployment, the agent's requests
leave the device. In a *secure edge* deployment — a workstation, a kiosk, a
vehicle, a medical or industrial system — they must not.
We want an edge platform where:
* inference runs **locally**, in its own VM, with the GPU;
* agents run in **their own VMs**, without the GPU and without the weights;
* other functions (desktop, networking, storage) run in **other VMs**, each
isolated;
* the whole stack is **attestable**, so remote parties can verify it;
* and nothing in this design requires the model server, the agent, or the GPU
driver to be modified.
That platform is HAAEEP. Open-IOV is its GPU component.
## 2. Architecture
```
┌──────────────────────────────────────────────────────────────────────────┐
│ EDGE DEVICE — one GPU, local inference only │
│ │
│ ┌─────────────────┐ ┌──────────────────┐ ┌───────────────────┐ │
│ │ Agent VM │ │ Inference VM │ │ Infrastructure │ │
│ │ capability- │────▶│ vLLM / SGLang │ │ desktop, net, │ │
│ │ constrained │ API │ owns GPU via │ │ storage VMs │ │
│ │ no GPU │ │ Open-IOV mdev or │ │ optional GPU accel│ │
│ │ │ │ xen-pci + relaying│ │ via Open-IOV │ │
│ └─────────────────┘ └────────┬─────────┘ └───────────────────┘ │
│ │ │
│ ┌────────────────────────────────┴───────────────────────────────────┐ │
│ │ Hypervisor: KVM / Xen + Argo/OpenXT / Qubes OS Air │ │
│ │ • Flask-XSM / HMX mandatory access control between VMs │ │
│ │ • TrenchBoot measured boot: firmware → hypervisor → OS │ │
│ │ • Open-IOV: per-VM virtual GPUs, IOMMU-enforced isolation │ │
│ └────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
### 2.1 Roles and trust
| Component | Trust | Why |
|---|---|---|
| Agent VM | untrusted | runs generated code, may be compromised |
| Inference VM | high (holds weights) | isolated from agents by MAC and by device virtualization |
| GPU-owning VM (optional) | high | owns the physical device; can serve others through Open-IOV |
| Infrastructure VMs | medium | GPU acceleration optional, no access to weights |
| Hypervisor | TCB | enforced by the platform (Argo/OpenXT/Qubes) |
| Open-IOV device | TCB-adjacent | must fail closed; audited (see the security doc) |
### 2.2 Data flow
1. An agent VM issues an API call to the inference VM over an internal,
MAC-controlled channel.
2. The inference VM's vLLM/SGLang (unchanged) serves the request with the GPU,
using its **KV-cache plugins** (also unchanged).
3. No request ever leaves the device: the platform's network policy forbids
external inference endpoints by construction.
### 2.3 Optional topology: GPU relay domain
Where the platform prefers it (Xen with `xen-pci`), a single domain owns the
physical GPU and **relays GPU acceleration to other domains via Open-IOV**. This
gives a stronger separation: the process running the model is itself isolated
from the domain that owns the silicon, and agents are isolated from both.
## 3. Why this needs Open-IOV (and not something else)
* **Discrete passthrough** gives the GPU to exactly one VM — but HAAEEP needs
the GPU in the inference VM *and* optional acceleration elsewhere.
* **Process-level sharing** (host CUDA with containers) does not give
VM-level isolation, which is the whole point of the platform.
* **Proprietary vGPU** could do much of this — but you cannot build a
sovereign, auditable, redistributable edge platform on a licensed host stack
you cannot inspect or modify.
* **Software GPU emulation** cannot run models usefully.
Open-IOV is the only component in this design that *must* be open, permissively
licensed and version-durable — so it is the component we are building.
## 4. Roadmap for HAAEEP
| Milestone | Depends on |
|---|---|
| CUDA execution in a guest (rung 3) | Open-IOV forwarding work |
| Two-VM coexistence (inference + agent) with isolation | Open-IOV multi-tenancy + platform MAC |
| Xen/OpenXT or Qubes Air variant | Open-IOV transport backends |
| Attested boot-to-inference chain | TrenchBoot integration |
| Published reference platform (scripts, docs) | all of the above |
We will document each milestone as it lands, with the same evidence discipline
as the rest of the project.
## 5. Collaboration models
### 5.1 Open source projects
Adopt the code under BSD-3-Clause (kernel modules are GPL-2.0). We will:
* treat your interface requirements as design inputs (transports, IRQ models,
memory interfaces),
* provide the transport abstraction you need rather than asking you to fork,
* test against your platform in our matrix.
### 5.2 Commercial partners
Welcome, with no license fee:
* ship Open-IOV in products; differentiate above it,
* sponsor support for specific hardware/driver versions,
* co-develop under the same permissive terms.
### 5.3 Hardware vendors
Nothing is required from you. Our goal is that new hardware works through
configuration and measurement. If you want to accelerate support, share
hardware and traces; we publish what we learn (within the bounds of what can be
published).
### 5.4 Security researchers
The emulation's attack surface is small and documented. Break it. We will
publish findings and fixes, and we will credit you.
## 6. Cross-project notes
| Project | Relationship |
|---|---|
| **Qubes OS Air** | provides compartmentalization on modern hardware; Open-IOV makes GPU acceleration usable per compartment (especially AI workloads) |
| **TrenchBoot** | provides measured/attested boot; combined, the platform can attest the inference path |
| **Argo / OpenXT** | provide hypervisor-mediated exchange (HMX) and Flask-XSM MAC; Open-IOV respects and complements that policy |
| **KVM/VFIO gaming** | established the fast-passthrough pattern; Open-IOV serves the users that need *multiple* VMs or cannot dedicate the GPU |
| **vLLM / SGLang** | remain the inference answer; Open-IOV isolates them without changing them; KV-cache plugins keep working |
## 7. How to help
* **Platform builders**: bring your hypervisor, tell us the transport you need.
* **Security engineers**: review the isolation model and the debug surface.
* **AI-infrastructure engineers**: define the reference HAAEEP deployment with
us (model server, KV cache, agent harness).
* **Everyone**: run the ladder on your hardware and send the evidence.
Contact and participation channels are listed in the repository README. We
would rather have a hundred partners than a hundred forks.
|
