1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
|
# Open-IOV vGPU-IO Portable Architecture Design
> **Status: superseded (historical).** Part of the pre-2026-09-23 docs; kept
> as a record. Current entry point: [docs/README.md](README.md). Portability
> replacement:
> [06-portability/hardware-software-matrix.md](06-portability/hardware-software-matrix.md).
## Design Goals
### 1. Driver Version Portability
- **Support NVIDIA OpenRM 550+** without code changes per version
- **Automatic version detection** at runtime
- **Backward/forward compatibility** for RPC interfaces
- **Graceful degradation** when features unavailable
### 2. Hardware Portability
- **Support multiple GPU architectures**: Turing (TU102/TU104), Ampere (GA100/GA102), Hopper (GH100), Blackwell (GB202/GB203/GB100), future
- **Support multiple form factors**: Consumer (RTX 3090/4090/5090), Data Center (A100/H100/B200), DGX Spark, DGX Station
- **Automatic hardware detection** via PCI device IDs and firmware signatures
- **Capability-based feature enablement**
### 3. Zero Guest Modification
- **Guest runs stock NVIDIA driver** from NVIDIA website
- **No Open-IOV code in guest** kernel or userspace
- **Standard VFIO/mdev interface** only
---
## Core Architecture: Three-Layer Abstraction
```
╔═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║ PORTABLE ARCHITECTURE LAYERS ║
╠═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║ ║
║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║
║ │ LAYER 1: HARDWARE ABSTRACTION LAYER (HAL) │ ║
║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║
║ │ │ GpuArchitecture (abstract base) │ │ ║
║ │ │ ├── detect(pci_dev) → GpuArchitecture* │ │ ║
║ │ │ ├── get_pci_device_ids() → vector<DeviceID> │ │ ║
║ │ │ ├── get_bar_layout() → BarLayout │ │ ║
║ │ │ ├── get_gsp_firmware_info() → FirmwareInfo │ │ ║
║ │ │ ├── get_rpc_dispatcher_addresses() → RpcAddresses │ │ ║
║ │ │ ├── get_capabilities() → GpuCapabilities │ │ ║
║ │ │ ├── create_rpc_handler(rpc_type) → RpcHandler* │ │ ║
║ │ │ └── create_bar_emulator() → BarEmulator* │ │ ║
║ │ │ │ │ ║
║ │ │ Concrete Implementations (per architecture): │ │ ║
║ │ │ ├── TuringArchitecture (TU102, TU104, TU106, TU116, TU117) │ │ ║
║ │ │ ├── AmpereArchitecture (GA100, GA102, GA103, GA104, GA106, GA107) │ │ ║
║ │ │ ├── HopperArchitecture (GH100, GH200) │ │ ║
║ │ │ ├── BlackwellArchitecture (GB202, GB203, GB205, GB206, GB100) │ │ ║
║ │ │ └── FutureArchitecture (auto-generated from firmware analysis) │ │ ║
║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║
║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║
║ ║
║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║
║ │ LAYER 2: VERSION ADAPTATION LAYER (VAL) │ ║
║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║
║ │ │ DriverVersion (semantic: major.minor.patch) │ │ ║
║ │ │ ├── parse(version_string) → DriverVersion │ │ ║
║ │ │ ├── is_compatible(min_version, max_version) → bool │ │ ║
║ │ │ ├── get_rpc_interface_version() → RpcInterfaceVersion │ │ ║
║ │ │ ├── get_firmware_signature() → string (SHA256 of firmware) │ │ ║
║ │ │ └── get_feature_flags() → FeatureFlags │ │ ║
║ │ │ │ │ ║
║ │ │ RpcInterfaceVersion (per driver version): │ │ ║
║ │ │ ├── get_rpc_type_count() → int │ │ ║
║ │ │ ├── get_rpc_struct_layout(rpc_type) → StructLayout │ │ ║
║ │ │ ├── get_subcommand_mapping() → SubcommandMap │ │ ║
║ │ │ ├── get_error_codes() → ErrorCodeMap │ │ ║
║ │ │ └── get_capability_bits() → CapabilityBits │ │ ║
║ │ │ │ │ ║
║ │ │ Version Registry (loaded at runtime): │ │ ║
║ │ │ ├── 550.x → RpcInterfaceV1 │ │ ║
║ │ │ ├── 560.x → RpcInterfaceV2 (added MIGRATE subcommands) │ │ ║
║ │ │ ├── 570.x → RpcInterfaceV3 (added DEBUG, NOTIFY enhancements) │ │ ║
║ │ │ ├── 580.x → RpcInterfaceV4 (confidential computing, TEE) │ │ ║
║ │ │ └── auto-detect from firmware signature │ │ ║
║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║
║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║
║ ║
║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║
║ │ LAYER 3: FIRMWARE-DRIVEN RPC ENGINE (FDRE) │ ║
║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║
║ │ │ FirmwareAnalyzer (runs at module load) │ │ ║
║ │ │ ├── load_firmware(firmware_path) → FirmwareImage │ │ ║
║ │ │ ├── find_rpc_dispatcher() → address │ │ ║
║ │ │ ├── find_jump_table() → address │ │ ║
║ │ │ ├── find_handler_addresses() → map<rpc_type, address> │ │ ║
║ │ │ ├── extract_struct_layouts() → map<rpc_type, StructLayout> │ │ ║
║ │ │ ├── compute_firmware_signature() → SHA256 │ │ ║
║ │ │ └── validate_against_known_versions() → DriverVersion │ │ ║
║ │ │ │ │ ║
║ │ │ DynamicRpcDispatcher: │ │ ║
║ │ │ ├── dispatch(rpc_msg) → uses discovered handler addresses │ │ ║
║ │ │ ├── fallback_to_emulation(rpc_type) → if handler not found │ │ ║
║ │ │ └── hot_patch_handler(rpc_type, new_address) → for live updates │ │ ║
║ │ │ │ │ ║
║ │ │ EmulationFallbackRegistry: │ │ ║
║ │ │ ├── register_emulation(rpc_type, EmulationHandler*) │ │ ║
║ │ │ ├── get_emulation(rpc_type) → EmulationHandler* │ │ ║
║ │ │ └── supports_emulation(rpc_type) → bool │ │ ║
║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║
║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║
║ ║
╚════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
```
---
## Configuration-Driven Design
### 1. Hardware Configuration Files
```yaml
# config/hardware/turing.yaml
architecture: "turing"
pci_device_ids:
- vendor: 0x10de
device: 0x1e07 # TU102
name: "RTX 2080 Ti"
- vendor: 0x10de
device: 0x1e08 # TU104
name: "RTX 2080 / T4"
- vendor: 0x10de
device: 0x1ed3 # TU104 mobile
name: "RTX 2080 Super Mobile"
bar_layout:
bar0_size: 16MB
bar1_size: 256MB
bar2_size: 32MB
bar3_size: 32MB
bar0_prefetchable: true
bar1_prefetchable: true
gsp:
firmware_base: 0x04000000
doorbell_offset: 0x110C00
shared_mem_offset: 0x1000
shared_mem_size: 4KB
rpc_dispatcher_primary: 0x057A1AC0
rpc_dispatcher_secondary: 0x04ED39DC
rpc_jump_table: 0x059BFEB0
handlers:
alloc: 0x057E94EC
schedule: 0x051EFBF4
bar_trap: 0x04CB8188
wpr_1: 0x04F00B0C
wpr_2: 0x04F00C78
migration_table_1: 0x04179ED8
migration_table_2: 0x04179FBC
capabilities:
max_vfs: 7
supports_mig: true
supports_wpr: true
supports_bar_trap: true
supports_scheduling: true
supports_notifications: true
msi_x_vectors: 32
```
```yaml
# config/hardware/ampere.yaml
architecture: "ampere"
pci_device_ids:
- vendor: 0x10de
device: 0x20b0 # GA100
name: "A100"
- vendor: 0x10de
device: 0x2204 # GA102
name: "RTX 3090"
- vendor: 0x10de
device: 0x2206 # GA102
name: "RTX 3080"
bar_layout:
bar0_size: 16MB
bar1_size: 512MB # Larger FB on Ampere
bar2_size: 64MB
bar3_size: 64MB
gsp:
firmware_base: 0x04000000
doorbell_offset: 0x110C00
shared_mem_offset: 0x1000
# Addresses discovered at runtime via FirmwareAnalyzer
capabilities:
max_vfs: 7
supports_mig: true
supports_mig_mig: true # MIG (Multi-Instance GPU)
supports_confidential: false
supports_wpr: true
supports_bar_trap: true
msi_x_vectors: 64
```
```yaml
# config/hardware/blackwell.yaml
architecture: "blackwell"
pci_device_ids:
- vendor: 0x10de
device: 0x2480 # GB202 (RTX 5090)
name: "RTX 5090"
- vendor: 0x10de
device: 0x2380 # GB100 (B200)
name: "B200"
- vendor: 0x10de
device: 0x2384 # GB205
name: "RTX 5070"
bar_layout:
bar0_size: 16MB
bar1_size: 1GB # Blackwell unified memory
bar2_size: 128MB
bar3_size: 128MB
gsp:
firmware_base: 0x04000000
doorbell_offset: 0x110C00
shared_mem_offset: 0x1000
capabilities:
max_vfs: 8
supports_mig: true
supports_mig_mig: true
supports_confidential: true # TEE support
supports_wpr: true
supports_bar_trap: true
supports_nvlink: true
msi_x_vectors: 128
```
### 2. Driver Version Configuration
```yaml
# config/versions/550.yaml
version: "550"
rpc_interface_version: 1
rpc_types:
- ALLOC: 0x01
- FREE: 0x02
- MAP: 0x03
- UNMAP: 0x04
- PIN: 0x05
- UNPIN: 0x06
- SCHEDULE: 0x07
- YIELD: 0x08
- NOTIFY: 0x09
- BAR_TRAP: 0x0A
- MIGRATE: 0x0B
- DEBUG: 0x0C
struct_layouts:
ALLOC:
size: 48
fields:
- {name: size, offset: 0, type: u64}
- {name: alignment, offset: 8, type: u64}
- {name: mem_type, offset: 16, type: u32}
- {name: flags, offset: 20, type: u32}
- {name: addr_hint, offset: 24, type: u64}
- {name: pid, offset: 32, type: u32}
capabilities:
supports_migrate: true
migrate_subcommands:
- SAVE: 0x04
- RESTORE: 0x05
supports_notifications: basic
```
```yaml
# config/versions/570.yaml
version: "570"
rpc_interface_version: 3
extends: "550"
rpc_types:
- NOTIFY: 0x09 # Enhanced with subcommands
- MIGRATE: 0x0B # Enhanced with 4-phase
struct_layouts:
NOTIFY:
size: 32
fields:
- {name: type, offset: 0, type: u32}
- {name: subcmd, offset: 4, type: u32}
- {name: event_fd, offset: 8, type: u64}
- {name: flags, offset: 16, type: u32}
capabilities:
supports_notifications: enhanced
supports_migrate_4phase: true
migrate_phases: [QUERY, PREPARE, TRANSFER, COMMIT, CLEANUP]
```
```yaml
# config/versions/580.yaml
version: "580"
rpc_interface_version: 4
extends: "570"
rpc_types:
- TEE_ATTEST: 0x0D # New in 580 for confidential computing
- TEE_SEAL: 0x0E
- TEE_UNSEAL: 0x0F
capabilities:
supports_confidential: true
supports_tee: true
tee_attestation_types: [CPU, GPU, MEMORY]
```
---
## Runtime Detection & Adaptation Flow
```
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ MODULE LOAD: RUNTIME DETECTION FLOW │
├─────────────────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. open-iov-vfio.ko loads │
│ │ │
│ ▼ │
│ 2. Probe PCI device │
│ │ │
│ ▼ │
│ 3. Match PCI Device ID → Hardware Config (turing.yaml, ampere.yaml, etc.) │
│ │ │
│ ▼ │
│ 4. Load Hardware Config → Instantiate GpuArchitecture subclass │
│ │ │
│ ▼ │
│ 5. Load GSP Firmware (from /lib/firmware or host nvidia.ko) │
│ │ │
│ ▼ │
│ 6. FirmwareAnalyzer.analyze(firmware) │
│ ├── Compute firmware SHA256 │
│ ├── Find RPC dispatcher via pattern matching │
│ ├── Find jump table via cross-references │
│ ├── Extract handler addresses for all 12+ RPC types │
│ ├── Extract struct layouts via pattern matching │
│ └── Match firmware signature → DriverVersion (550, 560, 570, 580, etc.) │
│ │ │
│ ▼ │
│ 7. Load Version Config → Instantiate RpcInterfaceVersion │
│ │ │
│ ▼ │
│ 8. Validate: Hardware Config + Version Config compatible? │
│ │ │
│ ├── YES → Initialize DynamicRpcDispatcher with discovered addresses │
│ ├── PARTIAL → Enable emulation for missing handlers │
│ └── NO → Fail with clear error: "Unsupported GPU/driver combination" │
│ │ │
│ ▼ │
│ 9. Register EmulationFallbackRegistry for all RPC types │
│ │ │
│ ▼ │
│ 10. Start GSP Emulation Thread (per mdev) │
│ │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
---
## FirmwareAnalyzer - Automatic Address Discovery
```c
// kernel/firmware_analyzer.c
struct firmware_signature {
uint8_t sha256[32];
char version_string[64];
uint32_t crc32;
};
struct discovered_rpc_handlers {
uint64_t primary_dispatcher;
uint64_t secondary_dispatcher;
uint64_t jump_table;
struct {
uint32_t rpc_type;
uint64_t handler_address;
bool found;
} handlers[16]; // Support up to 16 RPC types
uint32_t handler_count;
};
struct firmware_analyzer {
const uint8_t *firmware;
size_t firmware_size;
uint64_t firmware_base;
struct discovered_rpc_handlers discovered;
struct firmware_signature signature;
};
int firmware_analyzer_init(struct firmware_analyzer *fa, const char *firmware_path) {
// Load firmware file
fa->firmware = load_firmware_file(firmware_path, &fa->firmware_size);
if (!fa->firmware)
return -ENOENT;
// Compute signature
sha256(fa->firmware, fa->firmware_size, fa->signature.sha256);
fa->signature.crc32 = crc32(0, fa->firmware, fa->firmware_size);
// Set base address (from hardware config)
fa->firmware_base = get_hardware_config()->gsp.firmware_base;
return 0;
}
int firmware_analyzer_discover(struct firmware_analyzer *fa) {
// 1. Find primary dispatcher: search for known pattern
// Pattern: "RPC_DISPATCH" string reference or known instruction sequence
fa->discovered.primary_dispatcher = find_pattern(fa->firmware, fa->firmware_size,
"RPC_DISPATCH", fa->firmware_base);
// 2. Find jump table: cross-reference from dispatcher
fa->discovered.jump_table = find_jump_table(fa->firmware, fa->firmware_size,
fa->discovered.primary_dispatcher);
// 3. Parse jump table entries (32 entries for 12 RPC types)
parse_jump_table(fa->firmware, fa->discovered.jump_table, &fa->discovered);
// 4. Find secondary dispatcher
fa->discovered.secondary_dispatcher = find_secondary_dispatcher(fa->firmware,
fa->firmware_size, fa->discovered.primary_dispatcher);
// 5. Find specialized handlers via string references
find_handler_by_string(fa, "BAR_TRAP", "BAR1", "ForceBarAccess", &fa->discovered.handlers[GSP_RPC_BAR_TRAP]);
find_handler_by_string(fa, "WPR", "completion", "timeout", &fa->discovered.handlers[GSP_RPC_NOTIFY]);
find_handler_by_string(fa, "ALLOC", "size", "alignment", &fa->discovered.handlers[GSP_RPC_ALLOC]);
find_handler_by_string(fa, "SCHEDULE", "work", "callback", &fa->discovered.handlers[GSP_RPC_SCHEDULE]);
find_handler_by_string(fa, "MIGRATE", "checkpoint", "progress", &fa->discovered.handlers[GSP_RPC_MIGRATE]);
// 6. Extract struct layouts via pattern matching on handler prologues
extract_struct_layouts(fa);
// 7. Match signature against known versions
match_firmware_version(fa);
return 0;
}
void firmware_analyzer_match_version(struct firmware_analyzer *fa) {
// Compare signature against known database
static const struct {
const char *version;
uint8_t sha256[32];
uint32_t crc32;
} known_versions[] = {
{"550.163.01", {0x...}, 0x...},
{"560.28.03", {0x...}, 0x...},
{"570.144.00", {0x...}, 0x...},
{"570.144.01", {0x...}, 0x...},
{"580.00.00", {0x...}, 0x...},
};
for (int i = 0; i < ARRAY_SIZE(known_versions); i++) {
if (memcmp(fa->signature.sha256, known_versions[i].sha256, 32) == 0) {
strncpy(fa->signature.version_string, known_versions[i].version, 63);
return;
}
}
// Unknown version - use heuristic based on handler count and features
if (fa->discovered.handler_count >= 12)
strcpy(fa->signature.version_string, "570+ (unknown)");
else if (fa->discovered.handler_count >= 10)
strcpy(fa->signature.version_string, "560+ (unknown)");
else
strcpy(fa->signature.version_string, "550+ (unknown)");
}
```
---
## Dynamic RPC Dispatcher
```c
// kernel/dynamic_rpc_dispatcher.c
struct dynamic_rpc_dispatcher {
struct discovered_rpc_handlers *discovered;
struct rpc_interface_version *interface;
struct emulation_fallback_registry *fallbacks;
spinlock_t lock;
uint64_t dispatch_stats[16];
};
int dynamic_rpc_dispatcher_dispatch(struct dynamic_rpc_dispatcher *dr,
struct gsp_rpc_msg *req,
struct gsp_rpc_response *rsp) {
// 1. Validate RPC type
if (req->type >= dr->interface->rpc_type_count)
return -ENOTTY;
// 2. Check if we have a native handler address
uint64_t handler_addr = 0;
for (int i = 0; i < dr->discovered->handler_count; i++) {
if (dr->discovered->handlers[i].rpc_type == req->type) {
handler_addr = dr->discovered->handlers[i].handler_address;
break;
}
}
// 3. Dispatch to native handler if available
if (handler_addr) {
return dynamic_rpc_call_native(dr, handler_addr, req, rsp);
}
// 4. Fallback to emulation
struct emulation_handler *emu = emulation_fallback_get(dr->fallbacks, req->type);
if (emu) {
dr->dispatch_stats[req->type]++; // Track emulation usage
return emu->handle(req, rsp);
}
// 5. No handler, no emulation
rsp->status = -ENOTTY;
return -ENOTTY;
}
int dynamic_rpc_call_native(struct dynamic_rpc_dispatcher *dr,
uint64_t handler_addr,
struct gsp_rpc_msg *req,
struct gsp_rpc_response *rsp) {
// Use firmware-decompiled calling convention
// This calls the actual GSP firmware handler via the discovered address
// In emulation mode, we call our C reimplementation instead
if (gsp_emulation_mode_enabled()) {
// Call our C reimplementation (from gsp_rpc_client.c)
return gsp_rpc_client_call(req->type, req, rsp);
} else {
// Native mode: would call into GSP firmware
// Not used in our architecture (we always emulate in host)
return -ENOTSUPP;
}
}
```
---
## Capability Negotiation with Guest
```
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ GUEST DRIVER CAPABILITY NEGOTIATION │
├─────────────────────────────────────────────────────────────────────────────────────┤
│ │
│ Guest nvidia.ko probes vGPU: │
│ 1. Reads PCI Config Space → gets device ID, BAR sizes, capabilities │
│ 2. Maps BAR0 → finds GSP doorbell + shared memory │
│ 3. Sends RPC: QUERY_CAPABILITIES (new RPC type or subcommand) │
│ │
│ Host DynamicRpcDispatcher receives QUERY_CAPABILITIES: │
│ 1. Constructs response from: │
│ - Hardware Config capabilities │
│ - Version Config capabilities │
│ - EmulationFallbackRegistry (what we can emulate) │
│ 2. Returns capability bitmap to guest │
│ │
│ Guest nvidia.ko enables features based on response: │
│ - If MIG supported → enables MIG code paths │
│ - If WPR supported → enables WPR code paths │
│ - If CONFIDENTIAL supported → enables TEE code paths │
│ - If NOTIFY enhanced → uses async eventfd │
│ │
│ This allows SAME guest driver to work across ALL hardware/versions! │
│ │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
---
## Update Strategy for New Driver Versions
### 1. Adding New Driver Version (e.g., 590)
```bash
# 1. Add version config
cp config/versions/580.yaml config/versions/590.yaml
# Edit: version, rpc_interface_version, new RPC types, capabilities
# 2. Test with new firmware
# FirmwareAnalyzer auto-discovers addresses - NO CODE CHANGES NEEDED
# 3. If new RPC types added:
# - Add to config/versions/590.yaml
# - Add emulation handler in kernel/emulation_fallbacks.c
# - Register in EmulationFallbackRegistry
# 4. If struct layouts changed:
# - FirmwareAnalyzer extracts new layouts automatically
# - Version config specifies layout version
```
### 2. Adding New GPU Architecture (e.g., Rubin R100)
```bash
# 1. Add hardware config
cp config/hardware/blackwell.yaml config/hardware/rubin.yaml
# Edit: PCI device IDs, BAR layout, capabilities
# 2. Add GpuArchitecture subclass (if special handling needed)
# kernel/arch/rubin_architecture.c
# 3. Register in architecture factory
# kernel/architecture_factory.c
# 4. Test - FirmwareAnalyzer handles address discovery
```
### 3. Automated Firmware Analysis Pipeline
```yaml
# .github/workflows/firmware-analysis.yml
name: Firmware Analysis
on:
schedule:
- cron: '0 0 * * 0' # Weekly
workflow_dispatch:
inputs:
firmware_url:
description: 'NVIDIA firmware download URL'
required: true
jobs:
analyze:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Download firmware
run: wget -O firmware.bin "${{ inputs.firmware_url }}"
- name: Run FirmwareAnalyzer
run: |
./tools/firmware_analyzer firmware.bin > analysis.json
- name: Compare with known versions
run: ./tools/compare_analysis.py analysis.json
- name: Generate config if new
run: |
if [ "$(cat analysis.json | jq -r '.is_new')" = "true" ]; then
./tools/generate_config.py analysis.json > config/versions/NEW.yaml
git add config/versions/NEW.yaml
git commit -m "Add support for new firmware version"
git push
fi
```
---
## Testing Matrix for Portability
| GPU Architecture | PCI Device IDs | Driver Versions | Test Status |
|------------------|----------------|-----------------|-------------|
| **Turing** | 10de:1e07, 10de:1e08, 10de:1ed3 | 550, 560, 570, 580 | ✅ Baseline |
| **Ampere** | 10de:20b0, 10de:2204, 10de:2206 | 550, 560, 570, 580 | 🔄 In Progress |
| **Hopper** | 10de:2330, 10de:2334 | 560, 570, 580 | 🔄 Planned |
| **Blackwell** | 10de:2480, 10de:2380, 10de:2384 | 570, 580 | 🔄 Planned |
| **Rubin (Future)** | TBD | 590+ | 📋 Designed |
### Per-Combination Tests
```bash
# Automated test matrix
for arch in turing ampere hopper blackwell; do
for version in 550 560 570 580; do
if [ -f "config/hardware/${arch}.yaml" ] && [ -f "config/versions/${version}.yaml" ]; then
echo "Testing ${arch} + ${version}..."
./scripts/test_combination.sh ${arch} ${version}
fi
done
done
```
---
## Implementation File Structure (Portable)
```
open-iov/
├── config/
│ ├── hardware/
│ │ ├── turing.yaml
│ │ ├── ampere.yaml
│ │ ├── hopper.yaml
│ │ ├── blackwell.yaml
│ │ └── rubin.yaml (future)
│ ├── versions/
│ │ ├── 550.yaml
│ │ ├── 560.yaml
│ │ ├── 570.yaml
│ │ ├── 580.yaml
│ │ └── 590.yaml (future)
│ └── defaults.yaml
├── kernel/
│ ├── architecture/
│ │ ├── architecture.h # Abstract base class
│ │ ├── architecture_factory.c # Detection & instantiation
│ │ ├── turing_architecture.c
│ │ ├── ampere_architecture.c
│ │ ├── hopper_architecture.c
│ │ ├── blackwell_architecture.c
│ │ └── generic_architecture.c # Fallback
│ ├── version/
│ │ ├── version.h # DriverVersion, RpcInterfaceVersion
│ │ ├── version_registry.c # Loads version configs
│ │ ├── interface_v1.c # 550 interface
│ │ ├── interface_v2.c # 560 interface
│ │ ├── interface_v3.c # 570 interface
│ │ └── interface_v4.c # 580 interface
│ ├── firmware/
│ │ ├── firmware_analyzer.c # Auto-discovers addresses
│ │ ├── firmware_analyzer.h
│ │ ├── pattern_matcher.c # Binary pattern matching
│ │ └── struct_layout_extractor.c
│ ├── dispatch/
│ │ ├── dynamic_rpc_dispatcher.c
│ │ ├── dynamic_rpc_dispatcher.h
│ │ ├── emulation_fallback_registry.c
│ │ └── emulation_fallback_registry.h
│ ├── gsp_emulation.c # Uses dynamic dispatcher
│ ├── main.c # Integrates all layers
│ └── Makefile
├── tools/
│ ├── firmware_analyzer/ # Standalone analysis tool
│ ├── compare_analysis.py
│ ├── generate_config.py
│ └── test_combination.sh
└── docs/
├── PORTABLE_ARCHITECTURE.md
└── VERSION_COMPATIBILITY.md
```
---
## Key Design Principles Summary
| Principle | Implementation |
|-----------|----------------|
| **No hardcoded addresses** | FirmwareAnalyzer discovers at runtime |
| **No hardcoded struct layouts** | Version config + FirmwareAnalyzer extraction |
| **No hardcoded device IDs** | Hardware config YAML files |
| **No hardcoded capabilities** | Capability negotiation via QUERY_CAPABILITIES RPC |
| **Graceful degradation** | EmulationFallbackRegistry for missing handlers |
| **Forward compatibility** | Version config `extends` mechanism |
| **Automated updates** | Firmware analysis pipeline + config generation |
| **Test matrix** | Automated CI across architecture × version combinations |
---
*Document Version: 1.0*
*Repository: $OPEN_IOV_ROOT*
*Branch: refactor/host-gsp-emulation*
*Base Commit: f9ed5d8*
|