diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-23 13:48:07 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-09-23 13:48:07 -0400 |
| commit | fade84f8b5b781d29daad40699d07928dedea550 (patch) | |
| tree | e4156b3d0a6a1c794c90ef6834b949321cdfb5cb /docs/PORTABLE_ARCHITECTURE.md | |
| parent | c587eac377c1c5b720142edadebaace5cb1b17c0 (diff) | |
| download | open-iov-fade84f8b5b781d29daad40699d07928dedea550.tar.gz open-iov-fade84f8b5b781d29daad40699d07928dedea550.zip | |
docs: merge the legacy flat documents, add the CUDA deep-dive and link checks
* Merge and remove the pre-reboot flat documents, preserving their still-useful
material in the structured tree: debugging workflows and the EPT-livelock
case study (05-operations/02), performance microbenchmarks and A/B
methodology (05-operations/01), the version-upgrade runbook
(05-operations/02 section 4, with corrected host/guest framing), kernel-code
pitfalls (07-developers) and the OpenRM build/load order
(03-architecture/06 section 7). The remaining legacy files were verified
redundant and removed.
* docs/README now records where each migrated topic lives.
* Portability: the config YAML files are called out as design targets that no
code reads yet, with what adding support actually requires today.
* Getting started: the OpenRM step is marked optional (native vGPU-management
workflows only; the emulated path uses the distro driver).
* tools docs: rmtool/rmtop marked as planned, pointing at the implemented
tools/rmq subset.
* wiki: new CUDA-Userspace-Driver deep-dive page (public-safe, links the
canonical userspace/cuda analysis) and index update.
* scripts/check-docs-links.py plus a CI step so relative links cannot rot.
* STATE.md references to removed documents repointed.
Diffstat (limited to 'docs/PORTABLE_ARCHITECTURE.md')
| -rw-r--r-- | docs/PORTABLE_ARCHITECTURE.md | 747 |
1 files changed, 0 insertions, 747 deletions
diff --git a/docs/PORTABLE_ARCHITECTURE.md b/docs/PORTABLE_ARCHITECTURE.md deleted file mode 100644 index 4842407..0000000 --- a/docs/PORTABLE_ARCHITECTURE.md +++ /dev/null @@ -1,747 +0,0 @@ -# Open-IOV vGPU-IO Portable Architecture Design - -> **Status: superseded (historical).** Part of the pre-2026-09-23 docs; kept -> as a record. Current entry point: [docs/README.md](README.md). Portability -> replacement: -> [06-portability/hardware-software-matrix.md](06-portability/hardware-software-matrix.md). - -## Design Goals - -### 1. Driver Version Portability -- **Support NVIDIA OpenRM 550+** without code changes per version -- **Automatic version detection** at runtime -- **Backward/forward compatibility** for RPC interfaces -- **Graceful degradation** when features unavailable - -### 2. Hardware Portability -- **Support multiple GPU architectures**: Turing (TU102/TU104), Ampere (GA100/GA102), Hopper (GH100), Blackwell (GB202/GB203/GB100), future -- **Support multiple form factors**: Consumer (RTX 3090/4090/5090), Data Center (A100/H100/B200), DGX Spark, DGX Station -- **Automatic hardware detection** via PCI device IDs and firmware signatures -- **Capability-based feature enablement** - -### 3. Zero Guest Modification -- **Guest runs stock NVIDIA driver** from NVIDIA website -- **No Open-IOV code in guest** kernel or userspace -- **Standard VFIO/mdev interface** only - ---- - -## Core Architecture: Three-Layer Abstraction - -``` -╔═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗ -║ PORTABLE ARCHITECTURE LAYERS ║ -╠═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣ -║ ║ -║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║ -║ │ LAYER 1: HARDWARE ABSTRACTION LAYER (HAL) │ ║ -║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║ -║ │ │ GpuArchitecture (abstract base) │ │ ║ -║ │ │ ├── detect(pci_dev) → GpuArchitecture* │ │ ║ -║ │ │ ├── get_pci_device_ids() → vector<DeviceID> │ │ ║ -║ │ │ ├── get_bar_layout() → BarLayout │ │ ║ -║ │ │ ├── get_gsp_firmware_info() → FirmwareInfo │ │ ║ -║ │ │ ├── get_rpc_dispatcher_addresses() → RpcAddresses │ │ ║ -║ │ │ ├── get_capabilities() → GpuCapabilities │ │ ║ -║ │ │ ├── create_rpc_handler(rpc_type) → RpcHandler* │ │ ║ -║ │ │ └── create_bar_emulator() → BarEmulator* │ │ ║ -║ │ │ │ │ ║ -║ │ │ Concrete Implementations (per architecture): │ │ ║ -║ │ │ ├── TuringArchitecture (TU102, TU104, TU106, TU116, TU117) │ │ ║ -║ │ │ ├── AmpereArchitecture (GA100, GA102, GA103, GA104, GA106, GA107) │ │ ║ -║ │ │ ├── HopperArchitecture (GH100, GH200) │ │ ║ -║ │ │ ├── BlackwellArchitecture (GB202, GB203, GB205, GB206, GB100) │ │ ║ -║ │ │ └── FutureArchitecture (auto-generated from firmware analysis) │ │ ║ -║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║ -║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║ -║ ║ -║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║ -║ │ LAYER 2: VERSION ADAPTATION LAYER (VAL) │ ║ -║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║ -║ │ │ DriverVersion (semantic: major.minor.patch) │ │ ║ -║ │ │ ├── parse(version_string) → DriverVersion │ │ ║ -║ │ │ ├── is_compatible(min_version, max_version) → bool │ │ ║ -║ │ │ ├── get_rpc_interface_version() → RpcInterfaceVersion │ │ ║ -║ │ │ ├── get_firmware_signature() → string (SHA256 of firmware) │ │ ║ -║ │ │ └── get_feature_flags() → FeatureFlags │ │ ║ -║ │ │ │ │ ║ -║ │ │ RpcInterfaceVersion (per driver version): │ │ ║ -║ │ │ ├── get_rpc_type_count() → int │ │ ║ -║ │ │ ├── get_rpc_struct_layout(rpc_type) → StructLayout │ │ ║ -║ │ │ ├── get_subcommand_mapping() → SubcommandMap │ │ ║ -║ │ │ ├── get_error_codes() → ErrorCodeMap │ │ ║ -║ │ │ └── get_capability_bits() → CapabilityBits │ │ ║ -║ │ │ │ │ ║ -║ │ │ Version Registry (loaded at runtime): │ │ ║ -║ │ │ ├── 550.x → RpcInterfaceV1 │ │ ║ -║ │ │ ├── 560.x → RpcInterfaceV2 (added MIGRATE subcommands) │ │ ║ -║ │ │ ├── 570.x → RpcInterfaceV3 (added DEBUG, NOTIFY enhancements) │ │ ║ -║ │ │ ├── 580.x → RpcInterfaceV4 (confidential computing, TEE) │ │ ║ -║ │ │ └── auto-detect from firmware signature │ │ ║ -║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║ -║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║ -║ ║ -║ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ║ -║ │ LAYER 3: FIRMWARE-DRIVEN RPC ENGINE (FDRE) │ ║ -║ │ ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ║ -║ │ │ FirmwareAnalyzer (runs at module load) │ │ ║ -║ │ │ ├── load_firmware(firmware_path) → FirmwareImage │ │ ║ -║ │ │ ├── find_rpc_dispatcher() → address │ │ ║ -║ │ │ ├── find_jump_table() → address │ │ ║ -║ │ │ ├── find_handler_addresses() → map<rpc_type, address> │ │ ║ -║ │ │ ├── extract_struct_layouts() → map<rpc_type, StructLayout> │ │ ║ -║ │ │ ├── compute_firmware_signature() → SHA256 │ │ ║ -║ │ │ └── validate_against_known_versions() → DriverVersion │ │ ║ -║ │ │ │ │ ║ -║ │ │ DynamicRpcDispatcher: │ │ ║ -║ │ │ ├── dispatch(rpc_msg) → uses discovered handler addresses │ │ ║ -║ │ │ ├── fallback_to_emulation(rpc_type) → if handler not found │ │ ║ -║ │ │ └── hot_patch_handler(rpc_type, new_address) → for live updates │ │ ║ -║ │ │ │ │ ║ -║ │ │ EmulationFallbackRegistry: │ │ ║ -║ │ │ ├── register_emulation(rpc_type, EmulationHandler*) │ │ ║ -║ │ │ ├── get_emulation(rpc_type) → EmulationHandler* │ │ ║ -║ │ │ └── supports_emulation(rpc_type) → bool │ │ ║ -║ │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ ║ -║ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ║ -║ ║ -╚════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝ -``` - ---- - -## Configuration-Driven Design - -### 1. Hardware Configuration Files - -```yaml -# config/hardware/turing.yaml -architecture: "turing" -pci_device_ids: - - vendor: 0x10de - device: 0x1e07 # TU102 - name: "RTX 2080 Ti" - - vendor: 0x10de - device: 0x1e08 # TU104 - name: "RTX 2080 / T4" - - vendor: 0x10de - device: 0x1ed3 # TU104 mobile - name: "RTX 2080 Super Mobile" -bar_layout: - bar0_size: 16MB - bar1_size: 256MB - bar2_size: 32MB - bar3_size: 32MB - bar0_prefetchable: true - bar1_prefetchable: true -gsp: - firmware_base: 0x04000000 - doorbell_offset: 0x110C00 - shared_mem_offset: 0x1000 - shared_mem_size: 4KB - rpc_dispatcher_primary: 0x057A1AC0 - rpc_dispatcher_secondary: 0x04ED39DC - rpc_jump_table: 0x059BFEB0 - handlers: - alloc: 0x057E94EC - schedule: 0x051EFBF4 - bar_trap: 0x04CB8188 - wpr_1: 0x04F00B0C - wpr_2: 0x04F00C78 - migration_table_1: 0x04179ED8 - migration_table_2: 0x04179FBC -capabilities: - max_vfs: 7 - supports_mig: true - supports_wpr: true - supports_bar_trap: true - supports_scheduling: true - supports_notifications: true - msi_x_vectors: 32 -``` - -```yaml -# config/hardware/ampere.yaml -architecture: "ampere" -pci_device_ids: - - vendor: 0x10de - device: 0x20b0 # GA100 - name: "A100" - - vendor: 0x10de - device: 0x2204 # GA102 - name: "RTX 3090" - - vendor: 0x10de - device: 0x2206 # GA102 - name: "RTX 3080" -bar_layout: - bar0_size: 16MB - bar1_size: 512MB # Larger FB on Ampere - bar2_size: 64MB - bar3_size: 64MB -gsp: - firmware_base: 0x04000000 - doorbell_offset: 0x110C00 - shared_mem_offset: 0x1000 - # Addresses discovered at runtime via FirmwareAnalyzer -capabilities: - max_vfs: 7 - supports_mig: true - supports_mig_mig: true # MIG (Multi-Instance GPU) - supports_confidential: false - supports_wpr: true - supports_bar_trap: true - msi_x_vectors: 64 -``` - -```yaml -# config/hardware/blackwell.yaml -architecture: "blackwell" -pci_device_ids: - - vendor: 0x10de - device: 0x2480 # GB202 (RTX 5090) - name: "RTX 5090" - - vendor: 0x10de - device: 0x2380 # GB100 (B200) - name: "B200" - - vendor: 0x10de - device: 0x2384 # GB205 - name: "RTX 5070" -bar_layout: - bar0_size: 16MB - bar1_size: 1GB # Blackwell unified memory - bar2_size: 128MB - bar3_size: 128MB -gsp: - firmware_base: 0x04000000 - doorbell_offset: 0x110C00 - shared_mem_offset: 0x1000 -capabilities: - max_vfs: 8 - supports_mig: true - supports_mig_mig: true - supports_confidential: true # TEE support - supports_wpr: true - supports_bar_trap: true - supports_nvlink: true - msi_x_vectors: 128 -``` - -### 2. Driver Version Configuration - -```yaml -# config/versions/550.yaml -version: "550" -rpc_interface_version: 1 -rpc_types: - - ALLOC: 0x01 - - FREE: 0x02 - - MAP: 0x03 - - UNMAP: 0x04 - - PIN: 0x05 - - UNPIN: 0x06 - - SCHEDULE: 0x07 - - YIELD: 0x08 - - NOTIFY: 0x09 - - BAR_TRAP: 0x0A - - MIGRATE: 0x0B - - DEBUG: 0x0C -struct_layouts: - ALLOC: - size: 48 - fields: - - {name: size, offset: 0, type: u64} - - {name: alignment, offset: 8, type: u64} - - {name: mem_type, offset: 16, type: u32} - - {name: flags, offset: 20, type: u32} - - {name: addr_hint, offset: 24, type: u64} - - {name: pid, offset: 32, type: u32} -capabilities: - supports_migrate: true - migrate_subcommands: - - SAVE: 0x04 - - RESTORE: 0x05 - supports_notifications: basic -``` - -```yaml -# config/versions/570.yaml -version: "570" -rpc_interface_version: 3 -extends: "550" -rpc_types: - - NOTIFY: 0x09 # Enhanced with subcommands - - MIGRATE: 0x0B # Enhanced with 4-phase -struct_layouts: - NOTIFY: - size: 32 - fields: - - {name: type, offset: 0, type: u32} - - {name: subcmd, offset: 4, type: u32} - - {name: event_fd, offset: 8, type: u64} - - {name: flags, offset: 16, type: u32} -capabilities: - supports_notifications: enhanced - supports_migrate_4phase: true - migrate_phases: [QUERY, PREPARE, TRANSFER, COMMIT, CLEANUP] -``` - -```yaml -# config/versions/580.yaml -version: "580" -rpc_interface_version: 4 -extends: "570" -rpc_types: - - TEE_ATTEST: 0x0D # New in 580 for confidential computing - - TEE_SEAL: 0x0E - - TEE_UNSEAL: 0x0F -capabilities: - supports_confidential: true - supports_tee: true - tee_attestation_types: [CPU, GPU, MEMORY] -``` - ---- - -## Runtime Detection & Adaptation Flow - -``` -┌─────────────────────────────────────────────────────────────────────────────────────┐ -│ MODULE LOAD: RUNTIME DETECTION FLOW │ -├─────────────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ 1. open-iov-vfio.ko loads │ -│ │ │ -│ ▼ │ -│ 2. Probe PCI device │ -│ │ │ -│ ▼ │ -│ 3. Match PCI Device ID → Hardware Config (turing.yaml, ampere.yaml, etc.) │ -│ │ │ -│ ▼ │ -│ 4. Load Hardware Config → Instantiate GpuArchitecture subclass │ -│ │ │ -│ ▼ │ -│ 5. Load GSP Firmware (from /lib/firmware or host nvidia.ko) │ -│ │ │ -│ ▼ │ -│ 6. FirmwareAnalyzer.analyze(firmware) │ -│ ├── Compute firmware SHA256 │ -│ ├── Find RPC dispatcher via pattern matching │ -│ ├── Find jump table via cross-references │ -│ ├── Extract handler addresses for all 12+ RPC types │ -│ ├── Extract struct layouts via pattern matching │ -│ └── Match firmware signature → DriverVersion (550, 560, 570, 580, etc.) │ -│ │ │ -│ ▼ │ -│ 7. Load Version Config → Instantiate RpcInterfaceVersion │ -│ │ │ -│ ▼ │ -│ 8. Validate: Hardware Config + Version Config compatible? │ -│ │ │ -│ ├── YES → Initialize DynamicRpcDispatcher with discovered addresses │ -│ ├── PARTIAL → Enable emulation for missing handlers │ -│ └── NO → Fail with clear error: "Unsupported GPU/driver combination" │ -│ │ │ -│ ▼ │ -│ 9. Register EmulationFallbackRegistry for all RPC types │ -│ │ │ -│ ▼ │ -│ 10. Start GSP Emulation Thread (per mdev) │ -│ │ -└─────────────────────────────────────────────────────────────────────────────────────┘ -``` - ---- - -## FirmwareAnalyzer - Automatic Address Discovery - -```c -// kernel/firmware_analyzer.c - -struct firmware_signature { - uint8_t sha256[32]; - char version_string[64]; - uint32_t crc32; -}; - -struct discovered_rpc_handlers { - uint64_t primary_dispatcher; - uint64_t secondary_dispatcher; - uint64_t jump_table; - struct { - uint32_t rpc_type; - uint64_t handler_address; - bool found; - } handlers[16]; // Support up to 16 RPC types - uint32_t handler_count; -}; - -struct firmware_analyzer { - const uint8_t *firmware; - size_t firmware_size; - uint64_t firmware_base; - struct discovered_rpc_handlers discovered; - struct firmware_signature signature; -}; - -int firmware_analyzer_init(struct firmware_analyzer *fa, const char *firmware_path) { - // Load firmware file - fa->firmware = load_firmware_file(firmware_path, &fa->firmware_size); - if (!fa->firmware) - return -ENOENT; - - // Compute signature - sha256(fa->firmware, fa->firmware_size, fa->signature.sha256); - fa->signature.crc32 = crc32(0, fa->firmware, fa->firmware_size); - - // Set base address (from hardware config) - fa->firmware_base = get_hardware_config()->gsp.firmware_base; - - return 0; -} - -int firmware_analyzer_discover(struct firmware_analyzer *fa) { - // 1. Find primary dispatcher: search for known pattern - // Pattern: "RPC_DISPATCH" string reference or known instruction sequence - fa->discovered.primary_dispatcher = find_pattern(fa->firmware, fa->firmware_size, - "RPC_DISPATCH", fa->firmware_base); - - // 2. Find jump table: cross-reference from dispatcher - fa->discovered.jump_table = find_jump_table(fa->firmware, fa->firmware_size, - fa->discovered.primary_dispatcher); - - // 3. Parse jump table entries (32 entries for 12 RPC types) - parse_jump_table(fa->firmware, fa->discovered.jump_table, &fa->discovered); - - // 4. Find secondary dispatcher - fa->discovered.secondary_dispatcher = find_secondary_dispatcher(fa->firmware, - fa->firmware_size, fa->discovered.primary_dispatcher); - - // 5. Find specialized handlers via string references - find_handler_by_string(fa, "BAR_TRAP", "BAR1", "ForceBarAccess", &fa->discovered.handlers[GSP_RPC_BAR_TRAP]); - find_handler_by_string(fa, "WPR", "completion", "timeout", &fa->discovered.handlers[GSP_RPC_NOTIFY]); - find_handler_by_string(fa, "ALLOC", "size", "alignment", &fa->discovered.handlers[GSP_RPC_ALLOC]); - find_handler_by_string(fa, "SCHEDULE", "work", "callback", &fa->discovered.handlers[GSP_RPC_SCHEDULE]); - find_handler_by_string(fa, "MIGRATE", "checkpoint", "progress", &fa->discovered.handlers[GSP_RPC_MIGRATE]); - - // 6. Extract struct layouts via pattern matching on handler prologues - extract_struct_layouts(fa); - - // 7. Match signature against known versions - match_firmware_version(fa); - - return 0; -} - -void firmware_analyzer_match_version(struct firmware_analyzer *fa) { - // Compare signature against known database - static const struct { - const char *version; - uint8_t sha256[32]; - uint32_t crc32; - } known_versions[] = { - {"550.163.01", {0x...}, 0x...}, - {"560.28.03", {0x...}, 0x...}, - {"570.144.00", {0x...}, 0x...}, - {"570.144.01", {0x...}, 0x...}, - {"580.00.00", {0x...}, 0x...}, - }; - - for (int i = 0; i < ARRAY_SIZE(known_versions); i++) { - if (memcmp(fa->signature.sha256, known_versions[i].sha256, 32) == 0) { - strncpy(fa->signature.version_string, known_versions[i].version, 63); - return; - } - } - - // Unknown version - use heuristic based on handler count and features - if (fa->discovered.handler_count >= 12) - strcpy(fa->signature.version_string, "570+ (unknown)"); - else if (fa->discovered.handler_count >= 10) - strcpy(fa->signature.version_string, "560+ (unknown)"); - else - strcpy(fa->signature.version_string, "550+ (unknown)"); -} -``` - ---- - -## Dynamic RPC Dispatcher - -```c -// kernel/dynamic_rpc_dispatcher.c - -struct dynamic_rpc_dispatcher { - struct discovered_rpc_handlers *discovered; - struct rpc_interface_version *interface; - struct emulation_fallback_registry *fallbacks; - spinlock_t lock; - uint64_t dispatch_stats[16]; -}; - -int dynamic_rpc_dispatcher_dispatch(struct dynamic_rpc_dispatcher *dr, - struct gsp_rpc_msg *req, - struct gsp_rpc_response *rsp) { - // 1. Validate RPC type - if (req->type >= dr->interface->rpc_type_count) - return -ENOTTY; - - // 2. Check if we have a native handler address - uint64_t handler_addr = 0; - for (int i = 0; i < dr->discovered->handler_count; i++) { - if (dr->discovered->handlers[i].rpc_type == req->type) { - handler_addr = dr->discovered->handlers[i].handler_address; - break; - } - } - - // 3. Dispatch to native handler if available - if (handler_addr) { - return dynamic_rpc_call_native(dr, handler_addr, req, rsp); - } - - // 4. Fallback to emulation - struct emulation_handler *emu = emulation_fallback_get(dr->fallbacks, req->type); - if (emu) { - dr->dispatch_stats[req->type]++; // Track emulation usage - return emu->handle(req, rsp); - } - - // 5. No handler, no emulation - rsp->status = -ENOTTY; - return -ENOTTY; -} - -int dynamic_rpc_call_native(struct dynamic_rpc_dispatcher *dr, - uint64_t handler_addr, - struct gsp_rpc_msg *req, - struct gsp_rpc_response *rsp) { - // Use firmware-decompiled calling convention - // This calls the actual GSP firmware handler via the discovered address - // In emulation mode, we call our C reimplementation instead - - if (gsp_emulation_mode_enabled()) { - // Call our C reimplementation (from gsp_rpc_client.c) - return gsp_rpc_client_call(req->type, req, rsp); - } else { - // Native mode: would call into GSP firmware - // Not used in our architecture (we always emulate in host) - return -ENOTSUPP; - } -} -``` - ---- - -## Capability Negotiation with Guest - -``` -┌─────────────────────────────────────────────────────────────────────────────────────┐ -│ GUEST DRIVER CAPABILITY NEGOTIATION │ -├─────────────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ Guest nvidia.ko probes vGPU: │ -│ 1. Reads PCI Config Space → gets device ID, BAR sizes, capabilities │ -│ 2. Maps BAR0 → finds GSP doorbell + shared memory │ -│ 3. Sends RPC: QUERY_CAPABILITIES (new RPC type or subcommand) │ -│ │ -│ Host DynamicRpcDispatcher receives QUERY_CAPABILITIES: │ -│ 1. Constructs response from: │ -│ - Hardware Config capabilities │ -│ - Version Config capabilities │ -│ - EmulationFallbackRegistry (what we can emulate) │ -│ 2. Returns capability bitmap to guest │ -│ │ -│ Guest nvidia.ko enables features based on response: │ -│ - If MIG supported → enables MIG code paths │ -│ - If WPR supported → enables WPR code paths │ -│ - If CONFIDENTIAL supported → enables TEE code paths │ -│ - If NOTIFY enhanced → uses async eventfd │ -│ │ -│ This allows SAME guest driver to work across ALL hardware/versions! │ -│ │ -└─────────────────────────────────────────────────────────────────────────────────────┘ -``` - ---- - -## Update Strategy for New Driver Versions - -### 1. Adding New Driver Version (e.g., 590) - -```bash -# 1. Add version config -cp config/versions/580.yaml config/versions/590.yaml -# Edit: version, rpc_interface_version, new RPC types, capabilities - -# 2. Test with new firmware -# FirmwareAnalyzer auto-discovers addresses - NO CODE CHANGES NEEDED - -# 3. If new RPC types added: -# - Add to config/versions/590.yaml -# - Add emulation handler in kernel/emulation_fallbacks.c -# - Register in EmulationFallbackRegistry - -# 4. If struct layouts changed: -# - FirmwareAnalyzer extracts new layouts automatically -# - Version config specifies layout version -``` - -### 2. Adding New GPU Architecture (e.g., Rubin R100) - -```bash -# 1. Add hardware config -cp config/hardware/blackwell.yaml config/hardware/rubin.yaml -# Edit: PCI device IDs, BAR layout, capabilities - -# 2. Add GpuArchitecture subclass (if special handling needed) -# kernel/arch/rubin_architecture.c - -# 3. Register in architecture factory -# kernel/architecture_factory.c - -# 4. Test - FirmwareAnalyzer handles address discovery -``` - -### 3. Automated Firmware Analysis Pipeline - -```yaml -# .github/workflows/firmware-analysis.yml -name: Firmware Analysis -on: - schedule: - - cron: '0 0 * * 0' # Weekly - workflow_dispatch: - inputs: - firmware_url: - description: 'NVIDIA firmware download URL' - required: true - -jobs: - analyze: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - name: Download firmware - run: wget -O firmware.bin "${{ inputs.firmware_url }}" - - name: Run FirmwareAnalyzer - run: | - ./tools/firmware_analyzer firmware.bin > analysis.json - - name: Compare with known versions - run: ./tools/compare_analysis.py analysis.json - - name: Generate config if new - run: | - if [ "$(cat analysis.json | jq -r '.is_new')" = "true" ]; then - ./tools/generate_config.py analysis.json > config/versions/NEW.yaml - git add config/versions/NEW.yaml - git commit -m "Add support for new firmware version" - git push - fi -``` - ---- - -## Testing Matrix for Portability - -| GPU Architecture | PCI Device IDs | Driver Versions | Test Status | -|------------------|----------------|-----------------|-------------| -| **Turing** | 10de:1e07, 10de:1e08, 10de:1ed3 | 550, 560, 570, 580 | ✅ Baseline | -| **Ampere** | 10de:20b0, 10de:2204, 10de:2206 | 550, 560, 570, 580 | 🔄 In Progress | -| **Hopper** | 10de:2330, 10de:2334 | 560, 570, 580 | 🔄 Planned | -| **Blackwell** | 10de:2480, 10de:2380, 10de:2384 | 570, 580 | 🔄 Planned | -| **Rubin (Future)** | TBD | 590+ | 📋 Designed | - -### Per-Combination Tests - -```bash -# Automated test matrix -for arch in turing ampere hopper blackwell; do - for version in 550 560 570 580; do - if [ -f "config/hardware/${arch}.yaml" ] && [ -f "config/versions/${version}.yaml" ]; then - echo "Testing ${arch} + ${version}..." - ./scripts/test_combination.sh ${arch} ${version} - fi - done -done -``` - ---- - -## Implementation File Structure (Portable) - -``` -open-iov/ -├── config/ -│ ├── hardware/ -│ │ ├── turing.yaml -│ │ ├── ampere.yaml -│ │ ├── hopper.yaml -│ │ ├── blackwell.yaml -│ │ └── rubin.yaml (future) -│ ├── versions/ -│ │ ├── 550.yaml -│ │ ├── 560.yaml -│ │ ├── 570.yaml -│ │ ├── 580.yaml -│ │ └── 590.yaml (future) -│ └── defaults.yaml -├── kernel/ -│ ├── architecture/ -│ │ ├── architecture.h # Abstract base class -│ │ ├── architecture_factory.c # Detection & instantiation -│ │ ├── turing_architecture.c -│ │ ├── ampere_architecture.c -│ │ ├── hopper_architecture.c -│ │ ├── blackwell_architecture.c -│ │ └── generic_architecture.c # Fallback -│ ├── version/ -│ │ ├── version.h # DriverVersion, RpcInterfaceVersion -│ │ ├── version_registry.c # Loads version configs -│ │ ├── interface_v1.c # 550 interface -│ │ ├── interface_v2.c # 560 interface -│ │ ├── interface_v3.c # 570 interface -│ │ └── interface_v4.c # 580 interface -│ ├── firmware/ -│ │ ├── firmware_analyzer.c # Auto-discovers addresses -│ │ ├── firmware_analyzer.h -│ │ ├── pattern_matcher.c # Binary pattern matching -│ │ └── struct_layout_extractor.c -│ ├── dispatch/ -│ │ ├── dynamic_rpc_dispatcher.c -│ │ ├── dynamic_rpc_dispatcher.h -│ │ ├── emulation_fallback_registry.c -│ │ └── emulation_fallback_registry.h -│ ├── gsp_emulation.c # Uses dynamic dispatcher -│ ├── main.c # Integrates all layers -│ └── Makefile -├── tools/ -│ ├── firmware_analyzer/ # Standalone analysis tool -│ ├── compare_analysis.py -│ ├── generate_config.py -│ └── test_combination.sh -└── docs/ - ├── PORTABLE_ARCHITECTURE.md - └── VERSION_COMPATIBILITY.md -``` - ---- - -## Key Design Principles Summary - -| Principle | Implementation | -|-----------|----------------| -| **No hardcoded addresses** | FirmwareAnalyzer discovers at runtime | -| **No hardcoded struct layouts** | Version config + FirmwareAnalyzer extraction | -| **No hardcoded device IDs** | Hardware config YAML files | -| **No hardcoded capabilities** | Capability negotiation via QUERY_CAPABILITIES RPC | -| **Graceful degradation** | EmulationFallbackRegistry for missing handlers | -| **Forward compatibility** | Version config `extends` mechanism | -| **Automated updates** | Firmware analysis pipeline + config generation | -| **Test matrix** | Automated CI across architecture × version combinations | - ---- - -*Document Version: 1.0* -*Repository: $OPEN_IOV_ROOT* -*Branch: refactor/host-gsp-emulation* -*Base Commit: f9ed5d8*
\ No newline at end of file |
