Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 204 insertions, 0 deletions

diff --git a/firmware/57X-GSP/570.144/IMPLEMENTATION_MAP.md b/firmware/57X-GSP/570.144/IMPLEMENTATION_MAP.md
new file mode 100644
index 0000000..316a7b5
--- /dev/null
+++ b/firmware/57X-GSP/570.144/IMPLEMENTATION_MAP.md
@@ -0,0 +1,204 @@
+# GSP 570.144W - Implementation Map for vGPU-IO Integration
+
+## Complete Function Map
+
+### 1. Entry Point & Initialization
+| Address | Function | Description |
+|---------|----------|-------------|
+| 0x04009000 | `gsp_entry` | Bootstrap entry, self-loops at 0x04009070 |
+
+### 2. RPC Dispatcher Hierarchy
+
+#### Primary Dispatcher (Top-Level RPC Type Switch)
+| Address | Name | Type | Stack | Notes |
+|---------|------|------|-------|-------|
+| 0x057a1ac0 | `rpc_dispatch_primary` | Jump table | None | 224 callers |
+| 0x057a3ad8 | `primary_jump_table` | 32 entries | - | Index by sub-command |
+
+**RPC Types Handled**: ALLOC(0x01), UNMAP(0x04), PIN(0x05), MIGRATE(0x0B), FREE(0x02), SCHEDULE(0x07)
+
+#### Secondary Dispatcher (Full RPC Implementation)
+| Address | Name | Type | Stack | Callers |
+|---------|------|------|-------|---------|
+| 0x04ed39dc | `rpc_dispatch_secondary` | Full func | Dynamic | 139 |
+
+**RPC Types Handled**: ALL 11 types (MAP, ALLOC, YIELD, SCHEDULE, UNPIN, PIN, UNMAP, FREE, MIGRATE, BAR_TRAP, NOTIFY)
+
+**Handler Entry Points (from branch targets):**
+| RPC Type | Handler Address | Function |
+|----------|----------------|----------|
+| PIN (0x05) | 0x04ed34b8 | `rpc_pin_handler` |
+| SCHEDULE (0x07) | 0x04ed4364 | `rpc_schedule_handler_1` |
+| SCHEDULE (0x07) | 0x04ed45ac | `rpc_schedule_handler_2` |
+| PIN/ALLOC | 0x04ed4764 | `rpc_pin_alloc_handler` |
+| ALLOC (0x01) | 0x04ed4930 | `rpc_alloc_handler_1` |
+| ALLOC (0x01) | 0x04ed4984 | `rpc_alloc_handler_2` |
+| ALLOC (0x01) | 0x04ed49f8 | `rpc_alloc_handler_3` |
+| ALLOC/FREE | 0x04ed4a64 | `rpc_alloc_free_handler` |
+
+#### Tertiary Dispatcher
+| Address | Name | Stack | Callers |
+|---------|------|-------|---------|
+| 0x04cc58f8 | `rpc_dispatch_tertiary` | 160 bytes | 53 |
+
+#### Quaternary Dispatcher
+| Address | Name | Stack | Callers |
+|---------|------|-------|---------|
+| 0x05723ef4 | `rpc_dispatch_quaternary` | 96 bytes | 53 |
+
+### 3. Specialized RPC Handlers
+
+| Address | Name | Primary RPC | Calls | Stack |
+|---------|------|-------------|-------|-------|
+| 0x057e94ec | `rpc_alloc_main` | ALLOC | 64 | - |
+| 0x04d00a18 | `rpc_alloc_alt` | ALLOC | - | - |
+| 0x051efbf4 | `rpc_schedule_main` | SCHEDULE | 81 | 48 |
+| 0x04cb8189 | `rpc_bar_trap_handler` | BAR_TRAP | 4 | - |
+
+### 4. BAR Emulation
+| Address | Name | Description |
+|---------|------|-------------|
+| 0x059be7f4 | `bar_jump_table` | 12-entry encoded table (12 RPC types) |
+| 0x04cb8189 | `bar_trap_handler` | BAR register trap handler |
+| 0x04bc1665 | `bar_trap_subfunc` | Called from BAR trap handler |
+
+**BAR Strings**: BAR1, Bar1Rest, ForceBarAccess, Bar1Mapping, Bar1Aperture, Bar1Size
+
+### 5. WPR (Write Protected Region) Management
+**String References (60 locations):**
+- WPR_WRITE_COMPLETE
+- WPR_READ_COMPLETE
+- WPR_WRITE_TIMEOUT
+- WPR_READ_TIMEOUT
+- NV_WPR_WRITE_COMPLETE
+- NV_WPR_READ_COMPLETE
+- NV_WPR_WRITE_TIMEOUT
+- NV_WPR_READ_TIMEOUT
+
+### 6. Migration Protocol
+**RPC Type**: MIGRATE (0x0B)
+**References in dispatchers**: Offsets 280, 316, 324, 448, 656, etc.
+
+### 7. Calling Convention (Decoded)
+```assembly
+# PROLOGUE
+addi sp, sp, -N              # N = 32-560 (typically 96, 160, 560)
+sd s1, offset(sp)            # Save callee-saved s1-s11
+sd s2, offset(sp)
+...
+addi s0, sp, N               # Frame pointer (s0 = sp + N)
+mv sX, ra                    # Save return address to s1/s4/s10
+
+# EPILOGUE
+mv ra, sX                    # Restore return address
+ld s1, offset(sp)            # Restore callee-saved
+ld s2, offset(sp)
+...
+addi sp, sp, N               # Deallocate stack
+ret                          # jalr x0, 0(ra)
+```
+
+### 8. RPC Protocol (from gsp_comm.h)
+| Value | Name | Handler Location |
+|-------|------|------------------|
+| 0x01 | ALLOC | Primary/Secondary/Tertiary/Quaternary + 0x057e94ec |
+| 0x02 | FREE | Primary/Secondary/Tertiary/Quaternary |
+| 0x03 | MAP | Secondary/Tertiary/Quaternary |
+| 0x04 | UNMAP | Primary/Secondary/Tertiary/Quaternary |
+| 0x05 | PIN | Primary/Secondary/Tertiary/Quaternary + 0x04ed34b8 |
+| 0x06 | UNPIN | Secondary/Tertiary/Quaternary |
+| 0x07 | SCHEDULE | Primary/Secondary/Tertiary/Quaternary + 0x051efbf4 |
+| 0x08 | YIELD | Secondary/Tertiary/Quaternary |
+| 0x09 | NOTIFY | Secondary/Tertiary/Quaternary |
+| 0x0A | BAR_TRAP | Secondary + 0x04cb8189 (via 0x059be7f4) |
+| 0x0B | MIGRATE | Primary/Secondary/Tertiary/Quaternary |
+| 0xFF | DEBUG | Tertiary |
+
+### 9. Key Data Structures (from migration.h)
+```c
+// Migration state structure (referenced by RPC_MIGRATE handlers)
+struct gsp_migration_state {
+    uint64_t src_gpu_id;
+    uint64_t dst_gpu_id;
+    uint64_t memory_bitmap;
+    uint64_t page_table_ptr;
+    uint32_t state;
+    uint32_t flags;
+};
+```
+
+### 10. Integration Points for vGPU-IO Kernel
+
+#### gsp_comm.c - RPC Communication
+```c
+// RPC message structure (matches firmware expectations)
+struct gsp_rpc_msg {
+    uint32_t type;      // 0x01-0x0B, 0xFF
+    uint32_t subcmd;    // Sub-command for ALLOC, etc.
+    uint64_t payload[6]; // Variable payload
+    uint32_t seq_id;
+    uint32_t flags;
+};
+
+// Primary entry: gsp_rpc_dispatch(msg) -> calls firmware at 0x057a1ac0
+```
+
+#### bar_emulation.c - BAR Trapping
+```c
+// BAR trap handler registration
+// Firmware handler at 0x04cb8189 (from jump table 0x059be7f4)
+// Called when guest accesses emulated BAR registers
+void gsp_bar_trap_handler(uint64_t bar_addr, uint32_t access_type, uint64_t value);
+```
+
+#### migration.h - Migration Protocol
+```c
+// Migration RPC handlers in firmware:
+// - MIGRATE (0x0B) at primary dispatcher
+// - Save/restore GPU state
+// - Transfer page tables and memory bitmaps
+```
+
+## Implementation Priority for vGPU-IO
+
+### Phase 1: Core RPC Dispatch (Week 1)
+1. Implement `gsp_rpc_dispatch()` calling primary dispatcher at 0x057a1ac0
+2. Handle ALLOC/FREE/MAP/UNMAP for memory management
+3. Integrate with kernel page allocator
+
+### Phase 2: BAR Emulation (Week 2)
+1. Register BAR trap handler at 0x04cb8189
+2. Implement `ForceBarAccess` emulation
+3. Handle Bar1Rest/Bar1Mapping for GPU reset
+
+### Phase 3: WPR Management (Week 2)
+1. Implement WPR_WRITE/READ_COMPLETE handlers
+2. Add timeout handling for WPR operations
+
+### Phase 4: Migration (Week 3)
+1. Implement MIGRATE (0x0B) RPC handler
+2. State save/restore for live migration
+3. Page table transfer protocol
+
+### Phase 5: Scheduling & Pinning (Week 3)
+1. SCHEDULE/YIELD for vGPU time-slicing
+2. PIN/UNPIN for page locking during DMA
+
+## Files for Reference
+```
+$OPEN_IOV_ROOT/firmware/570-GSP/
+├── FINAL_REPORT.md              # Complete analysis report
+├── IMPLEMENTATION_MAP.md        # This file
+├── gsp-570.144.fwimage.bin      # Raw firmware (28.5 MB)
+├── all_call_targets.txt         # 134K call targets
+├── analysis_results.json        # Top 500 functions
+├── decompiled_*.c (10 files)    # Pseudo-C decompilation
+└── analyze_firmware.py          # Analysis tools
+```
+
+## Next Steps
+1. **Decompile secondary dispatcher handlers** (0x04ed34b8, 0x04ed4364, etc.) to pseudo-C
+2. **Map jump table at 0x059be7f4** - decode encoded pointers to actual handler addresses
+3. **Trace WPR string references** to find WPR management functions
+4. **Extract migration protocol** from RPC_MIGRATE handlers
+5. **Build kernel integration** using the function addresses above