Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 456 insertions, 0 deletions

diff --git a/firmware/57X-GSP/570.144/API_REFERENCE.md b/firmware/57X-GSP/570.144/API_REFERENCE.md
new file mode 100644
index 0000000..fbfbce2
--- /dev/null
+++ b/firmware/57X-GSP/570.144/API_REFERENCE.md
@@ -0,0 +1,456 @@
+# GSP 570.144W RPC Client API Reference
+
+## Overview
+
+This document describes the kernel-space RPC client API for communicating with the NVIDIA GSP (GPU System Processor) firmware version 570.144W (TU102/TU104). The API is derived from full firmware decompilation and provides direct access to all 12 RPC types and their sub-commands.
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Kernel Driver                            │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │            gsp_rpc_client.c/h                        │   │
+│  │  - Message encoding/decoding                         │   │
+│  │  - RPC dispatch via doorbell                         │   │
+│  │  - Response handling                                 │   │
+│  └─────────────────────────────────────────────────────┘   │
+│                              │                              │
+│                              ▼                              │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │            gsp_rpc_integration.c/h                   │   │
+│  │  - Integration with existing gsp_comm.c              │   │
+│  │  - Enhanced RPC operations with firmware addresses   │   │
+│  │  - WPR management                                    │   │
+│  └─────────────────────────────────────────────────────┘   │
+│                              │                              │
+│                              ▼                              │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │            gsp_comm.c (existing)                     │   │
+│  │  - Firmware loading/boot                             │   │
+│  │  - Doorbell/shared memory management                 │   │
+│  │  - IRQ handling                                      │   │
+│  └─────────────────────────────────────────────────────┘   │
+└─────────────────────────────────────────────────────────────┘
+                              │
+                    PCIe BAR0 Doorbell
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────┐
+│                    GSP Firmware (570.144W)                  │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │  Primary Dispatcher (0x057a1ac0)                    │   │
+│  │  - Jump table for 6 RPC types                       │   │
+│  │  - 32 ALLOC sub-commands at 0x057a3ad8              │   │
+│  └─────────────────────────────────────────────────────┘   │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │  Secondary Dispatcher (0x04ed39dc)                  │   │
+│  │  - Full implementation for all 11 RPC types         │   │
+│  └─────────────────────────────────────────────────────┘   │
+│  ┌─────────────────────────────────────────────────────┐   │
+│  │  Specialized Handlers                                │   │
+│  │  - ALLOC: 0x057e94ec                                │   │
+│  │  - SCHEDULE: 0x051efbf4                             │   │
+│  │  - BAR_TRAP: 0x057675b7 / 0x04cb8189                │   │
+│  │  - MAP/PIN/UNPIN/SCHEDULE/YIELD/NOTIFY (jump table) │   │
+│  └─────────────────────────────────────────────────────┘   │
+└─────────────────────────────────────────────────────────────┘
+```
+
+## Header Files
+
+### `include/gsp_rpc_client.h`
+Core RPC client API with all firmware addresses, message structures, and function prototypes.
+
+### `include/gsp_rpc_integration.h`
+Integration layer API that bridges the new RPC client with existing `gsp_comm.c`.
+
+## RPC Types
+
+| Type | Value | Description |
+|------|-------|-------------|
+| `GSP_RPC_ALLOC` | 0x01 | Memory allocation |
+| `GSP_RPC_FREE` | 0x02 | Memory free |
+| `GSP_RPC_MAP` | 0x03 | Map GPU VA to physical |
+| `GSP_RPC_UNMAP` | 0x04 | Unmap GPU VA |
+| `GSP_RPC_PIN` | 0x05 | Pin pages for DMA |
+| `GSP_RPC_UNPIN` | 0x06 | Unpin pages |
+| `GSP_RPC_SCHEDULE` | 0x07 | Schedule work |
+| `GSP_RPC_YIELD` | 0x08 | Yield current work |
+| `GSP_RPC_NOTIFY` | 0x09 | Notification/event |
+| `GSP_RPC_BAR_TRAP` | 0x0A | BAR register trap emulation |
+| `GSP_RPC_MIGRATE` | 0x0B | Live migration |
+| `GSP_RPC_DEBUG` | 0xFF | Debug/diagnostic |
+
+## Core API Functions
+
+### Client Lifecycle
+
+```c
+struct gsp_client *gsp_client_init(struct device *dev, phys_addr_t fw_base, phys_addr_t doorbell);
+void gsp_client_fini(struct gsp_client *client);
+```
+
+### Memory Management
+
+```c
+int gsp_rpc_alloc(struct gsp_client *client,
+                   u64 size, u64 alignment, u32 mem_type,
+                   u32 flags, u64 addr_hint, u32 pid,
+                   u64 *gpu_va, u64 *handle, u64 *actual_size);
+
+int gsp_rpc_free(struct gsp_client *client, u64 handle);
+
+int gsp_rpc_map(struct gsp_client *client,
+                 u64 handle, u64 gpu_va, u64 size,
+                 u32 prot, u32 flags, u64 offset,
+                 u64 *mapped_va);
+
+int gsp_rpc_unmap(struct gsp_client *client, u64 handle, u64 gpu_va, u64 size);
+```
+
+### Page Pinning
+
+```c
+int gsp_rpc_pin(struct gsp_client *client,
+                 u64 handle, u64 offset, u64 size,
+                 u32 flags, u64 *dma_addr, u32 *num_pages);
+
+int gsp_rpc_unpin(struct gsp_client *client, u64 handle, u64 offset, u64 size);
+```
+
+### Scheduling
+
+```c
+int gsp_rpc_schedule(struct gsp_client *client,
+                      u64 work_handle, u64 callback, u64 arg,
+                      u32 priority, u32 flags);
+
+int gsp_rpc_yield(struct gsp_client *client);
+```
+
+### BAR Trap Emulation
+
+```c
+int gsp_rpc_bar_trap(struct gsp_client *client,
+                      u64 bar_addr, u32 access_type, u32 size,
+                      u64 value, u32 bar_num, u64 *ret_value);
+```
+
+### Migration
+
+```c
+int gsp_rpc_migrate_save(struct gsp_client *client,
+                          u64 src_gpu_id, u64 dst_gpu_id,
+                          u64 memory_bitmap, u64 page_table_ptr,
+                          u64 *checkpoint);
+
+int gsp_rpc_migrate_restore(struct gsp_client *client,
+                             u64 src_gpu_id, u64 dst_gpu_id,
+                             u64 memory_bitmap, u64 page_table_ptr,
+                             u64 checkpoint);
+```
+
+### WPR Management
+
+```c
+int gsp_wpr_write_mutex_acquire(struct gsp_client *client, u32 wpr_id);
+int gsp_wpr_write_mutex_release(struct gsp_client *client, u32 wpr_id);
+int gsp_wpr_scratch_mutex_acquire(struct gsp_client *client, u32 wpr_id);
+int gsp_wpr_scratch_mutex_release(struct gsp_client *client, u32 wpr_id);
+```
+
+## Integration API
+
+### Initialization/Cleanup
+
+```c
+int nv_vgpu_gsp_rpc_integration_init(struct nv_vgpu_device *vgpu,
+                                      struct nv_vgpu_gsp_comm_ctx *comm_ctx);
+
+void nv_vgpu_gsp_rpc_integration_cleanup(struct nv_vgpu_device *vgpu);
+```
+
+### Enhanced RPC Operations
+
+All functions follow the pattern `nv_vgpu_gsp_rpc_<op>_fw()` and provide full firmware parameter support:
+
+```c
+int nv_vgpu_gsp_rpc_alloc_fw(struct nv_vgpu_device *vgpu,
+                              u64 size, u64 alignment, u32 mem_type,
+                              u32 flags, u64 addr_hint, u32 pid,
+                              u64 *gpu_va, u64 *handle, u64 *actual_size);
+
+int nv_vgpu_gsp_rpc_map_fw(struct nv_vgpu_device *vgpu,
+                            u64 handle, u64 gpu_va, u64 size,
+                            u32 prot, u32 flags, u64 offset,
+                            u64 *mapped_va);
+
+int nv_vgpu_gsp_rpc_pin_fw(struct nv_vgpu_device *vgpu,
+                            u64 handle, u64 offset, u64 size,
+                            u32 flags, u64 *dma_addr, u32 *num_pages);
+
+int nv_vgpu_gsp_rpc_schedule_fw(struct nv_vgpu_device *vgpu,
+                                 u64 work_handle, u64 callback, u64 arg,
+                                 u32 priority, u32 flags);
+
+int nv_vgpu_gsp_rpc_bar_trap_fw(struct nv_vgpu_device *vgpu,
+                                 u64 bar_addr, u32 access_type, u32 size,
+                                 u64 value, u32 bar_num, u64 *ret_value);
+
+int nv_vgpu_gsp_rpc_migrate_save_fw(struct nv_vgpu_device *vgpu,
+                                     u64 src_gpu_id, u64 dst_gpu_id,
+                                     u64 memory_bitmap, u64 page_table_ptr,
+                                     u64 *checkpoint);
+
+int nv_vgpu_gsp_rpc_migrate_restore_fw(struct nv_vgpu_device *vgpu,
+                                        u64 src_gpu_id, u64 dst_gpu_id,
+                                        u64 memory_bitmap, u64 page_table_ptr,
+                                        u64 checkpoint);
+```
+
+### WPR Management
+
+```c
+int nv_vgpu_gsp_wpr_write_mutex_acquire(struct nv_vgpu_device *vgpu, u32 wpr_id);
+int nv_vgpu_gsp_wpr_write_mutex_release(struct nv_vgpu_device *vgpu, u32 wpr_id);
+int nv_vgpu_gsp_wpr_scratch_mutex_acquire(struct nv_vgpu_device *vgpu, u32 wpr_id);
+int nv_vgpu_gsp_wpr_scratch_mutex_release(struct nv_vgpu_device *vgpu, u32 wpr_id);
+```
+
+### Firmware Address Accessors
+
+```c
+u64 nv_vgpu_gsp_get_primary_dispatcher(void);       // 0x057a1ac0
+u64 nv_vgpu_gsp_get_secondary_dispatcher(void);     // 0x04ed39dc
+u64 nv_vgpu_gsp_get_alloc_main(void);               // 0x057e94ec
+u64 nv_vgpu_gsp_get_schedule_main(void);            // 0x051efbf4
+u64 nv_vgpu_gsp_get_bar_trap_handler_jt(void);      // 0x057675b7
+u64 nv_vgpu_gsp_get_bar_trap_handler_primary(void); // 0x04cb8189
+u64 nv_vgpu_gsp_get_entry_point(void);              // 0x04009000
+```
+
+## Message Structures
+
+All structures use little-endian encoding (`__le32`, `__le64`) for cross-architecture compatibility.
+
+### `struct gsp_rpc_msg` (64 bytes)
+```c
+struct gsp_rpc_msg {
+    __le32 type;           // RPC type (0x01-0x0B, 0xFF)
+    __le32 subcmd;         // Sub-command (type-specific)
+    __le64 payload[6];     // 48-byte payload for request data
+    __le32 seq_id;         // Sequence ID for matching responses
+    __le32 flags;          // Message flags
+    __le32 reserved[2];    // Reserved for future use
+} __packed;
+```
+
+### `struct gsp_alloc_req` (48 bytes)
+```c
+struct gsp_alloc_req {
+    __le64 size;           // Allocation size in bytes
+    __le64 alignment;      // Required alignment
+    __le32 mem_type;       // Memory type (VRAM, sysmem, etc.)
+    __le32 flags;          // Allocation flags
+    __le64 addr_hint;      // Preferred address hint
+    __le32 pid;            // Process ID
+    __le32 reserved;       // Reserved
+} __packed;
+```
+
+### `struct gsp_map_req` (56 bytes)
+```c
+struct gsp_map_req {
+    __le64 handle;         // Allocation handle from ALLOC
+    __le64 gpu_va;         // Target GPU virtual address
+    __le64 size;           // Mapping size
+    __le32 prot;           // Protection (R/W/X)
+    __le32 flags;          // Mapping flags
+    __le64 offset;         // Offset into allocation
+    __le32 reserved[2];    // Reserved
+} __packed;
+```
+
+### `struct gsp_pin_req` (48 bytes)
+```c
+struct gsp_pin_req {
+    __le64 handle;         // Allocation handle
+    __le64 offset;         // Offset into allocation
+    __le64 size;           // Size to pin
+    __le32 flags;          // Pin flags
+    __le32 reserved[3];    // Reserved
+} __packed;
+```
+
+### `struct gsp_bar_trap_req` (40 bytes)
+```c
+struct gsp_bar_trap_req {
+    __le64 bar_addr;       // BAR register address
+    __le32 access_type;    // 0=read, 1=write
+    __le32 size;           // Access size (1/2/4/8)
+    __le64 value;          // Value for write operations
+    __le32 bar_num;        // BAR number (0-5)
+    __le32 reserved;       // Reserved
+} __packed;
+```
+
+### `struct gsp_migrate_req` (56 bytes)
+```c
+struct gsp_migrate_req {
+    __le64 src_gpu_id;     // Source GPU ID
+    __le64 dst_gpu_id;     // Destination GPU ID
+    __le64 memory_bitmap;  // Pages to migrate
+    __le64 page_table_ptr; // Page table pointer
+    __le32 state;          // Migration state (save/restore)
+    __le32 flags;          // Migration flags
+    __le64 context;        // Checkpoint context
+    __le32 reserved;       // Reserved
+} __packed;
+```
+
+## ALLOC Sub-Commands (Primary Dispatcher Jump Table)
+
+The primary dispatcher at `0x057a1ac0` uses a jump table at `0x057a3ad8` with 32 entries for ALLOC sub-commands:
+
+| Index | Constant | Description |
+|-------|----------|-------------|
+| 0 | `GSP_ALLOC_SUBCMD_DEFAULT` | Default allocation |
+| 1 | `GSP_ALLOC_SUBCMD_TYPE1` | Type 1 allocation |
+| 3 | `GSP_ALLOC_SUBCMD_TYPE2` | Type 2 allocation |
+| 4 | `GSP_ALLOC_SUBCMD_TYPE3` | Type 3 allocation |
+| 5 | `GSP_ALLOC_SUBCMD_TYPE4` | Type 4 allocation |
+| 8 | `GSP_ALLOC_SUBCMD_TYPE5` | Type 5 allocation |
+| 10 | `GSP_ALLOC_SUBCMD_TYPE6` | Type 6 allocation |
+| 12 | `GSP_ALLOC_SUBCMD_TYPE7` | Type 7 allocation |
+| 17 | `GSP_ALLOC_SUBCMD_TYPE8` | Type 8 allocation |
+| 19 | `GSP_ALLOC_SUBCMD_TYPE9` | Type 9 allocation |
+| 20 | `GSP_ALLOC_SUBCMD_TYPE10` | Type 10 allocation |
+| 21 | `GSP_ALLOC_SUBCMD_TYPE11` | Type 11 allocation |
+| 22 | `GSP_ALLOC_SUBCMD_TYPE12` | Type 12 allocation |
+| 23 | `GSP_ALLOC_SUBCMD_TYPE13` | Type 13 allocation |
+| 24 | `GSP_ALLOC_SUBCMD_TYPE14` | Type 14 allocation |
+| 25 | `GSP_ALLOC_SUBCMD_TYPE15` | Type 15 allocation |
+| 26 | `GSP_ALLOC_SUBCMD_TYPE16` | Type 16 allocation |
+| 27 | `GSP_ALLOC_SUBCMD_TYPE17` | Type 17 allocation |
+| 28 | `GSP_ALLOC_SUBCMD_TYPE18` | Type 18 allocation |
+| 29 | `GSP_ALLOC_SUBCMD_TYPE19` | Type 19 allocation |
+| 30 | `GSP_ALLOC_SUBCMD_TYPE20` | Type 20 allocation |
+| 31 | `GSP_ALLOC_SUBCMD_TYPE21` | Type 21 allocation |
+
+## MIGRATE Sub-Commands
+
+| Value | Constant | Description |
+|-------|----------|-------------|
+| 0x04 | `GSP_MIGRATE_SUBCMD_SAVE` | Save migration state |
+| 0x05 | `GSP_MIGRATE_SUBCMD_RESTORE` | Restore migration state |
+
+## Firmware Address Reference
+
+All addresses are relative to `GSP_FW_BASE = 0x04000000`.
+
+| Symbol | Address | Description |
+|--------|---------|-------------|
+| `GSP_RPC_DISPATCH_PRIMARY` | 0x057a1ac0 | Primary dispatcher (jump table) |
+| `GSP_RPC_JUMP_TABLE` | 0x057a3ad8 | 32-entry ALLOC sub-command table |
+| `GSP_RPC_DISPATCH_SECONDARY` | 0x04ed39dc | Secondary dispatcher (full 11 types) |
+| `GSP_RPC_DISPATCH_TERTIARY` | 0x04cc58f8 | Tertiary dispatcher (stack=160) |
+| `GSP_RPC_DISPATCH_QUATERNARY` | 0x05723ef4 | Quaternary dispatcher (stack=96) |
+| `GSP_ALLOC_MAIN` | 0x057e94ec | Main ALLOC implementation |
+| `GSP_SCHEDULE_MAIN` | 0x051efbf4 | Main SCHEDULE implementation |
+| `GSP_BAR_TRAP_HANDLER_JT` | 0x057675b7 | BAR trap handler (jump table) |
+| `GSP_BAR_TRAP_HANDLER_PRIM` | 0x04cb8189 | BAR trap handler (primary) |
+| `GSP_PIN_HANDLER_SEC` | 0x04ed34b8 | PIN handler (secondary) |
+| `GSP_MAP_HANDLER_A` | 0x05766ab7 | MAP handler A |
+| `GSP_MAP_HANDLER_B` | 0x05778a37 | MAP handler B |
+| `GSP_UNMAP_HANDLER_A` | 0x057669b7 | UNMAP handler A |
+| `GSP_UNMAP_HANDLER_B` | 0x05779937 | UNMAP handler B |
+| `GSP_PIN_HANDLER_A` | 0x057664b7 | PIN handler A |
+| `GSP_PIN_HANDLER_B` | 0x0577a3b7 | PIN handler B |
+| `GSP_UNPIN_HANDLER_A` | 0x057682b7 | UNPIN handler A |
+| `GSP_UNPIN_HANDLER_B` | 0x05779fb7 | UNPIN handler B |
+| `GSP_SCHEDULE_HANDLER_A` | 0x05767f37 | SCHEDULE handler A |
+| `GSP_SCHEDULE_HANDLER_B` | 0x0575aeb7 | SCHEDULE handler B |
+| `GSP_YIELD_HANDLER_A` | 0x0575ae37 | YIELD handler A |
+| `GSP_YIELD_HANDLER_B` | 0x05767337 | YIELD handler B |
+| `GSP_NOTIFY_HANDLER_A` | 0x0577c8b7 | NOTIFY handler A |
+| `GSP_NOTIFY_HANDLER_B` | 0x05767837 | NOTIFY handler B |
+| `GSP_ENTRY_POINT` | 0x04009000 | Firmware entry point |
+
+## WPR Mutex Strings (from Firmware)
+
+The firmware contains 60 references to WPR mutex strings:
+- `VPR_WPR_WRITE_MUTEX_ACQUIRE`
+- `VPR_WPR_WRITE_MUTEX_RELEASE`
+- `VPR_WPR_SCRATCH_MUTEX_ACQUIRE`
+- `VPR_WPR_SCRATCH_MUTEX_RELEASE`
+- `ACQUIRE_VPR_WPR_WRITE_MUTEX`
+- `RELEASE_VPR_WPR_WRITE_MUTEX`
+
+These are used by functions in the `0x0407xxxx` region for WPR (Write-Protected Region) management.
+
+## BAR Emulation Strings (from Firmware)
+
+28 references to BAR-related strings:
+- `BAR1`, `Bar1Rest`, `ForceBarAccess`
+- `Bar1Mapping`, `Bar1Aperture`, `Bar1Size`
+- `BAR0`, `BAR2`, `BAR3`
+
+## Building
+
+```bash
+# Build main module with RPC client
+cd $OPEN_IOV_ROOT/kernel
+make
+
+# Build test module
+cd tests
+make
+```
+
+## Testing
+
+Load the test module to validate message encoding/decoding:
+
+```bash
+insmod tests/gsp_rpc_test.ko
+dmesg | grep -i "gsp rpc test"
+```
+
+Expected output:
+```
+GSP RPC Client Test Suite Starting
+=== Testing Structure Sizes ===
+PASS: gsp_rpc_msg size
+PASS: gsp_rpc_response size
+...
+=== Testing Firmware Address Constants ===
+PASS: GSP_FW_BASE
+PASS: Primary Dispatcher
+...
+Test Results: 85 total, 85 passed, 0 failed
+TEST SUITE PASSED
+```
+
+## Integration with Existing Code
+
+The integration layer (`gsp_rpc_integration.c`) connects to existing `gsp_comm.c`:
+
+1. Call `nv_vgpu_gsp_rpc_integration_init(vgpu, comm_ctx)` during `nv_vgpu_gsp_init()`
+2. Use `nv_vgpu_gsp_rpc_*_fw()` functions for enhanced RPC operations
+3. Call `nv_vgpu_gsp_rpc_integration_cleanup(vgpu)` during `nv_vgpu_gsp_cleanup()`
+
+## Error Codes
+
+All functions return standard Linux kernel error codes:
+- `0` - Success
+- `-EINVAL` - Invalid parameter
+- `-ENOMEM` - Out of memory
+- `-ENODEV` - Device not available
+- `-ETIMEDOUT` - RPC timeout
+- `-EIO` - I/O error
+- `-ENOTSUPP` - Operation not supported
+
+## License
+
+GPL v2 - Open-IOV Project