Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 292 insertions, 0 deletions

diff --git a/firmware/57X-GSP/570.144/INTEGRATION_GUIDE.md b/firmware/57X-GSP/570.144/INTEGRATION_GUIDE.md
new file mode 100644
index 0000000..8ca495f
--- /dev/null
+++ b/firmware/57X-GSP/570.144/INTEGRATION_GUIDE.md
@@ -0,0 +1,292 @@
+# GSP 570.144W RPC Client Integration Guide
+
+## Overview
+
+This guide describes how to integrate the firmware-decompiled GSP RPC client into the vGPU-IO kernel driver. The integration provides access to all 12 RPC types with full firmware parameter support, using addresses extracted from the 28.5MB GSP firmware binary.
+
+## Prerequisites
+
+- Linux kernel 5.10+
+- NVIDIA GPU (TU102/TU104) with GSP firmware 570.144W
+- Existing vGPU-IO kernel driver (`nvidia-vgpu-vfio.ko`)
+- Kernel headers for current kernel
+
+## Files Added
+
+### Core RPC Client
+| File | Purpose |
+|------|---------|
+| `kernel/gsp_rpc_client.c` | Core RPC client implementation |
+| `kernel/include/gsp_rpc_client.h` | Core API header with all firmware addresses |
+
+### Integration Layer
+| File | Purpose |
+|------|---------|
+| `kernel/gsp_rpc_integration.c` | Bridge to existing `gsp_comm.c` |
+| `kernel/include/gsp_rpc_integration.h` | Integration API header |
+
+### Testing
+| File | Purpose |
+|------|---------|
+| `kernel/tests/gsp_rpc_test.c` | Validation test module |
+| `kernel/tests/Makefile` | Test build configuration |
+
+### Documentation
+| File | Purpose |
+|------|---------|
+| `firmware/570-GSP/API_REFERENCE.md` | Complete API documentation |
+| `firmware/570-GSP/INTEGRATION_GUIDE.md` | This file |
+
+## Build Instructions
+
+### 1. Build Main Module
+
+```bash
+cd $OPEN_IOV_ROOT/kernel
+make clean
+make
+```
+
+This builds `nvidia-vgpu-vfio.ko` with the new RPC client integrated.
+
+### 2. Build Test Module
+
+```bash
+cd $OPEN_IOV_ROOT/kernel/tests
+make clean
+make
+```
+
+This builds `gsp_rpc_test.ko` for validation.
+
+### 3. Install
+
+```bash
+# Install main module
+cd $OPEN_IOV_ROOT/kernel
+sudo make install
+
+# Install test module
+cd tests
+sudo make install
+```
+
+## Integration Steps
+
+### Step 1: Update gsp_comm.c to Use Integration Layer
+
+In `kernel/gsp_comm.c`, add the integration initialization:
+
+```c
+#include "gsp_rpc_integration.h"
+
+/* In nv_vgpu_gsp_init(), after existing init: */
+ret = nv_vgpu_gsp_rpc_integration_init(vgpu, ctx);
+if (ret) {
+    nv_vgpu_warn(vgpu, "GSP RPC integration init failed: %d", ret);
+    /* Non-fatal - continue with legacy RPC */
+}
+
+/* In nv_vgpu_gsp_cleanup(), before existing cleanup: */
+nv_vgpu_gsp_rpc_integration_cleanup(vgpu);
+```
+
+### Step 2: Replace Legacy RPC Calls
+
+Replace existing RPC functions with enhanced versions:
+
+```c
+/* Old: */
+ret = nv_vgpu_gsp_alloc(vgpu, size, align, handle);
+
+/* New: */
+ret = nv_vgpu_gsp_rpc_alloc_fw(vgpu, size, align, mem_type, flags,
+                                addr_hint, pid, gpu_va, handle, actual_size);
+```
+
+### Step 3: Add New Capabilities
+
+Use new RPC types not previously available:
+
+```c
+/* Page pinning for DMA */
+ret = nv_vgpu_gsp_rpc_pin_fw(vgpu, handle, offset, size, flags,
+                              &dma_addr, &num_pages);
+
+/* BAR trap emulation */
+ret = nv_vgpu_gsp_rpc_bar_trap_fw(vgpu, bar_addr, access_type, size,
+                                   value, bar_num, &ret_value);
+
+/* Live migration */
+ret = nv_vgpu_gsp_rpc_migrate_save_fw(vgpu, src_gpu, dst_gpu,
+                                       mem_bitmap, pt_ptr, &checkpoint);
+
+ret = nv_vgpu_gsp_rpc_migrate_restore_fw(vgpu, src_gpu, dst_gpu,
+                                          mem_bitmap, pt_ptr, checkpoint);
+
+/* WPR management */
+ret = nv_vgpu_gsp_wpr_write_mutex_acquire(vgpu, wpr_id);
+ret = nv_vgpu_gsp_wpr_write_mutex_release(vgpu, wpr_id);
+```
+
+## Firmware Address Validation
+
+The integration layer exports firmware address accessors for runtime validation:
+
+```c
+/* Verify firmware matches expected version */
+u64 primary = nv_vgpu_gsp_get_primary_dispatcher();
+u64 entry = nv_vgpu_gsp_get_entry_point();
+
+if (primary != 0x057a1ac0 || entry != 0x04009000) {
+    pr_warn("Firmware version mismatch! Expected 570.144W\n");
+}
+```
+
+## Testing
+
+### Run Validation Tests
+
+```bash
+# Load test module
+sudo insmod tests/gsp_rpc_test.ko
+
+# Check results
+dmesg -T | grep -i "gsp rpc test"
+
+# Unload
+sudo rmmod gsp_rpc_test
+```
+
+Expected output:
+```
+GSP RPC Client Test Suite Starting
+=== Testing Structure Sizes ===
+PASS: gsp_rpc_msg size
+PASS: gsp_rpc_response size
+...
+=== Testing Firmware Address Constants ===
+PASS: GSP_FW_BASE
+PASS: Primary Dispatcher
+...
+Test Results: 85 total, 85 passed, 0 failed
+TEST SUITE PASSED
+```
+
+### Test with Real Hardware
+
+```bash
+# Load main module
+sudo insmod nvidia-vgpu-vfio.ko
+
+# Check GSP initialization
+dmesg -T | grep -i "gsp"
+
+# Create mdev device
+echo "nvidia-vgpu-turing" | sudo tee /sys/bus/mdev/devices/.../create
+
+# Check RPC integration
+dmesg -T | grep -i "rpc integration"
+```
+
+## Debugging
+
+### Enable Debug Logging
+
+Add to `kernel/Makefile`:
+```makefile
+ccflags-y += -DDEBUG
+```
+
+Or dynamically:
+```bash
+echo 8 > /sys/module/nvidia_vgpu_vfio/parameters/debug
+```
+
+### Verify Firmware Addresses
+
+```bash
+# Check firmware symbols in kernel log
+dmesg | grep -E "(0x057a1ac0|0x04ed39dc|0x057e94ec|0x051efbf4)"
+
+# Verify jump table
+dmesg | grep "0x057a3ad8"
+```
+
+### Common Issues
+
+| Issue | Cause | Solution |
+|-------|-------|----------|
+| `gsp_client_init` returns `-ENODEV` | BAR0 not mapped | Ensure `nv_vgpu_bar_init()` called first |
+| RPC timeout | Doorbell not working | Check IRQ registration in `gsp_comm.c` |
+| Structure size mismatch | Kernel version difference | Verify `__packed` attribute on all structs |
+| Migration fails | WPR mutex not available | Check `0x0407xxxx` region in firmware |
+
+## Migration from Legacy RPC
+
+### Before (Legacy)
+```c
+int nv_vgpu_gsp_alloc(struct nv_vgpu_device *vgpu, u64 size, u32 align, u64 *handle)
+{
+    struct { u64 size; u32 align; u64 handle; } msg = { .size = size, .align = align };
+    return nv_vgpu_gsp_send_rpc(vgpu, &msg, sizeof(msg));
+}
+```
+
+### After (Enhanced)
+```c
+int nv_vgpu_gsp_rpc_alloc_fw(struct nv_vgpu_device *vgpu,
+                              u64 size, u64 alignment, u32 mem_type,
+                              u32 flags, u64 addr_hint, u32 pid,
+                              u64 *gpu_va, u64 *handle, u64 *actual_size)
+{
+    struct gsp_rpc_integration *integ = g_integration;
+    return gsp_rpc_alloc(integ->rpc_client, size, alignment, mem_type,
+                          flags, addr_hint, pid, gpu_va, handle, actual_size);
+}
+```
+
+## Performance Considerations
+
+1. **Message Batching**: The RPC client uses shared memory for message passing - batch multiple operations when possible.
+
+2. **Doorbell Optimization**: Write sequence ID to doorbell register only once per batch.
+
+3. **WPR Mutex**: Cache WPR mutex state to avoid redundant acquire/release cycles.
+
+4. **Migration**: Use `GSP_MIGRATE_SUBCMD_SAVE`/`RESTORE` for checkpoint-based migration.
+
+## Firmware Version Compatibility
+
+| Firmware Version | Base Address | Primary Dispatcher | Compatible |
+|------------------|--------------|-------------------|------------|
+| 570.144W | 0x04000000 | 0x057a1ac0 | ✅ Yes |
+| 535.x | 0x04000000 | Different | ❌ No |
+| 550.x | 0x04000000 | Different | ⚠️ Partial |
+
+**Note**: This implementation is specific to GSP firmware 570.144W. Addresses will differ for other versions.
+
+## Troubleshooting Checklist
+
+- [ ] Kernel module builds without errors
+- [ ] Test module passes all 85 tests
+- [ ] Main module loads without warnings
+- [ ] GSP firmware loads (check `dmesg` for "GSP firmware loaded")
+- [ ] GSP boot sequence initiates (check for "GSP boot sequence initiated")
+- [ ] RPC integration initializes (check for "GSP RPC integration initialized")
+- [ ] Doorbell IRQ fires on RPC completion
+- [ ] ALLOC/MAP/PIN operations return success
+- [ ] Migration save/restore works
+- [ ] WPR mutex operations work
+
+## Support
+
+For issues or questions:
+- Check `firmware/570-GSP/DECOMPILATION_COMPLETE.md` for full decompilation details
+- Check `firmware/570-GSP/RPC_HANDLER_MAP.md` for complete handler address map
+- Check `firmware/570-GSP/IMPLEMENTATION_MAP.md` for 7-phase integration plan
+- Review `kernel/tests/gsp_rpc_test.c` for usage examples
+
+## License
+
+GPL v2 - Open-IOV Project