Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 288 insertions, 0 deletions

diff --git a/wiki/WPR-Management.md b/wiki/WPR-Management.md
new file mode 100644
index 0000000..dbbe5b1
--- /dev/null
+++ b/wiki/WPR-Management.md
@@ -0,0 +1,288 @@
+# WPR (Write/Read Protect Region) Management
+
+## Overview
+WPR (Write Protect Region / Read Protect Region) provides memory isolation for vGPU by protecting guest memory regions from unauthorized access. The GSP firmware implements WPR through two handlers at **0x04F00B0C** and **0x04F00C78** (62 KB each).
+
+## Key Addresses
+
+| Handler | Address | Size | Role |
+|---------|---------|------|------|
+| WPR Handler 1 | 0x04F00B0C | 62 KB | Primary WPR management |
+| WPR Handler 2 | 0x04F00C78 | 62 KB | Secondary/timeout handling |
+| String Table | 60 locations | — | Completion/timeout strings |
+
+## String References (60 Total)
+
+### Completion Events
+```
+WPR_WRITE_COMPLETE      0x5E478F8
+WPR_READ_COMPLETE       0x5E478E0
+NV_WPR_WRITE_COMPLETE   0x5E478C8
+NV_WPR_READ_COMPLETE    0x5E478B0
+```
+
+### Timeout Events
+```
+WPR_WRITE_TIMEOUT       0x5E47898
+WPR_READ_TIMEOUT        0x5E47880
+NV_WPR_WRITE_TIMEOUT    0x5E47868
+NV_WPR_READ_TIMEOUT     0x5E47850
+```
+
+## WPR State Machine
+
+### Handler 1: 0x04F00B0C (Primary)
+
+```c
+typedef enum {
+    WPR_STATE_IDLE = 0,
+    WPR_STATE_WRITE_PENDING = 1,
+    WPR_STATE_READ_PENDING = 2,
+    WPR_STATE_COMPLETE = 3,
+    WPR_STATE_TIMEOUT = 4,
+    WPR_STATE_ERROR = 0xFFFF
+} wpr_state_t;
+
+typedef struct {
+    uint64_t region_base;      // Protected region base
+    uint64_t region_size;      // Region size (page-aligned)
+    uint32_t vf_id;            // Owning VF
+    uint32_t flags;            // WPR_WRITE | WPR_READ | WPR_EXECUTE
+    wpr_state_t state;
+    uint64_t timeout_ns;       // Operation timeout
+    void *completion_cb;       // Callback on complete
+    void *timeout_cb;          // Callback on timeout
+} wpr_region_t;
+```
+
+### Operation Flow
+
+#### Write Protect
+```
+1. VF requests WPR_WRITE on region [base, base+size)
+2. Handler validates alignment (4KB) and permissions
+3. Programs MMU/IOMMU write-protect bits
+4. Starts timeout timer (configurable)
+5. On GPU write attempt:
+   a. Trap to GSP via page fault
+   b. Check if VF owns region
+   c. If allowed: emulate write, return OK
+   d. If denied: trigger WPR_WRITE_COMPLETE/NV_WPR_WRITE_COMPLETE
+6. On timeout: trigger WPR_WRITE_TIMEOUT/NV_WPR_WRITE_TIMEOUT
+```
+
+#### Read Protect
+```
+1. VF requests WPR_READ on region
+2. Programs MMU/IOMMU read-protect bits
+3. On GPU read attempt:
+   a. Trap to GSP
+   b. Return zero/poison data or emulate
+   c. Trigger WPR_READ_COMPLETE/NV_WPR_READ_COMPLETE
+4. On timeout: trigger WPR_READ_TIMEOUT/NV_WPR_READ_TIMEOUT
+```
+
+## VBIOS Parsing (Handler 2: 0x04F00C78)
+
+The WPR handlers parse VBIOS structures for legacy compatibility:
+
+```c
+int parse_vbios_for_wpr(void *vbios_ptr, size_t size) {
+    // Search for signatures:
+    // 0xAA55 - Standard VBIOS signature
+    // 0x57464F4E ("WFON") - WPR enable
+    // 0x5746564E ("WFVN") - WPR version
+    
+    for (offset = 0; offset < size; offset += 0x1000) {
+        uint32_t sig = *(uint32_t*)(vbios_ptr + offset);
+        
+        if (sig == 0xAA55) {
+            // Check for WPR structures
+            parse_wpr_tables(vbios_ptr + offset);
+        }
+        else if (sig == 0x57464F4E || sig == 0x5746564E) {
+            // WPR enabled in VBIOS
+            wpr_vbios_enabled = true;
+            wpr_vbios_offset = offset;
+        }
+    }
+}
+```
+
+## Integration with GSP Subsystems
+
+### Memory Allocator (0x429B450)
+```c
+// Allocate WPR metadata structures
+wpr_region_t *wpr_alloc_region() {
+    return UNK_ram_058A1A9C(0x429B450, sizeof(wpr_region_t));
+}
+
+// Allocate bounce buffers for emulation
+void *wpr_alloc_bounce(size_t size) {
+    return UNK_ram_058A1A9C(0x429B450, size);
+}
+```
+
+### DMA/Copy Engine (0x0588F9C4, 0x05AEC86C)
+```c
+// Copy data for WPR emulation
+int wpr_emulate_access(wpr_region_t *rgn, void *dst, void *src, size_t size) {
+    return UNK_ram_0588F9C4(dma_engine, src, dst, size);
+}
+```
+
+### Logging (0x05AB7DA0)
+```c
+// Log WPR events
+void wpr_log_event(const char *event_str, uint32_t vf_id, uint64_t addr) {
+    UNK_ram_05AB7DA0(1, &event_str);  // Log string
+    UNK_ram_05AB7DA0(2, &vf_id);      // Log VF ID
+    UNK_ram_05AB7DA0(3, &addr);       // Log address
+}
+```
+
+## vGPU-IO Integration
+
+### Kernel WPR API
+```c
+struct gsp_wpr_ops {
+    int (*protect_write)(struct gsp_device *gsp, 
+                         uint32_t vf_id,
+                         uint64_t base, uint64_t size);
+    int (*protect_read)(struct gsp_device *gsp,
+                        uint32_t vf_id,
+                        uint64_t base, uint64_t size);
+    int (*unprotect)(struct gsp_device *gsp,
+                     uint32_t vf_id,
+                     uint64_t base);
+    int (*set_timeout)(struct gsp_device *gsp,
+                       uint32_t vf_id,
+                       uint64_t timeout_ns);
+    int (*register_callback)(struct gsp_device *gsp,
+                             uint32_t vf_id,
+                             enum wpr_event_type type,
+                             void (*cb)(struct wpr_event *));
+};
+
+static const struct gsp_wpr_ops gsp_wpr_ops = {
+    .protect_write = gsp_wpr_write_protect,
+    .protect_read = gsp_wpr_read_protect,
+    .unprotect = gsp_wpr_unprotect,
+    .set_timeout = gsp_wpr_set_timeout,
+    .register_callback = gsp_wpr_register_callback,
+};
+```
+
+### VF WPR Lifecycle
+```c
+// VF initialization: register WPR regions
+int gsp_vf_wpr_init(struct gsp_vf *vf) {
+    // Protect VF's page tables
+    gsp_wpr_ops.protect_write(vf->gsp, vf->id,
+                              vf->pt_base, vf->pt_size);
+    
+    // Protect VF's command buffers
+    gsp_wpr_ops.protect_write(vf->gsp, vf->id,
+                              vf->cmd_base, vf->cmd_size);
+    
+    // Set default timeout (10ms)
+    gsp_wpr_ops.set_timeout(vf->gsp, vf->id, 10000000);
+    
+    // Register completion callbacks
+    gsp_wpr_ops.register_callback(vf->gsp, vf->id,
+                                  WPR_EVENT_WRITE_COMPLETE,
+                                  vf_wpr_write_complete_cb);
+    gsp_wpr_ops.register_callback(vf->gsp, vf->id,
+                                  WPR_EVENT_TIMEOUT,
+                                  vf_wpr_timeout_cb);
+    
+    return 0;
+}
+
+// VF cleanup
+void gsp_vf_wpr_fini(struct gsp_vf *vf) {
+    gsp_wpr_ops.unprotect(vf->gsp, vf->id, vf->pt_base);
+    gsp_wpr_ops.unprotect(vf->gsp, vf->id, vf->cmd_base);
+    // ... unregister callbacks
+}
+```
+
+### Callback Handling
+```c
+static void vf_wpr_write_complete_cb(struct wpr_event *evt) {
+    struct gsp_vf *vf = evt->vf;
+    
+    // Log the write attempt
+    dev_dbg(vf->dev, "WPR write complete: VF=%u addr=0x%llx\n",
+            vf->id, evt->address);
+    
+    // If this was a page table update, invalidate TLB
+    if (evt->address >= vf->pt_base && 
+        evt->address < vf->pt_base + vf->pt_size) {
+        gsp_tlb_invalidate(vf->gsp, vf->id);
+    }
+}
+
+static void vf_wpr_timeout_cb(struct wpr_event *evt) {
+    struct gsp_vf *vf = evt->vf;
+    
+    dev_warn(vf->dev, "WPR timeout: VF=%u type=%d addr=0x%llx\n",
+             vf->id, evt->type, evt->address);
+    
+    // Escalate: reset VF or throttle
+    if (evt->type == WPR_EVENT_WRITE_TIMEOUT) {
+        gsp_vf_throttle(vf);
+    } else {
+        gsp_vf_reset(vf);
+    }
+}
+```
+
+## Error Codes
+
+| Code | Name | Description |
+|------|------|-------------|
+| 0x1E | ETIME | Operation timed out |
+| 0x1F | EDEV | Device error |
+| 0x25 | EOVERFLOW | Region too large |
+| 0x37 | EINVAL | Invalid alignment |
+| 0x51 | ENOMEM | Out of memory |
+| 0x56 | EBUSY | Region busy |
+| 0xFFFF | EPERM | Permission denied |
+
+## Debugging
+
+### Monitor These Addresses
+```
+0x04296048  : Global error flag
+0x0429B750+0x111300 : Error status register
+0x05E47850-0x05E478F8 : WPR string table
+```
+
+### GSP Log Patterns
+```
+WPR_WRITE_COMPLETE    VF=<id> ADDR=<addr>
+WPR_READ_COMPLETE     VF=<id> ADDR=<addr>
+WPR_WRITE_TIMEOUT     VF=<id> ADDR=<addr>
+WPR_READ_TIMEOUT      VF=<id> ADDR=<addr>
+NV_WPR_WRITE_COMPLETE VF=<id> ADDR=<addr>
+NV_WPR_READ_COMPLETE  VF=<id> ADDR=<addr>
+NV_WPR_WRITE_TIMEOUT  VF=<id> ADDR=<addr>
+NV_WPR_READ_TIMEOUT   VF=<id> ADDR=<addr>
+```
+
+### Trace Points
+```c
+// In kernel driver
+trace_gsp_wpr_protect(vf_id, base, size, type);
+trace_gsp_wpr_event(vf_id, event_type, address, status);
+trace_gsp_wpr_timeout(vf_id, address, elapsed_ns);
+```
+
+---
+
+## Reference
+- Handler 1: `FUN_ram_04f00b0c_ram_04f00b0c.c` (62 KB)
+- Handler 2: `FUN_ram_04f00c78_ram_04f00c78.c` (62 KB)
+- RPC Types: Handled via primary/secondary dispatchers
\ No newline at end of file