summaryrefslogtreecommitdiffstats
path: root/docs/refactored-implementation-to-do.md
diff options
context:
space:
mode:
authorArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-23 13:48:07 -0400
committerArthur Hanson Rasmusson <arthur@vgpu.io>2026-09-23 13:48:07 -0400
commit681494a79e6c0e084354eec0229386a696a314c0 (patch)
treef0d69e9583cd6ed752da7e40c8167df24e228a9a /docs/refactored-implementation-to-do.md
parentbfcb856e09207d8edb714373d775537165d06b05 (diff)
downloadopen-iov-681494a79e6c0e084354eec0229386a696a314c0.tar.gz
open-iov-681494a79e6c0e084354eec0229386a696a314c0.zip
docs: merge the legacy flat documents, add the CUDA deep-dive and link checks
* Merge and remove the pre-reboot flat documents, preserving their still-useful material in the structured tree: debugging workflows and the EPT-livelock case study (05-operations/02), performance microbenchmarks and A/B methodology (05-operations/01), the version-upgrade runbook (05-operations/02 section 4, with corrected host/guest framing), kernel-code pitfalls (07-developers) and the OpenRM build/load order (03-architecture/06 section 7). The remaining legacy files were verified redundant and removed. * docs/README now records where each migrated topic lives. * Portability: the config YAML files are called out as design targets that no code reads yet, with what adding support actually requires today. * Getting started: the OpenRM step is marked optional (native vGPU-management workflows only; the emulated path uses the distro driver). * tools docs: rmtool/rmtop marked as planned, pointing at the implemented tools/rmq subset. * wiki: new CUDA-Userspace-Driver deep-dive page (public-safe, links the canonical userspace/cuda analysis) and index update. * scripts/check-docs-links.py plus a CI step so relative links cannot rot. * STATE.md references to removed documents repointed.
Diffstat (limited to 'docs/refactored-implementation-to-do.md')
-rw-r--r--docs/refactored-implementation-to-do.md516
1 files changed, 0 insertions, 516 deletions
diff --git a/docs/refactored-implementation-to-do.md b/docs/refactored-implementation-to-do.md
deleted file mode 100644
index e8ddd3c..0000000
--- a/docs/refactored-implementation-to-do.md
+++ /dev/null
@@ -1,516 +0,0 @@
-# Open-IOV vGPU-IO Refactored Implementation TODO List
-
-> **Status: superseded (historical).** Part of the pre-2026-09-23 docs; kept
-> as a record. Current entry point: [docs/README.md](README.md). For the
-> current architecture and work list see
-> [03-architecture/01](03-architecture/01-system-overview.md),
-> [03-architecture/06](03-architecture/06-host-module-and-hybrid-model.md) and
-> [04-implementation/01](04-implementation/01-known-knowns-and-unknowns.md).
-
-## Overview
-This document tracks the granular implementation steps to refactor Open-IOV from guest-side emulation to **host-side GSP emulation**, enabling unmodified guest NVIDIA drivers to run with full functionality.
-
-**Restore Point**: Git commit `f9ed5d8` - complete current stack before refactoring
-**Target**: Host-only GSP emulation in `open-iov-vfio.ko`, guest runs pure unmodified `nvidia.ko`
-
----
-
-## Phase 0: Repository & Infrastructure Setup
-
-### 0.1 Git Branching
-- [ ] `git checkout -b refactor/host-gsp-emulation` from `f9ed5d8`
-- [ ] Create `docs/old/` with previous ARCHITECTURE.md (DONE)
-- [ ] Create `docs/refactored-architecture.md` (DONE)
-- [ ] Create this TODO document
-
-### 0.2 Build System Verification
-- [ ] Verify current build works: `make clean && make` in open-iov root
-- [ ] Verify all 3 kernel modules build: `nvidia.ko`, `open-iov-vfio.ko`, `gsp-vgpu.ko`
-- [ ] Verify userspace builds: `libiov.so`, `example_vgpu`
-- [ ] Document current build commands for regression testing
-
-### 0.3 Test Infrastructure
-- [x] Create `scripts/test_gsp_emu.sh` - automated GSP emulation unit tests
-- [x] Create `scripts/test_guest_driver.sh` - guest driver validation suite
-- [x] Create `scripts/test_cuda_workloads.sh` - CUDA workload validation
-- [x] Create `scripts/test_graphics_workloads.sh` - graphics validation
-- [x] Create `scripts/test_windows_guest.sh` - Windows guest validation
-- [x] Create `scripts/test_vgpu_stack.sh` - comprehensive test runner
-- [ ] Set up CI pipeline for incremental testing
-
----
-
-## Phase 1: Host Kernel - GSP Emulation Engine (Core)
-
-### 1.1 New File: `kernel/gsp_emulation.h`
-- [x] Define `struct gsp_emu_context` with:
- - [x] `struct nv_vgpu_device *vgpu`
- - [x] `struct task_struct *thread`
- - [x] `void __iomem *bar0_vaddr`, `*doorbell_reg`, `*shared_mem`
- - [x] `struct gsp_client *rpc_client`
- - [x] `spinlock_t lock`
- - [x] `bool running`
- - [x] Statistics: `u64 processed_rpcs`, `failed_rpcs`, `total_latency_ns`
-- [x] Function prototypes:
- - [x] `int gsp_emu_init(struct nv_vgpu_device *vgpu)`
- - [x] `void gsp_emu_cleanup(struct nv_vgpu_device *vgpu)`
- - [x] `int gsp_emu_process_rpc(struct gsp_emu_context *ctx)`
- - [x] `void gsp_emu_inject_completion_interrupt(struct gsp_emu_context *ctx, u32 seq_id)`
-
-### 1.2 New File: `kernel/gsp_emulation.c`
-- [x] Implement `gsp_emu_thread_fn()` - per-mdev polling thread
- - [x] Read doorbell at `bar0_vaddr + 0x110C00`
- - [x] ACK doorbell immediately (write 0)
- - [x] Call `gsp_emu_process_rpc()`
- - [x] Adaptive polling: busy-wait first 100μs, then `usleep_range(50, 100)`
- - [x] Handle `kthread_should_stop()` for clean shutdown
-- [x] Implement `gsp_emu_init()`
- - [x] Allocate `gsp_emu_context`
- - [x] Map BAR0, locate doorbell reg + shared mem offset (0x1000)
- - [x] Allocate shared memory (4KB, page-aligned, WC mapping)
- - [x] Map shared memory into guest BAR0 at 0x1000
- - [x] Initialize `gsp_client` via `gsp_client_init()` (reuse existing)
- - [x] Start kthread `gsp_emu_thread_fn`
-- [x] Implement `gsp_emu_cleanup()`
- - [x] Stop kthread
- - [x] Free shared memory
- - [x] Call `gsp_client_fini()`
- - [x] Free context
-- [x] Implement `gsp_emu_process_rpc()` - dispatch all 12 RPC types
- - [x] Copy request from shared memory (offset 0x0000)
- - [x] Validate seq_id != 0
- - [x] Switch on `req.type` → call appropriate `gsp_rpc_*` function
- - [x] Write response to shared memory (offset 0x0080)
- - [x] Call `gsp_emu_inject_completion_interrupt()`
-- [x] Implement `gsp_emu_inject_completion_interrupt()`
- - [x] Write to MSI-X PBA at `bar0_vaddr + 0x3000` (vector 0)
- - [x] Signal eventfd: `eventfd_signal(vgpu->msix_trigger[0])`
-
-### 1.3 Modify `kernel/main.c`
-- [x] Add `struct gsp_emu_context *gsp_emu` to `struct nv_vgpu_device` (reused `gsp_comm_ctx`)
-- [x] In `nv_vgpu_device_init_work()`: call `gsp_emu_init(vgpu)` after `nv_vgpu_gsp_init()`
-- [x] In `nv_vgpu_device_cleanup_work()`: call `gsp_emu_cleanup(vgpu)` before `nv_vgpu_gsp_cleanup()`
-- [x] Include `gsp_emulation.h`
-
-### 1.4 Modify `kernel/Makefile`
-- [x] Add `gsp_emulation.o` to `open-iov-vfio-y` or module objects
-- [x] Verify build: `make clean && make`
-
-### 1.5 Unit Tests for GSP Emulation (kernel)
-- [x] Create `kernel/tests/gsp_emulation_test.c`
-- [x] Create `kernel/tests/Makefile`
-- [x] Test ALLOC/FREE round-trip
-- [x] Test MAP/UNMAP with various sizes/alignments
-- [x] Test PIN/UNPIN returns valid DMA addresses
-- [x] Test SCHEDULE/YIELD completes
-- [x] Test BAR_TRAP read/write emulation
-- [x] Test NOTIFY register/unregister
-- [x] Test MIGRATE 4-phase state machine
-- [x] Test concurrent RPCs from multiple threads
-- [x] Test error handling (invalid seq_id, bad payload)
-- [x] Add to `kernel/tests/Makefile`
-
----
-
-## Phase 2: PCI Config Space & BAR Enhancements
-
-### 2.1 Enhance PCI Config Space (`kernel/main.c`)
-- [x] Verify MSI-X capability at 0x70 points to Table@BAR0+0x2000, PBA@BAR0+0x3000
-- [x] Ensure `VFIO_PCI_CONFIG_REGION_INDEX` returns full 256B config space
-- [x] Add config space write handlers for:
- - [x] Command register (0x04): Memory/IO/BusMaster enable
- - [x] MSI-X Message Control (0x72): Enable/disable, table size
- - [x] MSI-X Table Offset (0x74-0x77): Verify BAR0+0x2000
- - [x] MSI-X PBA Offset (0x78-0x7B): Verify BAR0+0x3000
-
-### 2.2 BAR Region VFIO Mapping (`kernel/bar_emulation.c`)
-- [x] Implement `VFIO_PCI_BAR0_REGION_INDEX` → 16MB emulated config space
-- [x] Implement `VFIO_PCI_BAR1_REGION_INDEX` → 256MB VRAM (backed by host allocation)
-- [x] Implement `VFIO_PCI_BAR2_REGION_INDEX` → 32MB reserved
-- [x] Implement `VFIO_PCI_BAR3_REGION_INDEX` → 32MB reserved
-- [x] Implement `VFIO_PCI_MSIX_REGION_INDEX` → MSI-X Table (0x2000) + PBA (0x3000)
-- [x] Ensure BAR0 mmap includes doorbell @ 0x110C00 and shared mem @ 0x1000
-
-### 2.3 MSI-X Table/PBA Emulation
-- [x] Allocate MSI-X table (32 entries × 16 bytes = 512 bytes) in BAR0 at 0x2000
-- [x] Allocate PBA (32 bits = 4 bytes, aligned to 64 bytes) in BAR0 at 0x3000
-- [x] Guest writes to MSI-X table entries → host stores for interrupt injection
-- [x] Guest reads PBA → host returns pending bits
-
----
-
-## Phase 3: GSP RPC Handler Completion
-
-### 3.1 Verify All 12 RPC Handlers in `gsp_rpc_client.c`
-- [x] ALLOC (0x01): VRAM allocation via host nvidia.ko
-- [x] FREE (0x02): Free allocation
-- [x] MAP (0x03): Map to GPU VA
-- [x] UNMAP (0x04): Unmap GPU VA
-- [x] PIN (0x05): Pin for DMA, return DMA address
-- [x] UNPIN (0x06): Unpin pages
-- [x] SCHEDULE (0x07): Submit work to local workqueue
-- [x] YIELD (0x08): No-op success
-- [x] NOTIFY (0x09): Async notification registration
-- [x] BAR_TRAP (0x0A): Shadow register read/write
-- [x] MIGRATE (0x0B): 4-phase protocol
-- [x] DEBUG (0xFF): Firmware introspection
-
-### 3.2 Enhance Handler Implementations
-- [x] ALLOC: Use host page allocation + DMA mapping
-- [x] MAP: GPU VA mapping management
-- [x] PIN: Return DMA address from mapping
-- [x] BAR_TRAP: Use bar_emulation.c shadow memory
-- [x] MIGRATE: Use migration.c 4-phase state machine
-- [x] NOTIFY: Use gsp_rpc_notify_register() + eventfd for guest delivery
-
-### 3.3 Add RPC Validation & Error Handling
-- [x] Sequence ID tracking (detect duplicates, gaps)
-- [x] Payload size validation per RPC type
-- [x] Timeout handling (guest watchdog)
-- [x] Proper error codes in response.status
-
----
-
-## Phase 4: Portable Architecture Implementation
-
-### 4.1 Firmware Analyzer (`kernel/firmware_analyzer.c`)
-- [x] Compute firmware CRC32 and SHA256 signatures
-- [x] Pattern matching for RPC dispatcher discovery
-- [x] Pattern matching for jump table discovery
-- [x] Pattern matching for ALLOC handler discovery
-- [x] Known firmware signature database
-- [x] Auto-discovery fallback for unknown firmware
-- [x] `fw_analyzer_analyze()` - analyze firmware blob
-- [x] `fw_analyzer_load_and_analyze()` - load and analyze from filesystem
-- [x] `fw_analyzer_print_result()` - debug output
-
-### 4.2 Dynamic RPC Dispatcher (`kernel/dynamic_rpc_dispatcher.c`)
-- [x] Version-specific interface registry (550, 560, 570, 580)
-- [x] Native firmware handler function pointers
-- [x] Emulation fallback interface
-- [x] Automatic interface selection based on firmware analysis
-- [x] Per-RPC-type emulation toggle
-- [x] Dispatcher API for all 12 RPC types
-- [x] `dynamic_dispatcher_init()` - initialize with firmware path
-- [x] `dynamic_dispatcher_get_interface()` - get current interface
-- [x] `dynamic_dispatcher_use_emulation()` - check if emulation used for type
-
-### 4.3 Hardware Architecture Layer (`kernel/architecture/`)
-- [x] `architecture.h` - hardware abstraction API
-- [x] `architecture.c` - architecture implementations
-- [x] Turing architecture (TU102/TU104) config
-- [x] Ampere architecture (GA100/GA102/GA104) config
-- [x] Blackwell architecture (GB100/GB202) config
-- [x] Architecture lookup by name and PCI ID
-- [x] BAR configuration queries
-- [x] Doorbell/shared memory offset queries
-- [x] MSI-X vector count queries
-- [x] Feature support queries (MIG, SR-IOV)
-
-### 4.4 Driver Version Adaptation Layer (`kernel/version/`)
-- [x] `version.h` - version adaptation API
-- [x] `version.c` - version implementations
-- [x] Version 550 interface
-- [x] Version 560 interface
-- [x] Version 570 interface
-- [x] Version 580 interface
-- [x] Feature detection per version
-- [x] Interface lookup by version string
-
-### 4.5 Declarative Configuration Files
-- [x] `config/hardware/turing.yaml` - Turing hardware config
-- [x] `config/hardware/ampere.yaml` - Ampere hardware config
-- [x] `config/hardware/blackwell.yaml` - Blackwell hardware config
-- [x] `config/versions/550.yaml` - Driver 550 config
-- [x] `config/versions/560.yaml` - Driver 560 config
-- [x] `config/versions/570.yaml` - Driver 570 config
-- [x] `config/versions/580.yaml` - Driver 580 config
-
----
-
-## Phase 5: Guest Driver Integration Testing
-
-### 4.1 Host Module Loading Sequence
-- [ ] `sudo insmod nvidia-open-gpu-kernel-modules/kernel-open/nvidia.ko`
-- [ ] `sudo insmod kernel/open-iov-vfio.ko`
-- [ ] Verify mdev types appear: `ls /sys/bus/pci/devices/0000:XX:XX.X/mdev_supported_types/`
-- [ ] Create mdev: `echo <UUID> | sudo tee /sys/bus/pci/devices/.../mdev_supported_types/nvidia-vgpu-turing/create`
-
-### 4.2 QEMU Launch with mdev
-- [ ] `qemu-system-x86_64 -device vfio-pci,sysfsdev=/sys/bus/mdev/devices/<UUID>,display=off`
-- [ ] Verify guest sees PCI device: `lspci -d 10de:` → should show 10de:1ed3
-- [ ] Verify BARs: `lspci -v -d 10de:` → BAR0=16MB, BAR1=256MB, BAR2=32MB, BAR3=32MB
-- [ ] Verify MSI-X: `lspci -v -d 10de:` → MSI-X capability present
-
-### 4.3 Guest NVIDIA Driver Installation
-- [ ] Install OpenRM 570.144 in guest: `./NVIDIA-Linux-x86_64-570.144.run --silent --dkms`
-- [ ] Install firmware: Copy `bootloader`, `booter_load`, `gsp` binaries to `/lib/firmware/nvidia/tu102/gsp/`
-- [ ] Load driver: `sudo modprobe nvidia`
-- [ ] Verify: `nvidia-smi` shows vGPU (T4 or equivalent)
-- [ ] Verify: `dmesg | grep -i nvidia` shows successful probe
-
-### 4.4 Basic Functionality Tests
-- [ ] `nvidia-smi -q` shows all fields populated
-- [ ] `nvidia-smi dmon` shows utilization counters
-- [ ] `nvidia-smi mig` shows MIG not supported (expected for vGPU)
-- [ ] `cat /proc/driver/nvidia/gpus/0000:00:00.0/information` shows correct info
-
----
-
-## Phase 5: CUDA Workload Validation
-
-### 5.1 CUDA SDK Tests
-- [ ] Compile and run `vectorAdd` sample
-- [ ] Compile and run `matrixMul` sample
-- [ ] Compile and run `simpleCUBLAS` sample
-- [ ] Compile and run `simpleCUFFT` sample
-- [ ] Verify numerical correctness vs CPU reference
-
-### 5.2 cuBLAS / cuDNN Tests
-- [ ] SGEMM/DGEMM various sizes (64x64 to 4096x4096)
-- [ ] Batched GEMM
-- [ ] cuDNN convolution forward/backward
-- [ ] cuDNN RNN/LSTM
-- [ ] Performance within 10% of native
-
-### 5.3 PyTorch / TensorFlow Integration
-- [ ] PyTorch: `torch.cuda.is_available()`, tensor ops, autograd
-- [ ] PyTorch: Training loop (ResNet-50, BERT-small)
-- [ ] TensorFlow: `tf.config.list_physical_devices('GPU')`
-- [ ] TensorFlow: Keras model training
-- [ ] Mixed precision (FP16/BF16) works
-
-### 5.4 Multi-GPU / NCCL Tests
-- [ ] Create 2+ vGPUs on same physical GPU
-- [ ] NCCL all-reduce across vGPUs
-- [ ] NCCL broadcast/reduce
-- [ ] Multi-process CUDA (MPS) if supported
-
-### 5.5 CUDA Graphs & Streams
-- [ ] CUDA Graph capture/replay
-- [ ] Multiple streams concurrent execution
-- [ ] Stream priorities
-- [ ] Event synchronization
-
----
-
-## Phase 6: Graphics Workload Validation
-
-### 6.1 Vulkan Tests
-- [ ] `vulkaninfo` shows GPU properties
-- [ ] `vkcube` renders correctly
-- [ ] `vkcubepp` (Vulkan 1.1+) renders
-- [ ] Sascha Willems Vulkan samples (deferred, compute, ray tracing if supported)
-- [ ] VK_KHR_swapchain, VK_KHR_dedicated_allocation work
-
-### 6.2 OpenGL Tests
-- [ ] `glxinfo | grep -i nvidia` shows correct renderer
-- [ ] `glxgears` runs at expected FPS
-- [ ] `glmark2` benchmark completes
-- [ ] Piglit test suite (subset) passes
-
-### 6.3 Video Encode/Decode
-- [ ] FFmpeg + NVENC: H.264 encode 1080p60
-- [ ] FFmpeg + NVENC: HEVC encode 4K30
-- [ ] FFmpeg + NVDEC: H.264 decode 1080p60
-- [ ] FFmpeg + NVDEC: HEVC decode 4K30
-- [ ] GStreamer nvh264enc/nvh265enc elements
-- [ ] GStreamer nvh264dec/nvh265dec elements
-
-### 6.4 Windows Guest Graphics (Separate Track)
-- [ ] Windows 11 guest with NVIDIA 570.144 driver
-- [ ] Device Manager: "NVIDIA T4" (or equivalent) with no error codes
-- [ ] GPU-Z / CUDA-Z detect vGPU correctly
-- [ ] D3D11: `D3D11CreateDevice` succeeds, render triangle
-- [ ] D3D12: `D3D12CreateDevice` succeeds, render triangle
-- [ ] Vulkan on Windows: `vkCreateInstance`, render
-- [ ] DXGI swapchain present works
-- [ ] Video encode via Media Foundation (MFT)
-- [ ] Video decode via Media Foundation (MFT)
-
----
-
-## Phase 7: Performance Benchmarking
-
-### 7.1 Microbenchmarks
-- [ ] GSP RPC round-trip latency (target < 10μs)
-- [ ] BAR register access latency (target < 2μs)
-- [ ] VRAM alloc/free latency (target < 50μs)
-- [ ] VRAM map/unmap latency (target < 20μs)
-- [ ] DMA pin/unpin latency (target < 30μs)
-- [ ] MSI-X injection latency (target < 5μs)
-
-### 7.2 Macrobenchmarks
-- [ ] CUDA kernel launch overhead (target < 5% vs native)
-- [ ] cuBLAS SGEMM 4096x4096 (target > 90% native GFLOPS)
-- [ ] cuDNN ResNet-50 training step (target > 85% native)
-- [ ] Vulkan frame time (target < 2ms @ 1080p)
-- [ ] NVENC 1080p60 encode latency (target < 5ms)
-- [ ] NVDEC 1080p60 decode latency (target < 3ms)
-
-### 7.3 Stress Tests
-- [ ] 24-hour CUDA compute stress (no errors, no memory leaks)
-- [ ] 24-hour graphics stress (Unigine Heaven/Valley loop)
-- [ ] 1000x VF create/destroy cycles
-- [ ] Concurrent CUDA + graphics workloads
-- [ ] Memory pressure test (allocate 90% VRAM, verify OOM handling)
-
----
-
-## Phase 8: Live Migration Validation
-
-### 8.1 Migration Protocol Tests
-- [ ] QUERY phase: source reports capabilities
-- [ ] PREPARE phase: destination allocates resources
-- [ ] TRANSFER phase: iterative VRAM + GSP state copy
-- [ ] COMMIT phase: cutover, guest resumes on destination
-- [ ] CLEANUP phase: source releases resources
-
-### 8.2 Migration Under Load
-- [ ] Migrate idle vGPU (downtime < 50ms)
-- [ ] Migrate running CUDA kernel (downtime < 100ms)
-- [ ] Migrate running graphics (no visual artifacts)
-- [ ] Migrate with pinned DMA buffers
-- [ ] Verify post-migration CUDA/graphics work correctly
-
-### 8.3 Migration State Verification
-- [ ] GSP context (registers, PC, stack) restored
-- [ ] Page tables identical
-- [ ] VRAM contents identical (checksum)
-- [ ] BAR register state identical
-- [ ] MSI-X table/PBA state identical
-
----
-
-## Phase 9: Windows Guest Validation
-
-### 9.1 Windows Guest Setup
-- [ ] Windows 10 22H2 / Windows 11 23H2 guest
-- [ ] VirtIO drivers for network/storage
-- [ ] QEMU guest agent
-- [ ] UEFI firmware (OVMF with TPM)
-
-### 9.2 NVIDIA Driver Installation
-- [ ] Download 570.144 Windows driver
-- [ ] Install with `setup.exe -s -noreboot` (silent)
-- [ ] Reboot guest
-- [ ] Verify in Device Manager: "NVIDIA T4" (Code 0 = working)
-
-### 9.3 Windows Compute Tests
-- [ ] `nvidia-smi.exe` shows vGPU
-- [ ] CUDA-Z detects GPU, runs device query
-- [ ] `vectorAdd.exe` (compiled in guest) executes
-- [ ] PyTorch/TensorFlow in Windows (WSL2 or native)
-
-### 9.4 Windows Graphics Tests
-- [ ] `dxdiag` shows correct GPU info
-- [ ] D3D11 tutorial samples render
-- [ ] D3D12 tutorial samples render
-- [ ] Vulkan SDK samples run
-- [ ] FurMark / 3DMark stress test
-
-### 9.5 Windows Video Tests
-- [ ] Media Foundation H.264/HEVC encode
-- [ ] Media Foundation H.264/HEVC decode
-- [ ] NVENC API direct usage
-- [ ] NVDEC API direct usage
-
----
-
-## Phase 10: Reliability & Production Hardening
-
-### 10.1 Error Handling & Recovery
-- [ ] Guest driver crash → host cleans up mdev state
-- [ ] Host kernel oops in GSP emulation → watchdog restarts thread
-- [ ] Physical GPU reset → host reinitializes GSP emulation
-- [ ] IOMMU fault → proper error to guest
-- [ ] VRAM OOM → proper CUDA_OUT_OF_MEMORY to guest
-
-### 10.2 Monitoring & Observability
-- [ ] Debugfs interface: `/sys/kernel/debug/open-iov-vfio/`
- - [ ] Per-mdev RPC statistics
- - [ ] Latency histograms
- - [ ] Error counters
- - [ ] GSP emulation thread CPU usage
-- [ ] Prometheus metrics exporter (optional)
-- [ ] Tracepoints for RPC processing
-
-### 10.3 Security Hardening
-- [ ] Validate all guest inputs (seq_id, payload sizes, pointers)
-- [ ] Isolate mdev memory (no cross-VF access)
-- [ ] Rate-limit doorbell writes (prevent DoS)
-- [ ] Audit VFIO ioctl permissions
-
-### 10.4 Documentation & Handoff
-- [ ] Update `docs/refactored-architecture.md` with as-built details
-- [ ] Create `docs/OPERATIONS.md` for deployment
-- [ ] Create `docs/TROUBLESHOOTING.md` for common issues
-- [ ] Record all firmware addresses used
-- [ ] Document test results for each phase
-
----
-
-## Success Criteria Checklist
-
-### Minimum Viable Product (Phases 0-4)
-- [ ] Host loads `open-iov-vfio.ko` with GSP emulation thread
-- [ ] mdev creates successfully with correct PCI config space
-- [ ] Guest `nvidia.ko` loads, probes vGPU successfully
-- [ ] `nvidia-smi` in guest shows vGPU
-- [ ] Simple CUDA kernel (`vectorAdd`) executes correctly
-
-### Compute Ready (Phases 5-6)
-- [ ] cuBLAS/cuDNN workloads pass numerical verification
-- [ ] PyTorch/TensorFlow training works
-- [ ] Vulkan/OpenGL rendering works
-- [ ] Video encode/decode works
-- [ ] Performance > 85% native for compute, > 90% for graphics
-
-### Production Ready (Phases 7-10)
-- [ ] 24-hour stress tests pass
-- [ ] Live migration works under load
-- [ ] Windows guest fully functional
-- [ ] Monitoring/observability in place
-- [ ] Security audit passed
-- [ ] Documentation complete
-
----
-
-## Dependencies & Prerequisites
-
-| Component | Version | Source |
-|-----------|---------|--------|
-| Linux Kernel | 6.8+ | Host distro |
-| NVIDIA OpenRM | 570.144 | `nvidia-open-gpu-kernel-modules` submodule |
-| QEMU | 8.0+ | Host distro / source |
-| libvfio-user | Latest | `libvfio-user` repo |
-| CUDA Toolkit | 12.4+ | Guest (for compilation) |
-| Vulkan SDK | 1.3+ | Guest |
-| Windows 10/11 | 22H2/23H2 | Guest ISO |
-
----
-
-## Tracking
-
-| Phase | Status | Start Date | End Date | Notes |
-|-------|--------|------------|----------|-------|
-| 0: Setup | ✅ DONE | - | - | Git repo, docs, restore point |
-| 1: GSP Emulation Engine | ✅ DONE | | | Core implementation complete |
-| 2: PCI/BAR Enhancements | ✅ DONE | | | Config space, MSI-X, BAR regions |
-| 3: RPC Handler Completion | ✅ DONE | | | All 12 RPC types with emulation |
-| 4: Portable Architecture | ✅ DONE | | | Firmware analyzer, dispatcher, arch/version layers, configs |
-| 5: Guest Driver Integration | 🔄 IN PROGRESS | | | Requires reboot to test |
-| 6: CUDA Validation | ⏳ PENDING | | | |
-| 7: Graphics Validation | ⏳ PENDING | | | |
-| 8: Performance Benchmarks | ⏳ PENDING | | | |
-| 9: Migration Validation | ⏳ PENDING | | | |
-| 10: Windows Guest | ⏳ PENDING | | | |
-| 11: Production Hardening | ⏳ PENDING | | | |
-| 10: Production Hardening | ⏳ PENDING | | | |
-
----
-
-*TODO List Version: 1.0*
-*Repository: $OPEN_IOV_ROOT*
-*Branch: refactor/host-gsp-emulation*
-*Base Commit: f9ed5d8* \ No newline at end of file