path: root/docs/refactored-implementation-to-do.md

# Open-IOV vGPU-IO Refactored Implementation TODO List

## Overview
This document tracks the granular implementation steps to refactor Open-IOV from guest-side emulation to **host-side GSP emulation**, enabling unmodified guest NVIDIA drivers to run with full functionality.

**Restore Point**: Git commit `f9ed5d8` - complete current stack before refactoring
**Target**: Host-only GSP emulation in `open-iov-vfio.ko`, guest runs pure unmodified `nvidia.ko`

---

## Phase 0: Repository & Infrastructure Setup

### 0.1 Git Branching
- [ ] `git checkout -b refactor/host-gsp-emulation` from `f9ed5d8`
- [ ] Create `docs/old/` with previous ARCHITECTURE.md (DONE)
- [ ] Create `docs/refactored-architecture.md` (DONE)
- [ ] Create this TODO document

### 0.2 Build System Verification
- [ ] Verify current build works: `make clean && make` in open-iov root
- [ ] Verify all 3 kernel modules build: `nvidia.ko`, `open-iov-vfio.ko`, `gsp-vgpu.ko`
- [ ] Verify userspace builds: `libiov.so`, `example_vgpu`
- [ ] Document current build commands for regression testing

### 0.3 Test Infrastructure
- [x] Create `scripts/test_gsp_emu.sh` - automated GSP emulation unit tests
- [x] Create `scripts/test_guest_driver.sh` - guest driver validation suite
- [x] Create `scripts/test_cuda_workloads.sh` - CUDA workload validation
- [x] Create `scripts/test_graphics_workloads.sh` - graphics validation
- [x] Create `scripts/test_windows_guest.sh` - Windows guest validation
- [x] Create `scripts/test_vgpu_stack.sh` - comprehensive test runner
- [ ] Set up CI pipeline for incremental testing

---

## Phase 1: Host Kernel - GSP Emulation Engine (Core)

### 1.1 New File: `kernel/gsp_emulation.h`
- [x] Define `struct gsp_emu_context` with:
  - [x] `struct nv_vgpu_device *vgpu`
  - [x] `struct task_struct *thread`
  - [x] `void __iomem *bar0_vaddr`, `*doorbell_reg`, `*shared_mem`
  - [x] `struct gsp_client *rpc_client`
  - [x] `spinlock_t lock`
  - [x] `bool running`
  - [x] Statistics: `u64 processed_rpcs`, `failed_rpcs`, `total_latency_ns`
- [x] Function prototypes:
  - [x] `int gsp_emu_init(struct nv_vgpu_device *vgpu)`
  - [x] `void gsp_emu_cleanup(struct nv_vgpu_device *vgpu)`
  - [x] `int gsp_emu_process_rpc(struct gsp_emu_context *ctx)`
  - [x] `void gsp_emu_inject_completion_interrupt(struct gsp_emu_context *ctx, u32 seq_id)`

### 1.2 New File: `kernel/gsp_emulation.c`
- [x] Implement `gsp_emu_thread_fn()` - per-mdev polling thread
  - [x] Read doorbell at `bar0_vaddr + 0x110C00`
  - [x] ACK doorbell immediately (write 0)
  - [x] Call `gsp_emu_process_rpc()`
  - [x] Adaptive polling: busy-wait first 100μs, then `usleep_range(50, 100)`
  - [x] Handle `kthread_should_stop()` for clean shutdown
- [x] Implement `gsp_emu_init()`
  - [x] Allocate `gsp_emu_context`
  - [x] Map BAR0, locate doorbell reg + shared mem offset (0x1000)
  - [x] Allocate shared memory (4KB, page-aligned, WC mapping)
  - [x] Map shared memory into guest BAR0 at 0x1000
  - [x] Initialize `gsp_client` via `gsp_client_init()` (reuse existing)
  - [x] Start kthread `gsp_emu_thread_fn`
- [x] Implement `gsp_emu_cleanup()`
  - [x] Stop kthread
  - [x] Free shared memory
  - [x] Call `gsp_client_fini()`
  - [x] Free context
- [x] Implement `gsp_emu_process_rpc()` - dispatch all 12 RPC types
  - [x] Copy request from shared memory (offset 0x0000)
  - [x] Validate seq_id != 0
  - [x] Switch on `req.type` → call appropriate `gsp_rpc_*` function
  - [x] Write response to shared memory (offset 0x0080)
  - [x] Call `gsp_emu_inject_completion_interrupt()`
- [x] Implement `gsp_emu_inject_completion_interrupt()`
  - [x] Write to MSI-X PBA at `bar0_vaddr + 0x3000` (vector 0)
  - [x] Signal eventfd: `eventfd_signal(vgpu->msix_trigger[0])`

### 1.3 Modify `kernel/main.c`
- [x] Add `struct gsp_emu_context *gsp_emu` to `struct nv_vgpu_device` (reused `gsp_comm_ctx`)
- [x] In `nv_vgpu_device_init_work()`: call `gsp_emu_init(vgpu)` after `nv_vgpu_gsp_init()`
- [x] In `nv_vgpu_device_cleanup_work()`: call `gsp_emu_cleanup(vgpu)` before `nv_vgpu_gsp_cleanup()`
- [x] Include `gsp_emulation.h`

### 1.4 Modify `kernel/Makefile`
- [x] Add `gsp_emulation.o` to `open-iov-vfio-y` or module objects
- [x] Verify build: `make clean && make`

### 1.5 Unit Tests for GSP Emulation (kernel)
- [x] Create `kernel/tests/gsp_emulation_test.c`
- [x] Create `kernel/tests/Makefile`
- [x] Test ALLOC/FREE round-trip
- [x] Test MAP/UNMAP with various sizes/alignments
- [x] Test PIN/UNPIN returns valid DMA addresses
- [x] Test SCHEDULE/YIELD completes
- [x] Test BAR_TRAP read/write emulation
- [x] Test NOTIFY register/unregister
- [x] Test MIGRATE 4-phase state machine
- [x] Test concurrent RPCs from multiple threads
- [x] Test error handling (invalid seq_id, bad payload)
- [x] Add to `kernel/tests/Makefile`

---

## Phase 2: PCI Config Space & BAR Enhancements

### 2.1 Enhance PCI Config Space (`kernel/main.c`)
- [x] Verify MSI-X capability at 0x70 points to Table@BAR0+0x2000, PBA@BAR0+0x3000
- [x] Ensure `VFIO_PCI_CONFIG_REGION_INDEX` returns full 256B config space
- [x] Add config space write handlers for:
  - [x] Command register (0x04): Memory/IO/BusMaster enable
  - [x] MSI-X Message Control (0x72): Enable/disable, table size
  - [x] MSI-X Table Offset (0x74-0x77): Verify BAR0+0x2000
  - [x] MSI-X PBA Offset (0x78-0x7B): Verify BAR0+0x3000

### 2.2 BAR Region VFIO Mapping (`kernel/bar_emulation.c`)
- [x] Implement `VFIO_PCI_BAR0_REGION_INDEX` → 16MB emulated config space
- [x] Implement `VFIO_PCI_BAR1_REGION_INDEX` → 256MB VRAM (backed by host allocation)
- [x] Implement `VFIO_PCI_BAR2_REGION_INDEX` → 32MB reserved
- [x] Implement `VFIO_PCI_BAR3_REGION_INDEX` → 32MB reserved
- [x] Implement `VFIO_PCI_MSIX_REGION_INDEX` → MSI-X Table (0x2000) + PBA (0x3000)
- [x] Ensure BAR0 mmap includes doorbell @ 0x110C00 and shared mem @ 0x1000

### 2.3 MSI-X Table/PBA Emulation
- [x] Allocate MSI-X table (32 entries × 16 bytes = 512 bytes) in BAR0 at 0x2000
- [x] Allocate PBA (32 bits = 4 bytes, aligned to 64 bytes) in BAR0 at 0x3000
- [x] Guest writes to MSI-X table entries → host stores for interrupt injection
- [x] Guest reads PBA → host returns pending bits

---

## Phase 3: GSP RPC Handler Completion

### 3.1 Verify All 12 RPC Handlers in `gsp_rpc_client.c`
- [x] ALLOC (0x01): VRAM allocation via host nvidia.ko
- [x] FREE (0x02): Free allocation
- [x] MAP (0x03): Map to GPU VA
- [x] UNMAP (0x04): Unmap GPU VA
- [x] PIN (0x05): Pin for DMA, return DMA address
- [x] UNPIN (0x06): Unpin pages
- [x] SCHEDULE (0x07): Submit work to local workqueue
- [x] YIELD (0x08): No-op success
- [x] NOTIFY (0x09): Async notification registration
- [x] BAR_TRAP (0x0A): Shadow register read/write
- [x] MIGRATE (0x0B): 4-phase protocol
- [x] DEBUG (0xFF): Firmware introspection

### 3.2 Enhance Handler Implementations
- [x] ALLOC: Use host page allocation + DMA mapping
- [x] MAP: GPU VA mapping management
- [x] PIN: Return DMA address from mapping
- [x] BAR_TRAP: Use bar_emulation.c shadow memory
- [x] MIGRATE: Use migration.c 4-phase state machine
- [x] NOTIFY: Use gsp_rpc_notify_register() + eventfd for guest delivery

### 3.3 Add RPC Validation & Error Handling
- [x] Sequence ID tracking (detect duplicates, gaps)
- [x] Payload size validation per RPC type
- [x] Timeout handling (guest watchdog)
- [x] Proper error codes in response.status

---

## Phase 4: Portable Architecture Implementation

### 4.1 Firmware Analyzer (`kernel/firmware_analyzer.c`)
- [x] Compute firmware CRC32 and SHA256 signatures
- [x] Pattern matching for RPC dispatcher discovery
- [x] Pattern matching for jump table discovery
- [x] Pattern matching for ALLOC handler discovery
- [x] Known firmware signature database
- [x] Auto-discovery fallback for unknown firmware
- [x] `fw_analyzer_analyze()` - analyze firmware blob
- [x] `fw_analyzer_load_and_analyze()` - load and analyze from filesystem
- [x] `fw_analyzer_print_result()` - debug output

### 4.2 Dynamic RPC Dispatcher (`kernel/dynamic_rpc_dispatcher.c`)
- [x] Version-specific interface registry (550, 560, 570, 580)
- [x] Native firmware handler function pointers
- [x] Emulation fallback interface
- [x] Automatic interface selection based on firmware analysis
- [x] Per-RPC-type emulation toggle
- [x] Dispatcher API for all 12 RPC types
- [x] `dynamic_dispatcher_init()` - initialize with firmware path
- [x] `dynamic_dispatcher_get_interface()` - get current interface
- [x] `dynamic_dispatcher_use_emulation()` - check if emulation used for type

### 4.3 Hardware Architecture Layer (`kernel/architecture/`)
- [x] `architecture.h` - hardware abstraction API
- [x] `architecture.c` - architecture implementations
- [x] Turing architecture (TU102/TU104) config
- [x] Ampere architecture (GA100/GA102/GA104) config
- [x] Blackwell architecture (GB100/GB202) config
- [x] Architecture lookup by name and PCI ID
- [x] BAR configuration queries
- [x] Doorbell/shared memory offset queries
- [x] MSI-X vector count queries
- [x] Feature support queries (MIG, SR-IOV)

### 4.4 Driver Version Adaptation Layer (`kernel/version/`)
- [x] `version.h` - version adaptation API
- [x] `version.c` - version implementations
- [x] Version 550 interface
- [x] Version 560 interface
- [x] Version 570 interface
- [x] Version 580 interface
- [x] Feature detection per version
- [x] Interface lookup by version string

### 4.5 Declarative Configuration Files
- [x] `config/hardware/turing.yaml` - Turing hardware config
- [x] `config/hardware/ampere.yaml` - Ampere hardware config
- [x] `config/hardware/blackwell.yaml` - Blackwell hardware config
- [x] `config/versions/550.yaml` - Driver 550 config
- [x] `config/versions/560.yaml` - Driver 560 config
- [x] `config/versions/570.yaml` - Driver 570 config
- [x] `config/versions/580.yaml` - Driver 580 config

---

## Phase 5: Guest Driver Integration Testing

### 4.1 Host Module Loading Sequence
- [ ] `sudo insmod nvidia-open-gpu-kernel-modules/kernel-open/nvidia.ko`
- [ ] `sudo insmod kernel/open-iov-vfio.ko`
- [ ] Verify mdev types appear: `ls /sys/bus/pci/devices/0000:XX:XX.X/mdev_supported_types/`
- [ ] Create mdev: `echo <UUID> | sudo tee /sys/bus/pci/devices/.../mdev_supported_types/nvidia-vgpu-turing/create`

### 4.2 QEMU Launch with mdev
- [ ] `qemu-system-x86_64 -device vfio-pci,sysfsdev=/sys/bus/mdev/devices/<UUID>,display=off`
- [ ] Verify guest sees PCI device: `lspci -d 10de:` → should show 10de:1ed3
- [ ] Verify BARs: `lspci -v -d 10de:` → BAR0=16MB, BAR1=256MB, BAR2=32MB, BAR3=32MB
- [ ] Verify MSI-X: `lspci -v -d 10de:` → MSI-X capability present

### 4.3 Guest NVIDIA Driver Installation
- [ ] Install OpenRM 570.144 in guest: `./NVIDIA-Linux-x86_64-570.144.run --silent --dkms`
- [ ] Install firmware: Copy `bootloader`, `booter_load`, `gsp` binaries to `/lib/firmware/nvidia/tu102/gsp/`
- [ ] Load driver: `sudo modprobe nvidia`
- [ ] Verify: `nvidia-smi` shows vGPU (T4 or equivalent)
- [ ] Verify: `dmesg | grep -i nvidia` shows successful probe

### 4.4 Basic Functionality Tests
- [ ] `nvidia-smi -q` shows all fields populated
- [ ] `nvidia-smi dmon` shows utilization counters
- [ ] `nvidia-smi mig` shows MIG not supported (expected for vGPU)
- [ ] `cat /proc/driver/nvidia/gpus/0000:00:00.0/information` shows correct info

---

## Phase 5: CUDA Workload Validation

### 5.1 CUDA SDK Tests
- [ ] Compile and run `vectorAdd` sample
- [ ] Compile and run `matrixMul` sample
- [ ] Compile and run `simpleCUBLAS` sample
- [ ] Compile and run `simpleCUFFT` sample
- [ ] Verify numerical correctness vs CPU reference

### 5.2 cuBLAS / cuDNN Tests
- [ ] SGEMM/DGEMM various sizes (64x64 to 4096x4096)
- [ ] Batched GEMM
- [ ] cuDNN convolution forward/backward
- [ ] cuDNN RNN/LSTM
- [ ] Performance within 10% of native

### 5.3 PyTorch / TensorFlow Integration
- [ ] PyTorch: `torch.cuda.is_available()`, tensor ops, autograd
- [ ] PyTorch: Training loop (ResNet-50, BERT-small)
- [ ] TensorFlow: `tf.config.list_physical_devices('GPU')`
- [ ] TensorFlow: Keras model training
- [ ] Mixed precision (FP16/BF16) works

### 5.4 Multi-GPU / NCCL Tests
- [ ] Create 2+ vGPUs on same physical GPU
- [ ] NCCL all-reduce across vGPUs
- [ ] NCCL broadcast/reduce
- [ ] Multi-process CUDA (MPS) if supported

### 5.5 CUDA Graphs & Streams
- [ ] CUDA Graph capture/replay
- [ ] Multiple streams concurrent execution
- [ ] Stream priorities
- [ ] Event synchronization

---

## Phase 6: Graphics Workload Validation

### 6.1 Vulkan Tests
- [ ] `vulkaninfo` shows GPU properties
- [ ] `vkcube` renders correctly
- [ ] `vkcubepp` (Vulkan 1.1+) renders
- [ ] Sascha Willems Vulkan samples (deferred, compute, ray tracing if supported)
- [ ] VK_KHR_swapchain, VK_KHR_dedicated_allocation work

### 6.2 OpenGL Tests
- [ ] `glxinfo | grep -i nvidia` shows correct renderer
- [ ] `glxgears` runs at expected FPS
- [ ] `glmark2` benchmark completes
- [ ] Piglit test suite (subset) passes

### 6.3 Video Encode/Decode
- [ ] FFmpeg + NVENC: H.264 encode 1080p60
- [ ] FFmpeg + NVENC: HEVC encode 4K30
- [ ] FFmpeg + NVDEC: H.264 decode 1080p60
- [ ] FFmpeg + NVDEC: HEVC decode 4K30
- [ ] GStreamer nvh264enc/nvh265enc elements
- [ ] GStreamer nvh264dec/nvh265dec elements

### 6.4 Windows Guest Graphics (Separate Track)
- [ ] Windows 11 guest with NVIDIA 570.144 driver
- [ ] Device Manager: "NVIDIA T4" (or equivalent) with no error codes
- [ ] GPU-Z / CUDA-Z detect vGPU correctly
- [ ] D3D11: `D3D11CreateDevice` succeeds, render triangle
- [ ] D3D12: `D3D12CreateDevice` succeeds, render triangle
- [ ] Vulkan on Windows: `vkCreateInstance`, render
- [ ] DXGI swapchain present works
- [ ] Video encode via Media Foundation (MFT)
- [ ] Video decode via Media Foundation (MFT)

---

## Phase 7: Performance Benchmarking

### 7.1 Microbenchmarks
- [ ] GSP RPC round-trip latency (target < 10μs)
- [ ] BAR register access latency (target < 2μs)
- [ ] VRAM alloc/free latency (target < 50μs)
- [ ] VRAM map/unmap latency (target < 20μs)
- [ ] DMA pin/unpin latency (target < 30μs)
- [ ] MSI-X injection latency (target < 5μs)

### 7.2 Macrobenchmarks
- [ ] CUDA kernel launch overhead (target < 5% vs native)
- [ ] cuBLAS SGEMM 4096x4096 (target > 90% native GFLOPS)
- [ ] cuDNN ResNet-50 training step (target > 85% native)
- [ ] Vulkan frame time (target < 2ms @ 1080p)
- [ ] NVENC 1080p60 encode latency (target < 5ms)
- [ ] NVDEC 1080p60 decode latency (target < 3ms)

### 7.3 Stress Tests
- [ ] 24-hour CUDA compute stress (no errors, no memory leaks)
- [ ] 24-hour graphics stress (Unigine Heaven/Valley loop)
- [ ] 1000x VF create/destroy cycles
- [ ] Concurrent CUDA + graphics workloads
- [ ] Memory pressure test (allocate 90% VRAM, verify OOM handling)

---

## Phase 8: Live Migration Validation

### 8.1 Migration Protocol Tests
- [ ] QUERY phase: source reports capabilities
- [ ] PREPARE phase: destination allocates resources
- [ ] TRANSFER phase: iterative VRAM + GSP state copy
- [ ] COMMIT phase: cutover, guest resumes on destination
- [ ] CLEANUP phase: source releases resources

### 8.2 Migration Under Load
- [ ] Migrate idle vGPU (downtime < 50ms)
- [ ] Migrate running CUDA kernel (downtime < 100ms)
- [ ] Migrate running graphics (no visual artifacts)
- [ ] Migrate with pinned DMA buffers
- [ ] Verify post-migration CUDA/graphics work correctly

### 8.3 Migration State Verification
- [ ] GSP context (registers, PC, stack) restored
- [ ] Page tables identical
- [ ] VRAM contents identical (checksum)
- [ ] BAR register state identical
- [ ] MSI-X table/PBA state identical

---

## Phase 9: Windows Guest Validation

### 9.1 Windows Guest Setup
- [ ] Windows 10 22H2 / Windows 11 23H2 guest
- [ ] VirtIO drivers for network/storage
- [ ] QEMU guest agent
- [ ] UEFI firmware (OVMF with TPM)

### 9.2 NVIDIA Driver Installation
- [ ] Download 570.144 Windows driver
- [ ] Install with `setup.exe -s -noreboot` (silent)
- [ ] Reboot guest
- [ ] Verify in Device Manager: "NVIDIA T4" (Code 0 = working)

### 9.3 Windows Compute Tests
- [ ] `nvidia-smi.exe` shows vGPU
- [ ] CUDA-Z detects GPU, runs device query
- [ ] `vectorAdd.exe` (compiled in guest) executes
- [ ] PyTorch/TensorFlow in Windows (WSL2 or native)

### 9.4 Windows Graphics Tests
- [ ] `dxdiag` shows correct GPU info
- [ ] D3D11 tutorial samples render
- [ ] D3D12 tutorial samples render
- [ ] Vulkan SDK samples run
- [ ] FurMark / 3DMark stress test

### 9.5 Windows Video Tests
- [ ] Media Foundation H.264/HEVC encode
- [ ] Media Foundation H.264/HEVC decode
- [ ] NVENC API direct usage
- [ ] NVDEC API direct usage

---

## Phase 10: Reliability & Production Hardening

### 10.1 Error Handling & Recovery
- [ ] Guest driver crash → host cleans up mdev state
- [ ] Host kernel oops in GSP emulation → watchdog restarts thread
- [ ] Physical GPU reset → host reinitializes GSP emulation
- [ ] IOMMU fault → proper error to guest
- [ ] VRAM OOM → proper CUDA_OUT_OF_MEMORY to guest

### 10.2 Monitoring & Observability
- [ ] Debugfs interface: `/sys/kernel/debug/open-iov-vfio/`
  - [ ] Per-mdev RPC statistics
  - [ ] Latency histograms
  - [ ] Error counters
  - [ ] GSP emulation thread CPU usage
- [ ] Prometheus metrics exporter (optional)
- [ ] Tracepoints for RPC processing

### 10.3 Security Hardening
- [ ] Validate all guest inputs (seq_id, payload sizes, pointers)
- [ ] Isolate mdev memory (no cross-VF access)
- [ ] Rate-limit doorbell writes (prevent DoS)
- [ ] Audit VFIO ioctl permissions

### 10.4 Documentation & Handoff
- [ ] Update `docs/refactored-architecture.md` with as-built details
- [ ] Create `docs/OPERATIONS.md` for deployment
- [ ] Create `docs/TROUBLESHOOTING.md` for common issues
- [ ] Record all firmware addresses used
- [ ] Document test results for each phase

---

## Success Criteria Checklist

### Minimum Viable Product (Phases 0-4)
- [ ] Host loads `open-iov-vfio.ko` with GSP emulation thread
- [ ] mdev creates successfully with correct PCI config space
- [ ] Guest `nvidia.ko` loads, probes vGPU successfully
- [ ] `nvidia-smi` in guest shows vGPU
- [ ] Simple CUDA kernel (`vectorAdd`) executes correctly

### Compute Ready (Phases 5-6)
- [ ] cuBLAS/cuDNN workloads pass numerical verification
- [ ] PyTorch/TensorFlow training works
- [ ] Vulkan/OpenGL rendering works
- [ ] Video encode/decode works
- [ ] Performance > 85% native for compute, > 90% for graphics

### Production Ready (Phases 7-10)
- [ ] 24-hour stress tests pass
- [ ] Live migration works under load
- [ ] Windows guest fully functional
- [ ] Monitoring/observability in place
- [ ] Security audit passed
- [ ] Documentation complete

---

## Dependencies & Prerequisites

| Component | Version | Source |
|-----------|---------|--------|
| Linux Kernel | 6.8+ | Host distro |
| NVIDIA OpenRM | 570.144 | `nvidia-open-gpu-kernel-modules` submodule |
| QEMU | 8.0+ | Host distro / source |
| libvfio-user | Latest | `libvfio-user` repo |
| CUDA Toolkit | 12.4+ | Guest (for compilation) |
| Vulkan SDK | 1.3+ | Guest |
| Windows 10/11 | 22H2/23H2 | Guest ISO |

---

## Tracking

| Phase | Status | Start Date | End Date | Notes |
|-------|--------|------------|----------|-------|
| 0: Setup | ✅ DONE | - | - | Git repo, docs, restore point |
| 1: GSP Emulation Engine | ✅ DONE | | | Core implementation complete |
| 2: PCI/BAR Enhancements | ✅ DONE | | | Config space, MSI-X, BAR regions |
| 3: RPC Handler Completion | ✅ DONE | | | All 12 RPC types with emulation |
| 4: Portable Architecture | ✅ DONE | | | Firmware analyzer, dispatcher, arch/version layers, configs |
| 5: Guest Driver Integration | 🔄 IN PROGRESS | | | Requires reboot to test |
| 6: CUDA Validation | ⏳ PENDING | | | |
| 7: Graphics Validation | ⏳ PENDING | | | |
| 8: Performance Benchmarks | ⏳ PENDING | | | |
| 9: Migration Validation | ⏳ PENDING | | | |
| 10: Windows Guest | ⏳ PENDING | | | |
| 11: Production Hardening | ⏳ PENDING | | | |
| 10: Production Hardening | ⏳ PENDING | | | |

---

*TODO List Version: 1.0*
*Repository: $OPEN_IOV_ROOT*
*Branch: refactor/host-gsp-emulation*
*Base Commit: f9ed5d8*