1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
|
# GSP 570.144W - Implementation Map for vGPU-IO Integration
## Complete Function Map
### 1. Entry Point & Initialization
| Address | Function | Description |
|---------|----------|-------------|
| 0x04009000 | `gsp_entry` | Bootstrap entry, self-loops at 0x04009070 |
### 2. RPC Dispatcher Hierarchy
#### Primary Dispatcher (Top-Level RPC Type Switch)
| Address | Name | Type | Stack | Notes |
|---------|------|------|-------|-------|
| 0x057a1ac0 | `rpc_dispatch_primary` | Jump table | None | 224 callers |
| 0x057a3ad8 | `primary_jump_table` | 32 entries | - | Index by sub-command |
**RPC Types Handled**: ALLOC(0x01), UNMAP(0x04), PIN(0x05), MIGRATE(0x0B), FREE(0x02), SCHEDULE(0x07)
#### Secondary Dispatcher (Full RPC Implementation)
| Address | Name | Type | Stack | Callers |
|---------|------|------|-------|---------|
| 0x04ed39dc | `rpc_dispatch_secondary` | Full func | Dynamic | 139 |
**RPC Types Handled**: ALL 11 types (MAP, ALLOC, YIELD, SCHEDULE, UNPIN, PIN, UNMAP, FREE, MIGRATE, BAR_TRAP, NOTIFY)
**Handler Entry Points (from branch targets):**
| RPC Type | Handler Address | Function |
|----------|----------------|----------|
| PIN (0x05) | 0x04ed34b8 | `rpc_pin_handler` |
| SCHEDULE (0x07) | 0x04ed4364 | `rpc_schedule_handler_1` |
| SCHEDULE (0x07) | 0x04ed45ac | `rpc_schedule_handler_2` |
| PIN/ALLOC | 0x04ed4764 | `rpc_pin_alloc_handler` |
| ALLOC (0x01) | 0x04ed4930 | `rpc_alloc_handler_1` |
| ALLOC (0x01) | 0x04ed4984 | `rpc_alloc_handler_2` |
| ALLOC (0x01) | 0x04ed49f8 | `rpc_alloc_handler_3` |
| ALLOC/FREE | 0x04ed4a64 | `rpc_alloc_free_handler` |
#### Tertiary Dispatcher
| Address | Name | Stack | Callers |
|---------|------|-------|---------|
| 0x04cc58f8 | `rpc_dispatch_tertiary` | 160 bytes | 53 |
#### Quaternary Dispatcher
| Address | Name | Stack | Callers |
|---------|------|-------|---------|
| 0x05723ef4 | `rpc_dispatch_quaternary` | 96 bytes | 53 |
### 3. Specialized RPC Handlers
| Address | Name | Primary RPC | Calls | Stack |
|---------|------|-------------|-------|-------|
| 0x057e94ec | `rpc_alloc_main` | ALLOC | 64 | - |
| 0x04d00a18 | `rpc_alloc_alt` | ALLOC | - | - |
| 0x051efbf4 | `rpc_schedule_main` | SCHEDULE | 81 | 48 |
| 0x04cb8189 | `rpc_bar_trap_handler` | BAR_TRAP | 4 | - |
### 4. BAR Emulation
| Address | Name | Description |
|---------|------|-------------|
| 0x059be7f4 | `bar_jump_table` | 12-entry encoded table (12 RPC types) |
| 0x04cb8189 | `bar_trap_handler` | BAR register trap handler |
| 0x04bc1665 | `bar_trap_subfunc` | Called from BAR trap handler |
**BAR Strings**: BAR1, Bar1Rest, ForceBarAccess, Bar1Mapping, Bar1Aperture, Bar1Size
### 5. WPR (Write Protected Region) Management
**String References (60 locations):**
- WPR_WRITE_COMPLETE
- WPR_READ_COMPLETE
- WPR_WRITE_TIMEOUT
- WPR_READ_TIMEOUT
- NV_WPR_WRITE_COMPLETE
- NV_WPR_READ_COMPLETE
- NV_WPR_WRITE_TIMEOUT
- NV_WPR_READ_TIMEOUT
### 6. Migration Protocol
**RPC Type**: MIGRATE (0x0B)
**References in dispatchers**: Offsets 280, 316, 324, 448, 656, etc.
### 7. Calling Convention (Decoded)
```assembly
# PROLOGUE
addi sp, sp, -N # N = 32-560 (typically 96, 160, 560)
sd s1, offset(sp) # Save callee-saved s1-s11
sd s2, offset(sp)
...
addi s0, sp, N # Frame pointer (s0 = sp + N)
mv sX, ra # Save return address to s1/s4/s10
# EPILOGUE
mv ra, sX # Restore return address
ld s1, offset(sp) # Restore callee-saved
ld s2, offset(sp)
...
addi sp, sp, N # Deallocate stack
ret # jalr x0, 0(ra)
```
### 8. RPC Protocol (from gsp_comm.h)
| Value | Name | Handler Location |
|-------|------|------------------|
| 0x01 | ALLOC | Primary/Secondary/Tertiary/Quaternary + 0x057e94ec |
| 0x02 | FREE | Primary/Secondary/Tertiary/Quaternary |
| 0x03 | MAP | Secondary/Tertiary/Quaternary |
| 0x04 | UNMAP | Primary/Secondary/Tertiary/Quaternary |
| 0x05 | PIN | Primary/Secondary/Tertiary/Quaternary + 0x04ed34b8 |
| 0x06 | UNPIN | Secondary/Tertiary/Quaternary |
| 0x07 | SCHEDULE | Primary/Secondary/Tertiary/Quaternary + 0x051efbf4 |
| 0x08 | YIELD | Secondary/Tertiary/Quaternary |
| 0x09 | NOTIFY | Secondary/Tertiary/Quaternary |
| 0x0A | BAR_TRAP | Secondary + 0x04cb8189 (via 0x059be7f4) |
| 0x0B | MIGRATE | Primary/Secondary/Tertiary/Quaternary |
| 0xFF | DEBUG | Tertiary |
### 9. Key Data Structures (from migration.h)
```c
// Migration state structure (referenced by RPC_MIGRATE handlers)
struct gsp_migration_state {
uint64_t src_gpu_id;
uint64_t dst_gpu_id;
uint64_t memory_bitmap;
uint64_t page_table_ptr;
uint32_t state;
uint32_t flags;
};
```
### 10. Integration Points for vGPU-IO Kernel
#### gsp_comm.c - RPC Communication
```c
// RPC message structure (matches firmware expectations)
struct gsp_rpc_msg {
uint32_t type; // 0x01-0x0B, 0xFF
uint32_t subcmd; // Sub-command for ALLOC, etc.
uint64_t payload[6]; // Variable payload
uint32_t seq_id;
uint32_t flags;
};
// Primary entry: gsp_rpc_dispatch(msg) -> calls firmware at 0x057a1ac0
```
#### bar_emulation.c - BAR Trapping
```c
// BAR trap handler registration
// Firmware handler at 0x04cb8189 (from jump table 0x059be7f4)
// Called when guest accesses emulated BAR registers
void gsp_bar_trap_handler(uint64_t bar_addr, uint32_t access_type, uint64_t value);
```
#### migration.h - Migration Protocol
```c
// Migration RPC handlers in firmware:
// - MIGRATE (0x0B) at primary dispatcher
// - Save/restore GPU state
// - Transfer page tables and memory bitmaps
```
## Implementation Priority for vGPU-IO
### Phase 1: Core RPC Dispatch (Week 1)
1. Implement `gsp_rpc_dispatch()` calling primary dispatcher at 0x057a1ac0
2. Handle ALLOC/FREE/MAP/UNMAP for memory management
3. Integrate with kernel page allocator
### Phase 2: BAR Emulation (Week 2)
1. Register BAR trap handler at 0x04cb8189
2. Implement `ForceBarAccess` emulation
3. Handle Bar1Rest/Bar1Mapping for GPU reset
### Phase 3: WPR Management (Week 2)
1. Implement WPR_WRITE/READ_COMPLETE handlers
2. Add timeout handling for WPR operations
### Phase 4: Migration (Week 3)
1. Implement MIGRATE (0x0B) RPC handler
2. State save/restore for live migration
3. Page table transfer protocol
### Phase 5: Scheduling & Pinning (Week 3)
1. SCHEDULE/YIELD for vGPU time-slicing
2. PIN/UNPIN for page locking during DMA
## Files for Reference
```
$OPEN_IOV_ROOT/firmware/570-GSP/
├── FINAL_REPORT.md # Complete analysis report
├── IMPLEMENTATION_MAP.md # This file
├── gsp-570.144.fwimage.bin # Raw firmware (28.5 MB)
├── all_call_targets.txt # 134K call targets
├── analysis_results.json # Top 500 functions
├── decompiled_*.c (10 files) # Pseudo-C decompilation
└── analyze_firmware.py # Analysis tools
```
## Next Steps
1. **Decompile secondary dispatcher handlers** (0x04ed34b8, 0x04ed4364, etc.) to pseudo-C
2. **Map jump table at 0x059be7f4** - decode encoded pointers to actual handler addresses
3. **Trace WPR string references** to find WPR management functions
4. **Extract migration protocol** from RPC_MIGRATE handlers
5. **Build kernel integration** using the function addresses above
|
