diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-20 01:56:24 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-20 01:56:24 -0400 |
| commit | c72f89483dec5507024efc26d945bf0674c66db7 (patch) | |
| tree | b3810912ae4cc173e3bf5cf3e82e817d8be19c8e /docs/tools | |
| parent | ae7848021ec10dd25c48f3244c4bca7adc79f2d5 (diff) | |
| download | open-iov-c72f89483dec5507024efc26d945bf0674c66db7.tar.gz open-iov-c72f89483dec5507024efc26d945bf0674c66db7.zip | |
vgsp: read the status-queue consumption cursor from the command queue (SWAP_RX)
Under MSGQ_FLAGS_SWAP_RX (both sides set it; flags=1 in both headers) the
guest driver's msgqRxMarkConsumed writes its STATUS readPtr to
pOurRxHdr->readPtr, and the driver's pOurRxHdr is the COMMAND-queue buffer's
RX header (msgq.c msgqRxLink swap branch: pReadOutgoing = &pOurRxHdr->readPtr
with pOurRxHdr from msgqTxCreate's pBackingStore = pCommandQueue). The
status queue's own +rxHdrOff is the mirror we publish - our command-queue
read position.
The status free-space check read the mirror, so it tracked our own writes:
with the guest mid-drain the ring was declared full (free=0, peer=0,
write=62/63), replies were written into it anyway, unread records were
overwritten, and the guest RM read a newer record - GspMsgQueueReceiveStatus:
Bad sequence number. Expected 447 got 510. Possible memory corruption. -
after which every RPC failed (status 0x25) and the first client's cudaMalloc
returned device busy or unavailable. That is the boot lottery: on a lost
boot the first client (mail) FAILs and the boot is poisoned for every later
client.
Evidence (r330 vs r332): r330 mail FAILed at the first cudaMalloc and only
one channel unregistered in the whole boot (the guest RM never tore its
channels down after the status desync); with the fix r332 mail + matmul pass
and every client's channels unregister (79 unregisters, one session flush
per client).
Also adds bounded diagnostics: sema-d2hmap (per-page destination resolution
of >=1MB host-dest copies) and a raised DIVERGE witness budget.
Diffstat (limited to 'docs/tools')
0 files changed, 0 insertions, 0 deletions
