diff options
| author | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 07:54:29 -0400 |
|---|---|---|
| committer | Arthur Hanson Rasmusson <arthur@vgpu.io> | 2026-10-08 07:54:29 -0400 |
| commit | 54cb61b70968c7c8cbc41f669041f4e28d5aa7c4 (patch) | |
| tree | 9b1b4acad8d5438ea122d0948f3eea246ac99b93 /kernel/gsp_sema_service.c | |
| parent | 5bb2d53ec14f0d802726bc879b994c0286f85415 (diff) | |
| download | open-iov-54cb61b70968c7c8cbc41f669041f4e28d5aa7c4.tar.gz open-iov-54cb61b70968c7c8cbc41f669041f4e28d5aa7c4.zip | |
sema: fix GR counter-slot mapping to fixed pool total (20)
The counter slot for GR rank r is 0x20400fec0 + (TOTAL-1-r)*0x10 with
TOTAL the fixed pool total. The autoreg used a live high-water channel
count instead; channels register progressively, so an early pass with a
partial count (16) registered the ranks present at the wrong slots and
every later rank's correct slot was then rejected as owned. Result:
ranks 10..19 never got a counter, no counter was ever published, and the
client's userspace tracker never advanced (cudaMalloc stall while the
guest RM polled MC_SERVICE_INTERRUPTS).
Validated on a live boot: 20/20 slots registered with 0 collisions
(previously ranks 6..9 installed at the n_gr=16 mapping and ranks 10..19
rejected).
Diffstat (limited to 'kernel/gsp_sema_service.c')
| -rw-r--r-- | kernel/gsp_sema_service.c | 32 |
1 files changed, 18 insertions, 14 deletions
diff --git a/kernel/gsp_sema_service.c b/kernel/gsp_sema_service.c index dc4b7b1..48c070b 100644 --- a/kernel/gsp_sema_service.c +++ b/kernel/gsp_sema_service.c @@ -120,6 +120,18 @@ struct sema_map_entry { /* ------------------------------------------------------------------ */ #define SVC_MAX_CHANNELS 128 +/* + * Fixed GR completion-counter pool total for the emulated device profile: + * the client allocates this many GR channels and zips them in ascending + * handle order onto descending 0x10-byte counter slots, so rank r owns + * slot 0x20400fec0 + (TOTAL-1-r)*0x10. The live channel count must never + * be used here: channels register progressively, and an early pass with a + * partial count claims the slots that the full mapping needs for later + * ranks (observed: ranks 6..9 installed at the n_gr=16 mapping, then every + * later rank's correct slot was already owned and rejected, no GR counter + * was ever published and the client's tracker never advanced). + */ +#define SVC_GR_COUNTER_TOTAL 20 #define SVC_GPFIFO_ENTRY_SIZE 8 /* GPFIFO slots tracked for the increment-release one-shot bitmap. The rings * are 1024 entries in practice; the cap only bounds the per-channel state. */ @@ -10357,31 +10369,23 @@ static void svc_pump_channels(struct nv_vgpu_device *vgpu) /* Only ever called from the service thread. */ static struct svc_channel local[SVC_MAX_CHANNELS]; /* - * Total GR channels the client ever had in the current generation. - * The counter-slot layout is fixed when the client allocates all its - * channels, so the slot must not be recomputed with a shrinking live - * count (a teardown pass would re-target the wrong slot). + * Fixed GR counter pool total (see SVC_GR_COUNTER_TOTAL). The live + * count grows as the client registers channels, so it must not be + * used to derive counter slots: an early pass with a partial count + * poisons every later rank's slot. */ - static int svc_gr_max; unsigned long flags; int i, k, n = 0, n_gr = 0; bool stalled = false; spin_lock_irqsave(&svc.chan_lock, flags); for (i = 0; i < SVC_MAX_CHANNELS; i++) { - if (svc.chan[i].valid) { + if (svc.chan[i].valid) local[n++] = svc.chan[i]; - if (!svc.chan[i].engine_type) - n_gr++; - } } spin_unlock_irqrestore(&svc.chan_lock, flags); - if (n_gr > svc_gr_max) - svc_gr_max = n_gr; - else if (!n_gr) - svc_gr_max = 0; - n_gr = svc_gr_max; + n_gr = SVC_GR_COUNTER_TOTAL; /* * P6b wave 1.5: a hold whose channel is absent from this pass's |
