From 624438bb0901680f416b879727b30a411dd3d78b Mon Sep 17 00:00:00 2001 From: Arthur Hanson Rasmusson Date: Wed, 14 Oct 2026 11:39:30 -0400 Subject: vgsp-sema: author the PMA scrubber channel's progress (third-client scrub freeze) The guest PMA scrubber (in-guest CeUtils CE channel, 4096 GPFIFO entries) waits on its channel buffer's finish payload (offset 0x6c004) after the third CUDA session's pages must be scrubbed; nothing in the emulation authored it, so cudaMalloc froze in scrubCheckAndWaitForSize. - channel_add: detect the 4096-entry scrubber channel and stash its USERD PA/ring geometry (the channel itself is left untouched: no rank change, no pump change - the colliding numeric-handle FREE may still unregister it, the geometry survives). - sema_scrubbuf_add debug param: register the finish/host-semaphore word PAs the guest scanner resolves (vmallocinfo + kcore walk), because the RM kernel VA space collides numerically with the client CE window in the global VA map and cannot be resolved through it. - svc_scrub_pass (service thread, per pass): from the channel's own USERD GPPut, track submitted scrub blocks across wraps, then author USERD GP_GET = put, host semaphore = put and finish payload = submitted. Forward-only, idempotent, zero writes while the scrubber idles. Live-validated mechanism (r137 VM): authoring the finish word at the kcore- resolved PA let the frozen victory_matmul leave scrubCheckAndWaitForSize and proceed to its launch. --- kernel/main.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) (limited to 'kernel/main.c') diff --git a/kernel/main.c b/kernel/main.c index ac970f1..d0308c4 100644 --- a/kernel/main.c +++ b/kernel/main.c @@ -2543,6 +2543,37 @@ static const struct kernel_param_ops nv_vgpu_sema_probe_ops = { module_param_cb(sema_probe, &nv_vgpu_sema_probe_ops, NULL, 0200); MODULE_PARM_DESC(sema_probe, "Dump sema-service map + slot values for a GPU VA"); +/* + * Register the guest PMA scrubber channel buffer words (finish payload PA, + * host-semaphore PA) the guest scanner resolved: + * echo 0x11a8ff004,0x11a8ff000 > .../parameters/sema_scrubbuf_add + */ +static int nv_vgpu_sema_scrubbuf_set(const char *val, + const struct kernel_param *kp) +{ + char buf[64]; + char *comma; + u64 fin_pa = 0, hsem_pa = 0; + + strscpy(buf, val, sizeof(buf)); + comma = strchr(buf, ','); + if (!comma) + return -EINVAL; + *comma = '\0'; + if (kstrtoull(buf, 0, &fin_pa) || kstrtoull(comma + 1, 0, &hsem_pa)) + return -EINVAL; + + return gsp_sema_service_scrubbuf_add(fin_pa, hsem_pa); +} + +static const struct kernel_param_ops nv_vgpu_sema_scrubbuf_ops = { + .set = nv_vgpu_sema_scrubbuf_set, +}; + +module_param_cb(sema_scrubbuf_add, &nv_vgpu_sema_scrubbuf_ops, NULL, 0200); +MODULE_PARM_DESC(sema_scrubbuf_add, + "Register the PMA scrubber finish/host-sema PAs (hex fin,hex hsem)"); + /* * Debug: run the page-table walks for a GPU VA with per-level tracing, * bypassing the pin/lat/pair map (see gsp_sema_service_walk_probe): -- cgit v1.2.3