| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
content-hash once gate)
The grace-expiry fake path consumed a held CE entry without marking its
GPFIFO slot as consumed for the put epoch, and the replay guard only ran
while the channel was not deferring. A later pin-batch rewind then
re-read the slot, found no recorded identity, and executed the push's
copies against a staging buffer the client had refilled after the faked
release - a deterministic vadd input shift (a[k] == h_a[k + 2 MB]).
- grace-fake consume path now calls svc_replay_note() before advancing;
- the svc_replay_seen() replay skip is evaluated unconditionally: a slot
consumed in the current epoch is skipped on every replay, deferral or
not, until a put wrap clears the mark;
- the once-only identity gate gains a content witness (FNV-1a hash of the
recorded push bytes): the same entry words with different push content
is a new push (clients rewrite push buffers in place) and runs its ops
with a cleared done bitmap; suppression requires identical content;
- snapshot write-back persists the hash array; new drain counter
reident= counts content-mismatch cancellations;
- the grace window stays at 8 s: a faked entry's work is never run, so
the hold is that work's only chance (a 2.5 s experiment made red-class
runs complete with corrupt data instead of timing out).
Verified: module 8e90fc7e..., three consecutive end-to-end acceptance
runs (MAIL + VADD bad=0 checksum=392344090.500000 + STATUS OK).
|
| |
|
|
|
|
| |
A later supervised cycle reproduced the held-entry retry over a
refilled staging buffer, the completed operation was skipped, and the
end-to-end transfer verified byte-for-byte (bad=0, expected checksum).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pump's operand-targeted-scan hold decodes a push, runs every op that
translates, then leaves the entry pending for a retry. The retry
re-decoded the whole push and re-ran the operations that had already
completed. A CE copy reads a client staging buffer the client refills
per chunk, so the retry overwrote a finished chunk with the following
chunk's data (observed: the bridge's vadd input half replaced by the
next 1 MB chunk, first half of the device buffer byte-identical to the
source shifted by exactly 2 MB; the same 1 MB copy logged twice with
different staging contents).
The service now records, per GPFIFO slot, the entry identity plus a
bitmap of the decoded operation indexes whose work has executed for
that identity, and a complete flag for a pass that ended with no
operand pending. svc_run_ops skips already-executed ops on a retry and
marks the rest as they succeed; the pump stores the record
immediately after decode, before any hold path can break, so the
pending path can no longer lose it. A replay of a complete identity
suppresses all ops; a partial identity retries only what is owed.
Also carries the ops_once diagnostic counter in the service snapshot
line and documents the operation-granularity rule in the architecture
doc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The no-claim emulation path now runs a stock CUDA program end to end:
device properties, cudaMalloc, a GPU-authored string kernel read back
byte-for-byte, two 4 MB host-to-device copies, a 1 M-element vectorAdd
executed on the physical GPU through the host bridge, and the
device-to-host copy-back verified element by element. Three consecutive
supervised cycles report MAIL + VADD bad=0 checksum=392344090.500000 +
STATUS OK with identical input/output hashes.
Two data-path correctness fixes make the transfers sound:
- Once-only op execution per GPFIFO entry. The pump replays recently
consumed ring entries when translation batches are injected so missed
releases can land; re-executing a CE copy is not idempotent because
its source is a client staging buffer that may have been refilled.
Slots now record their raw entry words and a replayed entry matching
that identity suppresses operations while still processing releases.
- Per-page translation of virtual operands. svc_copy resolved only the
first page of a VA range and walked the PA linearly; client staging
ranges are frequently split (observed 195 + 61 pages), so the tail of
alternating chunks was read from unrelated memory. Multi-page copies
now re-resolve each 4 KB page through the trusted resolver.
Also carries the earlier uncommitted emulation fixes that got the client
to the launch (QMD address-window gate, counter registration
persistence, 512 MiB emulated FB, bridge slot hygiene) and bounded
diagnostics (sema-bigop, sema-bigcopy, bridge-in/out hashes, daemon
INCHECK/OUTCHECK/dumps).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
registry variant; complete vgsp rename; docs/tooling refresh
- gsp_phys_booter: GR promote bisection modes 4-17 (two-phase zero-field VA,
batched entries, hVirtMemory, bare VA, KMD SKIP_CTXBUFFER_ALLOC channel,
old-interface GPU_INITIALIZE_CTX, FBMEM context buffer, UVM-faithful PA-only,
PATCH/PM auxiliary promotes, server-managed no-promote path); PATCH/PM
geometry queries and buffers; pre-boot SET_REGISTRY profile selection
- gsp_phys_gpu: gsp_phys_rmcall_gr_promote clamp 0..17 and gsp_phys_srv_ctxbuf
parameter; registry image variant with RMSetClientRMAllocatedCtxBuffer=0
- vgsp rename across kernel, scripts, tools and docs (module vgsp.ko,
/sys/kernel/debug/vgsp/, vgsp-<subsystem>: log prefixes)
- risc-v-gsp-rm-debugger user-facing rename and docs/CI refresh; guest-diag
tool updates; README/SBOM/install/Makefile sync
Live status: CE ladder and P6 guest-copy forwarding green; GR engine context
commit still open (all promote shapes accepted or cleanly refused; the channel
stays CTXNOTVALID) - see Handoff-vGPU-IO.md Addendum 21 (local).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- docs/eng-ops.md gains Part II, live physical-backend engineering
operations: hard rules, the full module-parameter matrix, the routine
supervised cycle and clean-release checklist, the debugfs inventory,
how to read gsp-phys-state, a debugging scenario catalogue
(CTXNOTVALID/promote wedge, fail-safe WPR2, unload hangs, guest wedges,
MMU/PTE and PBDMA faults, relay symptoms), a hazard register (crash
causes and debugging side effects), the physical-backend OODA loop, and
the work-context map with porting flows
- architecture, implementation, operations, orientation, getting-started,
portability, ecosystem and reference docs corrected to the current
state: CUDA driver-API smoke green, CE-class work forwarding and
engine-authored releases plus fault relay in place (default off), GR
execution blocked on engine-context validity, the promote wedge and
designed locked fail-safe, and P5 wave-2 subsumed
- internal-reference cleanup in tracked docs (no analysis-source or
local-archive references); historical architecture document bannered
- README reading list and STATE-CURRENT frontier refreshed; new pages
(runbook, green-eye user guide, porting sequence, porting/packaging
notes) linked from the tree
Link check: scripts/check-docs-links.py passes (56 files, no broken links).
|
| |
|
|
| |
physical-backend lock
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- new 03-architecture/09-full-system-picture.md: end-state map, GSP boot,
guest init, RPC lifecycle, memory model, work path (bridge and end state),
scheduling, isolation levels, KVM/mdev/VFIO, BAR/engine/channel/GMMU,
portability, performance model, implementation/test status
- new 03-architecture/10-interaction-and-scheduling.md: interaction inventory,
control-plane call path, msgq element lifecycle, channel pump, concurrency,
timing budgets, multi-tenant scheduling, teardown ordering, versioning
- README: mermaid system diagram replaces the ASCII block; reading list
extended and renumbered
- component docs: GSP boot sequence, mdev/VFIO lifecycle, end-state
interaction sequence; STATE-CURRENT links
|
| |
|
|
|
|
|
|
|
|
|
| |
- add docs/03-architecture/08-physical-gpu-integration.md: interfaces that
exist, mechanism options A-E, chosen path (module-owned GPU driven through
its real GSP firmware), sharing policy, decision record
- update 06-host-module-and-hybrid-model, 02-practical-challenges, system
overview, known-knowns, docs/README reading order and CUDA-side docs to
match
- correct stale engine-topology/COPY14 blocker claims across the tree
- add STATE-CURRENT.md as the dated public status snapshot
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- admit {GPU VA -> PA} pairs only when the preceding bytes form a valid
client memory descriptor header (type 0/1, CPU+GPU mapped bits), prefer
validated pairs over headerless candidates under first-wins dedup, and
invalidate the translation cache on every rebuild; fixes stale identity
entries shadowing live pool pages and lets the guest UVM channel
bring-up complete on its own
- capture OS event allocations (class 0x79) and post event records
(function 0x1003) into the guest status queue when a channel's busy
notification is consumed, with a broadcast fallback, serialized with
replies via a queue lock
- add BAR0 shadow introspection (dump_bar0/scan_bar0) and guest-side
diagnostic scripts for the VM cycle
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Merge and remove the pre-reboot flat documents, preserving their still-useful
material in the structured tree: debugging workflows and the EPT-livelock
case study (05-operations/02), performance microbenchmarks and A/B
methodology (05-operations/01), the version-upgrade runbook
(05-operations/02 section 4, with corrected host/guest framing), kernel-code
pitfalls (07-developers) and the OpenRM build/load order
(03-architecture/06 section 7). The remaining legacy files were verified
redundant and removed.
* docs/README now records where each migrated topic lives.
* Portability: the config YAML files are called out as design targets that no
code reads yet, with what adding support actually requires today.
* Getting started: the OpenRM step is marked optional (native vGPU-management
workflows only; the emulated path uses the distro driver).
* tools docs: rmtool/rmtop marked as planned, pointing at the implemented
tools/rmq subset.
* wiki: new CUDA-Userspace-Driver deep-dive page (public-safe, links the
canonical userspace/cuda analysis) and index update.
* scripts/check-docs-links.py plus a CI step so relative links cannot rot.
* STATE.md references to removed documents repointed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Status now matches the live state: guest driver probe and nvidia-smi work;
cuInit/device queries work; cuCtxCreate returns 999 because the emulated
engine list is missing the async copy engine (COPY14). The userspace
libcuda segfault is recorded as solved, with its mechanism (an RM
allocation reply must echo request parameters; zeroing them corrupted the
caller's saved frame pointer).
* Correct the emulation claims: only a subset of RM controls are
special-cased; the rest return a success reply with zeroed parameters whose
consumed fields are still being corrected; allocation replies are coherent
but no server-side object model exists.
* Add the eight-point emulation checklist and reply-fidelity ladder as
explicit gates, documented in userspace/cuda and linked from the
architecture, implementation and operations docs.
* Fix operational errors: build/test commands, module load order, mdev type
names, LFS references, debugfs claims, in-repo firmware claims, CI claims,
portability matrix (guest status, config layout, userspace driver facts).
* Mark historical material: one superseded banner per legacy document with a
pointer to its replacement; STATE.md marked as a 2026-09-21 snapshot.
* Wiki: fix the index, add status banners and in-place factual corrections
(msgq transport vs legacy mailbox, BAR definitions, error statuses,
migration, glossary), and note the archived 2023 imports.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
