| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
copies
- new default-off gsp_phys_work_defer_release: a forwarded guest push's
releases are recorded and its GPFIFO entry is held (no consume, no
GP_GET write, no counter publish/stall) until the work item publishes
completion; the service thread then delivers the releases, with a
fallback on failure/timeout (500 ms cap enforced every pump pass)/
pause/quiesce/channel-delete
- new default-off gsp_phys_work_hw_primary: the software copy is skipped
for forwarded ops, so the physical engine authors the data; a once-only
software copy runs before the releases on any hardware failure or when
the request cannot be queued
- release-apply helpers extracted; exactly-once latches; channel-delete
and teardown handling; counters and a work_guard state line
- live: deferred-release run engaged=75 applied=75 timeouts=0 with the
full smoke green; hardware-primary run req=100 ok=100 verify_ok=100
fallback=0 copy_skipped=119 copy_fail=0, smoke ALL OK 64/64 (guest read
data the engine authored), with 19 fail-closed inline software
fallbacks under single-inflight contention; a range-unregistered run
proved the fallback (121 skipped copies, all recovered, smoke green)
- defaults off; the software path and all existing guards unchanged
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- the sema service captures the first eligible copy per push from the
software resolver's own operands (the real push shape mixes guest RAM
with emulated-VRAM shadow addresses, including physical destinations)
- HPA resolution: guest RAM through the shared guest-RAM file page cache,
emulated VRAM through the mapping of its kernel shadow
- new scratch page-table pool: 16 zeroed page-table pages with their
page-directory entries built before any engine access, handing out one
whole 128-byte GMMU PTE fetch line per copy pair (16 PTEs, 512 pairs
per boot, never reused; a PTE written into an already-fetched line is
not observed and there is no invalidate path)
- the backend submits the copy on the module-owned channel and verifies
the result host-side; every miss falls back to the ladder shadow copy,
and the software pump remains authoritative
- live (full smoke, both parameters on): req=42 ok=42 verify_ok=42
verify_fail=0 fallback=0, guest payload roundtrip ALL OK 64/64,
clean latched release; defaults stay off
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- file-backed guest RAM (launcher memory-backend-file share=on) and
in-kernel GPA->HPA translation through the shared file page cache
(read_cache_page + page_to_phys + pfn_valid), opened O_LARGEFILE;
new default-off gsp_phys_guest_mem parameter
- debugfs gsp-phys-guest-ram (path + gpa/off/size ranges) and
gsp-phys-guest-gpa (one pinned GPA per boot); rm-call triggers 6/7 run
host->guest and guest->host 4 KiB CE copies; trigger 8 is a PTE
self-test through a fresh slot
- guest page-table slots are pre-built during ladder stage 3: runtime
PTE writes into an already-walked page-table line are not observed
without a TLB invalidate
- guest RAM layout (q35 4.2, 4 GiB): below-4G GPA == file offset,
above-4G offset = GPA - 0x80000000 (2 GiB PCI hole)
- tests: tools/test/gpage_test.c guest probe; docs + STATE-CURRENT
record the live green (G2H/W2G with semaphore release, guest observed
the write, smoke ALL OK 64/64, clean latched release)
|
| |
|
|
| |
P5 wave-1 shadow execution
|
| |
|
|
|
|
|
|
| |
- STATE-CURRENT snapshot moves to the P3 frontier: the host-freeze latch
and readiness gates are validated, the static identity overlay and the
read-only control forwarding are landed and live-verified
- the physical-backend guest-wiring design is now marked P0-P3 implemented
with the P2/P3 acceptance results
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
physical-backend guest-wiring design
- STATE-CURRENT snapshot 2026-10-01 21:05: the host RM-client ladder now
allocates VAS+PDB+TSG+channel and executes a real 4 KiB VA-mode CE copy
(semaphore released, dst==src) on the physical engine; next is wiring the
live session into the guest-facing mdev/vfio path.
- docs/03-architecture/11-physical-backend-guest-wiring.md: phased design
(P0-P6) for bridging the physical backend into the guest-facing emulation,
with the seam table, lifecycle changes and static-config flow.
- docs/README.md: reading order updated for the new document; execution
status refreshed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fill the GspSystemInfo chipset fields the production client syncs before
the GSP boot instead of leaving them zero:
- clPdbProperties (bitmask in the client's clSyncWithGsp order): bit 0
(config space accessible, from the OS MCFG/ECAM availability), bit 7
and bit 21 (Intel Skylake-family upstream-port setup: ASPM POR list,
root-port nosnoop WAR), bit 13 (upstream port ASPM L1 capable), bit 27
(LTR supported+enabled up to the root port) and bit 32 (IO coherent).
PDB_PROP_CL_UNSUPPORTED_CHIPSET is never assigned by the client, so the
faithful unmatched-chipset value is 0.
- Chipset = 0 (CS_UNKNOWN) for a host bridge with no chipsetInfo[] entry,
the reference's no-match fallback.
- FHBBusInfo/chipsetIDInfo: 10-byte BUSINFO blobs of the first host
bridge of the GPU's domain (device, vendor, subdevice, subvendor,
revision), with the subsystem-capability fallback on chipsetIDInfo.
acpiMethodData is intentionally sent all-zero (bValid=0): no ACPI/AML
calls are made and every consumer tolerates the no-data encoding. All
device lookups use refcounted pci_get_device()/pci_dev_put().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The real-GSP boot path staged the message queue but never queued the two
fire-and-forget init RPCs the production client places in it before the
booter starts. GSP-RM therefore created its GPU object without any system
information and aborted RM init (live GSP_INIT_DONE rpc_result=0x59), with
the resource-manager's object-database lookups asserting from the first
journal record.
Submit function 72 (GSP_SET_SYSTEM_INFO, 928-byte v17_00 payload carrying
the BAR physical bases, domain/bus/slot, PCI IDs, the BAR0 config-mirror
offset, link/FLR capabilities and upstream-port flags read from the
pci_dev) and function 73 (SET_REGISTRY, header-only empty table, which the
server explicitly accepts) through the existing submit-only queue producer
after all stage-3d preconditions and before the GSP falcon reset/Booter
Load kick. The wire layouts are re-declared in the module with per-field
static asserts against the reference ABI; payloads are heap-allocated, so
no new stack frames are introduced.
|
| |
|
|
| |
STATE-CURRENT and the integration page reflect the implemented sequencer, clean-shutdown RPC and static-info probe awaiting the next supervised live cycle.
|
| |
|
|
| |
Adds a minimal msgq producer on the command queue (element layout, checksum, sequence, dma_wmb before writePtr, doorbell) and submits the UNLOADING_GUEST_DRIVER RPC so GSP-RM shuts down and sets the scratch14 handoff bit the Booter Unload gate needs. Release attempts the RPC once before quiescing when a boot ran; on-demand testing is available through the debugfs file gsp-phys-unload-rpc while the module owns the GPU. GSP-RM processor suspend is polled on the GSP mailbox.
|
| |
|
|
| |
STATE-CURRENT and the architecture docs now describe the observed order (FWSEC -> Booter Load -> RISC-V active -> event 0x1002 run-sequencer -> 0x1001 init-done), the four-handoff-bit clean-release constraint, and the --boot/--keep harness contract.
|
| |
|
|
| |
physical-backend lock
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
physical backend
- New 4.2: describe behaviourally how the option-D claim path can
power-gate the board on driver release (platform power resource behind
the root port, off/on sequence via the embedded controller), the two
observed host freezes (no crash record), and the D0 lock now applied
across release -> claim -> rebind: runtime-PM forbid + D3cold disable on
the GPU, its board companion functions and the root port, config-space
verification while claimed and again before rebind, abort with the locks
held on any power-loss signature, runtime-PM restore after the
distribution driver's probe.
- 4.1/4.3: the lock is a required ownership step; returning to the normal
driver is conditional, not assumed; real GSP boot not started.
- 2/3(A): fresh no-unbind evidence - no exported RM/vGPU symbols and no
manager daemon in the stock distribution build.
- 6: driver-API baseline re-verified 64/64; ownership + firmware staging
verified; power-domain lock implemented pending supervised verification.
- 7/8: residual out-of-band power-cut risk, supervised-test requirement,
and the gsp_phys_gpu + gsp_phys_hazard_ack operating constraint.
|
| |
|
|
| |
verified, real boot next
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- new 03-architecture/09-full-system-picture.md: end-state map, GSP boot,
guest init, RPC lifecycle, memory model, work path (bridge and end state),
scheduling, isolation levels, KVM/mdev/VFIO, BAR/engine/channel/GMMU,
portability, performance model, implementation/test status
- new 03-architecture/10-interaction-and-scheduling.md: interaction inventory,
control-plane call path, msgq element lifecycle, channel pump, concurrency,
timing budgets, multi-tenant scheduling, teardown ordering, versioning
- README: mermaid system diagram replaces the ASCII block; reading list
extended and renumbered
- component docs: GSP boot sequence, mdev/VFIO lifecycle, end-state
interaction sequence; STATE-CURRENT links
|
| |
|
|
|
|
|
|
|
|
|
| |
- add docs/03-architecture/08-physical-gpu-integration.md: interfaces that
exist, mechanism options A-E, chosen path (module-owned GPU driven through
its real GSP firmware), sharing policy, decision record
- update 06-host-module-and-hybrid-model, 02-practical-challenges, system
overview, known-knowns, docs/README reading order and CUDA-side docs to
match
- correct stale engine-topology/COPY14 blocker claims across the tree
- add STATE-CURRENT.md as the dated public status snapshot
|
| |
|
|
| |
pitfall
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- admit {GPU VA -> PA} pairs only when the preceding bytes form a valid
client memory descriptor header (type 0/1, CPU+GPU mapped bits), prefer
validated pairs over headerless candidates under first-wins dedup, and
invalidate the translation cache on every rebuild; fixes stale identity
entries shadowing live pool pages and lets the guest UVM channel
bring-up complete on its own
- capture OS event allocations (class 0x79) and post event records
(function 0x1003) into the guest status queue when a channel's busy
notification is consumed, with a broadcast fallback, serialized with
replies via a queue lock
- add BAR0 shadow introspection (dump_bar0/scan_bar0) and guest-side
diagnostic scripts for the VM cycle
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Merge and remove the pre-reboot flat documents, preserving their still-useful
material in the structured tree: debugging workflows and the EPT-livelock
case study (05-operations/02), performance microbenchmarks and A/B
methodology (05-operations/01), the version-upgrade runbook
(05-operations/02 section 4, with corrected host/guest framing), kernel-code
pitfalls (07-developers) and the OpenRM build/load order
(03-architecture/06 section 7). The remaining legacy files were verified
redundant and removed.
* docs/README now records where each migrated topic lives.
* Portability: the config YAML files are called out as design targets that no
code reads yet, with what adding support actually requires today.
* Getting started: the OpenRM step is marked optional (native vGPU-management
workflows only; the emulated path uses the distro driver).
* tools docs: rmtool/rmtop marked as planned, pointing at the implemented
tools/rmq subset.
* wiki: new CUDA-Userspace-Driver deep-dive page (public-safe, links the
canonical userspace/cuda analysis) and index update.
* scripts/check-docs-links.py plus a CI step so relative links cannot rot.
* STATE.md references to removed documents repointed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Status now matches the live state: guest driver probe and nvidia-smi work;
cuInit/device queries work; cuCtxCreate returns 999 because the emulated
engine list is missing the async copy engine (COPY14). The userspace
libcuda segfault is recorded as solved, with its mechanism (an RM
allocation reply must echo request parameters; zeroing them corrupted the
caller's saved frame pointer).
* Correct the emulation claims: only a subset of RM controls are
special-cased; the rest return a success reply with zeroed parameters whose
consumed fields are still being corrected; allocation replies are coherent
but no server-side object model exists.
* Add the eight-point emulation checklist and reply-fidelity ladder as
explicit gates, documented in userspace/cuda and linked from the
architecture, implementation and operations docs.
* Fix operational errors: build/test commands, module load order, mdev type
names, LFS references, debugfs claims, in-repo firmware claims, CI claims,
portability matrix (guest status, config layout, userspace driver facts).
* Mark historical material: one superseded banner per legacy document with a
pointer to its replacement; STATE.md marked as a 2026-09-21 snapshot.
* Wiki: fix the index, add status banners and in-place factual corrections
(msgq transport vs legacy mailbox, BAR definitions, error statuses,
migration, glossary), and note the archived 2023 imports.
|
|
|
Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.
Core principle: emulate the interface, pass through the work.
kernel/ open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
(PCI config space, BAR shadows, MSI-X), GSP boot emulation,
GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
guest-PA discovery) and the semaphore-completion service that
advances guest UVM progress.
userspace/ libiov.so management API and examples.
tools/ guest diagnostics, firmware analysis wrappers, RM probes.
scripts/ stack manager, VM harness, build/load/test automation.
config/ hardware/version profiles (the portability surface).
docs/ documentation tree - start at docs/README.md.
wiki/ long-form firmware and protocol deep dives.
Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.
License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.
|
