Open-IOV: initial public release

Open-IOV virtualizes NVIDIA GPUs so that unmodified guest drivers run with
real GPU functionality - CUDA first, 3D and video to follow - on unmodified
hosts, without a proprietary vGPU license and without patching any driver,
firmware or hypervisor.

Core principle: emulate the interface, pass through the work.

  kernel/     open-iov-vfio.ko - host VFIO mdev driver: virtual GPU device
              (PCI config space, BAR shadows, MSI-X), GSP boot emulation,
              GSP-RM message-queue/RPC emulation, memory model (GPU-VA to
              guest-PA discovery) and the semaphore-completion service that
              advances guest UVM progress.
  userspace/  libiov.so management API and examples.
  tools/      guest diagnostics, firmware analysis wrappers, RM probes.
  scripts/    stack manager, VM harness, build/load/test automation.
  config/     hardware/version profiles (the portability surface).
  docs/       documentation tree - start at docs/README.md.
  wiki/       long-form firmware and protocol deep dives.

Status: initialization works end to end for a stock 570.144 Linux guest on
Turing (TU104): driver probe, nvidia-smi, the full GSP-RM control battery,
object creation and the early CUDA/UVM start-up path. GPU work forwarding -
the step that makes CUDA kernels execute on real hardware - is the active
work item; see docs/04-implementation/01-known-knowns-and-unknowns.md.

License: BSD-3-Clause for the project; GPL-2.0 for kernel modules that link
GPL-only kernel interfaces. See LICENSE and docs/09-reference/legal-and-licensing.md.

1 files changed, 150 insertions, 0 deletions

diff --git a/docs/tools/rmtool.md b/docs/tools/rmtool.md
new file mode 100644
index 0000000..a7ec2c6
--- /dev/null
+++ b/docs/tools/rmtool.md
@@ -0,0 +1,150 @@
+# rmtool - RM API Swiss Army Knife
+
+## Overview
+`rmtool` is a comprehensive userspace utility for interacting with the NVIDIA Resource Manager (RM) API. It provides direct access to RM internal state, engine control, memory management, and performance telemetry. Built on top of driver components studied through reverse engineering and
+decompilation (referenced by component).
+
+## Source Code Reuse
+| Component | Source Location | Purpose |
+|-----------|-----------------|---------|
+| nvidia-smi component | CLI framework, NV2080 ioctls, GPU enumeration |
+| gpumon / perfmon component | Performance monitoring, sampling infrastructure |
+| NV2080 control component | RM control interfaces (classes, methods, events) |
+| Resource server component | Resource Server (resserve) API |
+| CUDA software (CUDASW) component | CUDA internal layer |
+| Resource manager component | Resman API |
+
+## Core Commands
+
+### GPU Enumeration & Topology
+```bash
+rmtool gpu list                    # List all GPUs with PCI info
+rmtool gpu topology                # Show GPU topology (NVLink, PCIe, PCI IDs)
+rmtool gpu info -d 0000:65:00.0    # Detailed GPU info (like nvidia-smi -q)
+```
+
+### Engine Management
+```bash
+rmtool engine list                 # List all engines (GR, CE, NVDEC, NVENC, NVJPG, OFA)
+rmtool engine info -d 0 -e gr      # Engine details (registers, capabilities)
+rmtool engine allocate -d 0 -e gr -c 4  # Allocate 4 GR channels
+rmtool engine start -d 0 -e gr -c 0     # Start engine channel
+rmtool engine stop -d 0 -e gr -c 0      # Stop engine channel
+rmtool engine config -d 0 -e nvenc --preset=h264 --bitrate=10M
+```
+
+### Channel & Context Control
+```bash
+rmtool channel list -d 0           # List all channels on GPU
+rmtool channel create -d 0 -e gr -t compute  # Create compute channel
+rmtool channel submit -d 0 -c 0 --pushbuf=work.bin  # Submit work
+rmtool channel status -d 0 -c 0    # Channel status (busy, pending, error)
+rmtool context list -d 0           # List all contexts
+rmtool context info -d 0 -c 0      # Context details (page tables, memory)
+```
+
+### Memory Operations
+```bash
+rmtool mem alloc -d 0 --size=1G --type=video    # Allocate VRAM
+rmtool mem alloc -d 0 --size=512M --type=system # Allocate sysmem
+rmtool mem map -d 0 -h 0x1234 --gpu-va=0x100000 # Map to GPU VA
+rmtool mem unmap -d 0 -h 0x1234                 # Unmap
+rmtool mem pin -d 0 -h 0x1234                   # Pin for DMA
+rmtool mem info -d 0 -h 0x1234                  # Allocation details
+rmtool mem stats -d 0                           # VRAM/sysmem usage breakdown
+```
+
+### Performance Sampling
+```bash
+rmtool sample engine -d 0 -e gr --interval=100ms --count=100
+rmtool sample clocks -d 0 --interval=1s
+rmtool sample power -d 0 --interval=1s
+rmtool sample thermal -d 0 --interval=1s
+rmtool sample pcie -d 0 --interval=1s
+```
+
+### NV2080 Direct Control
+```bash
+rmtool nv2080 class list -d 0                # List all NV2080 classes
+rmtool nv2080 method call -d 0 -c 0x906D -m 0x100 --params=...
+rmtool nv2080 event subscribe -d 0 -c 0x906D -e 0x100
+rmtool nv2080 event wait -d 0 -c 0x906D -e 0x100 --timeout=5000
+```
+
+### Resource Server (resserve) Integration
+```bash
+rmtool resserve status                       # Resource Server status
+rmtool resserve reservation list             # List all reservations
+rmtool resserve reservation create --gpu=0 --engine=gr --size=4
+rmtool resserve reservation release --id=123
+```
+
+### CUDA Internal Layer (CUDASW)
+```bash
+rmtool cudasw context list                   # CUDA contexts
+rmtool cudasw module list                    # Loaded CUDA modules
+rmtool cudasw stream list -c 0               # Streams in context
+rmtool cudasw event list -c 0                # Events in context
+```
+
+### Resource Manager (Resman)
+```bash
+rmtool resman client list                    # Resman clients
+rmtool resman resource list -c 0             # Resources owned by client
+rmtool resman heap info -d 0                 # Heap state
+```
+
+### GSP-RM Telemetry
+```bash
+rmtool gsp rpc-stats -d 0                    # RPC queue depths, latency
+rmtool gsp doorbell -d 0                     # Doorbell rates
+rmtool gsp firmware -d 0                     # Firmware version, threads
+rmtool gsp wpr -d 0                          # WPR state
+```
+
+## Output Formats
+```bash
+rmtool gpu list --format=json
+rmtool engine list --format=csv
+rmtool sample engine --format=prometheus
+rmtool mem stats --format=influx
+```
+
+## Build
+```bash
+cd open-iov/tools/rmtool
+make
+# Links against: libnvidia-rmapi.so (reused RM API component), libpciaccess, libnuma
+```
+
+## Architecture
+```
+rmtool/
+├── src/
+│   ├── main.c              # CLI parsing, command dispatch
+│   ├── gpu.c               # GPU enumeration, topology
+│   ├── engine.c            # Engine allocation/control
+│   ├── channel.c           # Channel/context management
+│   ├── memory.c            # Memory alloc/map/pin
+│   ├── sample.c            # Performance sampling
+│   ├── nv2080.c            # NV2080 direct control
+│   ├── resserve.c          # Resource Server client
+│   ├── cudasw.c            # CUDA Internal Layer
+│   ├── resman.c            # Resource Manager
+│   ├── gsp.c               # GSP-RM telemetry
+│   └── output.c            # JSON/CSV/Prometheus formatters
+├── include/
+│   └── rmtool.h            # Internal headers
+├── lib/
+│   ├── librmapi.a          # Recycled RM API code
+│   └── libgpumon.a         # Recycled perfmon code
+└── Makefile
+```
+
+## Future Extensions
+- **Remote gRPC/REST API** for fleet management
+- **Policy Engine** for automated engine allocation
+- **Migration Coordination** via resserve
+- **Telemetry Export** (Prometheus, OpenTelemetry, InfluxDB)
+- **Guest Introspection** via vGPU VFIO interface
+- **Workload Profiling** with CUDA/GR engine correlation
\ No newline at end of file