summaryrefslogtreecommitdiffstats
path: root/userspace/cuda/570.144/docs/ANALYSIS-egl-engine-selection.md
blob: 9c7ed675c7a6cd2e46128379eedd7dcf800cd648 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
# ANALYSIS - EGL 3D-engine selection wall (570.144 guest userspace)

Status: OPEN (2026-10-23). The EGL-device path initialises (devices enumerate,
`eglInitialize` succeeds, `vendor=NVIDIA`), but `eglMakeCurrent` fails with
`EGL_BAD_ALLOC (0x3003)` because the per-device EGL-core client creator
returns NULL after a lower-layer engine-init call fails.

This document records the *shipped-binary* evidence only (disassembly +
live gdb traces). Method: run the probe under gdb with ASLR disabled
(gdb's default) so library offsets are stable, set breakpoints at the
recorded offsets, and read the arguments/returns.

## 1. The failing chain (libnvidia-eglcore.so.570.144)

```
eglMakeCurrent
  -> NvGlEglApiInit family: per-device EGL-core client creator  (+0xa246f0)
       -> +0xa06ed0  (device/engine selection wrapper)
            -> +0x9fe790  (engine-list walk)
                 list head: *( *(device + 0x28738) + 8 )
                 filters per engine:  engine[0x59] & 0x40
                                      engine[0x70] == 0
                 per-engine callback: *0x70( *(*(engine+0x28738)+0x10) )
                      -> +0xa22000  (callback)
                           rbx = *(reqs + 0xa0)          (device object)
                           mode = *(reqs + 0x20)
                           mode 6 -> +0xa09830
                           mode 7 -> +0xa0a4d0
                           else   -> +0x9fdfd0(device, engine, obj, reqs)
       NULL -> EGL_BAD_ALLOC
```

`+0x9fdfd0` ends in the rejection site:

```
9fe146: rax = *(r14 + 0x10)        ; r14 = *(engine + 0x28738)
9fe14d: rdi = *(rbx + 0x20)        ; rbx = *(reqs + 0xa0)  (device object)
9fe151: call *0x30(rax)            ; vtable slot 0x30
9fe154: test eax,eax
9fe156: je   9fe170                ; 0 == continue (success)
9fe158: free(tmp); return 0        ; non-zero == reject
```

## 2. The rejecting method (libnvidia-glsi.so.570.144)

The vtable at `*(*(engine+0x28738)+0x10)` is a *glsi* device-engine vtable;
its slot 0x30 resolves (live gdb: vtable `0x7ffff71ddfe0`, method
`+0xa29440` in eglcore terms = glsi `+0x5f480` base-adjusted) to the glsi
engine-init entry:

```
glsi +0x5f480(device, reqs):
  5f4c0: fd  = *(*(*(dev+0x108)+0x58))     ; int fd (nvidia device)
  5f4c8: rsi = *(dev+0x118)
  5f4dc: call +0x1aba0(fd, rsi, reqs)      ; FIRST call
  5f52e: call +0x1af20(...)                ; second call
```

Live trace (probe `egl_frame` under gdb, breakpoint at the callback's call
site): the **first** glsi call returns `0xee00001`; the caller propagates it
and the engine is rejected. `0xee00001` is a glsi-internal status produced by
two sites (`glsi +0x31f84`, `+0x33910`) that set flag bit `0x20` in the
device's `+0xb84` word and return "busy"-style; the engine-init state machine
(`glsi +0x301b0` and callees `+0x2fbe0` -> `+0x29760` -> `+0x32110`/`+0x2dc70`)
is what fails.

## 3. What is ruled out

* **Not an RM-control error.** During the failing probe the emulated GSP
  answers every `GSP_RM_CONTROL` with `status=0x0` (including the
  `0x20810107` BINAPI query that was previously suspected; that one is a red
  herring - a real class-0x2081 object on the live session answers
  `NV_OK`/`{0,0}` exactly like the emulation).
* **Not the engine-list shape alone.** The single candidate engine passes the
  walk's own filters (`flag59 & 0x40` set, `f70 == 0`), so the list contains
  a usable engine; the failure is inside the per-engine init.
* **Not ASLR-dependent.** All offsets above are stable under gdb
  (`set disable-randomization on`).

## 4. Next steps (for the successor)

1. Trace `glsi +0x301b0`'s internal calls (returns of `+0x3a6f0`,
   `+0x2ac10`, `+0x2fbe0`, `+0x29760`, `+0x32110`, `+0x2dc70`) to find the
   first non-zero return; that call is the true failing layer.
2. Compare the glsi device-engine state with the same probe on a physical
   Turing GPU (the host's own 570 driver runs the identical userspace; a
   host-side `egl_frame` run with the real GPU succeeds, so a differential
   dump of the engine records / the `+0xb84` flags word will show what the
   emulated device is missing).
3. The engine records glsi consumes come from the kernel driver's device
   data (engine table + class list). The CUDA path builds the analogous
   records from `GET_ENGINES_V2`; the CUDA engine list on this guest is
   known to be complete for GR + CE (see the private analysis area). The
   glsi path may require an entry (e.g. a specific engine kind or a caps
   bit) that the emulated device does not report.

## 5. Reproduction

```
# in the guest (570.144 stock driver):
gcc -O2 -o /tmp/egl_frame /tmp/egl_frame.c -lEGL -lGLESv2   # tools/guest-diag/egl_frame.c
timeout 90 /tmp/egl_frame      # EGL_CURRENT_FAIL err=0x3003

# under gdb (offsets stable, ASLR disabled):
gdb -q -batch -x egl-gdb11.txt ./egl_frame    # catches the vtable call site
```

Breakpoints used (no-ASLR addresses with eglcore base 0x7ffff5200000,
glsi base 0x7ffff7e2a000):

| address | meaning |
|---|---|
| `0x7ffff5bfe151` | eglcore rejection call `*0x30(vtable)` |
| `0x7ffff7e89480` | glsi engine-init entry (+0x5f480) |
| `0x7ffff7e894e1` | after glsi's first inner call (returns 0xee00001) |