Solved: Tesla P40 24GB working in a cMP 5,1 — no cross-flash needed, just `nvflash --gpumode graphics
@dizco @Borowski @keksikuningas following up on my earlier post. I've got a Tesla P40 POSTing and fully enumerating in a cMP 5,1, and the fix turned out to be a single reversible command rather than a vBIOS cross-flash. Writing it up in full in case it helps the next person.
---
System
- cMP 5,1, dual X5690, 128GB RAM
- Kubuntu, kernel 6.14
- RTX 4060 for display output
- Tesla P40 24GB, PG610 SKU 200, stock vBIOS (subsystem 11d9)
Symptom identical to the M40 thread: fans spin up, no startup chime, no POST. Remove the card and it boots normally every time.
The fix
The P40 ships in *compute mode*. Per NVIDIA's own product brief (Table 3), that mode requests a **32GB BAR1**. Graphics mode drops BAR1 to **256MB** and also changes the PCI sub-class from 3D Controller to VGA-Compatible Controller. The cMP's EFI is happy with the latter and not the former.
The switch is stored in the GPU's non-volatile memory, read at power-on, and is **fully reversible** with `--gpumode compute`. It is not a vBIOS replacement.
---
**Step by step**
**1. You need a modern PC with UEFI, temporarily.** The cMP can't enumerate the card, so nvflash can't reach it there. Any board that will POST with the P40 works. Boot Linux (I used a SystemRescue USB stick in UEFI mode) or Windows.
**2. Confirm the card enumerates on that machine.**
```
lspci -nn | grep 10de
```
Look for `1b38` — that's the P40's device ID.
**3. Back up the existing vBIOS first.**
```
./nvflash64 --list
./nvflash64 -i0 --save p40-original.rom
```
Copy it somewhere safe. There's no dual-BIOS on these cards.
**4. Switch the mode.**
```
./nvflash64 -i0 --gpumode graphics
```
Check `--help` first; syntax varies slightly by nvflash build.
**5. Full power down** — not a reboot. The mode is read at power-on.
**6. Verify before moving the card.**
```
lspci -s XX:00.0
cat /sys/bus/pci/devices/0000:XX:00.0/resource
```
You want the class to read **VGA compatible controller**, and BAR1 (second line) to show a **256MB** range instead of zeros.
**7. Install in the cMP and boot.** Chime, POST, boots normally.
**8. Drivers.** If `nvidia-smi` shows your display card but not the P40, check whether you're on the **open** kernel modules — those don't support Pascal. Switch to the proprietary package (`nvidia-driver-XXX` rather than `-open`) and reboot. I'm running 580.95.05 / CUDA 13.0 and both GPUs show correctly.
---
**Power**
The P40 uses a **CPU/EPS 8-pin**, not PCIe 8-pin. NVIDIA supplies a dongle that converts it to two PCIe 8-pin inputs, and Table 5 of the brief lists **only two-cable configurations** — the dongle splits the card's pins 4-and-4 across its two sockets, so both must be fed.
My chain: both logic-board mini 6-pins → dual-mini-6-to-single-PCIe-8-pin adapter → PCIe 8-pin to dual 8-pin splitter → both dongle sockets → card. No Pixlas mod, no second PSU. My display card runs off SATA power, which frees both mini 6-pins for the P40.
---
**The gotcha that cost me hours**
After the mode switch the machine chimed and booted, but the card didn't appear in `lspci` at all. `lspci -vv` on the root port showed:
```
SltSta: ... PresDet-
```
Presence detect is a passive short across two pins at opposite ends of the card edge — no power or firmware involved. `PresDet-` means the card isn't fully seated, full stop. It looked seated. It wasn't. If you hit this, check `PresDet` on the root port before chasing anything else, and note the P40 is a full 266.7mm board that may also ship with chassis extenders fitted to the far end.
---
**Current status**
Both GPUs enumerate, `nvidia-smi` reports the P40's full 24GB. I haven't fitted the blower yet, so I have no sustained thermal or stability data — the brief is clear this is a passively cooled board needing forced airflow, so I'd treat long-run behaviour as unproven for now. Happy to report back once it's cooled properly and I've run some real inference on it.
As far as I can find, nobody had documented a P40 running in a cMP. If anyone tries this on an M40 or another large-BAR Tesla, I'd be interested to know whether the mode switch works there too — it'd be a lot less involved than cross-flashing.
