I have a Mac Pro 5,1 (late 2009) which was sold to me with a Windows model NVidia GeForce EVGA GTX 680 4GB FTW graphics card. I don't know what most of those acronyms mean, but I was told it had been custom-flashed to work with Mac OS X, and so gave more power and VRAM for the price. The machine came with Mavericks but I rolled back to Mountain Lion, because I use software that did not yet support Mavericks at the time.
The card works very well under Mountain Lion 10.8.5, but I have encountered problems now that I've been wanting to move to Yosemite.
The card has two DVI outputs, one DVI-I and one DVI-D. I have two displays, a HD one with DVI support, and a smaller old one which only supports VGA. I have a VGA adapter plugged into the card's DVI-I port supporting my smaller display, and the primary display is plugged directly into the DVI-D port.
Under 10.8.5 this works fine, but when I have tried booting into 10.9 Mavericks or 10.10 Yosemite (from internal or external disks), the primary display will initially show the boot screen, and then it just shows digital garbage. The smaller display on the upper DVI-I port will still work fine, which allows me to boot into the OS and install the NVidia web driver for the card, but that makes no difference, even after rebooting. Plugging the primary display into the DVI-I port enables it to work, so it's not a problem with the display itself, but then I'm without my secondary display, and it's a massive pain to get into the back of the computer and switch the cables around every time I want to use an app that only supports 10.9+.
I can only surmise that the flashed firmware on the card needs updating, but I know nothing about this and wouldn't know where to start. The seller is unavailable, and successive Google expeditions have produced no useful results, except to point to vaguely related threads on this site and Hackintosh sites. I have also tried running Yosemite under Parallels Desktop (which would be preferable as I've no real desire to change my operating system, except that newer software demands it), but I can't get it to boot – an unrelated issue.
If anyone has any advice I would be very grateful!