Emperor of Mac Pros ;-) Dual GTX 970 GPGPU OS X

[Machines]

Ladies and Gentlemen , the emperor has entered the room - a fully functioning, dual high performance Maxwell Compute GPU Mac Pro running OS X . He sips power and breaths fire and was born earlier today . Enjoy and do not forget : The Nehalem Mac Pro remains a very robust machine .

Luxmark SALA (GPU only) seems to be a top score for a real Mac at 4499 .

Finally , a video subsystem that can handle both high performance Open CL and Cuda Core hardware acceleration . And everything powered internally from the factory .

Details are in the pictures , but a summary of the hardware profile is as follows :

Mac Pro 4,1 > 5,1 (Factory 2009 )
Hexacore 2.8 GHz
2 x EVGA GTX 970 Superclocked
Sonnet Tempo Pro Plus with 2 X 256 GB Samsung 850 Pro RAID 0 .

He's been benchmarked in Blender (Cuda) , Luxmark (Open CL) , Black Magic (Drives) and Heaven / Valley .

This cat is getting sleepy and needs to take a long nap . Additional information will be provided tomorrow and I will reply as best as possible to all requests

 

[Prince134]

Nice job , but not quite an empire yet.

compare to this:
https://forums.macrumors.com/attachment.php?attachmentid=473023&d=1400547536

 

[lexR]

should of gone down this route instead of dual R9 280X's... just can't get them to play nicely with my samsung U28U590D screen and OS X and DP 1.2... my last GPU was the GTX 770 and that run the monitor fine at 4K @60Hz no conflicts but as i use FCPX a lot i wanted better open CL performance, it seems like the newer nvidia cards have come on a long way with open CL?

 

[kennyman]

@Machines
Nice Setup, you have both OpenCL and CUDA, which is awesome.

Can you try FCPX BruceX benchmark and post your time?

 

[Synchro3]

you have both OpenCL and CUDA, which is awesome.

That's the point. Nice Setup!

 

[Machines]

Hello , Prince . Thank you very much for inspiring and advising me on how to build my first GPGPU array in a Mac . I used a pair of Sapphire reference AMD 7950 cards and got quite respectable results with FCPX . I build systems locally and soon discovered also a very high performance storage system is a requirement in order to write all the data being processed quickly enough to avoid latency issues . And that means , these days, the best PCIe flash/SSD card available .

With all due respect, my video subsystem is already outperforming your pair of AMD 7970 cards in Open CL . Which is why I called the build , “The Emperor.” Your Luxmark Sala score includes the contribution of a pair of X5690 Xeons . My Sala score is just from the pair of GTX 970 cards . I didn't even think to add my CPU to the mix . I think publicly you stated your Xeons added about a thousand points to the Sala score , which would bring your 5146 score down to 4146 .

If you like , I can add a 12 Core @ 3.46 GHz Tray into this Mac and see what happens

But all the rendering action these days seems to involve compute video arrays and high speed secondary storage systems . CPUs are not an economical contributor to rendering operations . And I'm already recommending to local still image / video clients on a budget that they should use single processor 6 Core Nehalem Mac Pros and simply build up the video and drive arrays to achieve the performance they need at a price tag they won't recoil from . These rigs are not cheap as it is ...

 

[drewsof07]

Is there logic behind the way you routed power? I would think the onboard 6-pin headers would be plenty to power the bottom card, leaving your two SATA-6-pin adapters for the top card. Very cool though.

 

[DEMinSoCAL]

I noticed there is no SLI cable between the video cards. I'm not really familiar with GPGPU, but is the lack of a SLI cable due to using it for GPGPU and not for enhanced video performance?

 

[drewsof07]

I noticed there is no SLI cable between the video cards. I'm not really familiar with GPGPU, but is the lack of a SLI cable due to using it for GPGPU and not for enhanced video performance?

SLI is not supported in OS X

 

[Machines]

Additional profile information :

Video cards used : 2 x EVGA GeForce GTX 970 Superclocked ACX 2.0 04G-P4-2974-KR

OS X version used : 10.10.1 Yosemite .

nVidia Web Driver version used : 343.01.02f01

nVidia Cuda Driver used : 6.5.25

I have had no problems with this combination of drivers so far . Everything seems to work and remains stable so far , but I am worried about the system breaking with an OS X or driver update .

 

[Machines]

@drewsof07

Yes, the logic is , it works under load without a mushroom cloud over the Mac !

Keep in mind , the reason cMP workstations look so pretty under the hood is because Apple routes the massive 980 W from the PSU into motherboard traces (and from there to connectors feeding system components ) . These traces are fragile compared to cables directly from the PSU as is found on the PC side . So, it's best to be conservative and not risk melting the traces when components are placed under load for an extended period . It's safer to spread the provided power over as many points as possible to a component .

Also , I don't think it's just a matter of total watts provided . I think it is more a matter of how the power is distributed across the three points each video card is powered from (the PCIe slot and the two 6 pin PCIe booster power connectors ) .

With previous GPGPU builds involving two AMD 7950 cards, I discovered I needed to provide a potential total of 225 W to each card even though under load each card only consumed around 130 Watts . That required each card to receive power from the PCIe slot and two 6 pin booster cables , even though the max power was never used . The cards would not work unless all three power points were fed . The GTX 970 cards are probably in the same boat .

----------

Total watts used, system wide, when the pair of GTX 970 cards are used at load in rendering does not exceed 450 W in this Mac . CPU was not used that much , intentionally . This high performance build is energy bill friendly thanks to the power sipping nVidia Maxwell GPU chips .

 

[Pressure]

It's quite the illusion, some Geforce GTX 980 and 970 cards are allowed to use 300 Watt (especially the factory overclocked ones) and the official TDP of the cards are 180 Watt for the reference design (and not the 165 Watt which they apparently get away with as a marketing bullet point).

 

[Machines]

@drewsof07

Also, each cMP SATA HDD backplane cannot provide the 75 W of power a 6 pin PCIe power booster cable is rated for . I would not exceed 35 W from that point for safety reasons alone . Remember , these backplane connectors are intended to feed a single HDD . 4TB Mechanical drives consume maybe 9 W maximum . So, in reality , the maximum power provided to each GTX 970 in this build is likely 75 W (PCIe slot) + 75 W (motherboard mini PCIe 6 pin connector ) + 35 W (backplane) = 185 W . Each card draws only a total of 135 W at load , evenly and safely distributed . This is a very conservative build . See the enclosed Max power draw figures in this chart showing this build in action .

 

[Machines]

last post for now as I have a day job lol .

for those of you familiar with blender and Mike Pan's classic BMW benchmark rendering test , here is the result for this dual GTX 970 build using Cuda Core hardware acceleration : 22.85 seconds .

 

[crjackson2134]

are these cards flashed, or are you working without boot screens?

I assume they're not since there was no mention.

 

[Pressure]

The EVGA GTX 970 Superclocked is averaging at 171 Watt and 188 Watt peak.

Not sure were you get the 130 Watt number from.

 

[Machines]

@kennyman

I need to transfer a copy of FCPX onto the drive first . It's a newly created OS installation and it took me forever just to load OS X because the Sonnet card did not like my local USB OS installation flash drives . So, it was a longgggg download from Apple's servers . To be fair to Sonnet, their card is uniformly awesomely fast with all OS X versions from 10.6.8 Snow to 10.10.1 Yosemite . Around 850-950 MB/s sequential reads and writes with the two Samsung 850 PRO SSDs in striped RAID .

 

[Machines]

@crjackson2134

No Maxwell video card has EFI roms yet , so these are just PC Editions . Also, no drivers are in Mac OS X for these cards . All drivers are straight from nVidia and must be loaded first before the cards are installed , or you'll get a black screen for sure at start up. I'm going to enter this build in EVGA's Mod Rigs program , hopefully to encourage their engineers to produce a Mac Edition . If anyone can , they will . EVGA made the GTX 285 and 680 Mac Editions .

 

[Machines]

@Pressure

Bresink Hardware Monitor reports the total power consumed by each card to be around 135 W at load .

See chart in post #13 , Max figures for the Hard Drive (Backplane) number 3 and 4 , PCIe Boost 1 and 2 (six pin mini PCIe mobo connectors) and PCIe slot 1 and 2 figures . These are the six power points the two cards jointly used . Max figures are when the cards were at load .

Also , the power consumption figures were independently verified with an externally attached Kill-A-Watt device . System power consumption was 128 W at idle (this figure includes the 20 W the two GTX 970 consume at idle) . When only the two GPUs were at load rendering , system power consumption peaked at 414 W . 414 – 128 = 286 . 286 / 2 cards = 143 W max consumed additionally by each card at load . Plus , each card idles at 10 W (which we need to add ) so let's say each card consumes an absolute maximum of 153 W at any time .

 

[theitsage]

Good to see someone got a pair of GTX 970s working in a cMP. About 2 months ago, I tried powering a GTX 970 by using a Y splitter to draw power from a single mini PCIe Booster. It worked perfectly for the time I had it.

I need boot screen. So R9 280x remains my GPU of choice for now.

 

[Machines]

@ongnoi

Thanks

All you need to do is install a GT 120 Mac Edition for your UI needs in an available slot and install the GTX 970 again for Compute purposes . Compute GPU cards work best anyways when they have no video channels connected to displays . The GTX 970 only works in Yosemite , though .

 

[Pressure]

@Pressure

Bresink Hardware Monitor reports the total power consumed by each card to be around 135 W at load .

See chart in post #13 , Max figures for the Hard Drive (Backplane) number 3 and 4 , PCIe Boost 1 and 2 (six pin mini PCIe mobo connectors) and PCIe slot 1 and 2 figures . These are the six power points the two cards jointly used . Max figures are when the cards were at load .

Also , the power consumption figures were independently verified with an externally attached Kill-A-Watt device . System power consumption was 128 W at idle (this figure includes the 20 W the two GTX 970 consume at idle) . When only the two GPUs were at load rendering , system power consumption peaked at 414 W . 414 – 128 = 286 . 286 / 2 cards = 143 W max consumed additionally by each card at load . Plus , each card idles at 10 W (which we need to add ) so let's say each card consumes an absolute maximum of 153 W at any time .

That isn't accurate. You need to measure the power from the PCIe bus, like they do at Techreport.

The reference card is allowed to use 180 Watt, an overclocked version will of course be using more. The average load will obviously be less but at peak it will draw more power.

 

[lexR]

appears the dual GTX 970's do out perform the R9 R9280X's in dual format, however being able to use the internal power is by far and a way the better option!! so in theory the FCPX bruceX export time would be the same as dual R9 280x's??

 

[Machines]

@Pressure

What you require for extreme precision, then , is what I lack in my shop – a Mac Pro Nehalem test bench with a bench multimeter and lots of current probes and a custom PCIe riser to get enough access to measure the various rails simultaneously . What I do have is accurate enough to serve me from a pragmatic system build standpoint . Good enough to build GPGPU arrays that are stable and safe at load .

I am perfectly satisfied with my methods – the Mac's built in sensors are accurate enough and are read properly by Hardware Monitor . Coupled with a Kill-A-Watt device that reports real time system consumption wattage , I have an excellent idea of the load various components pull at any given time under controlled circumstances .

If you maintain that each GTX 970 pulls 180 W at load , then two should pull 360 W simultaneously . This entire Mac Pro system pulled a maximum of 414 W when both GPUs were at load . Which means the rest of the Mac would be consuming just 54 W . This is absurd since a cMP PSU alone wastes around 50 W worth of power just by being turned on .

I'm quite certain these cards at load do not exceed more than 153 W each of power in my system . That's credit due to the Green Team at nVidia designing a power sipping GPU .

I might take the opportunity to install these cards in my Windows test bench , where I can evaluate these GPUs more accurately . But it will not shed any light on how they perform in a Mac Pro under OS X .

 

[Machines]

FCPX BruceX score is 101 seconds . This is an average of three scores , as recommended by the author . Both GPUs were involved in the processing , but only at around 35 percent load each . The load was equally balanced simultaneously across both cards . Obviously, the cards are underutilized with the test . Drivers not mature enough ? I'm using outdated drivers because they are known good . Maybe I'll do a little experimenting . Any thoughts ?