2013 15" Macbook Pro + GTX780Ti@16Gbps Thunderbolt2 eGPU implementation

[sergiobaschi]

Okay so I have a question for nando4 or anyone else. If I was to get this:

http://www.sonnettech.com/product/echoexpressse2.html

and put in a recent nVidia GPU into it and attach it to a MacBook Air via TB2, (I know this is the MBP forum, but for sake of proof of concept I thought I'd ask this here) would video editing in Adobe Premiere Pro CS6 benefit? Would it find its CUDA cores and allow me to render projects and even live effects faster than the MBA's HD 5000 chipset?

Also, would gaming be improved on OS X?

Thanks.

The MBA doesn't have TB2, that's an issue.

 

[MacVidCards]

TB2 is a 20 Gb/s connection, so it's somewhere between PCIe 2 (16 Gb/s) and PCIe 3 (32Gb/s). The original eGPU post in the Tech Inferno forums showed that the TB2 setup showed at least 90% of the performance when compared to being connected to a PCIe slot in a desktop motherboard.

When TB3 comes out, I'm very sure that the connectors will still be the same. After all, TB1 and TB2 shared the same connectors and TB1 peripherals works over a TB2 port (at TB1 speeds), and TB2 peripherals work over a TB1 port (but at TB1 speeds).

So all the talk about 'needing an adaptor' for TB3 - it's all hogwash to me.

Guess you haven't been keeping up with current events.

https://www.macrumors.com/2014/04/21/thunderbolt-third-generation-details/

Your math is all wrong on throughput as well. TB2 offers 4 lanes of PCIE2. So 1/4 of that available to a GPU in a 2008 Mac Pro.

Great idea to expand a MBP. Insanity as a plan for MP, which is where this thread started life. (And where I stand by my "move the engine into a trailer for a low hood line" analogy)

 

[nando4]

Your math is all wrong on throughput as well. TB2 offers 4 lanes of PCIE2. So 1/4 of that available to a GPU in a 2008 Mac Pro.

TB2 is 8 (x8 2.0) lanes limited on TB<->TB channel to 20Gbps TB1 is 4 lanes (x4 2.0) limited on the TB<->TB channel to 10Gbps.

 

[VanneDC]

Hmm , having sold my trusty 3,1 a few days ago, to fund the purchase of a nMP, this might offer me another alternative. I've got an old gtx480 lying around collecting dust, might just buy one of those sonnet thunderbolt boxes and do some DIY. Even though my MacBook Pro late 2011 is a few years old and only has the TB1 spec, from the results it doesn't look like it would impact performance much over 5-15 percent saturation anyways.
Hmm.. Very interesting...
Obviously this would only be used in windows gaming, but I also have a mac gtx 285 lying around from my old mac desktop..

Might have a look if I can find a cheap Sonnet box. ; )

 

[goMac]

I'm not sure how well 4 way crossfire will run over 6GBps (a lot less if you have any 4K displays or any other thunderbolt devices). Probably about 60-80% performance drop using next years graphics cards. Sad to pay all that money for expensive video cards and PCIe chassis and get such a huge and unnecessary bottleneck.

The Mac Pro has two cards internal, and two TB buses.

So one card on each bus, two cards internal, four way crossfire.

 

[MacVidCards]

TB2 is 8 (x8 2.0) lanes limited on TB<->TB channel to 20Gbps TB1 is 4 lanes (x4 2.0) limited on the TB<->TB channel to 10Gbps.

nMP has a grand total of x8 lanes split amongst all 6 TB ports via 3 controllers.

http://www.anandtech.com/show/7603/mac-pro-review-late-2013/8

TB2 itself specifies x4 lanes of PCIE 2.0.

http://www.anandtech.com/show/7049/intel-thunderbolt-2-everything-you-need-to-know

"Thunderbolt 2/Falcon Ridge still feed off of the same x4 PCIe 2.0 interface as the previous generation designs."

So, if you gave all 3 controllers 100% to an external GPU, it would allow 1/2 the bandwidth of using either of the x16 lane slots in 2008 or later Mac Pro.

And that would be assuming that the external PCIE GPU enclosure could somehow use the lanes that way. And of course you need to subtract the overhead of running all of these things through non-native interfaces.

So, any dreams of running multiple GPUs through less than half of a standard PCIE 2.0 16 lane setup remain exactly that, dreams. Meanwhile, PCIE has already doubled into PCIE 3.0, so you have less than a quarter of that available in nMP for external PCIE.

Not something for serious use with multiple GPUs.

 

[slughead]

The Mac Pro has two cards internal, and two TB buses.

So one card on each bus, two cards internal, four way crossfire.

nMP has a grand total of x8 lanes split amongst all 6 TB ports via 3 controllers.

http://www.anandtech.com/show/7603/mac-pro-review-late-2013/8

TB2 itself specifies x4 lanes of PCIE 2.0.

http://www.anandtech.com/show/7049/intel-thunderbolt-2-everything-you-need-to-know

"Thunderbolt 2/Falcon Ridge still feed off of the same x4 PCIe 2.0 interface as the previous generation designs."

So, if you gave all 3 controllers 100% to an external GPU, it would allow 1/2 the bandwidth of using either of the x16 lane slots in 2008 or later Mac Pro.

And that would be assuming that the external PCIE GPU enclosure could somehow use the lanes that way. And of course you need to subtract the overhead of running all of these things through non-native interfaces.

So, any dreams of running multiple GPUs through less than half of a standard PCIE 2.0 16 lane setup remain exactly that, dreams. Meanwhile, PCIE has already doubled into PCIE 3.0, so you have less than a quarter of that available in nMP for external PCIE.

Not something for serious use with multiple GPUs.

What he said. Also, you can't do crossfire over PCIe with those cards, you must have a bridge (ie: Impossible). Even if you just want it for FCP X, you also need matched video cards for OpenCL according to Rob Art at barefeats.com. So unless you can find some external D700 for sale anywhere, your 4-card Crossfire jerry-rig will have to have all 4 cards running through the TB2 bus.



----------

TB2 is a 20 Gb/s connection, so it's somewhere between PCIe 2 (16 Gb/s) and PCIe 3 (32Gb/s). The original eGPU post in the Tech Inferno forums showed that the TB2 setup showed at least 90% of the performance when compared to being connected to a PCIe slot in a desktop motherboard.

When TB3 comes out, I'm very sure that the connectors will still be the same. After all, TB1 and TB2 shared the same connectors and TB1 peripherals works over a TB2 port (at TB1 speeds), and TB2 peripherals work over a TB1 port (but at TB1 speeds).

So all the talk about 'needing an adaptor' for TB3 - it's all hogwash to me.

Well you are more than welcome to go back and read the details about Alpine ridge (aka TB3). Yes, its a different connector According to Intel. From what I've read elsewhere, TB2 on existing computers will not be forward compatible with Tb3 devices.

The six TB2 ports on the nMP are using a switched 8-lane chip-- see MVC's post above on how that is going to slow things down.

The amount of throttling also depends on the video card and the task. The faster the video card, the more it will slow down.

Keep in mind also that the newest AMD cards use the bus for crossfire... that's not going to go over well through an already throttled TB2 bus.

Face it, you're going to have to get a whole new computer to effectively upgrade the nMP's GPU capabilities.

Edit: Since this thread was moved from the Mac Pro forum, let me say that this is an excellent upgrade to any laptop! It's clearly not a big deal for desktop users though.

 

[nando4]

TB2 itself specifies x4 lanes of PCIE 2.0.

http://www.anandtech.com/show/7049/intel-thunderbolt-2-everything-you-need-to-know

"Thunderbolt 2/Falcon Ridge still feed off of the same x4 PCIe 2.0 interface as the previous generation designs."

Not according to Sonnet who've interfaced a 20Gbps TB2 chip to their III-D enclosure:http://www.sonnettech.com/product/echoexpress3d.html

Expansion Slots:
One x8 mechanical (x8 electrical) PCIe 2.0
One x16 mechanical (x8 electrical) PCIe 2.0
One x8 mechanical (x4 electrical) PCIe 2.0

It would make no sense to interface x8 2.0 to a x4 2.0 chip. x4 2.0 is 16Gbps so would be misleading to advertise it as 20Gbps. x8 2.0 is 32Gbps, which the TB2 channel then limits to 20Gbps and is then legimate advertising.

Would have also been a shock to have a 10Gbps->16Gbps 60% uprate going from TB1 to TB2. Having a 100% improvement, 10Gbps->20Gbps is more worthy of upgrade.

 

[slughead]

It would make no sense to interface x8 2.0 to a x4 2.0 chip. x4 2.0 is 16Gbps so would be misleading to advertise it as 20Gbps. x8 2.0 is 32Gbps, which the TB2 channel then limits to 20Gbps and is then legimate advertising.

Would have also been a shock to have a 10Gbps->16Gbps 60% uprate going from TB1 to TB2. Having a 100% improvement, 10Gbps->20Gbps is more worthy of upgrade.

TB2 is indeed limited to PCIe 2.0 4x speeds. The 20Gbps thing is kind of a lie, Anandtech wasn't able to get more than 11Gbps (1.34GBps) for storage. Likely it factors in the data used for displays et al. Though keep in mind plugging in a 4k display will dramatically reduce your throughput for the whole controller.

Moreover, the nMP specifically only has 8 lanes for all six TB2 ports, which is very limiting at PCIe 2.0.

 

[nando4]

TB2 is indeed limited to PCIe 2.0 4x speeds. The 20Gbps thing is kind of a lie, Anandtech wasn't able to get more than 11Gbps (1.34GBps) for storage. Likely it factors in the data used for displays et al. Though keep in mind plugging in a 4k display will dramatically reduce your throughput for the whole controller.

Moreover, the nMP specifically only has 8 lanes for all six TB2 ports, which is very limiting at PCIe 2.0.

I've asked squinks to do a CUDA-Z bandwidth test on his purported 20Gbps TB2 link. We have numbers for x2 2.0 and 10Gbps so will know for sure which it is once he posts the results.

It may well be that Sonnet to wired up a x8 2.0 electrical link to their 20Gbps TB2 enclosures since they are using a pci-e bridge to share that link with up to 3 slots.

Their single-slot Sonnet Echo Express SEL product is x4 2.0 electrically and is advertised as 20Gbps: http://www.sonnettech.com/product/echoexpresssel.html

I'm leaning towards it being more likely 16Gbps (x4 2.0), which would put a dampener on TB2. Since then it would only really be a 60% improvement over TB1 (10Gbps->16Gbps), rather than the 100% being advertised (10Gbps->20Gbps).

 

[MacVidCards]

Not according to Sonnet who've interfaced a 20Gbps TB2 chip to their III-D enclosure:http://www.sonnettech.com/product/echoexpress3d.html



It would make no sense to interface x8 2.0 to a x4 2.0 chip. x4 2.0 is 16Gbps so would be misleading to advertise it as 20Gbps. x8 2.0 is 32Gbps, which the TB2 channel then limits to 20Gbps and is then legimate advertising.

Would have also been a shock to have a 10Gbps->16Gbps 60% uprate going from TB1 to TB2. Having a 100% improvement, 10Gbps->20Gbps is more worthy of upgrade.

Sonnet doesn't define specs.

Intel did.

Please do some reading, the links I provided give you the answers

 

[slughead]

specs

I was thinking about the 8 lane switch -- I know this could be PCIe 3.0 (8GBps), but because of the way PCIe works, would this be PCIe 2.0 --4GBps total -- to the three TB2 controllers?

 

[Teuthos]

Based on all of this, what does it really cost at present to get a decent external GPU setup? By "decent" I mean powerful enough compared to the rMBP default (Iris Pro) that it justifies the effort and money needed to set it up, so somewhere around 3 times more powerful at least.

 

[nando4]

Based on all of this, what does it really cost at present to get a decent external GPU setup? By "decent" I mean powerful enough compared to the rMBP default (Iris Pro) that it justifies the effort and money needed to set it up, so somewhere around 3 times more powerful at least.

Either go the US$199-10Gbps Thundertek or US$399-16Gbps Sonnet SEL device and adapt as described here. FYI: squinks has done CUDA-Z bandwidth tests and we can confirm TB2 is using a x4 2.0 16Gbps pci-e link. So for eGPU purposes, TB2 is a 60% improvement over TB1, seeing bandwidth increase from 10Gbps to 16Gbps.

Iris Pro iGPU delivers a 3dmark11.gpu=1957 3dmark13.fs=1536 here. These tests are not bandwidth sensitive. On my own expresscard-equipped rig with a GTX770 I was able to return 9634 and 7343 respectively. Meaning, a GTX670 or even GTX660Ti would be seeing a three-fold increase in those bench results over the Iris Pro.

 

[ninja2000]

Either go the US$199-10Gbps Thundertek or US$399-16Gbps Sonnet SEL device and adapt as described here. FYI: squinks has done CUDA-Z bandwidth tests and we can confirm TB2 is using a x4 2.0 16Gbps pci-e link. So for eGPU purposes, TB2 is a 60% improvement over TB1, seeing bandwidth increase from 10Gbps to 16Gbps.

Iris Pro iGPU delivers a 3dmark11.gpu=1957 3dmark13.fs=1536 here. These tests are not bandwidth sensitive. On my own expresscard-equipped rig with a GTX770 I was able to return 9634 and 7343 respectively. Meaning, a GTX670 or even GTX660Ti would be seeing a three-fold increase in those bench results over the Iris Pro.

Nando, just a quick question for you. If I decided to take the egpu route I would want something small and compact to use with my rmbp 13 on its internal screen.
I have looked at the sonnet SEL and I reckon I could modify a maxwell 750TI to fit quite easily. My question would be, could I use an efi windows 8.1 bootcamp with this solution to enable optimus? I don't want t have to use a pcie extender cable as this would defeat the whole purpose of creating a small and portable egpu (for my use).

 

[nando4]

Nando, just a quick question for you. If I decided to take the egpu route I would want something small and compact to use with my rmbp 13 on its internal screen.
I have looked at the sonnet SEL and I reckon I could modify a maxwell 750TI to fit quite easily. My question would be, could I use an efi windows 8.1 bootcamp with this solution to enable optimus? I don't want t have to use a pcie extender cable as this would defeat the whole purpose of creating a small and portable egpu (for my use).

This is a very good question that I do not have a definitive answer for you. There is either a UEFI install (bootcamp 5.1) or MBR install (bootcamp 4.0) when the system has a pcie SSD.

With the rMBPs we've discovered the following about eGPU implementatations:

1. UEFI is plug'n'play for the eGPU side BUT has been problematic in getting the iGPU to work. A user M-LOW usd rEFInd to correct the problem here. Now the iGPU isn't necessary to be functional when using a external LCD with Win8.x however Optimus' accelerated internal LCD mode absolutely must have it working.

2. MBR has the iGPU working but the eGPU requires a PCI Reset Delay circuit and often pci-e preallocation prior to boot via Setup 1.x software. The PCI Reset Delay circuit can be implemented on a pcie riser so as to not modify the Thunderbolt chassis at all. Example of such an implementation is here.

It's feasible that Apple may have updated their firmware to correct (1) and Microsoft update Win8.1 to correct (2). Need someone to provide feedback there. I'll add that it was only in squinks latest eGPU implementation that we discovered the different UEFI/MBR Win8 installation method being used with bootcamp 4.0 and 5.1.

 

[ninja2000]

This is a very good question that I do not have a definitive answer for you. There is either a UEFI install (bootcamp 5.1) or MBR install (bootcamp 4.0) when the system has a pcie SSD.

With the rMBPs we've discovered the following about eGPU implementatations:

1. UEFI is plug'n'play for the eGPU side BUT has been problematic in getting the iGPU to work. A user M-LOW usd rEFInd to correct the problem here. Now the iGPU isn't necessary to be functional when using a external LCD with Win8.x however Optimus' accelerated internal LCD mode absolutely must have it working.

2. MBR has the iGPU working but the eGPU requires a PCI Reset Delay circuit and often pci-e preallocation prior to boot via Setup 1.x software. The PCI Reset Delay circuit can be implemented on a pcie riser so as to not modify the Thunderbolt chassis at all. Example of such an implementation is here.

It's feasible that Apple may have updated their firmware to correct (1) and Microsoft update Win8.1 to correct (2). Need someone to provide feedback there. I'll add that it was only in squinks latest eGPU implementation that we discovered the different UEFI/MBR Win8 installation method being used with bootcamp 4.0 and 5.1.

Thanks for the detailed response nando. I am so tempted to try this but it is a lot of money if it doesn't work like I hope, plus I do not want to have to resort to MBR bootcamp as I love the boot speed and response of the uefi install.

I am in the process of modding an iMac 2011 21" with a 780m and ssds but once that is finished I will take another look into this

 

[nando4]

Thanks for the detailed response nando. I am so tempted to try this but it is a lot of money if it doesn't work like I hope, plus I do not want to have to resort to MBR bootcamp as I love the boot speed and response of the uefi install.

I am in the process of modding an iMac 2011 21" with a 780m and ssds but once that is finished I will take another look into this

Since my last post here I've been alerted by a mid-2013 MBA that he was able to do a UEFI + Win8.1 complete plug and play eGPU installation. Details of which will be forthcoming. Keep an eye out for it on the Tech|Inferno eGPU area: http://forum.techinferno.com/diy-e-gpu-projects/

 

[ninja2000]

Since my last post here I've been alerted by a mid-2013 MBA that he was able to do a UEFI + Win8.1 complete plug and play eGPU installation. Details of which will be forthcoming. Keep an eye out for it on the Tech|Inferno eGPU area: http://forum.techinferno.com/diy-e-gpu-projects/

Wow sounds brilliant! will head over and take a look. I love the portability of my mbpr 13, if I could make a portable little egpu with the SEL I would be over the moon! no more lugging a large PC around for a game session with friends etc.

 

[traveyb]

I've been watching this with interest after getting a Late 2013 15'' rMBP (with 750m), and wanted to reported what I have done so far.

I already have a gaming rig with a 770GTX but would prefer to just use my mac with bootcamp for games. So i've pulled this out and plan to sell the rest of the components.

After some research I decided to go the budget route and opted for the PE4L + Sonnet EchoPro express card (I didn't want to splash out £900 for the OP's chassis without knowing how it all works/how I would get on with it).
Anyway, so my parts arrived today and all I can see is WOW.
Plugged it all in (following a guide on Techinferno), powered it up, booted into my Windows 8.1 bootcamp partion, installed the Nvidia drivers, rebooted and that was it!
Just a quick run of 3DMark Fire Strike and got the below results:
Before

After

Gaming PC

So for about £200, a nice upgrade! I think I will eventually get the chassis, makes it look a lot neater, but for the time being I'm very happy with the results!

 

[VanneDC]

standing by for the full details on the uefi win8.1 plug and play solution with sound hopefully. sounds is the deal breaker for me atm..
my Sonnet Echo Express came in the mail yesterday. now just gotta wait till my next trip to sydney to pick up my old GTX 480 and external PSU
so yeah Nando hanging for more details on the teaser above ;-D

 

[ninja2000]

I've been watching this with interest after getting a Late 2013 15'' rMBP (with 750m), and wanted to reported what I have done so far.

I already have a gaming rig with a 770GTX but would prefer to just use my mac with bootcamp for games. So i've pulled this out and plan to sell the rest of the components.

After some research I decided to go the budget route and opted for the PE4L + Sonnet EchoPro express card (I didn't want to splash out £900 for the OP's chassis without knowing how it all works/how I would get on with it).
Anyway, so my parts arrived today and all I can see is WOW.
Plugged it all in (following a guide on Techinferno), powered it up, booted into my Windows 8.1 bootcamp partion, installed the Nvidia drivers, rebooted and that was it!
Just a quick run of 3DMark Fire Strike and got the below results:
Before

After

Gaming PC

So for about £200, a nice upgrade! I think I will eventually get the chassis, makes it look a lot neater, but for the time being I'm very happy with the results!

Hey traveyb, where did you buy the sonnet and p4l from? did you import them?

 

[traveyb]

Hey traveyb, where did you buy the sonnet and p4l from? did you import them?

I bought the Sonnet from Amazon UK and the P4L direct from the company in Taiwan (ordered Saturday, shipped Wednesday and arrived Thursday (DHL is awesome)).

 

[nando4]

Hey traveyb, where did you buy the sonnet and p4l from? did you import them?

The Sonnet Echo Expresscard Pro + PE4L 2.1b is a 4Gbps (after overhead) Thunderbolt-to-expresscard solution that costs > US$230. This solution was the best value a number of months ago but isn't any longer.

Now there is a new Firmtek Thundertek 10Gbps enclosure available for US$199. So you can get more than double the bandwidth for lower cost. Only negative is a pcie riser is needed to extend the x4 slot to a x16 slot. Details at http://forum.techinferno.com/diy-e-...sonnet-echo-express-iii-d-win8.html#post91184

 

[ninja2000]

The Sonnet Echo Expresscard Pro + PE4L 2.1b is a 4Gbps (after overhead) Thunderbolt-to-expresscard solution that costs > US$230. This solution was the best value a number of months ago but isn't any longer.

Now there is a new Firmtek Thundertek 10Gbps enclosure available for US$199. So you can get more than double the bandwidth for lower cost. Only negative is a pcie riser is needed to extend the x4 slot to a x16 slot. Details at http://forum.techinferno.com/diy-e-...sonnet-echo-express-iii-d-win8.html#post91184

Thanks again nando. Could I cut the pcie slot? and could I use a pico psu? again, my goal is to make the device portable and I have no objections to making a case myself if I can get all the components