Netstor GPU Enclosures: NA211A & NA250A

Discussion in 'Mac Pro' started by 5050, May 22, 2014.

  1. 5050 macrumors regular

    Joined:
    May 28, 2009
    #1
    Considering a "starter" PCIe expansion box to house an extra GPU (GTX Titan) for my MacPro5,1.

    Netstor has some interesting solutions. I'm looking at both the NA211A and NA250A. Details below.

    NA211A
    http://www.netstor.com.tw/_03/03_02.php?OTc=

    NA250A
    http://www.netstor.com.tw/_03/03_02.php?OTM=#

    The NA211A lists 20 Gbps bandwidth, and the larger and more expensive NA250A lists 40 Gbps bandwidth. Will this make much difference when running the GTX Titan? I'd be using the extra Titan mostly for work in DaVinci Resolve 10.

    Power will be supplied by an external PSU, so no worries on the more limited PSU on the NA211A.

    Also, I tried Googling around but couldn't find this info. What's the PCIe bandwidth (Gbps) on the MacPro5,1 PCIe internal connections?

    Thanks in advance!

    Jonathan
     
  2. jimj740, May 22, 2014
    Last edited: May 22, 2014

    jimj740 macrumors regular

    jimj740

    Joined:
    Jul 25, 2013
    Location:
    Pining for the company that once gave us Clarus
    #2
    Ummm...

    Did you check out the power supply on the smaller unit?

    Not going to be happy with a Titan.... pushing the absolute limit.

    -JimJ
     
  3. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #3
    As noted in the original post, power to be supplied by external PSU, not internal PSU of PCIe expansion system.
     
  4. mcnallym macrumors 6502a

    Joined:
    Oct 28, 2008
    #4
    The PCI-E lanes in the Mac Pro are

    PSU
    x4
    x4
    x16
    x16
    CPU Tray

    I have said where position of the PSU and CPU tray are so that is very clear which orientation looking at from,

    The x16's are seperate bandwidth, however I seem to recall that the two x4 slots actually share there bandwidth when connecting into the Chipset.

    The NA211 uses a PCI-E x4 card and the NA250 uses a PCI-E x8 card. Just make sure that when you buy the enclosure you are getting the Host Card as well, as some places sell the enclosures and cards seperately.

    Personally would take the NA250 as has the extra connectivity speed. I wouldn't want to spend the money on a GTX Titan and then just providing it with a PCI-E 2.0 x4 slot. You would also be able to power via the PSU built into the Enclosure.
     
  5. mattspace macrumors 6502

    mattspace

    Joined:
    Jun 5, 2013
    Location:
    Australia
    #5
    do these expansion chassis' which run from a host card in a PCI slot have a difference from thunderbolt based ones in terms of drivers for graphics cards?
     
  6. mcnallym macrumors 6502a

    Joined:
    Oct 28, 2008
    #6
    In effect these PCI-E expansion chassis don't rely on the OS to support them. You are installing a PCI-E card into an Internal Slot and the Devices appear as if on the PCI-E bus.

    No need for the Driver to support them in the way that need Thunderbolt aware drivers
     
  7. MacVidCards Suspended

    Joined:
    Nov 17, 2008
    Location:
    Hollywood, CA
    #7
    Actually....I would look carefully. Cubix has reported serious issues in recent OS updates. Read up & ask questions before whipping out Credit Card.
     
  8. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #8
    I apologize in advance for my lack of technical understanding, but what bandwidth in Gbps is achieved with my current Titan in slot 1 (x16)?

    The Netstor enclosures are 20 Gbps and 40 Gbps. Is the Gbps throughput determined by the PCIe card being either x4 or x8? Sorry if this question is rudimentary.
     
  9. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #9
    Also, are there any other PCIe expansion chassis options for GPUs like the Titan other than Cubix and Netstor?
     
  10. mcnallym macrumors 6502a

    Joined:
    Oct 28, 2008
    #10
    That is correct. A PCI-E 2.0 x1 interface is 5Gbs per second

    5 x 4 = 20
    5 x 8 = 40

    The PCI-E x 16 slot provides 80Gbps bandwidth available to your card.
     
  11. fhenry macrumors regular

    Joined:
    Feb 28, 2012
    #11
    it depend for which application, on octane the bandwidth is not the limiting factor, on a 1,1 a dual titan shows the same results than on a 5,1 ! dual pci v1 8x (dual pci v2 4x)
     
  12. Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #12
    Although I'm widely known for being Mr. Cheapo, I'd go for the Dynapower USA Netstor 6-Slot PCIe 3.0 GPU Expansion Enclosure with Power Supply (PCIe 3.0) [ http://www.bhphotovideo.com/c/product/980261-REG/dynapower_usa_na255a_xgpu_netstor_6_slot_pcie.html ] if I intended ever to use more than one double wide GPU in it and/or to buy/build a new system in the future. The NA211A will support only one double wide GPU and is limited to a GPU that draws < 250 watts. The NA255A-XGPU will hold four of them and has a 1200 watt PSU.
     
  13. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #13
    Ahh. So the 20 Gbps bandwidth limit is due to the PCIe x4 host adapter, not the PCIe slots in the expansion box (which has two x8 lanes and one x4)? If I understand this correctly, if the host adapter card was x8, the bandwidth would be 40 Gbps instead of 20 Gbps, making full use of the x8 lane bandwidth in the expansion box?
     
  14. Tutor, May 27, 2014
    Last edited: May 28, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #14
    Eating bits and bites while driving in the fast lane ... .

    I’d rather have a bite of sweet potato pie rather than just a little bit of it because I have a big mouth and the better the pie the wider that mouth can open. Show me a computer interface solution that can mimic that feat.

    A given: 1 Gbit or Gb is equal to 125 megabytes - http://en.wikipedia.org/wiki/Gigabit.

    Yes, with a 40 Gbps total bandwidth limit, and if an x8 host adapter card is seated in an x8 or x16 PCIe slot of at least the same version and there's no other slot usage to account for in an external chassis, then it appears to be enough bandwidth to fully account for one occupied x8 slot's bandwidth; but for future growth please consider the following:

    1) Whether the bee that stung me is big (capitalized) or small (lower case) really matters to me a lot - http://www.translatorscafe.com/cafe/EN/units-converter/data-storage/15-16/ [ 8 gigabits (Gb) = 1 gigabyte (GB); so it takes 8 little bees to equal 1 big bee ] .
    Verified that size does really matter with other reliable sources: http://en.wikipedia.org/wiki/Gigabyte vs. http://en.wikipedia.org/wiki/Gigabit .

    2) Simple is as simple does - Keeping PCI express basics simple: Know your chipset, number of actual lanes per slot, slot version and the same for any other slots, if any - http://www.enthusiastpc.net/articles/00003/1.aspx - and what you're measuring, e.g., total traffic from all cars on the freeway (i.e., all lanes from all slots) vs. the traffic in the lanes exiting to the beach (all lanes related to a single slot). In other words, those "40 Gbps" [for the NA250A] and "5Gbps via laptop’s ExpressCard/34 interface as well as 20Gbps when linking to desktop PC or Mac computer" [for the NA211A] figures in the product summaries refer to measures of the total traffic per second that can be accommodated while moving data across the chassis-to-PC interface.

    3) Note what I take away from the article on PCI Express Basics that I cite in the last paragraph above :
    “What separates 1.x, 2.x and 3.x mostly is the transfer speed per lane:

    • A PCI Express 1.x lane can transfer up to 250MB/s
    • A PCI Express 2.x lane can transfer up to 500MB/s
    • A PCI Express 3.x lane can transfer up to 1GB/s

    These are Megabytes and Gigabytes not bits, so quite fast, even on just a single lane :)

    Obviously a 16 lane connection is still 16 times as fast as a single lane so:

    • PCI Express 1.x does 16 x 250MB/s = 4GB/s on a x16 connection
    • PCI Express 2.x does 16 x 500MB/s = 8GB/s on a x16 connection
    • PCI Express 3.x does 16 x 1GB/s = 16GB/s on a x16 connection .” (Emphasis added and rounding not disparaged.)

    BTW - 40Gb (gigabit) = 5 gigabytes. Five gigabytes is greater than 4 gigabytes (8GB[for x16 V2]/2[to account for use of x8 instead of x16]=4).

    4) Keep in mind, “What’s all in your system’s wallet?” or “How many slots (and what are their characteristics aggregated) in the system or external chassis that you're considering?”

    5) The NA255A-XGPU [ http://www.netstor.com.tw/_03/03_02.php?MTEx ] that I earlier recommended that you consider for future growth has data transfer rates up to 128Gbps between host and GPU enclosure (but that's for a PCIe V3 setup) and has 4x PCIe 3.0 x8 (in x16 connectors). A PCIe V3 x8 slot is about equal to a PCIe V2 x16. The NA250A has 4x PCIe 2.0 x8 (in x16 connectors). Thus the NA255A-XGPU has much greater potential to handle more than 3x more data than the NA250A. The 128 gigabits or 16 gigabytes per second data rate between host and GPU enclosure for the NA255A-XGPU (PCIe 3.0 x8) compares very favorably to the 5 gigabytes per second data rate between host and GPU enclosure for the NA250A (PCIe 2.0 x8), but the NA250A does costs about $470 less ( $2200 - $1730 = $470 ). However that $470 seems to me to be a small difference in price to pay when the choice is between feeding and being feed from 4 four double wide GPUs via a 16 gigabyte per second aggregate data rate vs. doing so via a 5 gigabyte per second aggregate data rate, particularly if your GPUs are PCIe V3 rated, as is a GTX Titan. I do, however, recognize that what can happen at the outer limits doesn't always happen at all times.


    Addendum: Take note of everything in the mix - unlike the case dealing with only internal slots, you talking about an aggregated system involving a couple of other variables - at least (a) an interface card that plugs into a PCIe slot on your PC, (b) a chassis cable that plugs into another (c) interface on the external chassis to communicate with (d) PCIe cards in (e) the PCIe slots within the chassis. So keep this in mind - THE SLOWEST VARIABLE DETERMINES THE FASTEST SPEED. So familiarize yourself with each of them to better ensure that what you get fully satisfies your needs. However, (and as is pointed out by fhenry in post #11, above), if the external GPU(s) are to be used for computation only or mostly (e.g., 3d or animation rendering), it probably won't matter as much if the top aggregate interface transfer rate appears low when compared to the total potential of all of your GPUs as it would matter if you were relying on the GPUs for display purposes.
     
  15. Tutor, May 28, 2014
    Last edited: May 28, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #15
  16. Tutor, May 28, 2014
    Last edited: May 28, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #16
    Your other thread, "2 x GTX Titan in Mac Pro 5,1?," answered some further questions that I had.
     
  17. riggles macrumors 6502

    Joined:
    Dec 2, 2013
    #17
    Tutor, lol, sometimes a person asks for a drink from a faucet and you turn on a fire hose. :D But your dedication to providing a very thorough amount of information is always impressive.
     
  18. registudio macrumors member

    registudio

    Joined:
    Aug 8, 2013
    Location:
    Montreal, QC
    #18

    Yep, I confirm.
    For further info, please check this out:
    http://forums.macrumors.com/showthread.php?t=1713478


    Cheers
    Regis
     
  19. Tutor, May 28, 2014
    Last edited: May 29, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #19
    Here's how I would have the cake you desire to eat.

    This method is not for the timid, those who intend to ensure maximum resale value of their MacPro to the largest audience possible or those who have a MacPro already filled to the brim with storage media that can't be parted with. The mods that I mention could raise the value of your MacPro to alike minded soul looking to do as I have done, but for fear or other hesitation finds it too great to do by his/herself or to one who's simply glad that it was already done for them. Only make this mod at your own risk and with your system not connected to any power source while you complete Steps 2 to 4.

    Step One: Determine how critical retaining the factory optical drive is for you. If it's critical go to Step Two. If it's not critical to you because you, like I, can satisfy optical drive needs with, e.g., an external USB drive like the Samsung Portable DVD Writer Model SE-208 (Costs about $30 thru Amazon - [ http://www.amazon.com/s/ref=nb_sb_n...ords=Samsung+Portable+DVD+Writer+Model+SE-208 ]), then go to Step Two.

    Step Two: Determine your drive storage needs and whether you can use within your Mac Pro a different storage drive location, leaving drive slots2 to 4 bare. If you have only a single drive and it's in Slot1, then proceed on. If you have lots of drives, filling every one of the 4 slots and you cannot use a different storage drive location, then this hack probably will not work for you. It did for me because I use the $120 MaxConnect for MacPro Optical Drive Bay Disk Mounting Assembly for my 3.5 inch Hard Drives [ http://www.maxupgrades.com/istore/index.cfm? fuseaction=product.display&product_ID=302&ParentCat=349 ] and I needed newer, faster, higher capacity ones. So I installed two 4T drives in the lower half of my optical drive bay, placing them on the lower level of the MaxConnect structure. I installed one 4T drive in the traditional drive slot1 location. I removed my old 1T drives, along with their sleds, from drive slots2-4, giving me access to install a double wide GPU in PCIe slot 4. So, I lost 4T storage to gain a faster 12T of storage - Seems like a good compromise to me!

    The GPU installed in slot4 could have been a Titan, for this works with GTX 780 TIs or GTX 680s. See my posts #s 972 and 978, beginning here: http://forums.macrumors.com/showthread.php?t=1333421&page=39 . Post 972 even shows, at the right side of the pic, how I modified the door with a Dremel saw (I cut about a one inch section from the inner frame of the door [after removing it, of course]) to allow me to pass cables from an FSP BoosterX5 [used to power the GPUs] that I installed above the 2x 4T drives in the optical bay.

    Step Three: If you can use within your Mac Pro a different storage drive location, then remove drive sleds 2-4, along with the drives, to make way for installing a double wide GPU in PCIe slot4. To get one of my GTX 680s to work in PCIe slot4, I had to straighten out the lip that forms a curl at the PCIe slot4 location on the end and top of the support at the back of the MacPro, to get that GPU to fit well at that location [there is no such lip on the other end of that support for the GPU that goes in slot1]. For further clarity, I'm referring to the support onto which you turn 2 screws to secure a metal plate to hold the PCIe cards in place. Install the GPU in PCIe slot4. If you have a double wide GPU in PCIe slot1, then you now have two double wide GPUs in your MacPro, plus you have PCIe slots 2and 3 to do with as you please. E.g., you could install a Sonnet Tempo SSD Pro in PCie slot2 and a Blackmagic Decklink Mini Monitor in PCIe slot3 or install a NA211A connection in PCIe slot3 and within the NA211A install a Blackmagic Decklink Mini Monitor and an eSATA/USB 3.0 PCIe card, leaving you with one open PCIe slot in the NA211A for future growth.

    Step Four: Install a BoosterX5 in the top section of the optical drive bay and cut the passage way for its power cables to the GPUs. You'll also need to have cut a small hole in the back grill of the MacPro to pass the BoosterX5's plastic rectangular power cable part through to power the BoosterX5 - just unscrew the plastic rectangular power cable part from the metal part that would otherwise be used to secure it to a rear metal PCIe slot opening. Make sure that the opening that you cut is just a little larger than needed to allow that plastic rectangular power cable part to pass through horizontally; then when you passed it through you can then turn it vertically to help prevent its falling back in to the inner case. I put a little duct tape around the area cut, placing it over some rubber protectors that came with my GPUs to cover the SLI connectors, to avoid injury to the cable. As extra precaution, I also wrap a rubber protector around the part of the power cable that is near the opening and taped it too. You can also re-attach the metal part to help secure the cable - so keep up with those screws.

    Step Five: Power up your system and enjoy.

    PS - Having a GPU in slot4 will not significantly affect its rendering compute ability.
     
  20. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #20
    Tutor, you're as wise as you are generous with your knowledge to the people in this forum. And I'm also a little envious of your hardware setups as well, lol. Thank you for the exhaustive explanation, sources, etc. Despite my short reply, everything you've mentioned here is a great help. Going to take a little time to digest it all!
     
  21. Tutor, May 28, 2014
    Last edited: May 29, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #21
    We truly are 100; were and always will be. Choice matters. So, be sure it's informed.

    Thanks for the compliment, but helping others helps me more than it helps them. It helps me to remain healthy and fully alive. It's like that in everything that we do: the giver receives more of what the giver gives than the receiver receives, whether its joy or pain or any other duality.

    Banish envy because at your request I'll help you, or anyone else, to replicate any of it for the lowest cost possible.

    You're more than welcome. My thanks go out to fellow MacRumor's member Milo for helping to teach me to support what I say.

    If I can be of further assistance to you or you have any questions or concerns regarding any of my posts, just let me know.
     
  22. 5050 thread starter macrumors regular

    Joined:
    May 28, 2009
    #22
    After some thought, it seems impossible to realize the performance I'm seeking without a larger PCIe expansion box (my original priority was maximizing my physical workspace and the idea of adding an enclosure that's the size of another computer wasn't too appealing). So I'm selling one of my Mac Pros to make space for a larger PCIe expansion box. The question is, if I plan to stay on the Mac platform, does the NA255A remain a great investment? The tower Mac Pro caps out at PCIe 2.0 and the new Mac Pros don't make use of PCIe in favor of Thunderbolt. Will that PCIe 3.0 bandwidth ever pay off?

    Given this scenario (or maybe you can conceive of a better option still within the Mac platform?), which is the best investment? NA255A or NA250?

    Thanks again Tutor!
     
  23. Tutor, May 29, 2014
    Last edited: May 30, 2014

    Tutor macrumors 65816

    Tutor

    Joined:
    Jun 25, 2009
    Location:
    Home of the Birmingham Civil Rights Institute
    #23

    If you're absolutely sure that you will be staying solely on the Mac platform, but the NA211A does not have the bandwidth that you need, then the lower priced 4 GPU chassis, i.e., the NA250, makes more sense so long as you don't need more than 1,000W. But keep in mind two things: (1) manufacturer sales hype and (2) that statement that I made earlier, but which on second thought I should have better explained, "I do, however, recognize that what can happen at the outer limits doesn't always happen at all times." By using the phrase "manufacturer sales hype," I'm talking about the tendency which I have seen of some products being hyped achieving a peak, but rarely achieved, level of whatever the sales gurus think will cause a willing purchaser to buy it. That hype could be compounded when you're considering the additive impact of using two or three things (let's say PCIe cards) at the same time, made by different hypers. My last statement also brings into play that observation which I could have done a better job of explaining: "... what can happen at the outer limits doesn't always happen at all times." By that I mean that Card A doesn't always operate at its peak all of the time. The same applies to Card B, and to Card C, etc. But what it also means is the Card A's peak may not necessarily and truly coincide with Card B's peak or Card C's peak (we are, after all, talking about computer time - which can be divvied up more finely than we tend to think of time). In fact, none of their peaks, even if actually achieved, may necessarily actually coincide temporally, particularly when factors such as, but not limited to, the cards' functions and/or the cards' roles in helping to carry out system functions differ. In sum, if you just use addition on peak numbers, you could be led to a conclusion that simply may not reflect what is actually happening. And another question also comes to mind: What if all three cards did saturate your bandwidth? Would the world end? Would your chassis explode? Would it fry your Mac? I doubt that any of these bad things might happen and that you'd perceive any effects of it. There might be a slow down, but we humans might not even perceive it. If you pose your ultimate question to Netstor, listen carefully to what they say and to what they're not saying before you decide between the NA211A and the NA250. But Netstor is likely tell you to ask the card manufacturers. A better route might be to buy the lowest priced option from a company that sells all three solutions, but has an exchange policy that will allow you to swap up if the lowest price option doesn't suite your needs. You'd probably need all of your components, tests and baselines on hand to try them out immediately upon receipt of the chassis.

    The reason why I made my "Hack Your Mac" suggestion to you [i.e., that which appears to me to be the lowest cost solution, unless you have lots of drives] is because when I read your other thread I got the impression that you would be more inclined to stay only on the Mac platform. If you are, like I am, prone to keep your Mac until its died an unrecoverable death, the solution that I gave you in my 2nd to last post allows you to run the cards that it appears to me (from the other thread) that you currently own, along with the eSATA/USB 3.0 card that you had mentioned desiring, along with another Titan, and not increase the footprint of your system much at all.

    In the end, it could be that my being an OS agnostic got the better of me on that first recommendation of the NA255A, except that it does provide you with 1200W of power should you, in the future, use GPUs that require more than 1000W. Since the Titans have a TDP of 250W each, the 1,000W PSU in the NA250 appears to be just enough to run up to four Titans. You should probably consider how long you project that you'll be using your MacPro because that too will affect what, for you, is the best choice economically.

    P.S. I'm not aware of any company providing a 4 slot PCIe chassis solution that allows you to swap interfaces to accommodate a move to Thunderbolt. That situation could, however, change; but I bet that greed may put it on the back burner, if not deep six it.
     
  24. riggles macrumors 6502

    Joined:
    Dec 2, 2013
    #24
    I scanned thru the posts again and didn't see what CUDA app(s) you are using the Titans for. I ask because, depending on the app, there may be the possibility of network GPU rendering. That's the case for me with Octane (in its next release) so I decided not to invest in a Netstor since I could build a dumb PC box or hackintosh to hold more cards for less money.

    If that's not an option and you want to stick with your Mac, which I totally understand, it seems the best Netstor is the one which meets your power requirements. The NA255A has two advantages, the way I see it. 1) higher capacity PSU if you decide to fill it up with four high-end cards. 2) PCIe 3.0 support. This doesn't sound helpful since your MP only supports PCIe 2.0, but should you ever decide to move to a newer PC or build a hackintosh, you would be able to take advantage of that quicker loading time.

    That being said, PCIe 2.0 only effects load time, not how fast your job renders on the cards. So if your render jobs are small, they will still transfer quickly and the difference between 2.0 and 3.0 will be negligible. If they are large, memory-hungry jobs, you are more likely to see a difference because 2.0 will take longer to load into memory.
     
  25. NukeUser macrumors newbie

    Joined:
    Sep 27, 2010
    #25
    Hey riggles- are your Titan Black cards flashed/firmware upgraded? Or are you using them straight out of the box?
     

Share This Page