Migrating your existing storage to the new Mac Pro

Discussion in 'Mac Pro' started by VirtualRain, Aug 22, 2013.

  1. VirtualRain macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #1
    Here's another essay on migrating your existing storage to the new Mac Pro :)

    What’s the problem:

    One of the chief objections from the slots ‘n bays crowd about the new Mac Pro is the cost of migrating their existing storage strategy. It’s a valid complaint if not a bit exaggerated. Sure, there are no internal SATA bays in the new Mac Pro but of all the challenges the new Mac Pro presents to existing Mac Pro customers, storage migration is probably the easiest to solve given the plethora of external storage solutions that currently exist (USB 3, NAS, and to a less extent TB). So I’m not sure the extent to which this complaint is an emotional response based on resistance to change.

    I think a lot of people are jumping to a conclusion that goes something like this… “I have 4 drives in my current Mac Pro so I need a $1000 Promise R4 Thunderbolt enclosure to accommodate my needs”. The fact is this is probably far from the truth.

    I can’t pretend to know everyone’s storage requirements, and I’m sure there are some situations which simply can’t migrate easily to the new Mac Pro at reasonable cost. For those situations, it will be interesting to hear about them.

    However, in a lot of cases, people serious about migrating and willing to relook at their storage requirements with an eye for what can be done to simplify and minimize costs should not have serious issues and maybe this thread can help those folks.

    Anytime you buy a new computer, it’s a great opportunity to look as simplifying and consolidating your storage requirements. Look at consolidating to fewer drives and being honest with yourself about the performance needs for any particular category of data. It may be no longer practical or necessary to run your old collection of four 1TB drives in RAID0 for your iTunes library (for example). It’s probably time to consolidate that and other less performance sensitive data into a single larger disk that can easily plug in with a $50 USB3 interface. Yes, it will require change, and a small investment, but this is a far cry from needing a $1000 Promise R4.

    As a side bar, not everyone here in this forum is concerned about absolute minimal expense. Some (like yours truly) are spending way more on storage media (like SSDs or large arrays with cached RAID controllers) than they really need. These enthusiasts are not going to mind migrating to a new form factor nearly as much as the seriously price-sensitive buyer. This thread is not really focused on the deep pocket enthusiast.

    What are the solutions:

    When you look at the types of data you need to store, it basically falls into one of two categories:

    1. Performance sensitive data:

    - OS/Apps: small size, but high random performance needed
    - Active work data (FCP, Logic, Aperture, Photoshop projects): medium size, high STR

    Ideal Options:

    SSD:
    In an ideal situation, you have your OS/Apps and active work data on solid state for optimal performance. The new Mac Pro addresses this by offering integrated PCIe SSD storage that should offer capacities to match anyone’s current setup. If you have multiple SSDs now, this is the ideal opportunity to consolidate to the single high performance SSD in the new Mac Pro, and if you must absolutely have additional SSD storage, use the affordable Promise J4 enclosure for $380.

    HDs in RAID0:
    If you simply can’t afford an SSD big enough to fit your active work data, then you’re likely using a RAID0 array of spinning disks. Anyone looking at spending a few grand on a workstations should really look at partitioning their work data set into small enough chunks to get as much onto an SSD you can afford in order to get the most out of the new workstation. However, I imagine there are some fringe cases where this is just not possible. That’s where a pair of WD Velociraptors in RAID0 can fill this void or even a pair of fast high-density drives might do.

    If you’re using a pair of Velociraptors in RAID0 (STR up to 400MB/s) then you will want Thunderbolt to not be a bottleneck. If you’re just using a pair of high-density drives in RAID0 then a USB3 enclosure offers enough performance at a lower price point. If you need more than a pair of drives, then use multiple enclosures (eg. WD TB Velociraptor Duo) or a larger multi-drive enclosure. USB 3 enclosures for multiple drives are typically under $200 and will provide up to 250MB/s based on reviews I’ve seen. If you need unconstrained performance of multi-drive RAID0 arrays, you’re probably better off getting larger drives in 2 drive enclosures like Western Digital’s 8TB Thunderbolt Duo product than investing in a $600-$1000 TB RAID enclosure.

    2. Less performance sensitive data:

    - Archived work data (eg. last month’s project): large size, medium STR
    - iTunes/media: medium-large size, slow STR
    - Backups: large size, slow STR

    This type of data is typically characterized as being high volume, with either infrequent access, or not demanding high performance.

    Anyone using RAID0 for this kind of data is (by definition of the performance needs of this data) is not really getting the benefit of the added STR, and is taking added risk. Anyone using RAID0 for this kind of data, is doing so because they can, not because they need to or it’s smart. Don’t do that. Consolidate this data onto one or fewer larger stand-alone drives.

    Stand alone HD:

    Any of this less performance sensitive data can live on a single drive of sufficient size without performance implications. If it’s grown beyond a singe large (3 or 4TB) drive, then you can either simplify your storage with fewer large drives, or look at USB3 JBOD enclosures. As I RAID0 for this kind of data offers little or no advantage in terms of performance and introduces unnecessary risk.

    eSATA enclosures

    Lots of folks may be using external enclosures connected by eSATA. If you’re just running JBOD, then USB3 should be more than adequate, and most eSATA enclosures probably have a USB3 port as well, meaning the investment is a cable. If you don’t have USB3 on your eSATA enclosure, the first thing to do is look at consolidating to fewer and larger drives and moving to USB3. If you’re running a large high-perforamnce RAID5 array, then the best options are explained

    Multi-drive RAID5 arrays:

    If you need huge high performance RAID5 protected storage, you probably already have invested in an ATTO, Areca or other cached RAID card for several hundred dollars and special RAID ready drives like WD RED, so you’re less price sensitive. Obviously you can solve this similarly with an external thunderbolt RAID box, a TB PCIe card box for your RAID card. But, if high performance is not that important (you can live without the cached controller) then USB 3 RAID enclosures with large drives or general purpose SOHO NAS offerings are perhaps a better solution (and the NAS can offer other benefits). What solution is ideal for you depends on your budget and need for speed. But there is a variety of solutions at nearly every budget.

    Conclusion:

    Let’s discuss storage migration to the new Mac Pro here, I’m certain there’s migration issues and even solutions I haven’t considered.

    My personal situation, like most people, is that I have an old Mac Pro full of drives. But I’m going to use this new Mac Pro as an opportunity to reassess my storage needs with an eye towards consolidation and simplification. I’m going to consolidate my OS/Apps and working photo library onto the integrated Mac Pro SSD. My archived photo libraries are on a 3TB internal disk that will move to USB3 and my media library is already on a NAS so that can stay as it is.

    I’m interested in what other folks who are planning to do about storage migration who are seriously considering the new Mac Pro. I’m also interested in hearing more from those who are dead set they can’t possibly migrate their storage affordably… as long as you’re open to some discussion and debate about your assumptions and requirements. :p
     
  2. flowrider, Aug 22, 2013
    Last edited: Aug 23, 2013

    flowrider macrumors 601

    flowrider

    Joined:
    Nov 23, 2012
    #2
    Well written, but I'm not buyin' it. Because Apple is making a more powerful but less configurable MacPro, I need to reassess my needs? I don't think so. I'm quite happy with my 6 HDDs and 1 SSD. Sure I have redundancy, but it's my backup. HDDs and SSDs fail, and if they do, fine, I still have my stuff.

    Over the years since my first expandable Mac, a Mac IICI, I have counted on being able to modify my Mac to fit my needs, not modify my needs to fit the needs my Mac.

    As you can guess, I do not plan to migrate to the next MacPro!

    Lou
     
  3. MacNerd1239 macrumors member

    Joined:
    Oct 21, 2007
    Location:
    Massachusetts, USA
    #3
    Why should I have to take a performance hit on my data when upgrading to a NEW machine, isn't the whole point to get better performance?
     
  4. portishead macrumors 65816

    Joined:
    Apr 4, 2007
    Location:
    los angeles
    #4
    Pretty much agree with you, but it just sucks at the moment because we're in a bad situation. SSD's are expensive, TB is expensive. People are highly critical now, which I get, but in 2 years when prices come down a lot, Apple is going to look like a genius.

    It's a mix of Apple looking ahead to the future, and current users being whiny without realizing they aren't being forced to upgrade.
     
  5. tamvly macrumors 6502a

    tamvly

    Joined:
    Nov 11, 2007
    #5
    I don't see it as a given that disk I/O performance will suffer on the nMP.

    The boot SSD will likely be faster for just about everyone in the new machine. Probably substantially faster.

    It remains to be seen how TB in the nMP will perform with external disks/SSDs. External rather than internal SATA? Yes. More expensive to implement? Certainly. Faster? Wait and see.
     
  6. scottsjack macrumors 68000

    Joined:
    Aug 25, 2010
    Location:
    Arizona
    #6
    My Mac Pro is at Apple right now getting a new front USB board (yeah AppleCare). I'm using its internal drives in an OWC Mercury Rack Pro plugged into a late 2009 mini via FW800.

    Not that my current setup should be in any way confused with new MP and external TB drives but I really miss the internal drives. They were always there, one button to start up, no external fan noise, no additional wiring.

    My drives and their external back ups are always configured as individual drives, never RAID. I see no point in a serious computer that cannot house a bunch of storage drives. Some call that old fashioned, I call it efficient.
     
  7. OS6-OSX macrumors 6502a

    OS6-OSX

    Joined:
    Jun 13, 2004
    Location:
    California
    #7
     
  8. VirtualRain thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #8
    YouTube?! :eek: :eek: :p

    (BTW, it seems like this quote jumped threads or something?)
     
  9. OS6-OSX macrumors 6502a

    OS6-OSX

    Joined:
    Jun 13, 2004
    Location:
    California
    #9
    You are correct about thread jumping. The quote was from the TB v. PCIe Essay! I was timed out in that post, copied and pasted in the incorrect Essay!
    So many essays to read while in the teachers lounge grading papers!:p
     
  10. rasputin666 macrumors member

    rasputin666

    Joined:
    Mar 1, 2009
    #10
    First-i’m in the group of having a nMP as an enthusiast.

    Where i was:

    MacPro 3,1
    128gb SSD OS/Apps Bay 1
    4 TB iTunes library in Bay2/Bay3 [Two 2TB Hitachi in raid]
    500gb apple stock drive in Bay4 for Data/workspace
    1 TB FW800 drive as backup for SSD and Data drives
    OWC Qx2 eSata to 3,1 for iTunes backup

    Now:

    nMP 6-core stock houses my OS, Apps, photos, docs, PDF’s, etc
    Qx2 4TB is now main iTunes Library connected via Lacie eSata hub to nMP via thunderbolt
    i had Two 3TB USB2 Seagate Desktop drives so i bought the USB3 base adapter for short money to expand iTunes Library
    2nd 3TB drive is backup to the other
    i bought a 4TB Seagate external [with the Thunderbolt adapter] to backup iTunes Library portion on the Qx2
    i took the 128GB SSD from 3,1 and connected it via GoFlex Thunderbolt adapter for additional workspace beyond the 256GB SSD in nMP
    i took the Two 2TB drives that was in my 3,1 and put them in two cheap USB3 external enclosures

    The move to the nMP from my 3,1 cost me [besides the nMP!] ONE 4TB drive for backup of the Qx2, the thunderbolt adapter for one of the two 3TB drives that’s extending my iTunes library. and the Lacie eSata hub. My 2 main externals, both part of iTunes library are the 4TB Qx2 Thunderbolt [via eSata] and 3TB Thunderbolt and both backup drives are USB3 [one 4TB and one 3TB].

    Reading your essay, i’d have to agree that most of my data is in Section 2 but i am starting not to like my implementation, even though i had the TWO 3TB drives just sitting waiting to be used. i now have 7 [SEVEN!!!] externals connected to my nMP.

    Qx2 4TB [1TB x 4 samsung drives]
    Qx2 4TB backup
    3TB itunes
    3TB itunes backup
    128GB SSD from 3,1 via TB adapter
    the two 2TB Hitachi drives from my 3,1

    unless i do something about the size of my external SSD, one of the drives in #6 is backing up my internal SSD and external SSD, without that, they are really not doing anything useful.
    1 option is to exchange the 1TB drives in the Qx2 with the two 2TB drives doing nothing much now, and buying two more 2TB and put in Qx2 getting to 8TB from 4TB. that removes need for 2nd iTunes 3TB drive and its backup. I have to see what my older Qx2 [2010 so no USB3] can handle in terms of max terabytes. I would still need to figure out backup of the Qx2. Secondly, bigger external SSD for workspace. I can use one of the leftover 4x1TB as backup of the two SSD’s
     
  11. VirtualRain thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #11
    This sounds confusing, probably because it's unnecessarily complicated.

    If you want some help simplifying this, that's cool...

    Forgetting how many drives you have, and just focusing on the data... How much actual storage do you need for:
    - iTunes?
    - Other data?
    - Backups?
    - SSD workspace?
     
  12. rasputin666 macrumors member

    rasputin666

    Joined:
    Mar 1, 2009
    #12
    it is confusing :confused: my attempt was to use the 2 drives i had sitting around and then have backups. what i should have done is what i do at work, go to the whiteboard and draw it out and i would have seen that i wouldn't like it and sell everything i had a use the nMP purchase as a way to start over...

    - iTunes? currently library is 4.5TB. i have it split in 4TB Qx2 [movies] and 3TB external [all other iTunes media]
    - Other data? 500GB
    - Backups? i'm OCD and backup/clone everything. 4.5TB for iTunes backup on 2 external and 1TB for Other Data and SSD space on 1 external
    - SSD workspace? 85GB used from the 384GB [256Int+128Ext]

    that is Used space right now.
     
  13. VirtualRain thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #13
    I see. Yeah, I agree using this opportunity to start from scratch might have been a great idea, but it's probably still easy to consolidate and simplify if you're open to this...

    I gather the Qx2 can hold 4 drives? And I gather that in terms of large drives you have a pair of 3TB drives and a 4TB drive.

    If this was my situation, I'd try to eliminate everything except the Qx2 and the SSD.

    You currently have about 5TB of data. 8TB of storage would be ideal long term, but 6TB will do for now. So, what I would do, is buy another 4TB HD, and put that in the Qx2 along with the other 4TB and the pair of 3TB drives. You could run this all in JBOD mode (just four separate disks) using a couple for data and a couple for backup. Alternatively you could RAID0 the two 4TB drives to have a single 8TB volume and RAID0 the two 3TB drives for a single 6TB volume. And then use the 8TB volume to store everything, and the 6TB volume to backup everything (or vice-versa). When you outgrow your 6TB volume, you would then replace those two 3TB drives with 4TB drives giving you 16TB in two 8TB volumes.

    I would then sell or gift everything else except the SSD and it's TB adapter which will come in handy to augment your internal SSD as you had planned.

    If there's any data that's irreplaceable if the house burns down, you could use any of your externals and left over drives for an offsite backup that you update occasionally but generally keep somewhere else.

    In the end, you have the Qx2 and SSD and everything is both fast and backed up.

    Of course, there are probably many ways to simplify this though... Let me know what you end up doing.
     
  14. mikepj macrumors regular

    Joined:
    Nov 2, 2004
    #14
    There are so many threads on this forum about storage and the nMP, but I think VirtualRain gave a good summary above. I took the opportunity to rethink my storage needs when purchasing a nMP (which still hasn't arrived). Here's what I am going to end up with:

    Internal storage: I CTOed the Mac Pro with a 512GB SSD, which should be plenty to store my apps and basic files (Home directory, etc).

    External storage: Purchased a Pegasus R4 diskless and put in four 3TB drives configured as a RAID 5. This gives me 9TB of working space to store my Aperture library, iTunes media, virtual machines, and other data.

    Backup: I have a Linux file server with 6TB of usable RAID 5 storage (and planning to add more shortly). Planning to use either Crashplan or Time Machine to backup here.

    Offsite Backup: Planning to use Crashplan.

    I also have an extra 3TB hard drive (from my old Mac Pro) that I haven't decided what to do with. Since it's an older disk, I don't trust it for primary data. However, it might be useful for some additional off-site backup, or perhaps a Time Machine volume for my smaller files to give an easier restore option for accidental (non hardware failure) data loss.

    Either way, I thought the above was a good way to have a lot of storage with a minimal amount of cable clutter. I'll have the nMP and the Pegasus on my desk… Everything else is remote.
     
  15. rasputin666 macrumors member

    rasputin666

    Joined:
    Mar 1, 2009
    #15
    Like i said yesterday, wish i read your post before :) great advice. 1 change i'll have to make, my Qx2 is from 2010 and that unit doesn't support JBOD or more than 4 x 2TB drives, so i can get to 8TB that way but i'll need another enclosure for the 2 4's and the 2 3's or at this point, sell/gift everything and get a low cost multi bay enclosure that can hold 4 x 4TB drives [or the 2 4's and 2 3's in two raido setup as you suggested. hhmmmm
     
  16. matoch macrumors member

    Joined:
    Oct 12, 2006
    #16
    This is a pretty good summary. There's so many people that complain about the lack of internal storage but I agree it's not a big issue. There are many options in a variety of price ranges depending on your needs.

    The mistake I think Apple made was moving to the New Mac Pro one generation early. If they had created one more Mac Pro with the old Style that had both USB 3 and Thunderbolt built in people would have had a chance to address storage needs earlier on before making the switch to external only solutions.

    I've known for 3 years that I wanted to go to an external thunderbolt solution but I was waiting for a new Mac Pro to do it. This turned into a huge investment to get everything I wanted in place.

    I'm still waiting on my New Mac Pro to be delivered but when finished I'll have a 6 core, 500GB, 32GB machine with a Pegasus2 R4 with 4x3TB drives for storage and an external blue ray burner.
     
  17. VirtualRain thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #17
    Thanks. When I wrote that, it was after WWDC when people were up-in-arms over the lack of internal storage. Since then, some new solutions have emerged which has made it a lot easier for some. Like this...

    :)

    ----------

    Incidentally, if you're interested, I also wrote somewhat of a similar essay on why they didn't do this and went straight to Thunderbolt, cutting out slots and bays cold turkey...

    http://forums.macrumors.com/showthread.php?t=1625114
     
  18. barmann macrumors 6502

    Joined:
    Oct 25, 2010
    Location:
    Germany
    #18
    VirtualRain, I appreciate your enthusiasm, but find your advice a little lacking .

    You keep claiming the nMP can easily replace an original MP, and you suggest it can be done without any major hassle and investment by a majority of users .
    Frankly, I believe this is utter nonsense .

    But by all means, prove me wrong .

    Please let me know what the additional costs and hardware requirements are for the following, rather modest setup :

    - Move four internal harddrives , of any kind, from an MP to external enclosures (use no more than two) . Performance and usability must be at least SATA III ; no additional fan noise , zero vibrations, max. cable length 6 feet, JBOD capable .
    This is high-mid performance only, project files, libs and scratch , iTunes stuff and backups can be on a slow NAS or Drobo or whatever .

    - Provide hubs compatible with existing pro video, audio and photo gear using (port powered) Firewire .

    - Mature USB 3.0 support and working hubs .

    - Two+ PCIe slots in external enclosure, no speed penalty .

    Something like an updated MP, which of course would have included most of that, at no extra cost . ;)
     
  19. matoch macrumors member

    Joined:
    Oct 12, 2006
    #19
    You know we got an updated MP and that it has none of that right?
     
  20. mikepj macrumors regular

    Joined:
    Nov 2, 2004
    #20
    This is borderline trolling, but I'll bite.

    Lot's of options here. One of the cheaper ones is the OWC Qx2. 4 disk JBOD with USB 3 connection. Should be just as fast as drives in an old Mac Pro. Less than $300. The new Mac Pro is so much quieter, that combined noise between the nMP and this unit should be quieter than an old Mac Pro.
    http://eshop.macsales.com/item/OWC/M3QX2KIT0GB/

    Buy an Apple Thunderbolt to Firewire adapter for $30. Then use your hubs from the old Mac Pro.

    Not a feature of the old Mac Pro, but you can use the built-in USB 3 ports. There are just as many USB 3 ports on the new Mac Pro as there are on the old one.

    External Thunderbolt 2 enclosures run at 8x PCIe speed and run for about $450. Other than GPUs, I don't know of any common PCIe cards that require 16x connections.


    My turn.
    Can you upgrade an old Mac Pro to have Thunderbolt ports?

    Can you tell me how I can get my old Mac Pro cut it's power usage by 60-70% and save around $75 in energy costs each year?

    Can you make the old Mac Pro smaller? It's taking up too much space in my office.
     

Share This Page