How do you organise your storage?

[Silly John Fatty]

I have to get some external storage, and I have to think how I'll organise it. I'm thinking for example to get a drive just for photos.

Do you guys split your content this way? Like one drive for photos, one for apps, one for music plugins, etc.?

Or do you also mix the contents of your drives?

What's best and what do you recommend?

 

[rm5]

Here's a little spreadsheet for you detailing how I organize stuff! But basically, to sum it up in short, all my programs are stored here on my boot drive, active projects and sample libraries on an external SSD, and then past projects get archived to an external drive. I also use cloud storage for smaller files.

As far as how to work your storage situation, I wouldn't recommend having one drive explicitly for photos, videos, etc. unless you REALLY want to do that and have a good reason for it.

 

[richmlow]

I have to get some external storage, and I have to think how I'll organise it. I'm thinking for example to get a drive just for photos.

Do you guys split your content this way? Like one drive for photos, one for apps, one for music plugins, etc.?

Or do you also mix the contents of your drives?

What's best and what do you recommend?

Folders (and sub-folders) are your friends!!

 

[Silly John Fatty]

Folders (and sub-folders) are your friends!!

That’s how I have it organised now, but I noticed some people use some drives specifically only for certain things.

As far as how to work your storage situation, I wouldn't recommend having one drive explicitly for photos, videos, etc. unless you REALLY want to do that and have a good reason for it.

Why would you not recommend it? Also thanks for your spreadsheet!

 

[rm5]

Why would you not recommend it? Also thanks for your spreadsheet!

No problem! I just think that you'd have to buy too many external drives, and it'd end up being sort of a waste of money, that's all

 

[Silly John Fatty]

No problem! I just think that you'd have to buy too many external drives, and it'd end up being sort of a waste of money, that's all

Oh okay, I thought there may be technical issues.

I want to get 40 Gb/s so it will be pricey either way (actually not even that pricey for what it is), but I'll have to learn more about how NVMe SSDs work and then decide which drives to use for what use.

It may very well be that I'll do mixed usage. But I'll have less drives than you have I believe, you have quite a lot.

I've read that pros use certain drives as so-called "scratch drives" that are used to be used intensely for example. Whereas for storage they use other drives. But even that depends on the kind of media, etc.

I'll have to look deeper into it.

 

[theorist9]

I want to get 40 Gb/s so it will be pricey either way (actually not even that pricey for what it is), but I'll have to learn more about how NVMe SSDs work and then decide which drives to use for what use.

You won't get 40 Gb/s from a single-cable TB drive. I think the best you'll see, for large sequential transfers, is ~3300 MB/s = 26 GB/s. If you're OK with those write speeds, but want faster read speeds (up to 5300 MB/s = 42 GB/s, you'll need a dual-TB RAID setup, like the iodyne Pro Data. But it's pricey ($4,500 for 12 TB).

iodyne Pro Data Review

The iodyne Pro Data is a laptop-sized RAID enclosure containing 12 NVMe SSDs, primarily focused on creative professional applications.

www.storagereview.com

 

[HDFan]

I noticed some people use some drives specifically only for certain things.

Simplest thing is to get a large hard disk and create folders for projects, photos, etc. This makes implementing a 3-2-1 strategy much easier.

I want to get 40 Gb/s

You won't get 40 Gb/s from a single-cable TB drive.

Think you asked this question elsewhere. Same answer there and here.

 

[Ben J.]

For me, storage organization has over the years come down to this:

Internal: System, apps, user folder etc.

External #1 (2TB, fastest drive):
Photos, Video, Studio (Music recording etc.). Everything essential and large.
Separate partition: Carbon Copy Cloner (CCC) clone of Internal.

External #2 (2TB, slower drive): Old movies and music that use a lot of space, but don't need fast storage, and I wouldn't care much if they were accidentally lost.
Separate partition: 2nd CCC clone of Internal.
Separate partition: CCC clone of External #1.

External #3 (2TB, slowest drive. Kept next door. Updated manually every now and then.):
Separate partition: 3rd CCC clone of Internal.
Separate partition: 2nd CCC clone of External #1.
Separate partition: CCC clone of External #2.

Carbon Copy Cloner is set to update backups every night.

 

[Silly John Fatty]

You won't get 40 Gb/s from a single-cable TB drive. I think the best you'll see, for large sequential transfers, is ~3300 MB/s = 26 GB/s. If you're OK with those write speeds, but want faster read speeds (up to 5300 MB/s = 42 GB/s, you'll need a dual-TB RAID setup, like the iodyne Pro Data. But it's pricey ($4,500 for 12 TB).

iodyne Pro Data Review

The iodyne Pro Data is a laptop-sized RAID enclosure containing 12 NVMe SSDs, primarily focused on creative professional applications.

www.storagereview.com

Think you asked this question elsewhere. Same answer there and here.

It is actually possible, some people on MacRumors did it. I'll get two enclosures and do a RAID 0 on both drives, this way I can get 40 Gb/s.

For me, storage organization has over the years come down to this:

Internal: System, apps, user folder etc.

External #1 (2TB, fastest drive):
Photos, Video, Studio (Music recording etc.). Everything essential and large.
Separate partition: Carbon Copy Cloner (CCC) clone of Internal.

External #2 (2TB, slower drive): Old movies and music that use a lot of space, but don't need fast storage, and I wouldn't care much if they were accidentally lost.
Separate partition: 2nd CCC clone of Internal.
Separate partition: CCC clone of External #1.

External #3 (2TB, slowest drive. Kept next door. Updated manually every now and then.):
Separate partition: 3rd CCC clone of Internal.
Separate partitions: CCC clone of External #1 and 2.

Carbon Copy Cloner is set to update backups every night.

Thanks, you have a lot of backups!

I was thinking of getting a single large HDD for backing up all the other drives. But not every night. Maybe once every 3 months. I've never had a drive fail yet. Or at least I don't think so. Maybe I have. Sometimes I have the feeling that some files are missing. Maybe that's some form of partial failing already, I'm not sure.

I feel like if I did a backup every night I'd just wear the drive more and therefore make it more probable that the backup drive will fail.

 

[Ben J.]

Thanks, you have a lot of backups!

I feel like if I did a backup every night I'd just wear the drive more and therefore make it more probable that the backup drive will fail.

I sleep well at night knowing that lightning could strike and I would be able to get a new mac and be up and running the next day.
Don't worry about drives failing. I've almost not ever seen it with over 30 yrs using macs. Besides, having more than one backup makes that a non-issue.

 

[meson]

There is nothing special about a scratch drive. A drive is a drive. A “scratch” drive is simply one that software like Adobe can use like ram when it swaps heavily. It just redirects the thrashing from the drive that houses the application to another drive. The idea is that if you wear out the scratch drive and it fails, nothing critical is lost.

Not that she listens, but the setup I recommended to my wife is to keep apps, some important often used files, and only the photo shoot or two that she is actively editing on the boot drive. From there, an external scratch SSD for photoshop to thrash. Then another external to archive completed work.

She usually ends up filling the boot drive and/or scratch drive with stuff and complains when she needs to reboot and/or clean up files.

 

[Silly John Fatty]

There is nothing special about a scratch drive. A drive is a drive. A “scratch” drive is simply one that software like Adobe can use like ram when it swaps heavily. It just redirects the thrashing from the drive that houses the application to another drive. The idea is that if you wear out the scratch drive and it fails, nothing critical is lost.

Not that she listens, but the setup I recommended to my wife is to keep apps, some important often used files, and only the photo shoot or two that she is actively editing on the boot drive. From there, an external scratch SSD for photoshop to thrash. Then another external to archive completed work.

She usually ends up filling the boot drive and/or scratch drive with stuff and complains when she needs to reboot and/or clean up files.

That's actually pretty much how I intended to do it. Have all apps on the boot drive. But have everything else on the external drives. And then a backup drive for internal + external. So I'd have Logic Pro libraries or my Photos libraries and Movies on external drives, but Logic Pro itself on the boot drive.

The only difference to you: I wouldn't copy projects I'm working on at the moment to the boot drive, but I'd rather work on them from the external drives. I don't want to wear the boot drive unnecessarily.

 

[theorist9]

It is actually possible, some people on MacRumors did it. I'll get two enclosures and do a RAID 0 on both drives, this way I can get 40 Gb/s.

I take it you're referring to something like this:

https://forums.macrumors.com/threads/thunderbolt-4-nvme-m-2-external-storage-read-write-speeds.2340127/page-8?post=32028925#post-32028925

 

[Silly John Fatty]

I take it you're referring to something like this:

https://forums.macrumors.com/threads/thunderbolt-4-nvme-m-2-external-storage-read-write-speeds.2340127/page-8?post=32028925#post-32028925

View attachment 2228987

For example, yes. I've seen multiple people do it, and it technically makes sense. 40 Gb/s with one Thunderbolt controller isn't possible for data, because around half of that is reserved for video/displays, etc.

But if you use two controllers, where you get around 20 Gb/s each and then put your drive in a RAID 0 format, you get around 40 Gb/s.

You'll need a Mac that has two Thunderbolt controllers, however. On the Mac minis it's one controller per two Thunderbolt ports. So the models with two TB ports have one TB controller, the models with four ports have two controllers.

 

[theorist9]

For example, yes. I've seen multiple people do it, and it technically makes sense. 40 Gb/s with one Thunderbolt controller isn't possible for data, because around half of that is reserved for video/displays, etc.

But if you use two controllers, where you get around 20 Gb/s each and then put your drive in a RAID 0 format, you get around 40 Gb/s.

You'll need a Mac that has two Thunderbolt controllers, however. On the Mac minis it's one controller per two Thunderbolt ports. So the models with two TB ports have one TB controller, the models with four ports have two controllers.

The Mini has two ports, the Pro Mini has four. Are you getting a Pro Mini so you have two controllers? And those two controllers have to carry not only data, but video. I assume if you were maxing out the Pro Mini's display load (2 x 6k + 4k), that would cut your data rate. So I'm wondering what's the max display load you can have without cutting the data rate. I suppose I could figure it out....

Also, I don't think what you wrote applies to the MBP's, since both the Pro and the Max have 3 TB Ports.

 

[Silly John Fatty]

The Mini has two ports, the Pro Mini has four. Are you getting a Pro Mini so you have two controllers? And those two controllers have to carry not only data, but video. I assume if you were maxing out the Pro Mini's display load (2 x 6k + 4k), that would cut your data rate. So I'm wondering what's the max display load you can have without cutting the data rate. I suppose I could figure it out....

Also, I don't think what you wrote applies to the MBP's, since both the Pro and the Max have 3 TB Ports.

I'm not sure about the MacBooks. I have the Mini M2 Pro and it has, from what I've heard, 2 TB controllers, and a total of four TB ports.

The 40 Gb/s refer (from what I've understood) to a single Thunderbolt controller, not to your Mac in total. So in total you'd theoretically have 80 Gb/s, if you had 2 controllers (it must be that way, otherwise some people wouldn't be able to get 40 Gb/s with data transfer).

So you have a total of 80 Gb/s, of which you can use a little more than half for your data transfer, while you can use the rest for your displays.

Edit: From this thread:

5120x2880 at 60Hz with 8-bit color needs about 27Gbps.

Depending on how many controllers and displays your have, you may or may not have enough for your displays. But that is apparently separated from the bandwidth you can use for data. I think you can't use more for data or more for your displays than the share that it is meant for.

 

[Clix Pix]

Over the years I have developed a system where I have several external drives, each with specific content on them which I back up periodically (images, every week, since I shoot a lot; other files, once a month). I have one drive which is specifically a backup of the current files and folders on my machine (Desktop, Documents, Movies, Music, Pictures), another "supplementary" drive which now is spanning what has become a distinctly not-insignificant several years' worth accumulation of files and folders in each of those categories. I began doing that some years ago when I wanted to keep my machine's internal drive reasonably "clean and mean" and so developed this system for stashing files and folders which weren't always needed immediately but which I still needed quickly and readily at hand from time to time. Technically speaking, it's also serving the purpose of an additional backup of sorts as well.

I tend to spend a lot of time shooting photos and working with RAW files and eventually coming up with results that I need to have available and/or archive; thus, I have a couple more external drives that I maintain for this purpose, as well as a separate archival file of older images. Edited image files for the current year, plus images in some phase of the reviewing/culling/editing process are retained on the computer's internal drive. Before beginning to process anything I do immediately back up the RAW files on yet another external drive for safety's sake.

So, yeah, I've got a lot of external drives floating around here, each with its own specific purpose, and each month a full set goes off to my safe deposit box at the bank to be swapped with the drives in there, which I bring home and update.

I'm retired, I've got plenty of time to mess around with this stuff, and I really prefer having full control over my data files and images rather than syncing or storing them on some online site. Years ago I started out with HDDs but now I am using external SSDs for everything, although I still do have some older HDDs still hanging around in a cabinet.

 

[eyoungren]

I'm not really a fan of single external drives. At a minimum it means cables and I already have a lot of those. Powered drives require being plugged in with an external power supply, while drives running off ports have a tendency to disappear from the Finder.

So my Mac Pro has four internal drives…

1) Boot drive, 1TB SSD. OS obviously, apps and personal documents like taxes, receipts, etc.

2) Storage drive for burning and long term storage, 6TB HD. I drop everything I ever download into this drive. Eventually it gets burned and deleted off the drive. Stuff I want to keep like installers, old versions of OS, etc are stored here. I also run any VMs off this drive.

3) Scratch/junk drive, 3TB HD. Used as a scratch disk and for any stuff I really don't care about losing.

4) Dropbox/Storage drive, 6TB, HD. I have 4TB reserved for Dropbox because I pay for 4TB with Dropbox. I keep drive images in Dropbox that all my Macs back up to on weekends. The other 2TB I have yet to use.

Oh and bonus… 5) Virtual RAM Drive, 10GB. I use this for Photoshop, InDesign, Illustrator and QuarkXPress as a scractch/backup drive. Also, anything I want loaded into RAM directly I'll put in there too.

Additionally I have…

1) NAS, 6TB. Music/video storage. Daily backups. My main video and music library are stored here. I don't use iTunes for music, I use other apps that can autoupdate from this central location whenever I add music. All my Macs backup daily to drive images on the NAS using Carbon Copy Cloner.

2) PowerMac G4 sharing a 6TB HD on the network. This drive is primarily used as a backup location for the music/media folder on the NAS. Carbon Copy Cloner replicates those folders to this drive.

3) 2TB RAID enclosure. Connected directly to my MacPro, this drive is used for photos, art, images and design work. I'm a graphic designer so I want all this stuff (and my art library) on one drive.

4) Lastly, one 150GB hard drive being shared by a 2006 Mac Mini. All downloads on all my Macs are directed to this shared drive. Makes it easier to sweep downloads to the second drive on my MacPro periodically. I'm not going to each Mac and transferring files.

My home network is Gig-E and all computers have Gigabit networking ports - including the PowerMac G4 which has two G4 XServe NICs installed. I'm not usually concerned about file transfer times.

 

[HDFan]

My system is simple. Have a large external RAID (128 TB) where I keep all of the data which won't fit on my boot SSD - pictures, media, etc. It is backed up to 2 NAS units as well as 3 online backup services. When I can get to the bank vault put irreplaceable data such as pictures to a HD with the correct size.

Boot disk just contains documents and index files for Music, Lightroom, etc. which are relatively small and can be backed up with multiple TM backups.

Even though I have terabytes of data my devices (thunderbolt and 10 GbE) don't need any additional interface speed, for video editing, file transfers, or whatever.

 

[dhershberger]

I have three external devices for storage: one NAS, one powered backup drive, and one TB3 SSD.

The NAS has this folder structure:
Backup
Financial
Music
Photos
Projects
Software
Travel
Videos

The powered backup drive is plugged into the NAS which automatically backs-up to it.
The TB3 SSD is used to hold difficult to replace data and sits in a firebox inside my safe.

 

[HobeSoundDarryl]

Oh okay, I thought there may be technical issues.

I want to get 40 Gb/s so it will be pricey either way (actually not even that pricey for what it is), but I'll have to learn more about how NVMe SSDs work and then decide which drives to use for what use.

It may very well be that I'll do mixed usage. But I'll have less drives than you have I believe, you have quite a lot.

I've read that pros use certain drives as so-called "scratch drives" that are used to be used intensely for example. Whereas for storage they use other drives. But even that depends on the kind of media, etc.

I'll have to look deeper into it.

Trying to maximize speed up to those levels for storage space for files like pictures and similar is probably wasting money unless you have some special circumstance where your photos are gigantic (RAW) and you want to see them as fast as possible on click. Else, the economics of HDD storage even if HDD (relatively) slow will be great for any such storage.

IMO, when to chase theoretical maximum speeds is for those niche situations where you are doing lots of big read/writes to complete a task: video editing is one such task... making MANY track music recording/production is another. Or if you are one of the relatively rare people where time (measured in seconds) really is significant money, that may justify such a target as well.

For browsing files like photos and similar, the good old HDD will be fast... NOT maximum fast but not aggravatingly slow either. If you need faster than single HDDs, get a RAID HDD setup and that will speed things up even more.

What others have offered applies: use folders to segregate types of files. Individual drives would likely be overkill except for very unique circumstances. If you are going to buy multiple drives, get a RAID enclosure, put all of them in there, set up as maybe RAID-0 for speed or RAID-5 for (self) backup (security) and that will give you great speed at relatively low total cost... especially versus a bunch of individual SSDs or a RAID SSD setup.

The "scratch" drive concept gets at the idea of wanting to do something- again like video editing- where you will be doing heavy read-writes to the drive. So you might move the project onto the speedy SSD-RAID or single SSD as your "scratch" do your video editing on it and then move it back to big HDD storage when you are done with the intense work. You reference Logic Pro and Plugins and those projects can be complicated enough to benefit from some very fast SSD-RAID or single SSD "scratch" especially if RAM is small and thus swaps will be used in a big way. However, again, you can store big Logic projects in big storage on HDD or HDD-RAID and then drag the project you want to work on now onto faster storage "scratch" while you work on it... then back again when done to free up the scratch space.

Chasing max-speed SSD for everything is going to be expensive and likely overkill. From what you've shared, you probably should go with a hybrid option of some fast SSD/SSD RAID storage for working "scratch" and big HDD/HDD-RAID for general purpose storage. Either way, be sure you can buy some backup drive(s) too because no choice is good without a backup to restore if any SSD or HDD setup fails.

 

[Silly John Fatty]

Trying to maximize speed up to those levels for storage space for files like pictures and similar is probably wasting money unless you have some special circumstance where your photos are gigantic (RAW) and you want to see them as fast as possible on click. Else, the economics of HDD storage even if HDD (relatively) slow will be great for any such storage.

IMO, when to chase theoretical maximum speeds is for those niche situations where you are doing lots of big read/writes to complete a task: video editing is one such task... making MANY track music recording/production is another. Or if you are one of the relatively rare people where time (measured in seconds) really is significant money, that may justify such a target as well.

For browsing files like photos and similar, the good old HDD will be fast... NOT maximum fast but not aggravatingly slow either. If you need faster than single HDDs, get a RAID HDD setup and that will speed things up even more.

What others have offered applies: use folders to segregate types of files. Individual drives would likely be overkill except for very unique circumstances. If you are going to buy multiple drives, get a RAID enclosure, put all of them in there, set up as maybe RAID-0 for speed or RAID-5 for (self) backup (security) and that will give you great speed at relatively low total cost... especially versus a bunch of individual SSDs or a RAID SSD setup.

The "scratch" drive concept gets at the idea of wanting to do something- again like video editing- where you will be doing heavy read-writes to the drive. So you might move the project onto the speedy SSD-RAID or single SSD as your "scratch" do your video editing on it and then move it back to big HDD storage when you are done with the intense work. You reference Logic Pro and Plugins and those projects can be complicated enough to benefit from some very fast SSD-RAID or single SSD "scratch" especially if RAM is small and thus swaps will be used in a big way. However, again, you can store big Logic projects in big storage on HDD or HDD-RAID and then drag the project you want to work on now onto faster storage "scratch" while you work on it... then back again when done to free up the scratch space.

Chasing max-speed SSD for everything is going to be expensive and likely overkill. From what you've shared, you probably should go with a hybrid option of some fast SSD/SSD RAID storage for working "scratch" and big HDD/HDD-RAID for general purpose storage. Either way, be sure you can buy some backup drive(s) too because no choice is good without a backup to restore if any SSD or HDD setup fails.

I'm already using HDDs and I think they're incredibly slow. I am also using SATA SSDs and I think they're slow too.

Even for storage I'd like to have fast speeds. Let's take my synchronised photos from my iPhone or iPad for example. I make a lot of them. I just filled my 2 TB HDD today with a single library. It did take me 10 years to fill it, though (it's all my iPhone photos basically).

It's extremely annoying synchronising to this library, because it is too slow. And it's annoying to go look for pictures as well because it's just slow when you have 100.000+ photos and videos suddenly opened at once.

With HDDs in RAID 0 that is of course a different story. I don't run a RAID 0 setup anywhere right now (and never have). This setup (HDD in RAID 0) is said to bring around 1000-1500 MB/s, which is around 5x less I could get with NVMe SSDs + RAID 0.

It's more expensive of course.

I planed to use HDDs only for backups. What also currently annoys me with HDDs is that when you access something on them, they take a while to turn on. You hear them spinning then, and finally they're "ready", but that's kind of annoying too I think. And they're inside my Mac Pro, in the drive bays (in case that makes them faster). Don't know if it will be the same if I have external NVMe SSDs on my Mac mini, maybe they need a moment to turn on as well.

It surely is a comfort thing in my case. I just like the snappiness. I'm on the computer the entire day and just want it to be super fast. But I'll yet have to think about which of my activities are very read- or write-intense.

But I have no idea what apps read or write things and in which way. I don't know for example if working with Logic does a lot of read or write activity, and how it does it.

 

[ovbacon]

External:

A fast 2TB ssd for all things photo/video related
Another 2TB ssd for occasional CCC backups
A 3TB HDD for files that I do not have a need for speed or I do not use much.
A 2TB HDD for Time Machine backups of the above and the internal HD.

 

[PhoenixDown]

My plan is to get a Synology server when the time comes.