External Hard Drive for Macs - Sync Multiple Users' Photo Libraries

[Thisismattwade]

Hi all, my wife and I share a Mac on a regular basis - base M1 MBA. It's a fantastic computer. We are both signed in with Fast User Switching enabled. The one problem is our iCloud (Shared) Photo Library is big - 555GB. She inherited a lot of our photos when we moved off OneDrive, so her piece of the Shared Library is 500GB and mine is 55GB. I'm considering switching our local syncing to an external hard drive, so when I'm working on our photo library I can work with local full-size originals. (It's very helpful to not have to wait for files, esp videos, to download when I'm watching them.) I have a few questions about how to make this work well.

- When I buy a new external SSD, what should I do it as soon as I get it to make sure it works well with MacOS?
- Can I use the external SSD as the "hard drive" that each of our iCloud Photos Libraries syncs to?
- If so, how does that work with each user? Does the syncing service just look for the hard drive to be connected and then sync any time there are changes? What happens if one of us forgets to connect the external hard drive? Will it corrupt anything, or just give some kind of syncing error?
- I'm pretty sure I know this answer, but is it possible to keep full-size originals on the external SSD, but then use Optimize Storage on the local HD?
- ETA: I’m assuming each user can choose the HD their library syncs to: I can choose to sync my library to the external SSS, but my wife can choose Optimize Storage and just use the local HD. Is that right?

Thank you!

 

[ChrisA]

Each user's photo library can only be in one place. So you can not choose to "optimize storage" and to not. It is either/or and only one location.

If the external drive is not plugged in and you try to run the Photos app, it will tell you it can't find the photos library and stop. It will not "fall back" to the build in SSD. You told it where the data is and that is where it must be.

What to do...
1) just buy a big external SSD. if you have 600GB of data, asume this will double over the life of the SSD. Buy a 1TB or 2TB size.
2) Buy an even larger mechanical disk, say 4TB, for Time Machine. Let it back up both the internal and external SSDs. This drive can be as slow as you like as it runs in the background an after the intial backup, not much data is written. But it needs to be about 2X the size of your data.
3) to make any kind of drive usable with MacOS, with an APFS files system using disk utility. Make sure to choose APFS a it is dramatically better than the other options.

You can actually still access the photos when you don't have the external SSD with you. You'd used the iCloud web browser interface, or an iPad with "optimize storage" enabled. But the notebook would always need the external plugged in.

What I did. I bought a Synology NAS. All the data goes on this device. I own several different computers, some iMac, Mac Mini and a few Linux-based PCs and iPads and iPhones. All the data goes on the NAS and then it is accessible to every computer, even if I am away from home. Yes network access is slower than a direct Thunderbolt cable. There are ways to get the data to cache onto the local machine or I just don't care. I have 10GB Ethernet that connects the NAS to my main computer.

The advantage of the NAS with notebook computers is you don't have cables and little boxes connected and the notebook remains portable. The NAS has redundant storage and can survive a disk failure. Still it all gets backed up to a remove location.

 

[RhetTbull]

Here's what I recommend in your situation. Keeping the Photos library on the external disk is absolutely possible but I'd not do so. You would create a folder for each user on the external disk and move the Photos library to the appropriate folder then open the library. Photos would sync to the external disk. The reason I'd not do this is that your ultra portable MBA is now less portable -- you have to take the external disk everywhere and it must always be plugged in if you want to use Photos. Rather, here's what I'd do:

Use "optimize Mac storage" in `Photos > Settings > iCloud` for each of you. This will reduce the size of the library to fit on your MBA internal disk (but you won't have the full resolution images). You can still edit or export an image because Photos will download it from iCloud when needed.

To also have a library with full size originals, which I recommend for backup purposes, I'd create a folder on the external disk for each of you as above. Then on the MBA, create new "shadow" accounts for each of you. Switch to the new account, log in to iCloud, then ensure "optimize Mac storage" is set for Photos (and iCloud drive if you use that). Don't sync mail or anything else to this shadow account. Use it only to sync Photos. Then in that shadow account move the Photos library to the external disk and change the settings to "download originals to this Mac". You can then switch back to the regular accounts using Fast User Switching and leave the shadow accounts active. They'll sync the entire library to the external disk as long as they're logged in. You can keep them logged in or periodically login to let them catch up. This gives you a full local copy on the external disk of each library but doesn't require you always have the external disk plugged in.

 

[Thisismattwade]

Hi, thanks for this!

The reason I want to have the Library (or Libraries) downloaded is that I've got a project that will take 3-6 months (just don't have enough free time) of cleaning up our (Shared) Photo Library. We're talking THOUSANDS of photos and videos. Waiting 2-5 seconds for each video (a lot are 4k, 350MB for example) to play adds up to precious time. For this timeframe, carrying around an external hard drive is a sacrifice I'd be willing to make. (Does having the full Library downloaded to my machine save this 2-5 seconds of time for each video?)

OPTION 1
I could use this opportunity to splurge on a 16GB/1TB M2 MBA ($1549 on Apple refurb) for this project. I can set up my User to keep the Photo library - which is still Shared so I have both our photos and videos - all on the Mac's hard drive, but set her User to Optimize storage. Thus, we won't have 555 x 2 GB downloaded onto the machine. Once I've finished this project, I would go back to Optimize storage for both Users. Effectively I never have to worry about local storage again.

OPTION 2
The biggest issue with OPTION 1 is the cost of the higher-spec M2 MBA. That's why my thought was buy a base M2 MBA ($929 on Apple refurb, also available for $899 at some retailers) which is otherwise completely useful for my family), sync my Library to an external hard drive (say the SanDisk Extreme 1TB for $123), work on the project for 3-6 months, then switch the Libraries back to Optimize storage and convert the external hard drive to a Time Machine. I could even technically just buy the external SSD, but when I'm not using the M1 MBA the external drive would not be plugged in. I imagine that would cause some syncing issues with my User. (Would 1TB be enough as a Time Machine?)

Ceteris paribus, OPTION 2 saves me over $500. I do like the idea of having a higher-spec'd MBA, but in reality my family doesn't need it. I just want to make sure I'm not missing anything in my comparison between the two options.

(I'm still holding out hope that one day Apple will allow us to backup Macs to iCloud, but oh well.)

 

[RhetTbull]

Option 3: buy a cheap refurb Mac Mini (I picked up one for $65 recently), add a $100 external monitor and the fast external drive and go to work!

 

[HobeSoundDarryl]

Another option: Purge those huge libraries into photos you want "local" and photos you would be OK keeping in "backup" Photos app libraries. Then store the backup "bigger" libraries on an external drive(s).

When I see sizes like 500GB, I visualize 30 shots of the same picture, trying to get ONE perfect one. I see this so often when people are browsing their photos libraries. They've basically taken a LOT of shots of any given photo in pursuit of perfection... but then don't go back and purge the imperfect copies.

Do this for most shots and you soon have gigantic libraries full of mostly the same images in 20-30 almost-the-same duplicate pics. If this is the case, keep doing what you/they are doing but develop the habit of keeping the "best one" in the main library and deleting or dumping the 2X near duplicates to the long-term library.

All this would take is duplicating the existing libraries and moving them to the external. Once you know they are there- and ideally backed up on yet another drive(s): Time Machine can make this very easy- start purging down the internal (drive) copy of the library to only hold the "best pic" versions.

Then it's a simple matter of holding down the OPTION key on the keyboard to have Photos load the smaller, carefully curated internal library or the giganto, NOT-curated external library.

The OTHER way this happens is storing VIDEO in the Photos library. Video eats up a LARGE amount of space and could easily be MOST of the 500GB. If that's the issue, you can easily, open "Media Types", Video, Select All of them, Right click mouse, Get Info and see how much of the 550MB is video. My suggestion would be to MOVE them ALL out of the Photos library and don't even store videos in Photos at all going forward (except temporarily right after they are shot).

Create a Home Movies or Videos file on that same external, move all Photos app videos to that folder and then you can use a simple tactic to make them available to the AppleTV app for easy viewing via AppleTV on a television and/or access them in that folder for easy direct access for playback in Quicktime on a Mac... without using ANY space on the internal drive. Just reply back if this (videos are most of it) issue and I'll share the "how?" answer.

The ONLY downside to this option is that videos ejected from Photos to external drive won't be playable when one is away from that external drive where they would then be stored. Remedy: take that drive with you when you travel OR temporarily drag videos you think you'll want to watch/see onto the internal drive ahead of such trips.

TIP: if there are a LOT of accumulated videos, consider organizing that new Videos/Home Movies folder on the external at least by year (create year folders within that folder) or even events within each year folder. This will help you quickly find specific videos when you want to locate little Joes 8th Birthday Party video or the Skiing Video from 2021, etc.

My guess is it will be a COMBINATION of BOTH (duplicate photos and lots of videos). Whatever the case, this concept of keeping "best of" in the internal library and the NOT "best of" pics + videos on the external should result in a relatively SMALL photos library on the internal drive without losing ANY photos or videos.

No need paying way too much for iCloud or other cloud storage. What you spend on so much Cloud space could buy you LOTS of local storage you will OWN.

ALL THAT OFFERED: BACKUP YOUR PHOTOS & VIDEOS!!!
Get your backup program in place. Photos & home movies can't be re-taken. Lose them and they are lost.

I suggest ALSO getting a couple of BIG HDDs to set up as Time Machine drives and backup the various Macs to them, ideally at least TWO copies (TWO TM backups) on at least TWO separate drives. Store ONE drive off site to protect against fire/flood/theft. Regularly swap the external TM drive with the local TM drive so that the backup is always pretty up to date. Be sure you are backing up the Macs PLUS the external drive now holding your only copy of home videos.

I follow a system like this and rotate my drives offsite-onsite approx. every 30 days. You may want it to be more frequent or less frequent than that, but pick your recurring time based upon the pain in a fire/flood/theft loss scenario of photos/videos newly shot since the last backup offsite was updated. Maybe that's 60 days? Maybe that's 15 days.

I also employ an answer referenced back in #2: a Synology NAS, which is a central way to TM-backup everyone's Mac and ALSO offer you the benefit of on-the-go (your own) cloud storage too if you like at $0/month subscription cost. If you can afford Synology, get a good one and it can play a big role in photos storage, collective TM backups and- optionally- on-the-go cloud storage of photos & videos too.

I store my offsite drive in a bank safe deposit box but any secure place away from the home can be fine. Getting it away from the home that could burn down, be robbed or flooded is the KEY concept here.

How big should these TM backup drives be: add up the size of all of the Macs internal drives + the external drive(s) you get to make this happen. If that turned out to total- say- 5TB, multiply by at least 2 to 3 and go with 10-15TB TM drives or larger. In general, the LARGER the better. HDDs of size like this are dirt cheap so invest in at least 2 of them to make this work. You don't want fire/flood/theft scenario with no way to recover precious photos & home movies. All the insurance claim money in the world could not re-shoot your memories.

 

[RhetTbull]

ALL THAT OFFERED: BACKUP YOUR PHOTOS & VIDEOS!!!
Get your backup program in place. Photos & home movies can't be re-taken. Lose them and they are lost.

I'm the author of the free open source tool osxphotos which includes a robust export command for exporting all your photos and videos along with all metadata in a form suitable for long term archive that's independent of the Apple Photos ecosystem. I export to external drive and backup that to Backblaze. iCloud Photos is great for syncing but it is NOT a backup system. Delete from one device and you delete from everywhere. If iCloud messes up and deletes or corrupts photos (I've seen this happen) then they're deleted/corrupted everywhere.

 

[Thisismattwade]

@HobeSoundDarryl , you make a really good point. In fact, part of the reason for this project is that I'm going through our Library from 2020 onwards for that exact reason: 20-30 shots (bursts are even more!) of our kids in almost the same stance. (There's also 1 minute long 4k videos where a kid got ahold of one of our phones and decided our floors needed to be in a video.) We've been much better about organizing in the last few months, but I want to have a clean Library for our future use. We also have photos and videos going back to 1985 in our Library, which have all been pretty much cleaned up, that account for a lot of the 500GB. I don't mind the size; I mind the above mentioned "duplicates" (quotation marks because Photos doesn't detect them with the Duplicates feature) and outtakes that will only get cleaned up by someone powering through the Photos Library.

I haven't decided whether to keep iCloud or not. I like the memories that Photos creates. I like that each of us has our whole library (on our phones) wherever we go. Plus, the kiddos like to look back at old photos on the iPad. I really don't mind the iCloud charge for that level of access. (Everyone has their price, I know.)

Thanks for all the feedback and helping me think through this!

 

[HobeSoundDarryl]

OK, I'm going to offer you a little more given those details.

CONSIDER centralizing your group library access in a new or old DESKTOP Mac- like Mac mini and then maybe 1+ AppleTVs put around the home for viewing photos, videos and music stored on it on the big TV screen(s) in your home (or on iDevice screens).

I presume that MB sometimes leaves the home. If it is THE home media "hub" for family members who may not always be going with it, everyone is locked out of access until it comes back. However, the anchored Mac (a desktop Mac) could be the home media hub for photos/videos/music access by all.

You could then sync new memories to that HUB for access by all. Create Photos (app) albums, Video and Music playlists to cater to everyone's interests instead of them having to work through 500GB of a whole "library" of photos.

Wife has her "Favorites" album, you have yours, kids have theirs, other albums of mutual interest, etc. You could do the same in Music with music playlists and in TV app with video playlists, etc.

Then everyone can get right to what they want on AppleTV(s) or any iDevices. You can also sync individual iDevices so the individual playlists/albums get onto each persons phone/iPad. Homesharing offers a LOT of great functionality in this way.

You keep using the MB for MOBILE computing and even dumping new photos/videos to it while traveling. But when you get home, you can move the new stuff to the "central hub" library to add the new stuff.

Unlike a mobile computer like MB where storing the photos library on an external drive is not ideal, that can change with a stationary Mac that is unlikely to ever disconnect from that external drive. I have a 20TB drive loaded up with lots of "external" media hooked to my desktop. Because it and the desktop never move, it all "just works" fine.

I have this setup in my home- a mix of Mobile Apple stuff and a stationary "central hub" Mac that functions exactly as described. Various phones, iPads and MBs can bring home new media that then is copied to the central hub. Anyone here puts favorites into their albums/playlists/etc and then sync those back to their phones & iPads. In general, with this setup, most of the actual viewing shifts to televisions via AppleTV or iDevices linked to the hub Mac by Home Sharing.

With a mobile Mac operating as your main Mac (between both you and your wife), there is enhanced risk of loss by simply dropping the thing. A cheap Mac Mini- even one a few years old- could be a great alternative to your thinking. Desktops generally don't "hit the road" and thus risk of dropping them is much lower. And again, it's left behind for the "rest" of the fam when the mobile Mac is out and about: enjoy the family media favorites at ANY time vs. only when the MB is at home too.

 

[Thisismattwade]

But wouldn't iCloud be the "hub" for our Library? I mean, it syncs our library to our ATV, MBA, and i devices right now. There's just too much bloat in our particular Library, but my project is to get that cleaned up. Are you trying to help me avoid paying the iCloud monthly cost?

I'm leaning toward a SanDisk external SSD, temporarily (3-6 months based on my workflow) sync my Photos Library to it, and then switch the SanDisk to a Time Machine backup when I finish my project. At that point I may consider taking a "snapshot" of the Photos Library and saving it all to the external SSD.

 

[HobeSoundDarryl]

Yes, that concept breaks the dependency on iCloud and its monthly 💵 toll. If you are fine with that toll, that’s fine too. There is a convenience benefit in shared storage space in the cloud. Some of the “off site” benefit is baked into cloud storage too but I wouldn’t view that as the same as a backup… especially one where you- not complete strangers- act as data caretaker.

The key difference is delete or corrupt a file in one place and iCloud will delete or corrupt it everywhere. On the other hand, delete or corrupt a file in a sizable TM backup and while the most recent (ones) will no longer be backing up to TM (it will be no longer storing a deleted file in new snapshots), older backup snapshots will still have the file (or uncorrupted version of the file) for some period of time (bigger the TM drive, the longer this time).

Just as the kid got control of iPhone and shot a video of your floor, what if instead they managed to delete a bunch of important photos or videos on that same phone? If so, the sync with iCloud would delete them there too. They would be lost.

With complete "strangers" in charge of iCloud, there is also at least some risk that their actions or their hardware or software issues losing or corrupting your data stored there. And iCloud is a much more desirable target to hackers, including those who will seek ransom for data if they get in.

In being your own "iCloud" with HDDs and TM, it is much harder to wipe out all backups... and thus much easier to recover from all of life's unfortunate data-endangering events. And hackers have little incentive to try to hack any individuals storage (because the monetary ransom prize is nowhere near other targets) but even if they do, your 1+ off-site (HDD-not cloud) backup would STILL give you a way to completely foil their efforts.

SSD vs. HDD
An SSD would offer the light weight and portable benefit of easily going with the MB when mobile. However, I suggest big HDDs for the TM storage. Speed is not an issue with TM and big storage simply means more backup history preserved at less cost per GB. TM doesn’t work any better with SSD- no special advantage. The same money spent on SSD would buy more HDD storage.

A good goal is to buy not for only what you need to backup today but also what you will likely need tomorrow. 500GB pruned to 100GB will be likely to grow towards 500GB again in the future (hopefully with much less pruning requirements if everyone does their own (20-50 copies of the same basic shot) pruning on the fly). Thus, you need enough TM storage for the future MAX vs. the today reduction (post prune). HDDs can easily last 5-10 or more years. I've got a few still going on year 20 or so.

Especially given relatively dirt-cheap HDD pricing even for sizable storage, my advice for TM HDD purchases is go big. The general rule of thumb is 2-3X the amount of storage on the Mac(s) to be backed up. But another way to go is up to 4-6X that amount. What "too much" TM storage will do is give you more snapshots before the disc fills and the oldest snapshots start getting deleted to create the space for even newer backups. So extra large storage means much more history from which to recover in any of the various data-loss scenarios.

Maybe you delete the floor video today but then have some reason to need a bit of video of the floor 6 or 12 months from now. With a big TM HDD, you could still recover a long-deleted "useless" file(s). With a small TM HDD or SSD drive, the old snapshot overwrites will come much sooner and going very far back in time to recover deleted files is not possible.

 

[Thisismattwade]

Thanks for following me in this discussion, it's been super helpful.

One last tangential question: if this kind of work is the heaviest lifting I see myself doing on these machines, a MBA is fine, right? I read about single core vs multi-core performance, and I just don't find myself in any of those "Pro" categories. I feel like our use case is what the MBA is made for, and it's an awesome machine. (We use Chrome/Safari/Edge to watch YouTube or NHL occasionally, read PDFs for work/school, play Music, do our taxes, etc, as well.)

 

[HobeSoundDarryl]

Any old Mac can easily manage all personal media: photos, music, videos, etc... sync with iDevices... flow media to AppleTVs... etc. MBair is plenty powerful enough for your tasks.

IMO: the only (modest) flaw with using a MBair as "central hub" is that a mobile computer can go mobile sometimes. Unless the FAM leaves home with it, your cloud dependency is absolute. Any old desktop Mac anchored back at home could give you the option of severing the iCloud dependency- and monthly cost- while still serving up all media to all family members when MBair is away.

If this interests you, consider perhaps a refurb or old Mac Mini or iMac to eventually take over the job of home hub. Then MBair is free to travel and those at home have a computer to use even when it is "on the road." Such a setup would also allow you to significantly free up storage on the MBair, since it would no longer be in charge of whole libraries but perhaps just synched favorite photos/music/video... because the desktop back at home manages all of the entire libraries of media.

 

[Fennec1212]

CONSIDER centralizing your group library access in a new or old DESKTOP Mac- like Mac mini and then maybe 1+ AppleTVs put around the home for viewing photos, videos and music stored on it on the big TV screen(s) in your home (or on iDevice screens).I

Great thread..something I need help with too. HobeSoundDaryl - could you expand a bit on how a centralized library on desktop would work? How are the family members uploading their files to this library? Also wouldn’t this library duplicate the amount of data in iCloud if logged into one of the family accounts? Thanks!

 

[HobeSoundDarryl]

In that centralized concept, think SERVER... like a corporate server... in which everyone's "creations" (media in this case) are stored on that central server so that everyone else has access.

Core idea here is that this SEVERS iCloud usage, so the FAM has access to the shared Music library or shared Photo library via AppleTVs, iDevices and Macs (home sharing) on the home network. If they want to have ready access to selections of media from the central store when away from home, they can drag those into their own device storage for local copies on their iDevices or Macs. No cloud required.

While each person could use the centralized Mac to add their own media, what generally works better is for someone to take the role of caretaker to basically merge in the "best" media and avoid the fluff. For example, if little Sally has shot 40 selfies in pursuit of a perfect one, you probably do NOT want 40 pics of almost the same pic in the centralized Photos library. Instead, identify the best one and add that one to that main library. If Sally wants to keep all 40 on her own iDevice, she can.

While I've used photos as the general example, the same works with music too. If Sally brings home a new CD and others would like to be able to listen to the same songs, (also) rip it to the central store Mac for "home sharing" access by all (and all AppleTVs).

Same with videos.

This is basically a dirt cheap way to have the best of all FAM members media available to the rest of the FAM. Very specific taste stuff NOT interesting to the rest of the FAM remains on individual's own devices (like those 39 other selfie pictures of Sally). For example, Sally really likes some music that no one else in the FAM likes. She gets it on her device but it doesn't go on the central store because only Sally wants to listen to it.

NOW, if there is a desire for this ABOVE this constraint of "home use only," a BETTER way to go is the true server approach, where you would then own your own cloud. A good device for this is Synology or other NAS competitors. Load various Apple-friendly apps on it up with media and then open access to all family members while AWAY from home. Instead of storing on Apple servers, you'd be storing on your own server. Instead of paying Apple forever (and relatively expensive) rent, you instead BUY your own cloud and pay NO rent.

Besides being your own cloud for all key forms of media, Synology is also easily able to be central Time Machine backup for all Macs in the home too... and many other benefits if desired, such as hosting your own website, linking with security cameras for video capture, whole home DVR and many more.

Myself?: I have both- a primary Mac as the central store for home sharing AND Synology for many benefits. For cloud, I only use Apple's free, Dropbox free and sometimes Microsoft & Google free: no rent for a hard drive in the sky when I own my own cloud outright and buy enough space on my iDevices to sync select favorite photo albums, best of music playlists, favorite videos, etc. I can basically offer me and mine ANY amount of free cloud space on that Synology, far above offerings from Apple at very steep rental rates. When one rents too much space like that, it only takes a relatively short amount of time paying high rent to fully cover the cost of OWNING their own cloud instead.