Sync two folders on two external hard drives

[smileman]

Hello,

I would like to mirror selected folders on two external hard drives attached to my MacMini. I do not want to mirror the entire drive contents, just the selected folders.

I periodically make changes to the contents of the folders on each respective drive, and then I will periodically plug-in both drives into my MacMini and would like the contents to synchronize across the selected folders.

So to be clear, there is not a Master Drive in this case; drive 1 has the master folder for my photos, and drive 2 has the master folder for my music. I would like the changes I make to photos on drive 1 to sync to drive 2, and I would like the changes I make to music on drive 2 to sync to drive 1.

Does that make sense? Suggestions? Thanks.

 

[CoastalOR]

I think ChronoSync would work for your needs:
https://www.econtechnologies.com/chronosync/overview.html

ChronoSync Documentation:
https://www.econtechnologies.com/chronosync/documentation.html

 

[I'm a ROb]

Depending on your bash/terminal experience: you could use rsync and schedule it with crontab.

 

[MacUser2525]

Hello,

I would like to mirror selected folders on two external hard drives attached to my MacMini. I do not want to mirror the entire drive contents, just the selected folders.

I periodically make changes to the contents of the folders on each respective drive, and then I will periodically plug-in both drives into my MacMini and would like the contents to synchronize across the selected folders.

So to be clear, there is not a Master Drive in this case; drive 1 has the master folder for my photos, and drive 2 has the master folder for my music. I would like the changes I make to photos on drive 1 to sync to drive 2, and I would like the changes I make to music on drive 2 to sync to drive 1.

Does that make sense? Suggestions? Thanks.

Sure makes sense as has been mentioned a bash script using rsync and either a cron job or launchd if wanting it done automatically when both drives are present. Here is the basic script to do exactly what you need.

#!/bin/bash

# A script to sync my photos and music directories on my external drives using rsync.

rsync -avP --delete /Volumes/drive\ 1/photos /Volumes/drive\ 2/
rsync -avP --delete /Volumes/drive\ 2/music /Volumes/drive\ 1/

Copy and paste to a text file say photo_music_sync.sh then in Terminal use chmod +x /path/to/photo_music_sync.sh to make it executable. To run it in Terminal use /path/to/photo_music_sync.sh and hit enter key. Or you could copy to say /usr/local/bin then it is available system wide so only photo_music_sync.sh would be needed. Personally I use a Bin directory in my home directory for the dozens of scripts I have for this type of thing if you wanted to do the same then add this to the file .bash_profile in your home directory. The first nano line edits the file to add the next four lines to it then sources the newly edited file to have the new path available in the current Terminal shell.

nano ~/.bash_profile

# set PATH so it includes user's private bin if it exists
if [ -d ~/Bin ] ; then
    PATH=~/Bin:"${PATH}"
fi

source ~/.bash_profile

 

[Huntn]

Sure makes sense as has been mentioned a bash script using rsync and either a cron job or launchd if wanting it done automatically when both drives are present. Here is the basic script to do exactly what you need.

#!/bin/bash

# A script to sync my photos and music directories on my external drives using rsync.

rsync -avP --delete /Volumes/drive\ 1/photos /Volumes/drive\ 2/
rsync -avP --delete /Volumes/drive\ 2/music /Volumes/drive\ 1/

Copy and paste to a text file say photo_music_sync.sh then in Terminal use chmod +x /path/to/photo_music_sync.sh to make it executable. To run it in Terminal use /path/to/photo_music_sync.sh and hit enter key. Or you could copy to say /usr/local/bin then it is available system wide so only photo_music_sync.sh would be needed. Personally I use a Bin directory in my home directory for the dozens of scripts I have for this type of thing if you wanted to do the same then add this to the file .bash_profile in your home directory. The first nano line edits the file to add the next four lines to it then sources the newly edited file to have the new path available in the current Terminal shell.

nano ~/.bash_profile

# set PATH so it includes user's private bin if it exists
if [ -d ~/Bin ] ; then
    PATH=~/Bin:"${PATH}"
fi

source ~/.bash_profile

I have invested in two different synch programs, one (Sync) s out of date (they are supposed to be working on a MacOS updated version and the other one (SyncMate) is not working properly.

Recently I've tried to follow the instructions in this article at chriswrites.com to use tools within the MacOS to sync folders, but when I try to run a sync method 2 Terminal, or method 3 automator, I get an error message. Anyone care to give this article a gander and tell me possibly where I'm tripping up?

I know without showing you what I'm typing that that is a tall order. I'm just wondering if there is a common mistake what people make when typing in these types of commands. I am familiar with the terminal and pathing, but when I drag a folder to the terminal to create a path, it does not look like it does in the examples. For automator, I think I either have to many or not enough ", / or \.

In your example above:
rsync -avP --delete /Volumes/drive\ 1/photos /Volumes/drive\ 2/
rsync -avP --delete /Volumes/drive\ 2/music /Volumes/drive\ 1/

Where does the original folder end and the folder to copy to begin?
Thanks!

 

[MacUser2525]

In your example above:
rsync -avP --delete /Volumes/drive\ 1/photos /Volumes/drive\ 2/

Where does the original folder end and the folder to copy to begin?

The original folder being synced is the in the first part of the line after the rsync options given the /Volumes/drive\ 1/photos in there that tells rsync to sync the directory photos on/in the /Volumes/drive\ 1 volume. The synced to destination is in the second part the /Volumes/drive\ 2/ volume. This when ran results in the photos directory on the /Volumes/drive\ 1 drive being synced to the /Volumes/drive\ 2/ drive in what will be a photos directory on it once the sync is done. In short when you want to sync a directory you go with this rsync options_used /Volumes/Drive/Directory_To_Sync then the destination where you want the directory to be created on/synced to the /Volumes/Drive_To_Sync_To/. You do not put the directory in the synced to part rsync does that automatically and the trailing / on the destination is needed to let it know to sync in that drives folder structure not the entire drive it self as the sync destination.

Edit: here is a snippet of my script I use to backup my main storage machine to a backup for my TV shows I have stored, it may help you see what is needed for your script you are trying to do.

MacUser2525:~$ cat Bin/primary.sh
#!/bin/bash

echo "Syncing TV_x265"

rsync -avP --delete --exclude ".*/" $1 /Volumes/TV_x265 storage1@192.168.0.102:/Sans_Digital/

Now here you can see it is a bash script by the first line the #!/bin/bash then I echo to the Terminal what it is going to do namely sync my tv shows that are in x265. Then the real task at hand use rysnc to archive the -a verbosely the v using progress option the P which allows you to killl of the process and have it resume without having to start from the beginning a file that has been partially copied. The --delete will delete any file on the destination that is not present on the source directory. The --exclude ".*/" skips any of the hidden files that OSX uses like the .ds_store as it is linux machine I am syncing to and they are not needed there. The $1 allows me to pass another option into the script at run time like a --dry-run to have it show me the changes that will be made without actually doing anything before I really run the script to have it done. I am syncing the directory /Volumes/TV_x265 on my source to the user storage1 at the 192.168.0.102 IP address on my home network in the destination directory /Sans_Digital/.

 

[Huntn]

The original folder being synced is the in the first part of the line after the rsync options given the /Volumes/drive\ 1/photos in there that tells rsync to sync the directory photos on/in the /Volumes/drive\ 1 volume. The synced to destination is in the second part the /Volumes/drive\ 2/ volume. This when ran results in the photos directory on the /Volumes/drive\ 1 drive being synced to the /Volumes/drive\ 2/ drive in what will be a photos directory on it once the sync is done. In short when you want to sync a directory you go with this rsync options_used /Volumes/Drive/Directory_To_Sync then the destination where you want the directory to be created on/synced to the /Volumes/Drive_To_Sync_To/. You do not put the directory in the synced to part rsync does that automatically and the trailing / on the destination is needed to let it know to sync in that drives folder structure not the entire drive it self as the sync destination.

Edit: here is a snippet of my script I use to backup my main storage machine to a backup for my TV shows I have stored, it may help you see what is needed for your script you are trying to do.

MacUser2525:~$ cat Bin/primary.sh
#!/bin/bash

echo "Syncing TV_x265"

rsync -avP --delete --exclude ".*/" $1 /Volumes/TV_x265 storage1@192.168.0.102:/Sans_Digital/

Now here you can see it is a bash script by the first line the #!/bin/bash then I echo to the Terminal what it is going to do namely sync my tv shows that are in x265. Then the real task at hand use rysnc to archive the -a verbosely the v using progress option the P which allows you to killl of the process and have it resume without having to start from the beginning a file that has been partially copied. The --delete will delete any file on the destination that is not present on the source directory. The --exclude ".*/" skips any of the hidden files that OSX uses like the .ds_store as it is linux machine I am syncing to and they are not needed there. The $1 allows me to pass another option into the script at run time like a --dry-run to have it show me the changes that will be made without actually doing anything before I really run the script to have it done. I am syncing the directory /Volumes/TV_x265 on my source to the user storage1 at the 192.168.0.102 IP address on my home network in the destination directory /Sans_Digital/.

I'm a novice at this although I have used the Terminal before to define file paths. In the past dragging in folders into the Terminal has been a good way to establish pathing, but this time it's not working so well.

Today using this link: https://www.bananica.com/Geek-Stuff/Synchronize-two-folders-on-a-Mac-with-Automator-and-Rsync/ I was actually able to create an automator script that syncs up the two files I choose, but I would prefer to have the folders set up in advance.

So with the goal of creating an Automator script, I still have to insert a terminal script. Based on that here is my example: I have two folders I am syncing on two different hard drives:
MBBB1 (Partition)/MBBB1 Backed up (folder)
WDFire 2 (Partition)/MBB1 Backed up (folder)

Now in the Terminal if I drag these folders into the Terminal, these are how the paths are displayed.
/Volumes/MBBB1/MBBB1\ Backed\ Up
/Volumes/WDFire2/MBBB1\ Backed\ Up

I'm trying to put together a script so the the second folder is synched with the first folder and old files are deleted. I'm thinking it should look something like this:
rsync -aE –delete ~/ <source folder> / “<destination folder>” or should there be two dashes like:
rsync -aE -–delete ~/ <source folder> / “<destination folder>” ?

But in the examples I found on line sometimes there are quotes ("") around both the source and destination file, and sometimes just around the destination file and sometimes neither. So it might look like one of the following:

rsync -aE –delete ~/Volumes/MBBB1/MBBB1\ Backed\ Up “/Volumes/WDFire2/MBBB1\ Backed\ Up”
or
rsync -aE –delete "~/Volumes/MBBB1/MBBB1\ Backed\ Up" “/Volumes/WDFire2/MBBB1\ Backed\ Up”
or
rsync -aE –delete ~/Volumes/MBBB1/MBBB1\ Backed\ Up /Volumes/WDFire2/MBBB1\ Backed\ Up

If it's not too much trouble could you tell me what you think the successful script would be? I've tried different variations and am continuing to get errors when using the terminal.
Thanks!

 

[MacUser2525]

rsync -aE –delete ~/Volumes/MBBB1/MBBB1\ Backed\ Up “/Volumes/WDFire2/MBBB1\ Backed\ Up”
or
rsync -aE –delete "~/Volumes/MBBB1/MBBB1\ Backed\ Up" “/Volumes/WDFire2/MBBB1\ Backed\ Up”
or
rsync -aE –delete ~/Volumes/MBBB1/MBBB1\ Backed\ Up /Volumes/WDFire2/MBBB1\ Backed\ Up

There is no /Volumes in the ~ (home directory of your user) remove the ~ from the line(s) the /Volumes is a root location and once you have the \ in a line to escape the illegal space in this case a " is not needed. As well on the destination part you do not want the directory specified unless you want a duplicte like /Volumes/WDFire2/MBBB1\ Backed\ Up/MBBB1\ Backed\ Up for the destination directory so your line should be something like this.

rsync -aE –delete /Volumes/MBBB1/MBBB1\ Backed\ Up /Volumes/WDFire2/

This will sync the directory /Volumes/MBBB1/MBBB1\ Backed\ Up to the directory /Volumes/WDFire2/ in a directory named MBBB1 Backed Up at that location, rsync will create the MBBB1 Backed Up directory if it does not already exist at that location during the syncing process.

 

[Huntn]

There is no /Volumes in the ~ (home directory of your user) remove the ~ from the line(s) the /Volumes is a root location and once you have the \ in a line to escape the illegal space in this case a " is not needed. As well on the destination part you do not want the directory specified unless you want a duplicte like /Volumes/WDFire2/MBBB1\ Backed\ Up/MBBB1\ Backed\ Up for the destination directory so your line should be something like this.

rsync -aE –delete /Volumes/MBBB1/MBBB1\ Backed\ Up /Volumes/WDFire2/

This will sync the directory /Volumes/MBBB1/MBBB1\ Backed\ Up to the directory /Volumes/WDFire2/ in a directory named MBBB1 Backed Up at that location, rsync will create the MBBB1 Backed Up directory if it does not already exist at that location during the syncing process.

Thanks for your help. Would it be wrong or inefficient to list the target folder in the back up drive in the command? I'm thinking I might change the name of the master folder to MBBB1 Master and the backed up folder to MBBB1 Backed Up.

 

[MacUser2525]

Thanks for your help. Would it be wrong or inefficient to list the target folder in the back up drive in the command? I'm thinking I might change the name of the master folder to MBBB1 Master and the backed up folder to MBBB1 Backed Up.

As the folder containing the backups? If so will not matter but the rsync will only compare indentical folder names so they (the backed up folders) need to named the same for it to work.

 

[Huntn]

As the folder containing the backups? If so will not matter but the rsync will only compare indentical folder names so they (the backed up folders) need to named the same for it to work.

Oh.... I'll keep them with the same name.

 

[Huntn]

Thanks for the help in this thread. I have created a backup using automator, but I've also purchased GoodSync which seems to be working well for my needs. I like constant automatic updating.