Go Back   MacRumors Forums > Apple Systems and Services > OS X

Reply
 
Thread Tools Search this Thread Display Modes
Old Feb 18, 2013, 06:12 PM   #1
switon
macrumors 6502a
 
Join Date: Sep 2012
Using "rsync" for differential backups...

Hi all,

I'm sure this is old hat to many of you, but I've promised to give some examples of the use of rsync and its remote update protocol. The following commands perform a few differential backup tasks and should serve as examples for your own rsync backups. Of course, there are numerous other ways of using the rsync commands, but these are a few of the ways that I find useful.

Note: Since I use different non-standard ports for my secure shell (SSH) servers on different machines, I have to set the proper SSH ports in the rsync commands. If you use the standard port of 22, you will not need to specify the port (-p) on the ssh command.

Scenario:
Two machines: server.private and har.private.
SSH on server.private uses port YYYYY.
SSH on har.private uses port XXXXX.
Two primary repository directories: server.private:/Users/switon/Repositories and server.private:/Shared Items/Repos/Repos are to be cloned.

The subversion repositories contained in these two directories are to be cloned to three volumes: SC-MyPassport/Repos and SCPegasus mounted on har.private, and SCDrobo mounted on server.private using the rsync Terminal commands from har.private:

Code:
# Use rsync on har.private to backup repositories from server.private to har.private:
rsync --rsh='ssh -pYYYYY' -avz server.private:/Users/switon/Repositories /Volumes/SC-MyPassport/Repos
rsync --rsh='ssh -pYYYYY' -avz 'server.private:/Shared\ Items/Repos/Repos' /Volumes/SC-MyPassport/Repos
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Repos /Volumes/SC-Seagate
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Repos /Volumes/SCPegasus
In the above commands, notice the judicious use of quote marks ('). The "rsync" command takes a "--rsh=" switch to set the secure shell with port YYYYY as the communications channel from server.private to har.private. This switch also sets the SSH port to XXXXX as the communications channel from har.private to har.private. When a space is included in the directory name (as in the "Shared Items" directory) , then quote marks surrounding the pathname are used and the space is delimited with a backslash (\ ).

Scenario:
Two volumes mounted on a different machine.

The "rsync" command cannot copy between two drives mounted on a different machine, so in this case I first ssh to the other machine in order to perform the "rsync" commands from the other machine:

Code:
# SSH to server.private to rsync between two drives mounted on server.private:
ssh -pYYYYY switon@server.private
rsync --rsh='ssh -pYYYYY' -avz /Users/switon/Repositories /Volumes/SCDrobo/Repos
cd /Shared\ Items/Repos
rsync --rsh='ssh -pYYYYY' -avz '/Repos' /Volumes/SCDrobo/Repos
Scenario:
Backup iTunes Library from local machine to shared volume.

To backup videos in an iTunes library named iTunes-Videos from har.private to the shared volume SCDrobo mounted on server.private:

Code:
# Backup iTunes-Videos from har.private to shared volume SCDrobo mounted on server.private:
rsync --rsh='ssh -pXXXXX' -avz 'har.private:/Volumes/SC-MyPassport/Videos/iTunes-Videos' /Volumes/SCDrobo/Videos
And lastly, backup the videos to two additional drives, SC-Seagate and SCPagasus:

Code:
# Backup the iTunes-Videos library to SC-Seagate and SDPegasus:
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Videos/iTunes-Videos /Volumes/SC-Seagate/Videos
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Videos/iTunes-Videos /Volumes/SCPegasus/Videos
Notice that these commands perform archived (-a), checksummed, and compressed (-z) differential backups, thus the cloned subversion repositories and iTunes libraries on each of the backup volumes are identical to one another and thus any of them can be used in Subversion or iTunes. These backup clones are also differential, so only new material added to any of the repositories or libraries will be synced during the next "rsync" backup. In other words, if you add a new video with thumbnails to the iTunes-Videos library, then a subsequent "rsync" will only transfer the new video and its associated thumbnails to the backups.

`Archived' means that ownerships and permissions and links are preserved, `checksummed' means that the original and backup copies are identical, and `compressed' means that the least amount of data is communicated over the network.

Caveat:
Of course, you must have to appropriate permissions to perform the Reads and Writes to each of these volumes. Sometimes you may encounter a situation where you do not have the proper permission to allow an rsync write in one direction, but then at these times you can often log into the other machine and use "rsync" from it to perform the backup (a "pull" instead of a "push" rsync).

Regards,
Switon
switon is offline   1 Reply With Quote
Old Feb 21, 2013, 04:35 PM   #2
maflynn
Moderator
 
maflynn's Avatar
 
Join Date: May 2009
Location: Boston
[MOD NOTE]
Thread moved back to OSX sub forum
__________________
~Mike Flynn
maflynn is online now   0 Reply With Quote
Old Feb 21, 2013, 08:37 PM   #3
switon
Thread Starter
macrumors 6502a
 
Join Date: Sep 2012
RE: Thanks...

Quote:
Originally Posted by maflynn View Post
[MOD NOTE]
Thread moved back to OSX sub forum
Thanks...much appreciated.

Regards,
Switon
switon is offline   0 Reply With Quote
Old Aug 19, 2013, 08:37 AM   #4
MacTribe
macrumors member
 
Join Date: Dec 2010
Location: London
Nice article and usage of rsync. I use it all the time for clients servers, linux to mac etc. I'll bookmark this, never know when you'll the extra switches!
__________________
iPhone 5 / MacBook Pro 15" / Hackintosh i5 Custom
MacTribe | Runlevel | Tech Blog | Tweets
MacTribe is offline   0 Reply With Quote

Reply
MacRumors Forums > Apple Systems and Services > OS X

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Similar Threads
thread Thread Starter Forum Replies Last Post
Superduper and Rsync backups dimme Mac Applications and Mac App Store 15 Aug 12, 2014 12:47 PM
Huge "Backups" section in my Mac HD? soxguy17 MacBook Pro 6 Nov 21, 2013 12:50 PM
Resolved: Can't install Mavericks because "This disk is used for Time Machine backups" xpipe OS X Mavericks (10.9) 2 Oct 23, 2013 10:03 AM
"Open With..." in Finder displays old version backups Lastmboy OS X 10.8 Mountain Lion 4 May 31, 2013 12:30 PM
iOS/connectivity problems saved into backups? Trying to understand "restore" chishifu iPhone Tips, Help and Troubleshooting 2 Mar 6, 2013 12:52 PM

Forum Jump

All times are GMT -5. The time now is 01:53 PM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC