Using "rsync" for differential backups...

Hi all,

I'm sure this is old hat to many of you, but I've promised to give some examples of the use of rsync and its remote update protocol. The following commands perform a few differential backup tasks and should serve as examples for your own rsync backups. Of course, there are numerous other ways of using the rsync commands, but these are a few of the ways that I find useful.

Note: Since I use different non-standard ports for my secure shell (SSH) servers on different machines, I have to set the proper SSH ports in the rsync commands. If you use the standard port of 22, you will not need to specify the port (-p) on the ssh command.

Scenario:
Two machines: server.private and har.private.
SSH on server.private uses port YYYYY.
SSH on har.private uses port XXXXX.
Two primary repository directories: server.private:/Users/switon/Repositories and server.private:/Shared Items/Repos/Repos are to be cloned.

The subversion repositories contained in these two directories are to be cloned to three volumes: SC-MyPassport/Repos and SCPegasus mounted on har.private, and SCDrobo mounted on server.private using the rsync Terminal commands from har.private:

Code:

# Use rsync on har.private to backup repositories from server.private to har.private:
rsync --rsh='ssh -pYYYYY' -avz server.private:/Users/switon/Repositories /Volumes/SC-MyPassport/Repos
rsync --rsh='ssh -pYYYYY' -avz 'server.private:/Shared\ Items/Repos/Repos' /Volumes/SC-MyPassport/Repos
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Repos /Volumes/SC-Seagate
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Repos /Volumes/SCPegasus

In the above commands, notice the judicious use of quote marks ('). The "rsync" command takes a "--rsh=" switch to set the secure shell with port YYYYY as the communications channel from server.private to har.private. This switch also sets the SSH port to XXXXX as the communications channel from har.private to har.private. When a space is included in the directory name (as in the "Shared Items" directory) , then quote marks surrounding the pathname are used and the space is delimited with a backslash (\ ).

Scenario:
Two volumes mounted on a different machine.

The "rsync" command cannot copy between two drives mounted on a different machine, so in this case I first ssh to the other machine in order to perform the "rsync" commands from the other machine:

Code:

# SSH to server.private to rsync between two drives mounted on server.private:
ssh -pYYYYY switon@server.private
rsync --rsh='ssh -pYYYYY' -avz /Users/switon/Repositories /Volumes/SCDrobo/Repos
cd /Shared\ Items/Repos
rsync --rsh='ssh -pYYYYY' -avz '/Repos' /Volumes/SCDrobo/Repos

Scenario:
Backup iTunes Library from local machine to shared volume.

To backup videos in an iTunes library named iTunes-Videos from har.private to the shared volume SCDrobo mounted on server.private:

Code:

# Backup iTunes-Videos from har.private to shared volume SCDrobo mounted on server.private:
rsync --rsh='ssh -pXXXXX' -avz 'har.private:/Volumes/SC-MyPassport/Videos/iTunes-Videos' /Volumes/SCDrobo/Videos

And lastly, backup the videos to two additional drives, SC-Seagate and SCPagasus:

Code:

# Backup the iTunes-Videos library to SC-Seagate and SDPegasus:
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Videos/iTunes-Videos /Volumes/SC-Seagate/Videos
rsync --rsh='ssh -pXXXXX' -avz /Volumes/SC-MyPassport/Videos/iTunes-Videos /Volumes/SCPegasus/Videos

Notice that these commands perform archived (-a), checksummed, and compressed (-z) differential backups, thus the cloned subversion repositories and iTunes libraries on each of the backup volumes are identical to one another and thus any of them can be used in Subversion or iTunes. These backup clones are also differential, so only new material added to any of the repositories or libraries will be synced during the next "rsync" backup. In other words, if you add a new video with thumbnails to the iTunes-Videos library, then a subsequent "rsync" will only transfer the new video and its associated thumbnails to the backups.

`Archived' means that ownerships and permissions and links are preserved, `checksummed' means that the original and backup copies are identical, and `compressed' means that the least amount of data is communicated over the network.

Caveat:
Of course, you must have to appropriate permissions to perform the Reads and Writes to each of these volumes. Sometimes you may encounter a situation where you do not have the proper permission to allow an rsync write in one direction, but then at these times you can often log into the other machine and use "rsync" from it to perform the backup (a "pull" instead of a "push" rsync).

Regards,
Switon

 

RE: Thanks...

Thanks...much appreciated.

Regards,
Switon