Superduper and Rsync backups

[dimme]

I am a big believer in backups. I have a nice backup plan for my data. I have recently been evaluating my strategy which uses both Superduper and arRsync which is a gui for rsync. I have been doing some testing and I have noticed if I copy data to a external disk with Arrsync and then at a later date update it with Superduper, Superduper will rewrite all the data not just the updater files. (I do have super duper set to update). I contacted super duper about this and that say that arrcync in not coping the media data. If I do the test in reverse, rsync just updates the super duper data. So that leaves me wondering, should I not be using arrcync. For backing up only certain directories arrcync is better suited that super duper.
Any thoughts?

 

[Weaselboy]

I think the SD dev meant "metadata" and this is part of the extended file attributes found in OS X.

The problem is OS X still ships with an ancient 2.6.9 version of rsync due to some licensing issues with the newer versions. 2.6.9 does not fully support OS X metadata and I suspect that is why you are having this issue.

What you can do if you still want to try this install a newer (open source) version of rsync. You can read about it here.

 

[dimme]

I understand now that Superduper copies the medadata and rsync does not. So the question is, are backups made with rsync or GUI rsync front ends not true backups. Should I not be trusting my data to these backups?

 

[Weaselboy]

I understand now that Superduper copies the medadata and rsync does not. So the question is, are backups made with rsync or GUI rsync front ends not true backups. Should I not be trusting my data to these backups?

You may be losing some meta data for files. For example, in OS X Spotlight uses meta data for searching things like file labels, and that might not be getting copied with the files. But as far as say a plain document file, the file itself would still be backed up.

From what I read, it looks like you can update the version of sync that came with OS X and that would solve the problem.

Or you could look at something like Carbon Copy Cloner that is an rsync GUI, but it uses its own version of rsync and not the one that comes with OS X.

 

[ZVH]

You could probably replace the version of rsync included in arRsync. It depends on whether or not they're bundling a copy of rsync with the application. If they are, I don't know why they wouldn't use the newer version. I'm guessing that they just launch rsync using the default OS version.

I have a command line script that I use to update files to various drives using rsync, but I'm using the newer version. I believe both SuperDuper and CCC both use this.

To see if the newer rsync is in SuperDuper it's usually inside the Resources folder of the application. You could check by doing the following:

1. Single click on the application icon.
2. When the icon is highlighted right click on it and when the menu comes down, click on show package contents.
3. A new finder window will appear with a folder named "Contents"
4. Double click on Contents
5. Another window will open. Double click on Resources.

If rsync is stored in there, you'll see it. You could copy (not move) it to another location and run rsync via the command line to see what version it is. If it's the newer version, you might want to try saving the old version of rsync to something like rsync.old and then moving the new one in its place.

There's no law saying a developer has to put that in Resources, they could put it anywhere if they like, they just typically do it in Resources. You could probably also try the same thing with arRsync, and possibly replace that version of rsync with the one from SuperDuper.

A lot of these are obviously kind of experimental, so be careful and make sure you save the originals somewhere safe before doing anything. rsync is open source so this shouldn't be a license problem of any type.

 

[dimme]

Thanks for all the insight. I think I have a pretty good understanding of what is going on. So if I continue to use the built in version of rcync all my data (word, excel and pictures) but what is missing is spotlight data or other files that os X may use?

 

[Weaselboy]

To see if the newer rsync is in SuperDuper it's usually inside the Resources folder of the application.

I know CCC uses a custom version of rsync, but I don't believe SD uses rsync at all.

----------

Thanks for all the insight. I think I have a pretty good understanding of what is going on. So if I continue to use the built in version of rcync all my data (word, excel and pictures) but what is missing is spotlight data or other files that os X may use?

You won't miss any files, what you may miss is metadata from those files.

 

[ZVH]

Thanks for all the insight. I think I have a pretty good understanding of what is going on. So if I continue to use the built in version of rcync all my data (word, excel and pictures) but what is missing is spotlight data or other files that os X may use?

It's the meta data Weasel Boy is referring to that's the problem. Meta data, is part of a file and it's actually stored in the index information about the file on the hard drive. When the old rsync copies one of these, instead of copying it as a regular file along with the meta data it actually splits it into pieces. This is why if you copied a file named abd.def you might end up with abd.def and something that looks like junk and often starts with the "._" characters.

I would think that one of the rsync development sites would have that binary available for download. Another option is to download the code, install XCode, and build it yourself.

 

[dimme]

So after a few week of using super to back up my data from my server everything on that end is fine. But what I need now is a good tool preferably free or almost free (money is real tight now) to update my data (mostly pictures & lightroom catalogs) from my macbook and workstations to my server. I thinking I want to stay away from anything that used rsync. I a shame super duper will not work but it will only save the info on the server as a disk image.

 

[Weaselboy]

So after a few week of using super to back up my data from my server everything on that end is fine. But what I need now is a good tool preferably free or almost free (money is real tight now) to update my data (mostly pictures & lightroom catalogs) from my macbook and workstations to my server. I thinking I want to stay away from anything that used rsync. I a shame super duper will not work but it will only save the info on the server as a disk image.

I still think CCC would be the best fit for this, but it is not free and it does use rysnc, but an updated version of rsync that handles meta data fine.

 

[dimme]

I gave CCC a try and I wall allow you to back one or 2 folders which is what I am looking for. However on the backup drive it will save the whole golfer path. I would just like to backup the information in the folder.

As a side thought when you copy a folder full of data to a windows server is the medadata striped away like rsync does?

 

[Vulkan]

Does anyone know if there is a way to schedule arrsync to run automatically?

On CCC I want to test it to replicate the contents of a folder across different servers Unidirectionally.

for example:

Server1 - folderABC ----> (Replicate) to Server2, Server3, Server4

Would I be able to do this on CCC?

 

[Weaselboy]

Does anyone know if there is a way to schedule arrsync to run automatically?

On CCC I want to test it to replicate the contents of a folder across different servers Unidirectionally.

for example:

Server1 - folderABC ----> (Replicate) to Server2, Server3, Server4

Would I be able to do this on CCC?

Yes, fairly easily. Just set up each sync job like you want then click "schedule task" in CCC and pick the time etc you want it to run.

 

[ChrisA]

I hope yu are also running Time machine.

ccc, SD, and rsync all have a problem that they over write old data with new data and do NOT save older versions of your data.

Look at this scenario:

1) Lets so you have a good Word document and a good backup of the Word document. You are happy

2) The word document get corrupted somehow. Either because of a software bug or you accidentally delete a few pages and did not notice. Either way the files is damaged.

3) your Rsync script runs and copies the corrupted Word document over top of the only good copy of the document you have wiping out your backup and replacing it with the corrupted file.

4) A week later you open the files and you are NOT happy. You go looking for an old backup and find every one of them is the same.

Had you used a better system like TM then you have have the entire change history of the Word document. All the old versions would be there. It is OK to make exact copies like you are doing but only as PART of a bigger system

 

[Vulkan]

I hope yu are also running Time machine.

ccc, SD, and rsync all have a problem that they over write old data with new data and do NOT save older versions of your data.

On my scenario, we are setting up our Filewave infrastructure to replicate across multiple servers for failover. So if one server goes down the data of that server is available on a secondary server. So all we have to do is remote into our backup server and turn on the Filewave server and it will continue to function.

I was hoping to do this via a normal replicating share with OSX server but apparently that functionality is no longer there.

Using Time Machine server on an enterprise deployment is really not a wise idea.

 

[Alrescha]

ccc, SD, and rsync all have a problem that they over write old data with new data and do NOT save older versions of your data.

Well, rsync does what you tell it. If you tell it to keep old versions, it will keep old versions. CCC too, if I recall correctly.

A.