Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Need help with Copying Files using Terminal

R

roachclip

macrumors newbie
Original poster
Sep 2, 2015
6
0
I have two external hard drives I use to archive large media files such as .mov and .fcpx bundles. One drive is the primary while the other is an archive / backup.

Right now I'm using Rsync to backup everything once a week but since I have 1.5 TB of data it takes a long time. I would like to know if there is a terminal command that will allow me to copy only new files or files that have been changed.

Thanks,
 
roachclip said:
I have two external hard drives I use to archive large media files such as .mov and .fcpx bundles. One drive is the primary while the other is an archive / backup.

Right now I'm using Rsync to backup everything once a week but since I have 1.5 TB of data it takes a long time. I would like to know if there is a terminal command that will allow me to copy only new files or files that have been changed.

Thanks,
Click to expand...

That is what is called an "incremental backup" and that is what rsync does by default. If you run it against the same source and target it should only copy new or modified files each time. Perhaps you have added a switch that is overriding the default behavior?
 
roachclip said:
Right now I'm using Rsync to backup everything once a week but since I have 1.5 TB of data it takes a long time. I would like to know if there is a terminal command that will allow me to copy only new files or files that have been changed.
Click to expand...

That's exactly what rsync is supposed to do. Copy over only the files that change. Are you sure that's not working? If you turn on logging with "-v", it will give you a "speedup" number at the end telling you how much time it saved by not copying some files. What exact commands are you executing? You should probably be using the "-a" flag to move the files to the external drive.

So in other words:

Code: 
rsync -av directory/ /Volumes/external/directory/

Another possibility is that rsync is working quite right because of timestamp issues. By default, I believe rsync uses file timestamps to figure out what to to copy and what not to copy. Maybe something is going wrong and the date can't be set properly on the external drive. What filesystem is on the external drive? That might matter.
 
  • Like
Reactions: Weaselboy
rsync's -c parameter will do skips based on checksum, not mod-time or size. This might be more reliable for you if you're having issues.

Considering ChronoSync in all likelihood is just using rsync, you're probably going to have the same issues with ChronoSync.
 
Weaselboy said:
That is what is called an "incremental backup" and that is what rsync does by default. If you run it against the same source and target it should only copy new or modified files each time. Perhaps you have added a switch that is overriding the default behavior?
Click to expand...

Ah, that very well could be. Here's the code I use:
sudo rsync -vaE --progress /Volumes/Studio/ /Volumes/Failover/

Do one of those switches cause the override?
 
Last edited by a moderator:
mfram said:
That's exactly what rsync is supposed to do. Copy over only the files that change. Are you sure that's not working? If you turn on logging with "-v", it will give you a "speedup" number at the end telling you how much time it saved by not copying some files. What exact commands are you executing? You should probably be using the "-a" flag to move the files to the external drive.

So in other words:

Code: 
rsync -av directory/ /Volumes/external/directory/

Another possibility is that rsync is working quite right because of timestamp issues. By default, I believe rsync uses file timestamps to figure out what to to copy and what not to copy. Maybe something is going wrong and the date can't be set properly on the external drive. What filesystem is on the external drive? That might matter.
Click to expand...

Mac OS Extended (Journaled) is the format of the Source and Destination
 
roachclip said:
Ah, that very well could be. Here's the code I use:
sudo rsync -vaE --progress /Volumes/Studio/ /Volumes/Failover/

Do one of those switches cause the override?
Click to expand...
I don't believe any of the switches would over ride the default incremental backup.
 
roachclip said:
I eliminated the E switch (extended attribute) and it seemed to work, only copying over the new files I added today. After that the routine terminated.
Click to expand...
Interesting... one would not think that switch would matter. Glad you figured it out. :)
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back