PDA

View Full Version : backup script with find and tar




wrathkeg
Sep 28, 2010, 04:14 AM
Hi all,

I have written a fairly simple script to back up files. They key bit is this line of code, which finds everything changed in the last day, and makes a tar file containing all those files:

find . -ctime -1 | xargs tar --no-recursion -czf $HOME/$TARFILE

If I change a filename but not the contents, the file isn't picked up by this line of code. I can get 'find' to identify files with changed name by using 'atime' rather than 'ctime'. The problem would be my script would back up everything I had accessed, rather than just those things which had either changed contents or a changed filename. Does anyone have any suggestions as to how to modify the above line to suit my needs? I am trying to avoid a rewrite of the whole script, or adoption of a new method....

Thanks in advance,

wrathkeg



ianray
Sep 28, 2010, 06:11 AM
Despite your wish to use the same method, I would have to suggest rsync.

One alternative is to compare the set of files on disk versus the archive, then remove deleted files and add renamed files to the archive. You could write some nice Perl to do this :)


tar -tzf $HOME/$TARFILE > tar.mf
find . -type f > disk.mf
echo files on disk, but not in tar
diff disk.mf tar.mf | grep "^<" | cut -b3-
echo files in tar, but not on disk
diff disk.mf tar.mf | grep "^>" | cut -b3-

willieva
Sep 28, 2010, 06:50 AM
would -mtime do it?

wrathkeg
Sep 28, 2010, 07:10 AM
Thanks for the suggestions.

I had tried to use rsync, but I am trying to back up to Dropbox and I don't have enough space on there to store everything I want to keep backed up, so I just want to backup (upload) things I have changed in the last few days. From time to time I back up everything to an external hard drive.

As far as I understand it, ctime only tells you when the file itself was last changed, and this includes changes to ownership and permissions, but not to filenames. Seems strange to me, but I'm sure there's a good reason.

wrathkeg
Sep 28, 2010, 07:47 AM
it occurs to me that one way round this might be to use the 'touch' command on a file after running 'mv' on it (I do pretty much all of my file manipulation from the terminal). I wonder if I can set up an alias for mv to do that........

cqexbesd
Sep 28, 2010, 08:18 AM
If I change a filename but not the contents, the file isn't picked up by this line of code.

If you kept an uncompressed copy of the tar file locally you might be able to use the -u option to tar. This would just make the tar file bigger each time but you said you backup in full periodically so you could then delete the tar file and start from scratch. You can still compress the tar file before upload.

HTH,

Andrew

wrathkeg
Sep 29, 2010, 05:20 AM
If you kept an uncompressed copy of the tar file locally you might be able to use the -u option to tar.
Great suggestion: thanks. I now have a script like this, which seems to do just what I wanted.
#!/bin/bash
# makes the dropbox folder if not there; prevents output to terminal
mkdir -p $HOME/Dropbox/Work
# sets up the filename
TARFILE=$HOME/dropboxbackup.tar
# changes to the folder you want to search recursively
cd $HOME/Work
# makes the tarfile in case it doesn't already exist
touch $TARFILE
# finds all recently modified files and tars them: the argument to
# -ctime specifies how recent things have to be, in days from current
# time. Currently set to 7 days. This means you need to run this at
# least once per week to keep things backed up.
find . -ctime -7 | xargs tar --no-recursion -uf $TARFILE
# goes into Dropbox
cd $HOME/Dropbox/Work
# unzips the file
tar xf $TARFILE
# changes to the directory containing tar file and deletes it
# get back to where we were
cd $PWD