rsync issues

Discussion in 'macOS' started by Spanky Deluxe, Apr 12, 2010.

  1. Spanky Deluxe macrumors 601

    Spanky Deluxe

    Joined:
    Mar 17, 2005
    Location:
    London, UK
    #1
    I've been using rsync to copy my itunes and iphoto files over to a mac mini connected to my TV in the lounge. This has been working absolutely fine for many many months (and still does). I recently got a second mac mini for use in a different room and am using exactly the same commands (obviously with a different IP address for when I mount the drive) but it refuses to work.

    My rsync command is:

    rsync -r -v "/Volumes/...../Music" "/Volumes/Mounted/.../userdir"

    One problem I'm getting is that if I don't have the added --exclude '.DS_Store' bit then rsync creates hundreds of thousands of files called ..DS_Store.000000 where 000000 is a six digit hexadecimal number. All zero KB big. If I exclude DS_Store then rsync gets gets stuck after a few files and just sits there doing nothing. The file can be just a few kb big.

    I've tried rsync with just a single file and it can copy it fine.

    All of my computers are connected via gigabit ethernet and are all running 10.6.3. The only difference between the mac minis as far as I can tell are that one is a 2009 C2D one (the one that works) and the other is an older CD one. I've already repaired disk permissions etc.
     
  2. v3nt macrumors newbie

    Joined:
    Apr 13, 2010
    #2
  3. Spanky Deluxe thread starter macrumors 601

    Spanky Deluxe

    Joined:
    Mar 17, 2005
    Location:
    London, UK
    #3
    I'm glad I'm not alone at least!

    I managed to get the files deleted but it took a while since it had created over 200,000 of them (I'd let it run for a while not realising it was broken). Finder struggled big time with the files and even using the terminal and the rm ..DS_Store.* command failed because the file list was too long. I had to create a script that looked like this:

    rm ..DS_Store.0*
    rm ..DS_Store.1*
    ...
    rm ..DS_Store.A*
    ...
    rm ..DS_Store.Z*


    That managed to get them deleted although the command still took about half and hour to get through them all.

    I just checked my first mac mini (the one that works) and I forgot to install the 10.6.3 update on it so that's definitely the root of the problem.

    Using rsync to copy files to a partition hosted on a machine running 10.6.3 simply doesn't work.
     
  4. azkennedy macrumors newbie

    Joined:
    Mar 30, 2010
    Location:
    The Desert...
    #4
    Any updates?

    Seen any more info on this issue? I'm fighting the exact same problem. :mad:
     
  5. viceroy macrumors newbie

    Joined:
    Jan 6, 2008
    #5
    I ran into the exact same problem as I was using rsync to sync my two macs, both running 10.6.3. If I rsync files from the mac I'm working on to the other mac, rsync bombs because it makes tens of thousands of copies of any hidden file starting with "."

    But I just found that if I copy files using rsync from the other mac to the mac I'm working on, it works fine. In other words,

    Code:
    rsync -avr  /Users/me/my_docs/ /Volumes/my_docs/
    bombs,
    but
    Code:
    rsync -avr /Volumes/my_docs/ /Users/me/my_docs/
    is okay.

    Both were fine before the 10.6.3 update. Definitely an update related issue. There's some discussion on the Apple support forum. Hopefully a future update will fix this.
     
  6. angelwatt Moderator emeritus

    angelwatt

    Joined:
    Aug 16, 2005
    Location:
    USA
    #6
    I use rsync a bit and haven't noticed this problem. I have rsync version 2.6.9, what version is your machine using (rsync --version)?
     
  7. viceroy macrumors newbie

    Joined:
    Jan 6, 2008
  8. viceroy macrumors newbie

    Joined:
    Jan 6, 2008
    #8
    To clarify, as the OP mentioned, it actually creates thousands of zero kb files named "..filename.xxxxxx" for any ".filename" and never really stops till the process is killed. You won't see this problem if you've excluded hidden files beginning with ".", or if such files haven't changed between source and target, or if you don't have such files. As far as I know the only files guaranteed to be present in every folder is .DS_store, so if you haven't excluded them in your rsync, I'm not sure why you are not seeing the same behavior.
     
  9. angelwatt Moderator emeritus

    angelwatt

    Joined:
    Aug 16, 2005
    Location:
    USA
    #9
    I backup hidden files as well. Here's the rsync command I use from a bash script. Maybe one of my arguments is helping my situation.

    Code:
    sudo rsync -aExzb --backup-dir=$bkup_dest --exclude-from=$excl --delete-after --log-file=$logfile $src -e "ssh -i $key" $host:$dest
    And here's my exclude file,
    Code:
    .Spotlight-V100
    /.fseventsd
    /.hotfiles.btree
    /.vol/*
    /Auth.bak
    /Cleanup At Startup
    /TheVolumeSettingsFolder
    .Trash
    .Trashes
    /TheFindByContentFolder
    /Shutdown Check
    Temporary Items
    /VM Storage
    /OpenFolderListDF
    /Desktop (Mac OS 9)
    /.com.apple.NetBootX
    /Volumes/*
    .VolumeIcon.icns
    /dev/*
    /automount
    /Network
    /private/var/vm/*
    /private/tmp/*
    /home
    /net
    ._*
    That's actually the same from the same user who created this thread. Though based on the last post in that thread maybe the 'z' argument for compression in my command above is enough to help.
     
  10. azkennedy macrumors newbie

    Joined:
    Mar 30, 2010
    Location:
    The Desert...
    #10
    Bug with AFP

    After digging around on the web for a while I saw someone mention that there was an issue with file sharing in 10.6.3 (wish I still had the link). Sure enough, I switched the target machines sharing type from AFP to SMB and the script runs fine now.
     
  11. daze macrumors 6502

    daze

    Joined:
    Mar 11, 2006
    Location:
    San Jose, California
    #11
    Here's how you can use the 'find' command to delete or perform actions from the shell when the file list is too big to process. Example for this particular case:

    Code:
    find . -name ".DS_Store*" -exec rm {} \;
    Warning: Do NOT run above command unless you really mean to delete the .DS_Store* files in the current directory!
     

Share This Page