Syncing files between two servers (mac mini)

Discussion in 'Mac OS X Server, Xserve, and Networking' started by unplugme71, Feb 19, 2014.

  1. unplugme71 macrumors 68030

    Joined:
    May 20, 2011
    Location:
    Earth
    #1
    I have two Mac Mini's with USB 3.0 RAID 10 storage.

    I work with files from one Mac Mini through my MacBook Air. After I save what I'm working on, I remote in, and run the rsync script that copies from server 1 to server 2.

    This is tedious and time consuming.

    I'm considering making rsync run every 15 minutes. My concern is, if rsync needs more time to copy before the next 15 min, will another rsync session start? And if it does start, will it try and copy the same file twice because rsync does not yet see the large file copied over completely yet?

    Also, I notice that even if I don't update some files, rsync will still copy them because their modified date changes or something.

    Just trying to figure out a way to keep the two servers in sync. Most of it is a one way sync (Server 1 -> Server 2).

    Server 3 backups Server 2.

    Otherwise, I might just run two copies of Windows Server 2012 and use DFS/Replication but I'm concerned about Mac files being on a windows share as I heard issues with file forks, meta data, etc.
     
  2. northerngit macrumors member

    Joined:
    Jul 16, 2007
    Location:
    England
    #2
    I'm unaware of native filesystem replication in OS X. Are you replicating for backup purposes, or to enable multiple users access to the files?

    Could you paste your rsync command? Certainly date changes shouldn't cause a replication event. Are you rsync'ing the full file on change, or just the deltas?

    There are many tools like ChronoSync and SyncMate that allow schedule to be set and agents run on local machines for seamless replication. ChronoSync is my personal favourite! Allowing bi-directional mirroring of content.

    Of course, this won't solve multiple user access issues and maintaining file locks.
     
  3. unplugme71 thread starter macrumors 68030

    Joined:
    May 20, 2011
    Location:
    Earth
    #3
    The replication i'm discussing is to maintain a spare 'live' file share. No one connects to the 2nd share unless the first one fails. I also have another drive that backs up every 4 hours (incremental) and then once a week (full).

    How can I sync the deltas? I think its a full file on change which is causing things to be slow. I'm sure its something with iTunes. I'm guessing the play count is stored within the movie or song? Could be why my replication is slow.

    If I can get just the changes to be copied over, that would be much quicker and save me a ton of time/resources!
     
  4. Alrescha macrumors 68020

    Joined:
    Jan 1, 2008
    #4
    You could create a script that never ends, sleeping for 15 minutes, running rsync, then going back to sleep.

    Alternatively, you could create a script that writes a lock file that subsequent invocations check for before they start rsync.

    A.
     
  5. jg900ss macrumors newbie

    jg900ss

    Joined:
    Nov 28, 2009
    Location:
    Europe, and Florida
    #5
    Syncing two servers, using SYNK

    I have three MacMini servers (the Core 2 Duo family) that are positioned in different countries. I used SYNK, a simple scripting product, to keep certain directories on the servers identical so that members could log into any one of the three servers and count on finding identical up-to-date information. The product was not expensive, and my Mac consultant was able to write scripts that kept those files/directories in sync at great distance, different time-zones, and securely using VPN. Something to explore? :)
     
  6. unplugme71 thread starter macrumors 68030

    Joined:
    May 20, 2011
    Location:
    Earth
    #6
    I might try your second option first, if that seems too difficult to perform I will go to the first.
     
  7. Alrescha, Feb 24, 2014
    Last edited: Feb 24, 2014

    Alrescha macrumors 68020

    Joined:
    Jan 1, 2008
    #7
    Code:
    # implement the lock file
    ls sync.lock > /dev/null 2> /dev/null
    if [ "$?" -eq "0" ]
    then
    exit
    fi
    #
    touch sync.lock
    #
    
    <do stuff>
    
    # remove the lock as we are done
    rm -f sync.lock
    A.
     
  8. mkelly macrumors regular

    Joined:
    Nov 29, 2007
    #8
    Bittorrent Sync

    I have a similar setup with two laptops and a Mac Mini server. I work primarily on one of the laptops but want the data available on both the secondary laptop and server.

    My first solution was some custom rsync scripts that would sync things around (and utilized a lock file to prevent more than one rsync from running at a time). It worked, but it wasn't pretty ;-)

    I've been experimenting with BitTorrent Sync for Mac, and I'm very happy with it. It seems to be keeping everything in sync nicely between the three machines, does so reasonably quickly, both on the LAN and remotely, and includes the option to keep a versioned history of changed files per directory (which you can enable/disable on a per-directory, per-machine basis). Also, it's free.

    http://www.bittorrent.com/sync
     
  9. unplugme71 thread starter macrumors 68030

    Joined:
    May 20, 2011
    Location:
    Earth
    #9
    Great! I'll give that a try.

    Thanks
     

Share This Page