NTFS -> HFS+ -> ZFS -- Migration Path for 6-7TB of Data?

[crazydiamonds]

Hi All,

This is my situation, I've been wanting to switch from windows desktop PC to MP, I already have MBP for all my productivity needs.

See here: https://forums.macrumors.com/threads/508567/

Even though current windows PCs are good enough for what I need, it seems OS X will have a brighter future with Snow leopard and my 3yo Amd X2 machine is slowing me down.

In order to make the transition I will need to transfer all my 6-7TB from NTFS to HFS+ through a buffer 1TB HD, since it seems there is no way to just convert the file system.

My question is, do you think when Snow leopard is released with ZFS it will be possible to convert HFS+ to ZFS without formatting? Moving terabytes worth of data won't be something fun to do, let alone twice.

I appreciate you input.

Ammar

 

[applefan69]

im not experienced on this so im gonna say everything i know right now and if im wrong then disregard my post (Cause its most likely wrong as well)

NTFS is windows file system format
HFS is OS X file system format
ZFS is a superior format to HFS, and is rumored snow leopard will adopt it

am I right?

If so then that means right now ALL OS X users are in HFS format, and when snow loepard comes out they will all need to convert to ZFS (most likely during the installation process)

im pretty sure apple wont force millions of users to re-format their drive just to upgrade to snow leopard, that would not only give bad press... but frusterate lots of customers. So im sure you'll be able to easily convert from HFS to ZFS.

Would someone mind telling me what is different in these formats and why one is superior tho?

 

[KelchM]

im not experienced on this so im gonna say everything i know right now and if im wrong then disregard my post (Cause its most likely wrong as well)

NTFS is windows file system format
HFS is OS X file system format
ZFS is a superior format to HFS, and is rumored snow leopard will adopt it

am I right?

If so then that means right now ALL OS X users are in HFS format, and when snow loepard comes out they will all need to convert to ZFS (most likely during the installation process)

im pretty sure apple wont force millions of users to re-format their drive just to upgrade to snow leopard, that would not only give bad press... but frusterate lots of customers. So im sure you'll be able to easily convert from HFS to ZFS.

Would someone mind telling me what is different in these formats and why one is superior tho?

ZFS is NOT ready to be a primary file system. Really, afaik it was never really intended to be used on boot drives either. It still has a relatively high overhead in processor usage for some events. Basically, ZFS is going to be useful in a server environment and with large arrays... Your not going to want to be using it on the drive your OS and applications are installed on.

That said, the likelihood of being able to convert from HFS+ to ZFS is going to be about nil.

That said, ZFS needs to happen as HFS+ is getting rather long in the tooth.

 

[bstreiff]

Typically, no, changing filesystems tends to require reformatting and starting over; this isn't to say that an in-place conversion isn't possible (Microsoft has a tool that converts FAT32 to NTFS, for instance), but I wouldn't expect it.

HFS+ isn't going to disappear overnight; ZFS will probably be made the default filesystem for new installations of Snow Leopard, while upgrades will continue to use the existing HFS+ filesystem.

Also, if you have 6-7TB of stuff, perhaps you should consider a dedicated fileserver-- I have one running Linux with ext3 filesystems in LVM, serving out files to Windows, Linux, and OS X clients. With this approach, it doesn't even matter what filesystem the clients support, as long as they understand SMB or NFS for file-sharing.

 

[crazydiamonds]

Thank you for replying,

applefan69: You are right about which FS belongs to which OS, HFS+ and NTFS are good filesystems, each has it's own cons and pros while ZFS is superior to both, I am most intrested in it's data intergrity, it's safer aginst data corruptions and the way it handles large files.

check this wiki: http://en.wikipedia.org/wiki/Comparison_of_file_systems


KelchM: I thought so too, ZFS is not a descendant from HFS+, not like Fat32 to NTFS.


bstreiff: A file server sounds like a good idea, how is the transfer speed?

I have 3tb+ of data that'll go inside MP and another 3tb+ for backup in an external multiple port e-sata enclosure. Do you think I can replace all of it with a file server?

Thank you

 

[bstreiff]

bstreiff: A file server sounds like a good idea, how is the transfer speed?

I have 3tb+ of data that'll go inside MP and another 3tb+ for backup in an external multiple port e-sata enclosure. Do you think I can replace all of it with a file server?

Just did a quick test with my setup (gigabit): Using scp (so, with encryption overhead) I was able to do 5 MB/s. My fileserver isn't anything special, just a JBOD (so no RAID or anything; I rsync home directories to another machine nightly, but that's it), but it's very easy to add disks to-- currently, I have a 400GB drive and three 500GBs.

It's not the 'easiest' approach; I'm very comfortable with Linux system administration, so this works best for me. I don't know what sorts of prepackaged systems are available or how much they would cost.

 

[KelchM]

A file server sounds like a good idea, how is the transfer speed?

I have 3tb+ of data that'll go inside MP and another 3tb+ for backup in an external multiple port e-sata enclosure. Do you think I can replace all of it with a file server?

The file server is the route I would go. With that much data it just makes more sense. Transfer speeds really shouldn't be a problem if you are using gigabit LAN.

 

[nanofrog]

From the info I've come across, ZFS will only be available in Snow Leopard Server. (Likely to become available to all in later versions).

If you do decide to go the file server route, it would be an option. For $500 increase, of course.

 

[mkrishnan]

If you do decide to go the file server route, it would be an option. For $500 increase, of course.

Ext4 will probably (?) hit final status before ZFS drops in OS X, so that might be another good option....

 

[nanofrog]

mkrishnan:

Ext4 earlier?

When reading the OP, I got the impression he was interested in OS X solutions.

If willing, Linux could offer a nice solution. Cheaper too!

 

[KelchM]

From the info I've come across, ZFS will only be available in Snow Leopard Server. (Likely to become available to all in later versions).

I outlined why this is in my previous post. ZFS is not intended for use in boot drives. That, and changing a file system brings with it many complications that might not apparent. Starting by introducing it at the server level makes a lot of sense in this situation.

 

[RedTomato]

ZFS is new and untried in the mass market. If you depend on your 6tb to earn a living, I would stick to what everyone else uses.

VMWare Fusion is able to run a lovely virtual XP pro or Vista in a window in your OSX desktop. The Windows Explorer in the virtual machine can also connect to external drives, and from there, it's just a simple drag and drop of the particular file you need into OSX. It's a little bit slow so I would only use it for particular files, not for tranferring 10 gig at a time.

For direct access to NTFS external drives from OSX, Paragon ($40) has a good reputation.

http://www.paragon-software.com/home/ntfs-mac/

This might be the best solution for you and smooth the transition.

There's also http://macntfs-3g.blogspot.com/ which is free but more work and not as fast as Paragon.

 

[Draeconis]

Hi All,

This is my situation, I've been wanting to switch from windows desktop PC to MP, I already have MBP for all my productivity needs.

See here: https://forums.macrumors.com/threads/508567/

Even though current windows PCs are good enough for what I need, it seems OS X will have a brighter future with Snow leopard and my 3yo Amd X2 machine is slowing me down.

In order to make the transition I will need to transfer all my 6-7TB from NTFS to HFS+ through a buffer 1TB HD, since it seems there is no way to just convert the file system.

My question is, do you think when Snow leopard is released with ZFS it will be possible to convert HFS+ to ZFS without formatting? Moving terabytes worth of data won't be something fun to do, let alone twice.

I appreciate you input.

Ammar

Install MacFUSE and NTFS-3G. Totally free and enables OS X to read/write to NTFS partitions so, if you decide to use Boot Camp or another machine running Windows you can still access your data. Transferring 6-7TB over to HFS+ or ZFS is all well and good if you're 100% sure you'll be only using OS X, but I've found it useful to have a format that can be read/written to by both operating systems.

 

[nanofrog]

I outlined why this is in my previous post. ZFS is not intended for use in boot drives. That, and changing a file system brings with it many complications that might not apparent. Starting by introducing it at the server level makes a lot of sense in this situation.

We seem to be on the same page.

I just wanted to mention the ZFS is effectively moot in the near future, unless the Server Edition were used. So I neglected any further details. Which you already had mentioned.

 

[crazydiamonds]

Install MacFUSE and NTFS-3G. Totally free and enables OS X to read/write to NTFS partitions so, if you decide to use Boot Camp or another machine running Windows you can still access your data. Transferring 6-7TB over to HFS+ or ZFS is all well and good if you're 100% sure you'll be only using OS X, but I've found it useful to have a format that can be read/written to by both operating systems.

Thank you for the links, I will look into these options. I just wish there was a good universal file system that plays nice on all OS', unfortonatly FAT32 isn't the answer.

 

[crazydiamonds]

ZFS is new and untried in the mass market. If you depend on your 6tb to earn a living, I would stick to what everyone else uses.

VMWare Fusion is able to run a lovely virtual XP pro or Vista in a window in your OSX desktop. The Windows Explorer in the virtual machine can also connect to external drives, and from there, it's just a simple drag and drop of the particular file you need into OSX. It's a little bit slow so I would only use it for particular files, not for tranferring 10 gig at a time.

For direct access to NTFS external drives from OSX, Paragon ($40) has a good reputation.

http://www.paragon-software.com/home/ntfs-mac/

This might be the best solution for you and smooth the transition.

There's also http://macntfs-3g.blogspot.com/ which is free but more work and not as fast as Paragon.

I wonder how reliable these programs are at dealing with NTFS drives. In the long run, it seems it's better to keep my NTFS partitions and not convert them to another file system, after all, NTFS is more universal than HFS+.

I will always have the boot drive with the native file system.

 

[crazydiamonds]

Thank you all for your very informative input. This is where I stand now.

1. Keep the NTFS drives intact and use one of the NTFS R/W programs.

2. Convert to HFS+ and forget about ZFS for now until it's tried and tested.

3. Forget about NTFS, HFS and ZFS and resort to a file server.

...

I'm leaning more towards the first two options for a smoother transition like RedTomato suggested, I will do a research to see how reliable these NTFS utilities are and whether or not they work with ChronoSyns or any other syncing programs with no problems. That way I can at least read my data from virtually any OS and I won't have to go through the hassle of moving all the data.

If problems arise from using NTFS on OS X, I will probably have to convert to HFS, I will buy new drives to copy the data to and store away the old drives as an off-site backup.

Moving everything to a file server is an excellent idea for using the data from multiple computers, even with different operating systems. Unfortunately this is an uncharted territory to me, I will have to do some research to see how it works and whether it's a viable long-term solution for my storage needs.

I guess we'll have to wait and see how ZFS looks in its final release, it definitely sounds like a great file system on paper.

Ammar

 

[Draeconis]

Well I can tell you from my experience having MacFUSE and NTFS-3G has been really useful. I have a load of files that are still only really relevant to Windows I want to keep, and converting the file system to HFS+ or FAT32 would be probably a bit better than leaving it as NTFS, but then Windows can't see the partition. It's worked flawlessly for me so far, which I'm happy about as I have close to 500Gb of audio files and folders I need access to a lot, so it'd been the perfect solution.