ZFS Comes to OS X Courtesy of Apple's Former Chief ZFS Architect

[Detrius]

I've never seen any corruption on HFS.

Logical fallacy: argument from ignorance.

Pre-journaling, it happened every single time the system was shut down improperly. Journaling has helped a LOT, but it still doesn't *completely* mitigate the issue of corrupt directory structures. BUT journaling is implemented as a hack on top of the existing HFS+. It really is an after-thought, and it works pretty well for a hack.

 

[Rocketman]

This plus Thunderbolt plus SSD raid plus magnetic redundancy = superior goodness.

 

[iindigo]

After reading all that, the first thought I had was, "These are the conditions under which Apple stops bothering with third parties and just creates their own solution."

I fully expect Apple to debut a new filesystem that is on par with or is better than ZFS with 10.8 or 10.9.

 

[Peace]

I would NOT trust ZFS for mission critical data until it has been proven to be completely non data destructive .

I sincerely hope this guy has it right because ZFS is a very good file system .

Far superior to HFS+ or NTFS.

 

[JAT]

Well I can send you a drive with a serious meta data error if you'd like. And no the drive is not dead according to the SMART status ...

Me, too.

 

[blow45]

apple should be ashamed of themselves for being the ubber tech company they are now and releasing an os called lion on top of such a paleolithic fs as hfs+. Ios fancy crap doesn't cut it when the os is in such a dire need for a good fs. Shame on the developers of lion.

 

[milo]

I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.

I've seen it, and it's a horrible experience. It can happen on any disk, the question is whether or not the file system can detect it soon enough that the data can be repaired, and before a corrupted file is backed up overwriting a good one. With HFS+ the corruption won't be detected until you try and copy an unreadable file, or if a file is corrupted but still readable it won't even detect then, you'd have to actually find the corruption yourself.

 

[waloshin]

Why not EXT4?
Or Btrfs?

 

[odedia]

This plus Thunderbolt plus SSD raid plus magnetic redundancy = superior goodness.

If I remember correctly, ZFS doesn't play well with SSD's. It plays, but not optimized.

 

[bakerzdosen]

For those who know ZFS version 28 is the base line for usability.

Zpool has been updated as well.

Um, I'd say I "know ZFS" (I have probably 20 systems at various customer sites running perhaps 500 OLTP clients and a few heavily hit websites (newspapers) running off of ZFS. We switched them to ZFS because VxFS didn't offer us the speed we needed on files over 200GB, and ZFS is MUCH more usable than VxFS) and the highest zpool version we've used thus far is 22. So, that's good to know that v.22 is not usable.

Personally, this changes the game a bit for me. I was thinking of switching my home OpenSolaris box over to FreeNAS 8 because I'm sick of things (that are included in FreeNas) not compiling on Solaris, but I haven't . However, this changes the game for me. A Mac Mini with ZFS and a few Thunderbolt external drives suddenly solves all my needs.

 

[star-affinity]

Logical fallacy: argument from ignorance.

Pre-journaling, it happened every single time the system was shut down improperly. Journaling has helped a LOT, but it still doesn't *completely* mitigate the issue of corrupt directory structures. BUT journaling is implemented as a hack on top of the existing HFS+. It really is an after-thought, and it works pretty well for a hack.

Ditto.
Very true post.

There's a reason DiskWarrior is still good to have around. But it's true journaling helps a lot. Still, while it's quite seldom it has happened that I've had to run DiskWarrior to save a broken HFS+ filesystem, even with journaling enabled. I work at a place with about 150 Macs.

I would NOT trust ZFS for mission critical data until it has been proven to be completely non data destructive.

You mean this implementation of ZFS for Mac OS X?
ZFS has been around for some time and it has been and is running on mission critical data just fine.

 

[bakerzdosen]

If I remember correctly, ZFS doesn't play well with SSD's. It plays, but not optimized.

Something you need to understand about ZFS: It has two different kinds of cacheing, read and write (L2ARC and ZIL) that are typically housed on SSD's. The ZIL is the write cache. That's probably where this misconception comes from. The ZIL is getting hammered (assuming an active system) with every write that occurs to the zpool. The problem is that using an mlc-based SSD as a ZIL causes them to wear out and fail quite quickly. You need a (much more expensive) slc-based SSD to be used as a ZIL drive.

Having a zpool made up entirely of SSD's is not only possible, but it works quite well. It also basically eliminates the need for separate drives for the ZIL and L2ARC. Yes, you don't have TRIM support, but based on the copy-on-write nature of ZFS, that's probably a good thing.

With that said, ZFS does NOT play well with nearly full (say, 85% or higher) zpools. Performance begins to drop off significantly - regardless of whether you're using rotational magnetic media or solid-state. Lack of TRIM support would probably exacerbate that problem, but it's already a problem.

 

[KevinN206]

Given the intention to converge OS X and iOS to as great an extent as possible, I wonder what Apple's next move is? Microsoft is introducing WinFS in the next version of Windows Server.

HFS+ probably does need to be replaced for the reasons listed in the Ars Technica article. It was fine in the 1990s and early 2000s, but it is getting old.

WinFS is not a new file system. It was very much a relational database that sits on TOP of NTFS. It was abandoned on Windows because of performance issues. ReFS is Microsoft's new file system that retains the same API as NTFS, but they made it more robust, resilient to errors, error corrections, storage pooling, etc. However, it's also still in infancy so there're many features that were part of NTFS that will not be available on the next Windows Server 8. More details here: http://en.wikipedia.org/wiki/Windows_Server_8#ReFS

 

[blow45]

Ditto.
Very true post.

There's a reason DiskWarrior is still good to have around. But it's true journaling helps a lot. Still, while it's quite seldom it has happened that I've had to run DiskWarrior to save a broken HFS+ filesystem, even with journaling enabled. I work at a place with about 150 Macs.




You mean this implementation of ZFS for Mac OS X?
ZFS has been around for some time and it has been and is running on mission critical data just fine.

chalk one up for diskwarrior from me too, a life saver. Even for regular maintenance it's highly recommended.

 

[AidenShaw]

apples and pears

I sincerely hope this guy has it right because ZFS is a very good file system .

Far superior to HFS+ or NTFS.

This is a bit unfair to NTFS (and probably HFS+) as well.

ZFS combines the functions of a volume manager and a file system - so the better comparison is against "NTFS + VDS" on Windows. ZFS does have some features that "NTFS + VDS" does not have, but VDS does add a lot of useful features that NTFS itself does not support - and since Windows includes VDS in all systems it's reasonable to compare the combination. (VDS (Virtual Disk Service) is also known as FTDisk (Fault Tolerant Disk).)

For example, you can expand NTFS volumes (either by extending a partition, or spanning the volume across multiple partitions).

You can also move an NTFS volume to a larger disk without taking it offline or rebooting. (Add the new disk, create a RAID-1 mirror with the existing volume on the new disk, when the mirror resynch is complete break the old volume off the mirror, and then extend the volume to use more of the new disk.)

No argument that ZFS doesn't have some interesting additional features, but just making the point that in the end it's what you can do on the system - not what the filesystem by itself is capable of doing.

 

[blow45]

This is a bit unfair to NTFS (and probably HFS+) as well.

ZFS combines the functions of a volume manager and a file system - so the better comparison is against "NTFS + VDS" on Windows. ZFS does have some features that "NTFS + VDS" does not have, but VDS does add a lot of useful features that NTFS itself does not support - and since Windows includes VDS in all systems it's reasonable to compare the combination. (VDS (Virtual Disk Service) is also known as FTDisk (Fault Tolerant Disk).)

you mean in their server systems, right? How is fault tolerance provided by via VDS in a run of the mill windows pc? I am not sure I follow you...

 

[Clete2]

I already host my Time Machine and my family's CrashPlan shares on a Gentoo Linux ZFS RAID-Z system. It is fantastic, simply put. I would never use any other filesystem that currently exists to keep my data safe and stored in an efficient manner.

 

[deconstruct60]

Building a start up around a filesystem seems to have only one end game--to sell it back to an OS vendor. There isn't an IPO in this future.

Vertias existed for many years as an independent company and "sold out" to Symantec ( a non OS vendor). http://en.wikipedia.org/wiki/VERITAS_Software

There is more than one path. Shirt Pocket (makers of SuperDuper) didn't fold when Apple released Time Machine. Both do back-ups in a general sense, but they have differences in approach. If TensComplement can find a difference to exploit and keep the corporate costs lean they could probably get by for a long time.


ZFS is 'different' than probably any refinement Apple might come out with in HFS+ ( or HFS++ or whatever they call it). Microsoft's ReFS might be tactic that Apple could take (http://blogs.msdn.com/b/b8/archive/...-generation-file-system-for-windows-refs.aspx ) where the "external" API to the file system stays the same but the internal metadata is stored more robustly. Selling ZFS back wouldn't do that.

I think this somewhat depends upon what the "Platinum" product does. If it allows someone to install that package on top of OS X Server and get a "NAS box with easy management" then I think it will get some traction. The file system isn't the "end game". The "end game" is providing storage as a service to folks.

These intro products would be mainly for kicking the tires and learning.

 

[zephonic]

After reading title: "Oh awesome."
After reading rest of article: "Ah, nevermind."

this

 

[Rocketman]

A Mac Mini with ZFS and a few Thunderbolt external drives suddenly solves all my needs.

That and a PB or MP with SSD and the crappy HFS+ FS on it will provide whatever is needed going forward for many Mac users. ZFS is a real game changer if it is real.

4K video, surveillance files, and a library of stuff can consume some space. All that is needed now is an uncorruptable long term backup medium for that library. Any suggestions?

Rocketman

Agree with Aiden Shaw.

 

[AidenShaw]

you mean in their server systems, right? How is fault tolerance provided by via VDS in a run of the mill windows pc? I am not sure I follow you...

Windows client systems support RAID-1 through VDS/FTdisk.

Notice the mirrored "Work (W: )" volume on my home PC....

 

[calderone]

Looks like they didn't plan of the traffic...

 

[blow45]

why would you not use some hardware raid config for this Aiden? (if I understand your set up correctly... or is that hardware raid?) In any case I would think one can hardly compare a raid set up to what zfs can offer. And can't one just get some hardware raid to work on hfs+ anyway in a mac pro say without some equivalent of vds for hfs+

(disclaimer: I might be making a few serious errors of understanding along the way in my post due to my rudimentary understanding here)

 

[AidenShaw]

why would you not use some hardware raid config for this Aiden? (if I understand your set up correctly... or is that hardware raid?) In any case I would think one can hardly compare a raid set up to what zfs can offer. And can't one just get some hardware raid to work on hfs+ anyway in a mac pro say without some equivalent of vds for hfs+

(disclaimer: I might be making a few serious errors of understanding along the way in my post due to my rudimentary understanding here)

The biggest difference is that hardware raid is usually very inflexible compared to software fault tolerance (but it is often much, much faster for parity-based RAID (5,6,50,60,...)).

The hardware systems can also be combined with software FT - for example you could get RAID-60 by software striping across hardware RAID-6 volumes. You could use VDS as I described to move an NTFS volume from a smaller hardware RAID volume to a larger one.

On my home PC, the 6 TiB volume is hardware RAID-5 on a 3ware controller with 256 MiB battery-backed write cache (the only way to get good performance from parity-based RAID IMO). The other large volumes are VDS-based RAID-0 stripes.

 

[fmaxwell]

Sweet. too bad Apple didn't do this themselves. how stable is it tho?

It doesn't matter how stable it is if installing a new version of OS X could render it inaccessible. We're not talking about a scanner or color printer that stops working -- this is way more serious.