ZFS Comes to OS X Courtesy of Apple's Former Chief ZFS Architect

[MacRumors]

Apple's flirtation with ZFS, the file system backed by Sun, has been well-documented with rumors and speculation dating back to 2006. The interest was fueled in mid-2007 by a claim by Sun CEO Jonathan Schwartz that ZFS would supplant HFS+ as the file system for Mac OS X Leopard. While ZFS did not materialize in Leopard, Apple clearly continued to work on the project, with claims of some support appearing in marketing materials for Mac OS X Server Snow Leopard.

Optimism surrounding that development was apparently premature, however, as all mentions of the file system later disappeared from Apple's site amid claims that licensing issues had led the company to scrap the project. By late 2009, Apple had shuttered even its open source ZFS project site, indicating that it had abandoned all work on the project.

In March of last year, Ars Technica posted a lengthy profile of Don Brady, the 20-year Apple engineer who had helped transition to the HFS+ file system for Mac OS X and also headed up Apple's internal ZFS team from 2006 until it was disbanded in 2009. As the report notes, ZFS offers a number of advantages over HFS+, including a 128-bit structure to address large file and volume sizes, as well as features for simplifying data management and increasing data integrity such as copy-on-write technology to preserve the file system structure should errors or failures occur while new data blocks are being written.

At one time, there was a lot of hope that Apple might transition from HFS+ to ZFS as a more modern replacement. In particular, ZFS's automatic snapshot feature was believed to be the perfect fit for Apple's Time Machine backup tool. When Snow Leopard was first announced in 2008, it was slated to have full read-write ZFS support, at least in the server version.

But when WWDC 2009 rolled around, all mention of ZFS support was scrubbed from Apple's website. A patent infringement lawsuit was still pending between NetApp and Sun, with NetApp claiming that it held patents on copy-on-write. Furthermore, Sun's CDDL open source license was also believed to make ZFS incompatible with Mac OS X, and that Apple couldn't reach suitable license terms with Sun.

With Apple shutting down its ZFS project in 2009, Brady soon left the company to form his own venture, Ten's Complement, with a plan to bring ZFS to the Mac. Brady and Ten's Complement have been working on the project since 2010, with an extensive beta program having been underway for quite some time, and it now appears that the company is ready to begin rolling out its ZFS products under the ZEVO name. First off the line is the Silver Edition, which is now available for $19.95 and brings some of the basic advantages of ZFS to Mac OS X.

Z E V O's Silver Edition brings you sophisticated ZFS storage technology in an easy-to-use solution.

It's ideal for those wanting modern, reliable storage that is simple to set up, validate, and manage. Our Z E V O Storage Setup Assistant makes setting up a single disk for Z E V O a snap. In a few simple steps you'll be up and running.

The company will soon be releasing its more advanced products, including the $39.95 Gold Edition with such features as Time Machine-like rotating data snapshots and data redundancy. A forthcoming Platinum Edition carrying additional features such as RAIDZ support, data deduplication, and an advanced management utility is set to launch this spring, with pricing yet to be announced. Finally, the company is also working on a Developer Edition that will offer a combination of GUI and command line interface tools for complete system control.

Ten's Complement is not the only group to still be working on bringing ZFS to Mac OS X, as the MacZFS open source project continues to build on the foundation laid by Apple and Sun. But Ten's Complement seems to be the first to bring ZFS to the Mac in a commercial package to facilitate implementation and maintenance of the file system for users looking at options for securing the integrity of their data.

Article Link: ZFS Comes to OS X Courtesy of Apple's Former Chief ZFS Architect

[marcusj0015]

Sweet. too bad Apple didn't do this themselves. how stable is it tho?

[jayhawk11]

HFS+ is held together with duct tape and a prayer at this point. I hope Apple has something brewing, because I'm not sure how much longer they can keep slapping more stuff on top.

[firedownunder]

Been using MacZFS for a while now, absolutely no issues. Read about ZFS in this forum, did a little research, haven't looked back. Took a little effort (not a lot of experience with terminal), but wasn't overly difficult. Ten's Compliment offers a nice gui, just got tired of waiting for a release.

[Porshuh944turbo]

Originally Posted by jayhawk11

HFS+ is held together with duct tape and a prayer at this point. I hope Apple has something brewing, because I'm not sure how much longer they can keep slapping more stuff on top.

What exactly is falling apart with HFS+? I use ZFS on a media server with raidz1 across 6 disks. It's great for that sort of thing, but for the average user with 1 or 2 HFS+ disks in an iMac, what are they missing?

[toxic]

Originally Posted by Porshuh944turbo

What exactly is falling apart with HFS+? I use ZFS on a media server with raidz1 across 6 disks. It's great for that sort of thing, but for the average user with 1 or 2 HFS+ disks in an iMac, what are they missing?

it corrupts easily, for one.

for anyone who hasn't read this: http://arstechnica.com/apple/reviews...12#file-system

[chrono1081]

Originally Posted by toxic

it corrupts easily, for one.

for anyone who hasn't read this: http://arstechnica.com/apple/reviews...12#file-system

I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.

[DHagan4755]

Why couldn't Apple with its near $100 billion in the bank just outright buy ZFS if it's that good? I mean HFS+ can't be all that bad....

[mrfrosty]

For the avoidance of any doubt, ZFS is the daddy. IMHO.

This product seems to suck a little though as you would have hoped you can convert your boot drive to ZFS, but apparently not.

[iEdd]

After reading title: "Oh awesome."
After reading rest of article: "Ah, nevermind."

[KPOM]

Given the intention to converge OS X and iOS to as great an extent as possible, I wonder what Apple's next move is? Microsoft is introducing WinFS in the next version of Windows Server.

HFS+ probably does need to be replaced for the reasons listed in the Ars Technica article. It was fine in the 1990s and early 2000s, but it is getting old.

[Kilamite]

Love that he continued to work on it.

However, given the "silver, gold and platinum" rubbish, maybe they should have made this for Windows. One version for all, even if the average user wouldn't use those features.

I appreciate the amount of effort and time, but fragmented releases like that suck. Especially if you had a "silver" edition and suddenly wish you had the "platinum" edition, you'd probably have to reformat.

[Peace]

I remember those days of testing ZFS in OSX. We couldn't seem to fix the problem where an attached USB drive formatted as ZFS would go to sleep and BAM ! kernel panic. Back then ZFS needed to be constantly connected (R/W) to any partition that was formatted ZFS. This is why the team at Apple gave up on it.

Since my NDA with that group no longer exists I can say that now


It looks like he's fixed the problem.

[OriginalMacRat]

Originally Posted by chrono1081

I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.

Wait! They were correct once?

[miknos]

What's the difference for the common consumer?

[blackburn]

We need a decent file system. I've experienced corruption with hfs+ but nothing as bad as with fat32 or ext2.

[WickedStealthy]

I'm certainly going to try it. ZFS has at least the advantage that with thunderbolt you could use some SSD & HHD in mixed mode to have storage and performance !
I hope they continue and get the stuff working so that Mac OSX can be booted from ZFS ... That would be nice.
Certainly for systems like the iMac with SSD + Big HD this could prove advantageous instead of ****ing around with separating user data from system data, which looks again something from 20y back ...

[cg0def]

Nice to see you guys covering Zevo. Ten's Complements has a great solutino but there's one issue that I really hope they've found a solution for. Time Machine drives apart from being a backup are also used as disaster recovery. If you store your backup on an FS that the OS does not by default know how to read, you will have a hell of a time restoring from it in the unlikely event of a disaster recovery. You would need some way to slipsteam the ZFS driver into the Lion recovery partition and then every time an OS update wipes out the hack you'd have to recreate it. The only other option would be the creation of a boot disk of some sort for the use of disaster recovery. I'd imagine that the first case breaks Apple's EULA and I am really hoping that Ten's Compliments has been working on creating a book disk of some sort. To be honest, for most OS X users, any backup other than a disaster recovery protection is a waste of time money and space.

Another problem is the fact that ZFS has absolutely no driver to Windows which is not quite the case with HFS but then again writing a Windows drivers is a bit out of the scope of Ten's Compliments.

Oh and as to why ZFS is something you need to care about here is a why:
HFS+ had gotten incrementally better over the years and I'd even go so far as to suggest that it's a marginally better FS than NTFS. However, the age of the FS is being hidden by the OS and in fact there are tons of hidden meta files that you never get to see. The problem with HFS+ and pretty much everything else out there is that they don't check for bit level file consistency. HFS+ however depends on the assumption that there are no bit level errors in your meta data in order to let you access your files. And when errors occur you are under a very very real danger of loosing your files. Apple would have you believe that it is impossible to ever loose files and then they'll go back and tie their pants by saying that you should always backup your files. Well here's what I know from personal experience. I had a set of data that I've been moving between OS upgrades and computer changes and hardware upgrades for the better part of 7 years. Yes, I do clean my preference files whenever it gets really messy. One day, after some sort of a combination of a hardware failure followed by a failed attempt of a software to save changes to a file, my whole documents folder literally disappeared. It was still there in terms of bits but HFS+ could not read it or repair the drive. Had that been ZFS, the error would have been caught long before occurring an ideally I would have known that the drive was failing with my data being written to the non failing parts of the disk (it was not a mechanical failure).
Anyway I lost some files and restored most but the point is even the best efforts of Apple can't quite cover up the fact that the FS design is a bit creaky 20+ years after it's creation.

As to Apple picking up ZFS once again, all is not at all lost. If I may point out, nowadays, Oracle does not have a single person from the Sun ZFS team working for them anymore. Stupid management decisions made it unacceptable for all the guys to stay there. ZFS improvements however, are still being contributed by those same guys. In fact the open source version of ZFS is currently the one that is ahead in terms of features and development efforts. There is still the issue of licence compatibility which I am not so sure if it really is an issue but my point is that if Apple ever chooses to reevaluate ZFS' suitability for OS X there is a very good chance of things turning out much different than they did last time.

Oh and a while back ArsTechnica did a very good piece of HFS+ and why a replacement is needed. Look it up for more details if you wish. It is factually correct.

[mrfrosty]

Originally Posted by Peace

I remember those days of testing ZFS in OSX. We couldn't seem to fix the problem where an attached USB drive formatted as ZFS would go to sleep and BAM ! kernel panic. Back then ZFS needed to be constantly connected (R/W) to any partition that was formatted ZFS. This is why the team at Apple gave up on it.

Since my NDA with that group no longer exists I can say that now


It looks like he's fixed the problem.

If a team of Apple Developers couldn't fix a repeatable problem like that, it's probably best they shelved it and let someone else handle it. Yes, i know this guy was head this area or something, that probably meant he just held a clipboard.

[Cougarcat]

Originally Posted by chrono1081

I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.

It's not just Ars, it's John Siracusa. He's written 20+ page OS X reviews since the beta. He knows his stuff.

Originally Posted by jayhawk11

HFS+ is held together with duct tape and a prayer at this point. I hope Apple has something brewing, because I'm not sure how much longer they can keep slapping more stuff on top.

If you've read Siracusa's review, he suggests that Apple's work on Core Storage (the biggest change in the OS X FS to date) might be an indication of the direction they're going. A few years ago they were hiring filesystem engineers, so they're definitely brewing something.

[cg0def]

Originally Posted by chrono1081

I've never seen any corruption on HFS. Also Arstechnica.com, while popular and full of good information, isn't 100% correct all the time.

Well I can send you a drive with a serious meta data error if you'd like. And no the drive is not dead according to the SMART status ...
I'm keeping it as a souvenir and just because YOU haven't seen something doesn't mean it does not exist. It is very rear though

[Casey4147]

...until I hit the "booting not supported in version 1.0" disclaimer.

[cg0def]

Originally Posted by mrfrosty

If a team of Apple Developers couldn't fix a repeatable problem like that, it's probably best they shelved it and let someone else handle it. Yes, i know this guy was head this area or something, that probably meant he just held a clipboard.

Actually Apple has a very different structure from what you thing (based on the comment). HE actually is a brilliant FS programmer and also he didn't fix anything. It was the ZFS community that has been adding features and fixing stuff in ZFS. What Ten's Complements did is port the OpenIndiana (i.e. Open Solaris) branch of the ZFS code. And even that one is about an year behind the latest stable releases but it gives them time to test and fix things since it is actually a very major component.

[mdriftmeyer]

With FreeBSD 9 latest Pre-release:

Source: http://www.freebsd.org/releases/9.0R/relnotes.html

The FreeBSD ZFS subsystem has been updated to the SPA (Storage Pool Allocator, also known as zpool) version 28. It now supports data deduplication, triple parity RAIDZ (raidz3), snapshot holds, log device removal, zfs diff, zpool split, zpool import -F, and read-only zpool import.[r219089]

For those who know ZFS version 28 is the base line for usability.

Zpool has been updated as well.

If Apple resumes ZFS support it will be able to pull in the work on the FreeBSD 9.x and 10.x branches to suit their needs.

Paying for this add-on only makes sense for older copies of OS X.

Building a start up around a filesystem seems to have only one end game--to sell it back to an OS vendor. There isn't an IPO in this future.

[AidenShaw]

Originally Posted by blackburn

We need a decent file system. I've experienced corruption with hfs+ but nothing as bad as with fat32 or ext2.

So, it's not as bad as two ancient filesystems that have both been replaced by more modern ones.