ZFS Comes to OS X Courtesy of Apple's Former Chief ZFS Architect

[calderone]

It doesn't matter how stable it is if installing a new version of OS X could render it inaccessible. We're not talking about a scanner or color printer that stops working -- this is way more serious.

This is why you resist the fanboy urge to upgrade right away and check into compatibility beforehand.

 

[blow45]

The biggest difference is that hardware raid is usually very inflexible compared to software fault tolerance (but it is often much, much faster for parity-based RAID (5,6,50,60,...)).

The hardware systems can also be combined with software FT - for example you could get RAID-60 by software striping across hardware RAID-6 volumes. You could use VDS as I described to move an NTFS volume from a smaller hardware RAID volume to a larger one.

On my home PC, the 6 TiB volume is hardware RAID-5 on a 3ware controller with 256 MiB battery-backed write cache (the only way to get good performance from parity-based RAID IMO). The other large volumes are VDS-based RAID-0 stripes.

cool, thanks!

 

[slackpacker]

the sites down ... me think Apple Lawyers have something to do with it.

 

[nutmac]

I assume 3rd party ZFS support is appropriate only for data volumes, which is still important, but diminishes my enthusiasm.

Problems with HFS+ is numerous, but my main beef is lack of data integrity check. For instance, if data on HDD or SSD becomes corrupted (which happens more often than you think), you will never know.

And lack of other modern features such as data deduplication and storage pool (where multiple disks, such as HDD and SSD can become one, similar to beyond RAID and with or without redundancy).

 

[loveturtle]

This is a bit unfair to NTFS (and probably HFS+) as well.
For example, you can expand NTFS volumes (either by extending a partition, or spanning the volume across multiple partitions).

You can also move an NTFS volume to a larger disk without taking it offline or rebooting. (Add the new disk, create a RAID-1 mirror with the existing volume on the new disk, when the mirror resynch is complete break the old volume off the mirror, and then extend the volume to use more of the new disk.)

You can do both of those things with ZFS. While you can't "expand a partition" you can expand the pool while live by a number of methods, and you really shouldn't be using partitions anyway. Whole disks for ZFS is the way to go..

As for moving the volume to a larger disk it is the same procedure you described with ZFS. Once you remove the smaller disk from the mirror (after they sync) the zpool will automatically grow to the size of the new larger disk.

----------

Um, I'd say I "know ZFS" (I have probably 20 systems at various customer sites running perhaps 500 OLTP clients and a few heavily hit websites (newspapers) running off of ZFS. We switched them to ZFS because VxFS didn't offer us the speed we needed on files over 200GB, and ZFS is MUCH more usable than VxFS) and the highest zpool version we've used thus far is 22. So, that's good to know that v.22 is not usable.

Personally, this changes the game a bit for me. I was thinking of switching my home OpenSolaris box over to FreeNAS 8 because I'm sick of things (that are included in FreeNas) not compiling on Solaris, but I haven't . However, this changes the game for me. A Mac Mini with ZFS and a few Thunderbolt external drives suddenly solves all my needs.

Go grab a copy of FreeBSD 9 and install it in your test environment of choice and give it a whirl. They've been working on it since 2007 and have done a wonderful job integrating it. I think you'll be pleasantly surprised coming from OpenSolaris...and the wealth of software available via the ports/pkg system does not hurt either.

 

[eyefoniac]

the sites down ... me think Apple Lawyers have something to do with it.

I wouldn't be surprised.

 

[AidenShaw]

You can do both of those things with ZFS. While you can't "expand a partition" you can expand the pool while live by a number of methods, and you really shouldn't be using partitions anyway. Whole disks for ZFS is the way to go..

As for moving the volume to a larger disk it is the same procedure you described with ZFS. Once you remove the smaller disk from the mirror (after they sync) the zpool will automatically grow to the size of the new larger disk.

Yes, I know this.

My point was simply that comparing ZFS to any particular filesystem is unfair.

ZFS is both a filesystem and a volume manager, so you should compare it to other filesystem/volume-manager combinations.

 

[marcusj0015]

Yeah, I loved Ars' Filesystem bit, they spelled it out and let you actually make sense of the ideas in that story. I learned a WHOLE lot about filesystems that day.

----------

It doesn't matter how stable it is if installing a new version of OS X could render it inaccessible. We're not talking about a scanner or color printer that stops working -- this is way more serious.

Yeah bro, I know how it works. you might wanna consider a custom OSX Installer disc if you're gonna rely heavily on ZFS, and won't have internet access.

 

[deconstruct60]

the sites down ... me think Apple Lawyers have something to do with it.

"slashdot" effect. (and also perhaps MacRumors). Looks like they exceeded their bandwidth quota. Not particularly surprising if monster amount of traffic suddenly show up at their site. I should have downloaded the manual earlier.

Besides .... Oracle lawyers would be about as likely as Apple's ones. LOL.

 

[iRCL]

Great, now a bunch of people who have no business running ZFS can do so more easily

 

[Can't Stop]

HFS+ is held together with duct tape and a prayer at this point. I hope Apple has something brewing, because I'm not sure how much longer they can keep slapping more stuff on top.

Pure nonsense.

 

[justperry]

Been using MacZFS for a while now, absolutely no issues. Read about ZFS in this forum, did a little research, haven't looked back. Took a little effort (not a lot of experience with terminal), but wasn't overly difficult. Ten's Compliment offers a nice gui, just got tired of waiting for a release.

No issues?
There's is a big bug in MacZFS, if you pull the cord from a USB external disk the whole system crashes:

http://code.google.com/p/maczfs/issues/detail?id=3

That IS a big issue.
I unmounted the disk, believing that I could pull the cord but was welcomed by a non responding OS.

Hope Ten's Complement does a better job.
I haven't read the whole topic yet so excuse Me if asked before, is it possible to put OS X on a ZFS disk?
Most probably not.

Edit: I should read whole topic first as I always do before I leave a comment, answers are there.

 

[sennekuyl]

Great, now a bunch of people who have no business running ZFS can do so more easily

Who pray tell should not be running a filesystem with end to end checksum, rather than the customary CRC checks? Or organic growth of storage?

Mirroring has a decent argument for it so for simplicity's sake mirroring should be done on many joe blow home networks. The silver edition works for them I believe.

@justperry: No, you can't boot OSX from zfs, though it maybe possible to have the root boot from hfs+ then have the users on zfs.

 

[Macman45]

For the avoidance of any doubt, ZFS is the daddy. IMHO.

This product seems to suck a little though as you would have hoped you can convert your boot drive to ZFS, but apparently not.

Agreed, 128 bit for larger video and music files would be good, nothing in these releases to make me want to buy.....Yet.

 

[justperry]

@justperry: No, you can't boot OSX from zfs, though it maybe possible to have the root boot from hfs+ then have the users on zfs.

Thanks Man, I already edited My post before I read yours, I should read first before leaving a comment

 

[Manderby]

I was hoping they introduce ZFS in Leopard (or was it Snow Leopard?) Server. They did not. And I think the reason is clear: Oracle. Same way that mysql is no longer part of the standard-package in Mac.

One can dream that ZFS would still find its way NATIVELY into MacOSX. But I think we all know that this is not going to happen. Would be nice. I trusted in the capabilities of Sun. Back then.

Althought now that the management has changed... who knows.

 

[Truffy]

I've never seen any corruption on HFS.

I have! Feel better?

 

[kocoman]

Do they have a 'timeframe' when booting from zfs will be supported?

 

[KnightWRX]

I have! Feel better?

I've seen corruption on ZFS.

I think the point is that there's nothing intrinsically wrong with HFS+'s design that causes corruption. Bugs exist in filesystem implementations, that's true of every filesystem out there, even the venerable VxFS from Symantec (Veritas) which is one of the superior ones hailing back the old days (and is standard on most HP-UX nowadays and still widely used on Solaris and other Unix systems).

Most people saying HFS+ is in dire need of a replacement don't quite understand what they are asking for. They are just repeating whatever they read on the Internet.

Anyway, when asked what they want from ZFS over HFS+, most people won't even be naming filesystem features that are lacking in HFS+, they'll be asking for a Volume Manager. That is sorely lacking on OS X. ZFS is more than just a filesystem, it's also a way to manage storage, agregate and abstract physical disks from mounted volumes and filesystems as to detach and make management of it completely transparent to the user (anyone who's worked with volume management knows the joys of online resizing, migration, POT snapshotting).

Give us LVM/HFS+ or ZFS or BTRFS, just give us something.

 

[Supermacguy]

I've had the same experience as others- folders full of images and I copied them to another drive. Only to find out that all the pictures were 0kb or corrupted! Thank God I could use DataRescue to recover it off an old HD that I hadn't erased yet. But ZFS is what I could have used then.

 

[firedownunder]

No issues?
There's is a big bug in MacZFS, if you pull the cord from a USB external disk the whole system crashes:

http://code.google.com/p/maczfs/issues/detail?id=3

That IS a big issue.
I unmounted the disk, believing that I could pull the cord but was welcomed by a non responding OS.

Hope Ten's Complement does a better job.
I haven't read the whole topic yet so excuse Me if asked before, is it possible to put OS X on a ZFS disk?
Most probably not.

Edit: I should read whole topic first as I always do before I leave a comment, answers are there.

If you read my signature, you'll see I have MacZFS running internally, all sata connections. Well aware of the usb issue. As I said, did a liittle research, no issues.

 

[KnightWRX]

I've had the same experience as others- folders full of images and I copied them to another drive. Only to find out that all the pictures were 0kb or corrupted!

And how do you know this is due to HFS+'s design ? How do you know it's not your copy tool that failed ? Or any other element in the chain from user space command to kernel back to storage system ?

That's why I really hate these conversations. People are so quick to blame HFS+ because they want to repeat the same stuff they heard here and join the "mob".

No objectivity, no investigation, no facts.

 

[haravikk]

What exactly can this paid implementation do that MacZFS can't? Doesn't ZFS already support block-level snapshots? So I expect the main difference is in GUI support, unless MacZFS only supports the basics?

Also, while MacZFS says it doesn't support ZFS encryption, would it support CoreStorage encryption? I've managed to set CoreStorage encryption on an AppleRAID volume, and what I understand of CoreStorage's encryption suggests that it doesn't actually matter what file-system is in-use.

Not that I'm that likely to upgrade to ZFS any time soon anyway, I do think that Apple's efforts toward CoreStorage do point to their own pooled storage solution, and I'm excited to see that happen. While the venerable HFS+ has served us well for decades, it's high-time for something new, especially when you realise how some of the features that HFS+ supports have had to be hacked into place (hard-links spring to mind).

 

[shaunp]

Wow

It's a shame Apple didn't to this themselves.

 

[stottmj]

Purpose of ZFS / ZEVO on Mac

1. It's only good for data volumes and booting from ZFS on Mac is not yet possible.
2. ZFS is extremely solid and reliable, it auto-corrects itself and verifies all read/writes are correct. Snapshots can be scheduled to allow you to rollback changes. ZRAID is software RAID but much much faster than any other software RAID system.
3. It will be much cheaper to license ZEVO from Ten's Compliment and setup a cheap SATA multiple drive bay using JBOD (Just a Bunch Of Disks) and connecting to a MacPro via eSATA addon PCI-X card. Each drive shows up individually and you add them to a ZFS ZPool and they become one drive.
4. ZFS will work on USB/Firewire/Thunderbolt external drives and if you get the Platinum / Developer version you can put them in a ZRAID.
5. Current ZEVO releases Silver/Gold do not support ZRAID, but they can mirror two disks and still have great reliability. Platinum / Developer releases are much more full featured.

It is best suited to those who have Terabytes of data to maintain. i.e. Photos, Videos, etc.

Not sure if ZEVO Developer Edition will support setting up a ZRAID pool of traditional drives and use an SSD as a high speed cache, I know this can be done with ZFS on Solaris.

Future features that the latest ZFS offers that ZEVO does not yet offer is ZFS diff which let's you calculate the differences between two different ZFS ZPools and xfer the bare minimum of data. This is good to move data from one ZFS SAN to another over a WAN, etc. It actually sends only enough data necessary to reconstruct it on the other side. Very slick feature but more of an enterprise feature.

Course, all this happens while hard disks are scarce and expensive due to the Thailand floods. Hopefully, the prices will start going back to normal when the ZEVO Platinum / Developer versions become available.

I am waiting for Platinum / Developer versions to ship and I will immediately buy it even if it costs $200+. I have a huge media library and I don't trust it to Drobo, ReadyNAS, etc. At least with ZFS I know I can trust the data integrity and that I can repair any problems that occur. With the other proprietary RAID systems from Drobo / ReadyNAS, I am not so sure...