ZFS (and Snow Leopard) to Speed up Solid State Drive Performance?

[MacRumors]

http://www.macrumors.com/images/macrumorsthreadlogo.gif (http://www.macrumors.com)

Infoworld reports (http://www.infoworld.com/article/08/08/07/Samsung_Microsoft_in_talks_to_speed_up_SSDs_on_Vista_1.html?source=rss&url=http://www.infoworld.com/article/08/08/07/Samsung_Microsoft_in_talks_to_speed_up_SSDs_on_Vista_1.html) (via MacsimumNews (http://www.macsimumnews.com/index.php/archive/mac_users_should_watch_zfs_development_closely/)) that Samsung has been working with developers to boost solid state drive (SSD) performance in operating systems. Samsung announced Wednesday that it has been in talks with Microsoft to boost performance in Windows:The speed and way in which SSDs fetch and cache data are different than hard drives, said Michael Yang, flash marketing manager at Sun. Samsung hopes to work with Microsoft to boost SSD performance on Windows by discovering optimal packet sizes for data transfers and the best ways to read and write files, for example.Of interest to Mac users is that Sun has already been working with Samsung to improve SSD support in their ZFS file system. Sun is adding capabilities to boost the durability and performance of SSDs on ZFS-based operating systems. For example, Sun may add defragmentation capabilities for SSDs, which organizes data in a particular order to enable quicker data access.Apple has announced that ZFS read/write support will be in Mac OS X 10.6 (Snow Leopard) server (http://www.apple.com/server/macosx/snowleopard/), although there has been no official word on the consumer version.

Solid State Drives are a new technology that promise faster disk drive performance but are presently at premium prices. Prices, of course, are dropping quickly. Apple recently dropped (http://www.macrumors.com/2008/07/03/1-8ghz-ssd-macbook-air-drops-500/) the price of the MacBook Air 64GB SSD upgrade from $999 to $599. While there were some controversial claims (http://www.macrumors.com/2008/07/02/solid-state-drives-ssd-reduce-battery-life/) from Tom's Hardware that SSDs actually reduced notebook battery life, a followup report (http://www.tomshardware.com/reviews/ssd-hard-drive,1968.html) indicates that this is not necessarily the case.



Article Link (http://www.macrumors.com/2008/08/08/zfs-and-snow-leopard-to-speed-up-solid-state-drive-performance/)

[Bruizer]

ZFS will be a step in the right direction IMO for sloving some of my personal storage systems issues. Don't know if SDD will be part of that but...

You never know.

[11800506]

Hopefully ZFS will make its way into the regular version of Snow Leopard.

[CWallace]

ZFS' "storage pool" concept seems to me to nicely complement Apple's digital lifestyle goals. As people add movies, pictures and music to their collections, the ability to treat it as one virtual pool regardless of where the files are actually located (different Macs or PCs on the home network, HDD enclosures, file servers, etc.) would make it easy to access that entire collection through a single portal (be it Finder, iTunes or Front Row).

While silicon still remains much more expensive then iron, and likely will remain so for many more years, SSDs still have advantages in many applications and if ZFS can improve their performance, so much the better.

[exscape]

ZFS' "storage pool" concept seems to me to nicely complement Apple's digital lifestyle goals. As people add movies, pictures and music to their collections, the ability to treat it as one virtual pool regardless of where the files are actually located (different Macs or PCs on the home network, HDD enclosures, file servers, etc.) would make it easy to access that entire collection through a single portal (be it Finder, iTunes or Front Row).

While silicon still remains much more expensive then iron, and likely will remain so for many more years, SSDs still have advantages in many applications and if ZFS can improve their performance, so much the better.
Unless Apple has made some *major* changes to ZFS, it cannot utilize other computers/file servers or such, only local disks can be used in a pool. I'm almost completely positive this is the case in Solaris (which I've used for a while).

[lostngone]

I know it would hurt speed but wouldn't you want fragmentation so you could spread out the writes over all the memory space so you wouldn't wear out the SSD as fast?

[jmadlena]

I am looking forward to HFS being phased out. ZFS appears to be the way forward, and I'm excited to see how it improves Snow Leopard performance!

[pyrodex]

Unless Apple has made some *major* changes to ZFS, it cannot utilize other computers/file servers or such, only local disks can be used in a pool. I'm almost completely positive this is the case in Solaris (which I've used for a while).

That is correct however it can use SAN disks which seems to be the case on many high end systems using ZFS. We use it today with multiple solaris boxes and 50TB of ZFS pools across them all.

[Apple Ink]

The real question is...... do we expect more SSD based devices in Apple's product mix?
The famous ultraportable Mac/iTablet any one?

[CWallace]

Unless Apple has made some *major* changes to ZFS, it cannot utilize other computers/file servers or such, only local disks can be used in a pool. I'm almost completely positive this is the case in Solaris (which I've used for a while).

I imagine Sun isn't going to stand still on extending ZFS' capabilities. And if they decide to stop, Apple might not.

[Wotan31]

Who gives a crap? SSD's suck at random I/O, and all but the best top dollar ones have only so-so throughput performance - they are a long ways out from being mainstream.

Give us some real news - Where is my Montevina MBP dammit!!!

[loveturtle]

it should be kept in mind that while no offical word has come noel from the zfs@apple mailing list said that as of now they have no plans to cripple zfs on the client but it will probably not have any gui tools. So only advanced users will be able to take advantage of it. Of course that is all subject to change but it is the closest thing we have to an offical comment

[rspeed]

It's worth pointing out that in Snow Leopard, the OS almost certainly won't be able to reside in a ZFS pool. Even Solaris, the OS for which the filesystem was originally developed, cannot boot off of ZFS.

[wrldwzrd89]

It's worth pointing out that in Snow Leopard, the OS almost certainly won't be able to reside in a ZFS pool. Even Solaris, the OS for which the filesystem was originally developed, cannot boot off of ZFS.
This was true a while ago. I remember several reports (http://www.google.com/search?q=zfs+boot&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) of ZFS boot support in Solaris on X86 being successfully implemented back on March 2007. Assuming this is true, then making Mac OS X capable of ZFS boot shouldn't be that much harder.

[poundsmack]

I am hoping snow lepord bring full ZFS across the board, both as the primary file system and as the boot file system. HFS+ needs ot be phased out. as soon as ZFS implimentation like that is in i will upgrade. Snow lepord really sounds like its going ot be amazing, presure is on apple :)

[wrldwzrd89]

I am hoping snow lepord bring full ZFS across the board, both as the primary file system and as the boot file system. HFS+ needs ot be phased out. as soon as ZFS implimentation like that is in i will upgrade. Snow lepord really sounds like its going ot be amazing, presure is on apple :)
You know what...

This was true a while ago. I remember several reports (http://www.google.com/search?q=zfs+boot&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) of ZFS boot support in Solaris on X86 being successfully implemented back on March 2007. Assuming this is true, then making Mac OS X capable of ZFS boot shouldn't be that much harder.
If what you said is true, and what I said here is also true... then the debate over whether or not Snow Leopard will be Intel-only or not is settled. Why? Sun and Apple won't bother to make ZFS boot support for PowerPC... and ZFS will be the native file system in Snow Leopard. :D

[bigwig]

I wonder what's changed to make ZFS work (other than the obvious). Even though OSX has case-sensitive HFS support in place, it doesn't actually work if you try to use it. OSX bombs horribly on case-sensitive HFS boot partitions, and case-sensitive HFS partitions from DMGs also have weird failure modes. For example, if you try to use case-sensitive HFS+ with FileVault you can't log in (password works fine, but hilarity ensues).

[Santa Rosa]

Who gives a crap? SSD's suck at random I/O, and all but the best top dollar ones have only so-so throughput performance - they are a long ways out from being mainstream.

Give us some real news - Where is my Montevina MBP dammit!!!

This is far more interesting than that.

[MShock]

This was true a while ago. I remember several reports (http://www.google.com/search?q=zfs+boot&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) of ZFS boot support in Solaris on X86 being successfully implemented back on March 2007. Assuming this is true, then making Mac OS X capable of ZFS boot shouldn't be that much harder.

The boot ability has been around for about a year in the OpenSolaris code base. ZFS is in fact the default filesystem for OpenSolaris 2008.05 and onwards. The problem is that boot support is very OS specific;therefore, making it a difficult feature to implement. Given some time, I feel ZFS will mature for 10.6 to be the default boot system on the client, as that is a Forge stated goal, however, I don't think the GUI tools will be around until Server update like 10.6.2 and Client update 10.6.8, similar to what Apple did with HFS+ Journaling in 10.2.

That having been said, THIS IS SWEET! One could compress the parts of the OS that they don't use, like the library, and get much faster I/O performance. Plus Time Machine and File Vault will be a lot smaller and efficient. If this works, I guarantee Apple will have some amazing implementation for networks in schools and business that will make Macs very secure and usable.

[lostngone]

Well ZFS looks really cool but companies like Adobe still don't fully support HFS+, how long before they get around to supporting ZFS, Fall 2030?

[SC68Cal]

Well ZFS looks really cool but companies like Adobe still don't fully support HFS+, how long before they get around to supporting ZFS, Fall 2030?

Nobody likes to support HFS+ anyway. I doubt that ZFS will ever be the filesystem of choice for OS X server or desktop. It'll probably be "supported" like UFS was.

[jackfrost123]

Does any of you guys have a good idea on how ZFS compares to ntfs or linux's ext3? Is it that much of a quality leap we are talking about?

[poundsmack]

Adobe would actualy benefit form them moving away from HFS+. it might take a little work in the begining but they have never really cared much for HFS+ and have not fully utilized its abilities with their software offerings because it was deemed to be a need more on the lines of optional.

[Peaceful]

Apple has announced that ZFS read/write support will be in Mac OS X 10.6 (Snow Leopard) server (http://www.apple.com/server/macosx/snowleopard/), although there has been no official word on the regular version.


Oh, for heaven's sake. Like I've said before, the client version IS confirmed to at least have the command-line zfs utilities:

http://lists.macosforge.org/pipermail/zfs-discuss/2008-June/000663.html

See, I said it right here:

http://forums.macrumors.com/showthread.php?t=505409

[bobrik]

One blog post of a Sun employee recently delt with a ZFS layer called L2ARC - it's a cache layer between RAM and harddisk, suited for SDDs - harddisk < SSD < RAM. He demonstrated how adding SSDs to act as L2ARC cache yielded 6x performance boost on a file server use case (read: the data was not primarily stored on SSDs, they were stored on harddisks, but SSDs cached the commonly accessed data so that harddisk does not have to be asked for it). Hope it yields comparable performance boost for desktop.

The blogpost is here: http://blogs.sun.com/brendan/entry/test

[Peaceful]

The real question is...... do we expect more SSD based devices in Apple's product mix?

You ought to expect more SSDs in everybody's mix. The hard drive is dead (http://nathanstocks.blogspot.com/2008/07/hard-drive-is-dead-long-live-solid.html), long live the SSD!

[steveh]

It's worth pointing out that in Snow Leopard, the OS almost certainly won't be able to reside in a ZFS pool. Even Solaris, the OS for which the filesystem was originally developed, cannot boot off of ZFS.

That became false for Open Solaris in early June.

ZFS booting will be in place before Snow Leopard arrives.

[poundsmack]

Does any of you guys have a good idea on how ZFS compares to ntfs or linux's ext3? Is it that much of a quality leap we are talking about?

http://lists.umanitoba.ca/pipermail/apple-list/2007/000292.html this is a good place to look for a ltitle bit of info.

this is a litle dated but still good http://zamwi.com/2007/01/16/why-do-geeks-have-lust-for-zfs/

[surferfromuk]

Can someone give me a super simple primer on ZFS - I looked at the wiki but it's all specification orientated.

why it's such a big deal -

how much better is it ? Is HFS a bottleneck?

who own's it ( ie will Apple put something like this underneath a million macs if Sun can suddenly get bought by say MS and it gets burned) -

where is it going ?

[jackfrost123]

hey poundsmack, thanks a lot!

[iOrlando]

I dont think i am the only one thinking this...I have no clue what this article even says/or is about. haha. Hence the low comment count. I guess this is for those in the know about this kind of stuff.

[bigwig]

Why don't Adobe apps work on HFS+? It takes real work to make an app care what filesystem it's running under. HFS+ isn't really any different than HFS, it's case-sensitive HFS+ that breaks things, which is just wrong, no Unix-derived system should break that way.

[SC68Cal]

which is just wrong, no Unix-derived system should break that way.

Makes you wonder, doesn't it?

[lowbatteries]

Oh, for heaven's sake. Like I've said before, the client version IS confirmed to at least have the command-line zfs utilities:

http://lists.macosforge.org/pipermail/zfs-discuss/2008-June/000663.html


From the link: We're not going to do anything to disable ZFS on the Snow Leopard
client, however it will likely be command line only form, so
accessible and usable for your filesystem pleasure for all of you who
are more "hard core" :)

Sadly that doesn't really sounds like "support" to me. In fact, I would say the message leans more towards "un-supported" - but available - in the client version. I want a GUI!

Of course, the ZFS command line tools are pretty easy to use, but it would be awesome to plug in a firewire drive and be asked "Do you want to add the unformatted drive XYZ to 'MacBook HD'? "

[jackfrost123]

aren't unix filesystems generally case sensitive?

[THX1139]

I know it would hurt speed but wouldn't you want fragmentation so you could spread out the writes over all the memory space so you wouldn't wear out the SSD as fast?

SSD has no moving parts so I don't know what you mean by "wear out."

[MRrainer]

Unless Apple has made some *major* changes to ZFS, it cannot utilize other computers/file servers or such, only local disks can be used in a pool. I'm almost completely positive this is the case in Solaris (which I've used for a while).


There's parallel NFS (pNFS) where your data can live on any server "in the cloud" and whoever needs it fetches it from that host (or from a central host, if the client doesn't speak pNFS).
Also comes with OpenSolaris in the 2008/11 release, AFAIK.
Apple _really_ made a winning move when they settled for the BSD-codebase as a foundation for their future OS. Another one was of course the move to Intel.
It remains to be seen how Snow Leopard looks from a system-administration point of view, but with ZFS on board, they've at least a good headstart.

[chadder007]

SSD has no moving parts so I don't know what you mean by "wear out."

The storage sectors on an SSD can only be read and written too a certain amount of times before failure also....nothing works forever.

[THX1139]

I just bought what is hopefully my final hard drive (crosses fingers). I hope to phase my system over to SSD as the prices drop - and along with ZFS, all my files will be more secure and accessed faster. Good times ahead!

[MRrainer]

From the link:

Sadly that doesn't really sounds like "support" to me. In fact, I would say the message leans more towards "un-supported" - but available - in the client version. I want a GUI!

Of course, the ZFS command line tools are pretty easy to use, but it would be awesome to plug in a firewire drive and be asked "Do you want to add the unformatted drive XYZ to 'MacBook HD'? "

Currently, you can't remove a drive from a ZFS pool, at least not in Solaris 10U5 (don't know about OpenSolaris).
So, such a feature would be a sure road to disaster.
While ZFS is relatively easy, it also offers a lot of ways to shoot yourself in your feet (and head). NOT giving endusers a GUI for it is a good thing.
If you think you really desperately need it, there's always the man-pages ;-)

[JensenJJ]

Can someone give me a super simple primer on ZFS - I looked at the wiki but it's all specification orientated.

why it's such a big deal -

how much better is it ? Is HFS a bottleneck?

who own's it ( ie will Apple put something like this underneath a million macs if Sun can suddenly get bought by say MS and it gets burned) -

where is it going ?

SUN owns it, but it's under the CDDL licens it's a Open Source licen
http://www.opensolaris.org/os/about/faq/licensing_faq/

Beside alot of cool server/storges featurs, then it tjek you data for error all the time, and not just when you run scandisk/chekdisk/what ever.

Then for the Time Mashine, rigth now time mashine hardlink all non change files, and get a new copy of changes files.
on ZFS you can go down in bit information.
So if you change a file it will not copy the hole file to the time mashine, but only the info about the change.
Realy nice if you have a DVD projekt that you change a little in, then will not copy the hole file again.

You can also have it as a snapshot funktion on the computer it self.

And then server side it can alot of cool stuff whit ZPools. But you don't need that as a privat person

[THX1139]

The storage sectors on an SSD can only be read and written too a certain amount of times before failure also....nothing works forever.

Okay... but if you consider the friction and the heat generated by the spinning platters and the wearing out of the read/write head of traditional drives, then compare it to SSD, it has virtually no risk of mechanical failure. True that SSD have limited life cycle for writing data, but if you combine that with ZFS, you have an almost fool proof way to store and access data.

http://en.wikipedia.org/wiki/Solid-state_drive#Comparison_with_hard_disk_drives

http://wiki.eeeuser.com/ssd_write_limit

[surferfromuk]

SUN owns it, but it's under the CDDL licens it's a Open Source licen
http://www.opensolaris.org/os/about/faq/licensing_faq/

Beside alot of cool server/storges featurs, then it tjek you data for error all the time, and not just when you run scandisk/chekdisk/what ever.

Then for the Time Mashine, rigth now time mashine hardlink all non change files, and get a new copy of changes files.
on ZFS you can go down in bit information.
So if you change a file it will not copy the hole file to the time mashine, but only the info about the change.
Realy nice if you have a DVD projekt that you change a little in, then will not copy the hole file again.

You can also have it as a snapshot funktion on the computer it self.

And then server side it can alot of cool stuff whit ZPools. But you don't need that as a privat person

Thanks for that.

[bigwig]

aren't unix filesystems generally case sensitive?
Yes, but by default HFS and HFS+ aren't. You can create a case-sensitive HFS+ filesystem (new in 10.4 I think), but strangely OSX won't work right if you do.

[andiwm2003]

all this convinces me that SSD's in the moment aren't used very efficiently. therefore it's probably better to hold off from buying one. the performance gains aren't that great. it's better IMHO to wait another year and then buy a much cheaper 256GB SSD for my notebook and that will then be used efficiently by the OS.

[cwoloszynski]

Yes, but by default HFS and HFS+ aren't. You can create a case-sensitive HFS+ filesystem (new in 10.4 I think), but strangely OSX won't work right if you do.

No sure what issues you are seeing, but we all moved to case-sensitive HFS+ on our move to Leopard and it works great for us. No issues in our team.

[MacsRgr8]

You know what...


If what you said is true, and what I said here is also true... then the debate over whether or not Snow Leopard will be Intel-only or not is settled. Why? Sun and Apple won't bother to make ZFS boot support for PowerPC... and ZFS will be the native file system in Snow Leopard. :D

ZFS won't be required...

I have stated in other forums (and, no... I am not trying to hijack this one), but I do hope Apple keep up developing for PPC. So developing Snow Leopard for PPC, but not supported and certainly not optimised (AltiVec etc), and keeping, especially the server OS, open to IBM's Power series.
You know what I mean.... Marklar, but then the opposite: Ralkram... developing Mac OS X secretly on PPC :p

[gnasher729]

aren't unix filesystems generally case sensitive?

HFS+ and the Posix layer use case-insensitive Unicode. If Apple dared to use a case sensitive file system as the default, tons of applications would break, and a lynch mob would go to Cupertino.

[winterspan]

The storage sectors on an SSD can only be read and written too a certain amount of times before failure also....nothing works forever.

Okay... but if you consider the friction and the heat generated by the spinning platters and the wearing out of the read/write head of traditional drives, then compare it to SSD, it has virtually no risk of mechanical failure. True that SSD have limited life cycle for writing data, but if you combine that with ZFS, you have an almost fool proof way to store and access data.


Yep, SSDs certainly don't last forever, but with different data storage techniques and new processes, SSD manufacturers have dramatically increased the lifetime of SSDs in the last 3 years. New SSDs have incredible MTBF ratings compared to even enterprise level SAS HDDs.

Here's a snippet from a TGdaily article referring to Samsung about their new SSD line:
"It states the new 128GB SSDs will last "approximately 20 times longer than the generally accepted 4-5 year life span of a notebook PC hard drive". That's 80-100 years before it kicks."

Who gives a crap? SSD's suck at random I/O, and all but the best top dollar ones have only so-so throughput performance - they are a long ways out from being mainstream.

Given the order of magnitude quicker seek time compared against 15K enterprise HDDs, SSDs should have the fastest random I/O performance. Are you confusing that metric with something else?

Also, manufacturers of SSDs have made amazing progress in the last 12 months. *Significantly cheaper* SSDs from the likes of Samsung, Intel, Micron and many lesser-known Taiwanese companies with with 100MB+ read/write rates are going to be available soon. I'm willing to bet that we'll see *fast* 128GB+ SSDs for $400 within 12-18 months.

Again, from a TGdaily article (Micron preps 256 GB SSDs for notebooks - http://www.tgdaily.com/content/view/38743/135/)
"Sources suggested that 128 GB and 256 GB SSDs will be in reach for enthusiasts later this year with 128 GB models falling well below the $500 mark and 256 GB versions targeting a price point “around $500”"

just a few links:
Micron preps 256 GB SSDs for notebooks
http://www.tgdaily.com/content/view/38743/135/

Samsung puts 128 GB SSDs into mass-production
http://www.tgdaily.com/content/view/38307/135/

Samsung fires up 128GB SSD massive attack
http://www.theregister.co.uk/2008/07/09/samsung_128gb_ssd_mass_production/



I dont think i am the only one thinking this...I have no clue what this article even says/or is about. haha. Hence the low comment count. I guess this is for those in the know about this kind of stuff.

http://en.wikipedia.org/wiki/Solid-state_drive
http://en.wikipedia.org/wiki/ZFS


all this convinces me that SSD's in the moment aren't used very efficiently. therefore it's probably better to hold off from buying one. the performance gains aren't that great. it's better IMHO to wait another year and then buy a much cheaper 256GB SSD for my notebook and that will then be used efficiently by the OS.

Well, that isn't really true. There is indeed a lot of work to be done to maximize their performance, but they already great to have. The primary problem has not been performance, just price. High speed SSDs (over 100MB/sec sustained read/write speed) have historically been very expensive, but the prices are coming down very fast, so It's a good idea to wait another year or so for these fast SSDs to trickle down into consumer products. (Google "Intel Micron SSD").

[ChrisA]

I imagine Sun isn't going to stand still on extending ZFS' capabilities. And if they decide to stop, Apple might not.

There are some good logical reason way a file system is limited to local disks. But if you did want to use networked storage you could "mount" a remove export into the local file system.

I seriously doubt Apple wants to "fork" ZFS. That would create some serious problems years down the road.

For all of you who want to try out ZFS. You can download Solaris for free from Sun's web site. Solaris of ccourse comes with a working implementation of ZFS and Solaris will run inside a VMware Fusion VM on any modern Mac. If you don't have Fusion, get Sun's "virtual box" it's almost the same thing as Fusion but Sun is offering it for free. It runs well in mac OS X. Anyone can have a working ZFS sytem and all the documentation on a few hours if they want.

[aburgh]

Even though OSX has case-sensitive HFS support in place, it doesn't actually work if you try to use it. OSX bombs horribly on case-sensitive HFS boot partitions, and case-sensitive HFS partitions from DMGs also have weird failure modes. For example, if you try to use case-sensitive HFS+ with FileVault you can't log in (password works fine, but hilarity ensues).

Can you elaborate? I have HFSX (i.e., case-sensitive) boot partitions on two machines (no FileVault), and haven't seen any problems with the OS or included apps. The only issue I've seen is that FileMaker's network component fails, so I can't use remote databases.

Also, the iPhone uses HFSX partitions.

[twoodcc]

can't wait for ZFS on my macs!

[Wotan31]

Does any of you guys have a good idea on how ZFS compares to ntfs or linux's ext3? Is it that much of a quality leap we are talking about?

Are you kidding? It leapfrogs either of those two crufty old filesystems entirely! Leapfrogs them by 2 generations even! ZFS is competition for ext4 (still in beta) and WinFS (supposed to be included with Vista but wasn't - currently not yet released), but contains WAY more features than ext4 or WinFS ever will.

NTFS is garbage and always has been. ext3 is basically journaling capability tacked onto the ancient ext2 filesystem.

The last time I was this excited over a filesystem was a few years ago, when SGI released their very very excellent XFS filesystem as open source - been using that on my Linux machine for a while now and I love it.

Basically ZFS is better than any existing UNIX filesystem in every possible metric *PLUS* it adds full LVM capability into the filesystem layer *PLUS* it adds software RAID into the filesystem layer *PLUS* it adds features only found in high-end enterprise SAN storage arrays such as copy-on-write clones, snapshots, etc. *PLUS* traditional FS maintenance like integrity checking and filesystem repair become non-disruptive online functions! ZFS is also the only 128 bit filesystem - it is impossible to ever fill a ZFS filesystem (all the storage ever created in the whole world couldn't fill a ZFS filesystem). No one else has anything even close to it. ZFS is the One Filesystem to Rule them All.

Apple would be very foolish to omit full ZFS support from 10.6, particularly since HFS+ sucks so so badly. HFS+ is just about one of the worst filesystems out there and should have been replaced years ago. I wish Apple had replaced HFS+ with SGI's XFS a few years ago. would have been a nice improvement for OSX, plus would have made the transition now to ZFS even easier...

Read here for more info: http://en.wikipedia.org/wiki/ZFS

[SC68Cal]

If Apple dared to use a case sensitive file system as the default, tons of applications would break, and a lynch mob would go to Cupertino.

Why, because people are too dense to understand case sensitivity?

[bigwig]

Why, because people are too dense to understand case sensitivity?
I know some people don't like it, but I don't agree with their reasons, which to me revolve around laziness. The real problem are apps like Adobe's, which don't work on case-sensitive filesystems. I don't know the exact reasons for why Adobe's apps break, but in general this takes real work to accomplish. Things like internally changing the case of the user's input before looking to the filesystem, assuming two files of the same name in the same directory but different cases are duplicates (which is absurd, if that assumption were true it would indicate the filesystem is trashed and needs reformatting) and ignoring one of them, using inconsistent-case filenames throughout the app to refer to the same file, and so on. Rather than keep it simple and rely on the filesystem's logic and rules, add rules in your app that attempt (poorly) to duplicate the underlying filesystem's rules. In other words, making an app that breaks on case-sensitive filesystems took actual engineering hours to accomplish in order to add extra code that causes failure, when having no code at all would have been less work and more reliable.

[lowbatteries]

Currently, you can't remove a drive from a ZFS pool, at least not in Solaris 10U5 (don't know about OpenSolaris).
So, such a feature would be a sure road to disaster.
While ZFS is relatively easy, it also offers a lot of ways to shoot yourself in your feet (and head). NOT giving endusers a GUI for it is a good thing.
If you think you really desperately need it, there's always the man-pages ;-)

Wait ... you can't remove a drive from a ZFS pool? Wouldn't that kind of make ZFS ... garbage?

I played with ZFS for a couple hours on my leopard machine, creating a pool, growing it, shrinking it ... I'm pretty certain I both added and removed drives from the pool. In fact, in "man zpool", it says "zpool remove pool vdev".

Unless you mean you can't remove the drive with the data still intact, which I guess is a consideration. But Disk Utility handles some pretty complicated stuff with RAID, I'm sure ZFS isn't much more of a leap.

http://lowbatteries.com/temp/disk-utility-raid-search.png

[tyr2]

Wait ... you can't remove a drive from a ZFS pool? Wouldn't that kind of make ZFS ... garbage?

I played with ZFS for a couple hours on my leopard machine, creating a pool, growing it, shrinking it ... I'm pretty certain I both added and removed drives from the pool. In fact, in "man zpool", it says "zpool remove pool vdev"

You should read further on that man page


zpool remove pool vdev

Removes the given vdev from the pool. This command currently only
supports removing hot spares. Devices which are part of a mirror
can be removed using the "zpool detach" command. Raidz and top-
level vdevs cannot be removed from a pool.


You can't remove a vdev entirely from a pool. Also if you have a RAIDZ vdev you can't remove devices from it (you can replace them with equivalent devices but not remove them entirely).

ZFS is not as flexible with disk management as people imagine but it is improving.

[gnasher729]

Why, because people are too dense to understand case sensitivity?

I think "dense" is not understanding the implications for the user interface. There are many reasons why more people use MacOS X than Linux, a case-insensitive file system is one of them.

Not wanting to put up with some **** doesn't make people "dense". Do you think everyone driving an automatic car is "dense" and too stupid to shift gears?

I know some people don't like it, but I don't agree with their reasons, which to me revolve around laziness.

Face it, people use Macs out of laziness. Because they are too lazy to defragment their hard drives, because they are too lazy to install and maintain virus checkers, because they are too lazy to learn the intricacies of a system when they can instead buy a computer (Macintosh) that lets them do everything with much less effort. They are also too lazy to figure out whether they called a file "MyFile" or "Myfile" or "myfile". Instead of "lazy" I call it "valuing their time".

PS. Your opinion that it takes actual effort to break an application under a case sensitive file system is nonsense. Take an application that creates a file "Preferences" for its preferences and then tries to open a file "preferences" to read them. How many of these do you think are out there?

PS. WWW.MacRumors.COM works absolutely fine for me. Do you think it shouldn't?

[gnasher729]

Of course, the ZFS command line tools are pretty easy to use, but it would be awesome to plug in a firewire drive and be asked "Do you want to add the unformatted drive XYZ to 'MacBook HD'? "

It would be awful if that happened and "MacBook HD" would stop functioning properly if that drive is not plugged in while I'm on the road. Normally, if I use an external drive I know exactly what is in my internal drive and what is on the external, so I know what I lose if I don't take it with me. With ZFS, any file can be anywhere on the two disks. That is fine (actually brilliant) on a Mac Pro if you are editing videos and every time you run out of space you just plug in another drive and it just works, but it would be disaster on a portable computer.

[gnasher729]

I know it would hurt speed but wouldn't you want fragmentation so you could spread out the writes over all the memory space so you wouldn't wear out the SSD as fast?

You wouldn't do that in the operating system, but in the SSD drive itself. The SSD drive pretends it has say 15 million blocks of 4 KB each. When it notices that the OS has written to block number 1,234,567 an awful lot of times, then it just moves that block to a different location on its own. The OS never knows that when it reads or writes block number 1,234,567, the SSD drive actually uses a different bit of physical memory for it.

[bigwig]

They are also too lazy to figure out whether they called a file "MyFile" or "Myfile" or "myfile". Instead of "lazy" I call it "valuing their time".

Which only matters on a command line. If you're using a GUI, if you called it MyFile it will display MyFile, as will file selection popups. No memory or extra time is required. Note that while HFS is not case sensitive, it is case preserving, so the distinction between MyFile and myfile still matters.

PS. Your opinion that it takes actual effort to break an application under a case sensitive file system is nonsense. Take an application that creates a file "Preferences" for its preferences and then tries to open a file "preferences" to read them. How many of these do you think are out there?

More than zero is too many. Any app that does that is a broken POS for using a hardcoded filename defined in more than one place, which is more work than defining it once. The same goes for an app that tries to "massage" filenames. It's broken behavior: when I type "MyFile" the app darn well ought to do something to "MyFile". Now if I type "MyFile" and that fails, I have no problem with the app saying "did you really mean 'myfile'?" should "myfile" exist. It had better not try to use "myfile" silently, I despise software that tries to be smarter than me. The point here is that the behavior I espouse is agnostic as to the case-sensitivity of the underlying filesystem. "Dumb" works everywhere. It's when apps try to be "smart" that they stop working when the underlying filesystem changes how it treats case.

[loveturtle]

Where did this case sensitivity conversation start? I've only been skimming but I keep seeing people bring it up. What does this have to do with anything? Did anyone notice that ZFS has a casesensitivity dataset property?

turtle@phoenix ~ $ zfs get casesensitivity phoenix
NAME PROPERTY VALUE SOURCE
phoenix casesensitivity sensitive -

Wow, look at that. I can make my entire zpool or any fs inside my zpool case insensitive. This feature isn't yet in the mac osx zfs implementation but I'm sure it will be updated soon. They're only at zpool v8, this is v10.

Anyway, ZFS is too complicated for home users in my opinion. Supporting it on osx server and having it available in command line only form on the client version is exactly what they should be doing. Maybe it would be an okay idea to eventually allow single disk zpools or two disk mirror zpools only on client (through the ui, and still allow every other fancy configuration via cli) The last thing Apple wants to support is stupid home users creating stupid zpool configurations with their usb disks that they can't undo without destroying the pool.

The kind of people that should be using ZFS in 10.6 are the kind of people who already are using ZFS in 10.5 and are just waiting for nicer integration with Finder & boot support.

[knweiss]

Apple has announced that ZFS read/write support will be in Mac OS X 10.6 (Snow Leopard) server (http://www.apple.com/server/macosx/snowleopard/), although there has been no official word on the consumer version.

The problem is that ZFS doesn't make sense with only a single hdisk. I.e. all the laptops are out.

However, for the Mac Pro with its four drive slots it would be great the have this option.

[knweiss]

ZFS is competition for ext4 (still in beta)
ext4 unfortunately doesn't even come close. As you've said ZFS is generations ahead.


Basically ZFS is better than any existing UNIX filesystem in every possible metric *PLUS* it adds full LVM capability into the filesystem layer *PLUS* it adds software RAID into the filesystem layer *PLUS* it adds features only found in high-end enterprise SAN storage arrays such as copy-on-write clones, snapshots, etc. *PLUS* traditional FS maintenance like integrity checking and filesystem repair become non-disruptive online functions!

It also provides end-to-end checksumming which IMHO is really important.

[aburgh]

I think "dense" is not understanding the implications for the user interface. There are many reasons why more people use MacOS X than Linux, a case-insensitive file system is one of them.

Not wanting to put up with some **** doesn't make people "dense". Do you think everyone driving an automatic car is "dense" and too stupid to shift gears?

Face it, people use Macs out of laziness. Because they are too lazy to defragment their hard drives, because they are too lazy to install and maintain virus checkers, because they are too lazy to learn the intricacies of a system when they can instead buy a computer (Macintosh) that lets them do everything with much less effort. They are also too lazy to figure out whether they called a file "MyFile" or "Myfile" or "myfile". Instead of "lazy" I call it "valuing their time".

PS. Your opinion that it takes actual effort to break an application under a case sensitive file system is nonsense. Take an application that creates a file "Preferences" for its preferences and then tries to open a file "preferences" to read them. How many of these do you think are out there?

PS. WWW.MacRumors.COM works absolutely fine for me. Do you think it shouldn't?

You are mixing user interaction with file system implementation, though. The file system can be case-sensitive and the features you describe implemented in the OS in a layer above the file system. There are many good reasons for case-sensitivity at the file system layer:

http://drewthaler.blogspot.com/2007/12/case-against-insensitivity.html

And, BTW, I have encountered many web sites that were case sensitive.

[paj]

...
And, BTW, I have encountered many web sites that were case sensitive.

You may have encountered many case sensitive web pages, but the actual web site or hostname is a function of the DNS and is never case sensitive.

[gnasher729]

You are mixing user interaction with file system implementation, though. The file system can be case-sensitive and the features you describe implemented in the OS in a layer above the file system. There are many good reasons for case-sensitivity at the file system layer:

http://drewthaler.blogspot.com/2007/12/case-against-insensitivity.html

And, BTW, I have encountered many web sites that were case sensitive.

I looked at the guys blog up to his first claim where he makes claims about the complexity of case sensitivity. The claim he makes there is nonsense. First, HFS+ stores filenames in canonical decomposed Unicode (you'd want one of the two canonical representation anyway, otherwise your file system will be in deep **** if you want to support Unicode, and you _do_ want to support Unicode), and that makes case insensitivity quite trivial. It _is_ trivial compared to creating a canonical representation.

He suggests that case insensitivity should be handled at the UI level instead of the file system. Good luck doing that if you can have 2048 files called "MYLETTER.TXT" in all possible variations of uppercase and lowercase on a folder.

He suggests problems with standards (like Unicode) changing. Guess what: It is part of the Unicode standard that canonical representations will _never_ change. And it is part of the definition of HFS+ that its algorithm for case insensitive comparison will _never_ change.

[jackfrost123]

You may have encountered many case sensitive web pages, but the actual web site or hostname is a function of the DNS and is never case sensitive.

I am not sure I understand what you are saying...shouldn't no webpage at all be case sensitive since they all translate via dns to an ip adress and the ip adress corresponds to a case insensitive name? I don't understand the distinction you are making between page and site.

[aburgh]

I am not sure I understand what you are saying...shouldn't no webpage at all be case sensitive since they all translate via dns to an ip adress and the ip adress corresponds to a case insensitive name? I don't understand the distinction you are making between page and site.

He is saying that the host name will always be converted to lowercase, but that the path to the page may be case-sensitive. In other words, if you type
http://WWW.macrumors.COM/iPhone/, the www.macrumors.com will be normalized to lowercase by the DNS server, so case won't matter, but the /iPhone part of the path is dependent on the configuration of server software (Apache, IIS, etc.). But, you illustrate the point I'm about to make, which is the distinction doesn't really matter for the point I was trying to make, which is that sometimes a URL *is* case sensitive (even if the hostname is not).

[jackfrost123]

He is saying that the host name will always be converted to lowercase, but that the path to the page may be case-sensitive. In other words, if you type
http://WWW.macrumors.COM/iPhone/, the www.macrumors.com will be normalized to lowercase by the DNS server, so case won't matter, but the /iPhone part of the path is dependent on the configuration of server software (Apache, IIS, etc.). But, you illustrate the point I'm about to make, which is the distinction doesn't really matter for the point I was trying to make, which is that sometimes a URL *is* case sensitive (even if the hostname is not).

Ah, ok, very clear explanation, much appreciated.

[aburgh]

I looked at the guys blog up to his first claim where he makes claims about the complexity of case sensitivity. The claim he makes there is nonsense. First, HFS+ stores filenames in canonical decomposed Unicode (you'd want one of the two canonical representation anyway, otherwise your file system will be in deep **** if you want to support Unicode, and you _do_ want to support Unicode), and that makes case insensitivity quite trivial. It _is_ trivial compared to creating a canonical representation.

But you are focused on the wrong use case--it's not about optimizing performance when the user is typing a name in a save or open panel, it's about the *millions* of times the file system perform filename comparisons in the process of regular activity, such as launching apps, copying files, Time Machine backups, etc. Did you read his post? Did you try his fs_usage example to see just how often the file system is accessed? Off the top of my head, this would happen for each open() and lstat() call you see, and probably others.

Take a simple example: TextEdit wants to save your preferences, which it will do periodically, so it opens /User/you/Library/Preferences/com.apple.TextEdit. To find the file, start with "User" compare it with each file in the root directory until you find a match. Then, compare "you" to each file in the /User folder, and so on. Each path component could result in dozens of (or more) filename comparisons (and I'm simplifying it). If the file system is HFSX case-sensitive (technically, HFS+ is always case-insensitive. HFSX is HFS+ with the option for either), then the comparison is simply a byte comparison, as noted here:

http://developer.apple.com/technotes/tn/tn1150.html#HFSX

If it's case-sensitive, then every comparison is a call to the FastUnicodeCompare() function:

http://developer.apple.com/technotes/tn/tn1150.html#UnicodeSubtleties

Also, CFString/NSString have multiple internal representations, including canonical, path-optimized versions. I haven't actually profiled the OS, but I suspect there is some (a lot?) of caching of these decomposed, canonical names within these objects and in other parts of the APIs.

Again, this isn't about the case when the user types a name--in that case the performance differences are completely inconsequential.

But, this is all speculation from both of us. What would be interesting is some actual benchmarks.


He suggests that case insensitivity should be handled at the UI level instead of the file system. Good luck doing that if you can have 2048 files called "MYLETTER.TXT" in all possible variations of uppercase and lowercase on a folder.

Umm, if the UI level is treating them all as the same file name, how did they get created in the first place? Certainly, a user could presumably do so on the command line, but let's keep this realistic--you are unlikely to have more than one or two conflicts in the unusual case of there actually being a conflict at all.


He suggests problems with standards (like Unicode) changing. Guess what: It is part of the Unicode standard that canonical representations will _never_ change. And it is part of the definition of HFS+ that its algorithm for case insensitive comparison will _never_ change.

I thought this discussion was about ZFS, too? So, if ZFS handles converting case for a case-insensitivity comparison slightly different than HFS+, then the user will have different experiences based on which type of file system they use. On the other hand, if both file systems are case-insensitive, then the Open Panel or Save Panel or Finder can ensure the experience is consistent regardless of the file system.

[lowbatteries]

Anyway, ZFS is too complicated for home users in my opinion. Supporting it on osx server and having it available in command line only form on the client version is exactly what they should be doing. Maybe it would be an okay idea to eventually allow single disk zpools or two disk mirror zpools only on client (through the ui, and still allow every other fancy configuration via cli) The last thing Apple wants to support is stupid home users creating stupid zpool configurations with their usb disks that they can't undo without destroying the pool.

The kind of people that should be using ZFS in 10.6 are the kind of people who already are using ZFS in 10.5 and are just waiting for nicer integration with Finder & boot support.

RAID is too complicated for the home users too, as are all the Unix command line tools, XCode, Applescript ... but these are all still provided in the client version.

However, if they get ZFS bootable, then it would be awesome to have it as the default file system. ZFS has many advantages besides zpools, especially snapshots - single-disk Time Machine would be awesome. Oh, and stability, dependability, and speed (the Snow Leopard promise).

[Apple Ink]

Agreed SSDs are the future but I for one, believe that in the near future...... SSDs cant match the capacities offered by HDDs!

[MShock]

The problem is that ZFS doesn't make sense with only a single hdisk. I.e. all the laptops are out.

That's simply not true at all. Kustarz's demonstrates this in his blog. As a student who loves his laptop, as someone who loves experimenting w/ programming, and as a general end user, ZFS on a laptop will ROCK!!!!!

http://blogs.sun.com/erickustarz/entry/zfs_on_a_laptop

[MShock]

RAID is too complicated for the home users too, as are all the Unix command line tools, XCode, Applescript ... but these are all still provided in the client version.

However, if they get ZFS bootable, then it would be awesome to have it as the default file system. ZFS has many advantages besides zpools, especially snapshots - single-disk Time Machine would be awesome. Oh, and stability, dependability, and speed (the Snow Leopard promise).

I definitely concur. Clearing up the I/O performance and increasing data integrity will massively benefit the general end user. They may not be aware of it, but it will be cool. After reading this forum, I think that Apple may be dividing up developer tools a bit more. They may leave the GUI to a seperate disk like the developer tools. Perhaps release a developer pack with the tools later instead of what they did with 10.2 and HFS+ Journal. This would make sense.

And it seems as though ZFS is already implemented by Apple. MobileMe and the "hickup" screams ZFS due to the pNFS capabilities. "In the cloud push" I believe was Schiller rhetoric. That makes it seem much more likely ZFS will be the filesystem of the future, and the default file system non the less.

[The Tall One]

I'm gonna give it 5 years until it will be a chore to buy non SSD macs. Although cool, they certainly are not worth the buy right now. But maybe someday.

[proc]

It seems that the Finder (partial) rewrite could have been stimulated by the wish to have bootable ZFS in Snow Leopard as this post (http://zfs.macosforge.org/trac/wiki/issues) on MacOSForge suggests.

A quote from the post:"Since ZFS has a much different structure than HFS+ (pool based storage instead of device based) a lot of assumptions for tools and other parts of the system like Finder are no longer valid. Hence there’s still quite a bit of work that needs to be done to make us fit seamlessly with the rest of the system."

Another post (http://zfs.macosforge.org/trac/wiki/faq) says this:"Not yet but we are working on it. Since booting off of ZFS requires changes to other parts of the system we won't likely be able to release it in a simple tarball. However our goal is to be able to have it available for the next OS X release if we can."

What do you think?

[lowbatteries]

It seems that the Finder (partial) rewrite could have been stimulated by the wish to have bootable ZFS in Snow Leopard as this post (http://zfs.macosforge.org/trac/wiki/issues) on MacOSForge suggests.

A quote from the post:"Since ZFS has a much different structure than HFS+ (pool based storage instead of device based) a lot of assumptions for tools and other parts of the system like Finder are no longer valid. Hence there’s still quite a bit of work that needs to be done to make us fit seamlessly with the rest of the system."

Another post (http://zfs.macosforge.org/trac/wiki/faq) says this:"Not yet but we are working on it. Since booting off of ZFS requires changes to other parts of the system we won't likely be able to release it in a simple tarball. However our goal is to be able to have it available for the next OS X release if we can."

What do you think?

That's an interesting theory, especially if OS X is somehow going to incorporate the snapshot feature of ZFS.

[MShock]

That's an interesting theory, especially if OS X is somehow going to incorporate the snapshot feature of ZFS.

I agree. OpenSolaris recently adapted GNOME's Nautilus file manager with an ability to scroll through snapshots. http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs

I think ZFS also provides the a fundamental goal of Snow Leopard - efficiency! If there is a Cocoa Finder rewrite, that should allow the Finder to be more flexible to take advantage of ZFS capabilities for the end user. Snapshots is just the beginning, networking will be second (much easier for business and students), and saving hard drive space - particularly on my laptop (zfs doesn't build new files with new saves, but has a core file and saves every modification, the user can select a modifiction and add it to the file). http://blogs.sun.com/erickustarz/entry/zfs_on_a_laptop.

I also hope this means a desktop GUI redesign, something like a cross of Sony.com (US) and the iPhone interface. I want tabs, more black glass, and system preferences pane for the Finder to modify it to add features similar to Pathfinder 5.0!

It would be nice to see this compete with Windows 7! Blow Vista rewrite out of the water!

[lowbatteries]

I agree. OpenSolaris recently adapted GNOME's Nautilus file manager with an ability to scroll through snapshots. http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs


Quick Look with a time slider! that would be sweet.



I also hope this means a desktop GUI redesign, something like a cross of Sony.com (US) and the iPhone interface. I want tabs, more black glass, and system preferences pane for the Finder to modify it to add features similar to Pathfinder 5.0!


*shudders at flash websites* - I don't see how either the iPhone UI or the Sony website resembles the top of a desk. Perhaps those paradigms are becoming antiquated? Either way I like the desktop/dock just the way it is. Though I would like to see overlay icons on the dock's stacks, as well as be able to make stacks out of smart folders.


It would be nice to see this compete with Windows 7! Blow Vista rewrite out of the water!

I have absolutely no doubt that it will. Even if they add no new features, as they've promised. Macs have never sold because they had the most features, but because they got the features they do have right.

[MShock]

[/QUOTE]
I have absolutely no doubt that it will. Even if they add no new features, as they've promised. Macs have never sold because they had the most features, but because they got the features they do have right.[/QUOTE]

I meant more on performance than on the market. Both would be nice, but market-wise, you're probably right.

[monguesto]

Has anyone heard discussion of a system that uses both ssd & hdd, for highest speed between the two? Sony has a system like that.

[kdarling]

Here's a snippet from a TGdaily article referring to Samsung about their new SSD line: "It states the new 128GB SSDs will last "approximately 20 times longer than the generally accepted 4-5 year life span of a notebook PC hard drive". That's 80-100 years before it kicks."

Too bad Samsung doesn't specify in their press releases (http://www.samsung.com/us/business/semiconductor/newsView.do?news_id=936.0)what they're referring to. The fact that the drive can stay powered up longer? Who knows. I don't think it's about cycle lifetime, that's for sure:

40nm MLC NAND, as used in those new cheap SSDs (and the iPhone, for that matter), has only a 5,000 erase/write cycle life. That's two or three orders of magnitude less than SLC NAND.

It should be kept in mind that it does not take 128 billion writes to "use up" a cycle. Each memory block is 256KB and must be erased/re-written totally. Change ONE BIT every 256K on disk, for example, and it only takes 500 thousand writes to hit all the blocks. Using the infamous "rogue recorder" example (http://www.storagesearch.com/ssd-slc-mlc-notes.html), we're talking about one year of life for a 128GB drive.

PS. Yes, I'm taking the extreme examples, but I think it's important to point out that MLC based SSD makers are going the other way by using some pretty rosy scenarios for their specs.

[chris200x9]

I think "dense" is not understanding the implications for the user interface. There are many reasons why more people use MacOS X than Linux, a case-insensitive file system is one of them.


hahahaha that is hilarious! you can't be serious, care to tell me how? Let's face it unless you are working in terminal would the end user even know they have a case sensitive file system especially if they are "lazy" like the rest of your post alleges?

[babyj]

So if I've read everything correctly, Snow Leopard will full ZFS support but Apple haven't said whether it will be the default filing system or if there will be an option for it to be for home users?

Personally, I'd be surprised if either is the case and that at best it will be severely crippled for end users. The snapshot functionality (and all the related stuff) is great but would cause a lot of confusion for end users, for example the total drive capacity minus the size of all files on the drive does not equal the remaining free space.

The worst part is when you change a file that was created via snapshot - you will often see free disk space reduce but the size of allocated files stay the same. Try explaining that to an end user.

Using it for Time Machine drives makes a whole lot of sense, but it would raise the question of why Apple didn't use it for Time Machine drives in the first place rather than in effect, hacking their own version of it instead.