Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Where is the ZFS?

CDDL and switching licenses

scaredpoet said:
… if Oracle is going to decide that the open source Sun software you're using is suddenly switching licenses …
Click to expand...

How could software that's subject to CDDL become no longer subject to that licence? I can't imagine it.

http://opensource.org/licenses/CDDL-1.0

http://open-zfs.org/wiki/FAQ#Licensing states:

… We do not have the power to change the (CDDL) license of OpenZFS. No single entity holds the copyright to all of the OpenZFS code and all contributors to OpenZFS maintain copyright to their changes. Changing the license would require the consent of each one whose changes are part of the current codebase. That is basically everyone who has contributed since 2001. …
Click to expand...
 
Last edited:
2009

wrldwzrd89 said:
… there shouldn't be a problem now. There definitely was a license clash before, though.
Click to expand...

Whilst I'm loath to revisit history yet again … I would not describe it as a license clash.

In a single sentence, essentially confirmed by Jeff Bonwick (co-founder of the ZFS project) in October 2009 (a few months before Oracle's January 2010 acquisition of Sun Microsystems):

David Magda said:
Apple can currently just take the ZFS CDDL code and incorporate it (like they did with DTrace), but it may be that they wanted a "private license" from Sun (with appropriate technical support and indemnification), and the two entities couldn't come to mutually agreeable terms.
Click to expand...
Click to expand...

The original author's preamble (emphasis added by me), in full:

I'm guessing that you'll never see direct evidence given the sensitivity that these negotiations can take. All you'll guess is rumours and leaks of various levels of reliability.
Click to expand...

In the four years since then: Jeff and other people in the know have remained suitably quiet on the subject in public. Let's draw a line under that history …

-----

… and with the last few weeks of 2013 and all of 2014 in mind: move on :)

:apple:
 
Last edited:
Reasons to prefer ZFS

hyhybt said:
Why do you want ZFS?
Click to expand...

Primarily: ZFS for integrity of data.

With HFS Plus I saw corruption of files far too often, with volumes that were – according to Disk Utility – apparently OK. Contradictions such as this should be unacceptable to any sensible end user. However, the vast majority of people who are affected by file corruption never realise the contradiction; never discover the true cause of difficulties with their Macs.

Beyond integrity of data: with ZFS I get much more than can be given by HFS Plus and Core Storage. ZFS is more flexible, more powerful, better documented and so on.

There's a very well established community of developers. The OpenZFS Developer Summit begins tomorrow morning …
 
ZFS with one- or two-disk systems

Bear said:
… ZFS, what will that really gain you on a small one or 2 disk system? …
Click to expand...

I use ZFS with Mountain Lion on a MacBookPro5,2 with a single internal disk, an sshd. A Core Storage logical volume is given to ZEVO for my ZFS home directory.

I have absolute certainty that my encrypted ZFS home directory is free from errors.

Two of my external hard disk drives, which remain at home, form another ZFS pool. Some of the ZFS file systems within that pool are for backups. Automated hourly snapshots of my home directory (much better than Mobile Time Machine) are sent to one of the backup file systems.

I have absolute certainty that the backup file systems are free from errors.

If ever an error affects a file in my home directory, ZFS will name that file and its path. I can then restore, or roll back …
 
ZFS and licensing (again)

danielsutton said:
Pardon my previous post if I did not sound clear enough. Apple had intended to use ZFS as part of OS X, but after Oracle purchased Sun Microsystems, ZFS moved to a new license which made it impossible for Apple to use it in its commercial software. So, as I remember reading, Apple hired some filesystem engineers to work on a new system for OS X, although since it takes a number of years to create a new filesystem, it was not included in Snow Leopard, Lion, Mountain Lion, or Mavericks. I am hoping that the new system will appear as part of OS X 10.10 next year.
Click to expand...

What was the new license? Where's the record of the move?

danielsutton said:
Apple is not able to use ZFS due to licensing terms, but the company did hire filesystem engineers several years ago to work on an alternative, which will likely include similar features to ZFS. I am hoping that OS X 10.10 will include said filesystem.
Click to expand...

Please see https://forums.macrumors.com/showthread.php?p=18084653#post18084653 and other posts in this topic.

I'm not aware of anything in the licence to prevent Apple from using ZFS.
 
grahamperrin said:
What was the new license? Where's the record of the move?



Please see https://forums.macrumors.com/showthread.php?p=18084653#post18084653 and other posts in this topic.

I'm not aware of anything in the licence to prevent Apple from using ZFS.
Click to expand...

My understanding is that Oracle moved ZFS and some other projects to a variation of the General Public License, which forbids their use in commercial software. There was quite a bit of press coverage of this issue as Apple was trying to include ZFS in Snow Leopard Server, only to remove it entirely without saying much about why it was removed. Later on, we discovered that the issue was one of licensing terms after Sun Microsystems, along with Solaris and its ZFS filesystem was acquired by Oracle. Please correct me if I am mistaken.

I do remember reading, though, about the hiring of filesystem engineers by Apple, after the ZFS project fell through. It takes a number of years to create a new system, and this may be why we haven't seen one as of yet. I am hoping, though, that HFS+ will be replaced with a more robust system, and that this will happen in OS X 10.10 next year.
 
ZFS licensing, ownership, copyrights and trademark

Thanks …

danielsutton said:
My understanding is that Oracle moved ZFS and some other projects to a variation of the General Public License …
Click to expand...

I'll seek a definitive answer.

In the meantime, I wonder whether people have different understandings of what's meant by the word ZFS. It was, for a while, abandoned as a trademark. So, for example, ZEVO is an implementation of ZFS (with multiple copyright holders – not owned by Oracle); and so on.

Now again the mark is owned by Oracle America, Inc. but I believe that a key fact remains:

  • the ZFS most commonly associated with Apple – the open source code for ZFS – is subject to the CDDL.
 
Right now I have little to add to the conversation other than to say that apparently that petition is still accepting signatures and to add a personal note to this whole matter:

I'd happily sacrifice many years of feature additions or stock applications being added to OS X if I could have ZFS in turn.
It's my only REAL BIG GRIPE with OS X. Sure, I have others, but none come to close to this.

Glassed Silver:mac
 
nutmac said:
Most folks would cite 3 benefits: data integrity, data deduplication, and storage pool.

Data integrity: I have had a number of hard disks whose data became corrupted completely unbeknownst to me. And HFS+ retrieves corrupted file as if nothing has gone wrong.
Click to expand...

I move TBs of data around all day long and I've never had a corrupted file, ever. I can replicate the German method to corrupt a file by using dd, but that is really academic in my opinion. I understand ZFS is technically better, but the reality is that for the average user, and even pro users, there's really nothing wrong with HFS+. Maybe...MAYBE if you have a transactional database running on a server this would be a little more of a concern. But data integrity has been wildly overblown as a reason to move from HFS+ to ZFS.
 
dBeats said:
… I've never had a corrupted file …
Click to expand...

I hear you, but with respect: how do you verify the integrity of all files on your HFS Plus file systems?

I do have disks with HFS Plus that seem entirely free from corruption – the first that comes to mind is in a Macintosh Server G4, going strong after more than a decade. Its longevity always makes me smile, but since working with ZFS I have learnt to be less blasé about its data.

I also have disks that seemed corruption-free with HFS Plus, that revealed a habit of corruption when given instead to ZFS. I could have given more detailed information in post 28 above, but I don't want to turn this topic into a set of case studies.

It is possible that you truly never had a corrupted file, but I suggest the possibility/likelihood that you never knowingly had one.

… for the average user, and even pro users, there's really nothing wrong with HFS+.
Click to expand...

I must disagree.

There is, at least, the latency issue. In the words of John Siracusa: "File system metadata structures in HFS+ have global locks. Only one process can update the file system at a time. This is an embarrassment …". Since I have experimented over the past year or so with a variety of disk arrangements, I realise why he used the word embarrassing.

… data integrity has been wildly overblown as a reason to move from HFS+ to ZFS.
Click to expand...

Maybe sometimes a little overblown for use cases where loss of data is bearable, but never wildly overblown. I'll seek some data on the subject.

In the meantime, there's this tweet from Don Brady, who has extraordinarily deep knowledge of both HFS Plus and ZFS on OS X:

https://twitter.com/DonJBrady/status/344599780120481792
 
grahamperrin said:
I hear you, but with respect: how do you verify the integrity of all files on your HFS Plus file systems?
...

It is possible that you truly never had a corrupted file, but I suggest the possibility/likelihood that you never knowingly had one.
Click to expand...

Well, from my point of view, if there's a flipped bit in my 20GB file (typical sizes for what I do), the file would be worthless since there's no redundancy built into this particular file format. It's either perfect or unreadable. So I guess my litmus test is no one has come back to me saying "Hey this file doesn't work." I'm sure clients would love to complain to me about that (they sure complain a lot about other things!!)

I'm sure it only takes one incident for me to change my attitude, maybe I'm just lucky enough to not have that incident yet. But I have a feeling if this was really a data integrity problem, that Apple would have addressed it. At least that's what I want to think!
 
I installed the latest zfs for OSX on my OS X Server 3.0.1, on Mavericks 10.9. Seems stable etc..

However, performance it is very very slow. A fresh "new" timemachine backup went from 9 hours to 4 days. On a blackmagic disk speed test I went from 300mb/s to 20mb/s on the same array.

To clarify. The old array was a simple 8 disk raid 5.

I have the ability to test multiple scenarios. I have 2x Raid5 (dual port SATA3) controllers that I can configure to be Raid0, 1, 5, 10 or JBOD. I am comparing the 300mb/s at Raid5 vs JBOD with raidz. In total I have 16 SATA3 drives. 8 are 3TB Red WD drives and 8 are 2TB Red WD drives.
 
Apple support for ZFS: petitions, feedback and more

Petitions

Glassed Silver said:
… that petition is still accepting signatures
Click to expand...

A link, for people who are unfamiliar –

Petition | Apple: OS X 10.9 - support OpenGL 4.3 and ZFS | Change.org

If/when the time comes for a comparable petition in relation to 10.10 – a petition focused solely on ZFS - we might add a link to the petition from the following topic:


(OT: for a new petition, I have in mind the text for a draft.)

… It's my only REAL BIG GRIPE with OS X. Sure, I have others, but none come to close to this.
Click to expand...

+1 (up-voted above).

Debatably much more important than petitions and votes:

Feedback to Apple

Apple - Mac OS X - Feedback

In a promoted comment under ZFS-loving Mac users demand support in OS X 10.9 | Ars Technica (2013-02-12, highlights):

… please strengthen your case by sending well-structured feedback to Apple. The communication may be one-way, but there's great value in using the recognised channel.

Hints: in Apple's form, keep feedback about ZFS separate from feedback about OpenGL; separate from feedback about HFS Plus. Stay positive. Make clear your use case.
Click to expand...

I disagreed with only one point in the article:

Chris Foresman said:
… the demand for ZFS support may fall on deaf ears.
Click to expand...

For a variety of reasons, I believe that Apple does pay attention to well-structured feedback.

Chris Foresman said:
Aside from the fact that Apple hasn't shown any indication that it will support anything other than HFS+ for the time being …
Click to expand...

Now: in Mavericks there are, to my eye, at least two subtle indications that the operating system is prepared to gain support, of some sort, for an additional file system.

Back to the opening question –
OS X Mavericks (10.9) > Where is the ZFS?
– my guess is that support from Apple may be present, or greater, in 10.10 – but 10.10 is off-topic …
 
grahamperrin said:
[...]

Now: in Mavericks there are, to my eye, at least two subtle indications that the operating system is prepared to gain support, of some sort, for an additional file system.

[...]
Click to expand...

What are you referring to? I'd love to know. :)

~~~

Also, I created a Google Group to educate others and ourselves on how we can push this most effectively.
Click here

I'm sure some of us would like a few pointer on what makes a good feedback or which categories to pick etc...
It's also an easy address to share to have a compendium of how to get involved overall. Anyone's free to add ideas.

Group description:
This group focusses on sharing ideas to spread the word about ZFS and building up pressure on Apple. This is not a technical support group, much more do we want to spread the word and find effective ways to get heard. If you want to actively work towards native ZFS for OS X, this group is for you!
Click to expand...

Feel free to share, join and converse in the group as much as you'd like.
I set it up as forum, so you're free to set up email notifications however you'd like or disable anything email related all-together. :)

I started by making a 10 Steps to good Feedback kind of write-up.

Glassed Silver:mac
 
Last edited:
Rally to bring native ZFS to the Mac (Google group)

Glassed Silver said:
… Also, I created a Google Group to educate others and ourselves on how we can push this most effectively.
Click here …
Click to expand...

Thanks!

The opening post there looks good to me. Smart thinking.

I have joined the group, but my contributions there may be sporadic. I'll be particularly busy with something ZFS-related for the next week or so.
 
Last edited:
ZFS licensing

danielsutton said:
My understanding is that Oracle moved ZFS and some other projects to a variation of the General Public License …
Click to expand...

Follow-up to post 32 above:

I asked around a few places, including some channels in IRC, and revisited some bookmarks. From what I gather: at least some of the confusion around GPL, in relation to ZFS, involves GNU GRUB. There is, for example, GRUB for Native ZFS : “Native ZFS for Linux” team – and GRUB is subject to a GPL – but that license does not extend to ZFS.
 
grahamperrin said:
Thanks …



I'll seek a definitive answer.

In the meantime, I wonder whether people have different understandings of what's meant by the word ZFS. It was, for a while, abandoned as a trademark. So, for example, ZEVO is an implementation of ZFS (with multiple copyright holders – not owned by Oracle); and so on.

Now again the mark is owned by Oracle America, Inc. but I believe that a key fact remains:

  • the ZFS most commonly associated with Apple – the open source code for ZFS – is subject to the CDDL.
Click to expand...

Thank you for the clarifications! It is good to hear more about this. I do remember, however, hearing that Apple hired some filesystem engineers a few years back, so it may be that the company wants to build its own filesystem, rather than use one that already exists. Apple does have a tendency to re-invent and re-imagine things, and so I am thinking that they may want to re-invent the filesystem as we know it. Would be interesting if they tied the local Mac and iOS filesystems in with iCloud in some way, to create a "CloudFS" implementation. I am brainstorming here, but this is one great thing about Forums, that we can have open discussions about technologies and make predictions about what may be to come.
 
If you want ZFS you'll have to use BSD.

HFS+ is pretty old and ZFS would be the logical choice but Apple will come up with something proprietary
 
Glassed Silver said:
What are you referring to? I'd love to know. :) …
Click to expand...

The very few things that I imagine to be clues about ZFS are nothing to get excited about. One thing involves a behaviour that's seen in relation to HFS Plus. Another involves a behaviour that's seen in Mountain Lion but not in Mavericks … the sort of thing that I'd expect with improvements to an operating system that "just works". Subtle, in an Apple way.

danielsutton said:
… Apple hired some filesystem engineers a few years back …
Click to expand...

Certainly there have been periods of focus on Apple Core Storage (a storage system but not a file system).

I don't doubt that Apple might internally and privately make great strides in file system development, but in my opinion there's a powerful and growing case for Apple to take best advantage of the expertise within and around the OpenZFS community. Some of what's around this open-source community can't be open in the same way, but can be equally valuable in raising the overall profile of OpenZFS as an outstanding storage platform.

Two potential challenges:

  • how to make the ZFS-oriented communities – as a whole (not the MacZFS community alone) – as attractive to Apple as they were, for a while, before OpenZFS existed
  • accepting that a mutually advantageous relationship might involve something less than complete openness from Apple and other sides.

I feel passionately about those two points, and can offer some suggestions, but I'd like to hear what others think.

… we can have open discussions about technologies and make predictions about what may be to come.
Click to expand...

I must emphasise: none of what I say here relates to 10.9.1 or greater. Such things are strictly confidential.

In the context of this topic: my experience is with prerelease and released builds of 10.9.

Hopefully some final points on licensing: there's a healthy interest, but my reactions to debate about ZFS, Apple and licensing are sometimes akin to an illumos community reaction to debate about package management :eek:

In part one of the series of videos from this week's OpenZFS Developer Summit there were, I recall, at least two questions about licensing. It's a healthy interest but the questions were politely, professionally deferred; I take this as a strong indication that licensing is not a hindrance to development :)
 
Reasons to not go solely proprietary

MyMac1976 said:
… Apple will come up with something proprietary
Click to expand...

For storage/file systems development, with proprietary (alone) comes the risk that what's delivered will be inferior, in too many ways, to systems that were more openly developed.

Take, for example, Microsoft ReFS. I don't want to get too much into bashing but I see an extremely wealthy organisation delivering a technology that's a poor substitute for ZFS. Yes, ReFS has data checksumming capability but if I'm not mistaken, that capability is disabled by default. From a ZFS perspective (more than twelve years mature with integrity in mind from the outset), I could brutally describe Microsoft's new ReFS as fundamentally retarded.

(Why is checksumming not enabled; what's the downside? Why is ReFS not an option for consumer editions of Microsoft's OSes? And so on.)
 
grahamperrin said:
For storage/file systems development, with proprietary (alone) comes the risk that what's delivered will be inferior, in too many ways, to systems that were more openly developed.

Take, for example, Microsoft ReFS. I don't want to get too much into bashing but I see an extremely wealthy organisation delivering a technology that's a poor substitute for ZFS. Yes, ReFS has data checksumming capability but if I'm not mistaken, that capability is disabled by default. From a ZFS perspective (more than twelve years mature with integrity in mind from the outset), I could brutally describe Microsoft's new ReFS as fundamentally retarded.

(Why is checksumming not enabled; what's the downside? Why is ReFS not an option for consumer editions of Microsoft's OSes? And so on.)
Click to expand...

I'm not justifying it, it's just Apple's MO..

I use butter I'm sure the butter team could use Apples resources just like I'm sure the the ZFS crew could use them too..

ZFS is about the best out there but Apple just needs to play ball and they don't like to do that.

Dunno man, it's be a big win for ZFS if adopted by Apple.
 
MyMac1976 said:
… Apple's MO.. … Apple just needs to play ball and they don't like to do that. … a big win for ZFS if adopted by Apple.
Click to expand...

Whilst I can't give a developer perspective (I'm an end user) I can say without hesitation that in at least some dealings with customers, Apple does proactively play ball … as best it can, given the numbers involved and the passions that are expressed. With so many millions of customers and such diversity, it simply can't be that "the customer is always right". ;)

Every individual has a modus operandi, or more than one. My M.O.s are sometimes far from perfect. (I recall private discussions where my enthusiasm for ZFS – my wish for people to realise its potential in relation to OS X – made me the target of some abuse. Combinations of private and public abuse, no matter how small the quantities, can quickly bring a person down and so I occasionally divorce myself from certain areas but never lose sight of the bigger picture, of which ZFS is a part. But enough about me.)

Organisational M.O.s? Try to not take too much notice of the press. It's occasionally entertaining to take an epic view of the Sun–Oracle–Apple thing as commonly portrayed by the press, but there's no substitute for personal experience. As much as I like to play down the Oracle part (it is, after all, easily viewed as just one milestone in the history of OpenZFS), I found myself unexpectedly touched by Deirdré Straughan's five-part series of blog posts (the sixth is not yet written). For anyone with an interest in the acquisition of Sun, recommended reading: Resistance is Futile: The Oracle Acquisition.

incidentally, when I think about the two potential challenges given above, I think first of the modi operandi of individuals and groups …
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back