ZFS is on Leopard... Sorta.

[SMM]

The migration to ZFS is not something to be accomplished easily. We just had an extensive thread about this. Those in the know, pointed out several technical challenges for doing this cleanly. Others were puzzled as to whether those with adequate storage resources, would be able to install Leopard, IF it required ZFS.

So, there will have to be some transition strategy. It could be a read-only version at first, then ZFS installed on NEW computers. Finally, it would be a process of helping others to upgrade. Come on people..use some sense.

 

[SMM]

Agreed. Apple's inability to get their ducks in a row on this one is almost enough to drive an otherwise-reasonable observer into the tinfoil hat schwartz-pissed-off-steve-so-much-they-yanked-it-at-the-eleventh-hour camp.

VSP - you figure that out newbie.

 

[jawzzy]

Hey guys/girls, thanks for all the answers! I get it now. THIS is why the forums are so useful!

 

[kalisphoenix]

Okay two things.
1) everyone who has thought time machine would use zfs does not understand how zfs works. I'll never understand why people who don't know what they're talking about like to pretend they do on the internet..there's plenty of things i don't know..i don't pretend to know any of them..

Time machine uses a second disk (or, an additional storage device). this is not how zfs snapshots work. zfs snapshots are created by not freeing the old data when modified data is written, that is all.

I know perfectly well how ZFS snapshots work, and their method doesn't preclude a modified (and immensely superior) Time Machine from using ZFS. In fact, ZFS snapshots, as essentially diffs of an entire filesystem, are immensely efficient and implemented in a practically infinitesimal amount of time. That's why I mentioned that Time Machine will be less impressive now that the possibility of it being based upon ZFS is basically nil.

i mean ZFS is cool and all, and i suspect it will one day be the system of choice on the Mac, but you have very little details available on time machine, and somehow, that is enough to magically tell you that the file system will transform it into a great app?!?

Yes. If you don't understand how ZFS could immensely improve Time Machine's functionality, then why are you bitching at me? It seems immensely simple to me.

Let's say I have a file. It contains a macrocosmic version of "ABCABCABCABC". Then I decide to change its contents to a macrocosmic "ABCDEFABCDEF".

ZFS would simply create a new file with "^DEF^DEF" with the carets signifying references to the data shared with the original version, hence having both versions of the file but taking only roughly 1.5 times the space, compared to HFS, which would take two times the space of the original file (not to mention taking a longer amount of time). ZFS would thus be more efficient both in terms of time and storage space.

By all means, feel free to correct me if I'm wrong. Please do. I welcome any chance to learn.

 

[AlBDamned]

hmm.... I guess before we go through this again.

I'll move it to page two, until we hear otherwise.

arn

Ah sorry Arn, has this point been covered before? I don't know much about ZFs, just read the comment and I thought it unusual for Schwartz wto go public saying that without being certain it was the case.

 

[rspeed]

Correcting loveturtle

Okay two things.
1) everyone who has thought time machine would use zfs does not understand how zfs works. I'll never understand why people who don't know what they're talking about like to pretend they do on the internet..there's plenty of things i don't know..i don't pretend to know any of them..

Time machine uses a second disk (or, an additional storage device). this is not how zfs snapshots work. zfs snapshots are created by not freeing the old data when modified data is written, that is all.

I'm sorry to say that while you're correct about how snapshots function, that doesn't mean that it couldn't be used for Time Machine. A few months ago when the details about ZFS support in the Leopard beta surfaced I (like many others) immediately thought of the benefit this could provide for Time Machine. It also occurred to me that this might not be possible since the snapshots would be occurring directly on the internal drive, thus ruling out the use of another partition (you're the only other person I've encountered that seems to have noticed). My theory at first was that you could set up two pools as RAID-1 and allow the system to take the external drive's pool offline when unplugged, then update it when brought back online. However, that seemed like it would be slow, messy and possibly dangerous. Luckily, my boss happens to be an expert on all things Solaris. In fact, he even wrote a rather slick ZFS-based backup system. I asked him about the issue and told him my theory (which, as I had thought, would be a bad idea). As it turns out ZFS can export snapshots from one pool to another.

That's it, problem solved. Time Machine could make a snapshot (which is essentially an instant operation) then export that snapshot to the external pool, transferring only the data which had changed since the previous snapshot. It would be very fast (at least compared to how Time Machine does it on HFS+), and would have the added benefit of being available for recovering corrupted files.

Thus my disappointment.

 

[Hopstretch]

VSP - you figure that out newbie.

Enchanted, I'm sure. But seriously, it isn't even a little odd that a company this obsessive about its public communications has to take several stabs at answering a simple question about a key aspect of one of its marquee products?

 

[MacVault]

OK, we got it, you are outraged that ZFS is not fully supported in Leopard. How many times do you have to sprinkle your F-word replacement characters all over this forum? In every thread about this issue, it is the same story. Maybe it is not stable enough yet, maybe it is not much use unless you are using Leopard server, who knows. All I know is that about 95% of the desktop or laptop users in this forum would not even know ZFS existed if not for the rumors and I am tired of your crude ranting. Just because you are not writing the whole word does not make it any more socially acceptable.

Take a pill...

Yea, whatever, theBB... I am dissapointed about lack of ZFS. But I'm outraged that after promising us at MWSF07 "Top Secret" features Apple yesterday showed us what they call a "feature complete" build of Leopard and correct me if I'm wrong but I didn't see much that lives up to their "Top Secret" feature hype. Dashboard? BootCamp? etc. Give me a break.

 

[nickelcokes]

I'm sorry to say that while you're correct about how snapshots function, that doesn't mean that it couldn't be used for Time Machine. A few months ago when the details about ZFS support in the Leopard beta surfaced I (like many others) immediately thought of the benefit this could provide for Time Machine. It also occurred to me that this might not be possible since the snapshots would be occurring directly on the internal drive, thus ruling out the use of another partition (you're the only other person I've encountered that seems to have noticed). My theory at first was that you could set up two pools as RAID-1 and allow the system to take the external drive's pool offline when unplugged, then update it when brought back online. However, that seemed like it would be slow, messy and possibly dangerous. Luckily, my boss happens to be an expert on all things Solaris. In fact, he even wrote a rather slick ZFS-based backup system. I asked him about the issue and told him my theory (which, as I had thought, would be a bad idea). As it turns out ZFS can export snapshots from one pool to another.

That's it, problem solved. Time Machine could make a snapshot (which is essentially an instant operation) then export that snapshot to the external pool, transferring only the data which had changed since the previous snapshot. It would be very fast (at least compared to how Time Machine does it on HFS+), and would have the added benefit of being available for recovering corrupted files.

Thus my disappointment.

Whoa, this changes everything. I spent a bunch of time this morning explaining why snapshots wouldn't work as a backup mechanism on an external disk, back in the other ZFS thread. Now that you mention exporting snapshots, I think I remember reading something similar at one point. This surely would make for a nice backup mechanism.

 

[kalisphoenix]

That's it, problem solved. Time Machine could make a snapshot (which is essentially an instant operation) then export that snapshot to the external pool, transferring only the data which had changed since the previous snapshot. It would be very fast (at least compared to how Time Machine does it on HFS+), and would have the added benefit of being available for recovering corrupted files.

Thanks for the post. I was beginning to think it was "Stomp the Hippie Day" or something.

That's pretty much exactly what I was expecting, and thought everyone else was talking about. I guess I was wrong

(Even though HFS+ can't do snapshots like ZFS, it would seem natural to provide some sort of low-level functionality that merely compares the blocks' checksums and transmits the changed blocks to the Time Machine backup drive rather than the entire new file. Could this be how Time Machine could be implemented with a ZFS external/networked drive and an HFS+ system drive?)

I mean, Linux users have been doing something like this for years with rsync. It's admittedly far less efficient, excruciatingly less sexy, and far less advanced, but it's a similar principle operating on a file level rather than on a raw level.

 

[shen]

Yes. If you don't understand how ZFS could immensely improve Time Machine's functionality, then why are you bitching at me? It seems immensely simple to me.

Let's say I have a file. It contains a macrocosmic version of "ABCABCABCABC". Then I decide to change its contents to a macrocosmic "ABCDEFABCDEF".

ZFS would simply create a new file with "^DEF^DEF" with the carets signifying references to the data shared with the original version, hence having both versions of the file but taking only roughly 1.5 times the space, compared to HFS, which would take two times the space of the original file (not to mention taking a longer amount of time). ZFS would thus be more efficient both in terms of time and storage space.

By all means, feel free to correct me if I'm wrong. Please do. I welcome any chance to learn.

oh i get it now! there is simply no way they could do that WITHOUT ZFS is there? i mean if they don't have your new fav L33T FS then there is no possible way that use any other method within the software of time machine to actually do that at all!

can i get an immense eye roll icon please?

so what you are saying, is that in your immense and infallible wisdom, apple could never manage to do the above without a new file system. is that your position?

i understand how ZFS can improve the app. i don't see that it is needed to have the same effect. further, since you can't even boot on the bloody format, i for one don't want it as the primary file system.

naturally i suspect you will explain to me how they could never do what they want without ZFS, yet somehow they can easily make it bootable, right?

 

[Analog Kid]

Bummer. I was hoping the delay to October would give Apple time to finish this up.

So much for Time Machine being incredible. Now it's just "really nifty."

Yeah, that was my thought too... As it stands, Time Machine seems like a workaround hack waiting for something like ZFS to make it really hum. Snapshotting at the file level is going to be pretty inefficient for media files.

Does anyone know if just changing metadata in a media file will trigger a snapshot?

Okay two things.
1) everyone who has thought time machine would use zfs does not understand how zfs works. I'll never understand why people who don't know what they're talking about like to pretend they do on the internet..there's plenty of things i don't know..i don't pretend to know any of them..

Since you clearly don't understand what kalisphoenix is talking about, it would probably be best not to pretend you do...

your kidding, right?

i mean ZFS is cool and all, and i suspect it will one day be the system of choice on the Mac, but you have very little details available on time machine, and somehow, that is enough to magically tell you that the file system will transform it into a great app?!?

read your tag line, and get back to us when the drugs wear off.....

Ditto.

Why do people get so rabid just over the perception that someone said something inaccurate? Courteous correction will make you look much less ridiculous when it turns out that you're the one who's wrong...

 

[kresh]

So with no ZFS, I guess we're back to Drobo? I like the way Drobo appears to work in the demo video and performs the features of ZFS that I was interested in. Has anyone used this little device?

http://www.drobo.com/products_demo.aspx

Edit: I was wondering if the Drobo box is running a flavor of Linux and it just hit me that maybe Drobo is using ZFS? Is it?

 

[Analog Kid]

oh i get it now! there is simply no way they could do that WITHOUT ZFS is there? ...

What part of "really nifty" are you missing?

What's with the wolves at the water hole today?

 

[DMann]

Finder

I'm outraged that after promising us at MWSF07 "Top Secret" features Apple yesterday showed us what they call a "feature complete" build of Leopard and correct me if I'm wrong but I didn't see much that lives up to their "Top Secret" feature hype. Dashboard? BootCamp? etc. Give me a break.

True, it's a shame that Steve didn't spend more than 10 minutes demonstrating the new, revamped Finder, which, by any standards, is sensational, and worthy of high praise. Perhaps the novel idea of utilizing 'Coverflow' as a GUI resource for searching isn't quite up to the rank of 'Top Secret,' insanely great, or mind blowing, but the development a multi-threaded finder which is lightning fast, has minimal latency, and sports a gorgeous graphic interface is highly praiseworthy. The new Finder really ought to have been the main focus of the 10 listed features. If a rehash of Time Machine was deemed necessary, then, at least, focus on the new refinements made to it (sure did seem more responsive and elegant than it did in build 9A410) OS X' Finder needed a major overhaul, and it got one. This alone will make the upgrade well worth it.

 

[Analog Kid]

So with no ZFS, I guess we're back to Drobo? I like the way Drobo appears to work in the demo video and performs the features of ZFS that I was interested in. Has anyone used this little device?

http://www.drobo.com/products_demo.aspx

Edit: I was wondering if the Drobo box is running a flavor of Linux and it just hit me that maybe Drobo is using ZFS? Is it?

Hey-- I like this idea! Where was it when I was looking for an external array... There are a few features missing that ZFS would provide though-- parallel access for higher throughput and the snapshots that seem so controversial right now...

 

[superbovine]

I predict that ZFS will become really big for people using XServe RAID and other large-capacity drive arrays. I don't see much point to using it on a computer that has only one hard drive.

That's quite a leap of logic, there.

They may also simply be attaching ZFS volumes formatted on Solaris via SCSI, FireWire or FibreChannel. What makes you think they have to actually run Solaris on the Mac to make this work?

I tend to agree, the partical implementation of zfs would be large fiber SAN storage. From the sounds of it, things like Oracle will scale well on the file system. More than likely, they will have zfs offering on XRAID

 

[jwthompson2]

Read only support is weird. I hope this changes and that I can create ZFS volumes across my iSCSI SAN for OS X Server. I guess we'll have to see where the betas go as we get closer to launch. Hopefully there will be something better than just read-only support in the final Leopard server. I'd hate to have to setup Solaris just so I can do ZFS across my iSCSI SAN when OS X showed promise in being able to do this in betas.

 

[dashiel]

The migration to ZFS is not something to be accomplished easily. We just had an extensive thread about this. Those in the know, pointed out several technical challenges for doing this cleanly. Others were puzzled as to whether those with adequate storage resources, would be able to install Leopard, IF it required ZFS.

So, there will have to be some transition strategy. It could be a read-only version at first, then ZFS installed on NEW computers. Finally, it would be a process of helping others to upgrade. Come on people..use some sense.

thank you for being a voice of reason amongst a sea of OMFG ZFS IS TEH ROXOR AND APPLE SUX FOR REMOVING IT!!!

put simply, ZFS is a very, very new technology and not ready for deployment to a user base of 20+ million.

 

[kalisphoenix]

oh i get it now!

I don't think you get it. I'm not interested in ZFS being default or even a choice for an OS X install. My interest in the FS extends no further than Time Machine, and merely for its clear and immense strengths. I'm normally not a filesystem nerd -- I've always run ext2/ext3 on my Gentoo boxes, for instance, because the marginal improvements of, say, ReiserFS or XFS aren't enough to catch my interest. I've never seen HFS+ get borked, which is enough to keep me using it until I'm basically forced to change ZFS's immense strengths for an application like Time Machine outweigh its relative youth and experimental nature. Which is saying something, since I've never been particularly willing to "experiment" with my backups.

The point is not the functionality, which one way or another can be accomplished with HFS+. Hell, I just mentioned rsync, which is FS- and even OS-independent The differences are in efficiency and interface. Trust me: rsyncing your whole drive from a cron script would work, it just wouldn't work very well. An HFS+ Time Machine is immensely better, even close to how it could be if it used ZFS, but not quite. As you said yourself:

i understand how ZFS can improve the app.

Which is all I said.
*shoots sarcasm bullets at you*
*dances*
*uses stars*

Yeah, that was my thought too... As it stands, Time Machine seems like a workaround hack waiting for something like ZFS to make it really hum. Snapshotting at the file level is going to be pretty inefficient for media files.

Yeah. I think of iMovie and cringe

What's with the wolves at the water hole today?

Hehehe. Thanks

 

[inkhead]

This is not exactly how ZFS works (close) for files it makes a snapshot, and It only saves snapshots of changed data, meaning that instead of traditional snapshots which mirror everything, it only saves the changes. So if your iTunes library at 40GB needs to take a snapshot, it only saves a backup of the 10 tunes you changes, instead of another 40GB file.

The part about linking to files is wrong, basically any any file i/o operation it works with a copy of the data, checksums it and doesn't work with the original data like traditional filesystems. This saves HUGE on i/o, and error checking. It kills off lots of overhead, and eliminates errors.

oh i get it now! there is simply no way they could do that WITHOUT ZFS is there? i mean if they don't have your new fav L33T FS then there is no possible way that use any other method within the software of time machine to actually do that at all!

can i get an immense eye roll icon please?

so what you are saying, is that in your immense and infallible wisdom, apple could never manage to do the above without a new file system. is that your position?

i understand how ZFS can improve the app. i don't see that it is needed to have the same effect. further, since you can't even boot on the bloody format, i for one don't want it as the primary file system.

naturally i suspect you will explain to me how they could never do what they want without ZFS, yet somehow they can easily make it bootable, right?

 

[kalisphoenix]

thank you for being a voice of reason amongst a sea of OMFG ZFS IS TEH ROXOR AND APPLE SUX FOR REMOVING IT!!!

Most people (at least me) aren't criticizing Apple. I'm merely disappointed they weren't able to implement it. It's an "aw shucks" rather than a "damn them" sort of moment.

Off-topic, but why is it that whenever a Macrumors poster disagrees with a previous poster, they feel forced to restate the previous poster's message all in 1337speak and capital letters? Do you really think that it makes your point any better? Do other forum readers come along and say, "wow, that Dashiel is a gentleman and a scholar. Look at what he thought up: restating Kalisphoenix's message all in 1337speak and capital letters!" Anyway, please cut it out. It's pretty lame

put simply, ZFS is a very, very new technology and not ready for deployment to a user base of 20+ million.

ZFS has been in the wild for quite some time now.

 

[DMann]

Zfs

ZFS has been in the wild for quite some time now.

True. There must be some reason for the delay. At least we have progress in the 'read only' department. I'm no expert in file management, but would it be difficult to integrate Time Machine with ZFS instead of HFS+? At least things seem to be heading in that direction.

 

[kalisphoenix]

True. There must be some reason for the delay. At least we have progress in the 'read only' department. I'm no expert in file management, but would it be difficult to integrate Time Machine with ZFS instead of HFS+? At least things seem to be heading in that direction.

FS programming is difficult, thankless, and unforgiving I was really hoping Apple could pull it off, though.

 

[twoodcc]

well i think they'll put it in leopard by october