Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
ZFS doesn't make sense for Time Machine

x2

ZFS + timemachine makes sense.

Despite what people say, ZFS makes no more sense for Time Machine than HFS+. I used to think ZFS would work well for Time Machine backups, but after thinking through it a bit more, I realize I was mistaken.

Snapshots don't work across multiple volumes. The point of snapshots is to duplicate the filesystem metadata, while pointing to the same data blocks. Only as the data blocks change are they duplicated to preserve the snapshot state. However, Time Machine is not about snapshots. It is about helping users replicate their data on an external volume for fault tolerance. Snapshots on the same volume as the original data provide no fault tolerance. It is an additional benefit that users can use Time Machine to easily access their multiple backups.

The only way ZFS snapshots would work with an external volume is to use mirroring. That isn't a solution, either. Mirroring requires that the backup volume be the same size as the original volume. Hence, if you added a 300 GB external disk to an iMac with an 80 GB internal disk, only 80 GB of the 300 GB drive could be used for mirroring. The other 220 GB would have to be formatted as an independent volume. With this setup, you are limited to 80 GB to store your current data and all backups. As the disk fills up, you will be forced to delete old backups more quickly than if you could use the entire 300 GB disk for backups.

The only way ZFS snapshots would work with Time Machine would be if the VFS were modified to designate certain disks as "backups". These backup disks would be quasi-mirrors, which replicate the file operations on the master disk, while also taking periodic snapshots. However, they wouldn't be proper mirrors, because the metadata wouldn't match exactly. For example, it would be impossible to guarantee that the inode number of a file on the master disk would be identical to the inode number of the same file on the backup volume. Things like creating hardlinks becomes more difficult to manage when this is the case. Bottom line, this sort of modification would be a very ugly perversion of mirroring. It greatly increases the bookkeeping involved with replicated filesystem changes and ruins the fundamental purpose of a mirror: to have an identical copy of the filesystem on every disk. This is not the way to solve this problem!

Of course, I would like snapshots for my own devices. I already replicate my data. I'm one of the people in that small sliver Jobs showed in the keynote. After many years of UNIX use with high-speed networking, I have it ingrained to make use of SSH, rsync and unison to replicate my data and work remotely. My important data is replicated on my office desktop, my home desktop and my laptop. I don't need Time Machine to make duplicates for me, but snapshots would be a great help indeed. Either ZFS or UFS2 would be of great value to me on a Mac.

ZFS offers some additional nice features. On-the-fly compression and data checksums would be invaluable. Sun is right, especially with multicore Macs: there are plenty of wasted cycles, so they might as well be used to compress or validate data. I really hope ZFS makes it into Leopard, at least well enough that I can think about moving my /Users partition to it. But it won't solve the Time Machine problem.
 
once someone has the beta theyre handing out installed im sure we'll know!

It would be incredibly easy to simple leave the ZFS option, in the UI, out of the distributed builds, so that unless someone REALLY went poking around they wouldn't have any idea. Who knows, I do think it's a bit of a blow tho.
 
Despite what people say, ZFS makes no more sense for Time Machine than HFS+. I used to think ZFS would work well for Time Machine backups, but after thinking through it a bit more, I realize I was mistaken.

Snapshots don't work across multiple volumes. The point of snapshots is to duplicate the filesystem metadata, while pointing to the same data blocks. Only as the data blocks change are they duplicated to preserve the snapshot state. However, Time Machine is not about snapshots. It is about helping users replicate their data on an external volume for fault tolerance. Snapshots on the same volume as the original data provide no fault tolerance. It is an additional benefit that users can use Time Machine to easily access their multiple backups.

Of course you can't do cross-volume snapshots, but that's not a problem. Do the following:

1. Make a complete backup of your drive onto a ZFS volume
2. Wait until midnight.
3. Create a snapshot
4. Copy the changed files over top of their original versions.
5. Go back to step 2.

These aren't cross-volume snapshots, but they still have the same basic effect. Of course, step zero, "Implement ZFS on Mac OS", appears to be the holdup...
 
Here is one "feature" of Leopard that few have mentioned on this thread even though it is tremendously important:

SHEER SPEED

If the machine Steve Jobs used for the demonstration is by any means "normal" (i.e. not a 4-core Pro with 16 GB of RAM or something) then the responsiveness of the new OS seems truly amazing. We can't underestimate this, especially with all these abundant "quick look" views and thumbnails.

Also, let's put things in a real-world perspective: I share the widespread disappointment. I also expected more daring ideas, more fundamental innovation, including even a new input method (multi-touch) or an entire new interface paradigm (based on Core Animation). But, at the same time, we must admit that the "minor tweaks" that Apple chose to focus on are all wisely chosen productivity boosters that will save us great amounts of time and frustration in the real world. They were, in other words, very mature choices of a design team that is serious about usability rather than experimenting. Compare these smart choices with the indiscriminate, distracting and occasionally downrightly hilarious interface "innovations" in Vista.

Only one complaint about Apple: I wish they would not withhold all the innovations they have in their labs for so long. I really feel like proposing a boycott on Apple products just to force them to make the leap from concepts to products more quickly.
 
Here is one "feature" of Leopard that few have mentioned on this thread even though it is tremendously important:

SHEER SPEED

If the machine Steve Jobs used for the demonstration is by any means "normal" (i.e. not a 4-core Pro with 16 GB of RAM or something) then the responsiveness of the new OS seems truly amazing.
Of course he put the fatest he got there, why wouldn't he?

My machine with OS X doesn't feel snappier than in Windows, boots faster thought. Me don't like the beachball ;/

Also imho "more speed" would just mean "not as slow", not "omg it's fast!"
Compare these smart choices with the indiscriminate, distracting and occasionally downrightly hilarious interface "innovations" in Vista.
Yeah, because I use dashboard so often ... Oh, and the Balmer effects is very necessary to.
 
Also can someone tell me what deep sixing is ? We don't have this phrase in england and if i were to guess it's meaning it wouldn't be printable :)

lol :D , did anyone answer this yet?
Nothing like what you're thinking, that's a deep something else.
It means to kill. I believe it's a reference to burying a body in a grave six feet deep.

Edit: nevermind, it looks like I've got it wrong by about 90 degrees. See the link just below
VVVVV
 
Also can someone tell me what deep sixing is ? We don't have this phrase in england and if i were to guess it's meaning it wouldn't be printable :)

It technically means to bury, or to dispose of by burial.

Check out the Urban Dictionary.
http://www.urbandictionary.com/define.php?term=deep+six

As far as "deep six" colloquially meaning "to kill," things that are buried alive usually don't live for long, do they. Hence it can also mean "to kill by burying" in addition to "it's dead and needs to be buried" or "kill it so there is a reason/excuse to bury it." Related term: "six feet under" (i.e. under the ground).

"Someone needs to deep six that dog carcass."
"Deep six that cell phone, or I will!"
"We are going to attack the fort and deep six anyone inside."
"The mafia deep sixed him because he testified in court."
"After Jonathan Schwartz leaked the information about Apple support of ZFS, Steve Jobs deep sixed the feature."
 
Time Machine

........ZFS offers some additional nice features. On-the-fly compression and data checksums would be invaluable. Sun is right, especially with multi-core Macs: there are plenty of wasted cycles, so they might as well be used to compress or validate data. I really hope ZFS makes it into Leopard, at least well enough that I can think about moving my /Users partition to it. But it won't solve the Time Machine problem.

Excellent post. thank you've helped answer a lot of the questions I had about what the impact of not having ZFS would be for using Time Machine to back up a Mac.

I don't mean for this to be a support topic but I thought that the example might help to discuss the pros and cons of Time Machine with or without ZFS might help those with greater understanding of the TM technology to explain some of the misconceptions that I and others seem to have about how they would interoperate and what benefit TM might or might not provide as a result.

When I first saw Time machine as a simple click and forget "incremental" backup system i.e. that it continually backed up each file from a given Mac as it was modified I thought it was a great idea. The part that I had a problem with was how large the external backup repository would have to be in comparison to the source data which was to be protected - given retention period, size and frequency of changes etc.

For Example
I currently have 2*500 Gb drives in my G5PM. No. 1 is used for system, apps and home drive. The other is used for aperture library, scratch pad for dumping data and "other data" that I don't want cluttering my main drive.

Current Process
In total I have about 600 GB of files (including system, apps, data etc.) that I back up regularly. I have two external 1TB drives which I use for backing up this system. Using a combination of manual copying, Deja Vu and the .Mac Backup Utility. I perform a Monthly Full backup (rotating between the two drives) and then let the tools perform their thing on a nightly basis to increment any data that has changes. In all cases the backups are nothing more than file replicas with a database log of what has been put where and when. Standard stuff I know..... Now since I eventually hit the limit of the external drives, every other month I wipe the entire drive and then go and delete the log and backup dB entered for that target volume, reset the backup and perform a new full backup and start again. That housekeeping task takes me about an hour (excluding the time it takes to do the actual backing up). The process then starts all over until the drive is at near 90% capacity. Generally I modify between 5 and 20 Gb of files a day on the system, so there is finite limit of incremental backups that the current devices can accommodate.

Stuff I would not use Time machine for

Aperture I is backed up using the Vault mechanism to dedicated 250GB ad 500GB external drives - I keep a dozen of them of them and rotate the backups in a sudo-GFS type rotation cycle, taking one drive out of the backup set every 3 months to keep off-line for 12 months. Since I need multiple instances for a high level of fault tolerance I understand that Time Machine is probably not going to be suitable for this.

Music/Movies: Other data such as the multiple TBs of Music, Podcasts, Movies and TV shows that I have on my Mac Mini (HTPMac) for use with Apple TV does not get dynamically backup up. I have a bank of multiple 500GB drives and for each drive that backs up I have an off-line replica which I keep stored. Sine the data is static, the only information I back up from this machine is the iTunes Library which contains the meta data and artwork etc.​

Hopes for Time machine

The way that TM was presented at MacWorld. I thought it was pretty cool in that it simplified the process of backing up a single client. however the more I thought about is the more I realised that control over retention and backup growth might be an issue since effectively you were keeping multiple copies of every file that would change - seemingly indefinitely. Obviously storage limits would be hit pretty quickly on a system which was doing only file level copying to a backup area. When I read about ZFS my hopes were that some sort of lower level technology was going to be used with TM which wold optimise the amount of storage required to keep those long term multiple version by reducing the amount of data needed to re-construct a file to a given state. Akin to copying only the blocks of the file which changes in each, rather than a fresh copy of the file every time it changes.

From what you were saying I take it that this is not now the case, and that time machine will only do file level copying? And have I and other therefore misunderstood that ZFS would ever have given that kind of advantage over using TM with HFS+?

thanks.
 
Secret feature #297: You can empty trash from the pulldown menu bar.

Sad but true. Sometimes it's weird what is considered a feature. But be assured lots of those secret features will be covered in the WWDC sessions, those where no press is allowed and attendees agreed to the NDA (as they are all ADC members).

I'd say the most significant features of Leopard will be Core Animation, Time Machine and QuickLook.
- Core Animation really makes everything way more intuitive for the user with barely additional effort for the developers.
- Time Machine... well, put an external HD on your Airport Extreme and you have a backup server—wireless, one click setup.
- QuickLook will replace Preview in the long run, I already love how Preview can open PDFs in a blink of an eye, now with QuickLook, you'll be able to open just about anything. No need to install Office just because someone e-mailed you something important as a word document (damn those people!, ODT files are even worse!). And you can finally enjoy Quicktime movies without shelling out 30 bucks for Quickime Pro (or writing down the "Apple Retail" license in your closest Apple Store...). Cover Flow in the Finder is great for lazy people like me who just dump stuff in a big folder and later find the stuff with spotlight. All in all this means you don't need to open Apps just to look at stuff. Major Memory Saver in my opinion.

Then you have spaces (the old linux feature, with a nicer implementation). Hitting F8 gives you the bird's eye view, you can even do Expose (F9-F11) in that view and still move windows around. Looks awesome with 100 windows open.

Once you're used to the menu bar (rumor has it, you can make it opaque), you'll love Leopard.
 
SHEER SPEED

If the machine Steve Jobs used for the demonstration is by any means "normal" (i.e. not a 4-core Pro with 16 GB of RAM or something) then the responsiveness of the new OS seems truly amazing.

I pose a question: Why would Steve Jobs choose anything less than an Octo-Core 16GB RAM MacPro to demo on?

Of course he has the fastest machine possible, he would be stupid not to.
 
[..]
- QuickLook will replace Preview in the long run, I already love how Preview can open PDFs in a blink of an eye, now with QuickLook, you'll be able to open just about anything. No need to install Office just because someone e-mailed you something important as a word document (damn those people!, ODT files are even worse!).
[..]

Although I assume QuickLook will open OpenDocument/ODF (.odt) files, is there any confirmation or proof somewhere?
 
Quicklook (and i hope CoverFlow view) supports plugins for Developers, so it'll only be a matter of time till ODF docs are supported.

According to the keynote, there will be extensions for more filetypes.

In addition, I think one of the big features/fixes/should have been there since 10.2 is the file sharing. If it works like Steve said, it'll be great (and .Mac might be worth it now with the "Back to my Mac" feature).

And I'm not so sure Steve would completely drop a core technology (if it is in fact there) just to get back to someone. He might have kicked it out of the keynote, and might deny it, but if it were there and had been well-developed into Leopard, Steve wouldn't have cut it.
 
Quicklook (and i hope CoverFlow view) supports plugins for Developers, so it'll only be a matter of time till ODF docs are supported.

I know, but since TextEdit is rumoured to read and write ODF without plugins I hope QuickLook will support them without plugins as well.

I definitely hope it will, plugins are nice but you can never rely on people to have an additional plugin installed. Native would be much better.
 
This is something I already mentioned

Of course you can't do cross-volume snapshots, but that's not a problem. Do the following:

1. Make a complete backup of your drive onto a ZFS volume
2. Wait until midnight.
3. Create a snapshot
4. Copy the changed files over top of their original versions.
5. Go back to step 2.

These aren't cross-volume snapshots, but they still have the same basic effect. Of course, step zero, "Implement ZFS on Mac OS", appears to be the holdup...

This process is what I'm talking about when I mention "quasi-mirrors". You can either copy the changed files over in one pass (using something like rsync), or you can modify the VFS to duplicate the file manipulations on both disks as they occur. You're right that these have the "same basic effect", but they are essentially a poor-man's version of mirroring and snapshots. I believe this is a clumsy solution because it breaks the functional interpretations of both mirroring and snapshots:

1. The current copy on the second disk is not an exactly replica of the data on the original disk. The metadata is different.

2. A snapshot is supposed to be an exact image of your filesystem at a specific time. However, these snapshots are images of the different filesystem, with different metadata.

Different metadata means things like different inodes for the same file on two different filesystems, and different data blocks consumed by that file. You will also have different creation times for the copies than for the originals. While you could modify the ZFS driver to write an arbitrary creation time, I think that violates the spirit of the creation time.

These differences are minor, and they probably don't affect the average user. However, they make it impossible to perform low-level operations between the filesystems. For example, suppose the inode for an important file on your original filesystem becomes corrupted, while at the same time the data on the backup becomes corrupted. You can't copy over the original with the backup, since the backup data is corrupted. Also, because the inode on the backup does not match the metadata on the original, you can't reconstruct the original's inode from the backup. You've lost your file! You will now have to scan your original disk block-by-block, hoping to identify the data blocks that belonged to your file so you can create a new inode for it.

If you're going to go to the trouble of copying data over and then making snapshots, you're better off making Time Machine a frontend to dump(8) and restore(8). These old UNIX utilities were designed to perfectly preserve filesystem states on other volumes, and are capable of incremental backups. With a few changes to the dump file formats to allow indexing and file-by-file extraction, dump and restore would be better solutions than the copy-and-snapshot process we've discussed. They also avoid the problem of corrupt inodes on your original and corrupt data in your backup; the inodes stored in the backup perfectly match those on the original. Hence, in the scenario above, a low-level utility would be able to recover your file without a costly block-by-block search. The best part is, dump files can be placed on a volume with any format at all.
 
Nope. It's available for Solaris and FreeBSD that I'm aware of.
Once FreeBSD can boot from ZFS will we still have to assign disk space to /, swap, /tmp, /use, etc., or is one of the advantages of ZFS is we do not have to worry about space?
 
Once FreeBSD can boot from ZFS will we still have to assign disk space to /, swap, /tmp, /use, etc., or is one of the advantages of ZFS is we do not have to worry about space?

One of the advantages of ZFS is... is... well, uh... there is none because Steve's ego decided to drop it from Leopard just to get back at Jonathan Schwartz :mad: :mad: :mad:
 
This is obviously just apples revenge on sun for spilling the beans. They told him, and he spilled apples big surprise. Now apple gets to steal away all his credibility!
 
I know, but since TextEdit is rumoured to read and write ODF without plugins I hope QuickLook will support them without plugins as well.

I definitely hope it will, plugins are nice but you can never rely on people to have an additional plugin installed. Native would be much better.

Hmm, Tiger's Textedit can open ODT files, but you only get some unreadable mess. It opens DOC files nicely tho.
 
According to the keynote, there will be extensions for more filetypes.

And I'm not so sure Steve would completely drop a core technology (if it is in fact there) just to get back to someone. He might have kicked it out of the keynote, and might deny it, but if it were there and had been well-developed into Leopard, Steve wouldn't have cut it.
What if ZFS makes its "debut" with OS X Server (Xsan?) instead of OS X Client?
 
It would make sense to put ZFS on a user partition though, allowing us to backup our files and our applications online - a real separation between the user and the OS.

This would be nice. I actually setup my first OS X machine this way (with separate boot & user partitions) but OS X had such a fit keeping applications and user data separate from the boot volume that I haven't done it since. Too bad.
 
One of the advantages of ZFS is... is... well, uh... there is none because Steve's ego decided to drop it from Leopard just to get back at Jonathan Schwartz :mad: :mad: :mad:


I'd be far more inclined to believe that Jonathan Schwartz made a mistake than believe that Steve Jobs would pull a feature that supposedly up until the beginning of the week was a major component of the OS. The business impact of reversing the decision, let alone the credibility loss that would entail would be substantial in comparison to a bruised ego. SJ might like to be the showman, but he's also a very shrewd business man and would likely take another form of action. After all, from what I've read Sun are not going to benefit hugely from Apple implementing ZFS - at least not from the majority of Mac, but Apple users could benefit from ZFS therefore it would be his loyal customers who would suffer as a result of that decision not Sun.

As dissapointing as it is, I think that the rumour was simply untrue and that Sun got some wires crossed and maybe confused over potential support for the file system vs. a wholesale move to ZFS over HFS. Who's to say that even the ZS references we have seen from older Dev seeds of 10.5 would in fact have made it to the release version as it is not uncommon for features to ramin in a product during development only to be removed before release because they are not fit for market.
 
Perhaps some hop

One of the advantages of ZFS is... is... well, uh... there is none because Steve's ego decided to drop it from Leopard just to get back at Jonathan Schwartz :mad: :mad: :mad:

Have a look at the following:

http://www.opensolaris.org/jive/thread.jspa?messageID=128542#128542

"I just installed the Leopard beta that was distributed at WWDC. Sadly, the installer provided no ZFS option (the only options were HFS Extended Journaled and a case-sensitive version of the same).

However, typing this in the terminal:

$ sudo zpool status

Returned this:

ZFS Readonly implemntation is loaded!
To download the full ZFS read/write kext with all functionality enabled, please go to http://developer.apple.com
no pools available"
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.