Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

autrefois

macrumors 65816
I think I'm beginning to understand what this is all about. People are making it sound like there are no limits, but for the average person is it correct to say that it doesn't mean you'll have unlimited space, just that ZFS can handle files/etc. of any size while having each individual thing take up less space, am I right?

each snapshot is a snapshot of how the blocks look, not your files. So the size of each snapshot is only however much space it takes to record what blocks have changed.

So in very simplified terms, snapshots are basically like wikipedia? It just keeps tracks of changes instead of creating a whole new copy of it, correct?
 

Trekkie

macrumors 6502a
Nov 13, 2002
920
29
Wake Forest, NC
This would be very exciting feature to have. The ability to have one single Macintosh HD and just keep adding drives (internal or external) to increase the space.

Sweeeet. :D
 

ChrisA

macrumors G5
Jan 5, 2006
12,561
1,671
Redondo Beach, California
For us poor plebeians, what the hell does the adoption of ZFS bring anyway? Is it just another geeky filesystem, or something that might really make a difference for ordinary users?

Did you read this?
http://opensolaris.org/os/community/zfs/whatis/

1) All operations are "copy on write".
data are never over written you can always go back to last month's version . The system saves space by only writing changes.

2) RAID and volume managment are integratd into the filesystem layer
File systems can span disks and you can add a disk to a file system without mess up the data.

Finally Mac OS can get rid of those disk drive icons. There can just be "storage" Much like you don't care much about how many RAM chips you have, you only care about the total amount of RAM. Disk can be like that too. Just open the box and slide in one more disk and the rest is "magic". Just like with RAM.

3) end to end checksums: These is no more thinking "maybe the file is corrupt". If anything goes wrong you will be told and conversely if you are not told you can be sure there is nothing wrong

4) there is not enough data on Earth to fil a ZFS file system. Or at least if you wrote to a disk 24x7 you would not live long enough to fill up ZFS. (although you would fill up quite a few physical drives)

5) priority based scheduling of I/O -- stuff that has to be read from the disk fast gets done ahead f stuff that can wait. ZFS may be better able to use all those CPU cores we will be seeing soon as four and eight core systems become common. ZFS can make a system seem more responsive even when it is under heavy I/O load.

6) data is compressed as it is writen to disk -- so less space is used but also because disks are slow, less data are written so it takes less time (assuming you have CPU power to "burn" which a four core system will have)
 

shawnce

macrumors 65816
Jun 1, 2004
1,442
0
This is one of the reasons I love being an OS X user. Can't wait to see what Apple eventually does with it.

I would love Apple to leverage the capabilities of ZFS (in particular zpools and related capabilities). Knowing them they will make it simple to manage, if not transparent in many cases. However I think it will be a longer road then 10.5 to get to that point.
 

MBP123

macrumors regular
May 26, 2006
192
0
Let's run through a simple example.

You have 1 hard drive right? You are running out of space. You go buy another drive. You want to add it to the system but now you have to decide whether you want that drive to just handle storage of files, and if so which files? Well, how about this instead? You can add the drive as a pool. It then "magically" appears as if your original drive is now x gigabytes larger than it is since it is using two drives as a pool. Convenient no?

Or how about you want to backup that drive instead? Ok, add a drive to the system. Add it to the pool, tell it to mirror the drive instead. It now copies the data from one drive to the other and any changes mirror the other. If a corrupt file is on drive 1 (your original working copy) it checks the other to see if the backup is non-corrupt. If so, it opens that file, and copies the good data to the original drive as good data. It does this with Checksums of the files.

Another for the geekiness factor is RAIDZ. One drive or two can be a parity drive. Ever use PAR files? Yup very similar. You have files or a whole drive disappear? You can pretty much restore it from the parity drive if enough of the data still exists, etc.

Or how about you like having "versions" of your filesystem. You're about to update OS X to 10.5.2 and you're afraid it might break your system. So you create a ZFS snapshot. It now olds this "snapshot" of your filesystem and the files in it. You install the new update, it does indeed bork your computer. So you tell OS X and ZFS to use a previous snapshot, boom. You're back to 10.5.1 and it works.

Also the idea of using ZFS with time machine, is really really cool. Also, on the fly compression of your files. With no real performance hit. Another neat one is built in encryption. So you'll be able to have file vault at the filesystem level rather than the OS X application/OS level.

ok but say i buy an external hard drive, and "pool" it with the internal hard drive. the external hard drive craps out and dies (like most hard drives do). is all my data then gone? or is some data still there from the HD that didnt fail
 

thejadedmonkey

macrumors G3
May 28, 2005
9,180
3,324
Pennsylvania
What happens if you add a drive into a pool, and at a later date wish to remove it from the system? Can you just tell the computer to move all of the data off of the drive and then pull it out (after turning off the computer, of course, or not if it's SATA) with no harm or lost files?
 

Doctor Q

Administrator
Staff member
Sep 19, 2002
39,782
7,514
Los Angeles
It's a natural fact: Over time, newer and better filesystems are developed. What's good to see is that Apple is willing to accept these changes and offer us new filesystems.

We don't want a "one size fits all" filesystem, and I mean that in both senses of the phrase: Users will benefit by being free to configure their disk space in a number of ways, and users need filesystems that are practial for variable-size files, from huge databases to the thousands of tiny files that come with some applications.
 

Goldfinger

macrumors 6502
Jan 7, 2006
329
73
Belgium
ZFS is cool but I'm afraid that the concept of pools is a bit complicated for the average consumer. How to manage backups ? What happens when a drive fails ? Time Machine has to be intelligent enough that it doesn't place data and backup on the same physical drive. Otherwise you could lose your original data AND the backup.

But, if it's thought out well then I'm all for it.
 

wmmk

macrumors 68020
Mar 28, 2006
2,414
0
The Library.
am i getting this wrong, or could I format my 100GB HD to ZFS and have it store as much data as a 1TB+ HFS drive? If so, then wow. Simply wow.
 

shawnce

macrumors 65816
Jun 1, 2004
1,442
0
Did you read this?
http://opensolaris.org/os/community/zfs/whatis/

1) All operations are "copy on write".
data are never over written you can always go back to last month's version . The system saves space by only writing changes.
Actually you are mixing up "copy on write" with snapshotting capabilities (however the former does help with the later). The COW aspect is talking about how file blocks are managed when they are changed including "file" blocks related to the file system itself. As a result the data on disk is always coherent which avoids the need for journaling and fsck.
 

longofest

Editor emeritus
Jul 10, 2003
2,924
1,682
Falls Church, VA
Please note that as of this time, developer builds of Time Machine are using HFS+ formatted disks, and ZFS is not used. While ZFS would certainly much-enhance Time Machine, this capability has yet to appear in the application.

am i getting this wrong, or could I format my 100GB HD to ZFS and have it store as much data as a 1TB+ HFS drive? If so, then wow. Simply wow.

Heh... While the compression algorithm may be good, you are talking about a factor of 10x + there, which means that you are going to see a noticable decrease in performance. Perhaps useful for archiving data, but not much else...
 

shawnce

macrumors 65816
Jun 1, 2004
1,442
0
ok but say i buy an external hard drive, and "pool" it with the internal hard drive. the external hard drive craps out and dies (like most hard drives do). is all my data then gone? or is some data still there from the HD that didnt fail
It depends. If you are mirroring data between the two drives then no. If you are using RAIDZ (likely not applicable in this simple example of yours) then likely no, your data would still exist and be accessible (has a storage and IO cost overhead to do this). If you are using simple concatenation (not sure of ZFS terminology) then a subset of your files (or file data) would disappear.
 

mkrishnan

Moderator emeritus
Jan 9, 2004
29,776
15
Grand Rapids, MI, USA
am i getting this wrong, or could I format my 100GB HD to ZFS and have it store as much data as a 1TB+ HFS drive? If so, then wow. Simply wow.

Erm... no, I don't think so. Not unless the situation were extraordinarily unuasual, like you had millions upon millions of teeny tiny files and basically no files of any significant size. Or you had inordinately compressible files...i.e. ones with basically junk in them.
 

killmoms

macrumors 68040
Jun 23, 2003
3,752
55
Durham, NC
am i getting this wrong, or could I format my 100GB HD to ZFS and have it store as much data as a 1TB+ HFS drive? If so, then wow. Simply wow.

Yeah, you're getting it very wrong. Some data can barely be compressed at all (since it's already compressed in other ways). What pools mean is that you can add storage to your pool almost limitlessly. Your drive still only stores 100GB on it. It's software, not sorcery. ;)
 

shawnce

macrumors 65816
Jun 1, 2004
1,442
0
What happens if you add a drive into a pool, and at a later date wish to remove it from the system? Can you just tell the computer to move all of the data off of the drive and then pull it out (after turning off the computer, of course, or not if it's SATA) with no harm or lost files?
Yes ZFS supports that type of operation. You can migrate data off of a storage device that you want to retire.
 

ChrisA

macrumors G5
Jan 5, 2006
12,561
1,671
Redondo Beach, California
This would be very exciting feature to have. The ability to have one single Macintosh HD and just keep adding drives (internal or external) to increase the space.

Sweeeet. :D

Yes it would be great. Every other UNIX system works like this already. So can the mac if you want to take the time to set it up. Put you may not want to use external drives (that might get unplugged) as art of your storage pool. Best to use drives that are screwed down for that.
 

shawnce

macrumors 65816
Jun 1, 2004
1,442
0
am i getting this wrong, or could I format my 100GB HD to ZFS and have it store as much data as a 1TB+ HFS drive?
No, the supported compression is not nearly that good (and already compress files for example are sufficiently random that it couldn't compress then further).
 

Westside guy

macrumors 603
Oct 15, 2003
6,339
4,156
The soggy side of the Pacific NW
ZFS is cool but I'm afraid that the concept of pools is a bit complicated for the average consumer. How to manage backups ? What happens when a drive fails ? Time Machine has to be intelligent enough that it doesn't place data and backup on the same physical drive. Otherwise you could lose your original data AND the backup.

But, if it's thought out well then I'm all for it.

Don't forget that ZFS was developed by Sun primarily as a next-generation filesystem for Solaris, their server OS. Some of its capabilities will not be useful to the average consumer; but that doesn't make it a negative. It's not like you HAVE to use multiple volumes.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.