How exactly does FileVault work?

[Ambrosia7177]

Can someone help me to better understand the technical details of how full-disk encryption (FDE) works with FileVault2 on a Mac?

This is my layman's understanding...

- You turn on FileVault.
- macOS provides a Recovery Key.
- macOS takes your login password and somehow creates an "encryption key". (Presumably it takes your login password, "salts" it, runs it through some fancy algorithm, and produces a hashed value which serves as your "encryption key".)
- macOS uses this "encryption key" to encrypt your entire hard-drive.
- macOS also stores this "encryption key" somewhere on your hard-drive.

- Later on when you start up your Mac, and log in, macOS takes your login password, runs it through the same process it used earlier (e.g. salting + some algorithm) and creates a hashed value.
- macOS then compares this hashed value against the stored "encryption key" on your hard-drive.
- If the hashed value matches yours "encyption key" then macOS unlocks (i.e. decrypts) your hard-drive.
- If the hashed value doesn't match, then you cannot access your hard-drive or the data on it.

Does this sound accurate?

(The description above is the standard way that you would code a "Log in" screen for an application/website...)

 

[chown33]

See here:
https://en.wikipedia.org/wiki/FileVault#FileVault_2


These are unnecessary, and would undermine security:

- macOS also stores this "encryption key" somewhere on your hard-drive.
...
- macOS then compares this hashed value against the stored "encryption key" on your hard-drive.
- If the hashed value matches yours "encyption key" then macOS unlocks (i.e. decrypts) your hard-drive.
- If the hashed value doesn't match, then you cannot access your hard-drive or the data on it.

It's not necessary to store the key, or to compare it.

Instead, the software can use the provided key to decrypt a known message. If the message doesn't decrypt correctly, then the key is incorrect.

The known message doesn't need to be revealed or stored anywhere in plaintext. Instead, a hash of it can be stored (in plaintext). Then the encrypted known message is decrypted and hashed. If the hashes match, the key is good.

The software can also rely on the fact that the header or other initial structural data of the disk volume is a "known message", in the sense that it has an expected structure and expected values. If that data is decrypted with the wrong key, then that structure will be incorrect.

An example in a non-disk-encryption case is a ZIP file, which starts and ends with specific identifiable data. If an encrypted zip file is decrypted with the wrong key, then the start and end are garbage.

 

[Ambrosia7177]

See here:
https://en.wikipedia.org/wiki/FileVault#FileVault_2


These are unnecessary, and would undermine security:


It's not necessary to store the key, or to compare it.

Instead, the software can use the provided key to decrypt a known message. If the message doesn't decrypt correctly, then the key is incorrect.

The known message doesn't need to be revealed or stored anywhere in plaintext. Instead, a hash of it can be stored (in plaintext). Then the encrypted known message is decrypted and hashed. If the hashes match, the key is good.

The software can also rely on the fact that the header or other initial structural data of the disk volume is a "known message", in the sense that it has an expected structure and expected values. If that data is decrypted with the wrong key, then that structure will be incorrect.

An example in a non-disk-encryption case is a ZIP file, which starts and ends with specific identifiable data. If an encrypted zip file is decrypted with the wrong key, then the start and end are garbage.

Sorry @chown33, you lost me... (That wikipedia page didn't help either.)

When I turn on FileVault2, what is macOS doing with my login password?

What is the sequence of events macOS takes to encrypt my drive?

And where does the "recovery key" come into play?

 

[NoBoMac]

The 101 level slide deck.

https://www.cl.cam.ac.uk/~osc22/docs/cl_fv2_presentation_2012.pdf

Slide 11 shows it all schematically.

In short:

• Encryption key generated
• Second encryption key generated to encrypt encryption key
• Second encryption key encrypted to user password(s) and recovery key
• Plist keychain created with the encrypted second key and hashed passwords/recovery key
• Plist encrypted with a different random key
• Random value saved to volume header

Reverse the process for login. Barebones OS is booted to prompt for passwords. If success, boots into real OS.

 

[chown33]

Can you describe how you discovered that slide deck?

I can find it when I search with author names, but that's cheating if I didn't know the authors.


I did find this sequence...

1. Google for: filevault analysis
2. One of the top results:

https://www.schneier.com/blog/archives/2012/08/an_analysis_of.html​

3. Which links to this summary:

https://www.lightbluetouchpaper.org/2012/08/06/analysis-of-filevault-2-apples-full-disk-encryption/​

4. Which links to this paper (PDF):

https://eprint.iacr.org/2012/374.pdf
Authors: Omar Choudary, Felix Gröbert, Joachim Metz
(Identical to slide deck's authors)
​

That paper is also Reference item [15] at the end of the Wikipedia FileVault article.

 

[NoBoMac]

"Filevault reverse engineering". Like the 5th hit on Google, a little deeper down on Duck Duck Go.

The PDF paper I read years ago, so, vaguely recalled where the authors were from, which awakened a memory when saw the slide deck.

Been using the slide deck link for a while now as it distills the paper down to something a lot less dense but still gets the basics across.

 

[Ambrosia7177]

The 101 level slide deck.

https://www.cl.cam.ac.uk/~osc22/docs/cl_fv2_presentation_2012.pdf

Slide 11 shows it all schematically.

In short:

• Encryption key generated
• Second encryption key generated to encrypt encryption key
• Second encryption key encrypted to user password(s) and recovery key
• Plist keychain created with the encrypted second key and hashed passwords/recovery key
• Plist encrypted with a different random key
• Random value saved to volume header

Reverse the process for login. Barebones OS is booted to prompt for passwords. If success, boots into real OS.

Thanks for the presentation link, but it assumes a person knows much more than I do, and it wasn't at all clear the sequence of events.

Isn't there some official Apple docs out there that explain the process of encrypting your hard-drive using FileVault2?

Or maybe something that Apple support techs might use?

 

[NoBoMac]

Unfortunately, it seems like Apple has no documentation on how FileVault works. I have never seen any, hence the links @chown33 and I provided.

 

[Ambrosia7177]

Unfortunately, it seems like Apple has no documentation on how FileVault works. I have never seen any, hence the links @chown33 and I provided.

Hmmm....

Well, thank you for posting the links above. I am reading and re-reading them trying to piece things together.

In the mean time, since you seem to know way mor ethan I do, could you please help me try and solve this puzzle?

Based on your knowledge and comments above...

When I log into my Macintosh, what is the first step that macOS/File Vault 2 take with my password?

@chown33, said of my guess, "These are unnecessary, and would undermine security" but after I looked at v3.0 of the Choudary, Grobert, Metz analysis, it does appear that "salting" of the login password occurs.

(Was at the mechanic when I stumbled across v3 of ths paper and now that I am back home I cannot find it?!)

 

[chown33]

@chown33, said of my guess, "These are unnecessary, and would undermine security" but after I looked at v3.0 of the Choudary, Grobert, Metz analysis, it does appear that "salting" of the login password occurs.

Only portions of what you wrote in your first post would undermine security. Specifically, the portions that I quoted in my post.

To address a specific point, which isn't in the portion I quoted, salting doesn't undermine security.

 

[NoBoMac]

When I log into my Macintosh, what is the first step that macOS/File Vault 2 take with my password?

As mentioned earlier, reverse of what I noted.

• Bare bone OS booted to prompt for password
• Keychain encryption key pulled from volume's header
• Encryption key de-crypts keychain
• Entered password is hashed and compared to value in keychain
• If match, password used to decrypt disk encryption key's encryption key
• Disk encryption key de-crypted via from above key
• Drive starts to de-crypt via disk encryption key, and system starts boot process

What one needs to worry about is if using a weak password (eg. "password") as someone with right tools can crack a FileVault drive. But in reality, unless one is in trouble with a three-letter agency, FV is highly secure for common issues (eg. common thievery). More likely to lose data via malware one might download on their device, or, via Capital One, Equifax, Target, IRS, DMV, doctor's office...

 

[SalisburySam]

I’ve never used FileVault but am thinking about doing so. What is the performance penalty, if any?

 

[Weaselboy]

I’ve never used FileVault but am thinking about doing so. What is the performance penalty, if any?

http://osxdaily.com/2011/08/10/filevault-2-benchmarks-disk-encryption-faster-mac-os-x-lion/

This is an older test, but still relevant. With newer Macs there is very little speed hit. Particularly with the newer T2 equipped models where the T2 chip handles encryption.

One of the first things I do on a new Mac is turn on FV and I have never had any issue with it.

 

[SalisburySam]

http://osxdaily.com/2011/08/10/filevault-2-benchmarks-disk-encryption-faster-mac-os-x-lion/

This is an older test, but still relevant. With newer Macs there is very little speed hit. Particularly with the newer T2 equipped models where the T2 chip handles encryption.

One of the first things I do on a new Mac is turn on FV and I have never had any issue with it.

Thanks for the vote of confidence for FV...sounds like a winner to me.

 

[chown33]

Even without T2, performance depends on the hardware. Many newer Intel CPUs have an AES instruction set, which speeds up the AES encryption used by FileVault.

Here's a list of the relevant Intel CPUs, by family name:
https://en.wikipedia.org/wiki/AES_instruction_set#Intel

It may be difficult to convert that name, e.g. Haswell, into the specific CPU in one's machine. It would be simpler to post the specific Mac model, so one can look up the CPU details.

 

[Weaselboy]

Many newer Intel CPUs have an AES instruction set, which speeds up the AES encryption used by FileVault.

Yep... I think anything Core i3 and up from 2010 on will have AES.

 

[wardie]

Sorry @chown33, you lost me... (That wikipedia page didn't help either.)

When I turn on FileVault2, what is macOS doing with my login password?

What is the sequence of events macOS takes to encrypt my drive?

And where does the "recovery key" come into play?

@Texas_Toast - what are you trying to find out with your question?

In the absence of official Apple documentation then the Wikipedia page plus the researcher’s presentation clearly lay out what is known of the architecture for this. Either the user’s password or the (randomly generated at FileVault-encryption-time) recovery key can be used to derive the master key for unlocking the drive. Page 11 of those slides shows this and the sequence of events. I’m not sure any other answer anyone can give will actually help you. 99.9% of people won’t understand this nor worry about how it works. Have you got an issue that needs fixing?

 

[chown33]

The PDF paper linked above (the 374.pdf link) describes what happens starting on page 4. Depending on one's experience, it might be wise to at least skim the sections before that.

Here's an excerpt, copied and pasted from the start of the relevant section on p. 4:

3 FileVault 2 architecture
3.1 Enabling FileVault 2

In Mac OS X Lion, after FileVault 2 is enabled, a series of events take place. First of all the user is presented with a 24 character recovery password (see figure 2). This password can be used to access the encrypted volume even if the user password is lost. We comment later on the security of this recovery method.

Next, the file system in the main volume is con- verted from the native HFSPlus type to CoreStorage (en- crypted). During this operation, the user can still use the system at will and the ConversionStatus field in the EncryptedRoot.plist file (details below) contains the string Converting. After the encryption process is com- plete the string changes to Complete.​

 

[Ambrosia7177]

@Texas_Toast - what are you trying to find out with your question?

In general, because I am so paranoid about people stealing my data, I figure I should understand how FileVault 2 does/does not protect my data...

In addition, having a better understand of how FV2 and "full-disk encryption" works would be useful for some other things I want to accomplish.

In the absence of official Apple documentation then the Wikipedia page plus the researcher’s presentation clearly lay out what is known of the architecture for this. Either the user’s password or the (randomly generated at FileVault-encryption-time) recovery key can be used to derive the master key for unlocking the drive. Page 11 of those slides shows this and the sequence of events. I’m not sure any other answer anyone can give will actually help you. 99.9% of people won’t understand this nor worry about how it works. Have you got an issue that needs fixing?

Fwiw, especially to @NoBoMac and @chown33, I have spent the last week intensively reading all of the links above plus looking things up online, watching videos on YouTube, and refreshing my memory of a lot of math concepts!

(Encryption, symmetric-key, assymetric-key, Diffie-Helman, Cryptography, cryptosystems, AES/Rijndael, block-ciphers, modes-of-operation, initialization vector, AES, ECB, CBC, CFB, OFB, CTR, AES-XTS, Tweakable block ciphers, Galois fields, polynomials, XOR, and much more!)

While the links above help somewhat, the truth of the matter is that they aren't written very well from a comprehension standpoint. (Maybe the author's first language isn't English?) My experience is that while engineers are great at solving problems, most suck at explaining things...

It looks like I won't be able to fully master FileVault2 processes, FDE, or cryptography in a week...
[doublepost=1565579129][/doublepost]

The PDF paper linked above (the 374.pdf link) describes what happens starting on page 4.

I just finished carefully reading and re-reading this entire paper...
Infiltrate the Vault: Security Analysis and Decryption of Lion Full Disk
Encryption

In the end, the author would *NOT* get an "A" in an English composition class...

 

[chown33]

Fwiw, especially to @NoBoMac and @chown33, I have spent the last week intensively reading all of the links above plus looking things up online, watching videos on YouTube, and refreshing my memory of a lot of math concepts!

(Encryption, symmetric-key, assymetric-key, Diffie-Helman, Cryptography, cryptosystems, AES/Rijndael, block-ciphers, modes-of-operation, initialization vector, AES, ECB, CBC, CFB, OFB, CTR, AES-XTS, Tweakable block ciphers, Galois fields, polynomials, XOR, and much more!)

While the links above help somewhat, the truth of the matter is that they aren't written very well from a comprehension standpoint. (Maybe the author's first language isn't English?) My experience is that while engineers are great at solving problems, most suck at explaining things...

It could be you're just reading the wrong ones. You didn't list any of the things you actually read for the subjects you listed, so there's no way for anyone else to evaluate them, or to say anything useful about them.

In my experience, Bruce Schneier writes well, and his "Applied Cryptography" is quite readable, although somewhat outdated. It's probably worth looking for as a used book. Another cryptographer who writes well is Matt Blaze, and I've read several things by him. It's been awhile since I read much about cryptography, so there are probably more I'd consider readable, but no other names spring to mind.

And honestly, if you're reading all that stuff in only a week, expecting to fully understand disk encryption, then your expectations are almost certainly misplaced. One can paddle a boat across Puget Sound, but I wouldn't recommend it as the first thing to do in your new kayak.

 

[Ambrosia7177]

It could be you're just reading the wrong ones. You didn't list any of the things you actually read for the subjects you listed, so there's no way for anyone else to evaluate them, or to say anything useful about them.

In my experience, Bruce Schneier writes well, and his "Applied Cryptography" is quite readable, although somewhat outdated. It's probably worth looking for as a used book. Another cryptographer who writes well is Matt Blaze, and I've read several things by him. It's been awhile since I read much about cryptography, so there are probably more I'd consider readable, but no other names spring to mind.

@chown33, my frustration was with the authors of these documents:

- "Security Analysis and Decryption of FileVault 2" (Omar Choudary, Felix Grobert, Joachim Metz)

- "Infiltrate the Vault: Security Analysis and Decryption of Lion Full Disk
Encryption" (Omar Choudary, Felix Grobert, Joachim Metz)

While I did learn some reading these two documents, the poor writing made things much more difficult than need be.

Some of my pet peves includes things like:

• Why does the paper not even have a date on it?
• Too much emphasis on "decrypting" without first explaining the "encryption" process. (Things are complicated enough it is foolish to say, "Just reverse them!")
• Talking about decryption in one sentence and immediately flipping to encryption in the next sentence.
• Grammatical errors
• Over reliance on the pronoun "it". WTF? When you are talking about 10 moving parts, "it" is a poor choice?!
• Not doing a good job tying the FileVault 2 encryption process with the AES-XTS process.

- And on and on...


*************
FWIW, the articles and videos I studied in just encryption were very educational, including:

- You Don't Want XTS (This guy is a *genius*!!)

- Lecture 8: Advanced Encryption Standard (AES) by Christof Paar (Love his accent, and the whole lecture series looks great!)

There were lots more, but I really liked the ones above.

And honestly, if you're reading all that stuff in only a week, expecting to fully understand disk encryption, then your expectations are almost certainly misplaced. One can paddle a boat across Puget Sound, but I wouldn't recommend it as the first thing to do in your new kayak.

I had hoped to fully understand how FileVault2 works in a weekend - that didn't happen.

And after spending the last 10 days reading about Cryptography, I am humbled to say the least!

Since protecting my data is so important to me, I plan to study more about Cryptography in my free time, but can see it will be a long road!

And as far as FileVault2 goes, I do have follow-up questions to the slide-show and paper from (Omar Choudary, Felix Grobert, Joachim Metz) if y'all are willing to entertain them here...

 

[NoBoMac]

And honestly, if you're reading all that stuff in only a week, expecting to fully understand disk encryption, then your expectations are almost certainly misplaced.

This.

Encryption and the ilk is the stuff that people use for thesis work on PHDs, MSs, etc. Focusing on the details of encryption is the wrong thing to be looking at, as Apple uses standard/recommended/approved algorithms. The encryption "just works". One needs to be concerned with how the keys are handled. And that becomes a whole new tangent (eg. how/where does Apple store the encryption key in RAM once the drive is decrypted? (guess: at least with T1/T2 Macs, probably in the T1/T2 chips, as that's how iOS works with basically same chip).

 

[chown33]

@chown33, my frustration was with the authors of these documents:

- "Security Analysis and Decryption of FileVault 2" (Omar Choudary, Felix Grobert, Joachim Metz)

- "Infiltrate the Vault: Security Analysis and Decryption of Lion Full Disk
Encryption" (Omar Choudary, Felix Grobert, Joachim Metz)

While I did learn some reading these two documents, the poor writing made things much more difficult than need be.

Here's what you wrote in post #19

I just finished carefully reading and re-reading this entire paper...
Infiltrate the Vault: Security Analysis and Decryption of Lion Full Disk
Encryption

In the end, the author would get an "A" in an English composition class...

So the same paper ("Infiltrate the Vault:...") would both "get an A" and contains "poor writing".


If you have follow-up questions, you should just ask them. If anyone wants to tackle them, they'll reply. If not, then there won't be replies. It seems much more practical to me to just ask the question, rather than not asking it.

 

[Ambrosia7177]

Here's what you wrote in post #19

So the same paper ("Infiltrate the Vault:...") would both "get an A" and contains "poor writing".

Oops! Thanks for catching that type-o from last night! (Have corrected the above)

What my brain was saying, but fingers seemed to miss was...

In the end, the author would *NOT* get an "A" in an English composition class...

 

[Ambrosia7177]

This.

Encryption and the ilk is the stuff that people use for thesis work on PHDs, MSs, etc. Focusing on the details of encryption is the wrong thing to be looking at, as Apple uses standard/recommended/approved algorithms. The encryption "just works".

I don't disagree, however I took a deep dive into cryptography this past week because I thought understanding the bascs would help me better understand how Filevault2 works. In this case, that wasn't necessarilu so.

One needs to be concerned with how the keys are handled. And that becomes a whole new tangent (eg. how/where does Apple store the encryption key in RAM once the drive is decrypted? (guess: at least with T1/T2 Macs, probably in the T1/T2 chips, as that's how iOS works with basically same chip).

I agree, and that is one of the main reasons I started this thread.