File size on disk vs actual file size - strange behaviour on large APFS drive?

[R2K2]

Edit: doesn't seem tied to large drives, but an APFS-specific problem. See bottom.

----
So, I'm trying to create backup of folder on a server volume to an external 14TB APFS-formatted hard drive.
The folder reports on the server as being 12TB in size. However, when copying to the 14TB drive, it doesn't fit.
I compared the folders that where already copied, to the source folders on the server. The folders reported as being the same size, but the 'on disk' part for some folders was larger than on the server (for example, a folder on the server reported as '512.846.824 bytes (512,9 MB on disk)' but on the 14TB drive as 512.846.824 bytes (551,6 MB on disk))

I checked the files in this folder, and indeed, some files took way more disk space than their actual size. One psd file was 68.253.265 bytes (68,3 MB on disk) on the server, but its copy on the 14TB drive was 68.253.265 bytes (84,9 MB on disk). A difference of over 16MB!

I thought: maybe the block size on the 14TB drive is too large, so when a file is just a bit larger than an exact number of blocks, it takes a whole extra block for just a bit of data. Although this would mean something like a 16MB block size, which didn't seem plausible.
But just for testing, I created a single 1MB file with 'dd if=/dev/random of=testfile.bin bs=1m count=1'. If block size was the problem, it would report as 16MB instead of 1MB.
Inspected the file, and it reported as 1.048.576 bytes (1 MB on disk). So this seemed okay.

Next I tried increasing the size of the testfile by changing the count in the dd command.
Up until 65 this was ok. This file reported as 68.157.440 bytes (68,2 MB in disk).
At 66 it changed. Now the file suddenly was 69.206.016 bytes (84,9 MB on disk)

To check if it was an OS or APFS thing, I copied this file to the internal drive of this mac (alsof APFS formatted). There it correctly reported as being 69.206.016 bytes (69,2 MB on disk)

So...any idea what is happening here? Is there something wrong with APFS on large drives? Or is this 'by design'?

This is on a late 2014 Mac Mini running Catalina 10.15.7.
----
Edit: This just got weirder. I did some more tests on my own macbook (M1 Air running 12.6):
If I create a clean, new 66MB file with 'dd if=/dev/random of=testfile.bin bs=1m count=66' on my internal ssd, it shows correctly in Finder info as '69.206.016 bytes (69,2 MB on disk)'.
If I overwrite this file with the same command ''dd if=/dev/random of=testfile.bin bs=1m count=66', it incorrectly shows as '69.206.016 bytes (84,9 MB on disk)'.
So it seems that for new, clean files everything is ok, but I you overwrite or append to a file, it's not.
Even weirder: when I duplicate this file, the duplicate shows correctly using 69,2MB on disk.

Does anyone here have an explanation for this behaviour? Or is able to confirm this?

 

[chabig]

It's probably related to the way APFS tracks changes. When you change a file, APFS keeps the original file and tracks the changes. This increases the space used.

 

[R2K2]

I thought about that as well, but wouldn't the file double in size then? Since I'm using /dev/random, and overwriting again from /dev/random, wouldn't that create byte-for-byte different files?

And if it is indeed some way of tracking changes, would there be a way to discard these? Thus shrinking files to their 'real' size and freeing up disk space?

 

[chabig]

This is pretty complicated and I don't know exactly how it works. I was just speculating. You might be able to reclaim some space by deleting snapshots. Read this...

Where did all that free space go on my APFS disk?

The volume should only be half full, but installers and other software complains there’s insufficient free space. What to do.

eclecticlight.co

 

[R2K2]

Yes, thought about this as well, but there are no snapshots. I think it's something in the way APFS stores files, but it don't understand what.

Steps to replicate are fairly easy:
- Open terminal and create a file in your homedir with dd: 'dd if=/dev/random of=testfile.bin bs=1m count=66'
- Locate this file in Finder and open Get Info on the file. It wil show as 69.206.016 bytes (69,2 MB on disk). Leave the window open.
- Run the same command in terminal again, thus overwriting the existing file with a file of exactly the same size.
- Return to the Get Info-window. You will see that the file has changed to being 69.206.016 bytes (84,9 MB on disk).
So..exactly the same number of bytes, but taking ~16MB more on disk.

(and duplicating this file does some random strange things to the on-disk size of both the original and duplicated file)

Does anyone here have some deeper APFS-knowledge to explain this?

 

[chown33]

Please try the command ls -l@ ~/testfile.bin then post the complete output from before and after the overwrite.

Also try it with dd reading from /dev/zero instead of /dev/random.

(and duplicating this file does some random strange things to the on-disk size of both the original and duplicated file)

Please describe the random strange things, or at least post numbers.

AFAIK, duplicating a file will make a copy-on-write duplicate (aka a clone). This means the new replica will share data with the original, and no copying actually occurs until a write occurs on one of the files.

How can you tell whether a file has been ‘cloned’ in APFS?

APFS can ‘clone’ files when copying or duplicating them within the same volume. But how can you tell whether any given file is a clone?

eclecticlight.co

Copy, move and clone files in APFS, a primer

What happens when you copy a large file to the same volume, or duplicate it, on the same APFS volume? Here are the answers.

eclecticlight.co

Search terms:
apfs copy site:eclecticlight.co

 

[R2K2]

Tried that, the file doesn't have any extended attributes.

But I did find a kind of solution: the files return to their 'normal' on-disk size after unmounting and remounting the drive. So it almost seems like some kind of cached write or clean-up that only happens when mounting or unmounting a drive.

This is what I did:
Formatted a 1TB usb hard drive as APFS
Created a single file 'dd if=/dev/random of=testfile.bin bs=1m count=66', and overwrote this file again with the same command.
After being overwritten it showed as 69.206.016 bytes (84,9 MB on disk) in Finder info.
Then I unmounted the drive, disconnected it and then reconnected.
Now the file showed the correct size in Finder info. Corresponding disk space (checked with 'df') was also freed.

So, still not sure what's happening here, seems an APFS thing (doesn't happen with HFS+ formatted drives).

 

[mfram]

Can't duplicate this result. I'm on Intel Ventura 13.0.1. Tried on both internal drive (Download folder) and an external APFS volume. I used the command:

dd if=/dev/random of=testfile.dat bs=1m count=10

In either case after overwriting the same file multiple times Finder reports "10,485,760 bytes (10.5 MB on disk)". Every time I overwrite the file I get the same result on Finder. This is on both the internal and external volumes.

Must be some other case. Or maybe there was a TM backup going on at the time?

 

[R2K2]

In either case after overwriting the same file multiple times Finder reports "10,485,760 bytes (10.5 MB on disk)". Every time I overwrite the file I get the same result on Finder. This is on both the internal and external volumes.

With smaller files there is no problem. The critical point for me was with a 66 MB file (count=66 in dd).

 

[mfram]

Yes, I was able to reproduce this with 66MB. My guess is that it has to do with pre-allocated block sizes. I did a couple other tests and the sizes just over a power of 2 seemed to have the biggest disparity. For example, 176MB didn't have a very big difference between on-disk and actual size. But 131 (just over 128) had a pretty big difference. So perhaps it allocated something to fit the 128MB and then preallocated some space for the data after 128MB in a bigger block. If you think about it, pre-allocating space on a file that's getting over-written makes some sense. So you don't keep fragmenting the file as it gets over-written (and possibly gets larger).