ssd swap - high usage of Terabytes Written

[Chuckeee]

A key point is to understand the failure mechanisms associated with SSD (NAND) failures. Specifically, the wear leveling, a low level NAND hardware process, will identify and isolate failed NAND memory sites. Up to a point this lost of viable memory (almost always an inability to write or address a unique memory location) is transparent to the user since the lost is absorbed within the wear leveling. Eventually it will become noticeable as a slow growing degradation to the SSD capacity. The effect (not the cause) is similar to how mobile phone batteries fail, the availability maximum capacity gets smaller and smaller. This is different from typical HDD failure that ultimately results in a critical failure (the entire device become inoperable). A result is a failed SSD declaration is somewhat arbitrary, 1%?, 5%?, 10%?, loss etc.

While there are other SSD failure mechanisms, this slow capacity degradation is the most common. Both usage (how much writing) and time (age of hardware) contribute to this failure. Environmental effects (e.g. temperature) also impact longevity.

 

[theorist9]

As mentioned, by the time FV2 rolled out, on sufficiently new Intel Macs, you could turn it on with nearly zero performance loss. The Secure Enclave approach has even less overhead than that, but I remember enabling FV2 on a 2011 MacBook Air and not noticing any difference.

I've had a different experience. I tested my 2019 i9 iMac*, which does not have Secure Enclave (meaning, as you know, that the encryption is done via software instead of dedicated hardware), both with and without FV2, and it felt more sluggish with FV2 on. That's consistent with this synthetic test of the internal SSD, which shows a significant decrease in performance with FV2 for 4k random writes, and high-QD 4k random reads.

*I believe this is the fastest Mac (for SC performance) Apple made that doesn't have Secure Enclave, so if any Mac is going to be able to overcome the performance penalty from FV2 software encryption, this would be it. Yet it doesn't. [OK, granted, what matters is actually encryption speed relative to storage I/O speed, so while it does have a fast processor, it also has a fast SSD.]

Notes: Testing done with MacOS Monterey v 12.4. The difference in sequential speeds is not significant (about within normal variation). Tests done with an aftermarket SSD (WD SN850). Apple stock SSD shown for comparison.

[Please ignore the "FireVault" typo. In thinking about security, I think my brain was mixing FileVault with firewall.]

 

[gank41]

Whelp, looking over my drive's stats again and comparing to what I was seeing years ago, this M1 MBP will last quite a long time... What a great machine.

Also of note, I had roughly 120some Unsafe Shutdowns while on Big Sur. Some of those betas on the M1 machines were pretty brutal. Lots of crashes. Knowing there's really only been roughly 30-40 in the years since is good news, too.

This is from the initial rollout of the late 2020 M1 13 inch MBP with 16GB/2TB

 

[gank41]

Finally got smartmontools working and it’s showing 22.7 TBW since I got my M1 MBP 16GB/2TB back in mid-November. Also shows 25 unsafe shutdowns! That’s probably the kernel panics I’m getting that a lot of people have reported when rebooting the M1 MBA or MBP in clamshell mode.

View attachment 1734066

Looking over from when I posted this above in Feb 2021 thru to now and this thing just continues to amaze me. Bar for some random macOS bugs here and there over time, The M1 MBP is by far and away the best computer I've ever used in my life. And I've also been using an Intel Mac mini with 64GB of Memory, along with a Windows computer with 64GB of Memory for work...

The post above was after roughly 3 months of heavy use. I do a lot of recording with Logic Pro along with all the various normal things people do with their computers. Most of the "Unsafe Shutdowns" occurred during the initial Big Sur betas where there was a lot of issues. The "Power On Hours" of 10,754 equates to roughly 1.23 continuous years of use, only power cycling it 213 times with a spare threshold of 99% still! As long as nothing else, hardware-wise, happens with this thing, it should bring in many more years of good use.

 

[shimpster]

So my I re-ran on my 2tb drive on my m1 Mac today and this is what I got. I have gone from 1% to 3% with 91.8tb written

 

[OxCxDx]

For anyone interested, smartmontools info for a refurbished MBP M1 Max, 64GB, 4TB:

• Read: 598 GB
• Write: 484 GB
• Host read commands: 23,517,537
• Host Write Commands: 13,000,335
• Power Cycles: 315
• Power on Hours: 10
• Unsafe Shutdowns: 11

Also, Battery Power cycles: 8

So my I re-ran on my 2tb drive on my m1 Mac today and this is what I got. I have gone from 1% to 3% with 91.8tb written

Interesting to compare the power cycle, 149 difference.

 

[BENCHPRESS]

For those with lots of data being written to disk.. do you use multiple monitors? I wonder if my dual external monitors somehow causes macOS to write more to disk.

My machine consistently writes around 60GB a day and I don't do anything that should warrant that much data to be written. All I do is read my news articles and watch my youtube videos. I don't even watch my videos at 4k!

I can't for the life of me figure out why there is so much data being written but I think I will give up on this fight.

 

[theorist9]

For those with lots of data being written to disk.. do you use multiple monitors? I wonder if my dual external monitors somehow causes macOS to write more to disk.

My machine consistently writes around 60GB a day and I don't do anything that should warrant that much data to be written. All I do is read my news articles and watch my youtube videos. I don't even watch my videos at 4k!

I can't for the life of me figure out why there is so much data being written but I think I will give up on this fight.

A) If you're curious, why not do an experiment for a day -- disconnect the externals and see if your writes change?

B) 60 GB/day is insignificant relative to the life of these SSD's. I have 150 GB writes/day (yes, I'm running two externals, but I have no idea if that contributes). But even you had double that (300 GB/day), and if your drive's lifetime were, say, 3000 TBW, that's 3000/.3/365 = 27 years.

 

[Knut Olsen]

For those with lots of data being written to disk.. do you use multiple monitors? I wonder if my dual external monitors somehow causes macOS to write more to disk.

I do not. This machine never had an external monitor connected to it.

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)

Critical Warning:                   0x00
Temperature:                        32 Celsius
Available Spare:                    100%
Available Spare Threshold:          99%
Percentage Used:                    37%
Data Units Read:                    2,521,062,826 [1.29 PB]
Data Units Written:                 2,289,600,448 [1.17 PB]
Host Read Commands:                 12,498,000,835
Host Write Commands:                7,023,130,600
Controller Busy Time:               0
Power Cycles:                       317
Power On Hours:                     5,433
Unsafe Shutdowns:                   50
Media and Data Integrity Errors:    0
Error Information Log Entries:      0


Read 1 entries from Error Information Log failed: GetLogPage failed: system=0x38, sub=0x0, code=745

 

[ChrisA]

All I do is read my news articles and watch my youtube videos.

One video frame is say 1000 by 1500 pixels, each pixel is three bytes. So we have 4.5 megabytes per frame. At 30 frames per second, that means. 800 megabytes per minute of data displayed on your screen. that's 0.8 gigabytes per minute. Don't think watching youtube is "doing nothing". That said, I wouldn't worry about it.

 

[mr_roboto]

One video frame is say 1000 by 1500 pixels, each pixel is three bytes. So we have 4.5 megabytes per frame. At 30 frames per second, that means. 800 megabytes per minute of data displayed on your screen. that's 0.8 gigabytes per minute. Don't think watching youtube is "doing nothing". That said, I wouldn't worry about it.

I agree with your conclusion but the numbers you used to arrive at it are simultaneously less and more pessimistic than they should be.

On the less side, there's lots of 4K or higher monitors out there, and macOS typically uses 32-bit (4-byte) color depth in frame buffers, not 24. The frame buffer for the built-in display of a 16" MBP is almost 30 megabytes.

On the too pessimistic side, displays are repeatedly refreshed from only two or three frame buffers (double or triple buffering). The system does not allocate a new buffer for every frame sent to the monitor. Thus, the memory usage due to adding a display is small and static, it doesn't grow over time with every frame sent to the display.

So, @BENCHPRESS , plugging in an external monitor does allocate a little bit of extra memory, but it's almost always less than 100MB. It does have to be wired memory (meaning, not allowed to be swapped out). That can increase memory pressure, but if 100MB is enough to push things from not swapping much to swapping a lot, you're running on the edge and you could choose almost anything as the last straw, rather than blaming it on the external monitor.

If you're mostly just watching youtube videos and reading news articles, I am going to guess a couple things. You tell me if they're true.

1. You don't have a lot of RAM in that Mac
2. Your primary browser is Chrome

Chrome is a known memory hog, and memory hog programs often cause excessive swapping on low-RAM computers. If that's what's happening, well, there's your problem. If my guesses are true and you don't have something forcing you to be on Chrome, you could try switching to Safari, which is much better at using memory efficiently.

 

[BENCHPRESS]

@ChrisA I overlooked all of that 🤦‍♂️

mr_roboto​

I primary watch my videos on 1080p, I use my machine on average about 13 hours a day. I have 32GB RAM and my primary browser is indeed Chrome 🙁.

 

[eldho]

@ChrisA I overlooked all of that 🤦‍♂️

mr_roboto​

I primary watch my videos on 1080p, I use my machine on average about 13 hours a day. I have 32GB RAM and my primary browser is indeed Chrome 🙁.

You can also check on your Activity Monitor whether you indeed have a lot of memory swapping occurring -(click on the memory column and look down at the bottom of the stats - in case you are new at this)

 

[mr_roboto]

I primary watch my videos on 1080p, I use my machine on average about 13 hours a day. I have 32GB RAM and my primary browser is indeed Chrome 🙁.

Google might have fixed it, but back in 2020 when people were still figuring out high write counts on Macs, the youtube channel Constant Geekery documented that if you used Mac Chrome to watch Youtube videos, Chrome would write the entire video stream to its content cache (files on your SSD).

It's pretty easy to check whether this is still happening. Using any of the tools which report the number of bytes written to the SSD, check before and after playing a long YT video in Chrome, preferably while not much else is going on on your computer. Try setting it to 4K to maybe make the test more obvious, since 4K videos are so much larger than 1080p.

 

[thebart]

I do not. This machine never had an external monitor connected to it.

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)

Critical Warning: 0x00
Temperature: 32 Celsius
Available Spare: 100%
Available Spare Threshold: 99%
Percentage Used: 37%
Data Units Read: 2,521,062,826 [1.29 PB]
Data Units Written: 2,289,600,448 [1.17 PB]
Host Read Commands: 12,498,000,835
Host Write Commands: 7,023,130,600
Controller Busy Time: 0
Power Cycles: 317
Power On Hours: 5,433
Unsafe Shutdowns: 50
Media and Data Integrity Errors: 0
Error Information Log Entries: 0


Read 1 entries from Error Information Log failed: GetLogPage failed: system=0x38, sub=0x0, code=745

What's your drive capacity?

 

[Beacons.things]

I’m clearly coming to this late, but I have a 2 month old MBA M1 with 18TBW. I only use this machine for lite tasks including syncing Onedrive (which for Sonoma I believe now uses Apple cloud services). I have just over 100GB stored in Onedrive. However, I am convinced that Apple’s cloud services for me are what is doing the excessive writing. For example, I can add a 2MB (or smaller) file to Onedrive and it will write 10s of GB to the drive. When setting up the machine, the initial 100GB sync with Onedtive wrote over 3TB alone!! When I turn off cloud services and syncing, writes to the SSD essentially fall to zero. So for me that’s what’s causing it, I’m just not sure why it’s so much higher than our Intel Macs…

 

[eldho]

I can add a 2MB (or smaller) file to Onedrive and it will write 10s of GB to the drive

That is certainly very extreme. You might be "late" to this thread but you might also have uncovered an extremely relevant new insight re the issue of this thread.

 

[theorist9]

I’m clearly coming to this late, but I have a 2 month old MBA M1 with 18TBW. I only use this machine for lite tasks including syncing Onedrive (which for Sonoma I believe now uses Apple cloud services). I have just over 100GB stored in Onedrive. However, I am convinced that Apple’s cloud services for me are what is doing the excessive writing.

Since OneDrive is a MS product, how do you know the problem is due to Apple's cloud services rather than OneDrive (or some unfortunate interaction between the two on your machine)?

To check this, I'd suggesting turning OneDrive off, and checking the writes for a couple of days where you are instead syncing your docs using iCloud Drive.

 

[trip1ex]

M1 Mac mini still going strong. ...hasn't blown up yet.

 

[Beacons.things]

Since OneDrive is a MS product, how do you know the problem is due to Apple's cloud services rather than OneDrive (or some unfortunate interaction between the two on your machine)?

To check this, I'd suggesting turning OneDrive off, and checking the writes for a couple of days where you are instead syncing your docs using iCloud Drive.

The reason I’m ‘assuming’ it is more Apple rather than MS is because within Activity Monitor the writes recorded by Onedrive are minuscule, whereas the overwhelming majority of writes to the SSD are attributed to ‘fileproviderd’ which I believe is an Apple process involved in managing cloud services or by the system itself…

I also have another conundrum which perhaps someone here can help me with - typically we see 512GB SSDs as per my machine) being described as warrantied/reliable (or within spec) for 300TBW or roughly 600 writes per cell (and I fully understand that this does not mean that they will die instantly at this point - rather manufacturers are guaranteeing that they will remain JEDEC compliant up to this point). But what I don’t understand is that when I have scrutinised varies peoples’ screenshots of their Smartctl outputs for their M silicon machines and I look at Data Units Written vs Percentage Used, my maths calculates that the ‘safe’ or warrantied period is routinely an order of roughly 10 times higher? Is this an issue with my maths/understanding; a Smartctl bug or actually the internal spec of the drive reporting to Smartctl that it is indeed operating within spec to a level far higher than is broadly described/written about online for SSDs???

 

[Mike Boreham]

The reason I’m ‘assuming’ it is more Apple rather than MS is because within Activity Monitor the writes recorded by Onedrive are minuscule, whereas the overwhelming majority of writes to the SSD are attributed to ‘fileproviderd’ which I believe is an Apple process involved in managing cloud services or by the system itself…

I also have another conundrum which perhaps someone here can help me with - typically we see 512GB SSDs as per my machine) being described as warrantied/reliable (or within spec) for 300TBW or roughly 600 writes per cell (and I fully understand that this does not mean that they will die instantly at this point - rather manufacturers are guaranteeing that they will remain JEDEC compliant up to this point). But what I don’t understand is that when I have scrutinised varies peoples’ screenshots of their Smartctl outputs for their M silicon machines and I look at Data Units Written vs Percentage Used, my maths calculates that the ‘safe’ or warrantied period is routinely an order of roughly 10 times higher? Is this an issue with my maths/understanding; a Smartctl bug or actually the internal spec of the drive reporting to Smartctl that it is indeed operating within spec to a level far higher than is broadly described/written about online for SSDs???

Read what @leman said in this post...replying the report two posts above his.

 

[Beacons.things]

Read what @leman said in this post...replying the report two posts above his.

Thank you Mike - this is very interesting. If true, and Apple are using enterprise class SSDs (based on the Smartctl metrics), perhaps it is to make up for/offset their M silicon machines writing far more to SSD in typical use-case scenarios on a day-to-day basis???

 

[theorist9]

The reason I’m ‘assuming’ it is more Apple rather than MS is because within Activity Monitor the writes recorded by Onedrive are minuscule, whereas the overwhelming majority of writes to the SSD are attributed to ‘fileproviderd’ which I believe is an Apple process involved in managing cloud services or by the system itself…

My view is that these things are complex, and rather than trying guess at what's going on, why not do the simple experiment I suggested to actually find out? Again: Turn OneDrive off, set up iCloud Drive, and check the writes for a couple of days while you are instead syncing your docs to iCloud Drive.

 

[NT1440]

My view is that these things are complex, and rather than trying guess at what's going on, why not do the simple experiment I suggested to actually find out? Again: Turn OneDrive off, set up iCloud Drive, and check the writes for a couple of days while you are instead syncing your docs to iCloud Drive.

fileproviderd is indeed the way Apple is allowing cloud file serves to run.

I don’t know if you guys remember a year or two back that Apple wiped out some of the ways cloud syncing applications worked when they disallowed kext file usage anymore.

From a quick search fileproviderd has always been a bit buggy, but it’s the way Apple dev documentation instructs cloud file syncing be implemented.

 

[theorist9]

fileproviderd is indeed the way Apple is allowing cloud file serves to run.

I don’t know if you guys remember a year or two back that Apple wiped out some of the ways cloud syncing applications worked when they disallowed kext file usage anymore.

From a quick search fileproviderd has always been a bit buggy, but it’s the way Apple dev documentation instructs cloud file syncing be implemented.

While that may all be true, I think it misses the big picture, which is what I was trying to focus on: If the reason the poster is seeing multiple TBW/month is really due purely to how MacOS syncs with cloud-based drives (as the poster believes), then anyone using cloud-based drives on Macs would be reporting that as well. But they don't.

The best way for the poster to begin tracking this issue down is to try switching from One Drive to iCloud and seeing if the issue persists. If it doesn't, it's due either to One Drive, or some bad interaction between One Drive and MacOS and the poster's specific config. If it does, then it's the poster's specific config, since we know this doesn't happen generally with MacOS and iCloud.