5,1 Slow AHCI M.2 Write Speeds

[JDLang76]

Booting to 4 SM951 in RAID 0 on PCIE 16 slot.
3 different speed tests have the Write starting at 1500MB/s but it drops to around 500 by the time the test is done.
Read speeds are consistently at 4000MB/s.

Anyone know whats up with the write speeds.

 

[tsialex]

Booting to 4 SM951 in RAID 0 on PCIE 16 slot.
3 different speed tests have the Write starting at 1500MB/s but it drops to around 500 by the time the test is done.
Read speeds are consistently at 4000MB/s.

Anyone know whats up with the write speeds.

Boot single user and run fsck -fy for each of your SM951-AHCI blades.

It's a know problem, SM951-AHCI don't do TRIM by himself correctly and you need to force it via fsck -fy from time to time.

 

[JDLang76]

Boot single user and run fsck -fy for each of your SM951-AHCI blades.

It's a know problem, SM951-AHCI don't do TRIM by himself correctly and you need to force it via fsck -fy from time to time.

Hm. When I ran it, it ran it for the whole RAID array and nothing changed.
I didnt get any option for doing individual blades.
Do I have to kill the RAID array first? Or am i missing something?

 

[tsialex]

Hm. When I ran it, it ran it for the whole RAID array and nothing changed.
I didnt get any option for doing individual blades.
Do I have to kill the RAID array first? Or am i missing something?

I don't know how you made your array, but you have to do for each blade individually and not the array. You will probably have to backup then destroy the array, format each blade, then do the fsck -fy when in single user mode to force the TRIM operation. After that, you will have normal write speeds.

 

[JDLang76]

So I scratched the raid array. Now I cant boot into single user mode. It goes to single usermode but before I can type anything, it switches to the recovery partition. I tried doing it in terminal from there, and it did it but only to the backup ssd I made. Is there a way to specify which disk I want top run it on?

 

[tsialex]

So I scratched the raid array. Now I cant boot into single user mode. It goes to single usermode but before I can type anything, it switches to the recovery partition. I tried doing it in terminal from there, and it did it but only to the backup ssd I made. Is there a way to specify which disk I want top run it on?

Recovery/Installer don’t work for this since Recovery/Installer automount everything and fsck -fy should only be run on unmounted filesystems.

 

[JDLang76]

Well I cant boot into single user mode anymore, and even if I could, there is NO option to select disk that I could see. Im looking at 3rd party trim enablers now. Thanks

Interesting., I just ran speed test on individual blades. 2 of them fast. 2 of them at 200MB/s. Weird

I also just discovered a terminal command - sudo trimforce enable - to enable trim on 3rd party ssds.

 

[tsialex]

Well I cant boot into single user mode anymore, and even if I could, there is NO option to select disk that I could see. Im looking at 3rd party trim enablers now. Thanks

Interesting., I just ran speed test on individual blades. 2 of them fast. 2 of them at 200MB/s. Weird

3rd party software will not resolve a firmware problem of SM951-AHCI. It's a lower level problem than the Apple drivers. Only the fsck -fy full TRIM operation done at check up time will resolve this. This has NOTHING to do with the trimforce command.

It's easy to revive the write speed. Install macOS to just one SM951-AHCI, boot from it with single user mode, run fsck -fy.

I have 1 512GB, 4 256GB and 1 128GB installed into various Mac Pros, and the only other option to revive SM951-AHCI write speed after totally full that I know of is the SATA secure erase.

 

[JDLang76]

3rd party software will not resolve a firmware problem of SM951-AHCI. It's a lower level problem than the Apple drivers. Only the fsck-fy full TRIM operation done at check up time will resolve this.

It's easy to revive the write speed. Install macOS to just one SM951-AHCI, boot from it with single user mode, run fsck -fy.

I used the sudo trimforce enable terminal command. All of the drives now testing well. The RAID array is also at normal write speeds. Looks like that was sufficient. Thanks for your help.

 

[tsialex]

I used the sudo trimforce enable terminal command. All of the drives now testing well. The RAID array is also at normal write speeds. Looks like that was sufficient. Thanks for your help.

The problem is that this will happen again sometime down the road, it's a firmware problem of various Samsung first generation PCIe M.2 drives.

The write speed decrease after full happens with:

• XP941,
• SM951-AHCI,
• 950Pro.

SM951-NVMe seems to not suffer or suffer less.

 

[JDLang76]

The problem is that this will happen again sometime down the road, it's a firmware problem of various Samsung first generation PCIe M.2 drives.

The write speed decrease after full happens with:

• XP941,
• SM951-AHCI,
• 950Pro.

SM951-NVMe seems to not suffer or suffer less.

But this trim command wasnt a one time thing. It keeps TRIM force-enabled until I disable it myself. So theoretically, that will prevent it, I would assume..

 

[tsialex]

But this trim command wasnt a one time thing. It keeps TRIM force-enabled until I disable it myself. So theoretically, that will prevent it, I would assume..

Seems you didn't understood the problem. TRIM operation and trimforce command are different things.

The trimforce command just tell to SSD that it can erase all the cells that the OS marked as erased, at the SSD firmware discretion. By itself it does absolutely nothing.

The TRIM operation is the SSD firmware operation that erases all cells marked for erase by the OS, this is done continuously in the background when the SSD is not in use. This takes time.

The write speed abrupt decrease is because the SSD has to erase the used cells before writing it again. TRIM when working correctly makes this unnecessary, since the cells are being always refreshed when the drive is not in use.

Samsung first generation PCIe M.2 drives have a firmware problem, probably related to garbage collection, that when the SSD is full (even if you erase it after), the TRIM operation stops to be done in the background and only is restarted with fsck -fy. This is the problem and why you see that from time to time write speed decrease from 1500MB/s to 200~250MB/s.

 

[JDLang76]

Seems you didn't understood the problem. TRIM operation and trimforce command are different things.

The trimforce command just tell to SSD that it can erase all the cells that the SO marked as erased, at the SSD firmware discretion. By itself it does absolutely nothing.

The TRIM operation is the SSD firmware operation that erases all cells marked for erase by the SO, this is done continuously in the background when the SSD is not in use. This takes time.

The write speed abrupt decrease is because the SSD has to erase the used cells before writing it again. TRIM when working correctly makes this unnecessary, since the cells are being always refreshed when the drive is not in use.

Samsung first generation PCIe M.2 drives have a firmware problem, probably related to garbage collection, that when the SSD is full (even if you erase it after), the TRIM operation stops to be done in the background and only is restarted with fsck -fy. This is the problem and why you see that from time to time write speed decrease from 1500MB/s to 200~250MB/s.

I was unaware of the difference. Thanks for the explanation. Since my write and read speeds are over 4500MB/s now, I’m gonna leave it until such a day where the performance goes down again. I need it up and running for a bit. Thanks again for all the help

 

[nico5.1]

Hello,

I think same problem here on a 2010 5.1 - 10.12.6 - dual 3,33 - 32gb ram - gtx980

I have 2 sm951 512 GB ahci on a Amfaltec squid blade since 2 years.. they are striped in raid 0 and it's my boot drive
At that time speed test (blackmagic) shows around 2500mb/s write and read speeds
I just ran the test 10 minutes ago : 220mb/s write and 1450mb/s read
Trim is enable since I bought them 2 years ago
I just tested the fsck command in single user and then rebooted... ran the speed test again... no real write speed improvement :-( but read speed is much better 🙂

Any ideas/solutions from the experts ? This is far beyond my knowledge

I love so much those machines 😉 want to keep it as long as possible

 

[tsialex]

Hello,

I think same problem here on a 2010 5.1 - 10.12.6 - dual 3,33 - 32gb ram - gtx980

I have 2 sm951 512 GB ahci on a Amfaltec squid blade since 2 years.. they are striped in raid 0 and it's my boot drive
At that time speed test (blackmagic) shows around 2500mb/s write and read speeds
I just ran the test 10 minutes ago : 220mb/s write and 1450mb/s read
Trim is enable since I bought them 2 years ago
I just tested the fsck command in single user and then rebooted... ran the speed test again... no real write speed improvement :-( but read speed is much better 🙂

Any ideas/solutions from the experts ? This is far beyond my knowledge

I love so much those machines 😉 want to keep it as long as possible

If no one have a better suggestion:

1. Backup your data,
2. Remove the RAID,
3. Install macOS for each blade and do the fsck -fy operation from single user for each blade.

Doing all this force the TRIM clean-up operation and the blades will have normal write speeds again. This has to be done from time to time.

 

[solaris8x86]

Booting to 4 SM951 in RAID 0 on PCIE 16 slot.
3 different speed tests have the Write starting at 1500MB/s but it drops to around 500 by the time the test is done.
Read speeds are consistently at 4000MB/s.

Anyone know whats up with the write speeds.

It's normal. I tried several brands SSDs in a POC including SAMSUNG 800 (by that time it was new.). The performance of SAMSUNG SSD drive was ok only at the beginning stage of benchmarking. After 10-20 seconds. It drops sharply (18,000Mbps --> 600Mbps only at the end of the test). So SAMSUNG was out of the game as an upgrade choice to my Mac Pro. OCZ has the same problem too. KINGSTON FuryX is a bit better but it still had a non-sustainable performance drop problem but it last longer.

The SANDISK Extreme Pro SSD (480GB or 980GB) is the only one without such problem in a huge data sustain read/write test. It could provide a sustain read/write speed at 20,000Mbps (4 SSDs in RAID-0, when the LUN was empty) in the whole stress test cycle. Flushing IN/OUT of 50GB data and it could always staying at the peak 20,000Mbps on symmetric read/write. Even now the LUN is almost 50% full. It could still have a sustain read/write peak speed at about 15,000Mbps symmetric speed when flushing IN/OUT of 50GB data. So I purchased 16 disks of SANDISK Extreme Pro SSDs for my RAID controller. Couldn't be happier by that time that I had a chance to do a POC before paying actual money. That also saved my troubleshooting time afterward.

Although I don't know how new is SM951. But I do believe it is a common characteristic of SAMSUNG SSD drives (NVMe or SSD doesn't matter) when working on an old Mac or these kind of drives. It must have somethings needed to be hacked or fine tuned to draw the performance in the OS. But no one knows now. If not, that being said. These drives are designed for session-based applications. Such as Office or Photoshop..etc which the application file size is small. Then they are ok.) Some people claim it is a "Ok" drive but they just make use of a small data traffic stream or their performance expectation is fairly non-demanding from a non-sustaining I/O application. So they said it is ok. But not friendly to people if you are telling me even the latest model of SAMSUNG drives remain having this "problem". May be it is not a "problem". It is just a design of the drive to handle smaller size files or daily desktop applications.

 

[tsialex]

It's normal. I tried several brands SSDs in a POC including SAMSUNG 800 (by that time it was new.). The performance of SAMSUNG SSD drive was ok only at the beginning stage of benchmarking. After 10-20 seconds. It drops sharply (18,000Mbps --> 600Mbps only at the end of the test). So SAMSUNG was out of the game as an upgrade choice to my Mac Pro. OCZ has the same problem too. KINGSTON FuryX is a bit better but it still had a non-sustainable performance drop problem but it last longer.

The SANDISK Extreme Pro SSD (480GB or 980GB) is the only one without such problem in a huge data sustain read/write test. It could provide a sustain read/write speed at 20,000Mbps (4 SSDs in RAID-0, when the LUN was empty) in the whole stress test cycle. Flushing IN/OUT of 50GB data and it could always staying at the peak 20,000Mbps on symmetric read/write. Even now the LUN is almost 50% full. It could still have a sustain read/write peak speed at about 15,000Mbps symmetric speed when flushing IN/OUT of 50GB data. So I purchased 16 disks of SANDISK Extreme Pro SSDs for my RAID controller. Couldn't be happier by that time that I had a chance to do a POC before paying actual money. That also saved my troubleshooting time afterward.

Although I don't know how new is SM951. But I do believe it is a common characteristic of SAMSUNG SSD drives (NVMe or SSD doesn't matter) when working on an old Mac or these kind of drives. It must have somethings needed to be hacked or fine tuned to draw the performance in the OS. But no one knows now. If not, that being said. These drives are designed for session-based applications. Such as Office or Photoshop..etc which the application file size is small. Then they are ok.) Some people claim it is a "Ok" drive but they just make use of a small data traffic stream or their performance expectation is fairly non-demanding from a non-sustaining I/O application. So they said it is ok. But not friendly to people if you are telling me even the latest model of SAMSUNG drives remain having this "problem". May be it is not a "problem". It is just a design of the drive to handle smaller size files or daily desktop applications.

This has absolutely nothing to do with sustained transfer rate/IOPs, it's a Samsung firmware problem with TRIM. Any SM951-AHCI makes a 3x sustained performance when comparing with a SanDisk Extreme Pro (a 2015 era drive).

I know what I'm talking and you don't, I have a SSD7101-A with 4 SM951-AHCI and another one alone, why do you post about things that you don't know?

 

[solaris8x86]

This has absolutely nothing to do with sustained transfer rate/IOPs, it's a Samsung firmware problem with TRIM.

I know what I'm talking and you don't, why do you post about things that you don't know?
View attachment 836000

Unfortunately, it was tried forcetrim on 10.12 and 10.13 (we have a few Mac Pros. Tried them all. Trim enabled/disabled, LUN created/deleted, HFS+, AFPS.., flesh OS installation.etc. Same result, no luck.)

By the time of 10.14. SAMSUNG has already been delisted from the vendor manifest of my company. They got no business from us thereafter. Poor company...

 

[crjackson2134]

This has absolutely nothing to do with sustained transfer rate/IOPs, it's a Samsung firmware problem with TRIM.

Absolutely correct. I've sold my SM951's AHCI long ago now, as these are OLD drives in terms of SSD/NVMe evolution. I too had to cause garbage collection to do it's job manually by booting into single user mode. It's not theory, it's a hard fact.

I've had no such issues after selling off the OLD SM951 SSD's and installing newer technology. Some people have very small usage and don't see this problem for a while. But it happens, and when it does, there is only forcing garbage collection to work. That is the in place fix. No other solution except replacement.

 

[tsialex]

Absolutely correct. I've sold my SM951's AHCI long ago now, as these are OLD drives in terms of SSD/NVMe evolution. I too had to cause garbage collection to do it's job manually by booting into single user mode. It's not theory, it's a hard fact.

I've had no such issues after selling off the OLD SM951 SSD's and installing newer technology. Some people have very small usage and don't see this problem for a while. But it happens, and when it does, there is only forcing garbage collection to work. That is the in place fix. No other solution except replacement.

Yes, for SM951, AHCI or NVMe, it's a complex problem and no one noticed that early on. It's a OEM drive.

Samsung don't provide firmware updates for OEM drives directly to end-users, even if the problem is already corrected for some ODM, we can't upgrade our blades firmware, unfortunately, since HP firmware can't be applied to Dell or Lenovo blades.

 

[dickiedunn]

Absolutely correct. I've sold my SM951's AHCI long ago now, as these are OLD drives in terms of SSD/NVMe evolution. I too had to cause garbage collection to do it's job manually by booting into single user mode. It's not theory, it's a hard fact.

I've had no such issues after selling off the OLD SM951 SSD's and installing newer technology. Some people have very small usage and don't see this problem for a while. But it happens, and when it does, there is only forcing garbage collection to work. That is the in place fix. No other solution except replacement.

Curious which drives you got/recommend after selling the SM951 AHCI. I'm also experiencing slow downs and am looking for more storage.

 

[crjackson2134]

Samsung 970 Pro NVMe

 

[dickiedunn]

Thanks. Can I ask a few last questions?
- what card are you using?
- what version of MacOS? I assume Mojave for NVMe?
- what kind of write/read speeds are you seeing?

I may just do the same thing you did and replace my 512GB SM951 AHCI

Thanks