USB drives slow when you fill them up

[Tech198]

I've must of gone through 5 USB drives over te last few months : all 4TB, and all different manufactures.. WD, Seagate, andd Hitachi, and i can't understand surly all of them just can't be the same conclusion.

This is basally the issue :

Why do my external USB hard drives become slower when I have filled them up only to a third?

The read/write speeds of my external USB3 hard drives (*) are significantly higher when the disks are empty (no normal files; less then 10MB used) compared to when they are moderately full (2.5 mil...

superuser.com

Is this true, and i could understand from same manufacture (a dud, or something) but every manufacture i've used ??

What are other people's experience....

Yes, as pointed out in the above link, i too have millions of small files... but i don't se any other way if you wish to store them.

Whats the solution ? or is this the "make drives cheap, but realiable enough" notion ? I've never encourced a read/write error, just slow performance, compared this to when empity, it was quicker.. allot quicker..

 

[Juicy Box]

Are you using MacOS? I wonder if lack of TRIM support is the issue.

but i don't se any other way if you wish to store them

Maybe consider using HDDs or if you need an SSD, try using a TB enclosure.

TB has TRIM support for MacOS, so this alone might fix your issue.

 

[jbarley]

Are you using MacOS? I wonder if lack of TRIM support is the issue.


Maybe consider using HDDs or if you need an SSD, try using a TB enclosure.

TB has TRIM support for MacOS, so this alone might fix your issue.

It appears he is using USB3, so his options are very limited.

 

[Juicy Box]

It appears he is using USB3, so his options are very limited.

Well, yeah the OP is using USB3, but the OP didn't say that they have to use USB3.

If the OP computer has TB1, TB2, or TB3, then I would use a TB3 NVMe drive instead of keep using USB drives that slow down.

Even a SATA drive over TB would probably be a better choice than USB, as it would have TRIM support.


If speed isn't important, and the OP needs to use USB for whatever reason, then consider a HDD.

What are other people's experience....

I have seen many threads with people complaining about slowdowns on SSDs, especially on write speeds, but also slightly slower on reads as well.

Enabling TRIM fixed these issues for a lot of people, so this could be your issue. I suspect that as the space on a drive becomes more and more used, the lack of TRIM would be even more apparent, with more dramatic slowdowns.

If you are using MacOS, there is no TRIM support for USB drives, so that is why I suggested using a TB solution or a HDD.

 

[Tech198]

ok ya,, it is USB3 drive and port i'm using.. Regular spinning drive..

Your coming from wrong direction i believe.... if it was a port issue, the drive would always be slow even if it was empty..

This slows down as you fill it up. Counting 3.76TB on a 4TB drive.. .

 

[Juicy Box]

if it was a port issue, the drive would always be slow even if it was empty..

This is definitely not true, as the problems you are having could be TRIM related, which is not supported on USB in MacOS.

The lack of TRIM wouldn't really be apparent on a new drive, but once the drive fills up, the drive could slow down.

That is why I was suggesting using a TB drive as TB has TRIM support and it might help the issues you are having.

There is a bunch of resources on TRIM, so feel free to research it, but with a quick search this was the top result on CNET:

Installing an SSD on your Mac? Don't make this mistake

Enable this one single command to ensure a longer life for your SSD.

www.cnet.com

TRIM allows the operating system to actively inform an SSD which blocks of data are no longer in use and can be wiped internally. This helps the drive work more efficiently and leads to faster performance and most importantly, longer lifespan.

A problem with USB SSDs on MacOS is that there isn't TRIM support by the OS.

Some SSDs have a TRIM feature of sort that allows the SSD controllers to handle garbage collection, but I am unsure if the USB SSDs you are using have this feature nor do I know if it handles free space as well as TRIM on the OS would.

Counting 3.76TB on a 4TB drive.. .

Even with TRIM support, when drives are nearly empty, performance could take a hit.


If just using it for storage purposes, HDDs don't have the same limitations that SSD do when it comes to filling them up. Maybe consider using them?

 

[jbarley]

If just using it for storage purposes, HDDs don't have the same limitations that SSD do when it comes to filling them up. Maybe consider using them?

The OP is already using an HDD, or as they call it "Regular spinning drive" ☕

 

[saudor]

why is everyone jumping to conclusions about TRIM? OP has clearly stated they are running a HDD which has no TRIM. Most people wouldnt go through 5 4TB SSDs either.


Anyways, All HDDs DO slow down as they fill up mainly because of physics. Data starts getting written at the edge of the drive and make its way inwards. Since the medium rotates, more data passes under the heads near the edge compared to the center at the same rotational speed. Also latency increases too. my 4TB drive starts at 200 mb/s+ on the outer edge and drops to 110 mb/s on the inner edge (end of the drive)

You also have to deal with more fragmentation as the drive fills up so performance starts to take a hit as the drive does more random seeks/reads to piece together the fragments

If you need consistent high speed, short stroke it (e.g. partition only 2.5 TB of your 4tb) For small files you're better off using a SSD though as those have way higher random read/write performance

 

[Fishrrman]

You didn't tell us WHAT FORMAT the drives are in.
For platter based drives, HFS+ would be the preferred format (not APFS).

Also, as a platter-based drive "fills up", it can become fragmented. Most notably, there can be many "pockets of free space" between files.

Run a defragmentation app on the drive, and it will concatenate all the fragmented files, and remove the free space between them. When new files are written to the drive, they will then "flow into" the remaining area of free space and the writes should go faster.

There used to be two very useful utility apps for OS X:
- iDefrag
- iPartition

Both were published by a company called Coriolis Systems that has now discontinued them, because changes in the Mac OS and the move from platter-based drives to SSDs has diminished the need for the above apps.

However, some folks may still have a need for them.
Coriolis has graciously made available all previous versions of their software.

You can freely download them from here:

Coriolis Systems

 

[Juicy Box]

The OP is already using an HDD, or as they call it "Regular spinning drive" ☕

Sorry, it was late, and I totally missed this. I think I was confusing this thread with another thread about 4TB SSDs, also.

Yes, TRIM will not be an issue at all for HDDs.

why is everyone jumping to conclusions about TRIM? OP has clearly stated they are running a HDD

It wasn't clear in the OP's first two posts, and I totally missed it in the OP's 3rd post.

 

[Juicy Box]

The OP is already using an HDD, or as they call it "Regular spinning drive" ☕

why is everyone jumping to conclusions about TRIM? OP has clearly stated they are running a HDD which has no TRIM.

I know why I didn't see the OP post about the "Regular spinning drive", the post was edited. I was really surprised that I could have missed that so I checked my email and this was the original post not mentioning anything about HDDs:

 

[Tech198]

why is everyone jumping to conclusions about TRIM? OP has clearly stated they are running a HDD which has no TRIM. Most people wouldnt go through 5 4TB SSDs either.


Anyways, All HDDs DO slow down as they fill up mainly because of physics. Data starts getting written at the edge of the drive and make its way inwards. Since the medium rotates, more data passes under the heads near the edge compared to the center at the same rotational speed. Also latency increases too. my 4TB drive starts at 200 mb/s+ on the outer edge and drops to 110 mb/s on the inner edge (end of the drive)

You also have to deal with more fragmentation as the drive fills up so performance starts to take a hit as the drive does more random seeks/reads to piece together the fragments

If you need consistent high speed, short stroke it (e.g. partition only 2.5 TB of your 4tb) For small files you're better off using a SSD though as those have way higher random read/write performance

ok . that makes sense.

 

[Hotwheeled]

Hi Tech198, is it possible for you to post the make/models of the drives?.

 

[Tech198]

Hi Tech198, is it possible for you to post the make/models of the drives?.

WD Elements SE

 

[Hotwheeled]

WD Elements SE

It may be SMR (shingled magnetic recording) drives that you are buying as you dont mention the model number.Your description mirrors my probems with newer external and internal storage options i have purchased in the last 18 months.(WD Elements 6TB,WD MYBook 8TB (Hitachi White Drive Inside), WD Passport 5TB and 5TB Seagate Backup plus recent purchases with SMR.all these are poor performing write drive for my needs in moving large amounts of files.)

Finding out what was slowing me down took awhile to figure out as i made more uninformed purchases trying to improve my results. (SMR instead of CMR). WD now lists which drives are SMR as they did not previously.Google is you friend on this one for future purchases.

Info on WD SMR
WD HDD Trim SMR

 

[AppleSmack]

As the post above says, it's SMR causing the slow down as the drive fills. Here's Wikipedia's explanation of what SMR is and why it causes the slowdown: https://en.m.wikipedia.org/wiki/Shingled_magnetic_recording

When you say you've "gone through" several drives, do you mean you've bought them to see if they're any better, or that you've actually ended up with drives that no longer function?