External hard disk speed

[cool11]

I am not sure if the problem I will describe, it is a hardware issue or a mac os issue.
So I decide it to post it here.

I have an external hard disk attached to my mbp, for backup purposes. I use superduper application, but also I am doing manual copying of files sometimes.
My mbp it is 16' m1 pro.
I use this hard disk almost from the moment I bought my mbp back in 2022. APFS formatted. Classic hard disk, Western Digital, not ssd.

The latest months, I observe a big decrease in speed performance.
I did not used special apps to measure it, I am telling what I see and what I experience everyday.

For example, though superduper has the same disk to copy/sync as always, nothing special changed, and I always use 'smart update' feature=incremental file copying,
it used to take about 30 minutes to complete till a few months ago,
but now it takes almost 2 hours!
Speed was about 150-200mb/s back in the days,
while now it is about 50mb/s!

Also, while I copy files there, it is really slower than in the past. The whole responsiveness seems slow. Even finder sometimes seem to stuck while I try to access it.

I unplugged cables and plug them again.

I cannot understand where this slowness is coming from.

And the reason I decided to post this problem here in 'mac os' subforum,
it is because I wonder if there is any issue like 'fragmentation' in windows,
or any other software related issue.

Is there any kind of maintenance that it could be done manually by me? Any checking?
Is anything new in sequoia updates that causes extra slowness?
I cannot accept it as a normal thing, as my internal mbp disk is always at the same lever of occupied gigabytes, and in general I have not made any serious changes besides the usual application updates.

I hope there is some kind of workaround for this problem.

 

[baconcat_8964]

I think it is caused by the APFS filesystem. APFS is designed for SSD. It can be used on HDDs, but will cause significant fragmentation issues over time and seriously affect drive performance. I would recommend backing up the drive and formatting it back to HFS+.

Using APFS On HDDs ... And Why You Might Not Want To

After 16 months of using and testing APFS—Apple's new file system—I’ve come to the conclusion that you probably don’t want to use it on HDDs (disks with rotating platters).

eshop.macsales.com

An analysis of APFS enumeration performance on rotational hard drives - Bombich Software

bombich.com

 

[cool11]

I think it is caused by the APFS filesystem. APFS is designed for SSD. It can be used on HDDs, but will cause significant fragmentation issues over time and seriously affect drive performance. I would recommend backing up the drive and formatting it back to HFS+.

Using APFS On HDDs ... And Why You Might Not Want To

After 16 months of using and testing APFS—Apple's new file system—I’ve come to the conclusion that you probably don’t want to use it on HDDs (disks with rotating platters).

eshop.macsales.com

An analysis of APFS enumeration performance on rotational hard drives - Bombich Software

bombich.com

I am searching if there is any non destructive way to make a conversion from APFS to HFS+, as it I am not having at this period of time, the time to make such a huge copy somewhere and then formatting.
There was a solution in the past, but no longer available:

APFS to HFS+ Converter | Paragon Software Group

Fast and lossless conversion of APFS volumes into HFS+ volumes for your Mac by Paragon Software.

www.paragon-software.com

I am looking for possible alternative solutions.

So, it seems that mainly APFS causes all this delay in performance?
Is there any maintenance tool to help this situation, if the external HD remains to APFS?

 

[baconcat_8964]

I am searching if there is any non destructive way to make a conversion from APFS to HFS+

From what I learned, sadly there's no way to convert APFS to HFS+ drive without erasing all data.

So, it seems that mainly APFS causes all this delay in performance?

For a three-year-old HDD, if there's no bad sector issue, this filesystem could be the major cause.

Is there any maintenance tool to help this situation, if the external HD remains to APFS?

Some say APFS has a defragementation tool (https://eclecticlight.co/2019/10/19/should-you-enable-defragmentation-on-apfs-hard-drives/). However it won't significantly help the performance, and the nature of APFS will generates a lot of fragements again for sure.

 

[Fishrrman]

Best solution:
Replace the HDD with an SSD.
I predict your problems will be ... solved.

 

[MayflyMaven]

I think it is caused by the APFS filesystem. APFS is designed for SSD. It can be used on HDDs, but will cause significant fragmentation issues over time and seriously affect drive performance.

I don't think fragmentation is the culprit here; APFS defragmentation probably won't make a substantive performance difference (certainly not retroactively). In my own testing, I struggled to find a real-world performance effect of fragmentation on reading performance, and in fact found faster performance of APFS on the same hardware (YMMV – not all rotational disks will perform as well). I ran some tests (this was last year, I didn't do this just now):

- Create 100 1GB files on 1TB partition at the beginning of a 4TB rotational disk (I was using a 7200RPM WD Black, IIRC) formatted APFS – 100GB initial data set
- Measure how long it takes to read the 100 files
- Create a snapshot on the volume (to force COW), then modify each file's data in 100MB chunks to artificially introduce fragmentation – 50% of the files' data were modified, thus adding 50GB of data to the source volume
- Measure how long it takes to read the 100 files
- Repeat the test 20 times (so eventually the 1TB partition is completely full)

I then repeated the same tests with HFS+, granted there was (likely) no fragmentation because I wasn't changing the sizes of the files. Results:

I can't explain why APFS performance improved right at the end as the disk filled up, but I repeated the read test and got roughly the same result. In every case, though, the decline in performance due to fragmentation was noticeable (~5% decline), but still faster than HFS+. I repeated all of these tests with APFS defragmentation enabled (immediately after formatting the volume) – no difference (and FWIW, I ran a total of 6 tests, and eventually measured the number of extents on the filesystem to objectively determine if defragmentation was having any substantive impact; the full analysis is probably straying too far from the topic at hand). Bear in mind this test was only designed to determine if there was a substantive seek penalty to reading fragmented files on an APFS volume on rotational storage.

I would guess that OP is seeing slower performance now for two reasons:

- APFS suffers worse performance than HFS+ while enumerating files and folders (vs. reading file data); the backup application is probably enumerating the entire destination every time (I have not used SuperDuper in many years, but I seem to recall that "Smart Update" means it will only copy the differences, but it still enumerates the entire source and destination to find the differences, vs. using FSEvents to find a smaller collection of modified files and folders)
- Rotational hard drive performance gets worse as the disk fills up – the platter spins slower at the "end" of the disk, which is the center of the donut. You can prove that one easily – partition a rotational disk into four equal partitions; you'll find a linear decline in performance from one partition to the next, and the first partition will be ~4x faster than the last partition. Naturally this point would only be applicable if OP's backup disk is getting full.

And of course there is the hardware-age wildcard – it could simply be showing signs of deterioration.

 

[cool11]

Best solution:
Replace the HDD with an SSD.
I predict your problems will be ... solved.

Buying a big ssd disk, it is a huge expense.
My classic external HD is 4TB.
Almost unthinkable to buy such drive in ssd for backup purposes....maybe in the future with price drops.

But alternativelyy I am thinking about usb flash drives, which are much cheaper, and with a good choice, it can be really fast. There are usb flash disks with 1gb capacity with very reasonable prices.
Would such a choice would be a solution, regarding speed?

 

[benwiggy]

After 3 years, it's possible that the drive could be failing. It's below average life, but these things do happen.

 

[MayflyMaven]

I am thinking about usb flash drives, which are much cheaper, and with a good choice, it can be really fast. There are usb flash disks with 1gb capacity with very reasonable prices.

Performance of flash storage can be even slower than rotational media. If I were choosing, it would be a choice between a 7200RPM rotational disk or an SSD. Personally, I'd choose SSD for backups of a startup disk (or any time you're backing up many small files), and would consider rotational storage for large media (e.g. a disk that holds movies) and the backup of that media.

 

[VitoBotta]

I think it is caused by the APFS filesystem. APFS is designed for SSD. It can be used on HDDs, but will cause significant fragmentation issues over time and seriously affect drive performance. I would recommend backing up the drive and formatting it back to HFS+.

Using APFS On HDDs ... And Why You Might Not Want To

After 16 months of using and testing APFS—Apple's new file system—I’ve come to the conclusion that you probably don’t want to use it on HDDs (disks with rotating platters).

eshop.macsales.com

An analysis of APFS enumeration performance on rotational hard drives - Bombich Software

bombich.com

CarbonCopyCloner needs APFS so it can work with snapshots. I am curious if SuperDuper needs APFS too.

 

[gilby101]

But alternatively I am thinking about usb flash drives, which are much cheaper, and with a good choice, it can be really fast.

1. They have much smaller cache than SSDs, so speed drops dramatically for anything except a modest amount of writing.
2. Life as a an active APFS backup disk unlikely to be long.

Maybe if you choose very carefully. You could start by looking for the most expensive flash drives.

The latest months, I observe a big decrease in speed performance.

How full is it? I would start to get nervous of HDD APFS performance once 70% full.

 

[cool11]

How full is it? I would start to get nervous of HDD APFS performance once 70% full.

None of my disks are anything close to full.
Below 50% of full capacity.

 

[Fishrrman]

Vito wrote:
"CarbonCopyCloner needs APFS so it can work with snapshots. I am curious if SuperDuper needs APFS too."

SD also creates snapshots (as far as I can tell).
So it needs APFS, as well.

The difference is that CCC allows the user to turn off snapshot creation.
With SD there's no option I know of to disable that...

 

[BrianBaughn]

I decided that having ANY old, mechanical HD attached to my M1 Mac Studio was slowing Finder activity down. I still have some but they're only attached for an occasional backup and then disconnected.