USB 3.1 Gen 2 vs Thunderbolt 3 for 4-bay external JBOD enclosure

[camner]

I need to replace my OWC Thunderbay 4 (TB2) enclosure that I've used for years with a 2015 27" iMac (read/write speeds have fallen to 10% of what they used to be...I've tried a different cable as well as connected the enclosure to both the 2015 iMac and my new 2020 iMac, with similar results).

I'm deciding between the OWC Thunderbay 4 Thunderbolt 4 enclosure ($390) and the OWC Mercury Elite Pro Quad (USB 3.1 Gen2) enclosure ($230).

I know that USB 3.1 Gen 2 maxes out at 10Gbps while Thunderbolt 3 maxes out at 40Gbps, but I wonder if in my use case I'll see any real difference in speeds.

In the external enclosure I will have my Time Machine drive (HDD), a drive (HDD) for daily system backups via Carbon Copy Cloner, and a 2 TB SATA SSD to serve as my external drive for the 27" iMac.

Will this setup ever really saturate the 10Gbps USB 3.1Gen 2 interface? From what I've read, I think the SSD will max out at about 400MB/s, while the HDDs will get, maybe, 150MB/s (depending on the characteristics of the individual drive). That comes to 700MB/s max, which, according to Google's conversion, is under 6Gbps.

I plan on keeping this setup for about 5 years. I can't imagine moving the backup drives to SSDs (I would need 4TB) unless the cost of 4TB drives came down incredibly much. I know that 2TB SSDs have come down from the ~$600 range a few years ago to ~$300 (I'm comparing Samsung 860 EVOs), but even at $300, I don't think I'd move the backup drives to SSDs.

 

[joevt]

Thunderbolt maxes out at 22 Gbps (the rest can be used by DisplayPort).

USB 3.1 gen 2 can do about 980 MB/s.
Thunderbolt 2 can do about 1500 MB/s.
Thunderbolt 3 can do about 2500 MB/s.

I'm not sure what a OWC Thunderbay 4 Thunderbolt 4 enclosure is. Do you mean OWC ThunderBay 4 Thunderbolt 3 enclosure? The max speed of that is 1527 or 1550 MB/s which is not much greater than USB 3.1 gen 2. That extra speed can probably only be achieved with large sequential reads and writes. I don't think you'll notice a difference. Plus your drives aren't fast enough to exceed USB 3.1 gen 2 speed.

You can get NVMe drives at 9¢ per GB. They may not be the fastest NVMe drives, but they're still faster than SATA and USB 3.1 gen 2.

Best SSDs: May 2021

www.anandtech.com

 

[camner]

Thunderbolt maxes out at 22 Gbps (the rest can be used by DisplayPort).

Interesting. Are you saying that only 22 Gbps out of the theoretical max of 40 can be used for data transfer, even if there is no display connected?

And, am I correct that since the 27" iMac has a single Thunderbolt 3 bus, the 40Gbps is shared by the two physical ports? (And similarly, that the 5Gbps of USB 3.1 Gen 1 is shared by the 4 USB-A ports on the back of the 27" iMac?)

USB 3.1 gen 2 can do about 980 MB/s.
Thunderbolt 2 can do about 2500 MB/s.

Do you mean Thunderbolt 3 here?

I'm not sure what a OWC Thunderbay 4 Thunderbolt 4 enclosure is. Do you mean OWC ThunderBay 4 Thunderbolt 3 enclosure?

Yes, I meant OWC Thunderbay 4 Thunderbolt 3 enclosure. Sorry for the fat fingers.

The max speed of that is 1527 or 1550 MB/s which is not much greater than USB 3.1 gen 2. That extra speed can probably only be achieved with large sequential reads and writes. I don't think you'll notice a difference. Plus your drives aren't fast enough to exceed USB 3.1 gen 2 speed.

So it sounds as if my best bet would be to get a USB 3.1 Gen2 enclosure for the HDDs and then add an external TB3 NVMe drive.

You can get NVMe drives at 9¢ per GB. They may not be the fastest NVMe drives, but they're still faster than SATA and USB 3.1 gen 2.

Best SSDs: May 2021

www.anandtech.com

I don't feel I need the fastest NVMe drive for my use case, but I would like to have one that is reliable. Thanks for the link to the anandtech article.

 

[camner]

Any thoughts about a good external NVMe enclosure? OWC is coming out with the Envoy Express at the end of September for $80. I've seen a JEYI on Alibaba for similar prices, but I don't know about the quality.

 

[joevt]

Interesting. Are you saying that only 22 Gbps out of the theoretical max of 40 can be used for data transfer, even if there is no display connected?

Approximately yes. 22 Gbps is 2750 MB/s. Some manufacturers say 2800 MB/s. There's a benchmark on YouTube for a Thunderbolt 4 controller/device that gets near 3000 MB/s.

And, am I correct that since the 27" iMac has a single Thunderbolt 3 bus, the 40Gbps is shared by the two physical ports?

No, each Thunderbolt 3 port can do 40 Gbps. Total PCIe bandwidth for both ports is limited to the PCIe 3.0 x4 of the discrete Thunderbolt controller which should allow ~3500 MB/s but is actually limited to something like 23 Gbps if you try to RAID 0 them together which is strange.
For maximum total bandwidth usage of both ports, you would do two displays from one port and a fast PCIe device from the other port.
When I did this test with my Mac mini 2018, I used two 4096x2304 68.595Hz 8 bpc displays (that was the max refresh I could get for that resolution with DisplayPort 1.2) and an eGPU. Total transmit bandwidth was 47.3 (31 Gbps for the displays and 16.3 Gbps (H2D) for the eGPU).
You might be able to do better with an XDR display (36.6 Gbps) and a fast NVMe drive (22 Gbps). For the XDR to use 36.6 Gbps, you need to disable DSC if your GPU supports it.
So it is impossible to use a full 80 Gbps but it is possible to exceed 40 Gbps of a single port.

MacBook Pro 13 inch 2020 or MacBook Air 2020 with 10th gen Intel CPU have integrated Thunderbolt controller so you can get full 22 Gbps from any two ports raided together (for approximately 37.3 Gbps of total PCIe traffic). https://egpu.io/forums/laptop-compu...hunderbolt-3-controller-bandwidth/#post-81885
Raiding 3 ports of integrated Thunderbolt 3 only gives a slight increase to 38.2 Gbps. Anyway, with integrated Thunderbolt 3, you can get closer to the 80 Gbps of two ports (one DisplayPort 1.4 display per port with a fast NVMe drive). The MacBook Pro 13 inch can't fill the bandwidth of four Thunderbolt ports because these Macs only support two displays, and like I said before, you can only max the PCIe bandwidth of two ports. This doesn't matter since you probably won't need to read or write to more than one port at a time (in other words, you probably won't make a RAID of two Thunderbolt devices).

(And similarly, that the 5Gbps of USB 3.1 Gen 1 is shared by the 4 USB-A ports on the back of the 27" iMac?)

That would only be true if the ports were part of a USB Hub.
I believe the ports are part of a USB controller (called a Bus in System Information.app).
Now, a USB controller can be limited by its PCIe connection, for example, if it were only PCIe 2.0 x1 (4 Gbps). It's probably at least PCIe 3.0 x1 (7.88 Gbps). I think each port is separately connected to the PCH (the chipset of the motherboard) so each should get 4 Gbps (USB 3.0 uses 8b/10b encoding similar to PCIe 2.0).
The PCH is connected to the CPU with DMI 3.0 (similar to PCIe 3.0 x4) so the total max of all devices connected to the PCH is 8 GT/s * 4 * 128b/130b = 31.5 Gbps. This includes USB, WiFi, SATA, NVMe, etc.

Do you mean Thunderbolt 3 here?

Thanks. Fixed it.

So it sounds as if my best bet would be to get a USB 3.1 Gen2 enclosure for the HDDs and then add an external TB3 NVMe drive.

I don't feel I need the fastest NVMe drive for my use case, but I would like to have one that is reliable. Thanks for the link to the anandtech article.

A USB NVMe might be more convenient. It's not as fast as Thunderbolt, but it's much less expensive, and more compatible with other computers. Combine an enclosure with an inexpensive, large capacity, NVMe.

ORICO Transparent NVMe M.2 Enclosure Tool-Free USB3.1 Type-C Gen2 10Gbps to M.2 SSD Enclosure for Intel 660p/Samsung 970 EVO/Samsung970 Pro 2230/2242/2260/2280 PCIe NVMe M-Key SSD up to 2TB - Red

ORICO Transparent NVMe M.2 Enclosure Tool-Free USB3.1 Type-C Gen2 10Gbps to M.2 SSD Enclosure for Intel 660p/Samsung 970 EVO/Samsung970 Pro 2230/2242/2260/2280 PCIe NVMe M-Key SSD up to 2TB - Red

www.amazon.com

I used an Intel 600p in mine. I think the latest is called 660p.

Any thoughts about a good external NVMe enclosure? OWC is coming out with the Envoy Express at the end of September for $80. I've seen a JEYI on Alibaba for similar prices, but I don't know about the quality.

The OWC Envoy Express is limited to PCIe 3.0 x2 (1553MB/s as they say).
I used a Trebleet which is similar to the JEYI. I don't think you'll have problems with either even if they are not Thunderbolt certified like the OWC Envoy Express.

 

[pldelisle]

I'd personally go Thunderbolt all the way. USB is slow compared to USB.

Why not a NAS ? Just asking for fun.

 

[Brian33]

Any thoughts about a good external NVMe enclosure? OWC is coming out with the Envoy Express at the end of September for $80. I've seen a JEYI on Alibaba for similar prices, but I don't know about the quality.

There is a lot of info in this thread, including user throughput tests:
https://forums.macrumors.com/threads/thunderbolt-3-m-2-nvme-ssd-enclosures.2027925/

 

[camner]

There is a lot of info in this thread, including user throughput tests:
https://forums.macrumors.com/threads/thunderbolt-3-m-2-nvme-ssd-enclosures.2027925/

Thanks for the link.

 

[camner]

A USB NVMe might be more convenient. It's not as fast as Thunderbolt, but it's much less expensive, and more compatible with other computers. Combine an enclosure with an inexpensive, large capacity, NVMe.

ORICO Transparent NVMe M.2 Enclosure Tool-Free USB3.1 Type-C Gen2 10Gbps to M.2 SSD Enclosure for Intel 660p/Samsung 970 EVO/Samsung970 Pro 2230/2242/2260/2280 PCIe NVMe M-Key SSD up to 2TB - Red

ORICO Transparent NVMe M.2 Enclosure Tool-Free USB3.1 Type-C Gen2 10Gbps to M.2 SSD Enclosure for Intel 660p/Samsung 970 EVO/Samsung970 Pro 2230/2242/2260/2280 PCIe NVMe M-Key SSD up to 2TB - Red

www.amazon.com

I used an Intel 600p in mine. I think the latest is called 660p.

The Amazon page for this Orico says "500GB" under "Digital Storage capacity"...does that mean this enclosure is limited to NVMe sticks with 500GB or less?
[/QUOTE]

And, thanks for detailed response with the analyses of access rates; it was interesting and helpful.

 

[Ynk]

I need to replace my OWC Thunderbay 4 (TB2) enclosure that I've used for years with a 2015 27" iMac (read/write speeds have fallen to 10% of what they used to be...I've tried a different cable as well as connected the enclosure to both the 2015 iMac and my new 2020 iMac, with similar results).

If problem is read/write speeds have decreased it might not be a hardware issue with the enclosure, are you using any of the RAID features and/or SSD disks? If so would suggest trying a format of the disks to at least rule out issues there before replacing the entire enclosure.

 

[camner]

Yes, read/write speeds have deteriorated significantly. I'm not using RAID, and I'm using a mixture of HDDs and SSDs.

The behavior is weirdly inconsistent. I'll take a given drive (SSD or HDD) and put it in a USB 3.1 Gen 1 dock and get consistent speed readings using Black Magic Disk Speed Test. I'll take exactly the save drive and put it into the Thunderbay 4 (TB 2) enclosure, and the speeds are always at least 30% slower. But, then sometimes the write speeds go down to 10% of normal (I've only seen that with an HDD, not an SSD).

I've seen similar behavior with the Thunderbay 4 connected to my old 2015 iMac as well as my new 2020 iMac, so I think I can rule out the computer being the issue.

 

[joevt]

The Amazon page for this Orico says "500GB" under "Digital Storage capacity"...does that mean this enclosure is limited to NVMe sticks with 500GB or less?

I have no idea why they would include a storage capacity number. Maybe 2TB drives didn't exist when they made that. I'm using a 2 TB drive. I'm sure it will work with larger drives as they become available. Maybe it doesn't work with some NVMe drives, but they don't give a list of those.