Thunderbolt 4 & NVMe M.2 External Storage: Read/Write Speeds?

[atonaldenim]

I've create a 32tb Raid0 striped volume with 4 units of OWC Thunderblade 8tb, each one connected to a different Thunderbolt4 port on the Mac Studio Ultra. The read/write speeds are amazing!

This is great to hear, and surprising because several people in this thread are saying they can’t connect more than 3 bus-powered Thunderbolt drives at the same time, on the Studio Ultra.

I wonder if the OWC Thunderblade is better in this regard than other Thunderbolt NVMe drives/ enclosures? (EDIT: Oh, because the Thunderblade is not bus powered. My mistake.)

For people making RAIDs out of multiple external drives, I’d be interested to hear your long term experiences, if they prove to be stable.

For people considering the possible speeds of Thunderbolt NVMe drives, it might be helpful to think in terms of PCIe lanes. One Thunderbolt port on the Mac Studio should provide x4 lanes of PCIe 3.0 bandwidth.

(Yes even though the M1 Max/Ultra is capable of PCIe 4.0 internally, PCIe 3.0 x4 speed is the TB3/4 spec.)

As one M.2 / U.2 NVMe SSD uses x4 PCIe lanes, you’ll only get max SSD speed with one NVMe SSD per Thunderbolt port.

With a PCIe NVMe card or enclosure that has a PCIe switch on it, like the Sonnet M.2 4x4, OWC Accelsior 4M2 etc, it can dynamically allocate those x4 lanes between one SSD or several depending on which are accessed, but x4 lanes is still the overall speed limit of the Thunderbolt port.

With a multi-SSD enclosure/card that lacks a PCIe switch, the x4 lanes are divided in a fixed way. The Sabrent TB3 Dual NVMe M.2 SSD Enclosure and Sonnet Echo Dual NVMe Thunderbolt Dock for example seem to provide x2 PCIe lanes to each SSD. The OWC Express 4M2 only provides x1 lane to each SSD. Any of these could theoretically reach max Thunderbolt speeds in RAID 0 with all SSD slots used. But each individual SSD would not be able to reach its full PCIe x4 speed if accessed individually, say in “JBOD” mode.

(Or a multi-SSD PCIe NVMe card that lacks a PCIe switch and requires “PCIe bifurcation” support on the motherboard, will usually provide x4 lanes to one SSD and none of the other SSDs will work.)

Same principle goes for multi-PCIe card enclosures like the Akitio Node Duo, the Mac’s Thunderbolt port will top out at x4 lanes only. That enclosure gives x2 lanes to each PCIe slot.

 

[handheldgames]

Yes, I think that this new version of the T7 is an attractive option for a USB 3.2 Gen 2 (10Gb/s) drive that offers shock/dust/water protection. My experience with T7 drives is that they're reliable, don't have heat/throttling issues and are plenty fast enough for most purposes. Most people are likely to evaluate the new T7 against the drives below, although there are other options. Prices are for 2TB in USD:

Samsung T7 Shield $240
SanDisk Extreme Portable v.2 $240
LaCie Rugged SSD $400

Whoops! Based on the performance, I had to dig deeper. Looks like the Samsung T7 is off topic as it's not NVME nor thunderbolt.

 

[Morseman]

Whoops! Based on the performance, I had to dig deeper. Looks like the Samsung T7 is off topic as it's not NVME nor thunderbolt.

A big issue for me with the T7 is the USB-C cable that comes with it. I get much better performance with a Thunderbolt cable I had lying around.

 

[handheldgames]

A big issue for me with the T7 is the USB-C cable that comes with it. I get much better performance with a Thunderbolt cable I had lying around.

What kind on an increase are you seeing?

 

[Morseman]

What kind on an increase are you seeing?

My mistake - the difference was not nearly so large as I first thought. I had used a generic USB-C cable for my first test and got 36Write/35Read, and 788W/717R with the Thunderbird cable. When I carefully selected a cable that I was sure came with the T7 the difference was much less: 710W/600R

 

[handheldgames]

Cables can make a difference @Morseman

FWIW... I've been running a Silently / Modified Highpoint 7101a switched NVMe adapter with 2 1TB 970 Pros and a 1 TB Sabrent Rocket without any disconnection issues. All 3 NVME SSD's average around 32C with the mod, which is only a degree or 2 higher than they had running with its original noisy fan.
Luckily, there are many options to get external storage on the Mac Studio

 

[swamyg1]

That Thunder Blade with 8tb is 2k, I would be tempted to just upgrade the internal 8tb… is the motivating factor for going external to have more flexibility with storage options down the road?

 

[F-Train]

The first post in this new thread has results for an additional setup: Comparison: Internal vs. External NVMe

The setup is:
Mac Studio: base M1 Max
RAID 0
Enclosure: Sabrent Dual NVMe Enclosure EC-T3DN
SSDs: 2TB Samsung 970 EVO + 2TB Samsung 970 EVO Plus

There are two problems with this setup. As discussed above, an effective RAID 0 requires two enclosures (or a dual enclosure with two outputs) and two Thunderbolt 4/3 ports. Also as discussed above, the 970 EVO Plus's write speed is very poor when used with a Thunderbolt 3 port rather than mounted on a motherboard.

Also, note posts in the new thread that raise the question of whether the results shown are accurate or instead reflect caching.

 

[F-Train]

I now have both a TEKQ Cube enclosure and the newer Acasis enclosure. In the photo below, the TEKQ is on the left and the Acasis is on the right.

As can be seen, the Acasis is larger because it's wider and a bit longer. It's also a bit deeper. It isn't obvious to me that the extra space in the enclosure has a function. I'm also skeptical that the additional surface area makes a significant difference to temperature.

The two enclosures are held together differently. The TEKQ uses small screws. The Acasis is "toolless". It's held together by friction. It also uses a small rubber piece (Acasis includes a spare) to fasten the end of the SSD to the enclosure. I didn't have the small rubber piece in place for this photo.

I'm not crazy about the absence of screws, but the Acasis will probably work OK if I don't open and close the enclosure frequently. I probably won't return the Acasis, but if I had this to do over again I would have purchased a second TEKQ.

I should receive an additional Samsung 980 Pro SSD tomorrow, at which point I'll test RAID 0 read and write speeds, connecting the enclosures to different Thunderbolt 4/3 ports. I'll set up the RAID with MacOS Disk Utility.

I ordered the additional 980 Pro with Samsung's heat sink option. Samsung has been asking a fair amount for the heat sink, but at current prices it only added US$5. I believe that Samsung and its competitors are now offering a heat sink due to demand for these SSDs arising from the Sony PlayStation 5. It will be interesting to see how the heat sink works out with these enclosures.

 

[F-Train]

Here is what Blackmagic Disk Speed says about the RAID 0 setup discussed in the post above. Samsung sent me the additional 980 Pro without the heat sink. I'll sort that out with them, but I can't comment at the moment on the suitability of the heat sink for these enclosures.

As several people have found, a single Thunderbolt 4/3 port limits read and write speed of an NVMe M.2 SSD drive or drives connected to it to somewhat below 2.8GB/s. These RAID 0 speeds using two ports and two 980 Pro SSDs are impressive, although for me they have limited practical application. I plan to use the two SSDs as separate drives.

It's perhaps worth noting that Samsung says that the 980 Pro, as a Gen. 4 SSD, is capable of up to 5100MB/s sequential writes and 7000MB/s sequential reads when used on a motherboard.

 

[Ploki]

I've had a spare large-ish heatsink and just putting my JEYI enclosure on it has directly has significantly dropped my 980 PRO temperature.
I think it makes more sense to couple the drive to the enclosure with thermal pads inside and put the heatsink outside

 

[SamC741]

It show significant performance on z690 Mob, I have Asus z690 TUF mob with Samsung EVO 980 M2 1 TB

 

[AfterglowMP]

suggestion of getting small T4 drives of

Quick update: I changed all the cables from Thunderbolt3 to Thunderbolt4, now read/write speeds have improved!! Amazing

View attachment 1986182

What OWC Thunderbolt units did you use?

 

[macguido]

Has anyone considered the OWC U2 Shuttle? I'm planning to expand my Mac studio and use my Akitio Thunder3 Quad X as an enclosure. The U2 Shuttle carrier is equipped with 4X NVMe slots to fit inside a 3.5 inch drive-bay. I have just emailed both OWC and Akitio just to be sure about compatibility. Also wondering what read/write speed I should expect.

 

[F-Train]

Has anyone considered the OWC U2 Shuttle? I'm planning to expand my Mac studio and use my Akitio Thunder3 Quad X as an enclosure. The U2 Shuttle carrier is equipped with 4X NVMe slots to fit inside a 3.5 inch drive-bay. I have just emailed both OWC and Akitio just to be sure about compatibility. Also wondering what read/write speed I should expect.

I haven't used either of these products but had a look at the specs. I would want to talk with OWC about the Thunderbolt ports. The Akitio enclosure has two Thunderbolt 3 ports. If the OWC Shuttle can use only one of those ports, your read/write speeds will be limited, as discussed in posts above, to something under 2800MB/s. If the shuttle can use both ports, the speeds should be higher depending on what NVMe SSDs you use. Post #85 shows RAID 0 speeds for two 2TB Samsung 980 Pros connected to two separate Mac Studio Thunderbolt 4/3 ports.

I would also want to know what OWC says about the enclosure's Thunderbolt 3 port limits. As I understand it, at least one OWC Thunderbolt 3 enclosure supports quite a bit less than 2800MB/s for a single NVMe SSD.

 

[macguido]

I haven't used either of these products but had a look at the specs. I would want to talk with OWC about the Thunderbolt ports. The Akitio enclosure has two Thunderbolt 3 ports. If the OWC Shuttle can use only one of those ports, your read/write speeds will be limited, as discussed in posts above, to something under 2800MB/s. If the shuttle can use both ports, the speeds should be higher depending on what NVMe SSDs you use. Post #85 shows RAID 0 speeds for two 2TB Samsung 980 Pros connected to two separate Mac Studio Thunderbolt 4/3 ports.

I would also want to know what OWC says about the enclosure's Thunderbolt 3 port limits. As I understand it, at least one OWC Thunderbolt 3 enclosure supports quite a bit less than 2800MB/s for a single NVMe SSD.

Right! I'll definitely ask OWC whether the Shuttle can use both ports, but I suspect it can't as it has to be mounted into a 3.5 drive bay. Why faster NVMes should impact performance, aren't they limited to the TB3 speed?

 

[F-Train]

Right! I'll definitely ask OWC whether the Shuttle can use both ports, but I suspect it can't as it has to be mounted into a 3.5 drive bay. Why faster NVMes should impact performance, aren't they limited to the TB3 speed?

I don't mean to suggest that Gen 4 NVMe SSDs will generally perform better than Gen 3 SSDs. However, Samsung's 970 EVO Plus SSD, in particular, has very poor write speed when used with Thunderbolt instead of on a motherboard. There's discussion about this earlier in the thread.

 

[handheldgames]

The problem may be an issue with MacOS compatibility and not just an issue with Thunderbolt on MacOS. There is a good thread documenting incompatibilities in the Mac Pro Forum that's worth checking out. Degradations in performance I've observed with the Sabrent Rocket 1TB, a Gen 4 PCIe SSD, happen on a thunderbolt based GPU enclosure and from a PCIe installation in a Highpoint 7101a.

 

[Ploki]

But then my full 4y old EVO non-plus wouldn't have better write speeds than brand new EVO Plus

 

[Rb70]

I've had a spare large-ish heatsink and just putting my JEYI enclosure on it has directly has significantly dropped my 980 PRO temperature.
I think it makes more sense to couple the drive to the enclosure with thermal pads inside and put the heatsink outside

Here is what Blackmagic Disk Speed says about the RAID 0 setup discussed in the post above. Samsung sent me the additional 980 Pro without the heat sink. I'll sort that out with them, but I can't comment at the moment on the suitability of the heat sink for these enclosures.

As several people have found, a single Thunderbolt 4/3 port limits read and write speed of an NVMe M.2 SSD drive or drives connected to it to somewhat below 2.8GB/s. These RAID 0 speeds using two ports and two 980 Pro SSDs are impressive, although for me they have limited practical application. I plan to use the two SSDs as separate drives.

It's perhaps worth noting that Samsung says that the 980 Pro, as a Gen. 4 SSD, is capable of up to 5100MB/s sequential writes and 7000MB/s sequential reads when used on a motherboard.


View attachment 2012599

F-Train,
I was wondering if you had a chance to evaluate the 980 heat sink and if it will fit in the enclosure? I’ve never worked with SSD m2 drives before. Thanks.

 

[F-Train]

F-Train,
I was wondering if you had a chance to evaluate the 980 heat sink and if it will fit in the enclosure? Thanks.

I wish I could answer that question, but there was an error in my order with Samsung and my SSD didn't come with the heat sink. They don't have a way to send me the heat sink by itself. I would have to send back the SSD at their expense and they would send me a new one with the heat sink. I'm not kidding I decided to just forget it.

 

[Rb70]

I wish I could answer that question, but there was an error in my order with Samsung and my SSD didn't come with the heat sink. They don't have a way to send me the heat sink by itself. I would have to send back the SSD at their expense and they would send me a new one with the heat sink. I'm not kidding I decided to just forget it.

Thanks for the reply.
Is the heat transfer tape that Tekq sends with the enclosure sufficient? Anything I should know about the TEKQ and 980 setup? Thanks Bob

 

[F-Train]

Thanks for the reply.
Is the heat transfer tape that Tekq sends with the enclosure sufficient? Anything I should know about the TEKQ and 980 setup? Thanks Bob

Have a look at post #84, where I talked about the difference between how the two enclosures are constructed. You may have a preference when it comes to an enclosure that's held together with screws (albeit small ones) and an enclosure that's held together by friction.

With the caveat that I haven't driven my 980 Pros hard over a long period, I have not seen a significant difference in performance between the two enclosures. Also, while the 980 gets pretty warm, I don't think overly so.

I think that you've just got to experiment. Tape is supposed to be in contact with both the SSD and the top of the enclosure, which is not so easy to judge given that you can't see through the aluminium There are also proponents of various brands of tape. It's unclear to me whether the alleged differences are real or whether this is just snake-oil marketing. Note that if you put one of these enclosures on top of an aluminium block, or indeed on top of your aluminium Mac Studio, they will draw some heat from the enclosure. I've also tried the enclosures with the top off and on.

I have no definitive conclusions.

From looking at videos of 980 installation in a PS5, I think that the Samsung heat sink will work with these enclosures. However, when I thought that I was buying the heat sink, it was a $5 surcharge. I just looked at B&H's current pricing, and it looks like the heat sink is currently an extra $40, which I think is crazy.

Not very helpful, am I

 

[Rb70]

Have a look at post #84, where I talked about the difference between how the two enclosures are constructed. You may have a preference when it comes to an enclosure that's held together with screws (albeit tiny ones) and an enclosure that's held together by friction.

With the caveat that I haven't driven my 980 Pros hard over a long period, I have not seen a significant difference in performance between the two enclosures. Also, while the 980 gets pretty warm, I don't think overly so.

I think that you've just got to experiment. Tape is supposed to be in contact with both the SSD and the top of the enclosure, which is not so easy to judge given that you can't see through the aluminium There are also proponents of various brands of tape. It's unclear to me whether the alleged differences are real or whether this is just snake-oil marketing. Note that if you put one of these enclosures on top of an aluminium block, or indeed on top of your aluminium Mac Studio, they will draw some heat from the enclosure. I've also tried the enclosures with the top off and on.

I have no definitive conclusions.

From looking at videos of 980 installation in a PS5, I think that the Samsung heat sink will work with these enclosures. However, when I thought that I was buying the heat sink, it was a $5 surcharge. I just looked at B&H's current pricing, and it looks like the heat sink is currently an extra $40, which I think is crazy.

Not very helpful, am I

You’re very helpful. I appreciate it. Thank you. I’m looking at the add on heat sinks that attach to the top of the enclosure. They are only around $10 so I’ll buy one and, if the enclosure seems hot, I’ll put it on.

 

[Ploki]

Thanks for the reply.
Is the heat transfer tape that Tekq sends with the enclosure sufficient? Anything I should know about the TEKQ and 980 setup? Thanks Bob

Hi, that's exactly is what i did. My 3 NVMe's are on the heatsink, with the chips facing the heatsink.
I observed slightly lower temperatures on the Thunderbolt enclosure and the heatsink is warm to the touch - if i stress the drives the heatsink gets significantly warmer which means it's doing its job.

They're also tucked neatly away under the desk (rack actually), so two birds with one stone