MP 7,1 Suggestions for best bootable PCI NVMe card for 2019 nCMP 7,1 Mac Pro

[ZombiePhysicist]

So I’m looking to use the 15tb micron 9300 pro drivehttps://www.amazon.com/Micron-15-36TB-Enterprise-Solid-State/dp/B07SK8GSYZ as my boot/main drive with the new 7,1 Mac Pro. Thread on that drive can be found here:

Mac Pro support for NVMe U2 Micron 9300 Pro 15.36TB U.2 Drive SSD?

ZombiePhysicist, where you able to get the 9300 SSD running in the Mac Pro 7,1? The 9300 is the drive I've settled on but I haven't been able to find any confirmation that the drive will work under Catalina. (I understand that the 9300 will need a PCIe to U.2 adapter or RAID controller.)

forums.macrumors.com

Any suggestions on what are the good pci cards that might work best on the 2019 Mac Pro? I’m guessing I could bracket the micron drive in the Pegasus 2ji bracket https://www.promise.com/us/Promotion/PegasusStorage .

Update 2020-02-02:
Just thought I'd update this thread in that I went with the Highpoint 7120 and Micron 9300 Pro 15.36TB drive option. Lot of details of getting it to work found in this thread:

Mac Pro support for NVMe U2 Micron 9300 Pro 15.36TB U.2 Drive SSD?

Ok so I got the drive. I got a bracket and it all seems to fit in decently tidy. Some pics: So I have the Pegasus J2i in there. Swapped the useless 8TB for a 16TB spinner. I got a 3.5" to 2.5" bracket. One note, the screws in the J2i that hold larger drives, are too big for both the 2.5"/3.5"...

forums.macrumors.com

 

[JesterJJZ]

Why do you need/want a boot drive that large? For best performance/practice you only keep your OS and apps on a boot drive, maybe user folders. Everything else goes on other drives, especially on an expandable system such as a Mac Pro.

 

[ZombiePhysicist]

Why do you need/want a boot drive that large? For best performance/practice you only keep your OS and apps on a boot drive, maybe user folders. Everything else goes on other drives, especially on an expandable system such as a Mac Pro.

Because it always ends up messing things up. When you have your photos library or music library on an external drive, and then that drive doesnt mount on some boot, and all the pointers get messed up. It's a pain. I've tried it several times. And in theory it works. In practice it inevitably fails and causes trouble. If it works for you, that's great. YMMV and all that.

Also, I see no downside. The 15TB is fast and spacious. I can put my entire world on it. Get a 2nd one to be a bootable recovery clone. Maybe throw in a 16TB spinner for a time machine archive of everything.
[doublepost=1562911071]https://forums.macrumors.com/thread...-pro-15-36-tb-nvme-ssd.2179713/#post-27323732 [/doublepost]

So from another thread, the Highpoint SSD7120 http://highpoint-tech.com/USA_new/series-ssd7120-overview.htm seems to be a good choice for the 5,1 Mac Pro. I wonder if it's also a good choice for the 7,1?

 

[ZombiePhysicist]

I could swear there was a slide from one of the WWDC presentations that showed at least one company was going to offer PCI cards for SSDs. Does anyone know of the company or recall/have a link to that slide?

 

[tsialex]

I could swear there was a slide from one of the WWDC presentations that showed at least one company was going to offer PCI cards for SSDs. Does anyone know of the company or recall/have a link to that slide?

The card pictured on the keynote was the Sonnet M.2 4x4 PCIe Card FUS-SSD-4X4-E3 and it's not a particular good card, several restrictions like the size and format of the blades.

I wrote about it on the Blade thread:

• Sonnet M.2 4x4 PCIe Card FUS-SSD-4X4-E3: Supports four single side 80mm M.2 blades. Uses PLX PEX8747 switch. Heatsink with a fan over the blades and PCIe switch. Needs a GPU with pre-boot configuration support. Do not buy it if you need to boot Windows from the blades or will upgrade to a PC later, this card can't boot Windows.

 

[ZombiePhysicist]

The card pictured on the keynote was the Sonnet M.2 4x4 PCIe Card FUS-SSD-4X4-E3 and it's not a particular good card, several restrictions like the size and format of the blades.

I wrote about it on the Blade thread:

• Sonnet M.2 4x4 PCIe Card FUS-SSD-4X4-E3: Supports four single side 80mm M.2 blades. Uses PLX PEX8747 switch. Heatsink with a fan over the blades and PCIe switch. Needs a GPU with pre-boot configuration support. Do not buy it if you need to boot Windows from the blades or will upgrade to a PC later, this card can't boot Windows.

Thank you so much. You dont happen to have a direct link to the post where you talk about it?

 

[tsialex]

Thank you so much. You dont happen to have a direct link to the post where you talk about it?

Start here #771, we discussed it for some time.

 

[ZombiePhysicist]

Start here #771, we discussed it for some time.

Thank you!

 

[ZombiePhysicist]

So I’m looking to use the 15tb micron 9300 pro drivehttps://www.amazon.com/Micron-15-36TB-Enterprise-Solid-State/dp/B07SK8GSYZ as my boot/main drive with the new 7,1 Mac Pro.

Any suggestions on what are the good pci cards that might work best on the 2019 Mac Pro? I’m guessing I could bracket the micron drive in the Pegasus 2ji bracket https://www.promise.com/us/Promotion/PegasusStorage .

Hmmm. So reading on the HighPoint site regarding the SSD7120, looks like it's not bootable. So that makes it a no-go. Bummer... I wonder if this might change for the 2019 Mac Pro?

Q: Can I use this card to boot from an NVMe drive or RAID array?
A: The SSD7101A-1 and SSD7120 cannot be used to boot. However, the SSD7102 and SSD7110 do support bootable NVMe drives or RAID arrays.​

 

[tsialex]

Hmmm. So reading on the HighPoint site regarding the SSD7120, looks like it's not bootable. So that makes it a no-go. Bummer... I wonder if this might change for the 2019 Mac Pro?

Q: Can I use this card to boot from an NVMe drive or RAID array?
A: The SSD7101A-1 and SSD7120 cannot be used to boot. However, the SSD7102 and SSD7110 do support bootable NVMe drives or RAID arrays.​

All Highpoint cards are bootable with a MP5,1. It's not the card firmware that makes it boot, but the Mac Pro BootROM infrastructure. This has some particularities.

My SSD7101-A boots from day one, but RAID booting is software dependent. Mojave don't have any official support from booting from an APFS array. Apple started to change things with Catalina DP3 and seems Catalina will finally have official support to boot from an APFS array.

With High Sierra you just need to use HFS+ and you can use a bootable 4-blade HFS+ array - use startosinstall to install.

With Mojave, to have correct support you boot from one blade and use the other three as a data array. If you make a 4-blade APFS array, you can't update macOS - software updates don't work at all.

 

[ZombiePhysicist]

All Highpoint cards are bootable with a MP5,1. It's not the card firmware that makes it boot, but the Mac Pro BootROM infrastructure. This has some particularities.

My SSD7101-A boots from day one, but RAID booting is software dependent. Mojave don't have any official support from booting from an APFS array. Apple started to change things with Catalina DP3 and seems Catalina will finally have official support to boot from an APFS array.

With High Sierra you just need to use HFS+ and you can use a bootable 4-blade HFS+ array - use startosinstall to install.

With Mojave, to have correct support you boot from one blade and use the other three as a data array. If you make a 4-blade APFS array, you can't update macOS - software updates don't work at all.

This is really interesting. So even though their FAQ says it's not bootable, it will be bootable with a single drive? So for example, I could have a single 15TB boot SSD drive, and then use the other 3 ports to do a software raid with 3 additional drives?

That's great news then! I wonder why they don't update their FAQs...

 

[tsialex]

This is really interesting. So even though their FAQ says it's not bootable, it will be bootable with a single drive? So for example, I could have a single 15TB boot SSD drive, and then use the other 3 ports to do a software raid with 3 additional drives?

That's great news then! I wonder why they don't update their FAQs...

It's bootable, every macOS release has it's quirks and limitations - only High Sierra supports everything.

Since it's a very complex subject, they play safe.
[doublepost=1563021244][/doublepost]

So for example, I could have a single 15TB boot SSD drive, and then use the other 3 ports to do a software raid with 3 additional drives?

Yes, you can. No one tested with drives bigger than 4TB that I know of.

 

[rxs0]

I recently received the SQUID Carrier Board™ PCI Express Gen 3 Carrier Board for 6 M.2 or NGSFF (NF1)
PCIe SSD modules, with x16 PCI Express upstream Adapter.

I am currently running this board in a windows box.

http://amfeltec.com/pci-express-gen-3-carrier-board-for-6-m2-or-ngsff-nf1-pcie-ssd-modules

I intend to test this system (with 6 Samsung 970 EVO 2TB NVMe SSD drives) in the 2019 Mac Pro computer when available. I will order the new Mac Pro as soon as it is available. I don't know if this will be bootable in RAID 0 on the new Mac Pro. I will update you with the results.

Thanks.

 

[Mac_User 0101]

I recently received the SQUID Carrier Board™ PCI Express Gen 3 Carrier Board for 6 M.2 or NGSFF (NF1)
PCIe SSD modules, with x16 PCI Express upstream Adapter.

I am currently running this board in a windows box.

http://amfeltec.com/pci-express-gen-3-carrier-board-for-6-m2-or-ngsff-nf1-pcie-ssd-modules

I intend to test this system (with 6 Samsung 970 EVO 2TB NVMe SSD drives) in the 2019 Mac Pro computer when available. I will order the new Mac Pro as soon as it is available. I don't know if this will be bootable in RAID 0 on the new Mac Pro. I will update you with the results.

Thanks.

That was the carrier I was going to recommend. With it's 6 M.2 slots, PEX8748 switch, adjustable fans and real time monitoring...I think it's one of the best NVMe carriers currently on the market and the OP should check it out. I'm considering purchasing one to use in a 5,1. Obviously, I won't be able to use it to it's full potential being that 3 high quality PCIe gen3x4 NVMe will fully saturate my PCIe 2.0 x16 slot. However, I am in the market to upgrade my gen2 Squid anyway and figure the sheer benefit of fitting up to 6 NVMe in a single PCIe slot is worth the price of admission.

 

[ZombiePhysicist]

Really interesting. But this card seems around the same speed. 13GB/sec but it will take 2.5" form factor NVMe drives:

http://highpoint-tech.com/USA_new/CS-product_nvme.htm#bottom

The 2.5" form factor drives for 15.36TB are going for under $2500 now, so price per gb is really good with 3.4GB/sec transfer speed:

http://www.nextwarehouse.com/item/?...MI_u3SjYSV5AIVxfZRCh0IcgSbEAkYASABEgJFo_D_BwE

I think the biggest NVME sticks are currently 2TB, so it is decently close to max it out to 12TB if you can raid the SQUID card and still boot? 2TB sticks go for about $480 or about $2880. Hmm, it is really interesting trade off. Then again, if the Highpoint lets me raid 4 of the 15TB drives, that would be interesting and crazy absurd with 60TB of SSDs that would theoretically work at around 14GB/sec...

 

[ZombiePhysicist]

I just ordered a 28core, 96GB ram, 256SSD (ergo why I need a PCI solution for storage), 580x, wheels (which annoyingly delay shipping by like 5 weeks, for freek'n wheels) 7,1 Mac Pro.


So for storage I thin I'm going with this:

Option A
Highpoint 7120 U.2 card. $400 bux:
https://www.amazon.com/gp/product/B...c45-cd78f07a84d8&pf_rd_r=Z5FT0J2SWTDZZHJHC1F9

And the 15.36 TB NVMe U.2 Micron 9300 pro drive. $2400bux:
http://www.nextwarehouse.com/item/?...MI_o2ewe6r5gIVxcDACh2KqQ9TEAkYASABEgK3IvD_BwE

The micron reads/writes at around 3500MB/sec. The Highpoint in theory would let you raid up to 4 of those together for 60+TB of SSD at probably over 12GB/sec throughput.

Option B
Another option is this Squid that takes 6 NVMe sticks (so you could fill it with 6 2TB sticks for around 12TB and reportedly around 12-14GB/sec read writes:
https://amfeltec.com/pci-express-gen-3-carrier-board-for-6-m2-or-ngsff-nf1-pcie-ssd-modules

Thoughts
I think I like the U.2 form factor better. Not clear if either of those solutions will let you boot from a RAID, but from another thread, it seems you can at least boot with a single drive on the U.2 solution:
https://forums.macrumors.com/thread...r-2019-ncmp-7-1-mac-pro.2189591/post-27538765


TLDR; Anyway, what was the config you ordered and what do you plan to do with regard to PCI SSD storage for your new 7,1 Mac Pro?

Update 2020-02-02:
Just thought I'd update this thread in that I went with the Highpoint 7120 and Micron 9300 Pro 15.36TB drive option. Lot of details of getting it to work found in this thread:

Mac Pro support for NVMe U2 Micron 9300 Pro 15.36TB U.2 Drive SSD?

Ok so I got the drive. I got a bracket and it all seems to fit in decently tidy. Some pics: So I have the Pegasus J2i in there. Swapped the useless 8TB for a 16TB spinner. I got a 3.5" to 2.5" bracket. One note, the screws in the J2i that hold larger drives, are too big for both the 2.5"/3.5"...

forums.macrumors.com

 

[SecuritySteve]

I'm very interested in this topic too. My work figures we need to use PCI SSDs to install ESXi on the MP 7,1 for our virtualization cluster, given that Apple and VMware are not playing nice with T2 chip support yet.

https://www.amazon.com/gp/product/B...d_p=ab873d20-a0ca-439b-ac45-cd78f07a84d8&th=1

I was looking into suggesting this one since it is bootable from raid.

 

[ZombiePhysicist]

I wonder if the 7120 card is raid bootable. If so, having over 60tb of ssd would be crazy useful.

 

[AidenShaw]

I'm very interested in this topic too. My work figures we need to use PCI SSDs to install ESXi on the MP 7,1 for our virtualization cluster, given that Apple and VMware are not playing nice with T2 chip support yet.

https://www.amazon.com/gp/product/B...d_p=ab873d20-a0ca-439b-ac45-cd78f07a84d8&th=1

I was looking into suggesting this one since it is bootable from raid.

My ESXi servers on ProLiants boot from SD cards - same with Dell. ESXi needs less than 2 GB of disk, and is memory resident for the most part after booting. (A couple of times per hour some logs are flushed.)

You might not need PCIe disks.

 

[SecuritySteve]

My ESXi servers on ProLiants boot from SD cards - same with Dell. ESXi needs less than 2 GB of disk, and is memory resident for the most part after booting. (A couple of times per hour some logs are flushed.)

You might not need PCIe disks.

The trouble is that you can't put ESXi on the stock Apple SSDs. The SSD controllers are actually embedded in the T2 chip, instead of on the SSD proper. This means ESXi cannot access the thunderbolt controller lanes either. It's the same problem the Mac mini has.

https://www.virtuallyghetto.com/2018/11/esxi-on-the-new-2018-apple-mac-mini.html

Essentially you can only install it via third party storage. This isn't a horrible thing either though - keeping the base 256GB drive for firmware updates on the board alone is useful.

As far as SD cards, I imagine we are going to have large storage needs. We are using 10 TB and are planning on using as much as 30 or more in the near future for VM storage alone. We could dangle it off the side as a datastore but then the performance tends to tank on the individual VMs.

So, PCIe storage for high performance and large capacity (even if expensive) is the best option.

 

[ZombiePhysicist]

So good news bad news. Good news is 16TB of super fast (for a spinner) enterprise grade storage is pretty cheap now, at around $420:

https://www.amazon.com/Seagate-256M..._1_2?keywords=16tb+exos&qid=1576020729&sr=8-2

The bad news is you have to pay $400 for a 2 drive bay that comes with completely uselss 8TB spinner drive for $399. It's evil because these drives are a complete waste for anyone in this segment. It's pure gouging and for a decent number of people, that 8TB spinner will end up in the trash heap:
https://www.apple.com/shop/product/...2i-8tb-internal-storage-enclosure-for-mac-pro

 

[H. Flower]

What do you guys think of the sonnet card with (4) 2 tb sabrent rockets , for around $1400? Fast for video editing ? Raid 5 for some protection ?

 

[ZombiePhysicist]

What do you guys think of the sonnet card with (4) 2 tb sabrent rockets , for around $1400? Fast for video editing ? Raid 5 for some protection ?

wouLDnt the squid 6 slot card I posted be better/faster and cheaper? Is there something else special about the 4 slot sonnet?

- - Post merged: Dec 11, 2019 - -

One thingo I wonder is anyone making an mpx to 6/8pin power cable adapter? I also want to put an amd 3x 9100 card in my new rig, and it would be cool and tidy to have it take all the power it needs from the mpx part of the pci slot it’s in with some kind of adapter.

 

[H. Flower]

wouLDnt the squid 6 slot card I posted be better/faster and cheaper? Is there something else special about the 4 slot sonnet?

- - Post merged: Dec 11, 2019 - -

One thingo I wonder is anyone making an mpx to 6/8pin power cable adapter? I also want to put an amd 3x 9100 card in my new rig, and it would be cool and tidy to have it take all the power it needs from the mpx part of the pci slot it’s in with some kind of adapter.

Well, 12 Gigs is a little too much for me, and 5 gigs just a hair too small. I'm looking for around 6 gigs storage. 4 slots at 2 gigs gives me 6 at Raid 5. Thats why i was thinking sonnet.

I missed the prices on the squids....are those posted anywhere?

 

[ZombiePhysicist]

Well, 12 Gigs is a little too much for me, and 5 gigs just a hair too small. I'm looking for around 6 gigs storage. 4 slots at 2 gigs gives me 6 at Raid 5. Thats why i was thinking sonnet.

I missed the prices on the squids....are those posted anywhere?

mhmm I don’t see the price on the squid. On the other hand, I don’t think you need to fill all 6slots of the squids. I think you can leave 2 slots open for future expansion