MP 7,1 Tests with a 1.5TB memory kit for $8,000 in a Mac Pro 7,1 ( 2019 )

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
For the last month or so , my shop has been experimenting with a very inexpensive memory kit that maxes out the official amount of memory for the MP7,1 .

The kit is twelve 128GB 2933 MHz DDR4 ECC Hynix memory modules model HMABAGL7C4R4N-WR T4 purchased for around $660 , per module . The total kit cost was around $8,000 . These are 8 rank modules using a modern , and unfamiliar to the Mac workstation community , density technology called 3DS ; specifically , 8 Gb 3DS . It is important to know if this type of memory is compatible with the new Mac Pro since very large capacity memory modules ( 256GB , 512GB , 1TB , etc ) going forward will embrace 3DS technology .

I decided to take a risk on these modules because they were specifically validated for use with second generation Intel Xeon Scalable processors ( code speak for Cascade Lake Xeons , which our MP7,1 use ) ; complete list available here :


The good news is the memory profiles just fine in macOS Catalina . It's really impressive to see the System Report windows . But simply getting memory to profile is not the same thing as actually having the full amount available reliably . As a System Builder , I always check memory before releasing a newly built or serviced System to client .

The actual results have been mixed and testing this large a configuration has been both challenging and time consuming .

First , there was no current method of verifying the integrity of the 128GB memory modules . The current computer industry wide program , Memtest , choked the first time I ran it on the full 1.5TB , shutting down the Mac and subsequent attempted passes of the entire suite of 14 individual tests never managed to complete even after a hundred hours .

With the assistance of the software publishing house , Passmark , a modified version of Memtest was especially made for large configurations in the new Mac Pro . The individual tests in particular that were shutting down the Mac were Test number 12 ( Random number sequence, 128-bit ) and 13 ( the grueling Hammer Test ) . Altering the Hammer Test step size to a manageable amount and then altering a blacklist configuration file so our MP7,1's logic board is supported ( Test 12 ) permitted the entire suite of tests to pass with a 768 GB ( 6 x 128 GB ) configuration . When one of my stations is free , I'll run the modified Memtest on the entire 1.5TB configuration . It will take quite a long time . Memtest was run in its own USB bootable environment and not within macOS or Mac OS X , as many Mac users are familiar with with the cMPs .

Secondly , I was unable to boot into Windows 10 Pro Workstation with the entire 1.5TB configuration . The 768GB ( 6 x 128 GB ) configuration booted up nicely in both Windows and macOS Catalina and was stable with both . Currently , our Mac Pro 7,1s can only run macOS Catalina and Windows 10 , officially .

Thirdly , the rated 21-21-21 "Y" timings of the modules were not realized , but downscaled to 24-21-21 actual . CAS latency thus was increased , but not by a horrible amount .

Fourthly , I am at a loss of how to actually test this large an amount of memory within a macOS environment . In Windows , the MP7,1 was able to run GIMPS , a Prime Number discovering program and utilized the entire 768GB of available memory . In macOS , either with the 768GB or 1.5TB kits , I just can't seem to utilize very much memory at all . User @tsialex recommended I set up a ramdisk , which is an excellent suggestion . When I have time , this will be attempted . It will be stressful on the modules , but not dangerous . The nice thing about main system memory modules is they are hard to burn out , unlike solid state secondary storage ( SSDs ) . So a good ol' fashioned ramdisk will be just the ticket to test the MP7,1 with memory capacities large enough to truly be useful . I imagine it will prove to be a wicked fast working drive ( where the project is written to in real time ) for video editing .

Fifthly , relative inexpensiveness of this 1.5TB ( 12 x 128GB ) kit compared to contemporary alternatives :

From Apple , $25,000 .
From OWC , $19,000 .
From TransIntl , $18,000 .
From Black Diamond ( Generic and unproven ) , $13,000 .
Snow Tiger's kit = $8,000 .


This initial posting will be updated occasionally with both the 768GB and 1.5TB configurations as various individual tests are completed .

W-3275M ( 28 Core @ 2.5 GHz ) with 768GB ( 6 x 128GB ) tests :
Runs macOS Catalina 10.15.x ? = Yes .
Runs Windows 10 Pro Workstation ? = Yes .
Memory bandwidth ( Memtest86 ) = 13.467 GB/s .
Memory Latency ( Memtest86 ) = 32.067 ns .
Memory Timings ( Memtest86 ) = 24-21-21-47 .
500 GB Ram disk performance ( AJA ) = Reads 4.040 GB/s ; Writes 3.851 GB/s
Modified Memtest86 v 8.3 Pro test , all individual tests = passed ( one pass ) .
Geekbench 3 64 bit Single Core = 4393 . Report : https://browser.geekbench.com/geekbench3/8855622
Geekbench 3 64 bit Multi Core = 89282 . Report : https://browser.geekbench.com/geekbench3/8855622
Geekbench 4 64 bit Single Core = 5241 . Report : https://browser.geekbench.com/v4/cpu/15362826
Geekbench 4 64 bit Multi Core = 77248 . Report : https://browser.geekbench.com/v4/cpu/15362826
Geekbench 5 64 bit Single Core = 1160 . Report : https://browser.geekbench.com/v5/cpu/1651788
Geekbench 5 64 bit Multi Core = 19656 . Report : https://browser.geekbench.com/v5/cpu/1651788

W-3275M ( 28 Core @ 2.5 GHz ) with 1.5TB ( 12 x 128GB ) tests .
Runs macOS Catalina 10.15.x ? = Yes .
Runs Windows 10 Pro Workstation ? = No , boots into Recovery Mode .
Memory bandwidth ( Memtest86 ) = 12.671 GB/s .
Memory Latency ( Memtest86 ) = 33.795 ns .
Memory Timings ( Memtest86 ) = 24-21-21-47 .
500 GB Ram disk performance ( AJA ) = Reads 3.872 GB/s ; Writes 3.595 GB/s
1.0 TB Ram disk performance ( AJA ) = Reads 4.070 GB/s ; Writes 3.878 GB/s
Modified Memtest86 v 8.3 Pro test , all individual tests = not tested yet .
Geekbench 3 64 bit Single Core = 4439 . Report : https://browser.geekbench.com/geekbench3/8855915
Geekbench 3 64 bit Multi Core = 89001 . Report : https://browser.geekbench.com/geekbench3/8855915
Geekbench 4 64 bit Single Core = 5172 . Report : https://browser.geekbench.com/v4/cpu/15365851
Geekbench 4 64 bit Multi Core = 76747 . Report : https://browser.geekbench.com/v4/cpu/15365851
Geekbench 5 64 bit Single Core = 1160 . Report : https://browser.geekbench.com/v5/cpu/1663479
Geekbench 5 64 bit Multi Core = 19528 . Report : https://browser.geekbench.com/v5/cpu/1663479

Some Eye Candy :

s-l1600.jpg


IMG_0437.jpg
 
Last edited:

erroneous

macrumors member
Jul 25, 2004
51
18
Thanks for the details!

When I tested the speed of MacOS ram discs on the Mac Pro, I didn't get very impressive results - about 2-4gbyte/sec. Which is... strange. Does anyone have any more impressive figures?
 

AidenShaw

macrumors P6
Feb 8, 2003
18,547
4,566
The Peninsula
Thanks for the details!

When I tested the speed of MacOS ram discs on the Mac Pro, I didn't get very impressive results - about 2-4gbyte/sec. Which is... strange. Does anyone have any more impressive figures?
RAM disks can be quite CPU intensive (shuffling bytes around in RAM), and may be single threaded. Real disks use DMA memory engines to move data without using CPU cycles, and can be massively parallel.

NVMe has eclipsed RAM disks for many tasks.
 

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
I built a 100 GB RAM disk in the 768 GB main System memory configuration Mac Pro 7,1 and benchmarked it against the 250 GB factory boot drive .

A large file was continuously read and written and after an elapsed time of one minute the results were recorded .

The test file was 16GB 4K Prores 4444 using the AJA drive utility . We are particularly interested in the write performance since a good purpose for a large installed memory kit would be to use it as a working drive for demanding video editing projects .

RAMDiskCreator V 1.2 was the macOS RAM disk utility used .


Results , factory boot drive . 1298 MB/s Write and 2237 MB/s Read :

Screen Shot 2020-03-26 at 5.50.54 PM.png


------------------------------------------------------------------------------------------------------------------------

Results , RAM disk . 3912 MB/s Write and 4013 MB/s Read :

Screen Shot 2020-03-26 at 5.54.08 PM.png


The good news is , the RAM disk is considerably faster .

The not so good news is a four slot NVMe SSD RAID 0 PCIe Card would be even faster and cheaper .
 
Last edited:

bcomer

macrumors member
Jan 25, 2008
84
25
Ottawa
I have a Corsair MP600 2TB SSD that gets 3GB per second reads and writes.

It is a little faster than the internal 1TB SSD.
 
Last edited:
  • Like
Reactions: erroneous

Pressure

macrumors 601
May 30, 2006
4,022
279
Denmark
I built a 100 GB RAM disk in the 768 GB main System memory configuration Mac Pro 7,1 and benchmarked it against the 250 GB factory boot drive .

A large file was continuously read and written and after an elapsed time of one minute the results were recorded .

The test file was 16GB 4K Prores 4444 using the AJA drive utility . We are particularly interested in the write performance since a good purpose for a large installed memory kit would be to use it as a working drive for demanding video editing projects .

RAMDiskCreator V 1.2 was the macOS RAM disk utility used .


Results , factory boot drive . 1298 MB/s Write and 2237 MB/s Read :

View attachment 901594

------------------------------------------------------------------------------------------------------------------------

Results , RAM disk . 3912 MB/s Write and 4013 MB/s Read :

View attachment 901595

The good news is , the RAM disk is considerably faster .

The not so good news is a four slot NVMe SSD RAID 0 PCIe Card would be even faster and cheaper .
The four slot NVMe drive will have a problem with sustained write performance once it runs out of SLC cache.

You would need four Samsung 970 Pro 1TB drives to be somewhat safe as they still use MLC NANDs and not TLC.
 

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
@Snow Tiger Could you do a Geekbench 4 and Geekbench 5 run with 1,5 TB RAM?
Currently , I'm testing with a 768 GB configuration . I'll start testing the 1.5 TB after I'm done with the smaller amount .

Frustratingly , I inexplicably lost the ability to run Test 12 ( Random number sequence, 128-bit ) of the self bootable memtest suite once it was modified to successfully run Test 13 ( Hammer Test ) . Passmark is looking at the debug log file now to determine why this is happening .

Both Passmark and my shop only recently started to examine memory configurations larger than 128 GB , so their software is struggling to test larger configurations .

Has anyone managed to run Windows 10 in a Mac Pro 7,1 with more than 768 GB of installed memory ? I cannot do this and I suspect its a bad driver . I can run Windows in Safe Mode with 1.5 TB of memory .

IMG_0440.jpg

- - Post merged: - -

The four slot NVMe drive will have a problem with sustained write performance once it runs out of SLC cache.

You would need four Samsung 970 Pro 1TB drives to be somewhat safe as they still use MLC NANDs and not TLC.
How large is that cache ? One of my System burn in tests is to continuously read and write 5 GB worth of data on the working drive to saturate the cache in order to obtain the actual available bandwidth .
 
Last edited:

Pressure

macrumors 601
May 30, 2006
4,022
279
Denmark
How large is that cache ? One of my System burn in tests is to continuously read and write 5 GB worth of data on the working drive to saturate the cache in order to obtain the actual available bandwidth .
Sorry, I was thinking about the Samsung 970 Evo. The 970 Pro doesn't rely on SLC cache at all, so can sustain performance longer. The 1TB 970 Evo Plus has 42GB of SLC cache (512GB has 22GB and 256GB only has 13GB) depending on how full the drive is. Write more than that and the drive is entirely at the mercy of the TLC NAND.
 

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
With the support of Passmark , I finally got the entire Memtest suite of 14 tests to run on a Mac Pro 7,1 with 768 GB ( 6 x 128 GB ) of installed memory , for anyone else interested in verifying the quality of a memory kit this large in this Mac . It is currently finishing the final test ( Hammer Test number 13 ) . The current elapsed time is 21 hours for the entire suite to run on a single pass and the final test ( Test 13 ) is around 75 percent finished .

The challenges of this are not to be understated , as the program will not work properly unmodified . Two modifications of the test code of Memtest86 v. 8.3 Pro were required , because of the size of the memory installed and also the platform the test runs on :

1 ) A change in the size of the hammer test step size from 16MB to a size more manageable . The configuration file parameter HAMMERSTEP setting must be changed to : HAMMERSTEP=0x10000000 . The method of performing this change is available here : https://www.memtest86.com/tech_configuring-memtest.html . This is to get Test 13 running .

2) A change to Memtest's logic board blacklist file so that our MP7,1's UEFI firmware is compatible with the test's single CPU mode . Add the following line to blacklist.cfg file found under EFI\BOOT\ of the USB drive : "Mac-27AD2F918AE68F61",ALL,EXACT,TEST12_SINGLECPU . This is to get Test 12 running .
 
Last edited:
  • Like
Reactions: ZombiePhysicist

OkiRun

macrumors 6502
Oct 25, 2019
484
256
Japan
With the support of Passmark , I finally got the entire Memtest suite of 14 tests to run on a Mac Pro 7,1 with 768 GB ( 6 x 128 GB ) of installed memory , for anyone else interested in verifying the quality of a memory kit this large in this Mac . It is currently finishing the final test ( Hammer Test number 13 ) . The current elapsed time is 21 hours for the entire suite to run on a single pass and the final test ( Test 13 ) is around 75 percent finished .

The challenges of this are not to be understated , as the program will not work properly unmodified . Two modifications of the test code of Memtest86 v. 8.3 Pro were required , because of the size of the memory installed and also the platform the test runs on :

1 ) A change in the size of the hammer test step size from 16MB to a size more manageable . The configuration file parameter HAMMERSTEP setting must be changed to : HAMMERSTEP=0x10000000 . The method of performing this change is available here : https://www.memtest86.com/tech_configuring-memtest.html . This is to get Test 13 running .

2) A change to Memtest's logic board blacklist file so that our MP7,1's UEFI firmware is compatible with the test's single CPU mode . Add the following line to blacklist.cfg file found under EFI\BOOT\ of the USB drive : "Mac-27AD2F918AE68F61",ALL,EXACT,TEST12_SINGLECPU . This is to get Test 12 running .
Sounds about right...:cool:
 

ZombiePhysicist

macrumors 65816
May 22, 2014
1,017
733
With the support of Passmark , I finally got the entire Memtest suite of 14 tests to run on a Mac Pro 7,1 with 768 GB ( 6 x 128 GB ) of installed memory , for anyone else interested in verifying the quality of a memory kit this large in this Mac . It is currently finishing the final test ( Hammer Test number 13 ) . The current elapsed time is 21 hours for the entire suite to run on a single pass and the final test ( Test 13 ) is around 75 percent finished .

The challenges of this are not to be understated , as the program will not work properly unmodified . Two modifications of the test code of Memtest86 v. 8.3 Pro were required , because of the size of the memory installed and also the platform the test runs on :

1 ) A change in the size of the hammer test step size from 16MB to a size more manageable . The configuration file parameter HAMMERSTEP setting must be changed to : HAMMERSTEP=0x10000000 . The method of performing this change is available here : https://www.memtest86.com/tech_configuring-memtest.html . This is to get Test 13 running .

2) A change to Memtest's logic board blacklist file so that our MP7,1's UEFI firmware is compatible with the test's single CPU mode . Add the following line to blacklist.cfg file found under EFI\BOOT\ of the USB drive : "Mac-27AD2F918AE68F61",ALL,EXACT,TEST12_SINGLECPU . This is to get Test 12 running .
So would you end up testing all of the 1.5TB by doing it on 768GB in one run, remove it, put in the other 768GB, and then run it on that, and if it clears both, then you can install all of it and assume all the memory is good?
 

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
So would you end up testing all of the 1.5TB by doing it on 768GB in one run, remove it, put in the other 768GB, and then run it on that, and if it clears both, then you can install all of it and assume all the memory is good?

I am deliberately working with 768GB to examine it's performance ( and also it is the largest configuration with at least this model of memory to run Windows 10 in bootcamp ) .

Once I am done with the 768GB , I'll install the 1.5TB and test that .
 

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
Completed the Geekbench 3 , 4 and 5 results for the 768GB configuration . Links to the full reports on my Geekbench account provided for those that want to look into the results in finer detail .
- - Post merged: - -

Interesting that the top of the line Mac Pro 7,1 28 Core @ 2.5 GHz with 768 GB ( 6 x 128 GB ) of memory is around three times as powerful as the fantastically desirable Mac Pro 5,1 12 Core @ 3.46 GHz with 96 GB ( 6 x 16 GB ) as recorded by Geekbench 3 64 Bit Multicore .
 
Last edited:

Snow Tiger

macrumors 6502a
Original poster
Dec 18, 2019
653
471
@Snow Tiger Could you do a Geekbench 4 and Geekbench 5 run with 1,5 TB RAM?
done .
- - Post merged: - -

So would you end up testing all of the 1.5TB by doing it on 768GB in one run, remove it, put in the other 768GB, and then run it on that, and if it clears both, then you can install all of it and assume all the memory is good?
Currently running Memtest86 with 1.5TB memory installed . It will take quite a while to even complete the first pass .
 
  • Like
Reactions: ZombiePhysicist
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.