Storage Solution & RAM Drive Slow?

[Forbidden Era]

Hey guys,

Looks like everyone has been hard at work with the old Mac Pro's.. I can't believe some of the things I am seeing in here..! I love that I am not the only one who knows that these computers can still have a little life left in them. Thunderbolt support seems like it's possible/being worked on, NVMe boot has been done..!

Anyhow, I have a 3,1.. I had 14GB of 800mhz in it, but I thought I'd try and see how it runs with 32gb of 667. Crammed it in, booted, fine, great, built a ram drive and tested..

I am only getting like 1200-1400MB write and like 700MB read..from a RAM DRIVE!#$

So, I am seriously considering adding some NVMe drives to my system. I can get a 970 Evo for like $160 for 500gb or $100 for 250gb, not too bad. I would eventually want 4 of these, possibly in some sort of RAID, on one of the carrier cards (amfeltec or highpoint) but the price of these cards is a bit much for me right now. I was hoping to use the SSD with just a basic M2->PCIe adapter for now.

I do love how those cards can do x4 3.0 -> x16 2.0 kind of thing.. These new PLX chips are cool, LTT was using some boards with PLX chips for graphics expansion trying to get like 7-8 high end GFX cards to pass thru to VMs in one system.

But anyway, I am worried that I won't see the performance that is possible. I am sure anything will be an upgrade from my 4x250gb raid0 which is barely pulling 200MB/s sequential anymore. My haxServe was running 4x300gb 15k SAS with 600MB/s sequential which wasn't bad..but it's time for SSDs in the workstations now..The laptops all have them.

If I can't get even 1500/1500 off a RAM drive, than solid state is a worry!

Anyone have any ideas what could possibly be causing this? Is my RAM and subsystem really just that slow? I mean, I am sure I have seen better benchmark results off people with SSDs and a 3,1 than I am posting here for a RAM drive..if it can't be moved in and out of ram faster than an SSD can handle, wouldn't the ram be the bottleneck?

I can throw my 800mhz back in and bench if needed but I am not sure that would make a huge diff. Running latest Yosemite if that makes any diff. This system was on Mavericks for a longtime and I only just went to Yosemite, always scared of breaking such an old system with an upgrade..! That, and, with Apple it's not always an UPGRADE..!

 

[h9826790]

How you create the RAM drive?

 

[Forbidden Era]

Last login: Tue Nov 13 23:04:44 on ttys000
Forbiddens-Pro2:/ forbiddenera$ diskutil erasevolume HFS+ "RamDisk" `hdiutil attach -nomount ram://8388608`
Started erase on disk8
Unmounting disk
Erasing
Initialized /dev/rdisk8 as a 4 GB case-insensitive HFS Plus volume
Mounting disk
Finished erase on disk8 RamDisk
[doublepost=1542345890][/doublepost]I am seriously considering buying a 970 EVO 512gb .. !

 

[h9826790]

Last login: Tue Nov 13 23:04:44 on ttys000
Forbiddens-Pro2:/ forbiddenera$ diskutil erasevolume HFS+ "RamDisk" `hdiutil attach -nomount ram://8388608`
Started erase on disk8
Unmounting disk
Erasing
Initialized /dev/rdisk8 as a 4 GB case-insensitive HFS Plus volume
Mounting disk
Finished erase on disk8 RamDisk
[doublepost=1542345890][/doublepost]I am seriously considering buying a 970 EVO 512gb .. !

So, you use 1GB test size?

Did you let it run few cycles to get a stabilised result?

Did you check if there is any CPU single thread limiting process in activity monitor?

 

[Forbidden Era]

So, you use 1GB test size?

Did you let it run few cycles to get a stabilised result?

Did you check if there is any CPU single thread limiting process in activity monitor?

1. The screenshot is from 2gb, although I tried various sizes with no different result.

2. Yes, the initial test before anything is cached does about 500-700 write and 700 read, so read isn't affected but write speed doubles after a cycle or so (assuming it is cached) and goes to ~1200

3. Not sure exactly what you are wanting me to check for.. There are no processes eating CPU, blackmagic is using about 30% during test or so..no single core is pegged.

[doublepost=1542362608][/doublepost]I'm having a hard time figuring out what numbers I should be getting, but I would think at LEAST 3000/4000?

I don't see how my read is so slow compared to my write. Although, as I said, on the initial run, the write is about equivalent to the read. If I do anything much, like, drag a window around, the speeds drop a bit.
[doublepost=1542363030][/doublepost]The current memory config is server-grade 8 x 4GB 667 FB-DIMM.

Previous config at 800mhz was Kingston and some other brand, with the big Apple style headspreaders, 2x4, 2x2, 2x1. I didn't bench a RAM drive in this config though. I doubt 667mhz vs 800mhz would make a huge difference - I must be seeing a bottleneck elsewhere..?

PCIe is:
Slot4: empty
Slot3: empty
Slot2: Quadro 2000 -> 2 screens (1x DP, 1x DVI)
Slot1: GeForce 560Ti -> 2 screens (1xDVI, 1xDVI->HDMI)

I do have the BT card. I do not have the wifi card.

 

[Forbidden Era]

Just in the thought, of simplicity, for now... While the SATA ports are SATA II, they are limited to 300MB/s per port right? So, if I used the two free SATA ports and RAID0 some cheap 2.5" SSDs I should be able to get maybe 550/550ish..not terrible..not great I guess..

I can get a 970EVO 250gb for like $100 right now and 500 for $170, so I wouldn't mind getting one but I'd need a way for it to fit/work - I would probably eventually get something like the highpoint card and run 4 of these SSDs, but for now - what's a good adapter to just get it to run/boot?
[doublepost=1542368233][/doublepost]I do get good transfer speed from mem to both x16 slots..? at least according to Cuda-Z?

You'd think based on this I should be able to see at least 3000MB to the ram drive?

I should try a benchmark for raw ram read/write..but the RAM drive is still an interesting test, especially since I am planning on adding flash storage - I'm hoping to see like 1500/1500 out of a new SSD if I can.

When the Quadro2000 was in slot 3, I was getting like 750mb/s only..(pcie1 speeds?)

after I figure out the Pro, I may have to figure out how to upgrade my MBP8,2..It has a 250 and 500gb sata 2.5" ssds in RAID0 atm..haven't done much testing to see if the mixed drives are any faster than a single..the 250 was my first ssd ever, prob 4 years old or so, the 500 came with the mbp but was used..I'm curious what the best solution for the mbp would be...perhaps putting an SSD in the WiFi slot might be an idea? otherwise I think I'm stuck with whatever's fastest that I can cram into the two sata ports..I think the 8,2 is sata3 tho? so if the two ports aren't shared I should be able to see 1000/100 maybe in raid0..

Right now tho, the MBP isn't bad, faster than the RAID0 in the Mac Pro..The 250gb ssd alone in my old 2009MBP about matched the sequential of the 4x250gb raid0 in the pro when I first got it. The newer 500gb ssd in the 8,2 is definitely faster though so I think raiding it with the 250 might actually be bottlenecking it but I was curious to try.

But I really want to update the cMP.. I think for now I might order a card like the DT120? I'm curious, will any adapter card work? I saw one that takes two SSDs, one M2 PCIe and a SATA one with a port on the back..I could maybe run 2 of those in 3,4 slots with 2 nvme and 2 sata with the sata going back to the extra optical ports..at least I wouldn't have to worry about power for the SATA..but I have two graphics cards and I do want to add USB3.1, and maybe T-Bolt which seems like might be possible soon? so.. ugh..

Definitely not easy but dammit, this cheese grater will live on! At least until I find a good 5,1, and then I am sure it'll still be a good secondary.
[doublepost=1542368892][/doublepost]With AJA and disk cache off:

AJA disk cache on:

 

[h9826790]

That ~3000MB/s in CUDA-Z is irrelevant. That's not measuring the system RAM speed.

And it seems the DDR2 speed limit is around 1500MB/s when using as RAM drive.

 

[startergo]

That ~3000MB/s in CUDA-Z is irrelevant. That's not measuring the system RAM speed.

And it seems the DDR2 speed limit is around 1500MB/s when using as RAM drive.

It is more than that

 

[h9826790]

It is more than that

Is that from 667MHz DDR2 on a 3,1?

 

[startergo]

Is that from 667MHz DDR2 on a 3,1?

from my hardware in the signature DDR3

 

[h9826790]

from my hardware in the signature DDR3

Then it's not related to OP's issue.

 

[Forbidden Era]

That ~3000MB/s in CUDA-Z is irrelevant. That's not measuring the system RAM speed.

And it seems the DDR2 speed limit is around 1500MB/s when using as RAM drive.

CUDA-Z shows "Memory copy" from "host to device" - as in, copying from system ram to device.

It seems? Are my results in line with what other people have seen?

I am sure I have seen someone post better results for an SSD inside a 3,1, how can you get faster read/write from an SSD than a RAM drive? Unless something is specifically slowing down the RAM drive.

I suppose I will pop in my 800mhz FB-DIMMS and see if that makes much difference, but really I was wondering what bottlenecks could be causing me to only see about 750/750 from memory? (remember the ~1200 or so write speeds were likely cached as the first cycle shows ~750)

I'm sure adding the SSD will improve performance regardless but I would like to see better than 750/750 out of it or else I would just add two 2.5" on the SATA ports and RAID0 them for simplicity sake. AHCI blades would be a great option but it seems like I can do $130-180 for a 500GB 970EVO or $500+ for any AHCI blade..!

Is there any good performance hacks for 3,1?

My goals:
- get a boot screen so I can debug stuff, bootcamp/linux (I think I read before it's possible using a hackintosh style bootloader, but I can't seem to find any info..I posted another thread about this..I'd be willing to flash my 560ti if I can figure out/find instructions on how to create my own proper EFI for it..I have a bunch of other cards I could potenitally rip an eeprom from if needed and have programmers tho nvflash should do it amirite? I did it for my fx4500 for my 1,1->2,1 before)
- add SSD for some HD performance. boot is required. fiilevault not necesarily required. fusion drive might be neat to try but not needed. recovery would be nice but I've lived without it until now too (apple sw raid0 4x250gb hd)
- OS X is my primary OS but I would like to boot windows native for games, linux native for other testing might be nice but I do have a big linux server
- add USB3,1 - not a huge priority, I don't have any USB3 devices but that's because I don't have any USB3 hosts probably..would like to future proof more..
- add thunderbolt - never thought this would ever be possible but seems like people are figuring it out!? omfgz

So..

I am curious, if someone more knowledgable can answer - now that we can modify/add drivers to the EFI for our systems, is there a way we can add the necesary stuff to the EFI to initialize a graphics card (with standard UEFI) on boot instead of flashing the graphics card itself?

 

[h9826790]

CUDA-Z shows "Memory copy" from "host to device" - as in, copying from system ram to device.

It seems? Are my results in line with what other people have seen?

I am sure I have seen someone post better results for an SSD inside a 3,1, how can you get faster read/write from an SSD than a RAM drive? Unless something is specifically slowing down the RAM drive.

I suppose I will pop in my 800mhz FB-DIMMS and see if that makes much difference, but really I was wondering what bottlenecks could be causing me to only see about 750/750 from memory? (remember the ~1200 or so write speeds were likely cached as the first cycle shows ~750)

I'm sure adding the SSD will improve performance regardless but I would like to see better than 750/750 out of it or else I would just add two 2.5" on the SATA ports and RAID0 them for simplicity sake. AHCI blades would be a great option but it seems like I can do $130-180 for a 500GB 970EVO or $500+ for any AHCI blade..!

Is there any good performance hacks for 3,1?

My goals:
- get a boot screen so I can debug stuff, bootcamp/linux (I think I read before it's possible using a hackintosh style bootloader, but I can't seem to find any info..I posted another thread about this..I'd be willing to flash my 560ti if I can figure out/find instructions on how to create my own proper EFI for it..I have a bunch of other cards I could potenitally rip an eeprom from if needed and have programmers tho nvflash should do it amirite? I did it for my fx4500 for my 1,1->2,1 before)
- add SSD for some HD performance. boot is required. fiilevault not necesarily required. fusion drive might be neat to try but not needed. recovery would be nice but I've lived without it until now too (apple sw raid0 4x250gb hd)
- OS X is my primary OS but I would like to boot windows native for games, linux native for other testing might be nice but I do have a big linux server
- add USB3,1 - not a huge priority, I don't have any USB3 devices but that's because I don't have any USB3 hosts probably..would like to future proof more..
- add thunderbolt - never thought this would ever be possible but seems like people are figuring it out!? omfgz

So..

I am curious, if someone more knowledgable can answer - now that we can modify/add drivers to the EFI for our systems, is there a way we can add the necesary stuff to the EFI to initialize a graphics card (with standard UEFI) on boot instead of flashing the graphics card itself?

VRAM isn’t not system RAM.

Cache not necessary make it faster. IMO test, turn off cache will make the RAM drive run faster in AJA. As you can see, the write speed drop to 560.

Anyway, RAM drives on cMP CANNOT run faster then NVMe (unless the NVMe is bottlenecked), the strong area of the RAM drive is the random read / write performance. Not the sequential speed.

And since even DDR3 only five ~2500MB/s, and you can achieve 1200MB/s on DDR2. That looks very reasonable to be.

For your info, DDR2 667MHz DIMM’s bandwidth is almost exactly half of DDR3 1333MHz DIMM can do.

Even though I also believe that ~750MB/s is a bit low. But from AJA, you can see that both read and write can achieve ~1500 range, means the RAM drive itself is normal. Just the software / cache etc make the difference.

Anyway, if the RAM drive is just 4GB in size, and you can fill it up in 3 seconds. I don’t thing it’s worth to spend too much time to analyse if that’s running too slow.

If that’s enough for your workflow, use it.

If that’s not enough for your workflow, go for NVMe or most likely you may need a new computer actually.

Also, the actual difference between 800 vs 667 may be much less than what you believe becasue they run with difference latency.

 

[Forbidden Era]

Fair enough. I just figured a RAM drive should be faster than any NVMe could be.

I want NVMe for an upgrade to my boot drive. I guess I'll grab an adapter card and a 970.

I can use any cheap adapter card if I just want to run 1 SSD for now right?

 

[h9826790]

Fair enough. I just figured a RAM drive should be faster than any NVMe could be.

I want NVMe for an upgrade to my boot drive. I guess I'll grab an adapter card and a 970.

I can use any cheap adapter card if I just want to run 1 SSD for now right?

Yeah, just need a cheap adaptor, however, make sure you have the correct OS which has NVMe driver. Also, check your PCIe slot config (both width and link speed). Without PCIe 2.0x4, the NVMe won’t able to achieve 1500B/s.

 

[Forbidden Era]

What OS is required for NVMe?

I plan to run OS X - currently I have 10.10.5 and possibly Windows 10 - whatever is most recent.

I would guess maybe 10.10.5 doesn't support NVMe? Or at least 3rd party? There is NVMe listed in System Profiler..?

I am pretty adverse to upgrading if I can avoid it. Going from Mavericks to Yosemite on my cMP was scary, but did turn out fine, although everytime I upgrade I loose something in compatibility. I believe in Yosemite, the control panel app for one of my pro firewire sound devices is no longer compatible (digi 002)..but if I can get TB3 going and some NVMe and such, then I can get a nice TB sound device! Fortunately my ProFire 610 is still fully supported, at least in Yosemite..

I do know I may have to set my PCI slots to 2.0 in my 3,1 to get full 2.0x4 - I would check this indeed. In fact, I only really use my Quadro2000 for extra screens and I don't think I'd loose much if any performance for desktop rendering if I switched that to a x4 slot anyway, so I could potentially use slot3 for the NVMe.

I wonder if I could use the ASUS adapter. I suppose it might work, even with multiple cards, just, they would probably negotiate low? I only thinking because it's the only one my local compshop has in stock ATM. I'll prob just order something cheaper though..any recommendations?

 

[h9826790]

What OS is required for NVMe?

I plan to run OS X - currently I have 10.10.5 and possibly Windows 10 - whatever is most recent.

I would guess maybe 10.10.5 doesn't support NVMe? Or at least 3rd party? There is NVMe listed in System Profiler..?

I am pretty adverse to upgrading if I can avoid it. Going from Mavericks to Yosemite on my cMP was scary, but did turn out fine, although everytime I upgrade I loose something in compatibility. I believe in Yosemite, the control panel app for one of my pro firewire sound devices is no longer compatible (digi 002)..but if I can get TB3 going and some NVMe and such, then I can get a nice TB sound device! Fortunately my ProFire 610 is still fully supported, at least in Yosemite..

I do know I may have to set my PCI slots to 2.0 in my 3,1 to get full 2.0x4 - I would check this indeed. In fact, I only really use my Quadro2000 for extra screens and I don't think I'd loose much if any performance for desktop rendering if I switched that to a x4 slot anyway, so I could potentially use slot3 for the NVMe.

I wonder if I could use the ASUS adapter. I suppose it might work, even with multiple cards, just, they would probably negotiate low? I only thinking because it's the only one my local compshop has in stock ATM. I'll prob just order something cheaper though..any recommendations?

You better study the NVMe thread

All your questions are answered / discussed there multiple times.

And the quick answer to you is NO. Your plan won’t work (in both hardware and software). You really have to do some study BEFORE start to buy anything.

 

[Forbidden Era]

I have been studying.

Picked up a 970 PRO today.

Anyone suggest any cheap adapter cards, preferably on amazon prime for quick shipping ? I may just get a dt-120 tho..

 

[handheldgames]

I have been studying.

Picked up a 970 PRO today.

Anyone suggest any cheap adapter cards, preferably on amazon prime for quick shipping ? I may just get a dt-120 tho..

There are many options better than the dt-120 as it's design is dated. It's best to look for an option with an integrated heatsink.

Some general info...

RAM disks and PCIe SSD's have limited performance compared to a 4,1/5,1 due to CPU and RAM bandwidth in the 3,1.

Write speed for ram disks can be increased by using a raid 0.

Performance of the ramdisk will drop with residual heat build up in the 3,1.

 

[h9826790]

I have been studying.

Picked up a 970 PRO today.

Anyone suggest any cheap adapter cards, preferably on amazon prime for quick shipping ? I may just get a dt-120 tho..

Won't work in 10.10.5