Look Out #1 - F@H

[mmomega]

still, that's one heck of a card. and i thought i read that the 1080 was more power efficient than the 980 ti. not bad. though the 1080s seem to be sold out everywhere

True.
Actually had my 980 Ti pull 700,000-750,000 PPD twice this week. I've decided to wait until I see some 1080 Ti's hitting the market before I upgrade.

I ended up picking up a 2nd used 980 Ti just because I got it at a very good price, cheaper than a 1070 or 2x R9 480's and I know how these cards perform. The latest I read was a 1070 Superclocked card is averaging what my card but is on a an overall much better system.

I have an older PC in the closet at work with two empty PCI slots. I think I'm just going to turn that into my new dedicated Folding machine and see how that does for the rest of the year before I even look at anything else.

 

[mmomega]

I'm in Mexico until tomorrow. New card and water coolers come in by Wednesday I believe. It's a slight frankenstein of old and new parts just thrown together. We'll see how it does by the next weekend.

The old/new Folding Machine is going to be:

Cooler Master Case RC-690-KKA1-GP
MSI Z68MA-ED55 Board
Intel i3 3220 3.3GHz LGA1155
16GB DDR3 1333MHz RAM (I think)
no HDD at the moment (will probably be a very small cheap SSD)
2 x EVGA GTX 980 Ti SC+ ACX 2.0 w/ Backplate
2 x Corsair H55 All-in-One Water Cooler
2 x Kraken G10 Water Cooling Bracket
[doublepost=1466975315][/doublepost]some info from another that put a 1080 Founders Edition in his rig and let it run over night.

So for me, getting both of my 980 Ti's for less than the cost of a 1080 and putting up PPD numbers that aren't too terribly far off. I'll definitely be keeping them for quite a while.
One of mine gets into 750,000PPD territory every once in a while. It's also on my nephews computer at the moment and I've been out of the country since Wednesday so I don't know when he's using it and possibly slowing down F@H a little.

Then I come across this post.

 

[rwh202]

So for me, getting both of my 980 Ti's for less than the cost of a 1080 and putting up PPD numbers that aren't too terribly far off. I'll definitely be keeping them for quite a while.
One of mine gets into 750,000PPD territory every once in a while. It's also on my nephews computer at the moment and I've been out of the country since Wednesday so I don't know when he's using it and possibly slowing down F@H a little.

Not a bad setup there!

I agree that the NV Pascal enthusiasm might be a bit early - I've got a stock 1080 FE and seen PPD ranging from 460k to 860k. I need to install HFM to see what it averages at, but it's still someway from being a 1M PPD card.

However, on the plus side, it is doing about the same as a pair of 980s and for half the power, so there's £200 a year saving from that, but a rather major initial outlay! (fortunately covered by the silly bitcoin price over the past few weeks)

The 480s could definitely be a winner though, might get a pair to replace some 750ti cards. I'm a bit cautious though since I've never managed a 100% stable AMD folding rig having been through 6 cards over the years.

 

[mmomega]

Not a bad setup there!

I agree that the NV Pascal enthusiasm might be a bit early - I've got a stock 1080 FE and seen PPD ranging from 460k to 860k. I need to install HFM to see what it averages at, but it's still someway from being a 1M PPD card.

However, on the plus side, it is doing about the same as a pair of 980s and for half the power, so there's £200 a year saving from that, but a rather major initial outlay! (fortunately covered by the silly bitcoin price over the past few weeks)

The 480s could definitely be a winner though, might get a pair to replace some 750ti cards. I'm a bit cautious though since I've never managed a 100% stable AMD folding rig having been through 6 cards over the years.

I'm still on the fence about snatching up an RX 480 on launch day if possible to let it run for a day or two and see what the outcome is.... I'm just curious. Then return / resell if I need to. I do have another rig it could go in if it turned out to be a very good card for the money. $199 is not bad at all if it comes close to 980-980Ti territory but I'm not convinced yet.

And I remember a i5 2500K I have in a small workstation that the i3 3220 would do just fine in. I may swap those two around on Wednesday just because the 2500 should be able to pump out just a few thousand more PPD.

This stuff can get addicting.

I'm just not letting my wife know about all the new "investments" unless she happens to ask about them. 🙂

 

[rwh202]

Have you gone for an RX 480?

I got as far as adding one to my basket, but chickened out - I've not seen folding numbers, but it looks like 970 levels of performance, but still at 970 power draw (all through just the slot and a single 6-pin connector - eek!).

Also, the price in the UK for the 8GB version is the same as for the clearance 970s so it looks like AMD will have to wait a bit longer before I dip my toes again.

 

[mmomega]

Have you gone for an RX 480?

I got as far as adding one to my basket, but chickened out - I've not seen folding numbers, but it looks like 970 levels of performance, but still at 970 power draw (all through just the slot and a single 6-pin connector - eek!).

Also, the price in the UK for the 8GB version is the same as for the clearance 970s so it looks like AMD will have to wait a bit longer before I dip my toes again.

I didn't... actually did the same as you, I added 1 to my cart two different times and chickened out also.

I did get my new rig running yesterday with the two 980Ti's, now to let them get some burn in and stable out on PPD.
At the moment it's showing approx 1.2Mil PPD.

And since this machines is going to be sitting and running and I didn't really want to worry about it much with heat or anything, I put a Corsair H55 All in One water cooling kit on each GPU and over 24 hours at 94% GPU load while folding the temps have stabled out at about 52C on one card and 41C on the second and both cards are overlocked by +150MHz.

Now I don't want to take the machine down again but I need to switch out those CPU from the i3 to the i5 and PPD should go up just a wee bit.

 

[twoodcc]

well i just happened to be a best buy that had one 1080 on the shelf....and now it's no longer on the shelf.

by the way, i'm still running windows 7 on my folding rigs. is windows 10 better for folding performance, or just keep what i got? both of these systems were built in 2010, with an i7 920/930 i believe.

 

[mmomega]

well i just happened to be a best buy that had one 1080 on the shelf....and now it's no longer on the shelf.

by the way, i'm still running windows 7 on my folding rigs. is windows 10 better for folding performance, or just keep what i got? both of these systems were built in 2010, with an i7 920/930 i believe.

congrats, was it a founders edition card?
Actually I never folded on Win 7. I just got into GPU folding this year and I only ever ran CPU folding in OS X on and off over the years.

 

[twoodcc]

congrats, was it a founders edition card?
Actually I never folded on Win 7. I just got into GPU folding this year and I only ever ran CPU folding in OS X on and off over the years.

thanks! yes, it is a founders edition. so not a cheap card.

what do you GPU fold with? i'm tempted to try linux, but since what i've already got installed seems to be working fine for now

 

[mmomega]

thanks! yes, it is a founders edition. so not a cheap card.

what do you GPU fold with? i'm tempted to try linux, but since what i've already got installed seems to be working fine for now

I'm currently on Windows 10 Enterprise. It's an older work PC, 1 pci slot is x16 and the other is x4 but the i3 CPU is a dual core with 16 PCI lanes total.

I've thought about Linux but I'm just not as familiar with it as Windows or OSX.

 

[twoodcc]

I'm currently on Windows 10 Enterprise. It's an older work PC, 1 pci slot is x16 and the other is x4 but the i3 CPU is a dual core with 16 PCI lanes total.

I've thought about Linux but I'm just not as familiar with it as Windows or OSX.

yeah i've casually tried linux a few times over the years, but never anything serious.

 

[mmomega]

yeah i've casually tried linux a few times over the years, but never anything serious.

So what kind of numbers does that card put out since you've had it running for a while now?

 

[twoodcc]

So what kind of numbers does that card put out since you've had it running for a while now?

not that great. around 500-700k depending upon the work unit. it could be partly due to being in my older system, and running on windows 7, but i'm not sure

 

[mmomega]

not that great. around 500-700k depending upon the work unit. it could be partly due to being in my older system, and running on windows 7, but i'm not sure

Ok, so very similar output as a 980Ti at the moment.

And I'm thinking my older motherboard "may" be a bottle neck as the PCI slots are PCIE2.0 @16 on the main slot and that card outputs roughly 600k. While the 2nd slot is PCIE1.1@4 and it outputs roughly 450k.

And these are identical cards.

When I had this same card on my nephews newer motherboard with an i5 and a PCIE3 @16 slot I would regularly see 700k PPD.

So a motherboard upgrade may be in my future. I can't say for sure when that would be but I'm thinking it's holding me back a good 200k at least.

When I check CPUID HWMonitor, one card is utilizing approximately 65-67% of the bus interface while the other card is 33-38% of the bus interface.

 

[twoodcc]

Ok, so very similar output as a 980Ti at the moment.

And I'm thinking my older motherboard "may" be a bottle neck as the PCI slots are PCIE2.0 @16 on the main slot and that card outputs roughly 600k. While the 2nd slot is PCIE1.1@4 and it outputs roughly 450k.

And these are identical cards.

When I had this same card on my nephews newer motherboard with an i5 and a PCIE3 @16 slot I would regularly see 700k PPD.

So a motherboard upgrade may be in my future. I can't say for sure when that would be but I'm thinking it's holding me back a good 200k at least.

When I check CPUID HWMonitor, one card is utilizing approximately 65-67% of the bus interface while the other card is 33-38% of the bus interface.

that is interesting. i also have a very old motherboard. though i don't think i'll be upgrading in the near future

 

[mmomega]

that is interesting. i also have a very old motherboard. though i don't think i'll be upgrading in the near future

I'll know for sure by mid to end of next week. I have someone that will let me plug my cards into a new Skylake setup and see if there is any performance difference in regards to Folding. Definitely not worth the price to upgrade but for me personally just the testing aspect of it is interesting.

 

[mmomega]

Question for you guys that do a mix of CPU and GPU Folding.

If you are primarily GPU folding, do you assign the CPU a slot to do WU's? Or not configure a CPU slot?
My understanding is that the CPU is still used in the background to assist the GPU's while they are folding or I may be misunderstanding or received incorrect information.

I ask this because on the other rig I had setup I only had GPU's configured to fold and the GPU's were outputting higher PPD.
I just reinstalled FAH on an essentially new build, same GPU's, and I let the FAH client set everything up as it would by default which has the CPU doing WU's and I am getting substantially less PPD overall while this is now a much more powerful system then previously.

It has only been running for 2-3 hours. Should I wait until tomorrow after a few WU's have cycled through and see what the PPD settles at tomorrow? Or would you suggest making any changes?

Just for information: New system:
i7 6700k SkyLake clocked at 4.4GHz
16GB RAM DDR4 3000MHz (though this shouldn't have much of difference on the system.
Dual EVGA GTX 980 Ti Superclocked+ overclocked +130MHz
Samsung 840 Pro SSD

CPU water cooled temps holding at 60C under 100% load across all 8 cores
Both GPU's water cooled temps holding at 44 - 47C under 80 - 87% load on both GPU

At the moment I am getting estimated 300,000 less PPD than on my other older much slower system.

 

[rwh202]

Question for you guys that do a mix of CPU and GPU Folding.

If you are primarily GPU folding, do you assign the CPU a slot to do WU's? Or not configure a CPU slot?
My understanding is that the CPU is still used in the background to assist the GPU's while they are folding or I may be misunderstanding or received incorrect information.

I ask this because on the other rig I had setup I only had GPU's configured to fold and the GPU's were outputting higher PPD.
I just reinstalled FAH on an essentially new build, same GPU's, and I let the FAH client set everything up as it would by default which has the CPU doing WU's and I am getting substantially less PPD overall while this is now a much more powerful system then previously.

It has only been running for 2-3 hours. Should I wait until tomorrow after a few WU's have cycled through and see what the PPD settles at tomorrow? Or would you suggest making any changes?

Just for information: New system:
i7 6700k SkyLake clocked at 4.4GHz
16GB RAM DDR4 3000MHz (though this shouldn't have much of difference on the system.
Dual EVGA GTX 980 Ti Superclocked+ overclocked +130MHz
Samsung 840 Pro SSD

CPU water cooled temps holding at 60C under 100% load across all 8 cores
Both GPU's water cooled temps holding at 44 - 47C under 80 - 87% load on both GPU

At the moment I am getting estimated 300,000 less PPD than on my other older much slower system.

View attachment 640887

I don't CPU fold with GPUs - the effect on PPD is too much to make it worth it. Maybe 30k ppd drop on a 980 thanks to a bit of extra heat floating around the system and extra latency waiting for bus, cpu and memory access.

However, what you're seeing is a bit extreme. You need to leave one thread free per GPU so that it has the resources it needs. The default settings only leave one core free when it detects a GPU present, so with two GPUs you'll need to adjust the CPU slot. I'm guessing that you'll be set to '-1' so you could try setting that to 6 to leave 2 threads free (assuming you're on a quad i7 with hyperthreading - giving 8 threads total). Maybe post the top of a log file to see what it's currently doing.

Hope that helps

Quick edit - just noticed you're system pic - very tidy!

 

[mmomega]

Thanks !!
You should have seen this in the other computer case, it was a nasty looking mess of wiring without much airflow.


--------------------------------------------------
Well I just did two things and this seemed to improve PPD quite a bit.

First was I remembered that I had SLI enabled as I was just benchmarking the system last night to see how well temperatures were going to be.
So I tried to disable SLI and I had no settings in nVidia Control Panel. I am remotely connecting to this computer and it would also not let me change display resolution which was stuck at 3440x1440, the monitor I was using last night to set the machine up.
I did not want to move the machine back to reconnect the monitor and I remembered I had an HDMI emulator in my headless Mac mini, I installed that into the 1st GPU and all settings came back in nVidia Control Panel and I disabled SLI, rebooted the system and left the emulator plugged in.

FAH started back up at login and within 3-5 minutes my estimated PPD for both cards jumped to roughly 500,000 each, so this is much closer to the performance these cards were getting previously. The card with the emulator installed is getting approx 34,000 more PPD (estimated) than the other card. Since they are running 2 different work units I don't know for sure if the emulator being installed is tricking the GPU into working more, which is why I have had it installed in the Mac mini.
I don't want to just pull the emulator and install into other GPU at the moment because I don't know if that may disturb anything and I'd rather wait until the WU is complete on both cards, then switch.
I went ahead and ordered two more emulators anyway, I already need one for another headless Mac mini and I'd like to have a spare laying around. If the GPU's fold better with the emulators then I'll order two more, if not then I'll have my spares.

But for now it seems as though SLI being enabled was causing the two cards to fold at a lower rate. I'll let is run undisturbed for the next few days and see where she settles in at average.


**
Now 1 is at 580k and the 2nd is at 530k. Much more normal.
***
And higher still. 1 GPU at 584k the 2nd at 750k. 1.35Million PPD Estimate at the moment.

 

[mmomega]

Finally broke into the top 10.

Only 9 more spots to go.

 

[twoodcc]

Finally broke into the top 10.

Only 9 more spots to go.

welcome to the top 10! 🙂

just curious - what do you use all those macs for?

 

[mmomega]

welcome to the top 10! 🙂

just curious - what do you use all those macs for?

At home I have an iMac and my wife has one. We come to the conclusion a while ago one desktop wasn't enough.
I have a 12" MacBook for personal and work on the go, when I'm on vacation, traveling, etc. I mainly just remote in to servers and computers to fix things.
I have a Mac Pro 6 Core at my office for work.

And all of the office workstations are Mac Mini's and a handful of hand-me-down iMacs. I used to build my own workstations at the office and after 5-6 years this became more and more, the older ones were showing their age and becoming less stable and becoming more work. I put in a 2010 Mac Mini back in 2010 as a workstation and 5 years later it was still running, 24/7 365. So I slowly started replacing the workstations with Mac Minis, when a workstation would require repair I'd put in a Mini.
We run primarily Windows, so I open a Mac Mini package and immediately format and install Windows10 Enterprise.
They are very small, I can either under-desk mount them or they can mount on the back of a monitor taking up also no space at all. They are very low-power and are great at being little workstations. If/when I ever upgrade them, their resell value is much greater than a PC equivalent also so a better return on them.

So I have about 30 Mac Mini's ranging from 2012/2014 models, 1 - 2010, 1 - 2011, 5 - iMacs 2009 - 2012 models, 3 PC Xeon servers and the MacPro at work.
We went from 3 employees, to 6 and now to almost 40.

2 - iMacs a 2015 (mine) and 2013 (wifes) 27" at home. 3 - MacBooks a 2012 12" retina (mine), 2013 Air (wifes travel), 2015 MacBook Pro 15" (wifes main laptop).

Everyone says I'm the Mac guy and have all the "toys", they don't see that my wife has twice as many devices as I do.

We own our business and it's growing pretty good at the moment, 2 locations with a third opening this year so we're fortunate to have that and the oppurtunities we have. We work hard, long hours and we like our technology. 🙂
Usually when we want to replace or put in place an iMac at work is when I'll upgrade my home iMac. So I get a new model, work buys my older model from me.


For folding though I only have that PC with the dual GPU's that I just built. My nephew occasionally runs FAH on his PC with an R9 380 and on the weekends I'll let the Mac Pro Fold.

 

[twoodcc]

At home I have an iMac and my wife has one. We come to the conclusion a while ago one desktop wasn't enough.
I have a 12" MacBook for personal and work on the go, when I'm on vacation, traveling, etc. I mainly just remote in to servers and computers to fix things.
I have a Mac Pro 6 Core at my office for work.

And all of the office workstations are Mac Mini's and a handful of hand-me-down iMacs. I used to build my own workstations at the office and after 5-6 years this became more and more, the older ones were showing their age and becoming less stable and becoming more work. I put in a 2010 Mac Mini back in 2010 as a workstation and 5 years later it was still running, 24/7 365. So I slowly started replacing the workstations with Mac Minis, when a workstation would require repair I'd put in a Mini.
We run primarily Windows, so I open a Mac Mini package and immediately format and install Windows10 Enterprise.
They are very small, I can either under-desk mount them or they can mount on the back of a monitor taking up also no space at all. They are very low-power and are great at being little workstations. If/when I ever upgrade them, their resell value is much greater than a PC equivalent also so a better return on them.

So I have about 30 Mac Mini's ranging from 2012/2014 models, 1 - 2010, 1 - 2011, 5 - iMacs 2009 - 2012 models, 3 PC Xeon servers and the MacPro at work.
We went from 3 employees, to 6 and now to almost 40.

2 - iMacs a 2015 (mine) and 2013 (wifes) 27" at home. 3 - MacBooks a 2012 12" retina (mine), 2013 Air (wifes travel), 2015 MacBook Pro 15" (wifes main laptop).

Everyone says I'm the Mac guy and have all the "toys", they don't see that my wife has twice as many devices as I do.

We own our business and it's growing pretty good at the moment, 2 locations with a third opening this year so we're fortunate to have that and the oppurtunities we have. We work hard, long hours and we like our technology. 🙂
Usually when we want to replace or put in place an iMac at work is when I'll upgrade my home iMac. So I get a new model, work buys my older model from me.


For folding though I only have that PC with the dual GPU's that I just built. My nephew occasionally runs FAH on his PC with an R9 380 and on the weekends I'll let the Mac Pro Fold.

that's awesome!

 

[mmomega]

that's awesome!

Thank You Sir
[doublepost=1469415030][/doublepost]Since my mind is always wandering. I like to try and get as much production out of machine as I can get.
Tomorrow I may put Ubuntu on my Folding Rig and see what type of and if any kind of improvement there will be.

I picked up about 175-200k PPD by putting those cards in with a better overall system. Now I am interested to see what a different Operating System such as Linux can bring.

I recently completely reinstalled Windows 10 Enterprise with absolutely nothing else installed other than last nVidia Drivers and a few motherboard drivers.

My last few numbers have been 1.3 Million +

Prior to new motherboard and CPU my average was 1 - 1.1 Million.

Folks have been saying an approximate 15% increase by going to Ubuntu or another flavor of Linux. So this equates to roughlt another 195,000 ppd.

So at the end of this week ---- if I get Ubuntu installed ASAP tomorrow somehow fit it in at lunch maybe.
My numbers will dip huge and avg will drop a significant chunk again but if all of this plays out fine.
I should be noting 1.5Million PPD from those two cards in Ubuntu

Here's a current screen shot before I head to bed and stop it from folding in the morning.

Do we have any thoughts on numbers?

Is 1.5million PPD a very reasonable number to expect?

Tune in tomorrow guys. and I hope any others reading this will join on, Team 3446, we are still in the top 50 teams ever.
Every little cpu crunching data at home can help make that number rise as we overtake another team. We are a damn good group of people here. Help me overtake the other teams and move us back up to the 30's.

 

[mmomega]

Ubuntu being installed at the moment.
Hoping for the best here.

---------------
Still wishing FAH would allow GPU folding on macOS.