Audio Decoding KILLS MacPro (2009) green factor

[lbodnar]

Interesting.. So for simplicity, Tcase is always somewhere in between the two values? What would typical Rdc and Rch values be?

Very rough estimation is Rdc ~ 0.1-0.2 K/W and Rch ~ 0.02-0.08
They can be deduced from actual hardware monitor measurements.

The whole idea of Intel thermal profiles is second-guessing core temperature based on Tcase and keeping it (Tcore) in control. The profiles are sloped because of the component related to dissipated power.

Here is what W3500 datasheet says:

The processor thermal specification uses the on-die Digital Thermal Sensor (DTS) value
reported via the PECI interface for all processor temperature measurements. The DTS
is a factory calibrated, analog to digital thermal sensor. As a result it will no longer be
necessary to measure the processors case temperature. Consequently, there will be no
need for a Thermal Profile specification defining the relationship between the
processors TCASE and power dissipation.

However, the question is: does SMC report actual heatsink temperature or does it second-guess what Tcase would be and reports that?

 

[lbodnar]

For those who have kernel panic on high CPU load - have you tried reducing CPU voltage a little bit?

If you have CHUD Tools installed try running

reggie_se -r -n IA32_PERF_CTL -i a

this will tell you your current PLL multiplier and core voltage settings. Try dropping voltage a few notches down and see if this helps. The command to do that is

sudo reggie_se -w [I]0x91E[/I] -n IA32_PERF_CTL -i a

here 9 is PLL multiplier (CPU core speed) and 1E is core voltage (dropped from 26.) These values are hexadecimal and relate to 3.0GHz X5365 Xeon. W35xx will have different multiplier and voltage but the idea is the same. Let us know if this helps. If you drop voltage too much you will start getting kernel panic again.

This will reset back after restart or wake-up from sleep.

 

[smacman]

Very rough estimation is Rdc ~ 0.1-0.2 K/W and Rch ~ 0.02-0.08
They can be deduced from actual hardware monitor measurements.

The whole idea of Intel thermal profiles is second-guessing core temperature based on Tcase and keeping it (Tcore) in control. The profiles are sloped because of the component related to dissipated power.

However, the question is: does SMC report actual heatsink temperature or does it second-guess what Tcase would be and reports that?

Ok. So what I am getting out of this, is that under load, the CPU Diode temperature will always register a slightly higher value than the actual Tcase due to thermal resistance. If my Diode temp was just a few degrees above 67.9C, this would be good news. In my case, I think I'm still out of Intel's thermal profile with diode temps as high as 86C seen.

 

[AZREOSpecialist]

Question for AZREOSpecialist:

Your results got me thinking, and I seem to recall you did some CPU transplants in your machine. Is your current W3580 chip installed by you or did you order a new machine from Apple? I'm thinking that perhaps the reason your temps are so much lower, is that with your self installed CPU, you probably did a much better job applying thermal paste than Apple did.

In any case, I'm curious to see if your temps remain ok when your cores are working hard. Any observations? In my case, audio + core stress = kernel panic. It seems like when Intel thermal protection logic kicks in, OSX kernel panics. In Windows, I observed what I think was CPU throttling. When running a stress test, one core reached 99C, then suddenly all four cores cooled down to 85C and slowly worked back up to 94C. Either way, audio should not be causing any significant rise in temps in OSX.

Lastly, I am still a little grey on which sensor equates best to Intel's Tcase. Is it CPU A Heat Sink or CPU A Temperature Diode?

Your temperatures are definitely a lot higher than mine, and the kernel panic when stressing your system would indicate a problem. I don't know whether the CPU A Temperature diode reads the same temperature as Intel's Tcase parameter, but it's as close as I'm able to get. The Tcase_max specification refers to the case temperature, not the core temp, so it's going to be either the CPU Temperature diode or the Heat sink temperature diode. Since the Mac Pro's heat sink sits atop a series of temperature monitors on the CPU board, I'm assuming the heat sink temperature reading is actually the heat sink and not the CPU case.

I did switch out my stock W3520 for a Core i7 975 Extreme and then later a W3580. When I removed my W3520, the heat spreader as well as the bottom of the heat sink appeared evenly coated with thermal paste. When I applied thermal paste to my CPUs, I used these instructions. See Core i7 application in lower right corner - that is the correct way of applying thermal paste to Core i7 and Nehalem CPUs. If there's nothing else wrong with your system, you may be suffering from a poor application of thermal paste. I believe Apple had this problem in the past with some Mac Book Pros, and in that case it was too much thermal paste causing overheating.

I would have it checked out at an Apple Store or authorized service center. Or, if you are brave, you can do it yourself! It's easy on a Quad, more complicated on an 8-core rig. On a quad, just pull out your CPU tray, unscrew the heat sink and pull it off, then unlatch and remove the CPU. Use rubbing alcohol and a Q-tip to remove the old thermal paste and re-apply using Arctic Silver 5 or better. That's an easy home remedy that you can try which shouldn't cost you more than $15 in parts and supplies and about 30 minutes of your time. Who knows, maybe that will fix your problem?

 

[Max(IT)]

Ok. So what I am getting out of this, is that under load, the CPU Diode temperature will always register a slightly higher value than the actual Tcase due to thermal resistance. If my Diode temp was just a few degrees above 67.9C, this would be good news. In my case, I think I'm still out of Intel's thermal profile with diode temps as high as 86C seen.

smacman your seems to be one of the worst scenario here: 86°c is very high.
Maybe your MP has a different problem that this "audio bug" (I mean, something else other that the audio bug that is in every MP) ?

I can't reach 60° C on our Quad, after 20 mins of iTunes (with several light applications in background), and I saw people reporting temperature in the 50-60°C range, here and in the Apple's forum.

With 86°C only playing music I can understand your attitude related to this issue

 

[AZREOSpecialist]

Either of them can be used to estimate Tcase.

Tcase=Tdiode - Rdc * P
Tcase=Theatsink + Rch * P

Rdc = thermal resistance diode-case (K/W)
Rch = thermal resistance case-heatsink (K/W)
P = CPU dissipated power (W)

I have left out thermal inertia component so this assumes some stable conditions.

So using these equations, Tcase based on CPU A Temperature diode is Tdiode minus some quantity. Estimating Tcase using the heat sink temperature is Theatsink plus some quantity. So in essence, the real Tcase temperature is always LESS by some variable quantity than what we see with CPU A Temperature diode (Tdiode), and greater by some variable quantity than what we see with the heat sink temperature (Theatsink). So it will always be somewhere between the CPU temp and heat sink temp. Based on my 53C CPU temp and 43C heat sink temp, I'm assuming my actual Tcase was somewhere around 48C.

Smacman, this makes sense. Most likely your actual Tcore temps aren't hitting 75C, but approach 67C or whatever is the limit on your particular CPU. You get a kernel panic as soon as the CPU starts to shut itself down. I am puzzled by the fact that OS X goes into a kernel panic when the CPU hits this state instead of recognizing what's happening and alerting the user prior to a forced shut down. This could very well be a thermal paste issue and worth a few bucks and time to see if that's all you need to fix the problem.

 

[lbodnar]

You can check how close your core temperatures to the Tcore limit by running the following command in terminal

reggie_se -r  -n IA32_THERM_STATUS -i a | awk '/IA/{print "0x" substr($2,13,2)}'

it shows hexadecimal value in ºC for each core. E.g. 0x20 is 32ºC below limit, 0x1 is just 1ºC below limit. The limit is usually 100ºC

 

[AZREOSpecialist]

You can check how close your core temperatures to the Tcore limit by running the following command in terminal
it shows hexadecimal value in ºC for each core. E.g. 0x20 is 32ºC below limit, 0x1 is just 1ºC below limit. The limit is usually 100ºC

Temperature Monitor shows individual core temp readout after installing some Intel-specific drivers. On average my core temps are 10C higher than the CPU temperature diode reading. At 53C CPU temp, my cores bounced around between 61C-63C. This while playing iTunes for 20 minutes w/ equalizer and visualizer on and playing a 1080p trailer streamed over the internet.

 

[AZREOSpecialist]

For those who have kernel panic on high CPU load - have you tried reducing CPU voltage a little bit?

If you have CHUD Tools installed try running

reggie_se -r -n IA32_PERF_CTL -i a

this will tell you your current PLL multiplier and core voltage settings. Try dropping voltage a few notches down and see if this helps. The command to do that is

sudo reggie_se -w [I]0x91E[/I] -n IA32_PERF_CTL -i a

here 9 is PLL multiplier (CPU core speed) and 1E is core voltage (dropped from 26.) These values are hexadecimal and relate to 3.0GHz X5365 Xeon. W35xx will have different multiplier and voltage but the idea is the same. Let us know if this helps. If you drop voltage too much you will start getting kernel panic again.

This will reset back after restart or wake-up from sleep.

Can this method be used to INCREASE the core speed and voltage as well? If so, how would we go about doing that? Also, is there any way to write these values into NVRAM or file so that the Mac can use these as default values at startup?

 

[lbodnar]

Can this method be used to INCREASE the core speed and voltage as well? If so, how would we go about doing that? Also, is there any way to write these values into NVRAM or file so that the Mac can use these as default values at startup?

You cannot increase CPU multiplier (core speed) above the marked speed, only decrease it (for energy saving.) However I have heard rumours that engineering samples can be clocked over the marked speed. I think the same applies to voltage. CPU uses default values after reset including wake-up from sleep. I guess making them "sticky" requires something like kext that would update them each time CPU is reset.

 

[handheldgames]

FWIW... The i7 975 is unlocked. Not sure about the w3580.

 

[rtrt]

1

 

[lbodnar]

So if i understand this correctly you can underclock and undervolt with a single terminal command?

Appreciate that the defaults are reinstated when power or sleep cycled - but many mp owners dont do this for a number of days/weeks even. So could be useful in certain circumstances.

Assuming answer to above is yes then is it valid for all MPs? Have a 2008 2.8 x 8

Yes, sure. This is supposed to be used for dynamic power control but OS X does not seem to use it at least on MacPros so any setting stays until CPU resets. I suggest you try yourself and see if it works. I can downclock X5365 in steps from 3.00GHz to 2.00GHz but decreasing voltage is much more beneficial.

 

[BillB50]

Just thought I would also bring up something not mentioned so far - Apple's installation of 8 Core Xeon CPUs in the MPs uses an installation WITHOUT the usual CPU cover - as those installed in CPUs purchased from a vendor.

Please see this article:
http://www.anandtech.com/mac/showdoc.aspx?i=3597

Direct link to pictures of the CPU as installed:
http://www.anandtech.com/mac/showdoc.aspx?i=3597&p=10

There is also some excellent photos of the heat sinks and fans and some information on cooling...

-bill

 

[Inconsequential]

I'm very interested! Ideally, we'd be able to keep the CPU A Temperature Diode below 67.9 under any conditions.

Ok well, i'll upload a test shortly then.

No idea if it will work in 64 bit kernal tho. I hope so :/

 

[smacman]

Ok well, i'll upload a test shortly then.

No idea if it will work in 64 bit kernal tho. I hope so :/

Thanks CR! This might be the only thing that makes my Mac Pro thermally stable..

 

[lbodnar]

Thanks CR! This might be the only thing that makes my Mac Pro thermally stable..

Try lowering core voltages - this should reduce generated power.

 

[smacman]

smacman your seems to be one of the worst scenario here: 86°c is very high.
Maybe your MP has a different problem that this "audio bug" (I mean, something else other that the audio bug that is in every MP) ?

I can't reach 60° C on our Quad, after 20 mins of iTunes (with several light applications in background), and I saw people reporting temperature in the 50-60°C range, here and in the Apple's forum.

With 86°C only playing music I can understand your attitude related to this issue

Although my temps are definitely higher than some peoples reported temps, they don't seem to be out of the realm. You may have misunderstood me however. 86C was my CPU A Temperature Diode reading when I was doing a CPU stress test. Playing audio results in temps in the 65-70C range. They are only a few degrees higher than the temps VirtualRain reported HERE, and I have seen several others that are within a couple of degrees of this. AZREOspecialist seems to have the coolest system, and I am very curious to know why his chip runs so cool. Good thermal paste application? Do W3580s just run cooler?

There could be two issues at play here. Issue 1 is the audio / cpu problem (which needs to be fixed if this machine is to be taken seriously), and Issue 2 is that Apple's thermal solution seems to be weak at best. If we ignore Issue 1 for a moment, I have my doubts that Apple's thermal solution is able to actually keep the CPU cooled sufficiently. I think I have proved this by observing TCC activation in OSX (Kernel Panic) and Windows (Normal TCC Event). With respect to Issue 2, this is all once again correctable by software. An SMC update could fix the fan logic, and an OSX update could allow OSX to actually survive a TCC event without crashing.

Have you installed Temperature Monitor yet? I think if we are going to compare temps, it helps if we are all using the same tool.

 

[smacman]

Try lowering core voltages - this should reduce generated power.

Thanks for all your info Ibodnar. It sounds like you are quite up to speed on this stuff. I don't have CHUD tools installed, and I am not even sure where to get them, but I will look into this option for sure. Assuming it works, then I would just have to figure out a way to make the setting stick after restart / sleepwake.

Thanks again for your info!

 

[lbodnar]

Thanks for all your info Ibodnar. It sounds like you are quite up to speed on this stuff. I don't have CHUD tools installed, and I am not even sure where to get them, but I will look into this option for sure. Assuming it works, then I would just have to figure out a way to make the setting stick after restart / sleepwake.

Thanks again for your info!

CHUD (Computer Hardware Understanding and Development) is a package that is usually a part of Xcode installation or optional installs. Otherwise you can download it from http://developer.apple.com/
Type reggie_se in terminal session - if it is not installed it would complain that command not found.

It would be really nice to get somebody to write a kernel extension that can control CPU voltages and also update CPU with the latest microcode. For somebody who knows kernel programming it is literally a few hours of work.

Instead of using a kernel extension another simple solution would be to use reggie_se in a chron task that updates CPU every 5 minutes or so. If it runs as root then sudo is not needed either.

 

[smacman]

CHUD (Computer Hardware Understanding and Development) is a package that is usually a part of Xcode installation or optional installs. Otherwise you can download it from http://developer.apple.com/
Type reggie_se in terminal session - if it is not installed it would complain that command not found.

Running that command on my machine results in:

ERROR: unknown processor type

 

[handheldgames]

Running that command on my machine results in:

ERROR: unknown processor type

I got the same error....

 

[Max(IT)]

Although my temps are definitely higher than some peoples reported temps, they don't seem to be out of the realm. You may have misunderstood me however. 86C was my CPU A Temperature Diode reading when I was doing a CPU stress test. Playing audio results in temps in the 65-70C range. They are only a few degrees higher than the temps VirtualRain reported HERE, and I have seen several others that are within a couple of degrees of this.

Much better
Yours are still higher than the average, but not so high as 86°C.

There could be two issues at play here. Issue 1 is the audio / cpu problem (which needs to be fixed if this machine is to be taken seriously), and Issue 2 is that Apple's thermal solution seems to be weak at best. If we ignore Issue 1 for a moment, I have my doubts that Apple's thermal solution is able to actually keep the CPU cooled sufficiently. I think I have proved this by observing TCC activation in OSX (Kernel Panic) and Windows (Normal TCC Event). With respect to Issue 2, this is all once again correctable by software. An SMC update could fix the fan logic, and an OSX update could allow OSX to actually survive a TCC event without crashing.

Regarding the Issue #2, did you monitor fan's behavior on your MP during stress test ?
From what I've seen in MPs, MBPs and MINIs, I think that Apple default settings seems to be much more focused on keeping computers quiet than cooled. Using simple applications like smcFanControl I can easily keep cpu's temperature under control raising fan's rotation speed.
I reported this to Apple, but I don't really think they are going to change their fan logic.

Have you installed Temperature Monitor yet? I think if we are going to compare temps, it helps if we are all using the same tool.

Yes I did.
And I can say that regarding heatsink and cpu diode, Temperature Monitor and iStat Pro report the same temperature.
In Temperature Monitor you can also monitor CPu Core temps.

 

[lbodnar]

Running that command on my machine results in:

ERROR: unknown processor type

I think this happens because version of CHUD is too old. I have checked the Apple developers site and the only standalone version available is 4.6.3
However I have downloaded the latest Xcode developer DVD installation (3.2.1) and it contains CHUD version 4.7.2. This is a free 790MB download from https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=20505

Cheers
Leo

 

[smacman]

However I have downloaded the latest Xcode developer DVD installation (3.2.1) and it contains CHUD version 4.7.2. This is a free 790MB download from https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=20505

Cheers
Leo

Actually that is what I downloaded and installed prior to receiving the error message..