Early/Late-2011 15" and 17" MBP (Sandy Bridge) GPU failures

[undesign]

Did you also monitor your temps before the logic board replacement?

I'd be curious to hear from anyone else who monitors temps and has had their logic board replaced for this problem.

It might be worth taking some extra steps to verify that heatsink fits correctly, that both sections of the heatsink fit flat against their corresponding dies. As I mentioned in the other thread, mine fit together perfectly (I've never had any problems with mine), but I'm not so sure that's always the case.

It would be helpful to know whether or not some manufacturing variations in the heatsink assembly is playing a role in these failures. All information is good. It would be helpful to know if it is. It would also be helpful to rule it out.

I have a couple screenshots of my desktop from a year+ ago which shows the temps in iStat. Click on the images for a larger view.

In April, 2012 running some apps (adium, coda, mamp, espresso and firefox for 2.3gb of RAM usage and minimal CPU usage) for a total of 71C:



In January, 2013, running completely idle with 1gb ram usage (likely after boot): 62C



I dont think I bothered much with the temperatures before.

 

[Mr. Retrofire]

Do a Google search on it and you'll see the number of failures.

Relative to what!?

 

[Swampus]

AS-5 is great when things fit together well. When dealing with unknowns, I'd be inclined to go with something known to work well under a wide range of conditions. Maybe MX-4 or TX-3 in this case?

Maybe even do some test drops to see how they spread out before doing it for real. Be sure to screw the heatsink down following this pattern:

Reinstall screws in the order shown and tighten down halfway. Then, following the same pattern, tighten them down the rest of the way. Then pull it back off to see what you have. If you're satisfied with the way it looks, do it for real.

 

[Gav Mack]

AS-5 is great when things fit together well. When dealing with unknowns, I'd be inclined to go with something known to work well under a wide range of conditions. Maybe MX-4 or TX-3 in this case?

Maybe even do some test drops to see how they spread out before doing it for real. Be sure to screw the heatsink down following this pattern:

Image

Reinstall screws in the order shown and tighten down halfway. Then, following the same pattern, tighten them down the rest of the way. Then pull it back off to see what you have. If you're satisfied with the way it looks, do it for real.

A good point but the Apple pipe assemblies are malleable, with a ruler/flat surface and placing it on the logic board naked you can usually see where it's bent and where to gently and gradually apply the pressure to bend it back. I have done it quite a lot before though

 

[Mr. Retrofire]

The GPU or video ram chips are starting to fail by the look of that especially if it locks up solid.

I've found that it is mostly a problem with the OS kernel and the graphics card switching. If you disable the graphics card switching with gfxCardStatus the freeze does no longer happen. So it is unlikely a hardware problem.

 

[Gav Mack]

I've found that it is mostly a problem with the OS kernel and the graphics card switching. If you disable the graphics card switching with gfxCardStatus the freeze does no longer happen. So it is unlikely a hardware problem.

With the info we have it could be either. Disable the switching and if it still happens...

 

[undesign]

With the info we have it could be either. Disable the switching and if it still happens...

Disabling switching doesn't allow external monitors to be used via thunderbolt though. This is the way I work 100% of the time at home.

 

[Gav Mack]

Disabling switching doesn't allow external monitors to be used via thunderbolt though. This is the way I work 100% of the time at home.

Disabling the switching is definitely not the road you should go down. You need to be cool

 

[Swampus]

A good point but the Apple pipe assemblies are malleable, with a ruler/flat surface and placing it on the logic board naked you can usually see where it's bent and where to gently and gradually apply the pressure to bend it back. I have done it quite a lot before though

Well, yeah, a flat ruler does sound like a much simpler idea!

Do you think these slight imperfections would escape the technician's attention at the depot repair when logic boards are being replaced?

 

[Gav Mack]

Well, yeah, a flat ruler does sound like a much simpler idea!

Do you think these slight imperfections would escape the technician's attention at the depot repair when logic boards are being replaced?

Thinking out the box thanks to my father and brother - both engineers and tool makers. Was using verniers micrometers before I even got my first computer lol.

I'm afraid unless these imperfections are on their checklist to think about it will all go over their head.

The rubbish thermal paste issue is a problem not only for apple, though in their case with razor tight thermals I despair they don't spend a few more cents on proper compound and train their assemblers and repairers to apply the stuff right in the first place.

 

[thundersteele]

Do a Google search on it and you'll see the number of failures.

It is hard to draw reliable information from that. And I'm not sure if anything but a class action suit will convince Apple to start an extended warranty program.

Anyways, after having suffered from a faulty 8600M GT and now having a potential problem with my 6750M, I know one thing: My next mac will have integrated graphics only!!!

Oh, it would also be interesting to know if the 6750M in the late 2011 model is less prone to failures than the early 2011 model.

 

[Gav Mack]

It is hard to draw reliable information from that. And I'm not sure if anything but a class action suit will convince Apple to start an extended warranty program.

Anyways, after having suffered from a faulty 8600M GT and now having a potential problem with my 6750M, I know one thing: My next mac will have integrated graphics only!!!

Oh, it would also be interesting to know if the 6750M in the late 2011 model is less prone to failures than the early 2011 model.

Integrated GPU's suck though. The 8600 was nvidia's fault. With newer models I believe it's down to the chassis, thermals, bad paste application and scrimping using cheap paste.

Off the top of my head (after a few Peronis) I reckon I've had 3/4 times more GPU failures on the 15 than the 17 inch. They get hotter faster which combined over time loosens the solder and dries out the crap factory paste.

 

[undesign]

You've got a similar setup to mine;

2.3ghz, 16GB RAM, Intel 320M 120GB in optibay, 6750M, AG screen, and 1TB WD.

Here's a closer look at my temps watching a simple youtube video:

[url=http://i44.tinypic.com/2i1c8yv.jpg]Image[/URL]



I think it was the intel gpu, notice my discrete graphics temperatures were hovering around 38C. (click on the photo above for larger view)

This is strange. I left my macbook at home all day (turned off). Booting up, it showed an idle temperature of 28-33C. I've been playing the SAME video
via chrome with nothing else running for the last 17 minutes and my temperature is at 57C. I jumped significantly once I started up Chrome and then played the video. The temperature has been increasing though every couple minutes by about 1C. It almost immediately jumped to around 45-48C once I started the video and its been slowly increasing. After 45 minutes of play, its hovering around 59C.

I've tried playing a 1hr 1080p youtube video, my temperature skyrocketed to 83C after 4 minutes of play and the fans are STILL at 2000 RPM.

At 84C the fans FINALLY kicked in, but SLIGHTLY. They ramped up to 2,200 RPM and dropped the temperature to 80C and turned down to 2,000 RPM again... and the temperature is again going back up to 84C.

 

[librarian]

for all the people having temp/fans out of control:
the SMC is the first component to get damaged by the heat, followed up by gpu failure.
By looking the official apple forums, failure rate of replacements are around 3 months, it looks like apple just replace the whole board with an used one with the same issue and nothing else.
For now the solution is to replace the Thermal paste, and eventually replace the GPU if it get busted (if its not damaged it can be reflowed without needing a new chip).

 

[yjchua95]

Are those any more reliable than the 2011 we have? I really don't want to use mine anymore...

Also, did they ever make them with an anti-glare screen at 1680x1050?

They still have the 1680x1050 AG screen, and I'm not sure whether they're more reliable or not, because although they use NVIDIA chips, the thermal paste might still be poorly applied as in the case of the 2011 MBPs.

 

[Doward]

No issues here - and I have the most beastly Sandy Bridge setup (2.5Ghz / HD6770M)

Might be due to this, though:

 

[Swampus]

I've found that it is mostly a problem with the OS kernel and the graphics card switching. If you disable the graphics card switching with gfxCardStatus the freeze does no longer happen. So it is unlikely a hardware problem.

It's a hardware problem. Once the problem starts, disabling switching usually only buys some time. Sometimes not even that.

The fact that some folks have been able to "fix" theirs, if only for a few weeks, with home-brew reflow methods is pretty clear evidence that we're dealing with broken solder joints. Those who have chosen a professional repair involving a reball or new GPU, so far, seem very pleased.

Apple's approach to the problem is to replace the entire logic board. That's been hit or miss so far.

There is probably more than one hardware variable in play and it may or may not also be the case that certain OS/software/firmware/SMC combinations aggravate the situation more than others.

 

[Swampus]

...The rubbish thermal paste issue is a problem not only for apple, though in their case with razor tight thermals I despair they don't spend a few more cents on proper compound and train their assemblers and repairers to apply the stuff right in the first place.

Aye. Absolutely agree that it's an industry-wide thing. I didn't meant to come across as singling out Apple. Though, that also serves to reinforce the idea that we're dealing with more than one variable with these GPU failures. The same chips were installed in BGA packages in laptops made by other companies with sloppy thermal compound and, at least to my knowledge, there hasn't been a lot of noise about failures in these other machines. But I suppose expectations could also play a role here? Someone whose HP failed at two years might have just thought they got a really good one?

I was under the impression that Apple still used a Shin-Etsu compound? But I've never seen anything made by Shin-Etsu look so dried and cracked after only two years. I'm not talking about mine, but some recent pictures I've seen that were torn down at about the two year mark. This one, for example:

 

[Doward]

At 84C the fans FINALLY kicked in, but SLIGHTLY. They ramped up to 2,200 RPM and dropped the temperature to 80C and turned down to 2,000 RPM again... and the temperature is again going back up to 84C.

That's normal. Apple doesn't start ramping up the fans until 85C or so. Full speed is usually attained ~90C.

If your temps stabilize over 95C, then you have issues.

 

[jmoore5196]

Like the OP, I have an early-2011 15" MBP (2.0GHz with Radeon 6490) that hasn't played up yet ... and I'm trying to determine whether it will in the near future. Are the iStat temps any indicator?

My MBP has spent most of its life tethered to a TBD and operating in clamshell mode. At idle closed, my GPU runs just less than 50º C. Open, it jumps up 10 degrees. I do a fair bit of CS6 work but seldom notice anything higher than the 71º that's often quoted as the upper range of the standard operating temperature. From overnight sleep, the GPU is around 33º and moves up rather quickly from there with even light load.

I'm thinking that running in clamshell mode so much of the time has likely saved my GPU. Those who are more tech-savvy than I may have their own ideas ... I'd welcome any input.

 

[yusukeaoki]

Early 2011 2.2GHz and had mine 4 days ago.

My MBP started to have glitchy screen and the second time it started to glitch, it froze.
Tried rebooting, SMC reset, PRAM reset, 2 HDDs with 10.7/8, 1 SSD with 10.9,
4/8/16GB DDR3 RAM and all failed.
Stuck on a white screen at boot.

Took it to a Apple store and unexpectedly, passed all the diagnosis.
They tried booting with their OSX and failed.
After that, they replaced my logic board in 3 days.
Thanks to AppleCare, the 400USD fix was all free.

 

[NewishMacGuy]

Are those any more reliable than the 2011 we have? I really don't want to use mine anymore...

Also, did they ever make them with an anti-glare screen at 1680x1050?

Too early to tell if they will be more reliable or not really.

Some were specced with the anti-glare, but I would guess that fewer were because the rMBP was out.

>

 

[undesign]

Aye. Absolutely agree that it's an industry-wide thing. I didn't meant to come across as singling out Apple. Though, that also serves to reinforce the idea that we're dealing with more than one variable with these GPU failures. The same chips were installed in BGA packages in laptops made by other companies with sloppy thermal compound and, at least to my knowledge, there hasn't been a lot of noise about failures in these other machines. But I suppose expectations could also play a role here? Someone whose HP failed at two years might have just thought they got a really good one?

I was under the impression that Apple still used a Shin-Etsu compound? But I've never seen anything made by Shin-Etsu look so dried and cracked after only two years. I'm not talking about mine, but some recent pictures I've seen that were torn down at about the two year mark. This one, for example:

Looking at the picture you posted, the thermal paste is overflowing down to the chip (the green area). Is it fair to say that the heat is being distributed BACK to the chip when this happens thus causing higher temperatures?

 

[Swampus]

Looking at the picture you posted, the thermal paste is overflowing down to the chip (the green area). Is it fair to say that the heat is being distributed BACK to the chip when this happens thus causing higher temperatures?

The excess that gets squished out the sides can play the role of insulator a little bit. When things fit together well, I don't think it's a huge deal--Maybe a three to five degree difference at full load? Folks who have redone the thermal compound generally report about a three to five degree improvement at idle and and about a five to eight degree improvement at load. I'd guestimate that's half attributed to doing a better job and half attributed to using a better quality compound.

As I've mentioned, the thing that I would be on the lookout for here is things not fitting well. Take a look at this thread for a good example:

So guys, I have an update. Yesterday, I decided to do another thermal paste reapply, as it seemed to me that the temperatures are worse each day.
Last time I used the spread method, covering whole die of cpu and gpu with paste (Arctic Cooling MX-4). Results after about 14 days you could see in my first writing in this topic.


This time I decided to use pea method for gpu (square) and line method for cpu (rectangle); both using the same MX-4 paste. When I removed the heatsink, I noticed that on the cpu die, where the Cores 2, 3 and 4 should be is almost no paste, which was kinda surprising to me, as on other parts of die and HS paste was, this led me to conclusion of higher temps of cores 1, 2 and 3. I've cleaned dies properly, heatsink also, put a little bit on dies and mounted carefully (no twisting with HS, just placed on).
Temps are slightly better under load for now (!), difference between core 1 temp and others is slightly less, around 5-6 C under heavy load, but is still there very noticeable, especially when C1 is 85 and rest is 89-92.
This leads me to conclusion there has to be some pretty bad imperfection on the die, or more likely, on the heatsink's surface. It just seemed, when I removed HS that the paste got squeezed out certain places where C2, C3, C4 are. I believe this is the main problem why I have problem with high and uneven temperature of cpu and any cpu paste reapply won't help. Temperature of gpu always seem quite ok, measuring 75C in average and not exceeding 80C.
I am thinking of ordering new genuine heatsink from ebay and if that wont help, well, my Mac can go to hell. After all, it's used to it's temperature environment already.

When there are actual gaps, thermal resistance will increase with the thickness of the interface material. Some compounds will hold their own better than others, but no compound is really meant to play this role.

I think most of our heatsinks fit together very well, but I'm starting to get the impression that imperfections are more common than we'd like to think. Not everyone is going to be as determined as the OP in that thread to figure out what is going on, but I think this could explain some of the outliers we've been seeing, such as what is happening in your case even with your new board.

 

[Swampus]

Like the OP, I have an early-2011 15" MBP (2.0GHz with Radeon 6490) that hasn't played up yet ... and I'm trying to determine whether it will in the near future. Are the iStat temps any indicator?...

I certainly think it's a good idea to monitor temperatures, but we don't really know what is going on with these 2011 failures. My best guess, given what is currently known, is that Apple made a mistake in the thermal recipe when they cooked all of this. Or, to similar effect, a faulty thermostat somewhere. Does this potential failure exist in all 2011 models? Or just ones ones from one particular assembly line at one particular location? A particular batch of materials? Too much phosphorus in the solder paste on some days?

If it's not a mistake in thermal profiling, then it probably comes down to several interrelated variables that push the existing CTE mismatch to become a destructive force.

I'm trying to get some pre-repair endoscopic images from a local shop that works on these. Maybe we'll eventually narrow in on some things based on the types of failures and the location of those failures.

For now, I'd try not to worry about it. It's too nice a machine to not use it the way you want. At worst, you might have to get a reball or new GPU installed one day for $250 or so.

I'm thinking that running in clamshell mode so much of the time has likely saved my GPU. Those who are more tech-savvy than I may have their own ideas ... I'd welcome any input.

I don't see anything alarming in your temperatures, but staying locked on the discreet is probably helpful in reducing the number of thermal cycles and the range of temperatures within those cycles. Those are both important variables in the Modified Coffin-Manson:

where:

Nf is the number of cycles to failure.
Α is a coefficient.
f is the cycling frequency.
ΔT is the temperature range during a cycle.
Α is the cycling frequency exponent.
Α is the temperature exponent.
G(Tmax) is equal to:

As far as I know, this is still considered very accurate in any case where maximum operating temperature is less than half of the melting point of the solder. That would certainly include our MBPs.

But that's just general long-term health of BGA solder joints. If yours has whatever defect we're seeing, it will fail much sooner rather than later and there isn't much you can do about it. Use it the way you want and deal with it when the time comes.