Lion - mds/mdworker spotlight indexing issue

[kbmb]

Upgraded my Mom's early 2008 Macbook to Lion this weekend. Everything went fine. As usual we waited for the initial Spotlight reindexing to finish.

Problem is, now, seemingly randomly, her fans crank up and cpu spikes as a few mds and mdworker processes eat up cpu.

Anyone else seeing this?

I did go in this morning....added her HD to the privacy section.....then removed it. At which point Spotlight reindexed again.....but even though it's finished, it still happens.

No real pattern....just one minute fans at 1800rpm as normal.....the next, CPU up and fans at 4500rpm with the mds/mdworker processes working.


FYI...the mds/mdworker processes are NOT owned by her....they are under the All Processes tab....I think owned by root.

-Kevin

[murdercitydevil]

Does she have TM backups enabled? Next time it happens check if TM is doing a backup.

[kbmb]

Originally Posted by murdercitydevil

Does she have TM backups enabled? Next time it happens check if TM is doing a backup.

She does has TM backups....to a Time Capsule.

We checked this morning....and at one point it was spiking when a TM backup was running....but after it finished.....15 minutes later or so, after it settled down, it spiked again with no TM backup running.

What's weird is I have the same 2008 Macbook....also backing up to the same style Time Capsule. My machine isn't having the issue.

-Kevin

[kbmb]

Ok, so looking at the system log....this might have something to do with Spotlight trying to index her time Machine backups.

No idea why....since I thought Spotlight automatically excludes the TM backups from indexing.

Now I know on my Mac Pro, which has a TM drive but it's hooked up via FW800.....I can see the drive in the TM prefs as excluded. What's weird on MY Macbook, which is backing up to a Time Capsule.....is in the options, I don't SEE the Time Capsule listed. However, I'm assuming Spotlight automatically excludes the TC backups from indexing?

I'm seeing errors like this in her system.log:

Aug 2 11:31:29 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 11:40:30 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 11:50:30 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 12:00:30 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 12:10:30 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 12:20:30 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb

and:

Aug 2 08:23:31 mds[3184]: (/Volumes/Time Machine Backups/.Spotlight-V100/Store-V2/57967A66-77F9-42FC-8EF1-A5E62D5CEE47)(Warning) IndexGeneral in _SIOpenIndex:datastore clean, index dirty, recovering...
Aug 2 08:23:34 mds[3184]: (Normal) DiskStore: Rebuilding index for /Volumes/Time Machine Backups/Backups.backupdb

and:

Aug 2 12:39:03 com.apple.mdworker.pool.2[8780]: I/O error : encoder error
Aug 2 12:40:36 com.apple.backupd[8445]: Waiting for Spotlight to finish indexing /Volumes/Time Machine Backups/Backups.backupdb
Aug 2 12:41:04 mdworker[8781]: *** Failed to decode data, treating as binary
Aug 2 12:41:39 mdworker32[8782]: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
Aug 2 12:45:24 com.apple.mdworker.pool.2[8780]: I/O error : encoder error

So.....because this is not her primary Mac....here's what I'm trying. I wiped the Time Capsule disk....and am starting over with the TM backups.

Guess I'll see if that solves it unless anyone else sees anything from the above logs.

-Kevin

[kbmb]

Interesting.....my Macbook just started showing some of these same Spotlight indexing errors today. Fans going full blast.

Anyone else that backs up to a Time Capsule see any of these errors?

And can anyone confirm....does Spotlight by default not index Time Machine volumes? I know on my Mac Pro that has a FW Time MAchine drive....I see it excluded in Spotlight settings. But for the Time Capsule one....since it's not mounted continuously, I don't see it (I don't see it even when it is mounted).

-Kevin

[murdercitydevil]

Originally Posted by kbmb

Interesting.....my Macbook just started showing some of these same Spotlight indexing errors today. Fans going full blast.

Anyone else that backs up to a Time Capsule see any of these errors?

And can anyone confirm....does Spotlight by default not index Time Machine volumes? I know on my Mac Pro that has a FW Time MAchine drive....I see it excluded in Spotlight settings. But for the Time Capsule one....since it's not mounted continuously, I don't see it (I don't see it even when it is mounted).

-Kevin

Spotlight will index TM unless you do a workaround to turn it off. The downside is that when you enter TM to find a file, you won't be able to use Spotlight to search.

[kbmb]

Originally Posted by murdercitydevil

Spotlight will index TM unless you do a workaround to turn it off. The downside is that when you enter TM to find a file, you won't be able to use Spotlight to search.

That's interesting, because I have my backup drive in the Spotlight privacy section, however, when I enter TM....I'm able to search just fine. I can even go back to the beginning of time....search for a file I know ONLY exists in TM....and it finds it.

Any ideas?

-Kevin

[kbmb]

So I found this command line utility that shows the current status.

$ mdutil -sa
/:
Indexing enabled.
/Volumes/MacBackup:
Indexing and searching disabled.
/Volumes/MacBackup/Backups.backupdb:
Indexing enabled.
/Volumes/MacExternal:
Indexing and searching disabled.
/Volumes/Personal:
Indexing enabled.
/Volumes/StorageHD1:
Indexing enabled.
/Volumes/StorageHD2:
Indexing enabled.
/Volumes/Work:
Indexing enabled.

You can see that my MacBackup has indexing AND searching disabled....but the TM folder has Indexing enabled.

So I removed the MacBackup from the privacy and added it back. I got the attached screenshot.

So yeah, OS X is smart enough to always allow indexing of TM.....and you can control whether other files on the disk are indexed and searched.

I guess the issue here is, after Lion, Spotlight on the Macbook I have, is trying to reindex the whole thing....and it's taking a long time.

Wonder if it's a bug.

-Kevin

[designgeek]

I've been having this issue too. mds won't die either, I've tried killing it but it can't be killed. I even tried killing spotlight all together and it still won't die... This is a bug that Apple needs to sort out. I have been having this problem since I upgraded to lion when it came out and as of yet no solution.

[kbmb]

Just an update on the mds processes for my Time Capsule TM backup.

I never wiped my TC backup.....and while it did take about a day....the mds processes that tried indexing the TM backup did eventually finish, and I don't see the spikes anymore.

-Kevin

[parish]

Found this thread when trying to find out about these mdworker processes and I think I now know what is causing the spikes in mdworker CPU usage.

I use TM on my MBP to backup to a USB disk which, because it's a laptop, is not connected a lot of the time. TM in Lion now makes local "snapshot" backups (in /.MobileBackups) when the TM backup disk is not available. I bet that it is these that are being indexed and causing the spikes and since TM does hourly backups they are happening at least that frequently.

However, it is probably not these processes that are causing the performance hit I'm seeing (which seems to coincide with these spikes) directly, it's because it's indexing the MBP's hard drive as opposed to the external one where the normal backups are located.

[DarkestRitual]

Originally Posted by designgeek

I've been having this issue too. mds won't die either, I've tried killing it but it can't be killed. I even tried killing spotlight all together and it still won't die... This is a bug that Apple needs to sort out. I have been having this problem since I upgraded to lion when it came out and as of yet no solution.

Yea, I've killed the process repeatedly, and eventually it gives up... still a pain in the rear!

[danistyping]

soooo....anyone ever solve this? It just started happening to me and its driving. me. nuts!

[kbmb]

Originally Posted by danistyping

soooo....anyone ever solve this? It just started happening to me and its driving. me. nuts!

Are you using a Time Capsule TM backup?

For my Mom....we wiped her backups are started over.
For mine.....I just made sure I left my MacBook on and running and after a while the processes settled down.

-Kevin

[danistyping]

I'm using a 3rd party external HDD to backup with time machine, but I haven't done this since the problem began in fear that I would mess up my backup somehow.

do I need to plug that in to let mdworker/mds/whatever resolve itself?

The computer is done indexing it seems. However, these processes continue to run AND the craziest part is that they are writing something on all of my free hard drive space! I can literally watch my 50 free gigs disappear over the course of about an hour or two in activity monitor....my internal HDD spins like crazy the whole time while mdworker is running at over 100% cpu.

This is def a software issue fyi, the guest account boots fine.

[kbmb]

Originally Posted by danistyping

I'm using a 3rd party external HDD to backup with time machine, but I haven't done this since the problem began in fear that I would mess up my backup somehow.

do I need to plug that in to let mdworker/mds/whatever resolve itself?

The computer is done indexing it seems. However, these processes continue to run AND the craziest part is that they are writing something on all of my free hard drive space! I can literally watch my 50 free gigs disappear over the course of about an hour or two in activity monitor....my internal HDD spins like crazy the whole time while mdworker is running at over 100% cpu.

This is def a software issue fyi, the guest account boots fine.

If those processes are running when your external isn't connected....then I'd say something else is causing it.

You can try going into Spotlight prefs and adding all your drives to the Privacy section. Wait a few minutes....then remove them. This should cause Spotlight to re-index the drive(s). This will cause MDS to spike again....but once it's done it should be fine.

-Kevin

[danistyping]

THANK. YOU. KEVIN!

all i had to do was add macintosh hd to privacy and then remove it. now its re-indexing like a NORMAL computer. I can't believe that no one at apple thought of this, nor did anyone else in any online forum. You are the s***.

[kbmb]

Originally Posted by danistyping

THANK. YOU. KEVIN!

all i had to do was add macintosh hd to privacy and then remove it. now its re-indexing like a NORMAL computer. I can't believe that no one at apple thought of this, nor did anyone else in any online forum. You are the s***.

Glad it's working. Fingers crossed that it finishes the full re-index and doesn't give any more issues!

-Kevin

[oneofthesedaves]

Is anyone using Time Machine and also an online backup service like Carbonite or Backblaze? I have found that for some reason using one of those backup services and Time Machine causes mdworker and mds to spike to 99% of CPU.

[madsb]

Originally Posted by oneofthesedaves

Is anyone using Time Machine and also an online backup service like Carbonite or Backblaze? I have found that for some reason using one of those backup services and Time Machine causes mdworker and mds to spike to 99% of CPU.

If you're using Backblaze, just add /Library/Backblaze/ in the privacy tab in Spotlight settings. I've made a post about it here:

http://madsb.com/post/4256208203/bac...roblem-in-os-x