Spotlight mdworker.shared file flooding system log; removal stops search index from working properly?

[donawalt]

@Brian33 I am new to ULBow so I probably did it wrong, I just typed mdworker into the filter window as I recall - in fact my further research below would seem to affirm that mdworker is NOT a problem right now.

What I have done since, which seems more reliable, is I uninstalled Malwarebytes (RTProtectionDaemon.Exclusions had 100s of thousands of entries in short periods of time), and set persist to off for DNRResponder and Bluetooth, as those choices could be selected from the Show dropdown of the Chart display - and they both seemed to be generating a lot of entries (but not as much as RTProtection...).

Since doing those 3 things, from Mints Logs button in the Information Pane, oldest log entry went from .87 days to 1.622 days in the last 20 hours. I will keep watching it. Eventually I will try setting the persist flag back for each, and then try reinstalling MWB once the oldest log entry stabilizes somewhat (it has been steadily climbing the whole 20 hours).

I do have MWB running on an M1 MacBook Pro, and its log is above 2 days, so maybe MWB on my M2 MacBook Pro got messed up somehow.

 

[auxbuss]

I'm running Monterey 12.6.3 and it doesn't seem to be a problem for me; I used the following 'log show' command and got only 5 for the last 30 minutes, and only 135 for the last 24 hours:

log show --info --last 30m --predicate 'sender == "launchd" &&  eventMessage contains[cd] "exited due to SIGKILL" '

Just to a data point: I'm running the same version of Monterey and got 84 entries in the past 30 mins. 778 in the past nine hours – as far back as it goes.

 

[donawalt]

If you want to get the Unified log file cleaned up, and mute subsystems that are depositing hundreds of thousands of log entries into the log (which causes other entires you may care about to be lost), check out this article and many more on the Unified log written by Howard Hoakley. No ads, no noise on his blog. This article on how to use his utility ULBow to see which subsystems are adding the most events to the Unified log, and how to easily mute them, was invaluable to me in clearing up a lot.

My log only had messages back about 16-18 hours, when it should be ideally around 2 days. I had so many log messages that the results of the XPR scans (malware protection) were overwritten in the log very quickly. I was shocked at which of the subsystems dropped LOTS of events at times - some of the com.apple.photos ones dropping 200K events in an hour; com.apple.bluetooth dropping 50K-100K in an hour, same for com.apple.mDNSResponder, com.apple.WebKit, and more. You need a tool that can aggregate and summarize the events to see which post the most. If you need to unmute one day to research an issue, it's easy to do as well.

I used to use console log, which is NOT complete. His stuff is way better imho.

 

[Brian33]

This article on how to use his utility ULBow to see which subsystems are adding the most events to the Unified log, and how to easily mute them, was invaluable to me in clearing up a lot.

My log only had messages back about 16-18 hours,

Thanks! I do often read the blog on eclecticlight.com and have learned a lot from it. Thanks to the pointer to the recent article on reducing log clutter which I hadn't seen yet.

I guess I'm lucky. Both of my Macs (running Monterey) have logs going back 30 days, according to 'log stats' command. Just out of curiosity I followed the procedure with his Ulbow utility and discovered the most prolific logger on my systems is com.apple.cache_delete. According to a comment this can be the result of running iStat Menus, which I do. However, since I have plenty of log history I don't feel the need to exclude those log entries. I will keep the technique in mind for the future, though!

 

[eropko]

Usually it's some app(s) constantly writing to disk and making Spotlight "look" there for changes. It's quite easy to see what folders to exclude in Activity Monitor. Filter process list by "mdworker", double click every running mdworker_shared process, go to third tab in a small window that pops up to see files open by process. After some dev/nulls you will see what folders is this process actually indexing now. Just add them to Spotlight ignore list and at some point you'll end up with a single mdworker spawning roughly once every several minutes with empty list of indexed folders in Activity Monitor.

For me it was
~/Library/Containers (Slack is writing there, I simply added whole folder to ignore list)
~/Library/Group Containers (Telegram is writing there, also added whole folder)
~/Library/Application Support/Firefox (Firefox is constantly writing something to user profile)