Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

SPOTLIGHT still not working on certain external drives

wfriedwald

wfriedwald

macrumors 6502a
Original poster
Jan 1, 2017
552
48
In December 2020, I had this problem with a new drive. Spotlight just failed to work. I wasn't able to solve it then:

<https://forums.macrumors.com/threads/easiest-way-to-force-spotlight-reindex.2275615/>

Since then, a few weeks ago, I copied the entire contents of that drive (Annex4_10TB) onto a new one (Annex5_14TB).

And now, to my disappointment, the problem reappears on the new new drive as well!

Since then I have upgraded to MONTEREY 12.1, and also tried rebuilding the directory structure with DISKWARRIOR. But no dice.

Am game for any more feedback.

Can someone give me specific terminal commands to turn SPOTLIGHT INDEXING off and on again for these specific drives
Annex4_10TB
Annex5_14TB

thanks very much again!

w
 
what file system does the hard drives use? You checked the Spotlight Privacy tab, I suppose? You can try these commands (as long as the file system format is supported of course).

to disable Spotlight globally:
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

to enable Spotlight globally:
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

to disable Spotlight (and delete cache) for one specific volume:
sudo mdutil -i off -d -E /Volumes/Annex4_10TB

to enable Spotlight for a specific volume:
sudo mdutil -i on -d -E /Volumes/Annex4_10TB
 
Last edited:
  • Like
Reactions: mar-ko
thanks!

Fisherking - I believe that these two drives are properly formatted. Is there a terminal command or something that can confirm that? (Love that movie, by the way!)

Slartibart - thanks for the Terminal commands. will try!

I just checked both drives in DISK UTILITY - it informs me that both drives are
"MAC OS Extended (Journaled)"

is that what you're asking?

thanks again. would love to solve this problem - this time!

w
 
format should be ok.

not familiar with those drives (annex? can't find them googliing). but, since this is happening on a 2 versions of the OS, the common denominator seems to be the drives themselves... and maybe worth contacting the manufacturer....
 
Further to Slartibart's post:

To check the current status of spotlight indexing for all mounted volumes in the Terminal:
Code: 
mdutil -sa


J
 
Hey thanks, "Annex" is just my name for the drive(s), not the manufacturer, I believe they're both WD drives.

----

WF: okay, here's what the first command told me:

Wills-Mac-mini:~ willfriedwald$ mdutil -sa
/:
Indexing enabled.
/System/Volumes/Data:
Indexing enabled.
/Volumes/Annex4_10TB:
Indexing and searching disabled.
/Volumes/Annex5_14TB:
Indexing and searching disabled.

WF: don't know why they are turned off!
I tried the command to turn them off and then on... but it doesn't work, it still does NOT turn indexing / searching back on, no matter what I do:

Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i off -d -E /Volumes/Annex4_10TB
Password:
/System/Volumes/Data/Volumes/Annex4_10TB:
2022-02-21 16:11:44.896 mdutil[1069:25296] mdutil disabling Spotlight: /System/Volumes/Data/Volumes/Annex4_10TB -> kMDConfigSearchLevelOff
Indexing and searching disabled.

Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i on -d -E /Volumes/Annex4_10TB
/System/Volumes/Data/Volumes/Annex4_10TB:
2022-02-21 16:11:54.878 mdutil[1073:25394] mdutil disabling Spotlight: /System/Volumes/Data/Volumes/Annex4_10TB -> kMDConfigSearchLevelOff
Indexing and searching disabled.

Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i on -d -E /Volumes/Annex5_14TB
/System/Volumes/Data/Volumes/Annex5_14TB:
2022-02-21 16:12:44.380 mdutil[1084:26639] mdutil disabling Spotlight: /System/Volumes/Data/Volumes/Annex5_14TB -> kMDConfigSearchLevelOff
Indexing and searching disabled.

Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i on -d -E /Volumes/Annex5_14TB
/System/Volumes/Data/Volumes/Annex5_14TB:
2022-02-21 16:12:47.538 mdutil[1087:26673] mdutil disabling Spotlight: /System/Volumes/Data/Volumes/Annex5_14TB -> kMDConfigSearchLevelOff
Indexing and searching disabled.
Wills-Mac-mini:~ willfriedwald$

WF: lastly, I tried to enable SPOTLIGHT "globally" and this is what I get:

Wills-Mac-mini:~ willfriedwald$ sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
/System/Library/LaunchDaemons/com.apple.metadata.mds.plist: service already loaded
Load failed: 37: Operation already in progress
Wills-Mac-mini:~ willfriedwald$

help! grateful for any additional suggestions!

w
 
Do you have a hidden .metadata_never_index file on the volume? You can check with:
Code: 
ls -al /Volumes/Annex4_10TB

Also, do you have a preferences file in /Library/Preferences/com.apple.SpotlightServer.plist? If so, does it contain a line to ignore external volumes?
 
thanks again - here's the response to those two queries - not sure if what we're looking for is here. thanks!

/Library/Preferences/com.apple.SpotlightServer.plist

Audio
com.apple.fsplaceholder
DirectoryService
Logging
OpenDirectory
SystemConfiguration
Xsan
com.adobe.acc.AdobeDesktopService.plist
com.adobe.AdobeGenuine.validator.plist
com.adobe.headlights.apip.plist
com.apple.airport.opproam.plist
com.apple.alf.plist
com.apple.AppleFileServer.plist
com.apple.apsd.launchd
com.apple.apsd.plist
com.apple.AssetCache.plist
com.apple.BezelServices.plist
com.apple.biometrickitd.plist
com.apple.Bluetooth.plist
com.apple.ByteRangeLocking.plist
com.apple.commerce.plist
com.apple.CoreRAIDServer.cfdb
com.apple.dock.plist
com.apple.driver.AppleIRController.plist
com.apple.FindMyMac.plist
com.apple.HIToolbox.plist
com.apple.keyboardtype.plist
com.apple.loginwindow.plist
com.apple.MCX.plist
com.apple.mdmclient.plist
com.apple.networkextension.cache.plist
com.apple.networkextension.necp.plist
com.apple.networkextension.plist
com.apple.networkextension.uuidcache.plist
com.apple.noticeboard.plist
com.apple.P2PSupervisor.plist
com.apple.powerlogd.plist
com.apple.PowerManagement.70A2CB65-6DF3-58E3-B202-240F39F85AB3.plist
com.apple.PowerManagement.plist
com.apple.security.appsandbox.plist
com.apple.security.plist
com.apple.security.systemidentities.plist
com.apple.SoftwareUpdate.plist
com.apple.systemprefs.plist
com.apple.TimeMachine.plist
com.apple.timezone.auto.plist
com.apple.updatesettings.plist
com.apple.ViewBridge.plist
com.apple.wifi.known-networks.plist
com.apple.windowserver.plist
com.microsoft.autoupdate2.plist
org.cups.printers.plist





Wills-Mac-mini:~ willfriedwald$ ls -al /Volumes/Annex4_10TB

total 34249984

drwxr-xr-x 4 willfriedwald wheel 136 Dec 24 2020 ((2020-12-13 +))
-rw-r--r--@ 1 willfriedwald wheel 810012133 Dec 27 2020 * * * * * * * CJ Holiday Title 2.mov
drwxrwxrwx 43 root wheel 1530 Feb 8 14:48 .
drwxr-xr-x 7 root wheel 224 Feb 21 16:08 ..
drwx------ 2 willfriedwald wheel 68 Nov 16 16:49 .CacheDeleteDiscardedCaches
d--x--x--x 9 root wheel 306 Feb 21 16:08 .DocumentRevisions-V100
d-wx--x--t 4 root wheel 136 Dec 11 2020 .TemporaryItems
d-wx--x--t 3 root wheel 102 Dec 11 2020 .Trashes
drwxrwxrwx 21 root wheel 714 Feb 21 11:58 .fseventsd
-rw-r--r-- 1 root wheel 0 Dec 10 2020 .metadata_never_index_unless_rootfs
-rw-r--r--@ 1 willfriedwald wheel 133 Dec 16 2020 .neofinder.abemeda.volinfo.xml
-rw-r--r--@ 1 willfriedwald wheel 3118882747 Apr 1 2021 1972-09-10 Zenith Presents A Salute To Television's 25th Anniversary.mkv
-rw-r--r--@ 1 willfriedwald wheel 18477637 Dec 14 2020 20201103_152619_002.mp4
-rw-r--r--@ 1 willfriedwald staff 1468115340 Jul 17 2019 CARMEN MCRAE Jazz Casual 1962-03-15 Simmons, Sproles, Perkins.mkv
-rw-r--r--@ 1 willfriedwald wheel 810012133 Dec 27 2020 CJ Holiday Title 2.mov
-rw-r--r--@ 1 willfriedwald staff 37671924 Dec 12 2020 Comic Book History of Animation 001 (2020) (digital) (Salem-Empire).cbr
-rw-r--r--@ 1 willfriedwald wheel 2013405184 Dec 13 2020 Dinah S&H Show - Nat King Cole, George Burns - New Years Show-12.29.61.mpg
drwxr-xr-x 1004 willfriedwald wheel 34136 Feb 21 11:12 Downloads
drwxr-xr-x 8 willfriedwald wheel 272 Dec 15 2020 FS XMAS 2020-12-15 (erase, dupes)
-rwxr--r--@ 1 willfriedwald staff 2537866906 May 24 2019 Frank Sinatra - Live in Frankfurt 10-5-91 [rare uncut version] [from David Rose].mkv
drwx------@ 8 willfriedwald staff 272 Dec 14 2020 Frank Sinatra MoFi Capitol Reissues
-rw-r--r--@ 1 willfriedwald wheel 449988643 Dec 14 2020 Frank Sinatra MoFi Capitol Reissues.zip
drwxr-xr-x 19 willfriedwald wheel 646 Mar 23 2021 Ina Ray Hutton (Misc consolidated)
-rw-r--r--@ 1 willfriedwald wheel 18723601 Jan 9 2021 Jack Jones & Paul Loweden (B3) blues excerpt.mov
-rw-r--r--@ 1 willfriedwald wheel 59753737 Mar 31 2021 Jimmy Durante 'One of Days' Goodnight HP 1968-03-30.mp4
-rw-r--r--@ 1 willfriedwald wheel 690591748 Mar 20 2021 NKC NBC 1957-01-14 Stuff Smith Portion With Restored Sound.mpg
-rw-r--r--@ 1 willfriedwald staff 71429 May 19 2021 NYAC LOGO BLACK.jpeg
-rw-r--r--@ 1 willfriedwald staff 8469 Mar 1 2021 NYAC logo.jpg
-rw-r--r-- 1 willfriedwald staff 669402615 Dec 12 2020 Petula Clark 1968 NBC special_SD_WEB.mp4
-rw-rw-rw- 1 willfriedwald staff 1480909845 Dec 14 2020 TONIGHT 1973-02-13 Carl Reiner, George Burns, Sammy Davis Jr [BK].mkv
-rw-r--r--@ 1 willfriedwald wheel 86048037 Apr 1 2021 billie holiday podcast EPISODE1MASTER.mp3
drwxrwxrwx 17 root wheel 578 Dec 17 14:04 bu annex 3 (God!)
-rw-r--r--@ 1 willfriedwald wheel 39205 Jan 27 2021 burke - van heusen songlist notes.docx
-rw-r--r--@ 1 willfriedwald staff 117920 May 4 2020 clip joint text virtual tip jar w kitty BEST.jpg
-rw-r--r-- 1 willfriedwald staff 2040784859 Aug 15 2020 erase.mkv
-rw-rw-r--@ 1 willfriedwald staff 1224790660 Jan 19 2020 jody.mp4
-rw-r--r--@ 1 willfriedwald wheel 87006 Mar 8 2021 phantom-of-the-paradise-vintage-movie-poster-original-40x60_600x.webp
drwxr-xr-x 20 willfriedwald wheel 680 Dec 14 22:20 stan kenton JPEGs
drwxr-xr-x 104 willfriedwald wheel 3536 Feb 9 00:24 usenet 2021
drwxr-xr-x 6 willfriedwald wheel 204 Jun 11 2021 zoom audio H4N audio
drwxr-xr-x 4 willfriedwald wheel 136 Dec 28 2020 {i} move to inactive drive
Wills-Mac-mini:~ willfriedwald$
 
Ok, this:

-rw-r--r-- 1 root wheel 0 Dec 10 2020 .metadata_never_index_unless_rootfs

Could be the problem. Try:

Code: 
sudo rm /Volumes/Annex4_10TB/.metadata_never_index_unless_rootfs

Then reset Spotlight indexing for the external:

Code: 
sudo mdutil -E /Volumes/Annex4_10TB
 
okay, here's the latest terminal response:

Wills-Mac-mini:~ willfriedwald$ sudo rm /Volumes/Annex4_10TB/.metadata_never_index_unless_rootfs

Password:

Wills-Mac-mini:~ willfriedwald$ sudo mdutil -E /Volumes/Annex4_10TB

/System/Volumes/Data/Volumes/Annex4_10TB:

2022-02-21 17:50:18.216 mdutil[2409:159288] mdutil disabling Spotlight: /System/Volumes/Data/Volumes/Annex4_10TB -> kMDConfigSearchLevelOff

Indexing and searching disabled.

Wills-Mac-mini:~ willfriedwald$


thanks yet again!

w
 
Can you check if the file got deleted with:

Code: 
ls -al /Volumes/Annex4_10TB/.metadata_never_index_unless_rootfs

Followed by:

Code: 
sudo mdutil -i on /Volumes/Annex4_10TB
 
thank you JCSCOL! Hopefully we are getting there... here's the latest!



Wills-Mac-mini:~ willfriedwald$ ls -al /Volumes/Annex4_10TB/.metadata_never_index_unless_rootfs

ls: /Volumes/Annex4_10TB/.metadata_never_index_unless_rootfs: No such file or directory



Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i on /Volumes/Annex4_10TB

Password:

/System/Volumes/Data/Volumes/Annex4_10TB:

Indexing and searching disabled.

Wills-Mac-mini:~ willfriedwald$
 
I'm running out of ideas with this one!

1) Have you checked that the volume isn't excluded in System Preferences -> Spotlight -> Privacy

2) Have you tried adding the volume to exclude it under Privacy tab, then removing it again (which usually kickstarts Spotlight indexing

3) (long shot) Try

Code: 
sudo defaults write /Library/Preferences/com.apple.SpotlightServer.plist ExternalVolumesIgnore -bool False
sudo mdutil -i on /Volumes/Annex4_10TB
sudo mdutil -sa
 
okay, I have done the SPOTLIGHT / PRIVACY trick before, but I tried it again, pulling the volume in and out of the privacy tab. That tab is currently EMPTY.

here's the results of the terminal commands:

Wills-Mac-mini:~ willfriedwald$ sudo defaults write /Library/Preferences/com.apple.SpotlightServer.plist ExternalVolumesIgnore -bool False

Password:


Wills-Mac-mini:~ willfriedwald$ sudo mdutil -i on /Volumes/Annex4_10TB

/System/Volumes/Data/Volumes/Annex4_10TB:

Indexing and searching disabled.


Wills-Mac-mini:~ willfriedwald$ sudo mdutil -sa

/:

Indexing enabled.

/System/Volumes/Data:

Indexing enabled.

/Volumes/12T1:

Indexing enabled.

/Volumes/Annex4_10TB:

Indexing and searching disabled.

/Volumes/Annex5_14TB:

Indexing and searching disabled.

/Volumes/CM_SUPP_1:

Indexing enabled.

/Volumes/USB DISK:

Indexing enabled.

Wills-Mac-mini:~ willfriedwald$

thanks again. I don't blame you for feeling frustrated. Believe me, I can relate!

w
 
thank you - here's this:

Wills-Mac-mini:~ willfriedwald$ sudo less /System/Volumes/Data/.metadata_never_index

Password:

/System/Volumes/Data/.metadata_never_index: No such file or directory

Wills-Mac-mini:~ willfriedwald$

thanks again!
 
Can you check with a different user?
And are Annex* partitions? If yes, is there any partition on the same drive excluded from Spotlight?
What happens if you drag&drop the icon of Annex4*&5* onto the Spotlight Privacy pane in its preferences?
 
This may be "late to the party" but after two days of wrestling with this very same problem, I have - by luck or stubbornness - solved it here. Thanks go to jcscol and wfriedwald for their back-and-forth which gave me clues. Long story short...

I tried the Spotlight preferences "add volume to privacy list then remove it" trick many times, no luck.

Using ls -al /Volumes/(volume name) I discovered all my external volumes did have the following listing:

-rw-r--r-- 1 root wheel 0 Dec 10 2020 .metadata_never_index_unless_rootfs

I got rid of that file using sudo rm /Volumes/(volume name)/.metadata_never_index_unless_rootfs

However, no amount of following that with sudo mdutil -i on /Volumes/(volume name) or similar commands would work to reestablish indexing.

Once more, with the "offending" file removed, I tried Spotlight preferences - added the volumes to the Privacy list, then removed them. But then I completely shut down my iMac, waited a bit, then rebooted. I noticed a LOT of external drive activity (read-write head sounds) as the drives mounted.

I then went to Terminal and used mdutil -sa and VOILA, every volume came back "Indexing enabled." It appears that the problem needs a very specific sequence of steps to work - at least, that's how it fell out here. Hope this helps.
 
  • Like
Reactions: mar-ko
oh gosh! thank you! I confess I had completely given up on this.

one factor that makes it even more complicated: in the last month or so, I've had tons of problems with slowdowns and crashes involving a combination of ZOOM, VLC (media player), and external USB3 drives. (there are long threads of discussion if you're interested!)

Anyhow, one suggestion was to turn off SPOTLIGHT completely. The theory was that it runs in the background and eats up system resources. (I've actually been using a program called NEOFINDER that does searching and cataloging independent of SPOTLIGHT. The main difference is NEO doesn't run in the background.) So I've actually barely used SPOTLIGHT at all for the last few months.

Anyhow, gotta try this. Literally, for the last 2-3 years this has been the most annoying problem ... I mean, SPOTLIGHT just up and refusing to index certain drives.

let me go through your steps and see what happens!

Thanks again,

W
 
Hi everyone,
I had the same problems. Thanks for all the hints here, it was a big help for me. 👍

After deleting the file ".metadata_never_index_unless_rootfs", I right-clicked the drive on the desktop, selected "information" (I hope that's the term, its "Informationen" in German) and in the window that popped up—at the very bottom—, I changed "ignore ownership on this volume" to "checked". (You need to click on the lock and provide your password first.)

After that, I could switch on indexing with

sudo mdutil -i on /Volumes/(volume name)

and

mdutil -sav

finally gave me "Indexing enabled." 🥳

Maybe some of the waiting and rebooting played a role, too (or even the main role), but it's now working.
Thanks everyone here helping with knowledge and patience; @wfriedwald I hope you found a solution, too.
 
Last edited:
thanks for the helpful new info. the short answer is NO! I have had this problem for years and tried everything, nothing seems to work. I could start from scratch and re-try all these suggestions from over the last two years.

I have never been able to find the file ".metadata_never_index_unless_rootfs"

(I guess I can only conclude it is not there, something else must be preventing spotlight?)

Okay, here are the results of my terminal session:

Last login: Sat Jul 29 12:53:52 on console
willfriedwald@Wills-MacBook-Air ~ % sudo rm /Volumes/Annex5_14TB/.metadata_never_index_unless_rootfs
Password:
willfriedwald@Wills-MacBook-Air ~ % sudo mdutil -i on /Volumes/Annex5_14TB
/System/Volumes/Data/Volumes/Annex5_14TB:
Indexing and searching disabled.
willfriedwald@Wills-MacBook-Air ~ % mdutil -sav
/:
Indexing enabled.
Scan base time: 2023-07-29 14:18:30 +0000 (132980 seconds ago), reasoning: '(null)'
/System/Volumes/Data:
Indexing enabled.
Scan base time: 2023-07-29 14:09:59 +0000 (133491 seconds ago), reasoning: '(null)'
/System/Volumes/Preboot:
Indexing enabled.
/Volumes/12T1:
Indexing enabled.
/Volumes/16TB:
Indexing enabled.
Scan base time: 2023-07-29 14:18:30 +0000 (132980 seconds ago), reasoning: '(null)'
/Volumes/3T:
Indexing and searching disabled.
/Volumes/6T:
Indexing enabled.
Scan base time: 2023-07-29 14:18:24 +0000 (132986 seconds ago), reasoning: '(null)'
/Volumes/8T:
Indexing enabled.
/Volumes/Annex5_14TB:
Indexing and searching disabled.
/Volumes/CM_2020(MAC2):
Indexing enabled.
/Volumes/Comics 4TB:
Indexing enabled.
/Volumes/JAZZ_4T:
Indexing and searching disabled.
/Volumes/OLD_STORAGE_5TB:
Indexing enabled.
Scan base time: 2023-07-29 14:18:29 +0000 (132981 seconds ago), reasoning: '(null)'
/Volumes/Vintage 7T:
Indexing enabled.
willfriedwald@Wills-MacBook-Air ~ %

As you can see, I have a whole bunch of external volumes (drives) connected.
For some reason, three of them are coming up as "searching disabled."

I still don't know how to fix it? Can anyone offer another suggestion?

Thanks again very much!

w
 
Oh thank you for getting back to me!

It seems like all my external drives, the ones that are properly indexing and the ones that are not, are all using the same format:

Mac OS Extended (Journaled)

is that what you're asking? again thanks for the very valuable help!

w
 
Can you please check or show here the output of e.g. ls -al /Volumes/JAZZ_4T/? This was commented before, and as @jcscol then pointed out, if there is a file .metadata_never_index_unless_rootfs, please delete it.

In the next step try removing the .Spotlight-V100 folder from the drive. Open a terminal and execute:

sudo rm -R /Volumes/JAZZ_4T/.Spotlight-V100/

Then restart your Mac. Check whether indexing is enabled. If it works, try the same procedure for the two other affected drives.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back