Spotlight index weights in at 20GB on a 160GB disk?

[tatonka]

Hey,

I usually had spotlight search disabled, cause I don't really use the search at all .. well that was until yesterday I was looking for a file on the backup drive. Needless to say it took quite a while to find it, so I went back to considering spotlight again.

Long story short I switched on indexing for both the local disk (160GB) and the USB Backup Disk (1.5TB) and left spotlight running over night .. it is not quite there yet .. but the index is already almost 40Gb in total .. that is 20GB locally and then another 19GB on the USB disk.

So my question .. is that normal? Does spotlight store the index of the external disk locally again? I can't imagine the index of the ~100GB I have stored locally to be that big ..
Is it something normal while indexing and will reduce itself when it is done?

People usually report their indexes well below 1GB for larger disk than I use here at least that was what I found googling.

Any tips, comment, help is appreciated.

Thanks for the answers.
T.

PS .. that is on a 10.6.4 system and a HFS file system on both drives.

 

[mulo]

yeah, depending on the amount of files you have on your external, 1.5TB is quite a bit to index.

 

[tatonka]

Well ... I am fine with the index on the USB Disk .. what I don't get is the 20 GB index on my local disk ..

T.

 

[Hal Itosis]

Well ... I am fine with the index on the USB Disk .. what I don't get is the 20 GB index on my local disk ..

T.

That sounds too large to me. How full is your HD, and how many files?

Spotlight tips

Perhaps you should turn off Spotlight, erase that index, check the HD with Disk Utility (and/or DiskWarrior), and then try indexing again.

 

[gnasher729]

Well ... I am fine with the index on the USB Disk .. what I don't get is the 20 GB index on my local disk ..

T.

Imagine a network drive connected via WiFi, you see how having it local would help. Now imagine a network drive with four Macs connected, all indexing at the same time into the same index file... That is not going to work.

 

[tatonka]

I have around 100GB data locally, there is quite a few small source files, but also good chunk of larger databases or the VM Ware disk files for example. So, a "normal" mix I would think.

I'll take the advice and try to rebuild the local index over night and see what it does.

@gnasher729
So that means, OSX is storing the indexes (indices?) of remote disks locally as well then?
In this particular case, it is a USB Disk, so only me who is accessing it .. besides it builds an index on the disc it as well (also ~30GB when I shutdown the disk earlier).


Thanks for the replys so far.
Tim

 

[glassmanCZ]

I ran into similar problem, I think. On my MacPro with 60GB SSD disk as boot the index went to around 10GB. This doesnt make any sense. There is second disk, 750GB, with quite large number of files, mostly jpegs. BUt even with that into account, the index seems way too large to me.

On my MBP with 120GBdisk, with about 100GB data on it its only about 180MB large.

The Spotlight indexing takes very long on the MacPro, even when I put the large disk into Privacy in prefs, so I beleive there must be something wrong with the mds service. Did all the Disk Utility checks and repairs and the filesystem looks clean and healthy.

Would like to identify the problem, as now I had to turn Spotlight off completely.

Pavel

I have around 100GB data locally, there is quite a few small source files, but also good chunk of larger databases or the VM Ware disk files for example. So, a "normal" mix I would think.

I'll take the advice and try to rebuild the local index over night and see what it does.

@gnasher729
So that means, OSX is storing the indexes (indices?) of remote disks locally as well then?
In this particular case, it is a USB Disk, so only me who is accessing it .. besides it builds an index on the disc it as well (also ~30GB when I shutdown the disk earlier).


Thanks for the replys so far.
Tim

 

[tatonka]

Hey,

I never got my issue solved. I rebuild the index several times with varying folders excluded, but never got to the bottom of it. I eventually gave up and do not use spotlight at all anymore.

The only thing I could suggest would be to exclude everything but one folder to find the folders/files where mdrun goes crazy.

I use quicksilver to launch applications, so my need for spotlight was rather limited .. I ussually know where my files are, so I rarely need to search, in which cases I use find from the shell now.

T.

 

[glassmanCZ]

Seems we ended up in exactly the same position. Also tried to rebuild many times with different tricks to no avail. I also use Quicksilver as launcher. Still it irks me that such a useful feature of the OS is not working properly.

Pavel

Hey,

I never got my issue solved. I rebuild the index several times with varying folders excluded, but never got to the bottom of it. I eventually gave up and do not use spotlight at all anymore.

The only thing I could suggest would be to exclude everything but one folder to find the folders/files where mdrun goes crazy.

I use quicksilver to launch applications, so my need for spotlight was rather limited .. I ussually know where my files are, so I rarely need to search, in which cases I use find from the shell now.

T.

 

[glassmanCZ]

Thread bump

Wondering if anyone has similar problem, and even better a solution...?

The mds today went into this vicious loop:

Jun 7 18:40:41 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:40:41 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk
Jun 7 18:40:41 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:40:41 MacPro WDDriveManagerStatusMenu[982]: *** attempt to pop an unknown autorelease pool (0x103d200)
Jun 7 18:41:15 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK, VQ_VERYLOWDISK (516)
Jun 7 18:41:15 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk, very low disk
Jun 7 18:41:15 MacPro mds[3812]: (Warning) Volume: Indexing suspended on volume "/" because it is low on disk space.
Jun 7 18:41:15 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:41:15 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:41:15 MacPro mds[3812]: (Warning) Volume: Indexing resumed on volume "/" because it is no longer low on disk space.
Jun 7 18:44:22 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:44:22 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk
Jun 7 18:44:22 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:44:36 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK, VQ_VERYLOWDISK (516)
Jun 7 18:44:36 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk, very low disk
Jun 7 18:44:36 MacPro mds[3812]: (Warning) Volume: Indexing suspended on volume "/" because it is low on disk space.
Jun 7 18:44:36 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:44:36 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:44:36 MacPro mds[3812]: (Warning) Volume: Indexing resumed on volume "/" because it is no longer low on disk space.
Jun 7 18:48:03 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:48:03 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk
Jun 7 18:48:03 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK, VQ_VERYLOWDISK (516)
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk, very low disk
Jun 7 18:48:19 MacPro mds[3812]: (Warning) Volume: Indexing suspended on volume "/" because it is low on disk space.
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 received event(s) VQ_LOWDISK (4)
Jun 7 18:48:19 MacPro mds[3812]: (Warning) Volume: Indexing resumed on volume "/" because it is no longer low on disk space.
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk1s2', low disk
Jun 7 18:48:19 MacPro KernelEventAgent[61]: tid 00000000 found 1 filesystem(s) with problem(s)

Th disk is 60GB with 11 GB free, even with spotlight index of 10GB size already on it. I am going really nuts because of this.

Pavel