Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

How can I delete over 1,000,000 tiny files?

Rachel Faith

Rachel Faith

macrumors regular
Original poster
Apr 24, 2007
126
17
Iowa
I moved a folder to the trash bin and sadly it had millions of tiny files in it and when I clicked empty trash it began the counting of those files and has been stuck counting for 2 weeks now.

Over 1 million and still "preparing" to empty the trash.

I will need to reboot sometime and it really I mean really hogs the cpu. LOL

So, is there some terminal command way to just empty them all and free up about 40 GB of space without letting this run for 2 months non stop?

I've exhausted google.

Thanks.
 
Already tried the option key it still tries to "read" the whole directory before hand.

By the way, I had to stop the previous counting, so now when I try some other option it starts over at 1 file, then 10, then a few minutes later it is nearing 1000. I am sure if I left all day, and came home tonight, it would be couting in the 10's of thousands and THAT is the problem.

Secure erase is NOT enabled and is NOT the problem.

The problem is the folder may have 10 million files and the OS tries to COUNT them before just doing the empty and the counting is what I must avoid and just get the space back.
 
Rachel Faith said:
Already tried the option key it still tries to "read" the whole directory before hand.

By the way, I had to stop the previous counting, so now when I try some other option it starts over at 1 file, then 10, then a few minutes later it is nearing 1000. I am sure if I left all day, and came home tonight, it would be couting in the 10's of thousands and THAT is the problem.

Secure erase is NOT enabled and is NOT the problem.

The problem is the folder may have 10 million files and the OS tries to COUNT them before just doing the empty and the counting is what I must avoid and just get the space back.
Click to expand...

Maybe try from Terminal? Open Terminal (in Utilities) and enter

Code: 
cd ~/.Trash; rm -fr *
 
On the apple.com forum, someone suggested

rm -R ~/.Trash

followed by

killall Finder

exit

What is the difference between this and your idea?

Thanks
 
Rachel Faith said:
On the apple.com forum, someone suggested

rm -R ~/.Trash

followed by

killall Finder

exit

What is the difference between this and your idea?

Thanks
Click to expand...

What's quoted here will remove the .Trash folder as well as its contents. The killall Finder command will simply stop and automatically restart Finder. I assume this will recreate the .Trash folder. The exit command will simply log you out of the Terminal shell you're in. (Not really needed; just quit Terminal when you're done.)

The -R is equivalent to my -r, and my -f means "force" removal.

So, in short, I think mine is better. :) And it works; I just used it. (Though I didn't have 10^6 files in my Trash).
 
Neither yours nor his worked. The folder is still there and so are all the files. 30 GB and over 1 million files. This is after reboot and terminal. Both tries, both separate times.

Is there a way to MOVE these files to the desktop and then force delete them? Windows would move, Mac seems to only copy which wouldn't help.
 
OK, I guess this is not near as easy as I had hoped.

Time for, as Paul Harvey used to say, "The REST of the Story".

Perhaps something in the details will present a clearer picture.

I run OS X SL. I run XP in bootcamp. The problem began over there on the XP C: drive. I ran a wipe program which crashed and left all these random data files with long names and random extensions.

I tried to clean it up there and could not. But I could MOVE the files. So I moved them to one directory and booted to OSX hoping that Mac wouldn't have the problems that XP is so well known for having.

I then used finder to remove that folder. POOF it worked. The folder was gone. Yea!!

All except that NOW.. OSX won't finish it. It did do what windows would not do, which was delete it, but OSX wants to CHECK IT before it actually deletes it.

So, I can't do anything with it in XP booting to C: because it is NOT there anymore. Windows passed it off to Mac and thinks its all done. But Mac won't finish the job it started. So the Recycle Bin says the files are GONE and Trash Can won't empty. The files are some damned place on this one partitioned drive but where and who is able to actually deal with them remains in international dispute. LOL

Next idea?
 
Sounds like whatever it is was originally on a different volume, that's why Mac is trying to copy to the desktop instead of move. Windows does the same thing by default.

To move hold down the command key to move instead of copy, but it's probably going to start counting again.

That also suggests to me that you're trying to remove the folder from the wrong .Trashes folder. If your million+ files were originally on a drive besides the System drive try:
Code: 
sudo rm -rf /Volumes/"[B]My Drive's Name[/B]"/.Trashes
 
Thanks Darth. That looks worth trying. Just tell me how to know the exact volume name? It is a standard C drive bootcamp generic set up with no frills by the book.

This rabbit hole keeps getting deeper. The drive is NTFS not FAT32. Google seems to think that this is the problem since Mac can't do something with NTFS.

Well. THAT did something... Terminal locked up with a beachball and has NOT returned to the Rachels-Macbook-Pro prompt. Just a little grey box on the line below the password... frozen? working? Or.... COUNTING secretly?

I think it is counting.... Sigh.

I am going to go out and get a bite to eat and come back in an hour. If terminal is still just sitting there, I will hard reboot and post again.
 
Last edited by a moderator:
FAT32
  • Read/Write FAT32 from both native Windows and native Mac OS X.
  • No individual file larger than 4GB.
  • Maximum volume size: 2TB
NTFS
  • Read/Write NTFS from native Windows.
  • Read only NTFS from native Mac OS X
  • To Read/Write/Format NTFS from Leopard or Snow Leopard: Install MacFUSE and NTFS-3G. You can install both with NTFS-3G for Mac OS X.
  • Maximum volume size: 256TB
 
I DO use macfuse to read and write my NTFS. That was part of the problem. I should have just let windows deal with it. I thought Max was soooo superior that it would just DO what I told it to do without trying to THINK for me.

Seems that even though we have 2TB hard drives these days, the OSs of Mac and windows both still cannot handle a folder with 100s of thousands of files in them. Legacy still haunts us.
 
Interesting problem.

1. Try going back to Windows and deleting them there. Do it via a CMD command (windows version of Terminal). I had to delete 500,000 small files on an old XP machine, and it was impossible in Windows, but a CMD command did the job in about an hour or two. I'm not skilled enough to give the right command to you, but it was similar to the OSX command.

2. Is it possible to take the HD out, and use it as an external drive on some other computer? That might work a bit better. Again, use a Terminal or CMD text interface to do it.

3. Finally, if the whole job takes too long, break it up. Try creating a text command that only deletes files starting with A. Then if that works, only delete files starting with B and so on.

You could test this by navigating in Terminal or CMD to the right directory and then listing only files starting with 'a': "ls a*.*" or "dir a*.*" For fast testing, try only listing files starting with 'aa' etc.

Hope that's a few ideas. Being able to navigate to the right folder in terminal or CMD is a vital skill.

cd changes directory (same in both os)
'cd Desktop/files/' takes you two folders down to Files.

cd .. goes up one folder level

ls = list (osx)
dir = directory (windows)

rm = remove

Osx - drives are in /Volumes. That means "cd /" which takes you to the topmost folder of the OS (similar to C: in windows, but different) then "cd Volumes" which takes you to a list of the drives/partitions on your Mac.

Hope that helps. Play around with cd and ls / dir, and Google is your friend.
 
I found a program in XP which deletes using the short name in a reboot bat file. Took 4 hours not 4 weeks. All is fine. Thanks to all who tried and gave me ideas.

PAX
 
you may be doing a fresh install soon ...

how did you ever decide that 1 million files need to go?

good luck:cool:
 
Rachel Faith said:
Seems that even though we have 2TB hard drives these days, the OSs of Mac and windows both still cannot handle a folder with 100s of thousands of files in them. Legacy still haunts us.
Click to expand...
For future reference, it's worth noting that while any OS is going to take a while to delete a million files (and that time will get longer for SSDs for OSes supporting TRIM), the specific issue here was that the 3rd party read-write NTFS drivers for the MacOS just aren't very fast. In particular, in my experience, they can take quite a while to delete files or copy large numbers of small files--seems to be some inefficient overhead relating to individual file transactions.

That's why using Windows to clear the trash contents, while still not speedy, was much faster. On an HFS volume OSX is considerably faster deleting things, but as with any other OS limited by the fact that the OS still needs to go through the directory data stored on the drive and remove literally a million entries, which takes a bit. The same would be true for any database, however efficient.
 
Rachel Faith said:
I found a program in XP which deletes using the short name in a reboot bat file. Took 4 hours not 4 weeks. All is fine. Thanks to all who tried and gave me ideas.

PAX
Click to expand...

4 hours is about right to delete a million small files from the command line. Which is probably more or less exactly what the BAT file did. You would have had the same results navigating to the same folder in CMD and running rm -r *.*

The guy above is right it would have been better to do it in Windows CMD rather than OSX Terminal as you're on NTFS and using the free but slow MacFuse driver not the commercial but faster driver (which only costs about $30 if I remember right).
 
Too late for the OP, but perhaps not too late for the next person with a similar problem.

I would have used SuperDuper or Carbon Copy Cloner, or similar, to create a bootable clone of the HD - not including the trash folder. Booted off the clone, format the HD, and then restored the clone back to the HD. Drastic measures, then there is the risk of an unrecoverable error... so hold it for a last ditch effort.
 
snberk103 said:
I would have used SuperDuper or Carbon Copy Cloner, or similar, to create a bootable clone of the HD - not including the trash folder.
Click to expand...
I don't think you read the whole thread; the problem was that the files weren't on the boot drive, they were on a secondary NTFS volume, so cloning the boot drive wouldn't have made any difference.

I'm pretty sure most Mac cloning utilities can't clone an NTFS volume, and (again, given the weakness of the free NTFS read-write drivers) even if they could, the clone operation would take a similarly long time unless there were only a few files on the drive that weren't getting deleted. In which case it'd be faster to bypass any sort of clone and just copy those few files to another drive, reformat, and copy them back. There's Winclone, but pretty sure that's a block-level clone utility, so you can't selectively not clone things on the drive.

You could have tried the same tack with a Windows cloning utility, but I doubt it'd be faster than just deleting them via Windows, which ended up working in a tolerable amount of time.
 
Makosuke said:
I don't think you read the whole thread; the problem was that the files weren't on the boot drive, they were on a secondary NTFS volume, so cloning the boot drive wouldn't have made any difference.

I'm pretty sure most Mac cloning utilities can't clone an NTFS volume, and (again, given the weakness of the free NTFS read-write drivers) even if they could, the clone operation would take a similarly long time unless there were only a few files on the drive that weren't getting deleted. In which case it'd be faster to bypass any sort of clone and just copy those few files to another drive, reformat, and copy them back. There's Winclone, but pretty sure that's a block-level clone utility, so you can't selectively not clone things on the drive.

You could have tried the same tack with a Windows cloning utility, but I doubt it'd be faster than just deleting them via Windows, which ended up working in a tolerable amount of time.
Click to expand...

oops... missed that part... thanks, and sorry....
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back