Help with iTunes duplicates

[Ravich]

I tried getting recommendations on how best to move itunes libraries to a new drive and received no response, so I tried it myself. I specified a new folder on a HDD in the advanced preferences, checked the box to automatically copy itunes to the new location, then dragged and dropped my 2 different iTunes libraries.

At first it seemed like it worked just fine, but I am finding duplicates EVERYWHERE. When I tell it to display duplicates, there is a list of 3377 tracks. I'm not deleting duplicates manually. I thought maybe it was because I dragged and dropped the files from one location and it copied them to another, but all of the duplicates are located in exactly the same location.

When I first discovered the problem I started deleting the duplicates, choosing to move the file to the trash. Given that duplicate tracks are in exactly the same location, how did this not create problems finding and playing said tracks? This doesnt make sense to me.




Can someone PLEASE tell me how I am supposed to go about this so as to not create a list of almost 4000 duplicate tracks?

 

[GGJstudios]

Here's a few of many threads on this topic that may have the solution you seek:

iTunes making duplicate tracks of the same songs
How to make an applescript to delete iTunes duplicates?
iTunes duplicate remover!
Does adding music to iTunes make duplicates?
Anyone have a solution for mass deleting duplicates in iTunes ...
Duplicate files - a growing problem

 

[stridemat]

http://dougscripts.com/itunes/itinfo/dupin.php

A nice Apple Script.

 

[Ravich]

I appreciate the links, but I'm not really willing to pay 15$ for something that makes iTunes run properly.

I tried to fix things up a bit by copying my empty iTunes folder from my home folder to the desired iTunes HDD and then dragging the folders from the desktop into iTunes. Since there was a bit of a change to the folder layout between 10.5 and 10.6 I think, I thought that might help. I still have over 3000 songs listed when I display duplicates.

The interesting thing is that I have 2 different music folders (both taken from other iTunes libraries I have been using), and one of them transfers over fine, while the other one basically duplicates every single track in that folder.



Anyone know what might be causing this?



The thing is that the problem isnt that iTunes is making copies of the files themselves. With duplicates, both tracks have the same source. The iTunes folder isnt any bigger than it should be given the amount of music. BUT, if I delete one of the duplicate tracks and select "move to trash", the file appears in the trash, but the remaining track in iTunes will still play, despite the fact that they had the same source file.

So... the problem exists purely within iTunes as far as I can tell.

 

[GGJstudios]

I appreciate the links, but I'm not really willing to pay 15$ for something that makes iTunes run properly.

What are you talking about that costs $15?

I prefer to manage my music files myself, rather than let iTunes make copies or manage the files. I have duplicates in my library, but only because I choose to. If a song by the same artist appears on two different albums by that artist, I want both albums to be complete, so I don't mind having duplicate songs in that case.

 

[Ravich]

Right, but the problem here isnt that there are legitimate duplicates. I'm talking about 1500+ tracks with the exact same info and source file appearing twice in my iTunes. Right next to one another.

In regards to the 15 dollars, I was referring to Dupin. As far as the threads you linked, the first one recommends manually deleting them (not an option for me), the second one I need to write my own script, the third one suggests Dupin, the fourth has no suggestion, the fifth opts for deleting and reimporting (already tried), and the sixth is about the Mac making actual copies of the files, which isnt what I'm dealing with.


And the reason I dont want to manage the files myself is because the iTunes interface is ideal for me as long as it isnt duplicating tracks for no reason while importing. As long as I can specify the album, artist, composer, genre, etc in iTunes and search for any keyword instantly, I dont so much care how the files themselves are organized.

 

[GGJstudios]

Right, but the problem here isnt that there are legitimate duplicates. I'm talking about 1500+ tracks with the exact same info and source file appearing twice in my iTunes. Right next to one another.

In regards to the 15 dollars, I was referring to Dupin. As far as the threads you linked, the first one recommends manually deleting them (not an option for me), the second one I need to write my own script, the third one suggests Dupin, the fourth has no suggestion, the fifth opts for deleting and reimporting (already tried), and the sixth is about the Mac making actual copies of the files, which isnt what I'm dealing with,

Of course, those weren't intended to be a definitive collection of threads on the topic, but just a sampling. The point is, searching the forum with MRoogle will many times reveal threads that contain solutions to your questions. It just takes some looking. The solution may be a manual one, even if that's not desirable. When it comes to deleting files, the safer approach is many times better, even if it's tedious. Doug's AppleScripts for iTunes is a great resource for managing your iTunes tags and files. You may find a free script there (most are free) that either meets your needs, or can be modified to do what you want.

 

[Ravich]

I did search google and MR for solutions, but didnt find anything. Since the problem exists purely within iTunes, I was hoping that even if I couldnt find a solution to automatically delete duplicates, that I could at least figure out what was causing this and reimport them. Like I said, going through 3500 tracks manually clicking on every other one isnt something I'm willing to do for now. As of right now I cant even figure out what the difference would be between simply deleting the track vs also moving the files to the trash.

 

[AndyMoore]

Sounds like your iTunes library file is getting itself into a bit of a state.
Have you tried consolidating the library?

If you're at the end of your tether, the last resort I would try is -

Stop iTunes
Back up your library file
Delete it
Restart iTunes
In preferences, let iTunes manage your music
Select where you want you music to go
Add the folders where your music is installed
iTunes will move it to where you selected
Job done


Now you will lose all your star ratings and play counts doing this.
As I said, last resort.

 

[Ravich]

Sounds like your iTunes library file is getting itself into a bit of a state.
Have you tried consolidating the library?

Just tried this but it didnt change anything.

If you're at the end of your tether, the last resort I would try is -

Stop iTunes
Back up your library file
Delete it
Restart iTunes
In preferences, let iTunes manage your music
Select where you want you music to go
Add the folders where your music is installed
iTunes will move it to where you selected
Job done

I tried this but I must be missing something. Closed iTunes, copied my iTunes library file (the one that was about 4kb in the iTunes folder, right?) to the desktop, then trashed it. Then I reopened iTunes, let iTunes manage my music (the box was already checked)... then I set my media folder to where it was already set (not sure what you mean for this step), and nothing happened, because the music was already there.

Now you will lose all your star ratings and play counts doing this.
As I said, last resort.

This definitely doesnt bother me, so no worries there.

 

[AndyMoore]

Just tried this but it didnt change anything.



I tried this but I must be missing something. Closed iTunes, copied my iTunes library file (the one that was about 4kb in the iTunes folder, right?) to the desktop, then trashed it.

How much music do you have? I'm not near my mac right now but I think that my iTunes library file is larger than 4k.
Set your media to be where you want it to be, not necessarily where it is at the moment. iTunes will then manage / move the files.

 

[tpg]

Did you delete both library files? There's an XML file as well as the binary file, both in the same folder. If either one has been deleted, iTunes will regenerate it from the other.

 

[Ravich]

Ah, ok. So I deleted both library files, then created a new iTunes folder on the HDD, and specified in preferences that that's where I want the media folder located. It updated the library for a bit and then asked me if I wanted it to do something to keep music organized according to the "keep music organized" option I had checked or whatever.

I said yes, and nothing changed. What exactly do you mean by "Add the folders where your music is installed"?

 

[AndyMoore]

I said yes, and nothing changed. What exactly do you mean by "Add the folders where your music is installed"?

I made an assumption, based on the fact that you were moving your music, that where your music is stored currently is not where you actually want it.

Open two finder windows and go to both the locations that you say your music is stored in and compare them. Are you sure that your duplicates in iTunes are not actually duplicated files between the folders?

 

[Ravich]

I already transferred the music to where I want it to be.

And upon closer analysis, there are in fact 2 files in my iTunes folder for any given duplicate. For example "The Awakening" and "The Awakening 1"

The thing is that the folder where I can copying the music from only has 1. Every time I add said folder into iTunes, it apparently makes a second copy of every. single. track in that folder, resulting in about 3500 tracks showing up when I tell iTunes to display duplicates. Why would this be?

 

[AndyMoore]

Ah. Vague memory flash . . .

In each folder, do you have a .m3u playlist file?

If you do then this is what is causing the duplicates.

 

[Ravich]

Hmm... no I cant find any m3u files. They are all m4a. I did find empty folders here and there, probably because this folder was a bunch of CDs I imported into iTunes and then altered from within iTunes. But I dont see why that would cause duplicate tracks to appear. The folders are empty and not common enough to cause every single track to be copied twice.

Would the m3u file not display for some reason?

 

[AndyMoore]

m3u files are not normally hidden, you should see them if they're there.

If you import a single track, does that get duplicated or is it only when you import a folder?

 

[Ravich]

Here's the thing: if I copy the entire folder (which contains within it a collection of folders), then it will duplicate tracks. If I look through the folder for a CD that I know it was duplicating all of the tracks from, and drag that folder individually, it doesnt duplicate any of the tracks.

Within the folder, there is:

-Automatically Add to iTunes (empty folder)
-Compilations (lots of folders inside)
-Movies (empty folder)
-Voice memos (empty folder)
-A couple of other folders that are specific albums

If I drag the "compilations" folder, it duplicates tracks. If I drag a specific folder from within the compilations folder that I know was previous duplicated, there is no issue. If I high light all of the folders within the compilations folder and then drag and drop them, I still get duplicate tracks.

So... there's something funky inside the compilations folder, right?

Thanks for all your help.


On a related note, in an absent minded course of action, I accidentally emptied my iTunes on my laptop instead of my Mac Pro (where I am configuring all of this). All of my laptops music files are now sitting in the trash. Is there an easy way to undo this? -.-

 

[AndyMoore]

On a related note, in an absent minded course of action, I accidentally emptied my iTunes on my laptop instead of my Mac Pro (where I am configuring all of this). All of my laptops music files are now sitting in the trash. Is there an easy way to undo this? -.-

In the Trash folder, there should be an option to Put them back.

Restoring deleted items to their previous location

If you put an item into the Trash from your Desktop or Finder window, you can restore it to where it was before it was placed in the Trash (if you haven't emptied the Trash yet).

Open the Trash (item 15 above) by clicking on it in the Dock. Select the item(s) to move back, then choose Put Back from the File menu. Or, you can Control-click or right-click the item and choose "Put Back" from the contextual menu.

 

[AndyMoore]

Here's the thing: if I copy the entire folder (which contains within it a collection of folders), then it will duplicate tracks. If I look through the folder for a CD that I know it was duplicating all of the tracks from, and drag that folder individually, it doesnt duplicate any of the tracks.

Within the folder, there is:

-Automatically Add to iTunes (empty folder)
-Compilations (lots of folders inside)
-Movies (empty folder)
-Voice memos (empty folder)
-A couple of other folders that are specific albums

If I drag the "compilations" folder, it duplicates tracks. If I drag a specific folder from within the compilations folder that I know was previous duplicated, there is no issue. If I high light all of the folders within the compilations folder and then drag and drop them, I still get duplicate tracks.

So... there's something funky inside the compilations folder, right?

Thanks for all your help.

As to this duplication issue, I'm at a loss. I'm not near a Mac so can't try any fault finding of my own. The only thing I can suggest now is to go back to the beginning.

Trash all you duplicates from Finder and start a new iTunes library.
Long winded I know with 3500 duplicates but unless anyone else has any suggestions then it's probably best.

 

[Ravich]

There doesnt seem to be a put back options either in the file menu or when I right click on the files in the trash,

Also, I'm not entirely sure what you mean by starting over with my library. I've already tried importing them into iTunes 4 times now, so a fresh start wont do anything.

 

[Ravich]

Figured it out...


within the compilations folder there was a bunch of folders (albums) in addition to another compilations folder with the duplicates. No idea why I didnt find those when I searched for duplicates earlier -.-

Anyway, sorry about that and thanks for the help.

 

[GGJstudios]

Figured it out...

Glad you got it worked out!

 

[AndyMoore]

Great, at least you got to the root of your problem.