Apple Seeds iTunes 10.5.1 Beta 2 for Continued iTunes Match Testing

[Tinmania]

OK I can finally confirm that a track that was ineligible due to low bitrate can in fact by up-converted to higher bitrate and then matched.

I just took an old 64kbps WMA from my old Pocket PC days, up-converted to 128kbps MP3 and imported into iTunes. It was matched very quickly.

By the way... you don't need to rescan your iTunes library for Match: anytime I add a new song to iTunes it immediately analyzes it for iTunes Match, and does whatever is needed. I have not had to manually start it.


Michael

 

[bookemdano]

That was me that did that, but it was possibly an unusual song offered for free on Facebook.

As for bringing up Shazam, I didn't mean they analyze as much as Match would need to. Only that they recognize tracks in DEPLORABLE conditions and get it done. We are talking a crappy radio station in a noisy car and yet it does it--probably at far less than 96kbps too. Also, Match uses the same wave form analysis as iTunes TuneUP and it has no such bitrate limitation. But it does not matter as that is the way Match works so deal with it we must.

First of all I know iTunes TuneUp claims to use the same technology as Apple but I don't believe it. Apple bought out Lala in order to use the proprietary analysis technology they developed. I just don't see how iTunes TuneUp figures into that picture.

Second of all, yes Shazam does a great job spitting out its best guess of what song you are listening to. But I have had it fail on me about 3 times out of 10--once where it couldn't make any guess at all and twice where it made suggestions but they were totally wrong. Furthermore, if I played it a live version of Alanis Morisette's "Ironic" it would identify the song as "Ironic" but could not tell me what specific recording of the song it was (that should be obvious anyway since Shazam and SoundHound don't even "listen" to the entire song).

My point was that matching 7/10 times is not good enough for Apple. And furthermore they want to be able to match very precise differences (remasters, edits, remixes, etc.) In order to get that level of precision you need a certain amount of audio quality to begin with (the old GIGO principle--"garbage in, garbage out). Extremely low bitrate files have already permanently lost so much detail (think of a severely compressed/pixellated JPEG) that Apple's matching algorithm can't differentiate it with sufficient confidence.

Edit: Ah, I see now that it's actually a gracenote technology that iTunes Tuneup uses, and that gracenote claims that's what Apple's using too for Match. I guess that makes sense, but I thought that's what the purchase of Lala was for. Maybe Apple is somehow using both technologies for more precision.

And like I said before, maybe transcoding a very low bitrate file up to a higher one will allow the file to match in some percentage of cases, but it's probably not reliable enough for Apple to allow the use of very low bitrates.

Edit2: Thanks for the report on subsequent adds of music to an already-matched library. I haven't seen that reported anywhere else so it's gratifying to see they will let you match additional music on an ad-hoc basis rather than forcing a complete re-analysis.

 

[Tinmania]

First of all I know iTunes TuneUp claims to use the same technology as Apple but I don't believe it. Apple bought out Lala in order to use the proprietary analysis technology they developed. I just don't see how iTunes TuneUp figures into that picture.

Second of all, yes Shazam does a great job spitting out its best guess of what song you are listening to. But I have had it fail on me about 3 times out of 10--once where it couldn't make any guess at all and twice where it made suggestions but they were totally wrong. Furthermore, if I played it a live version of Alanis Morisette's "Ironic" it would identify the song as "Ironic" but could not tell me what specific recording of the song it was (that should be obvious anyway since Shazam and SoundHound don't even "listen" to the entire song).

My point was that matching 7/10 times is not good enough for Apple. And furthermore they want to be able to match very precise differences (remasters, edits, remixes, etc.) In order to get that level of precision you need a certain amount of audio quality to begin with (the old GIGO principle--"garbage in, garbage out). Extremely low bitrate files have already permanently lost so much detail (think of a severely compressed/pixellated JPEG) that Apple's matching algorithm can't differentiate it with sufficient confidence.

Regardless I would bet I can take 100 random but known matchable tracks, down-convert to 64kbps, then up-convert to 128kbps and iTunes Match will still recognize them. Care to wager?

I have listened to 64kbps music when portable devices had mere megabytes of on-board storage and it was not THAT bad. I have little doubt there would not be much difference in a wave analysis situation since there are already differences between compression algorithms at the same bitrate.

I still assert it is more of a licensing issue: I can see how the record companies would not want that poor of a quality track turned into a much better 256kbps version.

And indeed that was the case of the track I just tested, that was 64kbps then up-converted and quite easily matched.

While I did not "buy" that track it was not obtained illegally: I used a Windows application called Total Record Pro, which had the ability to capture the audio stream and save it, to save songs from an Internet streaming service (which at the time streamed at only 64kbps). This was perfectly legal as it was akin to recording for personal use from the radio (USA). Total Recorder Pro could read song titles in the streaming player app and save and label songs separately. So I would let it stream and save for hours.

The results were merely adequate which is why I have not touched those tracks in years. Indeed this particular track is not the same length as the iTunes version. But it matched instantly. And I now have access to a MUCH better version of the same track.

But my point is if iTunes match worked fine with that track, considering its origins and not just bitrate, it would seem to bolster my assertion that the greater than 96kbps requirement is likely more than just a technical issue.


Michael

 

[bookemdano]

Image

Really strange. I notice both tracks were downloaded (they don't show the cloud icon). Did they appear mis-sorted prior to you downloading them? What happens if you turn off iTunes match on the iPhone and then turn it back on?

Do you have another iOS device you can turn iTunes Match on to see if the tracks appear mis-sorted there as well?

----------

Regardless I would bet I can take 100 random but known matchable tracks, down-convert to 64kbps, then up-convert to 128kbps and iTunes Match will still recognize them. Care to wager?

...

But my point is if iTunes match worked fine with that track, considering its origins and not just bitrate, it would seem to bolster my assertion that the greater than 96kbps requirement is likely more than just a technical issue.

Not interested in wagering. Let's face it, we're both just guessing here (unless you're privy to some insider knowledge I am not). I don't think it's possible to say conclusively either way without testing all kinds of different scenarios.

But I still say that even if transcoding a low-bitrate file to a higher one works in tricking iTunes Match some percentage of the time into matching doesn't mean that it works well enough for Apple to support it as a rule.

For example, a couple of pages back someone asked about his first edition of a U2 album vs. a remastered version that was released later. It's those fringe cases where iTunes Match might need higher resolution waveforms to tell the difference. Shazam could easily tell you the name of the song, but without high-enough resolution it may not be able to identify very subtle differences.

It may be the track you just tried as a test was an "easy" match and that's why it worked. I don't know... but feel free to keep testing and report back your progress

Edit: The reason I think it has nothing to do with the record companies desires is that they have already accepted that they are in essence giving amnesty to pirates via this service. I've pirated plenty of music in my time (yes, I feel bad about it) and not one file I downloaded illegally was ever lower than 128kbps (most are much higher than that). So if this bitrate rule were to try to prevent people from matching illegally-obtained music it's not likely to be very effective.

 

[Tinmania]

Well I have literally thousands of those 64kbps stream-captured tracks and have started a batch conversion up to 128kbps.

I have already loaded the first 221 of them into iTunes and they are being matched at a better rate than the songs that were in my iTunes library when recently Match became available again.

Fingers crossed!



Michael

 

[TexasTexan]

Well I have literally thousands of those 64kbps stream-captured tracks and have started a batch conversion up to 128kbps.

I have already loaded the first 221 of them into iTunes and they are being matched at a better rate than the songs that were in my iTunes library when recently Match became available again.

Fingers crossed!

Bet you are a happy camper. Ponders recording internet radio to see how well it works... lol

 

[Tinmania]

Bet you are a happy camper. Ponders recording internet radio to see how well it works... lol

To think how many times I nearly just deleted those files, thinking I would never bother with them again. Who knew!

Another thing I found interesting...

I am getting tracks matched that are definitely not available on iTunes. AC/DC's Highway to Hell matched and you can't get any AC/DC music in iTunes. Hmmmmm.....



Michael

 

[TexasTexan]

hmm that is strange you can match what isn't there. I have lots of assorted 80's cds which definitely don't seem to be getting matched. I was matched overnight before so I don't know what is taking so long. I am still at 543 to 2,643 left to upload. I even looked outside to make sure my internet antenna is pointed in the correct direction (no that is not a joke)

I love listening to streaming radio at times but hate hearing a song I don't have Well maybe I can, dying to try and see if it works well (for me anyways)

*edit* My Razor's Edge was match but Back in Black and Who Made Who are still "waiting" The list hasn't seem to update as it goes through the tracks. Guess it has to finish completely this time before they update.

 

[milo]

If they did that then if your meta data was for the remastered version, Apple checks your waveform against that one.

From a waveform standpoint, the difference between an older file of a song and a remaster is likely way too subtle for an algorithm to tell the difference. Things like live versions and fairly drastic remixes, explicit/non, sure (and in many cases those versions have the live/remix info in the tags).

Beyond that, I don't know if Apple wants to go to the trouble of having multiple masterings of the same song, and telling the difference. And I doubt that there are that many people who would even notice the difference on a remaster and not be satisfied with whatever the latest mastering of a song is.

And furthermore they want to be able to match very precise differences (remasters, edits, remixes, etc.)

Do they, at least in the case of remasters? Obviously some users want that but is there any reason to believe Apple agrees with that. They definitely should recognize edits and remixes as well as live versions but in most cases those should be fairly easy to figure out since the lengths are usually different and the metadata often mentions live/remix/etc. It doesn't seem like the extra quality should be necessary to tell the difference in those cases.

Regardless I would bet I can take 100 random but known matchable tracks, down-convert to 64kbps, then up-convert to 128kbps and iTunes Match will still recognize them.

Totally agree, as bad as 64k sounds it should still be close enough to identify tracks without much trouble. I'd bet the limitation is either based on licensing or just Apple's way of covering themselves when people have super low quality tracks it can't identify. So most of the time it should work but Apple is being conservative.

So can anyone with Match do a test of setting an audio file to a type other than "music" (audiobook etc) to see if that works for excluding files? The 25k limitation (with no way to manually exclude to stay below that limit) is probably the biggest thing that would keep me from trying it.

 

[Tinmania]

hmm that is strange you can match what isn't there. I have lots of assorted 80's cds which definitely don't seem to be getting matched. I was matched overnight before so I don't know what is taking so long. I am still at 543 to 2,643 left to upload. I even looked outside to make sure my internet antenna is pointed in the correct direction (no that is not a joke)

I love listening to streaming radio at times but hate hearing a song I don't have Well maybe I can, dying to try and see if it works well (for me anyways)

*edit* My Razor's Edge was match but Back in Black and Who Made Who are still "waiting" The list hasn't seem to update as it goes through the tracks. Guess it has to finish completely this time before they update.

Just a quick fyi that I could have never done that manually. The app I used was able to tell when songs ended, by pauses and by the track name changing, and auto create the files. But I am sure I could do it manually nowadays since we can pretty much "pick" a song of our choosing and have it play right away. Back then it was take what we give you lol.




Michael

 

[bookemdano]

From a waveform standpoint, the difference between an older file of a song and a remaster is likely way too subtle for an algorithm to tell the difference.

I disagree. Obviously it depends on the nature of the specific remaster but the idea that the waveform doesn't betray the (even inaudible) differences is just not true. Here's just one example--from Nine Inch Nails' "The Downward Spiral"-- http://www.theninhotline.net/features/halo.profiles/tds/zoomout.gif

Discussion here: http://www.theninhotline.net/features/halo.profiles/tds/

Beyond that, I don't know if Apple wants to go to the trouble of having multiple masterings of the same song, and telling the difference. And I doubt that there are that many people who would even notice the difference on a remaster and not be satisfied with whatever the latest mastering of a song is.

Now that you may be correct about--I doubt Apple keeps around multiple versions of albums that have been remastered (but I don't know). But I think that in developing their algorithms they probably wanted to give it the ability to discern between masterings in case they are presented with such cases in the future. It's always preferable to start out with more data than less. They can always compress and remove that data later but you can't add back data that's already been compressed out of existence. Or maybe, even if they don't have both pressings of the album in their store, they can still tell that your version is a different remastering and thus will upload it rather than match it. I guess we'll see. It would be nice if they allowed each user to upload and override matching a certain number of times for cases where it matched to something obviously wrong.

Do they, at least in the case of remasters? Obviously some users want that but is there any reason to believe Apple agrees with that. They definitely should recognize edits and remixes as well as live versions but in most cases those should be fairly easy to figure out since the lengths are usually different and the metadata often mentions live/remix/etc. It doesn't seem like the extra quality should be necessary to tell the difference in those cases.

This is Apple we're talking about. Even if in most cases the particular remix/etc/live version can be determined via track length, metadata, and other criteria, Apple has a history of being sticklers about things like this--based only on the fact that it helps them ensure a top quality experience for most users. They want to make the risk extremely low that their system will match the wrong version of a track, (like the report from someone that an explicit version of one of their songs got matched with a "clean" version). That shouldn't happen, and beginning with a sufficiently high quality original file helps ensure that it doesn't.

Totally agree, as bad as 64k sounds it should still be close enough to identify tracks without much trouble. I'd bet the limitation is either based on licensing or just Apple's way of covering themselves when people have super low quality tracks it can't identify. So most of the time it should work but Apple is being conservative.

Again, I think it's Apple covering its ass to help ensure a good matching (and thus listening) experience for users paying them $25 a year for the service. I maintain if the music industry was concerned at all about pirated files being matched it would never have even allowed iTunes Match to exist in the first place.

 

[TexasTexan]

Actually I found an app that in the paid version, you can create a wish list. When it is found playing by monitoring several thousand stations, it is recorded and saved. Can also record multiple stations at the same time by choosing the stations you want. If any of the tunes I record out of the trial work ok, they will get my money I think.

 

[Tinmania]

Actually I found an app that in the paid version, you can create a wish list. When it is found playing by monitoring several thousand stations, it is recorded and saved. Can also record multiple stations at the same time by choosing the stations you want. If any of the tunes I record out of the trial work ok, they will get my money I think.

They may get mine too... what is it?



Michael

 

[TexasTexan]

It's called Audials RaioTracker. Unfortunately I just figured out the wishlist doesn't work in the trial, so I can't test if that works well or not. Think the trial will record 40 songs so I will see if it works ok or not except for the wishlist. Some of the stations you can record continuously then break up or many stations will break them up automatically. It is at http://audials.com Seems pretty cool for $40

 

[Tinmania]

This is Apple we're talking about. Even if in most cases the particular remix/etc/live version can be determined via track length, metadata, and other criteria, Apple has a history of being sticklers about things like this...

No evidence that I see backs up your position though.

I have now been grabbing old music that I thought was not worth bringing into my current collection.

I just dragged and dropped an old MP3 (128kbps) into iTunes from a SD card that was inserted into the card slot of a wireless printer (so I could use any machine to access it). It took less than 3 seconds to copy over and also become matched in iTunes. I had never once had this track in iTunes, let alone iTunes Match.

I don't see how it could be doing the kind of analysis you are suggesting in such a short amount of time. It takes around 2 seconds just to copy it somewhere. I don't see a high-level of scrutiny occurring in only one second.



Michael

----------

It's called Audials RaioTracker. Unfortunately I just figured out the wishlist doesn't work in the trial, so I can't test if that works well or not. Think the trial will record 40 songs so I will see if it works ok or not except for the wishlist. Some of the stations you can record continuously then break up or many stations will break them up automatically. It is at http://audials.com Seems pretty cool for $40

Thanks!

 

[bookemdano]

I don't see how it could be doing the kind of analysis you are suggesting in such a short amount of time. It takes around 2 seconds just to copy it somewhere. I don't see a high-level of scrutiny occurring in only one second.

Well truth be told it's a big mystery what Apple is doing because they haven't talked about it. Perhaps it does a cursory analysis first and if that is good enough to get a high-confidence match then it uses that. And if the song doesn't match or matches multiple iTunes masters then it does a more detailed fingerprinting to obtain more source data.

Just as you say there is no evidence of my explanation for requiring high bitrate files I also see no evidence of your explanation (that the music labels somehow imposed it for "licensing" reasons). If they didn't want people using pirated music in iTunes Match then why would they have put their stamp of approval on it in the first place? It just doesn't make any sense.

 

[TexasTexan]

You're welcome! If you test it and the tracks work well, please let me know as I am still sitting in step 3 and can't test it myself. I am really, really curious how well the wishlist works. That is the part I want to test, so if you buy it and test that part, let me know

 

[milo]

I disagree. Obviously it depends on the nature of the specific remaster but the idea that the waveform doesn't betray the (even inaudible) differences is just not true.

You're right, I should have said some remasters. In the case of remastering and adding a ton of limiting/compression it's probably not hard to tell the difference. But other remasters are much more subtle, if it's just things like EQ and removing tape hiss I expect there would be some cases where it couldn't tell.

It really depends on the material and the remaster, but I wouldn't count on it being able to tell the difference every time.

It would be nice if they allowed each user to upload and override matching a certain number of times for cases where it matched to something obviously wrong.

I agree about that, there seem to be cases where it's wrong and it would be nice to force the upload instead of match in at least some situations where there's an audible difference.

Apple has a history of being sticklers about things like this--based only on the fact that it helps them ensure a top quality experience for most users.

I'd be more inclined to agree if iTunes didn't do such a bad job of finding album art. Sometimes seemingly random wrong art, other times nothing found for albums that are in the store with identical artist/album/title. I'd bet if someone did a test with a bunch of files converted down to 64k then back up, there'd be little if any difference in matching success rate.

I just dragged and dropped an old MP3 (128kbps) into iTunes from a SD card that was inserted into the card slot of a wireless printer (so I could use any machine to access it). It took less than 3 seconds to copy over and also become matched in iTunes. I had never once had this track in iTunes, let alone iTunes Match.

I forget, has anyone done a test switching the metadata on files? Is there any reason to believe that it looks at the waveform at all?

 

[bookemdano]

I'd be more inclined to agree if iTunes didn't do such a bad job of finding album art. Sometimes seemingly random wrong art, other times nothing found for albums that are in the store with identical artist/album/title. I'd bet if someone did a test with a bunch of files converted down to 64k then back up, there'd be little if any difference in matching success rate.

Yeah the album art fetching has been flakey--although any time I have used it in the last six months or so it worked. The only problems I had were for brand new music that had just been released... seems like it took a week or two before it could fetch new release album art.

But I think that's another reason Apple's allowing iTunes Match to use your existing cover art with the service. So people like me who are anal about tags and covers don't experience a situation where we go from having a perfect iTunes library to having a mess of mismatched or missing album art and other metadata after the match process. They know if you've bothered to embed album art in your files then you want to use that instead of whatever they guess you'll want to use.

I forget, has anyone done a test switching the metadata on files? Is there any reason to believe that it looks at the waveform at all?

Yes I think in this or one of the other iTunes match threads here someone mentioned trying to fool iTunes match by removing all the id3 tags from an mp3 file and renamed it to something like 123.mp3 and iTunes still matched it. So it either totally ignores metadata for the matching process or it uses it only in a secondary capacity if the waveform analysis doesn't yield a high-confidence single match.

Hopefully all of these little details will get figured out and published by smart folks once the service is released. I bet a there are some media organizations (ahem Ars Technica) that already have all of these details but are waiting to release them until after the service goes public.

 

[milo]

Yes I think in this or one of the other iTunes match threads here someone mentioned trying to fool iTunes match by removing all the id3 tags from an mp3 file and renamed it to something like 123.mp3 and iTunes still matched it.

I've been following these threads and don't remember seeing a confirmation either way. Could someone testing the service try it out and confirm either way?

 

[Senseotech]

I too noticed that AC/DC got matched despite it not existing in the store. Of course, all my AC/DC but "You Shook Me All Night Long" are matched, with that one being uploaded. Another artist thats not in the store, Tool, got totally uploaded, no matching there, as was my single country album by Garth Brooks, also not a store album, and thus was uploaded.

 

[Tinmania]

Well truth be told it's a big mystery what Apple is doing because they haven't talked about it. Perhaps it does a cursory analysis first and if that is good enough to get a high-confidence match then it uses that. And if the song doesn't match or matches multiple iTunes masters then it does a more detailed fingerprinting to obtain more source data.

I agree that it if it does not match right away it goes though a more detailed analysis. But many of the songs I am now getting matched don't even match the correct song length. In one the last 4 seconds were missing and it still matched. Granted, that could be because it matched right away and there was no other version of it to have to worry about so it matched it and moved on.

Just as you say there is no evidence of my explanation for requiring high bitrate files I also see no evidence of your explanation (that the music labels somehow imposed it for "licensing" reasons). If they didn't want people using pirated music in iTunes Match then why would they have put their stamp of approval on it in the first place? It just doesn't make any sense.

Sure it does. That could have been a negotiating point that Apple finally agreed to--or offered. I can very easily see how just allowing this at all would be a big hurdle. But once you get past that, surely you have to draw the line on what is at least somewhat acceptable proof that the end users at least have a decent copy of the music in question from the get-go. I would assume some of the iTunes Match revenue goes to the record companies as well.




Michael

 

[Educ8r]

Today I was listening to my iCloud playlists on my AppleTV and noticed that a "clean" version of a song played instead of the original song I have on my computer.

Has this happened to anyone else?

 

[Tinmania]

Today I was listening to my iCloud playlists on my AppleTV and noticed that a "clean" version of a song played instead of the original song I have on my computer.

Has this happened to anyone else?

Yes. It has been reported that an explicit version matched to the clean version.

I am about to check one of mine now...

OK the one I checked, and to be certain there was no chance of a local copy I did with an ATV, was indeed the explicit version. In this case it was Rehab's Bartender Song. So for sure it is not a case of "only family-friendly" songs in iCloud.


Michael

----------

I too noticed that AC/DC got matched despite it not existing in the store. Of course, all my AC/DC but "You Shook Me All Night Long" are matched, with that one being uploaded. Another artist thats not in the store, Tool, got totally uploaded, no matching there, as was my single country album by Garth Brooks, also not a store album, and thus was uploaded.

Same as me on AC/DC... only Shook Me was uploaded. Weird, but better than I expected (I thought "forget about ANY AC/DC").


Michael

 

[Starhawk]

Today I was listening to my iCloud playlists on my AppleTV and noticed that a "clean" version of a song played instead of the original song I have on my computer.

Has this happened to anyone else?

Yes. Gwen Stefani The Sweet Escape matched to the clean version when the original track was explicit. There are probably others but that's the one I caught yesterday.