Please help me figure this out (re: HD TV Show duplicate listings)

[sonictonic]

Hi,

I've been buying the HD version of Season 4 The Office all week, and it's been working great, up until tonight I just downloaded the last 3 episodes of the season, and just these three episodes, show up as duplicates on my Apple TV. Both the HD and SD versions are copied over. All the other episodes, it seemed to automatically sync the HD one over to ATV, and use the SD on my iPhone and iPod. So why now all of a sudden is it treating the last 3 episodes like they are each individuals?

This is kinda minor yeah, but pretty damn annoying. how does this happen? I can't figure it out... And how do I fix it? Any ideas?

Thanks in advance :) I'm trying to avoid having to call Apple support tomorrow. :p


EDIT:
my iPod just warned me too, that these 3 particular episodes cannot be synced because they are not compatible. The SD ones sync to my iPod, but I get a warning for the 3 HD copies. GRRR. :(

[sonictonic]

I guess my issue is maybe pretty isolated. Of course :p

I called Apple and they had never heard of this happening. They told me to email iTunes support for credit and to try and re-download the last 3 shows. So hopefully they will write back soon...

[camarobh]

I am having the same issue, only with all of The Office season 4 episodes. 30 Rock and other HD shows work correctly (although I have a different issue with that series as one of the episodes is mislabeled and actually the same as another episode). I tried changing the smart playlist I use to not sync season 4 (only HD season I bought) and once it was off the Apple TV I put it back and have the same problem.

[ManWhoCan]

Exact same problem here. The HD TV shows i downloaded in the first few days only show once on Apple TV but those I have downloaded in the last few days show twice with no indication which one is the HD copy.

It seems not to matter what the TV show is, just when it was downloaded.

[jaw04005]

We are all having this problem, unless you happen to be canceling the SD downloads (like me) and manually syncing.

The Apple TV needs a software update to beef up its compatibility with iTunes 8. Apparently, that's coming sometime within the next month.

As of now, it has no way to distinguish between the HD and SD versions of the file—and is just grabbing whichever (or in some cases both).

[camarobh]

The solution I came up with is to put the letters HD in the comment field for each HD episode, and then to add that to the smart playlist I use to sync content to my Apple TV. Works great, but not real elegant.

[sonictonic]

Well I hope you guys have emailed into iTunes tech support for this like I did. I've exchanged a couple emails with them now and the rep credited my account twice to try again 2 new times, and when the issue persisted, he forwarded it to the engineers and said it will be further investigated because it seems very odd and this should not be happening.

So, if you haven't already, please email into iTunes Support! I'm hoping for a fix soon because not only is the handful of dupes on my Apple TV annoying, but getting the error when syncing my iPhone or iPod that the HD copies cannot be syncd (duh) is annoying.

Oh, and the same issue just happened with the free Knight Rider episode 1 show too, btw.

When I hear back from iTunes support again I will post back. You guys too ok? :)

[RumMunkey]

Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_1 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F136 Safari/525.20)

what I want to know is, for that brief time when this WAS working, HOW was it working??

What was it in the SD version of the show that kept it out of the Appletv menu?

There's a bunch of stuff I'd love to apply this kind of filtering to.

Anybody know how this worked?

[zedsdead]

Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_1 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F136 Safari/525.20)

what I want to know is, for that brief time when this WAS working, HOW was it working??

What was it in the SD version of the show that kept it out of the Appletv menu?

There's a bunch of stuff I'd love to apply this kind of filtering to.

Anybody know how this worked?

I don't know, but I hope the people at Meta-X find out soon!;)

[jaw04005]

So, I woke up this morning to find iTunes complaining because my Apple TV's hard drive was completely full. :mad:

Apparently, it decided to sync both the SD and HD versions of House Season 2. I had to unsync (delete) the entire season to make the Apple TV even remotely operational again.

Now the device is running extraordinarily slow—despite freeing up disk space. I wonder if it's fragmented?

We need an Apple TV update and now. It's sort of ridiculous that the software update wasn't ready with iTunes 8.

[jhollington]

I suspect the problem actually exists with the content/headers of the TV show episodes themselves as they are distributed by the iTunes Store.

I'm having the same problem here with any new content now being downloaded... The series' which I downloaded the week of the announcement are all fine, including BSG S4, House S4 and Heroes S2. Now, over the past few days I've started seeing this problem with the first episode of House S5, the first TWO episodes of Heroes S3, and the Knight Rider 2008 first episode.

The distinction between the problem episodes and the working ones, however, is that iTunes 8 seems to have a "link" or "association" between the HD/SD pairs for those episodes that work, whereas the ones that are problematic are being treated as completely separate episodes.

With the working episode pairs, watching one version of an episode will mark the corresponding other version as watched also. Likewise, if you right-click on the HD or SD version of a "good" episode and choose an option such as "Mark as New" or "Mark as Not New" you should see both episodes change status.

With the problematic episodes, this doesn't happen... Only the specific episode that you're touching is actually affected -- the other one is ignored.

I can't find exactly how or where this "link" between the two episodes is supposed to exist... I suspect it lives somewhere in either a hidden header or an iTunes database entry, but at the same time I suspect iTunes 8 is supposed to get this information from the iTunes Store, and perhaps the episodes are not being flagged or associated properly at the original source, so iTunes sees them as two completely separate episodes rather than two versions of the same episode.

[sonictonic]

Exactly. I hope Apple finds and fixes this asap. I have gone in a full circle with iTunes email support. The rep said they passed it on to a tech to investigate, but the most recent email from him said he has credited my account for all the paid for episodes, and that's all he can do. He pretty much admitted the rest is over his head. What is annoying, is that he said to CALL Apple tech support. Well, I already did that first thing, and THEY said to email in for the support. WTF?

So guys, please call and email in. They need to know this is an issue and it isn't just the few of us I am sure. But we have to be the voice. :D

I suspect the problem actually exists with the content/headers of the TV show episodes themselves as they are distributed by the iTunes Store.

I'm having the same problem here with any new content now being downloaded... The series' which I downloaded the week of the announcement are all fine, including BSG S4, House S4 and Heroes S2. Now, over the past few days I've started seeing this problem with the first episode of House S5, the first TWO episodes of Heroes S3, and the Knight Rider 2008 first episode.

The distinction between the problem episodes and the working ones, however, is that iTunes 8 seems to have a "link" or "association" between the HD/SD pairs for those episodes that work, whereas the ones that are problematic are being treated as completely separate episodes.

With the working episode pairs, watching one version of an episode will mark the corresponding other version as watched also. Likewise, if you right-click on the HD or SD version of a "good" episode and choose an option such as "Mark as New" or "Mark as Not New" you should see both episodes change status.

With the problematic episodes, this doesn't happen... Only the specific episode that you're touching is actually affected -- the other one is ignored.

I can't find exactly how or where this "link" between the two episodes is supposed to exist... I suspect it lives somewhere in either a hidden header or an iTunes database entry, but at the same time I suspect iTunes 8 is supposed to get this information from the iTunes Store, and perhaps the episodes are not being flagged or associated properly at the original source, so iTunes sees them as two completely separate episodes rather than two versions of the same episode.

[CBlakeston]

I have the exact same problem (the first free episodes only show as HD on the Apple TV and now both are showing up) and thought I was going crazy so I'm so glad I'm not the only one :)

I think so much content has been added to the iTunes store lately some things have slipped. The premiere of Chuck season 2 for example, someone forgot to put a '2' in the Season number box in the info so it shows up as season less. (added it myself)

Here's hoping that an Apple TV Update will sort things out! Or Meta X comes to the rescue!

[jhollington]

I've tracked down what seems to be the cause of the problem, and there is a fix for existing episodes, but it's not necessarily for the faint of heart, as it involves hex-editing your video files.

The key is the cnID atom. This is an atom that seems to reflect an internal unique catalogue ID from the iTunes Store for each item in your iTunes library. For HD/SD episodes that are properly "matched" in my library, the cnID was the same for both versions. For the unmatched stuff, however, the cnID is different between the two.

AtomicParsley and related tools can display this atom, but I don't think they can actually change it, however you can open both episodes in a hex editor, search for "cnID" and ensure that the hex values about 4 bytes after the word "data" match between both episodes. In all of those that I've looked at thus far, this seems to be as simple as changing the first value of "51" in the HD episode to "11" to match the SD version (of course, I suppose you could do it the other way, but since the HD cnID number is so far out of the usual range I'm thinking that the SD is the more "correct" one).

Once you've made this change, you may need to reimport the specific episode into iTunes so that this value is re-read, but after that things should work as expected.

I'm speculating that perhaps this cnID change was to fix the other issue that users were having with HD/SD episodes overwriting each other during download.... It makes some sense that if they had the same cnID that iTunes might sometimes get confused if they're both arriving at the same time.

[CBlakeston]

If this rumour is true it could explain why we haven't seen an update yet. Whatever is coming, I hope I can update my current ATV to new software come Tuesday :)

http://www.tuaw.com/2008/09/27/rumor-is-the-apple-tv-being-replaced/

[jhollington]

While an Apple TV update would certainly be welcome, I doubt very much that it's going to fix this particular issue, particularly since the problem exists in iTunes itself and not merely on the Apple TV. The problem is merely most visible on the Apple TV as it's the only device that supports both formats (and hence shows duplicates). As mentioned by others above, the iPod will simply generate an error when trying to sync the HD version of an episode, although if you have previously checked the box to not show these warnings when syncing your iPod, you simply won't see that particular issue come up.

In every case I've observed thus far, the cnID for the HD episode is exactly 0x40000000 higher than the cnID for the SD episode (ie, the first hex octet is 51 vs 11). I suspect that iTunes 8.0.1 will solve this issue simply by "matching up" these episodes based on that difference.

[gtrogue]

I've tracked down what seems to be the cause of the problem, and there is a fix for existing episodes, but it's not necessarily for the faint of heart, as it involves hex-editing your video files.

The key is the cnID atom. This is an atom that seems to reflect an internal unique catalogue ID from the iTunes Store for each item in your iTunes library. For HD/SD episodes that are properly "matched" in my library, the cnID was the same for both versions. For the unmatched stuff, however, the cnID is different between the two.

AtomicParsley and related tools can display this atom, but I don't think they can actually change it, however you can open both episodes in a hex editor, search for "cnID" and ensure that the hex values about 4 bytes after the word "data" match between both episodes. In all of those that I've looked at thus far, this seems to be as simple as changing the first value of "51" in the HD episode to "11" to match the SD version (of course, I suppose you could do it the other way, but since the HD cnID number is so far out of the usual range I'm thinking that the SD is the more "correct" one).

Once you've made this change, you may need to reimport the specific episode into iTunes so that this value is re-read, but after that things should work as expected.

I'm speculating that perhaps this cnID change was to fix the other issue that users were having with HD/SD episodes overwriting each other during download.... It makes some sense that if they had the same cnID that iTunes might sometimes get confused if they're both arriving at the same time.

This works great. Just ran into one issue.
Season 4 of LOST doesn't have the "51" or "11" in the files like all the other files I looked at. Pretty weird because every other file is exactly the same but all the episodes of LOST are different.

[sonictonic]

Hoo-rah new iTunes version is out! I can't download it yet, I will tonight, but if anyone beats me to it please post here if our dupe issue is fixed!

*crosses fingers*

[NightStorm]

In iTunes, HD and SD versions are now combined into one entry. The HD plays by default unless you set the option in preferences to play the SD one. There is also a versions menu if you right click on the library entry.

The AppleTV now only shows the HD version along with a shiny HD logo on the Show Title and all HD episodes.

[sonictonic]

Yup looks like it's fixed! THANK YOU APPLE! :)

Having the episodes in iTunes listed as one SD/HD episode instead of 1 SD + 1 HD makes things much more neat and accurate as far as episode count too. This is really nice.

[jhollington]

This works great. Just ran into one issue.
Season 4 of LOST doesn't have the "51" or "11" in the files like all the other files I looked at. Pretty weird because every other file is exactly the same but all the episodes of LOST are different.
Not necessarily an issue any more, since iTunes 8.0.1 has solved the problem, but with LOST, the cnIDs are simply lower numbers, so you're looking at 10 for the SD version and 50 for the HD version, as opposed to 11 and 51.

Of course, if you downloaded Lost Season 4 the first week, they'll all be 10, since Apple started out with the HD/SD versions having matching cnIDs. My guess is that they decided to change them after receiving numerous reports of the SD/HD versions clobbering each other during download.

Regardless, however, the new system has them offset by exactly 0x40000000... iTunes 8.0.1 simply recognizes this offset as a legitimate version pairing and now links these episodes up properly.

[me3head]

So if I change the cnID to 0x40000000 higher on shows I ripped myself, will iTunes combine them?

[jhollington]

Theoretically, that would work. Howver, since the cnID doesn't exist on content not purchased from the iTunes Store, you would need to add it yourself to both tracks. Further, you would need to ensure that the numbers are unique from any other tracks in your library.

The other question would be whether iTunes needs any other tags that are typical of purchased tracks before it reads the cnID tag.

Of course, if you're ripping shows yourself, there's generally little to no point in ripping in HD resolutions unless you're actually getting the content from a Blu-Ray or HD-DVD or recording it from an HD broadcast signal. If you're ripping normal DVDs, you will gain nothing by ripping them in an HD format.

There might be some advantage to trying to combine iPod and SD Apple TV format episodes in this manner (so you could have a higher bit-rate Apple TV version), but in my own experience there's rarely a point in bothering with this for most DVDs, particularly now that the iPod supports anamorphic encoding so that you can get a proper 854x480 presentation.

[me3head]

Mainly, I wanted to combine a 854x480 Main Profile h.264 rip (as the "HD" one) with a iPhone screen-width limited Baseline Profile h.264 rip, so that I'd get the nice quality one in iTunes and auto-sync the iPod version as needed.

I also have a hdhomerun, so HD recordings are also on the table.

I just downloaded the free HD Ugly Betty short that has both HD and SD versions, and ran both versions through AtomicParsley. Turns out that the cnID is identical, but the info on which is HD and which is SD appears to be contained in an XML plist embedded in to the itunMOVI atom.

Unfortunately, AtomicParsely doesn't seem to be able to edit this Atom. I wonder if I were able to just use the exact data from that Atom in my files before iTunes import if I'd be successful. Anyone know of any tools that allow you to edit any arbitrary atom of an mpeg4 container?

Alternatively, could I use the existing container from the free show, replace the video and audio with mine, and then just change the metadata?

[jhollington]

Mainly, I wanted to combine a 854x480 Main Profile h.264 rip (as the "HD" one) with a iPhone screen-width limited Baseline Profile h.264 rip, so that I'd get the nice quality one in iTunes and auto-sync the iPod version as needed.
Depending on what you're encoding, that would make some sense. Modern content that's in a 16:9 aspect ratio can benefit from this somewhat, but since you can go anamorphic on the baseline profile (albeit only from a 640x480 source rather than a 720x480), you're not really losing all that much. The vertical resolution is the important point in that regard.

In fact, most of the newer iTunes movie downloads and almost all of the rentals using a 640x480 -> 854x480 anamorphic presentation, and I've found that the quality, even on my 62" DLP HDTV is not really that noticeable from an upconverted DVD unless I'm really looking for it.

I also have a hdhomerun, so HD recordings are also on the table.
HD recordings, or course, will definitely benefit from the 720p resolution available on the Apple TV.

I just downloaded the free HD Ugly Betty short that has both HD and SD versions, and ran both versions through AtomicParsley. Turns out that the cnID is identical, but the info on which is HD and which is SD appears to be contained in an XML plist embedded in to the itunMOVI atom.
Yes, iTunes has changed over their approach a couple of times on this one.... Initially, the cnIDs were the same, and then they started adding the 0x4000000 increment to them, likely to eliminate the problem with HD and SD downloads stomping on each other. Now they appear to have gone back to the original method, for at least some of their content, likely due to iTunes 8.0.1 fixing the original SD/HD conflict bug that caused the whole problem in the first place.

Regardless, iTunes 8.0.1 does recognize the 0x4000000 cnID offset as representing an HD/SD pair in purchased content, as all of the legacy content in my library from the late iTunes 8.0 period is still tagged in this manner with nothing else in the itunMOVI atom, and they retain their proper HD/SD pairing even when imported into a new library.

Unfortunately, AtomicParsely doesn't seem to be able to edit this Atom. I wonder if I were able to just use the exact data from that Atom in my files before iTunes import if I'd be successful. Anyone know of any tools that allow you to edit any arbitrary atom of an mpeg4 container?
This is just off the top of my head, but I think you can edit the iTunMOVI atom via the Atomicparsley reverseDNS switches.... This is the same way that cast and crew information gets embedded into iTunes movies, and there was a bunch of hacking around with that in AP before tools like Lostify and MetaX started providing direct GUI access to those "fields" (which are really just part of the same PLIST).

[angle]

Has there been any more progress on this front? I have a TON of HD quality rips that I've also re-encoded to smaller resolutions so they'll work on my iPhone. I'd love to be able to have them combined into one entry in iTunes though so as to not mess up my library. Would love to hear if anybody has tried and had success with this.

[angle]

Nobody has had anymore luck on this front? I've been toying around with it, but haven't been able to get iTunes to see the two files as a single file. Anyone else been trying that might be able to combine brainpower with me to figure this out?

[mallbritton]

I'd like to know how to accomplish this as well.

Regards,
Michael

[angle]

I've had luck with adding a CNID to a film, however I'm not sure if I'm doing it in such a manner that I'm damaging part of the file or not (adding it through a hex editor). I was unable to get iTunes to see the files as one though, so I'm not sure if I'm missing a step. Anybody who has an HD TV show / movie care to open it up in a hex editor and take screen shots of the associated parts where CNID is mentioned? See if maybe we can't figure out what needs to be done?

[angle]

Still hoping to get this to work. Anyone....anybody at all, have ANY luck with taking 3rd party content and getting it to sync up properly in iTunes? Would love to put together some type of tutorial so that everyone can take advantage of this!

[me3head]

Still hoping to get this to work. Anyone....anybody at all, have ANY luck with taking 3rd party content and getting it to sync up properly in iTunes? Would love to put together some type of tutorial so that everyone can take advantage of this!


I've been meaning to take a look when I get a chance... I dont think its the CNID that makes iTunes see them as one title. I beleive the key is in modifying the plist contained in the iTunMOVI atom.

[angle]

That's something new for me to look into! Thanks for the suggestion!

[angle]

Okay, I've tried doing some research but have been having some difficulty locating how to either A) view the iTUNmovi atom, or B) how I would edit it. Has ANYBODY made any other progress on this? I've been scouring the net trying to find people who might have had better luck, but it's looking glim.

If anyone can point me in the right direction, I'm more than willing to put the time into testing / trying stuff out just until we get it right!

[NightStorm]

The MP4v2 project (http://code.google.com/p/mp4v2/) is working on this as well... so far they have gotten the HD logo visible in iTunes (hdvd tag). I think they were the first to do so if you think you have something to offer, I'd get in touch with one of the administrators (a few work on Handbrake as well).

[jendrik]

Hey guys,

I just wanted to let you know that I think I have figured out which tags are needed in order to have SD and HD shows displayed as one. I am implementing this at the moment for my own app. (see http://iflicksapp.com or http://groups.google.de/group/iflicks?hl=en)

It all comes down to two tags:

1. both videos need the same cnID
2. one of the videos needs the hdvd tag set to 1 (0x1)

When the tags are set the videos have to be reimported. Just telling iTunes to reload the metadata from file is not enough.

Can anyone verify this?

Cheers,
Jendrik

[NightStorm]

Anyone know the proper technique to hex edit in a unique cnID number to a file?

[angle]

Proper? I'm not sure of....

HOWEVER...

I was able to create a CNID of my own in a file after some fudging around. I've included a link to a screen shot showing how I managed to add the CNID in. Basically, I tagged enough of the file to find where the tag information would be located in a hex editor. Then I just copy / pasted the CNID info from another file that iTunes had already tagged in, changed the number component, and called it a day.

I'm using a Windows box, so if anyone knows of an easier way to edit this please let me know! Also, I haven't mucked around with the MP4v2 stuff yet, so if anyone has had success, please leave a post! Thanks!

http://i299.photobucket.com/albums/mm295/deviantdope/CNID.jpg

[NightStorm]

I talked with one of the developers working on the mp4v2 project earlier today. He is working on adding the ability to add/change cnID info; if you look at their revision notes you can see he's been slowly adding all the pieces necessary. The SVN code can already set the hdvd tag to 1, which adds the HD logo in iTunes.

Once it is in place, we'll be able to run the files through mp4v2 and sync the cnIDs and flag the hdvd tag on the HD file and call it a day. In the meantime, I've started encoding both a HD and Universal (SD) version from my HD-DVD/Blurays in anticipation. :)

[angle]

So have you tested and confirmed that setting the same CNID and the HDVD tag to 1 allows the movies to be synced / seen as one entry in iTunes? This would be VERY good news if that's the case!

Also, is MP4v2 going to be Windows compatiable? I know that if you build it from source it should work, but I haven't taken the time to download Visual Studio Express to try that...

[angle]

I haven't tested the changing of the HDVD tag (haven't figured that one out yet) but I CAN confirm that simply having two files, on HD, one SD, same name (except for one being marked with a "1" at the end) in the same folder with the same CNID will NOT create a single entry in iTunes.

[NightStorm]

I can confirm that identical cnID + one file with hdvd == 1 does not group in iTunes. Back to the drawing board... :(

[angle]

Well that's not good. Does anybody actually have an HD / SD file that they can examine the tags of and report back? I'd gladly buy one if someone can tell me how / what I need to examine the file.

[NightStorm]

Well that's not good. Does anybody actually have an HD / SD file that they can examine the tags of and report back? I'd gladly buy one if someone can tell me how / what I need to examine the file.

This is currently in process. :)

[angle]

Almost 24 hours. Any differences in the comparison between files? Just curious what's going on with this movement. I feel like I should help, but I don't know how!

[angle]

Currently on iTunes there are some HD episodes of some new Lost material that can be downloaded for free. I believe this would be an excellent source for us to examine and can be used to compare HD / SD hex code / tags.

[me3head]

According to Jendrik's website for iFlicks, his app is supporting this now, so it's possible. I wonder if he would share the method he used and the apps he used for doing it (atomic parsley?)

[NightStorm]

OK, it's the Christmas season, so I'll give a hint.

- cnID must be equal in both files
- hdvd tag must be set to 1 in the HD file
- stik must not be 0 (Movies), set it to 10 (x0A) (TV Shows)

These files will import into iTunes combined, but listed in TV Shows. Use the iTunes "File Info" option to change it to Movies. We haven't been able to figure out why iTunes will not combine the titles if stik is set to 0 (Movies). I now have a couple movies with combined entries in my iTunes movie library.

At this point, I'm not willing to give exact specifics on how to do this; either figure out how to make these changes yourself, or wait for a program to be released (it is currently being worked on).

Special thanks to those who figured it out... :)

[angle]

Has anyone determined the proper (aka: safe) way to add the tags that ripped titles won't have to their files? I was able to add a cnID tag to a copy of Wall E that I ripped, but it made the file unusable at that point.

[mallbritton]

I'm back from my holiday travel and I'm willing and eager to help test any software solutions that might be working to solve this problem.

Regards,
Michael

[jendrik]

According to Jendrik's website for iFlicks, his app is supporting this now, so it's possible. I wonder if he would share the method he used and the apps he used for doing it (atomic parsley?)

I am using the QuickTime Metadata API to do this. Have a look at http://developer.apple.com/DOCUMENTATION/QuickTime/Conceptual/QT7UpdateGuide/QT7UpdateGuide.pdf on how to get started.

[angle]

So I'm guessing that no (efficient) way has been found to do this on a Windows machine? I've managed to botch a number of copies of movie files that I've tried to hex edit (god bless Copy / Paste).

Any more info would be appreciated!

[downinitjr]

I've waisted quite a few hours on this myself. Furthest I got was to compile mp4v2, and succesfully add the hd tag to a file. However this currently appears to corrupt pretty much all of the other meta-data in the file, making it unreadable and untaggable with meta-x. I've also messed around with hex-editors etc., but nothing useful has come out of that on this end.

[NightStorm]

I've waisted quite a few hours on this myself. Furthest I got was to compile mp4v2, and succesfully add the hd tag to a file. However this currently appears to corrupt pretty much all of the other meta-data in the file, making it unreadable and untaggable with meta-x. I've also messed around with hex-editors etc., but nothing useful has come out of that on this end.

This issue has been largely resolved in the newer SVN code from the mp4v2 project; however, the ability to add cnIDs has not been checked in (yet) while more testing is being done.

The current workflow would be mp4tags -> metax -> itunes.

[downinitjr]

Thanks for the heads up there with the svn code, it does seem to be working correctly now for adding the hd tag. I was able to add the Hdvd tag to a file already tagged with meta-x, and it didn't corrupt the other tags, and I was able to go back and still edit the previous tags with meta-x.

After thinking about this whole deal though I'm a little disappointed. It seems under the current scheme you can only combine two files if one is HD or at least tagged as "HD", and whatever else is needed (cnID etc). I do currently however have quite a few movies that are not HD, but I still keep a separate ipod version of them to keep the resolution as high as possible for tv, and file size as small as possible for ipod. It seems even if this is eventually implemented, it still won't really be a solution for these files.

I guess I'll keep my fingers crossed that enough people give feedback, and eventually we are allowed to combine files of are own choosing.

[realityking]

I just tag my full dvd res files also as hd, while tht doesn't help with separating sd from hd content at least I can combine the entries for the AppleTV and the iPod version

[downinitjr]

I just tag my full dvd res files also as hd, while tht doesn't help with separating sd from hd content at least I can combine the entries for the AppleTV and the iPod version

So are you saying that you have successfully combined your own content into a single file listing? If so how did you accomplish this?

[realityking]

So are you saying that you have successfully combined your own content into a single file listing? If so how did you accomplish this?

I'm one of the mp4v2 devs and it was me who added the support for the HD Tag and a lot more metadata improvements.
Together with NightStorm and refulgentis I was also able to add the functionality needed to combine HD and SD encodes, however there's currently no stable release of mp4v2 nor binaries that are available to the public (I posted some here after the HD Tag was added, afaik those are the last binaries available).

[downinitjr]

I'm one of the mp4v2 devs and it was me who added the support for the HD Tag and a lot more metadata improvements.
Together with NightStorm and refulgentis I was also able to add the functionality needed to combine HD and SD encodes, however there's currently no stable release of mp4v2 nor binaries that are available to the public (I posted some here after the HD Tag was added, afaik those are the last binaries available).
Gotcha, keep up the good work! I'll keep watching this and the other various forums around for developments.

[angle]

Been over two months and this thread has been quiet. Any word on a Windows compatible way to do this efficiently? Any news on how to do this efficiently at all?

[NightStorm]

Subler is a GUI that can set these tags (using the mp4v2 library). However, its Mac only.

http://code.google.com/p/subler/

[downinitjr]

Anyone have a chance to see how the new HD•SD movies from the itunes store are "put together". Seeing as we have to "trick" itunes right now by changing the type temporarily to a tv show before it will combine user encoded movies. From what I've read it seems the HD movie downloads from the itunes store are both a HD and SD version combined, but iTunes won't currently combine user files with their type set to movie. So how is apple doing it for movies?

[NightStorm]

Anyone have a chance to see how the new HD•SD movies from the itunes store are "put together". Seeing as we have to "trick" itunes right now by changing the type temporarily to a tv show before it will combine user encoded movies. From what I've read it seems the HD movie downloads from the itunes store are both a HD and SD version combined, but iTunes won't currently combine user files with their type set to movie. So how is apple doing it for movies?

Set stik == 9 (or movie kind == Short Movie). This is how iTunes is doing it now. If you notice, in 8.1 they have removed the "Short Movie" option from the media type drop down box. As long as you've followed the other steps (identical cnID, HD encode with hdvd atom == 1), it will show combined in Movies without the "trick".

Thanks to refulgentis for purchasing a movie from the iTunes Store tonight and reporting this finding in the mp4v2 IRC channel.

[downinitjr]

Set stik == 9 (or movie kind == Short Movie). This is how iTunes is doing it now. If you notice, in 8.1 they have removed the "Short Movie" option from the media type drop down box. As long as you've followed the other steps (identical cnID, HD encode with hdvd atom == 1), it will show combined in Movies without the "trick".

Thanks to refulgentis for purchasing a movie from the iTunes Store tonight and reporting this finding in the mp4v2 IRC channel.

That's exactly what I was looking for, Thanks! as well.

[angle]

Is anyone aware of how to perform this in Windows? Or better yet, has a Windows GUI been created? Any help would be greatly appreciated!

[RangerXML]

Is anyone aware of how to perform this in Windows? Or better yet, has a Windows GUI been created? Any help would be greatly appreciated!

Ditto

[angle]

Turns out this CAN be done in windows using the version of MetaX by Dan Hinsley. You have to assign your own Catalog ID, and tag the files as a "Short Film." One copy has to be marked as "HD" (also performed in MetaX) and the other just a regular version. From there, just drag / drop into iTunes and it works! :-D Reply back if you have problems and I'll try to help.

[cowm007]

I finally achieved consistent results using Subler (http://code.google.com/p/subler/). The only thing that bothered me was having to come up with my own cnIDs. So I'm just using the cnID iTunes uses for the episode (assuming it exists) by pulling the info from the iTunes store using the API.

Details for that can be found here http://www.apple.com/itunesaffiliates/API/AffiliatesSearch2.1.pdf

An example search for LOST season 4 episodes would be http://ax.phobos.apple.com.edgesuite.net/WebObjects/MZStoreServices.woa/wa/wsSearch?term=lost+season+4&media=tvShow

This returns all episodes in JSON format and the trackId value is the cnID iTunes uses. This could probably be automated somehow, but it works well enough for me at this point.

[djpaetkau]

Just wanted to say thanks to NightStorm for your recommendation of Subler, which perfectly created metadata so that files, when imported into iTunes, were automatically tagged as a combo HD/SD version.

Also big thanks to cowm007 for the iTunes API for pulling out the proper info. Being OCD about even my album and movie artwork being the official versions submitted to iTunes, getting the proper ID numbers is going to make my iTunes library cleaner and more official than ever before. Any chance you know how to access the iTunes movie cover art too?

[LostOnTheLine]

I know this post is old, but since I spent a lot of time looking for this information, I'll put it here for anyone searching for it in the future.

To build MP4v2 on a Mac, follow this tutorial Cheap Geek creating SD-HD files (http://www.thecheapgeek.org/creating-dual-resolution-video-files-in-itune)
On Windows it's a bit more complicated, but here's the basics:

Decompress the archive (I use 7-Zip which is free) you may need to do it twice, once for the .bz2 & once for the .tar.
In the folder will be a folder labeled 'vstudio9.0', goto that folder.
Open the 'MP4v2.sln' solution file in Visual C++ 2008 or 2010 (The free "Express" edition is available @ MS Visual Studio Express (http://www.microsoft.com/express/Downloads/).) 2010 will require a conversion wizard but it doesn't require you to do anything except click [Finish].

Either program will give you an error about not being able to read some of the properties of the solution & 2008 will give you an error about not supporting solutions folders & 'util' being unavailable, I don't know why, but if you ignore those warnings it should work fine, I haven't run into any problems yet.

Goto [Build Solution] in either the [Debug] Menu or the menu depending on the version you're using.
In the 'vstudio9.0' folder will be a folder labeled [B]'Debug' or 'Debug Static' which will contain your command line executables, along with some object files, debug files, & a DLL.
Either run the command line from that folder or rename & move it to system folder so you can run it from any directory.

It's not the cleanest way, but it works. (If you need to see the details goto MS MSDN Forums (http://social.msdn.microsoft.com/Forums/en-US/vcgeneral/thread/ffa87c6b-2c37-4d7f-bb98-c8babdccde2e))

MetaX now has the ability to write both the 'HD' tag & the 'cnID' atom. The later I only found out after writing it successfully with MP4v2 & going to check to make sure the rest of the metadata was still there. It's on the [Advanced] tab under [Catalog ID]. I also haven't had any trouble with loosing metadata after writing it in MP4v2 even though some have ridiculously long; lyrics, long description & cast info.
To import a SD-HD, single entry, item, you need to drag both files to iTunes together, one must have the HD tag, the other not, & they must have identical cnID #s. I haven't tried Movies but it works for both TV Shows & Music Videos.
As for just making up a cnID, if it's something avaliable on iTunes it's simple to find: find it in the store, Right-click or apple-click on the icon & select 'Copy Link' then paste it anywhere, & the 9 digit number after "id" at the end of the URL is the cnID {e.g. http://itunes.apple.com/us/video/do-you-wanna-date-my-avatar/id326378551}.
If you are making you're own, to prevent it from conflicting with a future iTunes purchase, make it less than 7 characters (you can use single digits like "2") OR, to prevent yourself from repeating & prevent conflicts with iTunes purchases, use the Date/time {e.g. right now it's 8/27/2010 @ 11:12pm so I'd use 201008272312 (Year2010-Month08-Day27-Hour(out of 24)23-minute12) which can also help you keep track if you for some reason need to know when you added a file or what order you added them or whatnot.
If you have a show with an 'SD-HD' icon & you want to delete one, but not both of them, for now the best I've got for you is to:

Delete the file of the one you don't want {e.g (Do You Wanna Date My) Avatar [feat. (HD).m4v}
Set the 'version' to the one you deleted (HD in the exampe) & itunes shows an ! next to it.
Delete the entry, it shouldn't try to delete the file, but if it does choose 'Keep File'.
Now drag the file back in.

You may want to note your rating first as Rating, Play Count & Watched/Unwatched info will be reset.
I would suggest Removing from iTunes, Editing in MetaX, & re-adding, in that order but I don't have a problem when I edit while it's still in the iTunes library, then delete & re-add. (For adding a single SD-HD entry to 2 files)

As a side note, the 'HD' file doesn't have to be High-Definition you could have a 120x90 file tagged as 'HD' & a 1280x720 file tagged as 'SD' but if your default is to play HD then, for that file, you'll see the tiny one. & if the timing is more than a second difference iTunes may add it as one, or may not, I'm not sure as to the whys just yet.
.
So, I jut realized I missed one question. Album Artwork

There have been a few ways in the past to get the actual, official, artwork directly from apple's server, but, the problem is, as soon as apple finds out about one of these, they quickly shut it down. Which is fine with me because I decided a while ago that I don't particularly like the square movie artwork anyways. BUT, if you do, there's always tagChimp with MetaX.

Signup for a tagChimp account @ www.tagchimp.com.
Goto your preferences.
One of the first sections is Cover Art.
Select 'iTunes Square Art' from the drop list.
Now, in MetaX, click on the Green notepad with a white & Yellow Cupcake(?) icon in the upper right corner (Preferences).
On the 'Data' tab you're gonna hafta select 'Share data with tagChimp' & put in your tagChimp ID.
Now the search results will have the stupid square art, many actually from iTunes.
If you still aren't finding what you want, there's a MacRumors forum for Unofficial Artwork (http://forums.macrumors.com/showthread.php?t=448145), as well as GetVideoArtwork.com (http://getvideoartwork.com/index.php) (There's already a link to it on the forums so I think apple either knows & can't do anything about it, or doesn't care since it's not coming from their servers.) Sometimes the art varies slightly, but that could be from the Store page having a different version than the actual file, or an updated version on the Store & the older one on the site.

If you don't want the Square Art, that's the most time consuming part of tagging, especially if you're, like me, picky. Try Amazon, IMDB, DVD Cover Art (http://www.dvdtown.com/moviedatabase/coverart), TV.com (http://www.tv.com/), The TV Database (http://thetvdb.com/), The show's official & unofficial Pages, and, this one works much better the more recent & more popular a title is: Google & Yahoo search. I use both because sometimes what I can't find on one I''ll find on the other. You can also, as a last resort, look for a torrent, many of them have the artwork, & you can choose to download just that, but if the FCC comes a knocking it's not my fault, show them your physical copy & live without your computer for a few months while they search it for anything that they could spin as illegal. Also, don't even search for a torrent without up-to-date Virus software. If you do, try ThePirateBay (http://thepiratebay.org/) as they're supposed to be one of the safer ones.

[darpar]

Sorry for reviving this old thread but I really do need help figuring something out, too as it is driving me crazy.

I have two versions of the same video (made with Handbrake), one HD, one SD (well, a little lower than that - 480x272).

Using Subler, I've then assigned them the same contentID and marked the HD video as such. When I import the files into iTunes, they do show up as just one title with the nice "HD·SD" tag. However, and this is a big HOWEVER, when I play the file, it will play the SD version only. :(

I've tried it on my 1st gen :apple:TV and it's the same there. I really can't figure this out - the files will play just fine if I import them individually, both on my computer and on the :apple:TV.

Any thoughts would be great! Thanks!

[bucksaddle]

iTunes Preferences -> Playback Tab -> Play videos using standard definition version.
This checked at all? If so, uncheck

[darpar]

Yes, thanks, it is unchecked, sorry, should have mentioned that!

I just noticed that I didn't check "Use iPod/iTunes friendly (.m4v) file extension for MP4" in the Handbrake preferences. My HD videos were .m4v, my SD videos .mp4. Still doesn't explain why my iTunes would prefer MP4 over m4v but I'm at the end of my rope here so I'll just re-convert the SD files and see what happens then.

[bucksaddle]

mp4 and m4v are exactly the same things, you can just rename the mp4 one.

Apple uses m4v to enable a few extra things like chapter markers, etc

[LostOnTheLine]

Try this:

Goto your Video, let's call it Bob, and right-click (ctrl-click) on Bob in iTunes.
In Bob's Drop Menu goto 'Version' and select the version you want to play whenever you click on Bob.
If you select 'HD' but still see the SD version, try setting Bob to 'SD' & see if Bob becomes a real boy.

What probably happened was that you set the HD tag on the wrong file. Easy enough to fix:
MetaX it & if you don't have the 'auto-submit to library' option on (I leave it off to preserve my play count), after you fix it, goto Get Info, switch to the other version, and Get Info again, this will update the files' tags in the iTunes Database.



M4V used to be the extension used for the lower resolution (iPod) version. There are no extra things in the M4V that aren't in the MP4. They both use the same tag system & I believe they always have.
I have a number of Dual-Resolution files (MP4=Hi, M4V=Low) and they both have chapter marks.
It used to be used for sorting purposes but now you just set Media Type, 'Movie' or 'TV Show', in the Tag settings.

[LostOnTheLine]

@darpar: Did you get your problem sorted?

I have an issue that some of the commentators on this thread might be able to help with:
I have a TV series that I've encoded with Handbrake & Tagged with MetaX. It has the cnIDs & HD Tags set properly.
In iTunes, when I choose the 'SD' version it plays the smaller one (480x272) & when I choose 'Default' or 'HD' it plays the larger one (576x320). So I know the HD tags are on the right files.
The problem is that when it syncs to my iPod it syncs the one set to play by default (HD) in iTunes.
Is there a way to make my iPod sync the SD tagged file even if it can play the HD one?

[prumor24]

I know this thread is old, but after I searched for a long time, I finally found this video which allowed me to created HD-SD movies in iTunes: http://www.youtube.com/watch?v=93bG1hUCbvI&feature=related


It worked perfectly and didn't take much time, so I recommend that anyone trying to combine HD and SD videos use this tutorial.

[mallbritton]

Thanks for the link to that video. I helped me figure out why a couple of iTunes Store purchases were not showing up as HD-SD in my iTunes library, but as individual entries.

Regards,
Michael

[StevoDevo]

Thanks for everyone's input. I now have HD SD tagged files in iTunes and I'm a happy camper. A question I have now is which version will get synced to my iPad? It looks like it's the HD version and if so is there anyway to control that or select which version goes to the iPad?

[angle]

When your iPad is plugged in, go to the "Summary" tab at the top, then scroll to the bottom of the window. You will see various tick boxes, one of which states something along the lines of "prefer standard definition video."

[StevoDevo]

When your iPad is plugged in, go to the "Summary" tab at the top, then scroll to the bottom of the window. You will see various tick boxes, one of which states something along the lines of "prefer standard definition video."

Perfect - thanks!