Import @ 1920x1080 or 960x540 (or why the Mac sucks for HD compared to PC)

[ArthurDaley]

Blindly like a sheep, I though Macs have a better name for video editing etc. (i.e. the creative stuff), so I bought a Mac Book Pro.

On the PC (XP) I'd used Sony Vegas or the free HDVsplit software to import my 1080i camcorder (HDV) tapes. It gave me a bit for bit import of HDV at 11 GB/hr (the output is .m2t files). Life was good. I wanted it better so got the Mac.

On switching to the Mac I found (iMovie, FCP) could handle the Sony files I'd imported on the PC; that is .m2t files without transcoding (!) to something call the Apple Intermediate Codec (AIC). My conclusion was there was no real option but to reimport everything from tape all over again (!) if I wanted my HD tapes to end up in a format that the Mac played nice with (every method to get .m2t to something the Mac liked seemed time consuming and nasty). Apple - 1

So once I'd accepted I'd need to start all over again importing from tape, I did my first import on the Mac and the output is 40 GB/hr - yes four times that of the PC! (and correct me if I am wrong but it's stored transcoded to AIC and still not a true bit by bit conversion). Apple - 2

I did a bit of background reading and Apple wants you instead of importing at 1920x1080 they want you to import at 960x540. Apple claims there is about a 4:1 ratio, file size wise between 1920x1080 compared to 960x540. So right enough I did an import at 960x540 and downscaling everything to Apple's proprietary format gave 12 GB/hr (for 960x540 video). Apple -3

Furthermore I've not had it confirmed but from iMovie08 reviews, it seems to be the case that even if you import and export at 1920x1080, iMovie will internally process it at 960x540.

A quick trawl around the web reveals quotes like "Ordinarily I'm among the strongest of Apple & Mac supporters, but this nonsense is inexcusable. And I'm not the only one who sees problems with Apple's wanderings. At this point there's no certainty about where Apple's video strategy is headed. I feel my time, money, and video is much safer with Sony Vegas as Sony's direction is crystal clear. Certainly they're not going to take their HD editor and chop it down to an SD editor as Apple has done." and "I've been upset with Apple's crppy MPEG2 support for years, and that's not going to get any better. Meanwhile, the Windows OS alternatives to this Apple mess just keep getting better. Sony Vegas has a huge-and-growing following including quite a few former Mac users. It's the new 'switch' campaign in reverse. Sony Vegas, BTW, offers no-hassles Blu-Ray burning since last year. It works directly with HDV "

So it would seem for HD, that your probably better off running a PC with Vegas. Which is a bummer since that is what I had!

My objective later on is to use Blu-ray discs for pristine 1080i distribution.

Now it COULD be the case that I'd not notice much difference between 1920x1080 or 960x540 (output display is 1080p, 42" and a few years down the line larger) assuming it's easy enough to make a blue-ray compat. disc with 960x540 res. material. Since my camcorder is 1080i and h.264 (used by BR) isn't as efficient with interlaced material and the camera really is 1440x1080 as with most 1080i camcorders, it scales to 1920x1080 on playback.

BUT it seems hard not to see it that Apple are really forcing you to near SD levels!

So two questions:

1) Can anybody refute the evidence above that Apple suck for consumer HD and I should have stuck with PC+Vegas?
2) What does the jury say, should I import at 1920x1080 or 960x540? If so doing it at 40GB/hr on the Mac sucks against 11 GB/hr on the PC.

 

[NickFalk]

Well in FCP, you can pretty much create your own specs when it comes to resolution/framerate etc. Just use the Apple 422 codec. This is way better for anything where you'd like to place titles etc. anyway as it doesn't compress colour as brutally as HDV.

HDV is really not a good format for post-work in any sense of the word which is why it is a good idea to convert it to another format.

 

[oceanzen]

I don't know why you're comparing a freebie (imovie) to a pro app (vegas) surely you should be using final cut pro if you are used to a pro video app

 

[ChemiosMurphy]

Whoa... Let's fix a few of your misconceptions first....

Why are you knocking FCP for transcoding the M2T files? You really shouldn't edit with a compressed format because it sucks up so much processing power. Hard drives are dirt cheap and RAID0 arrays are easy to make, so why not use a format with looser compression?

AIC or ProRes422 are so much better than Mpeg2 because of the loose compression. ProRes422 is the better choice because it records files with a chromasubsampling rate of 4:2:2 instead of HDV native 4:2:0. HDV is a lossy format which ditches some color information to save space, however, this makes things like keying a pain. This is why it makes you convert it, so that you CAN ACTUALLY EDIT WITH IT later.

I don't get why you keep knocking Apple's support for MPEG2 and native HDV editing. If anything the Sony is playing a cheap gimmick here by letting you edit in HDV, sure it saves space but sucks your processor power dry. HDV MPEG2 GOP uses interframe compression... In English this means that if you disturb the master frame it must work to regenerate part of the sequence.

I will in fact support you with the fact that iMovie sucks, but FCP rocks.

What does this mean "Certainly they're not going to take their HD editor and chop it down to an SD editor as Apple has done" and "At this point there's no certainty about where Apple's video strategy is headed". Look, I'm no fanboi of apple but uninformed people drive me nuts even more. No other company has rocked the video industry like Apple has. They took Avid off of their pedestal and are eating their lunch. Final Cut Pro supports nearly every HD format out there, albeit some with transcodes, but it works flawlessly overall if you know how to use it.

So thats part 1.

Part 2... Read this HD and Broadcast format guide from Apple's website. It will give you the best workflow for HDV. I can promise you that this booklet will solve your issues. And skip iMovie, and stick with FCP.

 

[ArthurDaley]

OK I will have a read of the guide.

Tomorrow I fly and I will be taking the tapes with me. Before I fly I want to ensure I have a copy by getting them onto an h/drive. Since FCP is not on the machine, it seems there is no option but to import them via iMovie? I will see about getting FCP on the machine later on.

The thing is the HD thing is further complicated, the camera (Sony HDR7) support http://en.wikipedia.org/wiki/XvYCC colour and nobody ever seems to mention that in terms of editing, preserving that colour space.

Also when you say

HDV is a lossy format which ditches some color information to save space

where does this loss occur as I thought the tape was HDV but the way you speak you mean going from tape to the computer?

 

[ArthurDaley]

I need to make a decision quick before I fly with the tapes - import via iMovie at 960x540 OR play them onto the PC into .m2t files? From what you guys are saying later on FCP will play happy with the .m2t files?

 

[LethalWolfe]

As the old saying goes, "measure twice, cut once" but apparently you didn't measure at all and are now blaming your new saw for cutting your board too short.

If Vegas meets your needs then why not stick w/it? There is no "one size fits all" program out there that is right for everyone.

AIC or ProRes422 are so much better than Mpeg2 because of the loose compression. ProRes422 is the better choice because it records files with a chromasubsampling rate of 4:2:2 instead of HDV native 4:2:0. HDV is a lossy format which ditches some color information to save space, however, this makes things like keying a pain. This is why it makes you convert it, so that you CAN ACTUALLY EDIT WITH IT later.

Or you can just change your sequence settings to render in ProRes and then you get to work w/HDV sized files buy all your transitions, effects, color correction, etc., are done in ProRes. While HDV is near the bottom rung of HD codecs to work with it's far from impossible unless you have a G4 or something. The bad thing about capturing HDV as AIC or ProRes via FW is that there is no batch capture and you lose the tape TC info so if you ever need to recapture a project you are up a creek and will have to rebuild it by hand. That and you are working w/much large file sizes that require bigger, faster drives and, depending on your needs, you may not see a sizable improvement in work flow for all the extra money spent on HDDs.

Horses for courses as they say. Sometimes working natively in HDV is the way to go and sometimes it's not.


Lethal

EDIT: No Apple product likes M2T files. You can use MPEG Stream Clip to transcode the M2T files into whatever QT codec you want (native HDV, AIC, ProRes, DVCPro HD, etc.,).

 

[ArthurDaley]

EDIT: No Apple product likes M2T files. You can use MPEG Stream Clip to transcode the M2T files into whatever QT codec you want (native HDV, AIC, ProRes, DVCPro HD, etc.,).

But then it ends up 4 times the size compared to the .mt2 on the PC.

Plan is to share using Blue Ray later and only edits are cuts that I plan. It all can't be that hard?

 

[ChemiosMurphy]

Hmm. The XvYCC color space sure is strange. I did some googling to try and understand how it fits in, but nothing popped up. I think it has something to do with the HDMI out, but don't quote me on it.

HDV is a lossy format which ditches some color information to save space

where does this loss occur as I thought the tape was HDV but the way you speak you mean going from tape to the computer?

HDV itself is a format. The Digital Video world has tons of formats that fit on mini-DV tapes. DVCAM, DV, HDV are three that come off the top of my mind. Yet all three are incompatible. A cheap mini-DV deck will only read mini-dv tapes, no DVCAM or HDV. Most DVCAm tapes will read mini-dv, and DVCAM, but no HDV. You get the idea... But HDV is its own flavor.

HDV completes with AVCHD and AVC-Intra and other formats. The big difference between HDV and AVC is that AVC employs H.264/Mpeg-4 part 10 Compression. H.264 gives much better compression and less artifacts when compared to HDV and many believe that AVCHD will be top dog someday, but that's speculation.

So in a longwinded answer HDV is a format. It is lossy from the begginning because you are cramming a full HD signal onto a mini-dv tape that has been used for years with 480i dv. All of that high-defness is getting crammed onto a 25 mb/s tape, so something has to give. That's where the MPEG2 compression comes into play. If you are filming a crazy fast action scene where tons of action is happening, you may get artifacts and stuff like that. It's lossy, meaning compressed.

The 4:2:0 chroma subsampling rate is a dealbreaker for many pro uses of HDV because it is only 8 bit colors compared to 10 a 4:2:2 for keying and stuff like that. This is where prores422 or AIC comes into play. If you shot stuff 4:2:0 and are not going to be color correcting or keying, use AIC to save file space. But if you are going to be editing vertically, capture it pro res.

I apologize for the long winded answer. The technology behind the art is so complex and one goof can cost you the project.


Here's an imovie/FCP workflow i found online for you....

first...Here's the workflow:

1. Capture a whole HDV (1080i or 720p) tape in iMovie.

Features scene detection, capture, demux and auto encode to the Apple Intermediate Codec. Audio sync is perfect because it isn't using the buggy MPEG2 QuickTime Component.

2. Show package content and move the media to a dedicated FCP folder

CTRL-Click on the iMovie project file and all your captured clips will reside in the media folder in the Apple Intermediate Codec (AIC).

3. Import all the mov (Apple Intermediate Codec) into FCP

You can simply import the AIC movs in FCP HD. They will play in realtime in full resolution (720p or 1080i). You will get a warning that the clips aren't optimized for FCP HD, ignore it for now.

4. Setup a sequence using the appropriate Apple Intermediate Codec format (1080i or 720p)

It is important to setup a proper sequence in FCP HD that matches the HDV specs. Clicking on 'Advance' for the compressor will allow you to set the codec to 720p or 1080i or other (more coming soon...)

5. Edit in realtime in FCP full res HD

At this point, you can edit your clips in full res and realtime. Even FX are realtime! It is a bit buggy but if you preferred the proxy approach you could always use media manager to make a DV or DVCPRO HD version of your project for true realtime.

6. Output timeline to Apple Intermediate Codec

Finsihed editing in realtime... simply render your timeline to the AIC in the proper resolution.

7. Import the final render in iMovie

Import that AIC file into iMovie. As long as it is the AIC, it'll import fine.

8. Output the imported final render from iMovie to the HDV camera (1080i or 720p)

Send it back to the camera (Sony or JVC) in full HDV resolution.


second.... iMovie takes the 1440x1080 HDV and transcodes to a 1440x1080 AIC QT file. Then it sets a flag in the file to playback at 1920x1080. QT Player and FCP read this flag correctly.

When you place this clip in a 1920x1080 timeline, it's the QT library unsqueezing the clip before FCP can do anything to it. This causes problems in FCP (and also in After Effects, as I found out). I was getting ghost chroma lines every few frames, and other artifacts.

So here's how you fix it: Open your iMovie AIC clips in QT Player Pro, Apple-J to get movie track properties. Select Video Track on the left, and then Size on the right pulldown. Resize the clip from 1920x1080 to 1440x1080. Save, then close.

Bring the clip back into FCP...now the clip will be un-anamorphasized. Go into the clips properties and tell FCP it's an anamorphic clip. Then use a Fredric's 1440 AIC sequence settings. Sucess!

I stole all this workflow from here

 

[LethalWolfe]

But then it ends up 4 times the size compared to the .mt2 on the PC.

No, it doesn't. Transcode to the appropriate Apple native HDV codec (in this case 1080i60) and the file sizes will remain around 11-12gigs.


Lethal

 

[LethalWolfe]

So in a longwinded answer HDV is a format. It is lossy from the begginning because you are cramming a full HD signal onto a mini-dv tape that has been used for years with 480i dv. All of that high-defness is getting crammed onto a 25 mb/s tape, so something has to give. That's where the MPEG2 compression comes into play. If you are filming a crazy fast action scene where tons of action is happening, you may get artifacts and stuff like that. It's lossy, meaning compressed.

Everything is lossy. Everything is compressed. The question is by how much. Sony Z1U cameras were used in certain POV shots during the beach attack scene in "Flags of Our Fathers" if the image created wasn't up to snuff I'm sure they would've gone a different route. HDV is not the devil.

The 4:2:0 chroma subsampling rate is a dealbreaker for many pro uses of HDV because it is only 8 bit colors compared to 10 a 4:2:2 for keying and stuff like that. This is where prores422 or AIC comes into play. If you shot stuff 4:2:0 and are not going to be color correcting or keying, use AIC to save file space. But if you are going to be editing vertically, capture it pro res.

8-bit and 10-bit refer to color bit depth, while 4:2:0, 4:1:1, 4:2:2 etc., refer to how the color information is sampled. Two different things. ProRes comes in 8-bit and 10-bit flavors and I think AIC is only 8-bit.

Here's an imovie/FCP workflow i found online for you....
.
.
.
I stole all this workflow from here

OMG. You stole an ancient workflow from January of 2005 designed before Apple implemented native HDV editing in FCP.

If the OP has FCP he can either select the HDV 1080i60 Easy Setup or select the AIC 1080i60 Easy Setup and be good to go. Or he can keep is current M2T files, transcode them into the Apple HDV 1080i60 codec using MPEG Stream Clip, and import them into FCP project created w/the HDV 1080i60 Easy Setup.


Lethal

EDIT: Sorry for the short response, but I wanted to post before the OP came in hear and had a heart attack after seeing the 3yr old solution you posted.

 

[ChemiosMurphy]

Sorry for messing up the color depth thing and being to verbose for the compression. The OP said he didn't have FCP and the imovie workaround would at least let him capture a difference way because he said he had problems before, this was just a different way of doing it. Sorry to offend.

And lethal, i didn't see your 1:19 pm response because it took me forever to post mine since i'm at work, I wouldn't have posted my garbage if i would have saw yours

 

[LethalWolfe]

Sorry for messing up the color depth thing and being to verbose for the compression. The OP said he didn't have FCP and the imovie workaround would at least let him capture a difference way because he said he had problems before, this was just a different way of doing it. Sorry to offend.

No need to apologize. I'm multitasking as well and missed that the OP didn't have FCP at all (I thought he just didn't have access to it right now). So, yeah, since ArthurDaley only has iMovie then he can only work w/HDV by transcoding it into AIC. So, the OP should probably stick w/Vegas and down the road see if it's worth the money to invest in FCP. Going from Vegas to iMovie is definitely a step backwards but that's not that surprising since iMovie is free and Vegas is over $500.


Lethal

 

[ArthurDaley]

Thanks folks for all the responses. I reckon there is some good things for Google to hit on for others searching around the same topic (if to import at 960 or 1920).

All I want is tape to h/drive, full res, no colour loss, i.e. great archive, lowest possible storage space. When I edit all I do is chop out pieces - I don't play with effects, colours etc. Simply cuts. So on the PC I can import to .MT2 and have no loss stored at 11 gb/hr. I can then trim it using free programs, but happen to have Vegas.

Now it seems to use the Mac the field is complicated. I am willing to learn. But it seems people knocked .mt2 because it is not good for editing due to being highly compressed. Well my main aim is archive and I've had no problems editing because I just do cuts. Maybe if I was applying effects etc. I'd notice.

So the PC setup seemed perfect for an average consumer with an HD camcorder and it can all be done free (editing and capture software can be found free on PC). So on the PC the average consumer can store at only 11 gig/hr, no loss and can edit (chop/cuts etc) without paying for software.

Now it seems on the Mac there is no way for me to transfer the tape without loss and store at 11 gb/hr.

It seems the best way if I can get hold of FCP on the Mac is

If the OP has FCP he can either select the HDV 1080i60 Easy Setup or select the AIC 1080i60 Easy Setup and be good to go. Or he can keep is current M2T files, transcode them into the Apple HDV 1080i60 codec using MPEG Stream Clip, and import them into FCP project created w/the HDV 1080i60 Easy Setup.[/quote/

This would seem to say that I should continue to capture to PC to .M2T for the 4:1 space saving and if I want to edit them follow the sequence above.

As regards the colour information, all the talk of 4:2:0 and sampling rates, 8 bit , 10 bit etc is complicated. Surely the tape - the HDV format has contraints and surely you can set the capture to match the best the tape has? i.e. if HDV is 8 bit there is no point using 10 bit capture?

As regards xvYCC, the camera (HDR7) and the display (Samsung) both support xvYCC, and I even made sure I got a 1.3 HDMI cable with 10gig bandwidth to support it. But I will be darned if my eyes notice a difference between using xvYCC or not. And how xvYCC fits into capture (i.e. making sure the computer gets the extra colour space) nobody knows. So I just imagine it does not exist.

 

[NickFalk]

You are quite right that you cannot actually improve upon the quality of the original HDV. The reason why most people (myself included) recommend changing the format, specifically in regards to colour-compression, is that the instant you do anything with the clips other than straight cuts the HDV-format looses a substantial amount of information.

Another point to keep in mind when you hold up the "average-consumer" is that 720p is more than good enough. The reality is that at an 3 meters viewing distance you need a 50" screen to actually be able to see all the information this resolution contains. The 1080 is in other words for consumers major overkill as they won't be able to enjoy whatever practical difference there is between the 1080 and 720 signal.

But of course if you feel that the standard use for HD-camcorders is to just to quick edits without any colour-correction, filters, titles etc. and watch the finished product on, say a 72" screen, then you're quite probably right.

 

[ArthurDaley]

ok so conclusion is to work with HD, capture on a Windows machine using the free HDVsplit software, that stores it at 1080i (no loss) @ 10 gig /hr. Simple edits can be done on the PC against the .m2ts file on PC or use mpegstreamclip software for Mac to bring it over but then it takes up 40 gig/hr.

 

[zioxide]

ok so conclusion is to work with HD, capture on a Windows machine using the free HDVsplit software, that stores it at 1080i (no loss) @ 10 gig /hr. Simple edits can be done on the PC against the .m2ts file on PC or use mpegstreamclip software for Mac to bring it over but then it takes up 40 gig/hr.

You seem to be ignoring the fact that FCP can capture and edit in native 1080i60 HDV. This is going to be the same 13GB/hr files as the m2t on the pc, just stored in a .mov container (afaik). The only way its going to be ~40GB/hr is if you set FCP to convert to AIC.

 

[ArthurDaley]

Oh OK, FCP can capture 1080i and store it at the same size and let me chop it? OK I recapture.