Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
Maybe I'm missing something, but why is there no tool available to easily import, catalog and keyword digital video?

I use Photo mechanic and lightroom for my pictures, and can easily catalog and store thousands of files and search / manage them without problem.

For video however, getting the footage off my camera and onto my computer is hard enough - and keywording, etc I've not found any way to do it.

I use CS6 and have started to try Prelude, but even getting the video off the camera doesn't actually work unless you transcode it. So if you use another program like chronosync, well that doesn't work because it labels the entire folder with many video clips as simply "AVCHD" and then you have ?? clips within it.

What am I missing? How do others import/archive/keyword video? Photography software just seems light years ahead here.
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
I have not. I would prefer not to completely switch video editing suites if possible, having invested a chuck of change into Premiere Pro / AE and CS6, but I could consider it I guess.
 

jasonvp

macrumors 6502a
Jun 29, 2007
604
0
Northern VA
What am I missing? How do others import/archive/keyword video?

I don't keyword video as it serves no purpose to and for me.

That said, I use a "old fashioned" directory structure style of archiving AVCHD from my Sony cameras. I have a mirrored volume on my Mac; on that volume I have a directory for videos (called, oddly enough, "videos"). Inside that directory, I have a tree of others that are named sometime that describes the event, including a date. Any files I need for that event, including the saved Premiere Pro file, are kept in that directory.

Also in that event directory, I'll make a sub-directory for each camera. Called "camera 1" and "camera 2" up to "camera n". The names aren't important, and I could make them more descriptive if I needed to. I pop the flash card into my USB reader and copy the entire "Private" directory structure to each of the aforementioned sub-directories.

When I "import" them into Premiere Pro (less so an import, per se), I just tell it to look in the aforementioned camera sub-directories, and voila.

The basic summary is: use the OS' tools to arrange, archive, and organize the raw data. Don't use the video editing software to do it.

jas
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
Thanks for that feedback.

When I attach my video camera and download the data it comes as a package and within that package are multiple directories and files. In the past I've copied just the actual .mts files which AFAIK is the actual video, but there are other directory and "play list" files which I've been told can be important to have.

If I were to follow your system and manage these things on an os level, do I just copy the MTS files? That would make it much easier.
 

HobeSoundDarryl

macrumors G5
Here's a closely related option: Go into the MTS files and rename just those files that kick off a new shoot (event). For example, 000165 might get renamed to 000165HSBasketballVsRival" followed by several (automatically) numerically named files until the next one you've renamed (maybe 000222 becomes) "000222familyreunion2012" followed by several numerically named files. Then use a tool like Chronosync for syncing the camera folder to hard drive backup until the camera's hard drive is full (as which point you'll start a new backup archive). At that point you can go back in and delete the duplicate files which will be copied in with those you've renamed. For example, if 000222 became "000222FamilyReunion2012", Chronosync will have copied 000222 over again, resulting in 2 copies of the same raw video. After you've started a new backup archive, you could go in and delete all those duplicates.

Another thing I use is the color coding options in Mac OS X. For stuff I want to get rid of after starting a new archive, I tag those files red (that's a great way to quickly prune all those duplicates). For stuff that kicks off a new segment of video, I tag the first file green. That becomes something like event markers or if I shoot one event over several days, green tags might segment out each day.

Basically, this is a pretty easy way to manage individual shoots (events) without making significant changes to how you organize the raw AVCHD on your disc.

In the bigger picture though, the place for tagging & keywords is after editing. Get the file into the final form then use a tool like MetaZ, MetaX or similar to richly describe the contents of the .m4v or mp4 video file. Then, for playback purposes, it is easy to quickly find the exact video you want, edited to whatever you consider the ideal bits within that video.
 

CaptainChunk

macrumors 68020
Apr 16, 2008
2,142
6
Phoenix, AZ
If you're looking to archive AVCHD video without having to transcode, you could use Disk Utility to make .dmg files for each card, which both keeps their original file hierarchies intact and gives you mountable volumes. Then, you would just place them in folders using names that make sense to you. This is how I've always archived AVCHD...

I seem to recall reading a thread on this forum that confirms that Mountain Lion will also play AVCHD media (the .mts files) natively in Quicktime, which would be useful if you wanted to quickly preview the media without transcoding or dropping it into an NLE.
 

jasonvp

macrumors 6502a
Jun 29, 2007
604
0
Northern VA
If I were to follow your system and manage these things on an os level, do I just copy the MTS files? That would make it much easier.

Does your camera have an SD card in it? Or does it cut the video directly to some internal storage/flash?

jas
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
I should have mentioned that - it's a hard disk based camera (sony HDR-xr520)

I've tried using chronosync but apparently with mountain lion 10.8.1 the entire folder is transferred as a "package" so even though there are just incremental changes (new files added since last transfer) chronosync sees it as an entirely new "AVCHD" package and so it copies everything all over again and moves the last copy to "archive" which then renames everything.

----------

If you're looking to archive AVCHD video without having to transcode, you could use Disk Utility to make .dmg files for each card, which both keeps their original file hierarchies intact and gives you mountable volumes. Then, you would just place them in folders using names that make sense to you. This is how I've always archived AVCHD...

I seem to recall reading a thread on this forum that confirms that Mountain Lion will also play AVCHD media (the .mts files) natively in Quicktime, which would be useful if you wanted to quickly preview the media without transcoding or dropping it into an NLE.

I'm not adverse to transcoding, I just don't know why or why not to do so.

I am aware that transcoded files can be much larger, and this is a concern for me as many of my projects are sports videos where I have say 2 hours of raw footage that I might clip just a few minutes out of.
 

HobeSoundDarryl

macrumors G5
I wouldn't transcode for archival purposes. Keep it in it's original form as your raw master file archive. Each generation can lose something so retaining the original retains the maximum quality of what you shot.

Consider transcoding for editing purposes. For example, I use clipwrap to take 1080p60fps from AVCHD to ProRes for FCP X editing.

That's sad news about Chronosync & Mountain Lion. Hopefully, that's a software fix away. I haven't upgraded to ML yet so it works very well for me.

For your sports video clips (small segments out of hours of footage), if you are regularly having to go back to the well (that is, you are having to go back to the raw footage to pull in other clips for other projects), it sounds like you need a clip art library-like solution. Slice them all up into the usable segments and then render them as master files. Tag those files with MetaZ, MetaX, etc and then draw from those second-generation masters for mixing & matching as needed in future projects. In other words, if your 2 hours of video has- say- 1 hour of usable clips with- say- a total of 30 clips in that 1 hour, edit and render those 30 clips at high quality h.264 and then tag those 30 clips with MetaZ or similar. Then use these as your on-demand library. Yes, they are second generation instead of raw but they will still look great in a 3rd generation project when you need them. That's probably the easiest way to accomplish what you want (if I'm guessing correctly at what you want).

If those small segments from hours of footage is the only bits you want to use permanently, edit them out and consider jettisoning the rest. If you don't want second generation (edited) archives, you could go back into the MTS folder and use VLC or similar to watch each clip to isolate those you want to keep. Then, drag the rest to the trashcan. That would keep first generation masters of just the bits you want.

Another option would be to edit together all of the clips you want to keep as one file and set chapter markers for each clip. This would yield one larger file with chapter markers to help you jump right to the clips you want. Then, when you need to pull video clips into subsequent projects, you can come back to a small number of "digest-like" clips and clip the pieces other than the chapters you want in your new project.

A variation on that one would be to organize all of like clips together into larger files. For example, all basketball video clips into a large file, all baseball video clips into another large file. Then, use chapter markers to segment all of those clips for the final render. This would organize all of your individual sports together in a single file or two. If you need something similar but organized by teams, same basic approach (all team #1 basketball, all team #1 baseball, etc).

Lastly, if money is not really an object, you could buy huge storage and transcode all of it into something for FCP-X and just have them all dynamically available when you need them (much like iMovie clips are readily available in iMovie). With HD footage, this will need a lot of hard drive space so you'll need to be looking at big (multi-drive) storage. Synology makes some 12 bay NAS devices that look favorable... stuff like: http://www.synology.com/products/product.php?product_name=DS2411+&lang=us with some room to grow beyond even 12 * 4TB. There's other solutions out there that can hold much greater storage at much higher prices.
 
Last edited:

jasonvp

macrumors 6502a
Jun 29, 2007
604
0
Northern VA
I should have mentioned that - it's a hard disk based camera (sony HDR-xr520)

Bleah, OK... When you connect it to the Mac, does it appear as another drive, I assume? If you go crawling through the mounted drive using Finder, I assume you don't see the usual directory structure associated with AVCHD? What if you pop open a Terminal window and crawl through it using the UNIX CLI?

jas
 

Madmic23

macrumors 6502a
Apr 21, 2004
894
980
I have an AVCHD camcorder, but I'm thinking of switching over to a Rebel T4i. The T4i, like other DSLR's, saves the video as mpeg4 which seems like it's easier to manage, but the file sizes are huge.

Sorry to thread jack, but I'm curious to see what everyone's take is on the AVCHD vs MPEG4 debate and the pros and cons of organizing / using each.
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
Bleah, OK... When you connect it to the Mac, does it appear as another drive, I assume? If you go crawling through the mounted drive using Finder, I assume you don't see the usual directory structure associated with AVCHD? What if you pop open a Terminal window and crawl through it using the UNIX CLI?

jas

yes it's a "no name" drive

I can get into the actual folders only by using the "open package content" option in the finder. But AFAIK chronosync doesn't do that so it sees a whole "new" file every time I add any video so it backs up the entire contents over again each time.

I think this is a ML issue because before I saw all the directories individually when I opened the drive.
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
well new drama in this project. I copied the mts files individually as suggested above into folders based on events/dates and organized them.

Now however when I try to ingest then work with them in adobe prelude I am unable to add subclip information (options are greyed out) and i'm reading that perhaps moving the file structure can cause this. not sure why but this is now even worse and I don't have the original file structure now.

Argh!

Do all prosumer camcorders nowdays use this avchd file structure? It just seems WAY more problem than it's worth
 

HobeSoundDarryl

macrumors G5
pprior, get Clipwrap, drag the video files into it first for conversion into a Quicktime movie container (I usually convert them to ProRes for FCP X work but there are other options), and all should work well.

However, yes, when you backup from camera, ideally you backup the whole file structure instead of just the video streams themselves. Clipwrap works either way but I pretty much always back up the whole structure too.

I wouldn't try organizing the raw files in folders. Do your organization in finished, edited videos (for which it is very easy). Else, see #10 again above and also consider picking up Chronosync. Once you backup a whole archive from camera (when you'll no longer be adding any new video to that archive), you can go back in and rename the clips with some helpful names for locating raw files. On a Mac, I also use simple color tagging and VLC previews of AVC for segmenting one AVC group of clips from the next.
 
Last edited:

floh

macrumors 6502
Nov 28, 2011
460
2
Stuttgart, Germany
well new drama in this project. I copied the mts files individually as suggested above into folders based on events/dates and organized them.

Many people do that, but it's a bad idea. The AVCHD container not only consists of the stream (.MTS) files but stores most of its metadata (timecode etc.) in the rest of the folder structure. In the future, try to backup the whole folder structure.

To save the .MTS files you have already seperated, you can either use the mentione ClipWrap or first try the free little tool (I just programmed a Mac version for) to change video containers:
http://sourceforge.net/projects/containerswitch/

Good luck!
 

floh

macrumors 6502
Nov 28, 2011
460
2
Stuttgart, Germany
I have an AVCHD camcorder, but I'm thinking of switching over to a Rebel T4i. The T4i, like other DSLR's, saves the video as mpeg4 which seems like it's easier to manage, but the file sizes are huge.

Sorry to thread jack, but I'm curious to see what everyone's take is on the AVCHD vs MPEG4 debate and the pros and cons of organizing / using each.

My personal opinion: AVCHD in its original file structure is more annoying to work with. People say (I haven't tried it) that Mountain Lion supports it natively, but so far it's tougher to play back the stream and manually organize it on your computer.

But (and this is a big but): I try not to organize the file structure manually. You can import files from the AVCHD folders into basically any editing software. And if you do so, the containers will be changed by the software and it will also allow you to edit the metadata to your needs and tag everything.

What I want to say: The codec inside is another matter, and if you (I recommend it) organize and handle your movie data within your editing suite, it's about the same comfort and speed, no matter the original container.

But that's not really why I'm answering. What I wanted to say is: Please don't make your camera choice dependent of a video container format. You will be able to live with both, and there are many very good arguments for and against every camera that are way more important than its container format.
 

daybreak

macrumors 6502a
Sep 4, 2009
531
0
Find me old fashion but i don't understand AVID software but reading these post, the main trouble seems to be logging clips from your camcorder?.
Should i be wrong stop reading now and just call me an idiot.
Working with FCP7 logging information is easy but in FCP-X it is a different ball game.
If you have difficulty logging and clip in a pacific software why not have a pen and pad and name your TAPE I with information and the same with card.
I'm editing a big project and FCP-X will not allow me to name individual clip. As i use tape my logging details are on paper.
TAPE ONE.(Graveyard).That is the info for my New Event and on paper. Then the paper trail list the main shoot.
Sometimes i think we need to go back to basics as some of the task we expect to do on our computer can fox us and loose our hair.
I do apologise if i'm reading this post wrong.:rolleyes:
 

Artful Dodger

macrumors 68020
I'm not sure I'm understanding this fully however if I can ask, I've been using AVCHD for a few years now and in the last month or a bit longer I purchased FCPX yet before that it was iMovie.

I would setup subfolders based upon my day or subject matter so my day would look like this: Day 1 Part 1 (1.5hrs long), Day 1 Part 2 (1.5hrs long) and so on, this would be placed into a main folder with say a presenters name. After the import or I should say archive I would add names and or dates and so on. Everything seemed to work out fine but again I just applied a simple way incase others needed to take my place for the day or event it would be ok.

I would just off load my video from the SD cards like that and check them with Toast 10 to make sure the audio was there and no issues. After the day was done then I'd start the edit process one section at a time. I'm not sure about ML as I don't have it yet, I always hold off a bit but I have read like you stated AVCHD is supported in ML.

Are you saying you name (keywords) while you work or on the import process you need more flexibility?

Again I just do the simple simple stuff because I was using iMovie so when people post stuff like you have going on I'm not sure what I'm missing or not being complete about with using AVCHD.
 

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
What is happening is that with 10.8.x the entire contents of the camera show up as a single "avchd" file. So any backup of the camera will basically re-copy every single video as "new" each time. When a camera has 200GB of stuff on it, that adds up quickly!

In addition, the file structure within that "file" (show contents under mac os) has multiple folders and if I have say several different project types on a single camera it's almost impossible to organize them because I can't just "extract" or pull files out of a single directory. I tried that as suggested above and now I can't do logging in prelude because I've screwed up something.

I tried re-wrapping into .mov format but then prelude was VERY sluggish trying to shuttle back and forth.

At this point I've given up for my current project and am using the .mts files I copied out of the folder structure. They play fine and I'm just creating subclips in premiere pro, but again the whole thing just seems rediculously difficult to manage compared with say managing my RAW files in lightroom for my photography stuff.

----------

Find me old fashion but i don't understand AVID software but reading these post, the main trouble seems to be logging clips from your camcorder?.
Should i be wrong stop reading now and just call me an idiot.
Working with FCP7 logging information is easy but in FCP-X it is a different ball game.
If you have difficulty logging and clip in a pacific software why not have a pen and pad and name your TAPE I with information and the same with card.
I'm editing a big project and FCP-X will not allow me to name individual clip. As i use tape my logging details are on paper.
TAPE ONE.(Graveyard).That is the info for my New Event and on paper. Then the paper trail list the main shoot.
Sometimes i think we need to go back to basics as some of the task we expect to do on our computer can fox us and loose our hair.
I do apologise if i'm reading this post wrong.:rolleyes:

I'm not using FCP. I believe their import and logging setup is somewhat better but I switched to adobe suite because of integration of after effects and overall workflow.

I'm also trying to organize and track footage overal several years, not just a single project. For example if I want to see all photographs of a certain girl dancing over the last 5 years I can get that in about 10 seconds in lightroom, but how can I do that with my video footage??
 

HobeSoundDarryl

macrumors G5
I'm also trying to organize and track footage overal several years, not just a single project. For example if I want to see all photographs of a certain girl dancing over the last 5 years I can get that in about 10 seconds in lightroom, but how can I do that with my video footage??

Don't work in raw video for dynamic archiving. It works for photographs because they are small files. It also works for music because songs are small files. Video is rarely small files. HD Video even more so.

For the dancing girl analogy, suppose she has dance events that you capture on video from time to time over that 5-year span. Edit that video out as either separate clips and render them as small final files. Then, tag them with a poster image, description and even search terms (her name, dance, dancing, etc) and put them in a folder ("EllensDanceVideos" or "Dance", etc). If you need to mix & match them in future events, go back and work with those renders (splice them together, mix them with other video, etc). All very easy to do.

I've got hundreds of "home movies" I've shot on AVCHD equipment over the last few years organized exactly this way. I split them out by year and name the files with names so that I know what video is rendered within them. I tag them with MetaX or MetaZ with a unique poster, a good description of their contents and many other details typical of iTunes video metadata (making them easy to sort & find via search). Yes, I also hang onto the masters (the original AVCHD file structure) should I ever need to go back to them for future editing work, but the edited & rendered files are easy to organize, search, etc.

OR, merge all of the video clips of her dancing into a single file and maybe use Chapters to identify one shot 5 years ago from one shot 4 years ago, etc. If she has some more dancing events in the future, you can just append new shoots onto these older renders.

As previously described, you're trying to organize in raw, unedited footage. Take the next step. Edit it into final renders (after editing) and then tag those rendered videos with everything you need to quickly locate them anytime you want.

OR, load up on hard drive storage (HUGE, HUGE storage) and read all of those raw clips into editors like FCP X in which you can then tag raw clips and have them all dynamically available (if you have enough storage for all of them) to mix & match as you see fit.

But the easy answer continues to be going from large, unedited raw files to edited and rendered smaller files. You'll be taking raw video and getting it into the consumable form... just like a collection of photos or music. The latter just doesn't take the same amount of processing by nature of how the raw files are generally the consumable form (or quickly become the consumable form on import).

If you just can't bring yourself to go in one of these ways, the next best option is to follow the film archival model. Open up a program like Word, enter each file ID name in the AVCHD stream folder, watch it with VLC and document its contents in that Word doc. Then, you can use the Word doc as a searchable doc to locate specific raw clips that contain whatever you want to find. A more sophisticated version of this would be to build your "search" database in Excel or Numbers or- even better- a database program.

If you really want an iPhoto-like experience, render each raw file as a small Quicktime video. Drag that into iPhoto and tag it like you can tag a photo. iPhoto will play video files. Name these files with the same names as the raw video (AVCHD) names or put the raw video names in the "notes" field. This won't be archiving the raw video itself but small file proxies of the raw video (much like the thumbnails vs. underlying bigger photo files used in iPhoto). However, this would be pretty close in functionality to what you seem to be seeking- an at-a-glance video clip-based database of all clips shot with AVCHD.

Yes that won't be as easy as just importing photos into iPhoto but it shouldn't be. Why? Because video is not a single shot but many shots per second of various subjects, scenes, times, etc. Thus whether you choose the film archival model or this iPhoto proxy approach, either will still need someone to watch the clip and type a description of it into tags for subsequent search purposes.
 
Last edited:

Menneisyys2

macrumors 603
Jun 7, 2011
5,997
1,101
Thanks for that feedback.

When I attach my video camera and download the data it comes as a package and within that package are multiple directories and files. In the past I've copied just the actual .mts files which AFAIK is the actual video, but there are other directory and "play list" files which I've been told can be important to have.

If I were to follow your system and manage these things on an os level, do I just copy the MTS files? That would make it much easier.

You can - you can build the entire directory structure from scratch, automatically. I've even written a full tutorial on it (this is part of my larger article on AVCHD in general):

2. Manually creating / adding files to AVCHD (camera) archives

Above, I've already mentioned [PRIVATE/]AVCHD/BDMV/index.bdm is the main AVCHD / Blu-ray descriptor, which, among other things, tells the AVCHD-capable apps what clips there are in the archive. This is why, for example, you can't just copy additional M(2)TS files to [PRIVATE/]AVCHD/BDMV/STREAM, not even if you name them properly (for example, if the last original file in the STREAM directory was "00007.MTS", you name your copied file "00008.MTS" and so on) – they simply won't be shown when trying to import the camera archive.

A quick glance into index.bdm reveals it's a binary file and, therefore, can't be edited manually – only via third-party apps. Of them, there are two very famous ones: bdedit and multiAVCHD. We'll need the latter (the former isn't able to add new files, albeit knows a lot of niceties like MTS demuxing.) Download and install it. Note that it'll work under CrossOver on the Mac just fine; I've made my screenshots below in this mode. (Note that, by default, the output files will be stored under ~/Library/Application Support/CrossOver/Bottles/multiAVCHD_4.1.exe/drive_c/multiAVCHD with CrossOver.)

First, click “Add video files” in the top left corner (annotated below). Add any number of files. In this example, I only add MTS ones; however, you could add even AVI's, MOV's etc. An example of adding several camera videos I've collected for my forthcoming Camera Connection Kit and iOS Compatibility bible (see their name in the “Compilation” group, after returning from “Add video files”):



Click “Start” in the bottom right (also annotated above). In the next dialog, select the (default) PS3 one (top left, annotated):



The AVCHD archive creation starts. When it ends, you'll be notified in the status row:



Note that multiAVCHD doesn't create a topmost "PRIVATE" directory. The three apps will import the files it creates nonetheless - you won't need to create the directory either.

Side note: THIS DBM editor has nothing to do with our index.bdm file – it's a database editor.

EDIT: slight modding of the text & changing images to much smaller, clickable thumbnails
 
Last edited:
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.