To set the default application to open any file type:
1) Select a file of the type in the Finder.
2) Do a "Get Info" on the file (command-i).
3) In the "Open with:" section of the info window, select the application you want to open that type.
4) Click the "Change All..." button underneath.
Now double-clicking on a file of that type will open it in the selected application. Repeat for each video type you want to open in Quicktime (that is, once for .mov, once for .avi, etc).