It is not your imagination.
The following information applies to older Quicktime Player versions and, it seems, to Quicktime X as well.
The Quicktime Player seeks out hidden tags in video files and tries to make the video look either A) the way the encoder intended, or B) the way apple thinks it would look best for your monitor. I work in video professionally and this is something that is a recognized issue/hassle when using QT Player.
Note that QT Player is also making an adjustment to the aspect ratio of the image (it is slightly squished vertically). These adjustments may be correct in that they display the image more like what the producer intended, or they may be totally wrong. If you don't like 'em, as far as I know, Apple doesn't allow you to shut them off. The only exception is if you have Final Cut Pro installed. If you do, then in QT Player a new option in the preferences will appear that gives you the option to ignore them but I believe this only applies within FCP itself (confusingly enough). I'm not a big FCP guy so maybe someone here who is could speak to that.
I'm not running Quicktime X Player personally, but the folks I know in the business who have looked into it tell me the situation hasn't changed for QT Player in terms of it "second guessing" the look of the material it displays.
The best check is to compare it to the original material before it was encoded (although I realize that is not always possible).
Good luck.