It's the H264ness of it all. I have been recently moving all shows over to iTunes and to h264 format for consistency. all my AVI files run around 700MB, encoded over to H264, 1Gig. I do know exactly what your talking about.
This is not a very useful test for measuring the size/quality of h264 vs. AVI files. The size of the resulting h264 file can be controlled by adjusting the quality settings in the h264 encoder. But, since you're encoding from an (encoded/compressed) AVI file, the quality of the resulting h264 file can be no better than the AVI file, and probably somewhat worse, due to the decoding/re-encoding process.
A better test would be to start from original, uncompressed video, and encode it to both AVI and h264, adjusting the settings so that the resulting quality is comparable in both. h264 claims to be better, so if that's true, then the resulting h264 file should be smaller than the AVI file.