I've got a compilation album with tracks pulled from other albums, all with their own individual cover-art.
The only one that shows up in cover flow is the first tracks cover-art (when viewing by album, or by default on the iPhone/iPod Touch). But, when you play a subsequent track, the cover-art thats displayed either in the bottom left window on iTunes or on the screen on your iPhone/iPod Touch/iPad is the cover-art of said track.
So, whatever you want the cover-art to be for the album, make sure its on your first track.