Are you referring to the Compilation tag in iTunes? If I remember correctly, that feature doesn't carry over to your iPod. So if you have a lot of CDs with multiple artists (like a movie soundtrack), you end up cluttering your iPod's Artists menu with a whole bunch of names.
My solution to the problem was this:
1. Remove the Compilation tag from all my files.
2. Copy the Artist field -> Composer field for every song in the library (
http://www.dougscripts.com/itunes/ has Applescripts that do this automatically).
3. Rename the Artist field in all compilation-style albums to "Various Artists".
4. Change my Smart Playlists that look in the Artist field to look in the Composer field instead.
This way, I can scroll through the Artists menu in my iPod and have all the compilations under the name "Various Artists", but my smart playlists will still find any specific artist's songs listed there.
Hope this helps.
Edit:
http://www.mp3tag.de/en/index.html has a terrific free tag editor for Windows, too. Just make sure you back up your AAC-encoded files before doing any mass-conversions, because it used to corrupt my files sometimes. Works much better now though!