Even better is to make sure that you have all artists and all albums selected in the iTunes window, and then go File>Print. You can print an album list, or song list - just muck about with the options.
If you don't want to print it, you can save it as a PDF.