The Library folder under your home folder contains a "Mail" folder - that contains all of your emails. If you copy that elsewhere, you'll have saved a copy of all of your emails. There might be a "Mail Downloads" folder as well, which you can also copy.
In that same Library folder (again, the one in your home account, not the "Library" folder right under "Macintosh HD") is a Preferences folder. In that folder is a file named "com.apple.mail.plist" which contains, I think, some settings as well.