When it comes to RAM...more is ALWAYS better. Basically you can NEVER have too much RAM.
LOL...I couldn't resist, get so tired of hearing this.
But yeah, basically what has been said by others above, fire up whatever program, FCP, Photoshop, Aperture, etc. you plan on using for the task at hand and check what Activity Monitor says under System Memory in the Used/Active portions. This will give you some idea of what your actual memory load is currently with your program of choice running and will help you decide on the proper amount.
Give yourself a little bit of headroom, consider the number of DIMM slots available and plan your RAM purchase accordingly so that if needed you can expand later without having to purchase all new DIMMS just to add a few more GB of RAM.
FYI, I have an 06 Mac Pro, I use Aperture to edit large RAW files from a Canon 5D Mark II, and I edit small HD video projects with FCP etc. and the 8GB of RAM I have installed has always been adequate for my needs.
I am building a new 6-core hackintosh that will have 12GB of RAM installed which is more than I need but the price was right at $269 for 3x4GB so I went for it.