Open activity monitor after a few days of use. Check your page-outs. If the number is high (several hundred MB or more) then you'll likely benefit a bit from more RAM.
I have 8GB and use the computer mostly for web browsing, video encoding and some other desk work. I occasionally run windows 7 in a virtual machine. With 8GB of RAM, I generally hover around 2-3GB free.