There's more to the picture than just free memory. You don't need to be concerned with how much free memory you have. The only thing you need to watch for is page outs.
Page outs are cumulative since your last restart, so the best way to check is to restart your computer and track page outs under your normal workload (the apps, browser pages and documents you normally would have open). If your page outs are significant (say 1GB or more) under normal use, you may benefit from more RAM. If your page outs are zero or very low during normal use, you probably won't see any performance improvement from adding RAM.