Launch Activity Monitor and click the System Memory tab at the bottom to check your page outs. Page outs are cumulative since your last restart, so the best way to check is to restart your computer and track page outs under your normal workload (the apps, browser pages and documents you normally would have open). If your page outs are significant (say 1GB or more) under normal use, you may benefit from more RAM.
Mac OS X: Reading system memory usage in Activity Monitor
Chances are very good that you can do everything you described without ever maxing out 4GB of RAM.