Sorry for the late reply, matey. I promise I'm not talking out of my rear-end; just going on my experience and what I've seen! I mentioned: "10.8 just hates reallocating RAM, it seems; no matter what, it'll insist on leaving the memory as 'inactive' and opt to write to the bloody HDD instead".
Although the RAM usage doesn't seem to be as high in your screenshot as when I was describing it, you'll still see there's a large 'unallocated' segment in your screenshot, leaving it to write to the HDD instead rather than using that unallocated segment. What I mean is that the unallocated should really be, well,
allocated to programs. Instead OS X seems to leave it that way, opting to write to the hard-drive instead, making its memory management a little confusing to me (well, compared to the Snow Leopard days). I'll be the first to admit that I don't know an awful lot about this and I'm only going from my experiences, so please feel free to tear me a new one if there's something about UNIX memory management that I don't appreciate.
I can't see VM being much of a problem when you've got an SSD due to the read/write speeds but with a 5400RPM drive I absolutely promise that it savagely crushes performance, and I'm inundated with calls about it daily -- when they're not even running much on their system either. It's really confused me and I've tried everything I can think of to fix the issue, without success. All I can say is that on every MacBook Pro running ML, 4GB RAM just isn't enough.
Hope this helps clarify my position in some way.