Mavericks will use whatever the ram you have available to the max. Compared to previous OS's like Lion or ML, it will look like it is using more ram, however don't be fooled by the actual ram usage. Look at the "page outs" and the amount of compressed memory. The new Activity Monitor shows the "Memory Pressure". If you see more page outs and the memory pressure being pushed to the red zone, then worry. In other words, even if you install 32Gb memory, Mavericks again will use almost all of it. It never empties its cache unless it has to, thinking that whatever is used once, it will be used again.
Here is a good article to read about:
http://arstechnica.com/apple/2013/10/os-x-10-9/17/#compressed-memory
At first, I was annoyed to see the memory being occupied more so compared to Mountain Lion but after reading those technical articles, I understand what is actually going on.
The only upgrade I am against is the new iMovie 10 (10.0.2 to be precise). It is the worse piece of software Apple has created so far. Stay away from it.
Oh, I’ve read articles on Mavericks memory compression technology and seen it in action and I have no problem with that when you have a system that has more RAM ,and therefore more available virtual memory and a higher amount of RAM that can be compressed to save the system from using the SWAP. I get that.
I also know that it will only use whatever Ram you have available to the max once you reach that max. If basic usage for me with the apps I had listed results in 4+ GB of memory usage in a system with 16GB RAM, then there is no memory pressure, no compression, no swap used, and no proportional increase in virtual memory.
That being said, the compression algorithm is great for compression of a memory cache to about half its size. Great. That means with 32GB RAM, you have roughly 16GB of RAM that can be compressed, which would result in having 48GB of virtual memory (I think). 16GB of Ram will give you a limit of 8GB that can be compressed resulting in a proportional increase in virtual memory and so on down the line.
So the OP only having 4GB on an older system which is probably the last time that his iMac will be able to be upgraded (even with the compression technology in Mavericks), Mavericks will only be able to fully compress 2GB an will still only give the OP 6 GB of virtual memory and will result in continued beach-balling or slow downs.
Sure, memory purging in older OSes wasn’t as sophisticated as it is in Mavericks, but the newer apps requiring more memory initially doesn’t make upgrading to Mavericks (or staying with Mavericks) seem like a good move unless some hardware upgrades are performed first to mitigate the concerns of Mavericks' memory requirements.
I of course could be wrong in my understanding of how the compression works. And if that's the case, then fine. But the way that you posted, it seems like you belie the memory compression happens continuously. If that IS what's happening, then cool, but that would still only result in a total compression of half the available memory. I would also think that the continual use of the compression algorithm that is multi-threaded would not be wise given the need for power efficiency.
Here are my stats from activity monitor right now with the following apps open and running:
Safari (3 tabs open)
Mail
TextEdit
Physical Memory: 16.00 GB
Memory Used: 5.56 GB
Virtual Memory: 16.00 GB
Swap Used: 0 bytes
App memory: 1.79 GB
File Cache: 2.47GB
Wired Memory: 1.30 GB
Compressed: 0 bytes
Memory Pressure: completely at the bottom and in the green.
And just for kicks, I decided to do a manual purge in terminal and these are the new stats:
PhysicalMem: 16GB
MemUsed: 3.25 GB
VirtualMem: 16GB
SwapUsed: 0 bytes
AppMem: 1.80GB
FileCache: 160.00 MB
WiredMem: 1.29 GB
Compressed: 0 bytes
Memory Pressure: still at the bottom and completely green.