Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

RAM Compression - what does it really mean?

brdeveloper

brdeveloper

macrumors 68030
Original poster
Apr 21, 2010
2,629
313
Brasil
Hello,

I'm planning upgrading my MacMini (mid-2010) to 16GB. I'm planning performing queries on my MySql databases on RAM. The tables have around 30M entries, usually filled with indexes to other tables, so each table entry is pretty small, but they have relationships which make the queries heavier.

I note that my 8GB Retina Macbook is doing a lot of swap while doing these queries (up to 12-15GB of "swap used" on Activity Monitor). The RAM pressure remains yellow most of the time and I get 4GB of "compressed memory".

My question is: with this information (12-15GB swap, 4GB compressed in a 8GB computer) does it mean that 16GB would be enough or maybe I'd need 32GB? I don't know if the compressed RAM feature is suitable for swapping compressed memory every time (since MySql probably needs accessing all entries at every moment) or is it useful only for storing inactive data from idle apps?

I don't know if OSX compresses data in the swap file too. If not so, the additional memory could keep the swap file empty while doing these queries since the uncompressed swap data should fit in 8GB of compressed RAM. What do you think?
 
Last edited:
maflynn

maflynn

macrumors Haswell
May 3, 2009
73,448
43,370
More ram will decrease or eliminate the swap space so your queries will run faster since you're not paging information out and back in.
 
brdeveloper

brdeveloper

macrumors 68030
Original poster
Apr 21, 2010
2,629
313
Brasil
maflynn said:
More ram will decrease or eliminate the swap space so your queries will run faster since you're not paging information out and back in.
Click to expand...

The problem is: what does it happen if my queries constantly need 20GB of memory space? It is the case when you have to compare n by m rows. Swapping is not an option (even with a SSD) since it will take forever until the query ends. You must have both tables in memory.

So look at this example:
****************

The orange area is occupied by the problematic sql query. The blue part is the other stuff that OSX needs for working, and other opened apps. Bold parts are compressed memory providing twice its original size. So there is a total of 20GB of "virtual RAM". Now, suppose that MySql needs reading some of the compressed portions of RAM. Will OSX be smart enough for compressing the uneeded area while it decompresses the now-needed area? Or will it do swap to SSD in this case?
 
maflynn

maflynn

macrumors Haswell
May 3, 2009
73,448
43,370
brdeveloper said:
The problem is: what does it happen if my queries constantly need 20GB of memory space? It is the case when you have to compare n by m rows. Swapping is not an option (even with a SSD) since it will take forever until the query ends. You must have both tables in memory.
Click to expand...
Rewrite the query to be more efficient. I mean is that if you have something that is going to chew up 20GB of ram, then something is wrong. What many people do with large queries is use temp tables to store intermediate results.

Oracle which is what I'm used to is very efficient for memory but it's a server application that is not designed to operate at peak efficiency on a desktop. Perhaps you're in a situation where you're expecting server type performance and results from a desktop.
 
C

cube

Suspended
May 10, 2004
17,011
4,972
I would think even in Brazil your best option is to buy the 16GiB and try it.
 
brdeveloper

brdeveloper

macrumors 68030
Original poster
Apr 21, 2010
2,629
313
Brasil
cube said:
I would think even in Brazil your best option is to buy the 16GiB and try it.
Click to expand...

I asked in the MacMini forum if the Mini/2010 accepts DDR3 1333 or 1600 RAM and most people are saying it don't. Anyway, my girlfriend's laptop has a 2GB/DDR3 1600 removable stick so I'm planning testing it on the Mini tonight.

Practically only OWC still sells 2x8GB DDR3 1066 RAM and it's pretty expensive for bringing it to Brazil (around USD 500). I would pay $250 just for trying, but $500 is a pretty high amount of money to spend on something that may not work as expected. On the other hand, DDR3 1600 is pretty affordable in Brazil (I could buy 2x8GB modules for $250).

----------
maflynn said:
Rewrite the query to be more efficient. I mean is that if you have something that is going to chew up 20GB of ram, then something is wrong. What many people do with large queries is use temp tables to store intermediate results.

Oracle which is what I'm used to is very efficient for memory but it's a server application that is not designed to operate at peak efficiency on a desktop. Perhaps you're in a situation where you're expecting server type performance and results from a desktop.
Click to expand...

I'll try, but I suspect that it won't work. Each time I make these queries, I need retrieving and joining all entries, so I think there isn't a workaround. This is for my master degree research and I don't have too much time to tweak my algorithms... I just need to see results faster from the data I collected. 16GB+ RAM seemed to be the easiest way to get it made.
 
C

cube

Suspended
May 10, 2004
17,011
4,972
brdeveloper said:
I asked in the MacMini forum if the Mini/2010 accepts DDR3 1333 or 1600 RAM and most people are saying it don't. Anyway, my girlfriend's laptop has a 2GB/DDR3 1600 removable stick so I'm planning testing it on the Mini tonight.
Click to expand...

It seems it should work with some, but probably not all.

I don't know what is the standard CAS level for that machine, but probably you should look at what might be the equivalent rating at 1333, and 1666.
 
C

cube

Suspended
May 10, 2004
17,011
4,972
cube said:
It seems it should work with some, but probably not all.

I don't know what is the standard CAS level for that machine, but probably you should look at what might be the equivalent rating at 1333, and 1666.
Click to expand...

As I was thinking, it seems to be 2 levels for each standard frequency jump:

http://en.wikipedia.org/wiki/DDR3_SDRAM#JEDEC_standard_modules

For example, I remember going from some modules at CAS 7 to others at CAS 9 at the next normal frequency above with my MBP and it worked.

I also remember buying faster modules for a picky Supermicro Atom board, and they did not work.
 
h9826790

h9826790

macrumors P6
Apr 3, 2014
16,614
8,545
Hong Kong
brdeveloper said:
My question is: with this information (12-15GB swap, 4GB compressed in a 8GB computer) does it mean that 16GB would be enough or maybe I'd need 32GB?
Click to expand...

4G Compressed with swap used means the system only able to compress 4G from that 8G RAM.

12-15G swap means all swap in + swap out = 12-15G. It only indicate that you need more RAM, the number won't tell you how much more RAM required.

The best parameter to check is actually "virtual memory" in activity monitor. Under high workload condition, it shows the system "demand" for how much RAM. However, it doesn't mean that you really need to buy so much RAM to fulfil that demand. e.g. Virtual memory shows 20G, you may only upgrade to 16G RAM, and let the system compress the RAM to fit that extra 4G demand under high workload condition.

Memory compression will have little bit performance hit, but you may not able to feel it. Again, as long as memory pressure shows green, your system is good.

"Demand" doesn't mean that the system "required" that amount. From the following screenshot, you can see that I only have 32G of RAM and the system "demand" for 48G, but it still able to keep the swap at zero by compressing 8G of memory. So, "virtual memory" is a good indicator, but really no need to go for that much. However, I am quite sure if I have 48G of RAM, the system will use all of them since that is the real demand.

attachment.php
 
Last edited:
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom