RAM Compression - what does it really mean?

Discussion in 'OS X Mavericks (10.9)' started by brdeveloper, Jun 9, 2014.

  1. brdeveloper, Jun 9, 2014
    Last edited: Jun 9, 2014

    brdeveloper macrumors 68020

    brdeveloper

    Joined:
    Apr 21, 2010
    Location:
    Brasil
    #1
    Hello,

    I'm planning upgrading my MacMini (mid-2010) to 16GB. I'm planning performing queries on my MySql databases on RAM. The tables have around 30M entries, usually filled with indexes to other tables, so each table entry is pretty small, but they have relationships which make the queries heavier.

    I note that my 8GB Retina Macbook is doing a lot of swap while doing these queries (up to 12-15GB of "swap used" on Activity Monitor). The RAM pressure remains yellow most of the time and I get 4GB of "compressed memory".

    My question is: with this information (12-15GB swap, 4GB compressed in a 8GB computer) does it mean that 16GB would be enough or maybe I'd need 32GB? I don't know if the compressed RAM feature is suitable for swapping compressed memory every time (since MySql probably needs accessing all entries at every moment) or is it useful only for storing inactive data from idle apps?

    I don't know if OSX compresses data in the swap file too. If not so, the additional memory could keep the swap file empty while doing these queries since the uncompressed swap data should fit in 8GB of compressed RAM. What do you think?
     
  2. maflynn Moderator

    maflynn

    Staff Member

    Joined:
    May 3, 2009
    Location:
    Boston
    #2
    More ram will decrease or eliminate the swap space so your queries will run faster since you're not paging information out and back in.
     
  3. brdeveloper thread starter macrumors 68020

    brdeveloper

    Joined:
    Apr 21, 2010
    Location:
    Brasil
    #3
    The problem is: what does it happen if my queries constantly need 20GB of memory space? It is the case when you have to compare n by m rows. Swapping is not an option (even with a SSD) since it will take forever until the query ends. You must have both tables in memory.

    So look at this example:
    ****************

    The orange area is occupied by the problematic sql query. The blue part is the other stuff that OSX needs for working, and other opened apps. Bold parts are compressed memory providing twice its original size. So there is a total of 20GB of "virtual RAM". Now, suppose that MySql needs reading some of the compressed portions of RAM. Will OSX be smart enough for compressing the uneeded area while it decompresses the now-needed area? Or will it do swap to SSD in this case?
     
  4. maflynn Moderator

    maflynn

    Staff Member

    Joined:
    May 3, 2009
    Location:
    Boston
    #4
    Rewrite the query to be more efficient. I mean is that if you have something that is going to chew up 20GB of ram, then something is wrong. What many people do with large queries is use temp tables to store intermediate results.

    Oracle which is what I'm used to is very efficient for memory but it's a server application that is not designed to operate at peak efficiency on a desktop. Perhaps you're in a situation where you're expecting server type performance and results from a desktop.
     
  5. cube macrumors G5

    Joined:
    May 10, 2004
    #5
    I would think even in Brazil your best option is to buy the 16GiB and try it.
     
  6. brdeveloper thread starter macrumors 68020

    brdeveloper

    Joined:
    Apr 21, 2010
    Location:
    Brasil
    #6
    I asked in the MacMini forum if the Mini/2010 accepts DDR3 1333 or 1600 RAM and most people are saying it don't. Anyway, my girlfriend's laptop has a 2GB/DDR3 1600 removable stick so I'm planning testing it on the Mini tonight.

    Practically only OWC still sells 2x8GB DDR3 1066 RAM and it's pretty expensive for bringing it to Brazil (around USD 500). I would pay $250 just for trying, but $500 is a pretty high amount of money to spend on something that may not work as expected. On the other hand, DDR3 1600 is pretty affordable in Brazil (I could buy 2x8GB modules for $250).

    ----------

    I'll try, but I suspect that it won't work. Each time I make these queries, I need retrieving and joining all entries, so I think there isn't a workaround. This is for my master degree research and I don't have too much time to tweak my algorithms... I just need to see results faster from the data I collected. 16GB+ RAM seemed to be the easiest way to get it made.
     
  7. cube macrumors G5

    Joined:
    May 10, 2004
    #7
    It seems it should work with some, but probably not all.

    I don't know what is the standard CAS level for that machine, but probably you should look at what might be the equivalent rating at 1333, and 1666.
     
  8. cube macrumors G5

    Joined:
    May 10, 2004
    #8
    As I was thinking, it seems to be 2 levels for each standard frequency jump:

    http://en.wikipedia.org/wiki/DDR3_SDRAM#JEDEC_standard_modules

    For example, I remember going from some modules at CAS 7 to others at CAS 9 at the next normal frequency above with my MBP and it worked.

    I also remember buying faster modules for a picky Supermicro Atom board, and they did not work.
     
  9. h9826790, Jun 10, 2014
    Last edited: Jun 10, 2014

    h9826790 macrumors 604

    h9826790

    Joined:
    Apr 3, 2014
    Location:
    Hong Kong
    #9
    4G Compressed with swap used means the system only able to compress 4G from that 8G RAM.

    12-15G swap means all swap in + swap out = 12-15G. It only indicate that you need more RAM, the number won't tell you how much more RAM required.

    The best parameter to check is actually "virtual memory" in activity monitor. Under high workload condition, it shows the system "demand" for how much RAM. However, it doesn't mean that you really need to buy so much RAM to fulfil that demand. e.g. Virtual memory shows 20G, you may only upgrade to 16G RAM, and let the system compress the RAM to fit that extra 4G demand under high workload condition.

    Memory compression will have little bit performance hit, but you may not able to feel it. Again, as long as memory pressure shows green, your system is good.

    "Demand" doesn't mean that the system "required" that amount. From the following screenshot, you can see that I only have 32G of RAM and the system "demand" for 48G, but it still able to keep the swap at zero by compressing 8G of memory. So, "virtual memory" is a good indicator, but really no need to go for that much. However, I am quite sure if I have 48G of RAM, the system will use all of them since that is the real demand.

    [​IMG]
     

Share This Page