Activity monitor reports huge VM size in SL

Discussion in 'macOS' started by Am3822, Sep 28, 2009.

  1. Am3822 macrumors 6502

    Am3822

    Joined:
    Aug 16, 2006
    Location:
    Groningen, The Netherlands
    #1
    I don't think it has been reported (or at least I couldn't find it), and a similar problem existed in 10.5 -- the amount of memory reported by the Activity Monitor application is exaggerated.

    Right now my reported VM size is 138GB, on a driver with a 100GB free. Any suggestions how to solve it?

    The computer in question is a Feb 2008 MBP, and the SL installation is an upgrade.
     

    Attached Files:

  2. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #2
    VM size has nothing to do with hard disk space or with the amount of ram being used. It's basically a useless measurement in 99% of situations.
     
  3. maflynn Moderator

    maflynn

    Staff Member

    Joined:
    May 3, 2009
    Location:
    Boston
  4. Am3822 thread starter macrumors 6502

    Am3822

    Joined:
    Aug 16, 2006
    Location:
    Groningen, The Netherlands
    #4
    Thanks for the replies, people.

    @Catfish_man : I thought that the VM size is the size of OSX's swap file.
     
  5. maflynn Moderator

    maflynn

    Staff Member

    Joined:
    May 3, 2009
    Location:
    Boston
    #5
    Nope.

    I have 500meg worth of swap files but my VM size is 155gig
     
  6. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #6
    This is a bit late, but it may still be useful. VM size is the number of addresses allocated to a program, whereas memory used is the number of addresses that actually point to memory. So if a program memory maps a file, then that range of addresses will point to the file on the hard disk, instead of into ram (for example).
     
  7. sidewinder macrumors 68020

    sidewinder

    Joined:
    Dec 10, 2008
    Location:
    Northern California
    #7
    This is more technically accurate than what Catfish_Man wrote:

    VM size

    "VM size" is the amount hard disk space being used as virtual memory (this number also includes the amount of installed RAM).

    Virtual memory allows Mac OS X to "virtually" use more memory than the amount of RAM you have by using hard disk space to supplement RAM. However, hard disks are much slower than RAM, so Mac OS X automatically distributes information between disk space and RAM for efficient performance.


    What does "memory maps a file" mean? All a program can do is load a file into memory, either partially or the whole thing.

    S-
     
  8. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #8
    Uh, if you don't know what memory mapping a file is, you've got no business saying your explanation is more accurate :p

    It is, in fact completely wrong, and I can link to the relevant kernel source code to prove it if you like. I can also write a demo app that has a VM size larger than your hard drive (assuming you're on a 64 bit machine), which would be impossible if what you quoted is true.

    As for memory mapping;

    http://developer.apple.com/Mac/library/documentation/Darwin/Reference/ManPages/man2/mmap.2.html has the detailed explanation

    In (attempting) layman's terms, it's a way of reading in a file on demand in small non-sequential chunks by pretending that it's actually ram. All shared libraries are mmap()ed for example, so that unused pieces of code aren't loaded unnecessarily.
     
  9. sidewinder macrumors 68020

    sidewinder

    Joined:
    Dec 10, 2008
    Location:
    Northern California
    #9
    Like I said, "All a program can do is load a file into memory, either partially or the whole thing." If you use the mmap function to do it or some other method, what I said is true. To be honest, I have not programmed on a Unix system in a long time and never used the mmap function so the terminology was not familiar to me and I suspect a lot of other people here.

    Here is where I got that quote on VM:

    http://support.apple.com/kb/HT1342

    Please explain how you can allocate more virtual memory than you have disk space plus RAM in an actual functioning program.

    S-
     
  10. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #10
    Here you go:
    Code:
    #include <mach/vm_map.h>
    #include <stdio.h>
    
    int main() {
        char* data;
        kern_return_t err;
        int i = 0;
        for(;;) {
        	for (i = 0; i < 10; i++) {
                //allocate a big address range
                err = vm_allocate(  (vm_map_t) mach_task_self(),
                            (vm_address_t*) &data,
                            4294967295,
                            VM_FLAGS_ANYWHERE);
                //touch the beginning and end of it. I forget why it doesn't take me seriously unless I do this; probably an optimization
                *data = 1;
                *(data + 4294967295-1) = 1;
       	}
            //Activity monitor freaks out and stops updating if I don't slow down a little
       	sleep(1);
        }
     
        return 0;
    }
    
    I saved that in vm.c and compiled with gcc vm.c, then ran with ./a.out.

    and a screenshot:

    [​IMG]

    As for how this works, it's as I said. VSIZE indicates allocated *address space*, not memory.

    <edit>
    I should really file a bug report with Apple to get that technote corrected. For now though, sleep.
    </edit>
     
  11. sidewinder macrumors 68020

    sidewinder

    Joined:
    Dec 10, 2008
    Location:
    Northern California
    #11
    What would happen if you tried to write data to each address in the VM address space?

    Isn't it the job of the programmer to make sure they don't allocate space that does not exist?

    S-
     
  12. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #12
    If you did that, then it would begin backing VM addresses with swapfiles on the hard drive once it ran out of physical ram for them. When it ran out of hard drive space it would presumably crash.
     
  13. sidewinder macrumors 68020

    sidewinder

    Joined:
    Dec 10, 2008
    Location:
    Northern California
    #13
    Isn't it the job of the programmer to make sure they don't allocate space that does not exist? I know thew answer is "Yes". So this:

    VM size

    "VM size" is the amount hard disk space being used as virtual memory (this number also includes the amount of installed RAM).

    Virtual memory allows Mac OS X to "virtually" use more memory than the amount of RAM you have by using hard disk space to supplement RAM. However, hard disks are much slower than RAM, so Mac OS X automatically distributes information between disk space and RAM for efficient performance.


    Is accurate and VM size is as defined. Anything more than what is defined would be considered a programming error or bug.

    S-
     
  14. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #14
    No, that is not accurate. The fact that it would be bad to run out of space to back VM with does not mean that all VM is backed.

    Put another way, the hard drive space is definitely not being used until something is written to the allocated address space. You can put other files in it freely.

    Also, you can back VM space with things other than hard drives. For example, one common way of working with serial ports was to back an address with the port, so that when you write to that address it sends data out the port.

    Finally, it would not be a programming error to run out of memory. All you need to do to see this is fill your hard drive with your own files (no particular program involved here), then start launching apps until you run out of ram. There's no possible way programmers can defend against this unless they know beforehand how much ram they will need, and pre-allocate all of it (which is a terrible idea for most apps).
     
  15. SHankara macrumors member

    Joined:
    Oct 5, 2007
    #15
    VM SIZE is the vm of all processess open added together,you should look at swap used instead.
     
  16. sidewinder macrumors 68020

    sidewinder

    Joined:
    Dec 10, 2008
    Location:
    Northern California
    #16
    Some OS's allocate swap space on disk when the address space is allocated. FreeBSD does not. Solaris used to and may still. For VM to be useful, it needs to be backed. A good programmer would make sure that the space required would be available. A program crash is ALWAYS a bug since it should never happen. A programmer should never allow low memory conditions to crash his app.

    S-
     
  17. Catfish_Man macrumors 68030

    Catfish_Man

    Joined:
    Sep 13, 2001
    Location:
    Portland, OR
    #17
    Pre-allocating swap space for all VM allocations just in case the user simultaneously fills up their disk and runs out of ram would be a fairly useless and disk-intensive safety net.

    In out of memory + disk conditions, it would be impossible to save your work anyway, since the space necessary for the create-write-rename-delete "atomic save" pattern wouldn't be available. If you can't keep using the program, and you can't save your work... that sounds like a crash by another name to me.

    Solaris presumably had some reason to do this if you're correct that it did, but I highly doubt that it's a safety measure that makes sense on OSX. Certainly no other desktop OS I'm aware of does anything similar.

    <edit>
    I've filed a bug with Apple to get the technote corrected.
    </edit>
     
  18. 1ofthedavids macrumors regular

    Joined:
    Jun 8, 2009
    #18
    Hey guys, I just noticed that my VM Size is over 100GB as well on a 13" MBP with 2GB Ram. I've rebooted, but even after a reboot, it's still over 100GB. After looking over this thread, it appears that it isn't anything to worry about, but I just wanted to make sure that I was right in that guess :-D So, just to clarify, is that large of a Virtual Memory Size something I should have concern with, or just brush it off?
     
  19. sbostedor macrumors newbie

    sbostedor

    Joined:
    Aug 6, 2008
    Location:
    Michigan
    #19
    Catfish_Man is correct. The VM Size is the measurement of the allocated space, not consumed space. Once it becomes used, it will, as he described, fill the hard drive to capacity and then crash.

    The Swap Used measurement should be where you look for VM space (larger than available RAM) that has been consumed by processes.

    For example, say that I want to measure how much money there is at my disposal. I have $1000 in cash (ram) and $4000 in savings (Hard Drive Space). Paraphrasing Catfish_Man's code, we want to buy something that costs more than our cash and savings combined. Let's say it's $6,000.

    The VM size would be $6000 because that is how much I've allocated for the purchase in the "budget". That doesn't mean that I've spent it yet. It still hasn't actually taken up a single dollar. It's just a measurement of my desire to spend.

    The problems only arise once I actually attempt to spend that amount ... my card is declined and I'm cleaning up a mess for some time to come.

    So, to sum this all up, the large VM size is an indicator of potential catastrophe should all of the processes cash in on their desires. Most processes do make sure that there is enough resources available when writing data to a file. If there is a deficit of HDD space at the time of need, it is up to the process to crash gracefully (not the OS).

    I hope that this clears it up in layman's terms for Sidewinder and anyone else who reads this.

    Thank you,

    Steve Bostedor
    Host - The iPad Show
    http://www.ipadshow.com
     

Share This Page