Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Activity monitor reports huge VM size in SL

Am3822

Am3822

macrumors 6502
Original poster
Aug 16, 2006
424
0
Groningen, The Netherlands
I don't think it has been reported (or at least I couldn't find it), and a similar problem existed in 10.5 -- the amount of memory reported by the Activity Monitor application is exaggerated.

Right now my reported VM size is 138GB, on a driver with a 100GB free. Any suggestions how to solve it?

The computer in question is a Feb 2008 MBP, and the SL installation is an upgrade.
 

Attachments

  • Screen shot 2009-09-28 at 6.09.21 PM.png
    Screen shot 2009-09-28 at 6.09.21 PM.png
    35.3 KB · Views: 830
VM size has nothing to do with hard disk space or with the amount of ram being used. It's basically a useless measurement in 99% of situations.
 
This is a bit late, but it may still be useful. VM size is the number of addresses allocated to a program, whereas memory used is the number of addresses that actually point to memory. So if a program memory maps a file, then that range of addresses will point to the file on the hard disk, instead of into ram (for example).
 
This is more technically accurate than what Catfish_Man wrote:

VM size

"VM size" is the amount hard disk space being used as virtual memory (this number also includes the amount of installed RAM).

Virtual memory allows Mac OS X to "virtually" use more memory than the amount of RAM you have by using hard disk space to supplement RAM. However, hard disks are much slower than RAM, so Mac OS X automatically distributes information between disk space and RAM for efficient performance.

What does "memory maps a file" mean? All a program can do is load a file into memory, either partially or the whole thing.

S-
 
Uh, if you don't know what memory mapping a file is, you've got no business saying your explanation is more accurate :p

It is, in fact completely wrong, and I can link to the relevant kernel source code to prove it if you like. I can also write a demo app that has a VM size larger than your hard drive (assuming you're on a 64 bit machine), which would be impossible if what you quoted is true.

As for memory mapping;

http://developer.apple.com/Mac/library/documentation/Darwin/Reference/ManPages/man2/mmap.2.html has the detailed explanation

In (attempting) layman's terms, it's a way of reading in a file on demand in small non-sequential chunks by pretending that it's actually ram. All shared libraries are mmap()ed for example, so that unused pieces of code aren't loaded unnecessarily.
 
Catfish_Man said:
Uh, if you don't know what memory mapping a file is, you've got no business saying your explanation is more accurate :p

It is, in fact completely wrong, and I can link to the relevant kernel source code to prove it if you like. I can also write a demo app that has a VM size larger than your hard drive (assuming you're on a 64 bit machine), which would be impossible if what you quoted is true.

As for memory mapping;

http://developer.apple.com/Mac/library/documentation/Darwin/Reference/ManPages/man2/mmap.2.html has the detailed explanation

In (attempting) layman's terms, it's a way of reading in a file on demand in small non-sequential chunks by pretending that it's actually ram. All shared libraries are mmap()ed for example, so that unused pieces of code aren't loaded unnecessarily.
Click to expand...

Like I said, "All a program can do is load a file into memory, either partially or the whole thing." If you use the mmap function to do it or some other method, what I said is true. To be honest, I have not programmed on a Unix system in a long time and never used the mmap function so the terminology was not familiar to me and I suspect a lot of other people here.

Here is where I got that quote on VM:

http://support.apple.com/kb/HT1342

Please explain how you can allocate more virtual memory than you have disk space plus RAM in an actual functioning program.

S-
 
Here you go:
Code: 
#include <mach/vm_map.h>
#include <stdio.h>

int main() {
    char* data;
    kern_return_t err;
    int i = 0;
    for(;;) {
    	for (i = 0; i < 10; i++) {
            //allocate a big address range
            err = vm_allocate(  (vm_map_t) mach_task_self(),
                        (vm_address_t*) &data,
                        4294967295,
                        VM_FLAGS_ANYWHERE);
            //touch the beginning and end of it. I forget why it doesn't take me seriously unless I do this; probably an optimization
            *data = 1;
            *(data + 4294967295-1) = 1;
   	}
        //Activity monitor freaks out and stops updating if I don't slow down a little
   	sleep(1);
    }
 
    return 0;
}

I saved that in vm.c and compiled with gcc vm.c, then ran with ./a.out.

and a screenshot:

vmproof.png


As for how this works, it's as I said. VSIZE indicates allocated *address space*, not memory.

<edit>
I should really file a bug report with Apple to get that technote corrected. For now though, sleep.
</edit>
 
What would happen if you tried to write data to each address in the VM address space?

Isn't it the job of the programmer to make sure they don't allocate space that does not exist?

S-
 
If you did that, then it would begin backing VM addresses with swapfiles on the hard drive once it ran out of physical ram for them. When it ran out of hard drive space it would presumably crash.
 
Catfish_Man said:
If you did that, then it would begin backing VM addresses with swapfiles on the hard drive once it ran out of physical ram for them. When it ran out of hard drive space it would presumably crash.
Click to expand...

Isn't it the job of the programmer to make sure they don't allocate space that does not exist? I know thew answer is "Yes". So this:

VM size

"VM size" is the amount hard disk space being used as virtual memory (this number also includes the amount of installed RAM).

Virtual memory allows Mac OS X to "virtually" use more memory than the amount of RAM you have by using hard disk space to supplement RAM. However, hard disks are much slower than RAM, so Mac OS X automatically distributes information between disk space and RAM for efficient performance.

Is accurate and VM size is as defined. Anything more than what is defined would be considered a programming error or bug.

S-
 
No, that is not accurate. The fact that it would be bad to run out of space to back VM with does not mean that all VM is backed.

Put another way, the hard drive space is definitely not being used until something is written to the allocated address space. You can put other files in it freely.

Also, you can back VM space with things other than hard drives. For example, one common way of working with serial ports was to back an address with the port, so that when you write to that address it sends data out the port.

Finally, it would not be a programming error to run out of memory. All you need to do to see this is fill your hard drive with your own files (no particular program involved here), then start launching apps until you run out of ram. There's no possible way programmers can defend against this unless they know beforehand how much ram they will need, and pre-allocate all of it (which is a terrible idea for most apps).
 
VM SIZE is the vm of all processess open added together,you should look at swap used instead.
 
Catfish_Man said:
No, that is not accurate. The fact that it would be bad to run out of space to back VM with does not mean that all VM is backed.

Put another way, the hard drive space is definitely not being used until something is written to the allocated address space. You can put other files in it freely.

Also, you can back VM space with things other than hard drives. For example, one common way of working with serial ports was to back an address with the port, so that when you write to that address it sends data out the port.

Finally, it would not be a programming error to run out of memory. All you need to do to see this is fill your hard drive with your own files (no particular program involved here), then start launching apps until you run out of ram. There's no possible way programmers can defend against this unless they know beforehand how much ram they will need, and pre-allocate all of it (which is a terrible idea for most apps).
Click to expand...
Some OS's allocate swap space on disk when the address space is allocated. FreeBSD does not. Solaris used to and may still. For VM to be useful, it needs to be backed. A good programmer would make sure that the space required would be available. A program crash is ALWAYS a bug since it should never happen. A programmer should never allow low memory conditions to crash his app.

S-
 
Pre-allocating swap space for all VM allocations just in case the user simultaneously fills up their disk and runs out of ram would be a fairly useless and disk-intensive safety net.

In out of memory + disk conditions, it would be impossible to save your work anyway, since the space necessary for the create-write-rename-delete "atomic save" pattern wouldn't be available. If you can't keep using the program, and you can't save your work... that sounds like a crash by another name to me.

Solaris presumably had some reason to do this if you're correct that it did, but I highly doubt that it's a safety measure that makes sense on OSX. Certainly no other desktop OS I'm aware of does anything similar.

<edit>
I've filed a bug with Apple to get the technote corrected.
</edit>
 
Hey guys, I just noticed that my VM Size is over 100GB as well on a 13" MBP with 2GB Ram. I've rebooted, but even after a reboot, it's still over 100GB. After looking over this thread, it appears that it isn't anything to worry about, but I just wanted to make sure that I was right in that guess :-D So, just to clarify, is that large of a Virtual Memory Size something I should have concern with, or just brush it off?
 
Catfish_Man is correct. The VM Size is the measurement of the allocated space, not consumed space. Once it becomes used, it will, as he described, fill the hard drive to capacity and then crash.

The Swap Used measurement should be where you look for VM space (larger than available RAM) that has been consumed by processes.

For example, say that I want to measure how much money there is at my disposal. I have $1000 in cash (ram) and $4000 in savings (Hard Drive Space). Paraphrasing Catfish_Man's code, we want to buy something that costs more than our cash and savings combined. Let's say it's $6,000.

The VM size would be $6000 because that is how much I've allocated for the purchase in the "budget". That doesn't mean that I've spent it yet. It still hasn't actually taken up a single dollar. It's just a measurement of my desire to spend.

The problems only arise once I actually attempt to spend that amount ... my card is declined and I'm cleaning up a mess for some time to come.

So, to sum this all up, the large VM size is an indicator of potential catastrophe should all of the processes cash in on their desires. Most processes do make sure that there is enough resources available when writing data to a file. If there is a deficit of HDD space at the time of need, it is up to the process to crash gracefully (not the OS).

I hope that this clears it up in layman's terms for Sidewinder and anyone else who reads this.

Thank you,

Steve Bostedor
Host - The iPad Show
http://www.ipadshow.com
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back