How do I increase swap space in Snow Leopard?

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
Did a cursory search of the forums and Google and I can't find the answer.

I'm running a 64 bit Matlab simulation on an enormous dataset and OS X (not Matlab) is telling me I'm running out of space on my startup disk for "application memory". I've got 300GB of free space on the hard drive, and only 60GB used as swap. I think the limit I'm hitting is the 64th swap file.

Anybody know how to grow that? I've tried booting to 64bit kernel, but no difference.
 

MWPULSE

macrumors 6502a
Dec 27, 2008
706
1
London
Just for the sake of the thread, what mac do you have? and how much RAM? Cos thems are the vital bits of info..

Anyways, have you tried looking in application preferences see if theres any way to assign more memory to certain processes? etc etc

The solution might be to get more RAM? but wouldnt know for certain until you tell us how much RAM you actually have in the first place lol.. :)

PTP
 

MisterMe

macrumors G4
Jul 17, 2002
10,650
28
USA
Did a cursory search of the forums and Google and I can't find the answer.

I'm running a 64 bit Matlab simulation on an enormous dataset and OS X (not Matlab) is telling me I'm running out of space on my startup disk for "application memory". I've got 300GB of free space on the hard drive, and only 60GB used as swap. I think the limit I'm hitting is the 64th swap file.

Anybody know how to grow that? I've tried booting to 64bit kernel, but no difference.
It sounds like you need to redesign your simulation. I tend to be suspicious of a simulation based on a large set of discrete data. My guess is that your problem is not a computer problem, but rather your attempt to substitute computer power for an understanding of your system under study. Once you understand your system, you will find that you will suddenly have a lot fewer application memory shortages.
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
Just for the sake of the thread, what mac do you have? and how much RAM? Cos thems are the vital bits of info..

Anyways, have you tried looking in application preferences see if theres any way to assign more memory to certain processes? etc etc

The solution might be to get more RAM? but wouldnt know for certain until you tell us how much RAM you actually have in the first place lol.. :)

PTP
Mac Pro, 2x4x2.66GHz
16 GB RAM installed

The problem seems to be coming from OS X, not any of the specific applications-- I get a window suggesting I start force quitting applications, so I don't think any application preferences would help (and I haven't found anything when I looked).

More RAM starts to get pricy. I've got enough in the machine for most stuff, and the sim only runs half an hour so the swap time isn't killing me. OS X just refuses to use more disk...

It sounds like you need to redesign your simulation. I tend to be suspicious of a simulation based on a large set of discrete data. My guess is that your problem is not a computer problem, but rather your attempt to substitute computer power for an understanding of your system under study. Once you understand your system, you will find that you will suddenly have a lot fewer application memory shortages.
I understand my subject quite well, but that's not really the point. Even if I was simply curious, I'd like to know how to increase the swap space. I only mention Matlab in case there's a known issue with that set of technologies (Matlab, X11, Java, etc).
 

MisterMe

macrumors G4
Jul 17, 2002
10,650
28
USA
Mac Pro, 2x4x2.66GHz
16 GB RAM installed

The problem seems to be coming from OS X, not any of the specific applications-- I get a window suggesting I start force quitting applications, so I don't think any application preferences would help (and I haven't found anything when I looked).

More RAM starts to get pricy. I've got enough in the machine for most stuff, and the sim only runs half an hour so the swap time isn't killing me. OS X just refuses to use more disk...


I understand my subject quite well, but that's not really the point. ...
I severely doubt that. You have have enormous resources at your disposal. You don't lack for robustness in your OS. You don't lack for RAM. You don't lack for swap file capacity.

My suggestion to you is to seek help with a better approach to your attempt to solve your problem.
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
In the hopes of simplifying the question, let me phrase it this way: "I am running a program that is allocating large chunks of memory and OS X is restricting the program to about 60GB of swap space while leaving 300GB of hard drive space unused. That's not what I expected. I suspect there is a limit set in the OS, either as a fraction of available storage, or as a multiple of available RAM, or as a maximum number of individual 1GB swap files. Does anyone know if this is true and if it can be modified?"



Now, to the peanut gallery:
I severely doubt that. You have have enormous resources at your disposal. You don't lack for robustness in your OS. You don't lack for RAM. You don't lack for swap file capacity.

My suggestion to you is to seek help with a better approach to your attempt to solve your problem.
So, of all the potential problems, you've decided it's merely a matter of my incompetence?

Kind of makes me think of a wise comment I've seen elsewhere:
My suggestion to you is that before you volunteer your opinion about the intelligence of people that you don't know and have never heard from, think. Can you do that?
While I'm impressed by the level of confidence required to question the professional judgement of someone you don't know in approaching work you know nothing about and using a tool as widely applicable as Matlab, I think I'll boil the discussion so far down to "MisterMe can't help me with my virtual memory problem", thank you for your time and for keeping this thread active, and hope that someone out there will focus on the question at hand.
 

blomma

macrumors member
Apr 13, 2010
73
0
Did a cursory search of the forums and Google and I can't find the answer.

I'm running a 64 bit Matlab simulation on an enormous dataset and OS X (not Matlab) is telling me I'm running out of space on my startup disk for "application memory". I've got 300GB of free space on the hard drive, and only 60GB used as swap. I think the limit I'm hitting is the 64th swap file.

Anybody know how to grow that? I've tried booting to 64bit kernel, but no difference.
Nope, by default virtual memory is unlimited. It might be that you are hitting an addressable ram limit or perhaps some kind of process limit.

http://www.appleinsider.com/articles/08/08/26/road_to_mac_os_x_10_6_snow_leopard_64_bits.html

http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/brh72ex-49.html

You might have better luck posting on matlab specifik forums about this, seems like the thing other people there might have run into.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
interesting topic, one that grabs my attention ;)

you cannot change the swap size in OSX. it is automatically chosen as needed.

you have clearly exceeded the free space on your drive by running these massive simulations, and really the ONLY way i can see you fixing it is by getting a larger startup drive - unless of course you can rewrite how OSX handles page files and whatnot ;)
 

tkingart

macrumors 6502
Apr 1, 2010
278
0
West Coast
Not sure if this is helpful or not. But from what I understand, OS X automatically de-fragments files 20mb and smaller. Anything larger than that, such as in your case- may be fragmented all over the place with many gaps in-between sectors.
 

Rayday

macrumors member
Aug 13, 2009
31
0
Michigan
While I'm impressed by the level of confidence required to question the professional judgement of someone you don't know in approaching work you know nothing about and using a tool as widely applicable as Matlab, I think I'll boil the discussion so far down to "MisterMe can't help me with my virtual memory problem", thank you for your time and for keeping this thread active, and hope that someone out there will focus on the question at hand.
Well said. While I don't have MatLab installed, your rebuttal was very well spoken and focused on the subject at hand.

Kudos.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
Not sure if this is helpful or not. But from what I understand, OS X automatically de-fragments files 20mb and smaller. Anything larger than that, such as in your case- may be fragmented all over the place with many gaps in-between sectors.
correct. could be a likely scenario i guess.
 

jpyc7

macrumors 6502
Mar 8, 2009
276
0
Denver, CO
Have you run the tools (from the Terminal) like top, vm_stat or vmmap?

They can tell you how big a VM region is. I don't know that the swap file has to be in unfragmented disk sectors, as suggested earlier. I haven't heard of multiple swap files either.

Total speculation on my part, but maybe the OS message is not correlated with the underlying condition. It says you have exceeded the disk space, but maybe it reports that when you have "excessive" paging. The tool like vm_stat or top can give you some indication of that sort of activity.
 

Ganesha

macrumors regular
Aug 4, 2009
112
1
You can easily test some of your hypotheses by customizing /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist and using the -S option to let you set the size of each paging file. Keep in mind OS X needs a block of continuous disk space for each swap file.

Also are you sure MatLab isn't somehow triggering the error by asking for some ridiculous amount of memory (i.e. copy on write) which the OS refuses to give.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
You can easily test some of your hypotheses by customizing /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist and using the -S option to let you set the size of each paging file. Keep in mind OS X needs a block of continuous disk space for each swap file.

Also are you sure MatLab isn't somehow triggering the error by asking for some ridiculous amount of memory (i.e. copy on write) which the OS refuses to give.
interesting! tell me more! :D
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
Wow, thanks for the sudden rush of information! I'll try a bunch of this tomorrow and report back.

Nope, by default virtual memory is unlimited. It might be that you are hitting an addressable ram limit or perhaps some kind of process limit.

You might have better luck posting on matlab specifik forums about this, seems like the thing other people there might have run into.
The error box I get is an OS X error, not a Matlab "out of memory" error, so something is triggering OS X to warn and prompt me to start force quitting tasks. It's a very similar warning box to the "your startup disk is almost full" error.
interesting topic, one that grabs my attention ;)

you cannot change the swap size in OSX. it is automatically chosen as needed.

you have clearly exceeded the free space on your drive by running these massive simulations, and really the ONLY way i can see you fixing it is by getting a larger startup drive - unless of course you can rewrite how OSX handles page files and whatnot ;)
I've still got 300GB free-- it seems that I'm a long way away from being full unless...
Not sure if this is helpful or not. But from what I understand, OS X automatically de-fragments files 20mb and smaller. Anything larger than that, such as in your case- may be fragmented all over the place with many gaps in-between sectors.
This could very well be the case. Each swap file after the first 4 or so, are about 1GB in size. If there weren't 1GB of contiguous disk, would that cause this problem? I wouldn't expect finding a free GB out of 300 would be hard, but doing it the 65th time might just be too much.

I used to do that ritualistically in the Windows world but haven't thought about it in over a decade now... Anybody have a favorite defragmenter?
Have you run the tools (from the Terminal) like top, vm_stat or vmmap?

They can tell you how big a VM region is. I don't know that the swap file has to be in unfragmented disk sectors, as suggested earlier. I haven't heard of multiple swap files either.

Total speculation on my part, but maybe the OS message is not correlated with the underlying condition. It says you have exceeded the disk space, but maybe it reports that when you have "excessive" paging. The tool like vm_stat or top can give you some indication of that sort of activity.
Thank you for both these pointers-- top I'm aware of, but the two vm specific tools I hadn't seen before. I'll see what I can learn when I'm back in the office tomorrow. vmmap looks like more than I'll be able to digest, but vm_stat looks manageable.

I've thought about the possibility that OS X is misdiagnosing the problem, or is just getting concerned by the rate of consumption rather than the total amount and trying to give me time to respond before it's too late. "At this rate, you've got 10 minutes to do something before the world ends" kind of thing...

It is doing massive paging. 1k+ pages continuously on MenuMeters for minutes.

The argument against this theory though is that once it stops consuming memory, and even after having killed non-Matlab processes to appease the gods, OS X won't stop posting the warning until I free memory within Matlab (or restart it).
You can easily test some of your hypotheses by customizing /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist and using the -S option to let you set the size of each paging file. Keep in mind OS X needs a block of continuous disk space for each swap file.

Also are you sure MatLab isn't somehow triggering the error by asking for some ridiculous amount of memory (i.e. copy on write) which the OS refuses to give.
There's the ticket! I'll give this one a shot tomorrow too, hopefully if I set the paging files to 2GB, everything will get happy. If fragmentation is the culprit, I'd expect it to fall over sooner with a 2GB swap file.

I don't think it's a major allocation request that trips the alarm-- I can set simulation parameters such that it will run, but OS X won't allow me to dismiss the force-quit dialog. I think that means that I'm right up against the line at that point-- OS X fulfilled the memory request but doesn't think it can fulfill the next one.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
I've still got 300GB free-- it seems that I'm a long way away from being full unless...
well im using 265GB of virtual memory currently! that is a cumulative total, but you can see that its a large amount.

you should be able to use roughly 24GB swap files - what is it at currently?

i think the defrag issue is the most likely problem, you should try to defrag and see how that goes!
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
Also are you sure MatLab isn't somehow triggering the error by asking for some ridiculous amount of memory (i.e. copy on write) which the OS refuses to give.
I should also mention that the "wired" memory is in the 1GB range, well short of the 16GB of available RAM.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
I should also mention that the "wired" memory is in the 1GB range, well short of the 16GB of available RAM.
OH! that kinda of changes things then!

the program must be only made in such a way that it can address ~1GB of RAM. older versions of PS were around 3GB.. so its possible for this.

are there any preferences you can change?
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
well im using 265GB of virtual memory currently! that is a cumulative total, but you can see that its a large amount.

you should be able to use roughly 24GB swap files - what is it at currently?

i think the defrag issue is the most likely problem, you should try to defrag and see how that goes!
If you 'ls -l /private/var/vm', you're seeing swap files as big as 24GB? My system was up to 63 or 64 swap files for a grand total of 60GB of virtual memory. Looking at the files, it looked like they started small, then ramped up to 1GB, but never got much bigger than that.

I agree, it sounds like I need to defrag.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
If you 'ls -l /private/var/vm', you're seeing swap files as big as 24GB? My system was up to 63 or 64 swap files for a grand total of 60GB of virtual memory. Looking at the files, it looked like they started small, then ramped up to 1GB, but never got much bigger than that.

I agree, it sounds like I need to defrag.
nah im at about 5.6GB swap used. its kinda of proportionat to your physical memory. swap file can go to roughly 1.5x your RAM. sometimes it will go larger in extreme cases (like yours).

each file can go to 1GB i believe.

can you do this for me, go into Activity Monitor, then the "memory" tab - and screen shot the info from there?
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
OH! that kinda of changes things then!

the program must be only made in such a way that it can address ~1GB of RAM. older versions of PS were around 3GB.. so its possible for this.

are there any preferences you can change?
I thought the relatively small amount of wired memory (system wide) was just proof that I wasn't being limited by available RAM. Everything not wired should be swappable.

The 3GB limit, I think, was a 32bit issue. Out of the 4 gig of address space, 1 gig was reserved for system and 3 gig went to the application. This is a 64 bit application, so that shouldn't be the problem here.
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
nah im at about 5.6GB swap used. its kinda of proportionat to your physical memory. swap file can go to roughly 1.5x your RAM. sometimes it will go larger in extreme cases (like yours).

each file can go to 1GB i believe.

can you do this for me, go into Activity Monitor, then the "memory" tab - and screen shot the info from there?
I don't have access to that system right now-- I'm a few time zones ahead of you... :D

I'll try to capture a shot tomorrow.
 

DoFoT9

macrumors P6
Jun 11, 2007
17,494
26
Singapore
I thought the relatively small amount of wired memory (system wide) was just proof that I wasn't being limited by available RAM. Everything not wired should be swappable.

The 3GB limit, I think, was a 32bit issue. Out of the 4 gig of address space, 1 gig was reserved for system and 3 gig went to the application. This is a 64 bit application, so that shouldn't be the problem here.
hmm i didnt think it was a limit on the "bitness" of the system though, more of what they actually chose the limit to be? im not sure though :(


I don't have access to that system right now-- I'm a few time zones ahead of you... :D

I'll try to capture a shot tomorrow.
LOL. you live in new zealand? i live in australia, you have to be in NZ to be ahead of me ;)

post screenie when u can!
 

Analog Kid

macrumors 601
Original poster
Mar 4, 2003
4,771
2,793
LOL. you live in new zealand? i live in australia, you have to be in NZ to be ahead of me ;)

post screenie when u can!
Sorry, should have said "a few time zones ahead of you, but yesterday..." :D

I'm in California.