macOS Matlab Jobs in Background

mjoshac · Oct 10, 2009

Anyone familiar with this?

http://www.zaphu.com/2007/10/19/run-matlab-jobs-in-the-background-on-a-mac-or-linux-server/

I would like to set up test runs and view results after a nite of running

Thank you

chown33 · Oct 10, 2009

Do you have a question? Are you saying that you can't understand the article? That you tried what it says but it didn't work? That you want something similar to article's description but not exactly?

Please clarify exactly what you're seeking that the article doesn't provide.

If you tried what the article says and it didn't work, you'll have to describe exactly what happened, including copying and pasting any error messages. If you didn't get any error messages, then you should post that fact.

I read the article. It seems pretty plain to me: make some shell scripts, run them with nohup. Since I don't have matlab, I can't tell you if they work as given, but the principles and the specific commands to use seem sound.

mjoshac · Oct 10, 2009

My apologies I have very little experience in this area

As shown in the link:

" Now you will need to set the permissions on the file to allow execution by entering the following at the prompt.

chmod u+rwx runJob "

I am not sure where to run this code?

Thank you for the help

chown33 · Oct 10, 2009

You run that command in Terminal.app.

Given the article's brevity and your inexperience, I will try to provide more specific steps.

1. Using TextEdit, open a new document. Make it plain text.

2. Paste the given script into that document, replacing YOUR_MATLAB_FILE with the full pathname of your matlab file (see discussion of "full pathname" on Wikipedia).

http://en.wikipedia.org/wiki/Path_(computing)

Note: if you drag-n-drop the matlab file from Finder onto the plain text document, its full pathname will appear as text in the document. It should look something like /Users/yourname/wherever/it/is/blahblah.m.

3. Save the document under the name "runJob", in your Home directory (NOT your Home/Documents directory). Ensure it does not have a ".txt" or any other extension.

4. Start Terminal.app. It's located in /Applications/Utilities. When it starts, its working directory will be your home folder. (see discussion of "working directory" on Wikipedia).

http://en.wikipedia.org/wiki/Working_directory

5. SIDEBAR: The concept of pathname is important because it's how you identify a specific file unambiguously. Every file has a unique pathname. The concept of working directory is important because it's how you navigate directories without having to always give the full pathname.

6. SIDEBAR #2: Some important navigational commands:

pwd
ls -l
cd pathname

pwd is "print working directory". This is the "what is my working directory" or "Where am I?" command.

ls -l means "list the working directory's contents in long format". Long format is essential to see what attributes or permissions each item has.

cd pathname means "change the working directory" to the given pathname. If no pathname is given, it will change to your home directory.

7. Issue the chmod command as given in the article. If it works, no confirmation will appear. If it doesn't work, an error message will appear. Copy and paste that error message, along with the output from 'ls -l' and we'll try to make sense of it.

Another option is to find someone who knows more about command-lines than you, who can write a complete self-contained AppleScript or bundled shell-script for you. It should be fairly easy to do for a specific matlab file, i.e. always the same file. It's somewhat harder to do for an arbitrary matlab file.

Another possibility might be a folder action, so simply copying a matlab file into a designated folder starts it as a background job. That's harder, but still in the range of a weekend project for someone with moderate skills and a free weekend.

mjoshac · Oct 11, 2009

I have no errors and I had my computer email me when complete

the email gives the path of the results but there is nothing there

I placed in that path (home directory) output.txt (since there is no command that creates that file?)

Tried the following:
1. made sure Matlab did not need to be opened while taking place
2. place the output.txt in the same directory as the matlab file

Do you have any insight, not sure where the results would be?

Thank you again for the nice and easy to follow description

chown33 · Oct 11, 2009

The results should be in the file "output.txt", located in the directory where your "runJob" was run.

There isn't a distinct command to create the output file. This command:

Code:

nohup matlab -nodisplay < YOUR_MATLAB_FILE.m > output.txt &

contains an "output redirection" that will create an output.txt file (or overwrite one that exists).

http://en.wikipedia.org/wiki/Redirection_(computing)

The problem is that the "output.txt" filename is a relative path (see earlier Wikipediate entry for Pathname, and find 'relative'). As a result, it will be created wherever the working directory is at the time the command-line is executed. If you don't have an extremely clear sense of where that is, then it could end up somewhere you didn't intend.

I'm going to suggest a slightly different command in your 'runJob' file:

Code:

nohup matlab -nodisplay < YOUR_MATLAB_FILE.m >~/Documents/Matlab-Output.txt &

The "~/Documents" represents the Documents folder of your home folder. The "Matlab-Output.txt" will be a file therein.

The output file should be created only a brief moment after you execute 'runJob'. It will initially be empty (zero bytes on disk). It will only receive data after matlab completes, or possibly as blocks of output data are produced.

If it still doesn't work, post again.

mjoshac · Oct 11, 2009

I made changes you suggested:

This is my runJob.txt file:

Code:

#! /bin/csh

nohup matlab -nodisplay < /Users/josh/Documents/MATLAB/Class/559/p2/BreakDown/10bit/START.m> /Users/josh/output.txt &

not sure but the output.txt was not created by my execution of:

Code:

chmod u+rwx runJob

I added the suggested code for an email to the matlab file and I do not get a confirmation email

should this be put in my runJob.txt: #! ?

therefore I am assuming it has not opened the file

mjoshac · Oct 11, 2009

are there any rules on format in the runJob.txt ?

Thank you again

mjoshac · Oct 11, 2009

sorry to reply I was wondering if this helps

I type matlab into the terminal and I get this error:

matlab: No MATLAB bin directory for this machine architecture.

ARCH = maci6

Is this relative to the issue?

Code:

cd /usr/local/bin
sudo ln -s /Applications/MATLAB6p5/bin/matlab .
rehash

to add matlab to your shell path

"I do not know what I am talking about here just did some google'n to see if I can help the problem"

Take care and thanks for the help

Ramashalanka · Oct 12, 2009

After reading your whole thread, I think you would find life much easier just running your matlab script using the normal graphical interface. Leave it on overnight, and see the results in the morning.

Anyway, answering your specific questions, and some comments:

mjoshac said:
This is my runJob.txt file:

It is normal (and the article you reference assumes) that files like runJob do not have a .txt extension. You could either rename the file in Finder, or you could use the .txt extension below.

mjoshac said:

That doesn't look too bad, but you need to be in the matlab directory (or have it in your path - you allude to this below) in order for the script to work. I prefer to put the full path of the matlab executable in the script, for example (replace the word "matlab" above with something like this: look in the applications folder in finder to find the matlab name)

Code:

/Applications/MATLAB_R2008a/bin/matlab

or for you, perhaps:

Code:

/Applications/MATLAB6p5/bin/matlab

then you can put both your csh script and matlab script in the same directory or at least not so far away on the path. If your START.m calls any of your functions, you may need to put a

Code:

addpath /Users/josh/Documents/MATLAB/Class/559/p2/BreakDown/10bit/

at the start of START.m.

mjoshac said:
not sure but the output.txt was not created by my execution of:

Code:

chmod u+rwx runJob

Didn't you get an error when you did this (the error would start with "usage ...")? It shouldn't have found the file, since yours seems to be called runJob.txt, not runJob. You need to try it with runJob.txt.

Anyway, all (for your purposes) that chmod does is make it executable (i.e. change the file type to a file that is allowed to run commands), so that you can run it, it doesn't actually run it. After the chmod, you need to type

Code:

./runJob.txt

to run it.

mjoshac said:
I added the suggested code for an email to the matlab file and I do not get a confirmation email

should this be put in my runJob.txt: #! ?

therefore I am assuming it has not opened the file

I don't think that matlab ever got run, let alone STARTUP.m. The suggested code should go in STARTUP.m not runJob.txt, since unless you have a quit in STARTUP.m the csh script will never get past calling matlab.

I wouldn't worry about the email thing until you get output.txt working. And before trying to put it in STARTUP.m, I'd test it in terminal: try typing:

Code:

mail -s abc yourname@youraddress

then some text, then Ctrl-D at the start of a line so see if mail is set up. If not, you have some more work to do on that before trying to put in it an matlab script.

mjoshac said:
are there any rules on format in the runJob.txt ?

Yes, it has to be a plain text file, and it has to contain commands, such as matlab, chmod etc, and many more. Since you start the file (on Mathworks suggestion) with csh, commands need to be in the c shell format (you probably won't have to worry about this if you're just using the normal things you can type in the terminal). I usually use bash (as I think most mac users do, since it's the default) instead of csh. But I wouldn't worry about any of that until you can get matlab running from the script.

mjoshac said:
I get this error:

matlab: No MATLAB bin directory for this machine architecture.

ARCH = maci6

Click to expand...

Is this relative to the issue?

Could be. Can you get matlab to run with the usual graphical user interface on your system? Are you sure the error says "ARCH = maci6" and not maci64? The error means that the version you are trying to run (6.5) doesn't have an intel mac (64bit if it is maci64) executable. Version 7.4 was the first official Intel Mac version. You may still be able to run it in rosetta, just don't worry about sudo ln. If your script doesn't work, you could start by adding the option -nojvm to the matlab call in your runJob script. I can't check this because I only have intel mac versions of matlab. Anyone?

mjoshac said:
Code:

cd /usr/local/bin sudo ln -s /Applications/MATLAB6p5/bin/matlab . rehash

to add matlab to your shell path

That doesn't add matlab to your shell path, it creates a link to the executable, like the icons you may have on your desktop. I wouldn't bother with it.

Hope this helps, but i'd say the best idea would be to run matlab graphically to get your work done. In the meantime, play around with commands in terminal to get used to the paths etc first. Start with pwd, cd, chmod, ls. Use "man pwd" etc to get help on what they do.

I hope you don't find any of the above patronizing, you may be a computer whizz, these are just the things that I thought may help based on your queries.

balamw · Oct 12, 2009

Ramashalanka said:
Hope this helps, but i'd say the best idea would be to run matlab graphically to get your work done. In the meantime, play around with commands in terminal to get used to the paths etc first. Start with pwd, cd, chmod, ls. Use "man pwd" etc to get help on what they do.

I agree 100% with this. The article seems to have been written for use on remote unix machines and really is only of any real use when you have a long job that ONLY displays its results to the MATLAB window.

I would hope that anyone who is using MATLAB for long jobs like that would learn about writing to files instead of to the display. It's really easy to save your workspace in a file rather than printing results to the screen and redirecting that to a file. (doc save or xlswrite or ...)

You should definitely read up on the various options for starting up matlab in the online help in "matlab (unix)" in "Startup and Shutdown" if you want to use command line options like -nodisplay.

Ramashalanka said:
That doesn't add matlab to your shell path, it creates a link to the executable, like the icons you may have on your desktop. I wouldn't bother with it.

A minor quibble, but in this case what it is doing is placing a symbolic link to the matlab executable to a folder that is in the path. Both types of link in unix are far more flexible and useful than the GUI shortcuts you are comparing them with. In this case, a symbolic link to the matlab executable in /usr/local/bin (or some other folder already in the path) is a perfectly fine way to get it in the path and is in fact the way matlab does it itself at least in 2009b. A far better way than polluting the path with all kinds of folders as is the only way to get things done in Windows land.

B

Ramashalanka · Oct 12, 2009

balamw said:
A minor quibble, but in this case what it is doing is placing a symbolic link to the matlab executable to a folder that is in the path. Both types of link in unix are far more flexible and useful than the GUI shortcuts you are comparing them with. In this case, a symbolic link to the matlab executable in /usr/local/bin (or some other folder already in the path) is a perfectly fine way to get it in the path and is in fact the way matlab does it itself at least in 2009b. A far better way than polluting the path with all kinds of folders as is the only way to get things done in Windows land.

Fair enough, and thanks for your input. I agree my analogy was overly simplistic. Personally I prefer to use the full path to call matlab in my scripts since on remote machines I don't usually have sudo access to be able to alter /usr/bin etc. I could make a directory just for putting the links in, but the full path feels like the easiest way. Also I can see directly in the script which version of matlab I'm using on someone else's machine.

mjoshac · Oct 13, 2009

If this works I will be able to run m-files without having matlab open? I could still use my save feature and later come back to view the plots?

This would be beneficial to me since my run times are 30 mins and I would like to run a few m-files.

I got the output.txt working and it displays the following in output.txt:

matlab: No MATLAB bin directory for this machine architecture.

ARCH = maci64

and the following on the command line:

Code:

 [1] 1045

I have Matlab 2009b running on snow leopard with the macbook pro 5,5

I am assuming this would not be needed then:

Code:

-nojvm

I tried using:

Code:

 /Applications/MATLAB_R2009b/bin/matlab

instead of

Code:

 matlab

and error was "Command not found"

mjoshac · Oct 13, 2009

correction*

and the following on the command line

should read

and the following on the terminal

Thanks again

Ramashalanka · Oct 13, 2009

mjoshac said:
If this works I will be able to run m-files without having matlab open? I could still use my save feature and later come back to view the plots?

This would be beneficial to me since my run times are 30 mins and I would like to run a few m-files.

You can run m-files without having the graphical interface open, or even in the background with the "&" at the end of the line. You won't be able to go back to look at the plots with output redirected (" > output.txt"), but if you put some commands to save the plot in your m file you can look at them later (type "help save" at the matlab prompt, this may be what you were referring to). I would tend to save the data rather than the plots, and plot them when they are all finished, but either is OK.

I reiterate, though, that isn't the way to go for what you are doing, judging from the kinds of questions you are asking. What I would do for your problem is to do it all within the graphical interface. Make an m-file that calls all of the m-files you want to run. With each new plot use the command "figure" (to a line by itself), don't use any "close" commands. Then when you come back the next day, or whatever, and all of the jobs have been run, you will have all of the plots there to look at. The approach you are trying is more difficult, because you can't see what's going on. If you have any deadlines for your work/study, I would avoid the terminal approach. However, if you want to waste some time seeing how it would be done, read on. I run matlab with no display via ssh or when I need multiple copies for the multicore packages. I wouldn't use terminal myself for what you are doing.

mjoshac said:
I got the output.txt working and it displays the following in output.txt:
the output.txt working and it displays the following in output.txt:

matlab: No MATLAB bin directory for this machine architecture.

ARCH = maci64

Click to expand...

and the following on the command line:

Code:

[1] 1045

I have Matlab 2009b running on snow leopard with the macbook pro 5,5

I am assuming this would not be needed then:

Code:

-nojvm

Right, so it was maci64, not maci6. The 1045 is the process id (pid). You can see a list of them by typing "ps". You need to change your call to matlab in runJob to have -maci64 as an option (near where you have -nodisplay in the command line). I still use -nojvm as well, but you'll want to avoid this if you want to plot (I just generate data which I plot later).

mjoshac said:
I tried using:

Code:

/Applications/MATLAB_R2009b/bin/matlab

instead of

Code:

matlab

and error was "Command not found"

(got your extra post - it's generally better to edit the original post unless someone's already replied).

Well, you need to find matlab on your computer. Either look in Applications in the finder, or type "/App" in terminal and press tab (it should become "/Applications/" then type "MAT" and press tab (if it doesn't do anything press tab twice and you should get some options, if that doesn't do anything change the "MAT" to "mat" and press tab), then type "bi" and press tab, "mat" and press tab. This should give you the full path. You then need to type -maci64. Press return to check that it runs. Then you can play with -nodisplay, -nojvm, running your script, redirecting the output, running in the background with "&" etc.

As I said before, I'd have a go at using the commands: ls, pwd, cd, ps, man before trying to do this.

balamw · Oct 14, 2009

Ramashalanka said:
Then you can play with -nodisplay, -nojvm, running your script, redirecting the output, running in the background with "&" etc.

As I said before, I'd have a go at using the commands: ls, pwd, cd, ps, man before trying to do this.

I concur, you should really try to understand the command shell and redirection with simple commands before you start messing with matlab. You also obviously have not read the online help about the startup options or you'd have a better shot at understanding the -maci -nojvm etc...

Take a step back and understand the shell. Just like leaning MATLAB it may not make the most sense at first but once you get familiar with it it will become very powerful.

As they say there are many ways to skin a cat, you need to find the workflow that makes the most sense to you and works for your type of problem.

Another useful command to try is "which" as it seems like you have the wrong architecture of matlab in your path. Try "which matlab" and it'll tell you the full path of the one it is trying to run.

B

mjoshac · Oct 15, 2009

It would be beneficial to me because I am not to good at planning ; )

so to be able to run one big project in background and then open Matlab and do some other homework would help. Which would allow me to look at results in my break time from homework and then put another run through

I am in the following path:

/Applications/MATLAB_R2009b.app/bin

but I can not open the next file you asked entitled: matlab

I type into the terminal: open matlab

and I get the following error

matlab: No MATLAB bin directory for this machine architecture.

ARCH = maci64

Does this error have to do with me running Matlab (32 bit) and I am in snow leopard (64 bit)

I also typed into the terminal: open bin

and I see that the matlab file you asking me open and its a unix executable file

I am familiar with all those commands except man ps and looking that up now

mjoshac · Oct 15, 2009

mjoshac · Oct 15, 2009

I modified runJob.txt to be:

Code:

#!/bin/csh

nohup /Applications/MATLAB_R2009b.app

</Users/yorhymes/Documents/MATLAB/Class/559/p2/3bit/START.m> output.txt &

entered into the terminal:

Code:

chmod u+rwx rubJob.txt

./runJob.txt

and I get the following error: Permission denied

balamw · Oct 15, 2009

mjoshac said:
I am in the following path:

/Applications/MATLAB_R2009b.app/bin

but I can not open the next file you asked entitled: matlab

I type into the terminal: open matlab

and I get the following error

You seem to be on a 64 bit capable Mac, but do not seem to have the 64 bit MATLAB installed or so it says. Do you have a folder named /Applications/MATLAB_R2009b.app/bin/maci64 or /Applications/MATLAB_R2009b.app/bin/maci.

Try not to use "open" it just confuses things because you may have mutiple "matlab"s and it may be picking the wrong one. Try ./matlab instead. i.e. the unix executable called matlab in this folder.

B

Ramashalanka · Oct 15, 2009

mjoshac said:
It would be beneficial to me because I am not to good at planning ; )

so to be able to run one big project in background and then open Matlab and do some other homework would help. Which would allow me to look at results in my break time from homework and then put another run through

But you can open matlab multiple times graphically. You could have one with your batch job and one for your other homework. This definately isn't the way to go. Anyway...

mjoshac said:
I am in the following path:

/Applications/MATLAB_R2009b.app/bin

but I can not open the next file you asked entitled: matlab

I type into the terminal: open matlab

and I get the following error

Are you sure the directory has ".app" at the end of it? my version of matlab doesn't use a package. Following from balamw's question, which of the following directories do you have?

Code:

/Applications/MATLAB_R2009b/bin/maci64
/Applications/MATLAB_R2009b/bin/maci
/Applications/MATLAB_R2009b.app/bin/maci64
/Applications/MATLAB_R2009b.app/bin/maci

mjoshac said:
Applications/MATLAB_R2009b.app/bin

I typed into the terminal: open bin

and I see that the matlab file you asking me open and its a unix executable file

are you trying to see if I can even open Matlab from the terminal?

I am familiar with all those commands except man ps and looking that up now

bin is the directory, so you don't want to "open" that, just cd to it.

"are you trying to see if I can even open Matlab from the terminal?" Yes: that is a first step before you start redirecting output.

Depending on where you are, you may need to put a sudo before you chmod, ie "sudo chmod u+rwx rubJob.txt".

chown33 · Oct 15, 2009

mjoshac said:
I modified runJob.txt to be:

Code:

#!/bin/csh nohup /Applications/MATLAB_R2009b.app </Users/yorhymes/Documents/MATLAB/Class/559/p2/3bit/START.m> output.txt &

If those are on separate lines in runJob.txt, then it's wrong. It needs to be on one line.

Also, you can't execute a .app from the command line. A .app is a directory (i.e. a folder) that contains multiple sub-components. Folders aren't executable.

Code:
entered into the terminal:

Code:

chmod u+rwx rubJob.txt ./runJob.txt

and I get the following error: Permission denied

Copy and paste the entire error message. There is often context that's important to a proper interpretation of the error.

I mention this because you've used "rubJob.txt" in the chmod command, and from what I can tell, there is no such file.

I recommend posting the output from the following command:

Code:

ls -ld /Applications/[Mm][Aa][Tt]*

Post the output even if it gives an error message. This will give us a better idea of exactly what you have on your machine.

I mention this because part of the problem seems to be basic navigation and use of pathnames.

balamw · Oct 15, 2009

chown33 said:
I mention this because part of the problem seems to be basic navigation and use of pathnames.

Ramashalanka said:
This definately isn't the way to go. Anyway...

Definitely seems like someone who is more comfortable with a GUI trying to jump in to command line usage without understanding or learning the basics.

Given what he was trying before there could be another matlab somewhere in the path, but it still would be useful to know what is in /Applications

B

mjoshac · Oct 17, 2009

I was able to find this solution with some help

My problem:

I am running snow leopard (64bit) and Matlab (32bit) and some settings need to be set when running this set up.

Here is my runJob.txt:

Code:

#!/bin/sh

ARCH=maci
MATLAB_ARCH=maci
export ARCH
export MATLAB_ARCH
cd /Users/josh/Documents/MATLAB/Class/559/p2/3bit
nohup matlab -nodisplay < START.m > output.txt &

In the terminal I used the chmod command on runJob and then ran runJob by: ./runJob.txt

It stores outputs in the current directory

Thank you all for your patience, sorry for the late reply classes are no out for the weekend so bit more time

Please excuse my simplicity I am very new to this realm : )

balamw · Oct 17, 2009

balamw said:
You seem to be on a 64 bit capable Mac, but do not seem to have the 64 bit MATLAB installed or so it says.

mjoshac said:
I am running snow leopard (64bit) and Matlab (32bit) and some settings need to be set when running this set up.

Yup. That's what I was saying.

mjoshac said:
Please excuse my simplicity I am very new to this realm : )

Glad to hear you have it working the way you want it!

B

macOS Matlab Jobs in Background

macrumors newbie

Moderator

macrumors newbie

Moderator

macrumors newbie

Moderator

macrumors newbie

macrumors newbie

macrumors newbie

macrumors regular

Moderator emeritus

macrumors regular

macrumors newbie

macrumors newbie

macrumors regular

Moderator emeritus

macrumors newbie

macrumors newbie

macrumors newbie

Moderator emeritus

macrumors regular

Moderator

Moderator emeritus

macrumors newbie

Moderator emeritus

Our Staff