macOS Fortran Gnuplot optimization - refresh terminal

[qgfreire]

Hi,

Problem description
I'm doing some parameter estimation, using gfortran on mac 10.6.7.
I would like to see de optimization advances (time vs. experimental vs. model).
For that I make a gnuplot script.

From fortran I call the gnuplot script that uses x11 term (or aqua).

If I call gnuplot -persist, I will have a lot of opening windows that eat all system resources.

If I call without persist and use a pause -xx (> 3 for seeing something) at the end of gnuplot script, my optimization stops for so long that turns impossible to wait for a solution (sometimes there are more than 1000 iterations).

Question

I'm googling for some 24h and didn't found any solution.
I'm a newbie (I'm not a programmer!), I was pro. but in ZX81 and QL from sinclair , lololol!

Something is escaping me, sure there is a sollution.

Can you help me, please?

FGF

Hi all, again,

I'm not pushing you to help me!


In the Fortran program for parameter estimation, my model is a set of ODE's.
I tried http://www.netlib.org/odepack/
but I didn't saw how to pass parameters to the ODE.

If someone know's how or know's another package, please.....

because I'm working now with a subroutine (by me) with Euler method not to ..., you know, made by me!


I love

FGF

 

[balamw]

Post some code. I think mainly the gnuplot script might be the most interesting to see how you are approaching it.

Usually for an optimization problem with a tracking plot, I wouldn't create a new window and plot every time, but just update the changed data.

I seem to recall something about a "reread" command that would get new data from the files.

B

 

[jiminaus]

I don't know gnuplot, and I've just hosed by macports install, so I can't install it right now to test this. But I'm assuming there's still a gnuplot process left over to manage the persisting window. If there is, perhaps you can kill it jsut before you're ready to open a new one.

The code below will terminate all running gnuplot processes.

/usr/bin/killall gnuplot

Perhaps try putting it in just before you invoke gnuplot.

 

[qgfreire]

Post some code. I think mainly the gnuplot script might be the most interesting to see how you are approaching it.

Usually for an optimization problem with a tracking plot, I wouldn't create a new window and plot every time, but just update the changed data.

I seem to recall something about a "reread" command that would get new data from the files.

B

Thanks for your interest and fast answer. I will try to post only the interesting parts:

Fortran side:
.
.
.

open( unit=10, file='resultados')
write( 10, '(5es20.10)') cdi(1), cdi(4), cdi(4),cdi(2:3)
do j=1,NN 
write( 10, '(5es20.10)') osdados(j)%x, osdados(j)%y, osdados(j)%yt,YY(j,:2)
end do
close(10)
call run_gnuplot('ajuste.gp')

.
.
.

!***********************************************************************************
	subroutine run_gnuplot(command_file_name)
!***********************************************************************************
	implicit none
	character (len = 100) command
	character (len = *) command_file_name
	integer status
	integer system
!***********************************************************************************
!  Issue a command to the system that will startup GNUPLOT, using
!  the file we just wrote as input.
!***********************************************************************************
	write (command, *) 'gnuplot ' // trim (command_file_name)		
	status=system(trim(command))	
	if (status.ne.0) then
		print *,'RUN_GNUPLOT - Fatal error!'
		stop
	end if	
	return
!***********************************************************************************
	end subroutine run_gnuplot

Gnuplot side
#ajuste.gp#

set term x11 
set xrange [0:]
set yrange [0:]
set xlabel "Tempo [sec]"
set ylabel "[]"
set grid
set title "Ajuste nao linear"
set key left
plot 'resultados' u 1:2 with points lw 3 title "dados",\
'resultados' using 1:3  with lines title "C",\
'resultados' using 1:4  smooth csp with lines title "A",\
'resultados' using 1:5 smooth csp with lines title "B"
pause 0.01

Sory, I'm portuguese and I use such language frequently

My best regards!
FGF

I don't know gnuplot, and I've just hosed by macports install, so I can't install it right now to test this. But I'm assuming there's still a gnuplot process left over to manage the persisting window. If there is, perhaps you can kill it jsut before you're ready to open a new one.

The code below will terminate all running gnuplot processes.

/usr/bin/killall gnuplot

Perhaps try putting it in just before you invoke gnuplot.

I there,
That's an good idea! I didn't remembered it! Thank you!
There is a drawback, it kills other gnuplots I'm running!
OK, it works, but.... thanks. It must exist a better way! No?

 

[jiminaus]

balamw's idea is far better than mine.

The idea is to run gnuplot just the first time and then modify your gnuplot script as follows.

set term x11 
set xrange [0:]
set yrange [0:]
set xlabel "Tempo [sec]"
set ylabel "[]"
set grid
set title "Ajuste nao linear"
set key left
plot 'resultados' [color=red]volatile[/color] u 1:2 with points lw 3 title "dados",\
'resultados' [color=red]volatile[/color] using 1:3  with lines title "C",\
'resultados' [color=red]volatile[/color] using 1:4  smooth csp with lines title "A",\
'resultados' [color=red]volatile[/color] using 1:5 smooth csp with lines title "B"
pause 0.01
[color=red]refresh[/color]

This should cause the plots to be redrawn using whatever is in resultados at time of the refresh. You probably want to increase the pause so it doesn't refresh too fast.

I don't know if this will work, I'm just working off the gnuplot manual. You might want to try reload instead of refresh. I'm not clear on the differences between them.

However to use this technique you'll need to have your Fortran program build up a separate data file and then move it to resultados when its done, otherwise gnuplot might start reading the file while your Fortram program is part way through writing to it.

ALTERNATIVE

Can you invert the situation? Instead of having your Fortran program running and calling gnuplout, and you run gnuplot and have it call your Fortran program?

The idea is to have this gnuplot script:

set term x11 
set xrange [0:]
set yrange [0:]
set xlabel "Tempo [sec]"
set ylabel "[]"
set grid
set title "Ajuste nao linear"
set key left
[color=blue]system(sprintf("yourprogram %f", arg))[/color]
plot 'resultados' [color=red]volatile[/color] u 1:2 with points lw 3 title "dados",\
'resultados' [color=red]volatile[/color] using 1:3  with lines title "C",\
'resultados' [color=red]volatile[/color] using 1:4  smooth csp with lines title "A",\
'resultados' [color=red]volatile[/color] using 1:5 smooth csp with lines title "B"
[color=blue]# Increment args to next iteration[/color]
[color=red]refresh[/color]

Where the system call in blue calls your Fortran program with arguments to generate a particular iteration into resultados. I don't know if this is feasible for your simulation.

 

[qgfreire]

balamw's idea is far better than mine.

The idea is to run gnuplot just the first time and then modify your gnuplot script as follows.

set term x11 
set xrange [0:]
set yrange [0:]
set xlabel "Tempo [sec]"
set ylabel "[]"
set grid
set title "Ajuste nao linear"
set key left
plot 'resultados' [color=red]volatile[/color] u 1:2 with points lw 3 title "dados",\
'resultados' [color=red]volatile[/color] using 1:3  with lines title "C",\
'resultados' [color=red]volatile[/color] using 1:4  smooth csp with lines title "A",\
'resultados' [color=red]volatile[/color] using 1:5 smooth csp with lines title "B"
pause 0.01
[color=red]refresh[/color]

This should cause the plots to be redrawn using whatever is in resultados at time of the refresh. You probably want to increase the pause so it doesn't refresh too fast.

I don't know if this will work, I'm just working off the gnuplot manual. You might want to try reload instead of refresh. I'm not clear on the differences between them.

However to use this technique you'll need to have your Fortran program build up a separate data file and then move it to resultados when its done, otherwise gnuplot might start reading the file while your Fortram program is part way through writing to it.

Dear jiminaus, This macrumors are great, I never believed that such a fast answers were possible.

I've a little child and now is lunch time here in Portugal, so thank you for your answer, I will try it in some moments and I will post the feed back.

Thank you all (balamw's)

The idea is to run gnuplot just the first time and then modify your gnuplot script as follows.

Code: from jiminaus
set term x11
set xrange [0:]
set yrange [0:]
set xlabel "Tempo [sec]"
set ylabel "[]"
set grid
set title "Ajuste nao linear"
set key left
plot 'resultados' volatile u 1:2 with points lw 3 title "dados",\
'resultados' volatile using 1:3 with lines title "C",\
'resultados' volatile using 1:4 smooth csp with lines title "A",\
'resultados' volatile using 1:5 smooth csp with lines title "B"
pause 0.01
refresh

I tried with reread at the end of gnuplot script, I couldn't implement volatile.
However the conclusion is that the control don't return to fortran routine, so the calculations don't continue. Any sugestions?
Now I will try the second idea. Put gnuplot over control.
If that don't result I will try


I tried the second idea to put gnuplot in control. Has I don't know how to implement refresh I substituted it by reload and that don't work because is always renewing the fortran program, so I need some kind of loop but not from the beginning of gnuplot script. I will google now!


OK!
I did it, I have now a loop to reread only the data plot and not the call to the fortran program.
I can post gnuplot script(s) if someone is interested.
But now I have another problem, is data synchronization, if gnuplot requires the data and fortran didn't completed it, I will receive a error, stoping gnuplot,

plot 'resultados' u 1:2 with points lw 3 title "dados",'resultados' using 1:3  with lines title "C",'resultados'  using 1:4  smooth csp with lines title "A",'resultados' using 1:5 smooth csp with lines title "B"
                                                                                                                          ^
"loop.plt", line 4: Can't calculate splines, need at least 3 points

Conclusion
OK!

I learned a lot. Thank you very much you two. Now that I have the bicycle I will learn how to drive. Thank you again!

I will use for now the killall command ! from jiminaus !

And thanks to macrumors, I will try to collaborate in what I know.

I will be happy if someone have ideas, or want, in parameter estimation on ODE models of chemical reactor kinetics. I don't use sundials because it seems complex, excel because is slow (solver 2011 is much better), kinteticus, because I can't register again (i forgot password) etc.


FGF
Thanks all

balamw's idea is far better than mine.
.
.
.
Where the system call in blue calls your Fortran program with arguments to generate a particular iteration into resultados. I don't know if this is feasible for your simulation.

That will give some work, but after trying your initial solution, I will try to see if I can implement this one. Thank you very much for your support.

FGF

Hello all,

Problem not solved!

When I use the killall gnuplot_x11 just before the new gnuplot command, I don't know what happened, but the -persist directive is not obeyed, before it was and opened lots of windows.


last NEWS
-persist it's working, the process stays but the xterm window disappears!
- but only if I call gnuplot subroutine from a subroutine. If I call gnuplot from main program, it does'nt disappear!





I love but ...


With set term aqua, aqua persists but not like a process possible to kill, lol
I don't know what todo !!!

Zero again!

Best regards.
FGF

 

[balamw]

Instead of having your Fortran program running and calling gnuplout, and you run gnuplot and have it call your Fortran program?

I like this concept a lot. It gets around the problem of having the plot out of sync with the optimization.

I have a feeling that once upon a time I did something along these lines using a lockfile that was generated by the sim when new results were available. Gnuplot called a shell script that waited until the lock file appeared.

I haven't digested the rest of the thread yet, but will take a look and see if I have any more thoughts.

B

 

[qgfreire]

I like this concept a lot. It gets around the problem of having the plot out of sync with the optimization.

I have a feeling that once upon a time I did something along these lines using a lockfile that was generated by the sim when new results were available. Gnuplot called a shell script that waited until the lock file appeared.

I haven't digested the rest of the thread yet, but will take a look and see if I have any more thoughts.

B

Thanks! I will wait

 

[balamw]

EDIT: I removed the code from the previous version as it wasn't working for me and there seems to be a much easier way.

http://www.cfd-online.com/Forums/main/12890-real-time-residual-results-plotting.html

c Fortran-gnuplot interaction for real-time animation using named pipe 
c 
c This is a simple example of how gnuplot can be used to create 
c real-time animation of data produced by running fortran 77 code. 
c 
c I put it in the searchable directory in a hope that it can be useful, 
c but no guarantees of any kind: use it at your own risk. Too many 
c frames per second can hang you system heavily. 
c 
c To understand how it works, you need to know what a named pipe is, and what 
c plot '-' means in gnuplot: read the manuals. 
c 
c And, PLEASE, if you know why a.out > FIFOfile works 
c but just a.out does not 
c even though the gnuplot commands are written to the pipe and not 
c to the standard output, send me an explanation to 
c chernysh@soton.ac.uk (Sergei Chernyshenko) or 
c by other means, see http://www.afm.ses.soton.ac.uk/ -> People. 
c 
c How to use: 
c 
c at the linux command prompt, type mkfifo FIFOfile 
c compile this file and name the executable a.out . 
c in an xterm window type gnuplot < FIFOfile 
c in another window type a.out > FIFOfile 
c enjoy 
c 
c do not use too many frames per second, the computer may hang. 
c 
c second(x) gives the CPU time in seconds. Replace if unavailable. 

nframe=0 
open(1,file='FIFOfile', status='unknown') 
write(1,*) 'set xrange [-10:10]' 
write(1,*) 'set yrange [-2:2]' 
write(1,*)'set title \'Demonstrating animation Fortran-gnuplot' 
1 ,' via named pipe\'' 
close(1) 
1 call second(x) 
c 15 frames per second: 
if(15*x.gt.nframe) then 
nframe=nframe+1 
open(1,file='FIFOfile', status='unknown') 
write(1,*) 'plot sin(x-',x,'),',' \'-\' w lp' 
write(1,*) 0.2 
write(1,*) 0.3 
write(1,*) 0.4 
write(1,*) 0.5 
i=(x/10) 
i=i*10 
write(1,*) (x-i)/10. 
write(1,*) 0.7 
write(1,*) 0.8 
write(1,*) 0.9 
write(1,*) 'e' 
close(1) 
endif 
goto 1 
stop 
end

B

 

[qgfreire]

EDIT: I removed the code from the previous version as it wasn't working for me and there seems to be a much easier way.

http://www.cfd-online.com/Forums/main/12890-real-time-residual-results-plotting.html

c Fortran-gnuplot interaction for real-time animation using named pipe 
c 
c This is a simple example of how gnuplot can be used to create 
c real-time animation of data produced by running fortran 77 code. 
.
.
.

stop 
end

B

After almost 1 year and after achieved very good results I returned to Fortran.
A new mechanism, a new reaction etc.
Your answer is very interesting! I tried it but I obtain this (with your example):

cat FIFOfile
set xrange [-10:10]
set yrange [-2:2]
set title 'Demonstrating animation Fortran-gnuplot via named pipe'

and in the other terminal:

./a.out > FIFOfile
Bus error: 10

So, pipe terminated because is blank? I'm not very good in this.

 

[balamw]

cat FIFOfile

Sounds like you haven't created the named pipe.

http://en.wikipedia.org/wiki/Named_pipe

You meed a mkfifo somewhere.

B

 

[qgfreire]

B[/QUOTE]

Sounds like you haven't created the named pipe.

http://en.wikipedia.org/wiki/Named_pipe

You meed a mkfifo somewhere.

B

Hello!
Still here after all this time??? UaU!

OK!, I've created de pipe. When error occurs, and I make

cat FIFOfile

set xrange [-10:10]
set yrange [-2:2]
set title 'Demonstrating animation Fortran-gnuplot via named pipe'


So the pipe is there, but stopped?

Chears