macOS Shell Scripting or AWK?

mar2194 · Dec 8, 2012

Hey everyone,

I'm currently running a program which accepts an input file. The input file has various parameters, 3 of which I need to change, run again, etc.
The range of those 3 parameters are 15 values for one, 12 values for another and 10 for another, so I have 15*12*10 (=1800) possible input files I need to make for the program. The output filename will need to match the parameters for that run as well. Obviously creating 3375 input files by hand would be impractical, not only because it would be time consuming but the risk of human error would be great. I need help with the following:

1. Varying the parameters of my input file. I know this can be done with AWK, or perhaps using shell script. The input file is plain text (ascii)

2. recursively running the script each time a parameter is changed. Suppose my input file looks like:

#input file mockup
#Value for A
1
#Value for B
2
#Value for C
4
#Value for D
12

Suppose the three parameters I want to change are A, B and D. Suppose each value of A is spaced 5 units apart, and I want to run over 3 values. My possible values of A would then be {1, 6, 11}
My possible values for B are spaced 3 units apart and there are 4 values {2, 5, 8, 11}
and my Values for D are spaced 1 apart and there are two of them {12, 13}
Allowing for repeated values, the sets I would want process are:

{1, 2, 12}
{1, 5, 12}
{1, 8, 12}
{1, 11, 12}

{1, 2, 13}
{1, 5, 13}
{1, 8, 13}
{1, 11, 13}

{6, 2, 12}
{6, 5, 12}
{6, 8, 12}
{6, 11, 12}

{6, 2, 13}
{6, 5, 13}
{6, 8, 13}
{6, 11, 13}

{11, 2, 12}
{11, 5, 12}
{11, 8, 12}
{11, 11, 12}

{11, 2, 13}
{11, 5, 13}
{11, 8, 13}
{11, 11, 13}

where each place in the brackets corresponds to the values for {A, B, D}

So my two problems are essentially:
1 Make a wrapper to run recursively through the program
2 Change the three variables so I get every combination and pass the corresponding input file to the program.

here's what I think it should look like in pseudo-code (I know this is wrong need some help):

A = 1
B = 2
D = 12

for B<11,
for A<11,
for D<13,
B+3, A+5, D++, run program

dmi · Dec 9, 2012

echo {1,6,11},{2,5,8,11},{12,13} | xargs -n 1 program

#or, to create a set of input files

for ABD in {1,6,11}.{2,5,8,11}.{12,13} ; do echo $ABD | tr '.' '\n' > file.$ABD ; done

mar2194 · Dec 9, 2012

dmi said:
echo {1,6,11},{2,5,8,11},{12,13} | xargs -n 1 program

#or, to create a set of input files

for ABD in {1,6,11}.{2,5,8,11}.{12,13} ; do echo $ABD | tr '.' '\n' > file.$ABD ; done

Will the second option create 24 input files with the correct variations in parameters?

Here is the input file (run1.input):

Code:

# grid size (sx1, sx2, sx3)
256 256 256
# sampling size, smoothing & nyquist (dx1,dx2,dx3,beta,nq)
0.5 0.5 0.5 0.5 2
# origin position & rotation
0 0 0
# geographic origin (longitude and latitude), UTM zone and real length unit (usually m or km)
# displacements and stress are converted to lon/lat geographic coordinates
# unit corresponds to a scaling from dx1,dx2,dx3 to real unit
# use unit = 1   if dimensions are described in units of m
# use unit = 1e3 if dimensions are described in units of km
120 22 51 1e3
# observation depth (displacements and stress) (stress is only exported in GRD)
0 5
# output directory (all output files are stored in the following directory)
output1
# the elastic parameters (Lambda and mu) are the Lame parameter and the shear modulus
# gamma=(1-nu)*rho*g/mu is a wavelength relevant to buoyancy.
# elastic parameters and gamma = (1-nu) rho g / mu = 8.33e-7 /m = 8.33e-4 /km
4.26E+04 2.6E+04 8.48e-4
# integration time, time step and scaling 
0 -1 1
# number of viscous observation slice 
0 
# number of observation points 
0 
# number of Coulomb patches
0
# number of prestress interfaces 
0 
# number of linear viscous interfaces 
0 
# number of power-law viscous interfaces 
0 
# number of friction faults 
0
# number of interseismic loading strike-slip and opening
0
0
# number of coseismic events (when slip distribution is prescribed) 
1
# number of shear dislocations (strike-slip and dip-slip faults) 
1
# no   slip xs ys zs length  width  strike dip rake
  1	2   0  0  40   20    10	    0      60	0
# number of tensile cracks 
0 
# number of dilatation sources 
0 
# number of surface traction 
0

Here is the shell-script I currently use to run over that input file using the program(run1.sh):

Code:

#!/bin/sh
WDIR=./output1

if [ ! -e $WDIR ]; then
	echo adding directory $WDIR
	mkdir $WDIR
fi

OMP_NUM_THREADS=4 relax $* < run1.input | tee $WDIR/in.param

As you can see, it copies the output of the program to an in.param file in the directory output1 after running the program relax on 4 threads using run1.input

I need to change 3 parameters from my input file and create a new directory for every run of new variable combination.

chown33 · Dec 9, 2012

mar2194 said:
Will the second option create 24 input files with the correct variations in parameters?

Try it. See what happens.

I need to change 3 parameters from my input file and create a new directory for every run of new variable combination.

Please identify which parameters you want replaced by variables. For example, repost run1.input, and hilite the parameter values that should be variables using bold or a color.

Search

Search

macOS Shell Scripting or AWK?

mar2194

macrumors member

dmi

macrumors regular

mar2194

macrumors member

chown33

Moderator

Our Staff