Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

macOS Undesirably Huge Executable !!! (Fortran)

lordava

lordava

macrumors newbie
Original poster
Aug 9, 2010
7
0
Hello

I only recently learnt to compile my using a makefile and modules. My problem is that I have to use two systems Linux and Mac.

Now my makefile works a charm on the Linux box: compiles fast and the file is only about 800 kb in size.

When I use the same makefile on my 10.6 MacBook. It takes much longer to compile and creates a massive executable almost 600MB in size. This is insane!!! And I don't know why it occurs.

One thing I noticed is that one of the object files created is also about the same size.

The executable runs but 600MB is a problem. Can anybody shed some advice? I would be immensely grateful.

Compiler: gfortran (from HPC Mac website)
libraries: FFTW3

Here is the makefile

*********************************
OBJS = dis.o \
var.o \
evo.o \
grd.o \
ini.o \
str.o \
ino.o \
gsf.o \
fft.o

FC=gfortran

FFLAGS= -O3 -I"/usr/local/include" -L"/usr/local/lib"

DIS: ${OBJS} Makefile
${FC} -o $@ ${OBJS} ${FFLAGS} -lfftw3

var.mod: var.o var.f
${FC} -c var.f
var.o: var.f
${FC} -c var.f

dis.o: var.mod dis.f
${FC} -c dis.f

evo.o: var.mod evo.f
${FC} -c evo.f
grd.o: var.mod grd.f
${FC} -c grd.f
ini.o: var.mod ini.f
${FC} -c ini.f
str.o: var.mod str.f
${FC} -c str.f
ino.o: var.mod ino.f
${FC} -c ino.f
gsf.o: var.mod gsf.f
${FC} -c gsf.f
fft.o: fft.f
${FC} -c fft.f ${FFLAGS} -lfftw3
clean:
rm -f *.o *~ *.mod

cleanall:
rm -f *.o *~ DIS *.dat fort.* *.prf *.med *.nh *.dis
 
which object is 600MB in size? What's the source file that generates that object file? Is it possible that the library is being bundled with your executable instead of dynamically linked?

-Lee
 
Thanks for replying.

The oversized object file is "var.o". That is my suspicion that the library is being compiled into the executable.

I don't know why the linking isn't dynamic though.

dis.f is the main program file
var.f is the module file
other.f files are subroutines

in particular the fft.f calls the FFTW 3 library
 
No it doesn't use any modules or files itself.

MODULE VAR
SAVE
C DEFINE COMMON PARAMETERS
INTEGER,PARAMETER :: NX=660
INTEGER,PARAMETER :: NY=32
INTEGER,PARAMETER :: NZ=180
INTEGER,PARAMETER :: NXYZ=NX*NY*NZ
INTEGER, PARAMETER :: NP=1 !! NO OF SLIP PLANES
INTEGER, PARAMETER :: NQ=3 !! NO OF IN-PLANE SLIP DIR.
REAL, PARAMETER :: SQRT3= 1.732050808 !! SQUARE ROOT OF 3
REAL, PARAMETER :: PI=3.14159265359 !! PI
REAL, PARAMETER :: TWOPI=2*PI !! 2xPI
C DEFINE COMMON VARIABLES FOR
REAL :: N(4,3) !! NORMAL VECTORS TO SLIP PLANES
REAL :: B(4,3,3) !! BURGERS VECTORS
REAL :: G2(NX,NY,NZ) !! K^2 CONSTANTS, LAPLACE OPERATOR IN K-SPACE
REAL :: G(3,NX,NY,NZ) !! K (IJK)COMPONENTS
REAL :: BPQ(NP,NQ,NP,NQ,NX,NY,NZ) !! BPQ MATRIX
REAL :: CIJKL(3,3,3,3) !! COMPLIANCE MATRIX
REAL :: E0(NP,NQ,3,3) !! TRANSFORMATION STRAIN
REAL :: S0(NP,NQ,3,3) !! TRANSFORMATION STRESS
REAL :: S_APP(3,3) !! EXTERNALlY APPLIED STRESS
COMPLEX :: ETAK(NP,NQ,NX,NY,NZ) !! FIELD VARIABLES, K-SPACE
REAL :: ETA(NP,NQ,NX,NY,NZ) !! FIELD VARIABLES, R-SPACE
INTEGER :: OPT0, OPT1
REAL :: TPR(NX,NY,NZ) !! TEMPORARY VARIABLES, K-SPACE
COMPLEX :: TPK(NX,NY,NZ) !! TEMPORARY VARIABLES, R-SPACE
REAL :: ECR(NX,NY,NZ) !! NICRISTRUCTURE VARIABLE
C DEFINE COMMON VARIABLES FOR INO.F
REAL :: A0
REAL :: POSRAT
REAL :: C11, C12, C44, C44X
REAL :: BETA, AA
INTEGER :: ISEED
REAL :: NOISEAMP
REAL :: T0, DT
REAL :: S_APP_MAG
INTEGER :: NIN
INTEGER :: MTL
INTEGER :: GSURFG, GSURFGP
REAL :: DXD
INTEGER :: BORDER !! CREATES FROZEN EVOLUTION FRAME
INTEGER :: NSS
INTEGER :: NSHEAR
REAL :: DX,DY,DZ
C
REAL :: DER(NP,NQ,3,NX,NY,NZ)
C DEFINE COMMON VARIABLES FOR GSF.F
REAL :: GC1(14), GC2(14)
C DEFINE FFTW VARIABLES
INTEGER, PARAMETER :: PF_FFTW3_INIT_TRUE =9000
INTEGER, PARAMETER :: PF_FFTW3_INIT_FALSE=9001
DOUBLE COMPLEX :: pfFFTW3_data(NX,NY,NZ)
INTEGER*8 :: pfFFTW3_plan_fwd, pfFFTW3_plan_bkwd
INTEGER :: pfFFTW3_initflag
END MODULE
 
Large COMMON block perhaps that gets included in the executable completely? Compiler thinks that you have 600 MB of data that can't just be initialised to zero but must be initialised to zeroes carefully stored in the executable?

Haven't done FORTRAN for ages, but try what happens if you remove the "SAVE".
 
I certainly don't know the specifics of gfortran, but I might be inclined to take all of those giant 3d arrays, make them allocatable, and have a var_init subroutine that's part of the module, that gets called at runtime. It would go through and allocate all of that memory for you. Maybe playing with your optimization level would get the compiler to decide to generate all of this for you at launch instead of having the big block in the image.

-Lee
 
It's a bug in gcc for darwin (the same kind of thing happens in C and C++): http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33120

Here is the recommended solution (given in comment #2 in the above link):
A workaround, and recommended best practice for new code, is to make arrays in modules allocatable, i.e. don't declare statically dimensioned large arrays. Then you must add allocate statements at the appropriate initialization locations.
Click to expand...

(Same as lee1210)
 
OK. I've dug out my Fortran textbook and I'll give it a go. Will let you know if it works.
 
Really shouldn't be too bad... just tedious at this point.

Each array will need the "allocatable" attribute added, and in place of each dimension, you place a :. Then you can do one allocate that lists all of them:
Code: 
subroutine init_vars
  allocate(G2(NX,NY,NZ),ETAK(NP,NQ,NX,NY,NZ),....)
end subroutine init_vars

It might actually be better to write the allocate first, so you can just copy from the current declarations so you don't have to remember the dimensions.

-Lee
 
Ok.

I've made some of these arrays allocatable. Then I placed the allocate commands allocating them the parameter dimensions.

However the compiler did not like it. The error said it was not a valid form of array reference.


dis.f:11.29:

ALLOCATE (ETAK(NP:NQ:NX:NY:NZ))
1
Error: Invalid form of array reference at (1)
dis.f:12.28:

ALLOCATE (TPR(NP:NQ:NX:NY:NZ))
1
Error: Invalid form of array reference at (1)
dis.f:13.28:

ALLOCATE (TPK(NP:NQ:NX:NY:NZ))
1
Error: Invalid form of array reference at (1)
dis.f:14.28:

ALLOCATE (BPQ(NP:NQ:NP:NQ:NX:NY:NZ))
1
Error: Invalid form of array reference at (1)
dis.f:15.28:

ALLOCATE (ETA(NP:NQ:NX:NY:NZ))
1
Error: Invalid form of array reference at (1)
dis.f:16.19:

ALLOCATE (ECR(NX:NY:NZ))
1
 
I think it should be something like:
Code: 
      MODULE VAR
      SAVE
C DEFINE COMMON PARAMETERS
      INTEGER,PARAMETER :: NX=660
      INTEGER,PARAMETER :: NY=32
      INTEGER,PARAMETER :: NZ=180
      INTEGER,PARAMETER :: NXYZ=NX*NY*NZ
      INTEGER, PARAMETER :: NP=1             !! NO OF SLIP PLANES
      INTEGER, PARAMETER :: NQ=3             !! NO OF IN-PLANE SLIP DIR. 
      REAL, PARAMETER :: SQRT3= 1.732050808  !! SQUARE ROOT OF 3
      REAL, PARAMETER :: PI=3.14159265359    !! PI
      REAL, PARAMETER :: TWOPI=2*PI          !! 2xPI
C DEFINE COMMON VARIABLES FOR 
      REAL :: N(4,3)        !! NORMAL VECTORS TO SLIP PLANES
      REAL :: B(4,3,3)      !! BURGERS VECTORS
      REAL,allocatable :: G2(:,:,:)  !! K^2 CONSTANTS, LAPLACE OPERATOR IN K-SPACE
      REAL,allocatable :: G(:,:,:,:) !! K (IJK)COMPONENTS
      REAL,allocatable :: BPQ(:,:,:,:,:,:,:) !! BPQ MATRIX
      REAL :: CIJKL(3,3,3,3)            !! COMPLIANCE MATRIX
      REAL :: E0(NP,NQ,3,3)             !! TRANSFORMATION STRAIN
      REAL :: S0(NP,NQ,3,3)             !! TRANSFORMATION STRESS
      REAL :: S_APP(3,3)                !! EXTERNALlY APPLIED STRESS
      COMPLEX,allocatable :: ETAK(:,:,:,:,:)   !! FIELD VARIABLES, K-SPACE
      REAL,allocatable :: ETA(:,:,:,:,:)    !! FIELD VARIABLES, R-SPACE
      INTEGER :: OPT0, OPT1
      REAL,allocatable :: TPR(:,:,:)           !! TEMPORARY VARIABLES, K-SPACE
      COMPLEX,allocatable :: TPK(:,:,:)           !! TEMPORARY VARIABLES, R-SPACE
      REAL,allocatable :: ECR(:,:,:)           !! NICRISTRUCTURE VARIABLE
C DEFINE COMMON VARIABLES FOR INO.F
      REAL :: A0
      REAL :: POSRAT
      REAL :: C11, C12, C44, C44X
      REAL :: BETA, AA
      INTEGER :: ISEED
      REAL :: NOISEAMP
      REAL :: T0, DT
      REAL :: S_APP_MAG
      INTEGER :: NIN
      INTEGER :: MTL
      INTEGER :: GSURFG, GSURFGP
      REAL :: DXD
      INTEGER :: BORDER !! CREATES FROZEN EVOLUTION FRAME
      INTEGER :: NSS       
      INTEGER :: NSHEAR    
      REAL :: DX,DY,DZ
C      
      REAL,allocatable :: DER(:,:,:,:,:,:)
C DEFINE COMMON VARIABLES FOR GSF.F
      REAL :: GC1(14), GC2(14)
C DEFINE FFTW VARIABLES
      INTEGER, PARAMETER :: PF_FFTW3_INIT_TRUE =9000
      INTEGER, PARAMETER :: PF_FFTW3_INIT_FALSE=9001
      DOUBLE COMPLEX,allocatable :: pfFFTW3_data(:,:,:)
      INTEGER*8 :: pfFFTW3_plan_fwd, pfFFTW3_plan_bkwd
      INTEGER :: pfFFTW3_initflag
      contains
      subroutine init_vars
        implicit none
        allocate(G2(NX,NY,NZ),G(3,NX,NY,NZ),BPQ(NP,NQ,NP,NQ,NX,NY,NZ))
        allocate(ETAK(NP,NQ,NX,NY,NZ),ETA(NP,NQ,NX,NY,NZ),TPR(NX,NY,NZ))
        allocate(TPK(NX,NY,NZ),ECR(NX,NY,NZ),DER(NP,NQ,3,NX,NY,NZ))
	allocate(pfFFTW3_data(NX,NY,NZ))
      end subroutine

      END MODULE

I didn't know what "dialect" you were using, so i tried to keep the lines short so I wouldn't have to guess at the continuation style of your dialect. It would be possible to just use a single allocate, otherwise.

I certainly didn't test this (don't have gfortran, just g95), but this compiled fine and generated an object file and .mod file.

Note I only made the REALLY big arrays allocatable. Things that were 81 elements long, etc. i left alone. You could make them allocatable, too, but you probably don't need to.

-Lee

Edit: Just saw your edit. The : is only a placeholder for an allocatable dimension of an array in the declaration. In your allocates use commas between dimensions. I'm guessing your compiler thought you were doing some slicing with : incorrectly.
 
Thanks very much for that last edit. You are a saint.

It turns out my book misses on the correct syntax by using a single variable array as an example.

I've been going crazy the whole day over this and the solution was so simple.

The executable now is 1.2MB, which is acceptable. Just have to check it gives sensible output.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back