Undesirably Huge Executable !!! (Fortran)

Discussion in 'Mac Programming' started by lordava, Aug 9, 2010.

  1. lordava macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #1
    Hello

    I only recently learnt to compile my using a makefile and modules. My problem is that I have to use two systems Linux and Mac.

    Now my makefile works a charm on the Linux box: compiles fast and the file is only about 800 kb in size.

    When I use the same makefile on my 10.6 MacBook. It takes much longer to compile and creates a massive executable almost 600MB in size. This is insane!!! And I don't know why it occurs.

    One thing I noticed is that one of the object files created is also about the same size.

    The executable runs but 600MB is a problem. Can anybody shed some advice? I would be immensely grateful.

    Compiler: gfortran (from HPC Mac website)
    libraries: FFTW3

    Here is the makefile

    *********************************
    OBJS = dis.o \
    var.o \
    evo.o \
    grd.o \
    ini.o \
    str.o \
    ino.o \
    gsf.o \
    fft.o

    FC=gfortran

    FFLAGS= -O3 -I"/usr/local/include" -L"/usr/local/lib"

    DIS: ${OBJS} Makefile
    ${FC} -o $@ ${OBJS} ${FFLAGS} -lfftw3

    var.mod: var.o var.f
    ${FC} -c var.f
    var.o: var.f
    ${FC} -c var.f

    dis.o: var.mod dis.f
    ${FC} -c dis.f

    evo.o: var.mod evo.f
    ${FC} -c evo.f
    grd.o: var.mod grd.f
    ${FC} -c grd.f
    ini.o: var.mod ini.f
    ${FC} -c ini.f
    str.o: var.mod str.f
    ${FC} -c str.f
    ino.o: var.mod ino.f
    ${FC} -c ino.f
    gsf.o: var.mod gsf.f
    ${FC} -c gsf.f
    fft.o: fft.f
    ${FC} -c fft.f ${FFLAGS} -lfftw3
    clean:
    rm -f *.o *~ *.mod

    cleanall:
    rm -f *.o *~ DIS *.dat fort.* *.prf *.med *.nh *.dis
     
  2. lee1210 macrumors 68040

    lee1210

    Joined:
    Jan 10, 2005
    Location:
    Dallas, TX
    #2
    which object is 600MB in size? What's the source file that generates that object file? Is it possible that the library is being bundled with your executable instead of dynamically linked?

    -Lee
     
  3. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #3
    Thanks for replying.

    The oversized object file is "var.o". That is my suspicion that the library is being compiled into the executable.

    I don't know why the linking isn't dynamic though.

    dis.f is the main program file
    var.f is the module file
    other.f files are subroutines

    in particular the fft.f calls the FFTW 3 library
     
  4. lee1210 macrumors 68040

    lee1210

    Joined:
    Jan 10, 2005
    Location:
    Dallas, TX
    #4
    Can you post var.f? Does it use any modules? Does it include any other files?

    -Lee
     
  5. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #5
    No it doesn't use any modules or files itself.

    MODULE VAR
    SAVE
    C DEFINE COMMON PARAMETERS
    INTEGER,PARAMETER :: NX=660
    INTEGER,PARAMETER :: NY=32
    INTEGER,PARAMETER :: NZ=180
    INTEGER,PARAMETER :: NXYZ=NX*NY*NZ
    INTEGER, PARAMETER :: NP=1 !! NO OF SLIP PLANES
    INTEGER, PARAMETER :: NQ=3 !! NO OF IN-PLANE SLIP DIR.
    REAL, PARAMETER :: SQRT3= 1.732050808 !! SQUARE ROOT OF 3
    REAL, PARAMETER :: PI=3.14159265359 !! PI
    REAL, PARAMETER :: TWOPI=2*PI !! 2xPI
    C DEFINE COMMON VARIABLES FOR
    REAL :: N(4,3) !! NORMAL VECTORS TO SLIP PLANES
    REAL :: B(4,3,3) !! BURGERS VECTORS
    REAL :: G2(NX,NY,NZ) !! K^2 CONSTANTS, LAPLACE OPERATOR IN K-SPACE
    REAL :: G(3,NX,NY,NZ) !! K (IJK)COMPONENTS
    REAL :: BPQ(NP,NQ,NP,NQ,NX,NY,NZ) !! BPQ MATRIX
    REAL :: CIJKL(3,3,3,3) !! COMPLIANCE MATRIX
    REAL :: E0(NP,NQ,3,3) !! TRANSFORMATION STRAIN
    REAL :: S0(NP,NQ,3,3) !! TRANSFORMATION STRESS
    REAL :: S_APP(3,3) !! EXTERNALlY APPLIED STRESS
    COMPLEX :: ETAK(NP,NQ,NX,NY,NZ) !! FIELD VARIABLES, K-SPACE
    REAL :: ETA(NP,NQ,NX,NY,NZ) !! FIELD VARIABLES, R-SPACE
    INTEGER :: OPT0, OPT1
    REAL :: TPR(NX,NY,NZ) !! TEMPORARY VARIABLES, K-SPACE
    COMPLEX :: TPK(NX,NY,NZ) !! TEMPORARY VARIABLES, R-SPACE
    REAL :: ECR(NX,NY,NZ) !! NICRISTRUCTURE VARIABLE
    C DEFINE COMMON VARIABLES FOR INO.F
    REAL :: A0
    REAL :: POSRAT
    REAL :: C11, C12, C44, C44X
    REAL :: BETA, AA
    INTEGER :: ISEED
    REAL :: NOISEAMP
    REAL :: T0, DT
    REAL :: S_APP_MAG
    INTEGER :: NIN
    INTEGER :: MTL
    INTEGER :: GSURFG, GSURFGP
    REAL :: DXD
    INTEGER :: BORDER !! CREATES FROZEN EVOLUTION FRAME
    INTEGER :: NSS
    INTEGER :: NSHEAR
    REAL :: DX,DY,DZ
    C
    REAL :: DER(NP,NQ,3,NX,NY,NZ)
    C DEFINE COMMON VARIABLES FOR GSF.F
    REAL :: GC1(14), GC2(14)
    C DEFINE FFTW VARIABLES
    INTEGER, PARAMETER :: PF_FFTW3_INIT_TRUE =9000
    INTEGER, PARAMETER :: PF_FFTW3_INIT_FALSE=9001
    DOUBLE COMPLEX :: pfFFTW3_data(NX,NY,NZ)
    INTEGER*8 :: pfFFTW3_plan_fwd, pfFFTW3_plan_bkwd
    INTEGER :: pfFFTW3_initflag
    END MODULE
     
  6. gnasher729 macrumors P6

    gnasher729

    Joined:
    Nov 25, 2005
    #6
    Large COMMON block perhaps that gets included in the executable completely? Compiler thinks that you have 600 MB of data that can't just be initialised to zero but must be initialised to zeroes carefully stored in the executable?

    Haven't done FORTRAN for ages, but try what happens if you remove the "SAVE".
     
  7. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
  8. lee1210 macrumors 68040

    lee1210

    Joined:
    Jan 10, 2005
    Location:
    Dallas, TX
    #8
    I certainly don't know the specifics of gfortran, but I might be inclined to take all of those giant 3d arrays, make them allocatable, and have a var_init subroutine that's part of the module, that gets called at runtime. It would go through and allocate all of that memory for you. Maybe playing with your optimization level would get the compiler to decide to generate all of this for you at launch instead of having the big block in the image.

    -Lee
     
  9. ncl macrumors member

    Joined:
    Aug 16, 2008
    #9
    It's a bug in gcc for darwin (the same kind of thing happens in C and C++): http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33120

    Here is the recommended solution (given in comment #2 in the above link):
    (Same as lee1210)
     
  10. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #10
    OK. I've dug out my Fortran textbook and I'll give it a go. Will let you know if it works.
     
  11. lee1210 macrumors 68040

    lee1210

    Joined:
    Jan 10, 2005
    Location:
    Dallas, TX
    #11
    Really shouldn't be too bad... just tedious at this point.

    Each array will need the "allocatable" attribute added, and in place of each dimension, you place a :. Then you can do one allocate that lists all of them:
    Code:
    subroutine init_vars
      allocate(G2(NX,NY,NZ),ETAK(NP,NQ,NX,NY,NZ),....)
    end subroutine init_vars
    It might actually be better to write the allocate first, so you can just copy from the current declarations so you don't have to remember the dimensions.

    -Lee
     
  12. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #12
    Ok.

    I've made some of these arrays allocatable. Then I placed the allocate commands allocating them the parameter dimensions.

    However the compiler did not like it. The error said it was not a valid form of array reference.


    dis.f:11.29:

    ALLOCATE (ETAK(NP:NQ:NX:NY:NZ))
    1
    Error: Invalid form of array reference at (1)
    dis.f:12.28:

    ALLOCATE (TPR(NP:NQ:NX:NY:NZ))
    1
    Error: Invalid form of array reference at (1)
    dis.f:13.28:

    ALLOCATE (TPK(NP:NQ:NX:NY:NZ))
    1
    Error: Invalid form of array reference at (1)
    dis.f:14.28:

    ALLOCATE (BPQ(NP:NQ:NP:NQ:NX:NY:NZ))
    1
    Error: Invalid form of array reference at (1)
    dis.f:15.28:

    ALLOCATE (ETA(NP:NQ:NX:NY:NZ))
    1
    Error: Invalid form of array reference at (1)
    dis.f:16.19:

    ALLOCATE (ECR(NX:NY:NZ))
    1
     
  13. lee1210 macrumors 68040

    lee1210

    Joined:
    Jan 10, 2005
    Location:
    Dallas, TX
    #13
    I think it should be something like:
    Code:
          MODULE VAR
          SAVE
    C DEFINE COMMON PARAMETERS
          INTEGER,PARAMETER :: NX=660
          INTEGER,PARAMETER :: NY=32
          INTEGER,PARAMETER :: NZ=180
          INTEGER,PARAMETER :: NXYZ=NX*NY*NZ
          INTEGER, PARAMETER :: NP=1             !! NO OF SLIP PLANES
          INTEGER, PARAMETER :: NQ=3             !! NO OF IN-PLANE SLIP DIR. 
          REAL, PARAMETER :: SQRT3= 1.732050808  !! SQUARE ROOT OF 3
          REAL, PARAMETER :: PI=3.14159265359    !! PI
          REAL, PARAMETER :: TWOPI=2*PI          !! 2xPI
    C DEFINE COMMON VARIABLES FOR 
          REAL :: N(4,3)        !! NORMAL VECTORS TO SLIP PLANES
          REAL :: B(4,3,3)      !! BURGERS VECTORS
          REAL,allocatable :: G2(:,:,:)  !! K^2 CONSTANTS, LAPLACE OPERATOR IN K-SPACE
          REAL,allocatable :: G(:,:,:,:) !! K (IJK)COMPONENTS
          REAL,allocatable :: BPQ(:,:,:,:,:,:,:) !! BPQ MATRIX
          REAL :: CIJKL(3,3,3,3)            !! COMPLIANCE MATRIX
          REAL :: E0(NP,NQ,3,3)             !! TRANSFORMATION STRAIN
          REAL :: S0(NP,NQ,3,3)             !! TRANSFORMATION STRESS
          REAL :: S_APP(3,3)                !! EXTERNALlY APPLIED STRESS
          COMPLEX,allocatable :: ETAK(:,:,:,:,:)   !! FIELD VARIABLES, K-SPACE
          REAL,allocatable :: ETA(:,:,:,:,:)    !! FIELD VARIABLES, R-SPACE
          INTEGER :: OPT0, OPT1
          REAL,allocatable :: TPR(:,:,:)           !! TEMPORARY VARIABLES, K-SPACE
          COMPLEX,allocatable :: TPK(:,:,:)           !! TEMPORARY VARIABLES, R-SPACE
          REAL,allocatable :: ECR(:,:,:)           !! NICRISTRUCTURE VARIABLE
    C DEFINE COMMON VARIABLES FOR INO.F
          REAL :: A0
          REAL :: POSRAT
          REAL :: C11, C12, C44, C44X
          REAL :: BETA, AA
          INTEGER :: ISEED
          REAL :: NOISEAMP
          REAL :: T0, DT
          REAL :: S_APP_MAG
          INTEGER :: NIN
          INTEGER :: MTL
          INTEGER :: GSURFG, GSURFGP
          REAL :: DXD
          INTEGER :: BORDER !! CREATES FROZEN EVOLUTION FRAME
          INTEGER :: NSS       
          INTEGER :: NSHEAR    
          REAL :: DX,DY,DZ
    C      
          REAL,allocatable :: DER(:,:,:,:,:,:)
    C DEFINE COMMON VARIABLES FOR GSF.F
          REAL :: GC1(14), GC2(14)
    C DEFINE FFTW VARIABLES
          INTEGER, PARAMETER :: PF_FFTW3_INIT_TRUE =9000
          INTEGER, PARAMETER :: PF_FFTW3_INIT_FALSE=9001
          DOUBLE COMPLEX,allocatable :: pfFFTW3_data(:,:,:)
          INTEGER*8 :: pfFFTW3_plan_fwd, pfFFTW3_plan_bkwd
          INTEGER :: pfFFTW3_initflag
          contains
          subroutine init_vars
            implicit none
            allocate(G2(NX,NY,NZ),G(3,NX,NY,NZ),BPQ(NP,NQ,NP,NQ,NX,NY,NZ))
            allocate(ETAK(NP,NQ,NX,NY,NZ),ETA(NP,NQ,NX,NY,NZ),TPR(NX,NY,NZ))
            allocate(TPK(NX,NY,NZ),ECR(NX,NY,NZ),DER(NP,NQ,3,NX,NY,NZ))
    	allocate(pfFFTW3_data(NX,NY,NZ))
          end subroutine
    
          END MODULE
    
    I didn't know what "dialect" you were using, so i tried to keep the lines short so I wouldn't have to guess at the continuation style of your dialect. It would be possible to just use a single allocate, otherwise.

    I certainly didn't test this (don't have gfortran, just g95), but this compiled fine and generated an object file and .mod file.

    Note I only made the REALLY big arrays allocatable. Things that were 81 elements long, etc. i left alone. You could make them allocatable, too, but you probably don't need to.

    -Lee

    Edit: Just saw your edit. The : is only a placeholder for an allocatable dimension of an array in the declaration. In your allocates use commas between dimensions. I'm guessing your compiler thought you were doing some slicing with : incorrectly.
     
  14. lordava thread starter macrumors newbie

    lordava

    Joined:
    Aug 9, 2010
    #14
    Thanks very much for that last edit. You are a saint.

    It turns out my book misses on the correct syntax by using a single variable array as an example.

    I've been going crazy the whole day over this and the solution was so simple.

    The executable now is 1.2MB, which is acceptable. Just have to check it gives sensible output.
     

Share This Page