Page tree
Skip to end of metadata
Go to start of metadata

Here's instructions on how to compile ASCOT5 on different platforms.

Vdiubuntu.aalto.fi

 You can use the nvidia version of vdi.aalto.fi to do VTK visualization  Here is what I (Simppa Äkäslompolo ) did to get libascot.so working on the vdiubuntu.
There is no hdf5 on the vdiubuntus, but you can get it more-or-less easily as part of the ascot5 environment. Install it using the ascot5.yaml-file in a5py folder of the python repository. There are some hints in the file.


Once you are in the (ascot5) environment, you need to change the Makefile:

(remember to update the folder of your miniconda environment)


+MINICONDA_ENV=/u/54/sjjamsa/unix/conda/miniconda3/envs/ascot5/
 ifneq ($(CC),h5cc)
        ifneq ($(CC),h5pcc)
-               CFLAGS+=-lhdf5 -lhdf5_hl
+               DEFINES+=-DH5_BUILT_AS_DYNAMIC_LIB=1
+               CFLAGS+=-I$(MINICONDA_ENV)include \
+               -isystem $(MINICONDA_ENV)include  \
+               -L$(MINICONDA_ENV)lib \
+               -Wl,-rpath,$(MINICONDA_ENV)lib \
+               -Wl,-rpath-link,$(MINICONDA_ENV)lib
        endif
 endif


libascot.so: libascot.o $(OBJS)
-       $(CC) $(CFLAGS) -o $@ $^
+       $(CC) $(CFLAGS) -o $@ $^ -lhdf5_hl -lhdf5


Then when compiling, use something like this:

  make VERBOSE=2 MPI=0 libascot.so CC=gcc FLAGS="-foffload=disable" -j



MPCDF Cobra

module load doxygen
module load intel
module load impi
module load git
module load hdf5-mpi
make ascot5_main MPI=1 VERBOSE=1 FLAGS="-qno-openmp-offload"

MPCDF Raven

module load doxygen
module load intel/19.1.2
module load impi/2019.8
module load git
module load hdf5-mpi
make ascot5_main MPI=1 VERBOSE=1 FLAGS="-qno-openmp-offload -diag-disable 3180"
FOR PYTHON INTERFACE
module load anaconda/3/2020.02
export LD_LIBRARY_PAHT="<PATH TO LIBASCOTDIR>:${LD_LIBRARY_PATH}"
python setup.py install --user


Marconi KNL

module load intel/pe-xe-2018--binary intelmpi/2018--binary szip/2.1--gnu--6.1.0 zlib/1.2.8--gnu--6.1.0 hdf5/1.8.18--intelmpi--2018--binary
module load python/3.5.2
make ascot5_main VERBOSE=2 MPI=1 FLAGS="-qno-openmp-offload -diag-disable 3180 -xmic-avx512 -vecabi=cmdtarget"

Marconi SKL

module load intel/pe-xe-2018--binary intelmpi/2018--binary szip/2.1--gnu--6.1.0 zlib/1.2.8--gnu--6.1.0 hdf5/1.8.18--intelmpi--2018--binary
module load python/3.5.2
make ascot5_main VERBOSE=2 MPI=1 FLAGS="-qno-openmp-offload -diag-disable 3180 -xcommon-avx512 -vecabi=cmdtarget"


Marconi M100 (GPU)

module load xl
module load spectrum_mpi/10.3.1--binary
module load gnu/8.4.0
module load hdf5/1.12.0--spectrum_mpi--10.3.1--binary
module load szip

NERSC Cori

module load cray-hdf5-parallel
make ascot5_main CC=h5cc VERBOSE=0 MPI=1 FLAGS="-qno-openmp-offload –diag-disable 3180"

Prior to running:

module load cray-hdf5-parallel
export PMI_NO_FORK=1
export PMI_NO_PREINITIALIZE=1
export HDF5_USE_FILE_LOCKING=FALSE

Marenostrum (WIP)

module load hdf5/1.8.19 intel/2018.4 impi/2018.4 zlib szip/2.1.1

make ascot5_main VERBOSE=2 MPI=1 FLAGS="-qno-openmp-offload -diag-disable 3180 -xcommon-avx512 -vecabi=cmdtarget"

CSC.fi puhti

module load StdEnv intel/19.0.4  hpcx-mpi/2.4.0  intel-mkl/2019.0.4  hdf5/1.10.4-mpi

make -j ascot5_main MPI=1 VERBOSE=1

TOK-cluster at AUG

module load intel/18.0.5 impi/2018.4 hdf5-mpi/1.8.21
make -j ascot5_main MPI=0 VERBOSE=1 FLAGS="-qno-openmp-offload -diag-disable 3180"

Triton.aalto.fi

For GCC:

module load hdf5/1.10.2-openmpi
make -j ascot5_main MPI=1 VERBOSE=1

For Intel:

module load intel-parallel-studio
module load hdf5/1.10.2-openmpi
export OMPI_MPICC=icc
make ascot5_main VERBOSE=2 MPI=1 FLAGS="-qno-openmp-offload -diag-disable 3180 -vecabi=cmdtarget"

(add -xcommon-avx512 to optimize for skl/csl nodes)


Aalto desktops

pkcon install hdf5-helpers
pkcon install libhdf5-dev
make -j ascot5_main VERBOSE=1 FLAGS="-foffload=disable"

You might also want to install hdf5-tools (provides h5copygroup, h5ls etc.):


pkcon install hdf5-tools

Lac8 at TCV

 make libascot CC=h5cc 
 make ascot5_main CC=h5cc MPI=0 VERBOSE=2
 make bbnbi5 CC=h5cc MPI=0 VERBOSE=1


Compiler flags

Use these with make ascot5_main FLAGS="..." when compiling.

-qno-openmp-offload -diag-disable 3180Ignore offload pragmas.
-xcommon-avx512Compile for KNL or Skylake processor.
-xmic-avx512Compile (and optimize) for KNL processor.
-xcore-avx512Compile (and optimize) for Skylake processor.
-vecabi=cmdtargetEnables vector instructions for NSIMD > 2.
-qopt-report=5 -qopt-report-phase=vecGenerate vectorization reports in *optrpt files.
-ipo

Interprocedural Optimization.

-foffload=disableDisables compiling for GPU. (fixes HSAIL segmentation fault)


See ascot5.h for ASCOT5 specific flags


Special cases with solutions



We had the issue that libascot.so depended on hdf5 (as usual), but the objects within h5*.a were not compiled with -fPIC. Thus we couldn't include them in libascot.so. What we did was add something -lhdf5_hl -lhdf5 to the Makefile line where libascot.so was linked. This isn't really an issue with ASCOT. How should I document it?

  • No labels