Finis Terrae

This document is under active development and discussion!

If you find errors or omissions in this document, please don’t hesitate to submit an issue or open a pull request with a fix. We also encourage you to ask questions and discuss any aspects of the project on the Feel++ Gitter forum. New contributors are always welcome!

CESGA Computing Center

1. Preriquisites

You have to own an account on the machine. See the cluster documentation for the first step on the cluster.
To use differents libraries version that match your need, the cluster uses environment modules You should familiarize first to know how to load specific softwares.
The cluster use slurm job supervisor. You should be familiar with job creation and job submission before going further.

2. Feel++ usage

2.1. Feel++ toolboxes

module load gcc/5.3.0
module load openmpi/1.10.2
module load singularity/2.2.1
module load feelpp-toolboxes/0.102.00

Now you should be able to access via the terminal to default feel++ applications.

To run an application,

mpirun feelpp-toolboxes  feelpp_qs_laplacian_2d

We provide some scripts to install dependencies, if one of the dependencies need to be recompiled, or updated.

2.2. Compilation from scratch

First of all, load the module for the Feel++ library

module load gcc/6.3.0
module load openmpi/2.0.2
module load feelpp

To compile {feelpp++}, download clone the sources in $Lustre directory.

cd ${LUSTRE}
git clone https://github.com/feelpp/feelpp.git

For the cmake step, we recommend you to use clang compiler. You have to export some environment variables for the compilation.

Here is an example of script for the cmake procedure.

#! /bin/sh

BUILDDIR=$PWD
SCRIPTPATH=$( cd $(dirname $0) ; pwd -P )
cd $BUILDDIR

. $SCRIPTPATH/environment
. $SCRIPTPATH/modules

PROF=gcc630
FEELPP_SOURCES=$LUSTRE/devel/feelpp.git
export CMAKE_BUILD_TYPE=RELEASE
export CLANG_DIR=/opt/cesga/easybuild/software/Clang/3.9.1-foss-2017a
export GMSH_DIR=${INSDIR}/$PROF/gmsh/2.16.0
export PETSC_DIR=${INSDIR}/$PROF/petsc/3.7.3
export CLN_DIR=${INSDIR}/$PROF/cln/1.3.4
export PETSC_ARCH=
export CC=${CLANG_DIR}/bin/clang
export CXX=${CLANG_DIR}/bin/clang++
#export CC=`which gcc` \
    #export CXX=`which g++` \

${INSDIR}/$PROF/cmake/3.8.0/bin/cmake \
    -DFEELPP_PETSC_ENABLE_TESTS=OFF \
    $FEELPP_SOURCES

Remember you can download the scripts to install Feel++

2.3. Singularity

See section for more details about this container solution.

First you need to load the specific module

module load singularity/2.2.1

Then you can download an image from one the official Feel++ singularity images.

2.4. Slurm (job scripts)

2.4.1. Feel++ toolboxes (module)

Script 1

Launch feelpp_qs_laplacian_2d in two task in different nodes.

#!/bin/bash
#SBATCH -p cola-corta
#SBATCH -N 2
#SBATCH -n 2
#SBATCH -t 00:10:00
#SBATCH --ntasks-per-node=1

module purge
module load gcc/5.3.0 openmpi/1.10.2 singularity/2.2.1 feelpp-toolboxes/0.102.00

mpirun feelpp-toolboxes  feelpp_qs_laplacian_2d

Script 2

Change Feel++ output directory and launch feelpp_toolbox_fluid_2d in two task in different nodes.

#!/bin/bash
#SBATCH -p cola-corta
#SBATCH -N 2
#SBATCH -n 2
#SBATCH -t 00:10:00
#SBATCH --ntasks-per-node=1

module purge
module load gcc/5.3.0 openmpi/1.10.2 singularity/2.2.1 feelpp-toolboxes/0.102.00

mpirun feelpp-toolboxes  "export FEELPP_REPOSITORY=$PWD && feelpp_toolbox_fluid_2d --config-file=PATH-TO/TurekHron/cfd1.cfg"

3. Acknowledgments

The support of Feel++ @ CESGA has been funded by the EU E-INFRA H2020 project MSO4SC.