Montana Tech High Performance Computing Montana Tech High Performance Computing

BeoMPI (based on MPICH) is the default Message Passing Interface (MPI) on the system. MPICH2[1], MVAPICH2[2], and OpenMPI[3] are available and easily selected with the env-modules package that allow users to change their environment variables from predefined files (currently only the gnu compiler is loaded). To see what modules are available:

module avail

and to load a module:

module load mpich2/gnu

MPICH2 does not use the InfiniBand (IB) Network, but MVAPICH2 does. Our preliminary benchmarking tests show the best performance with MVAPICH2.

Once the MPI environment is set, the corresponding mpicc and mpif90 are available. These wrappers use the gnu 4.8.5 compilers distributed with the Centos 7.3 are in /usr/bin.

Note OpenMPI will use IB with the mca flag:

mpirun --mca btl openib,sm,self -np ....

Example

To compile a C program call mpiprog.c using MPICH2:

module load mpich2/gnu

mpicc mpiprog.c -o mpiprog

To run using torque create a script mpijob with:

#!/bin/bash
#PBS -l nodes=2:ppn=32
#PBS -N MPIJob
#PBS -d /home/mtech/username
#PBS -S /bin/bash
#PBS -m e
#PBS -M username@mtech.edu
#PBS -l walltime=01:00:00

module load mpich2/gnu
mpirun -np 64 --hostfile $PBS_NODEFILE mpiprog

Then submit to moab

msub mpijob