Using MPI

Important comment: The guidelines below are given for MPICH, feel free to use either MPICH or LAM/MPI. Last year's experience showed LAM/MPI to have better cleaning commands for the cluster, so try to use LAM/MPI instead. Most commands are similar. I will try to post a manual for LAM/MPI. Otherwise, you can easily find the manual on the web by simply searching for LAM MPI.

Compile, link, run

You must include the header file mpi.h in your source file (#include <mpi.h>). In order to compile the programs you must use mpicc  if your program is written in C, or mpiCC if your program is in C++.
Also, your source files containing C++ code must have extension .C

mpicc yourprogram.c -o yourbinary

or

mpiCC yourprogram.C -o yourbinary

At this point you'll get an executable binary. In order to run this binary file, you must use mpirun:

mpirun -np <number of processes> yourbinary your parameters

Create a .rhosts file in your home directory containing these machine names:

chocolate.eecg.toronto.edu

platinum.eecg.toronto.edu

maroon.eecg.toronto.edu

mauve.eecg.toronto.edu

mint.eecg.toronto.edu

navy.eecg.toronto.edu

ochre.eecg.toronto.edu

The MPICH package can be found at /local/amza/mpich-1.2.6 on these machines. The LAM/MPI package can be found at /local/amza/lam-7.1.1 Alternatively, please feel free to download and use your own LAM-MPI package. In order for you to be able to run MPI commands from anywhere in your home directory please add /local/amza/mpich-1.2.6/bin or the bin directory corresponding to LAM MPI to your path in your .cshrc or .bashrc file in your home directory.

Just edit this file (e.g., .cshrc) and add the above path to your setenv command in this file.

This is how it should look like: setenv PATH “/usr/local/bin/:/usr/local: /local/amza/mpich-1.2.6/bin:.”

Note

MPI man pages are also installed, this is how you access them:

cd /local/amza/mpich-1.2.6

then type in: man the MPI command you want information on

e.g. “man mpirun” and “man MPI_Recv”.

Using MPICH with ssh

You will have to set an environment variable RSHCOMMAND to /usr/bin/ssh.

If you are using csh or tcsh the syntax is:

    setenv RSHCOMMAND /usr/bin/ssh

If you are using bash, the syntax is:

    RSHCOMMAND=/usr/bin/ssh; export RSHCOMMAND

Examples

The examples are on the course web page. For the TSP example, you will compile the program:

mpCC list.C tsp.C -o tsp

and to run it on 4 processors:

mpirun  -all-local -np 4 tsp // This will create 4 processes
mpirun –machinefile ~/.rhosts –np 8 tsp     // This will create 4 processes, distributed on different machines.