
% BB Oct 30 2007.

\documentclass[20pt,landscape,footrule]{foils}
\usepackage[dvips]{graphicx}
\usepackage[pdftex]{geometry}
\geometry{headsep=3ex,hscale=0.9}
\usepackage[pdftex]{color} % for the colored block
\date{\color{blue} Singapore MCMC Meeting, March 2004}
\usepackage[ignore]{pause}
%\usepackage{pause}
\usepackage{background}
%\usepackage{pp4slide}
%\MyLogo{AV HEP,CSIT}
\LogoOff
\begin{document}
\def\r{\color{red}}
\def\b{\color{blue}}
\def\s{\color{black}}

\foilhead{\r Message Passing Interface (MPI)} \b

\noindent In a typical Unix installation a MPI Fortran~77 program will 
compile with 
\begin{equation} \label{MPI_compile}
\s  {\tt \dots pathname/mpif77\ -\!O\ fln.f}
\end{equation}
and generates an executable file, {\s\tt a.out}, which runs with
something like
\begin{equation} \label{MPI_run}
\s {\tt \dots pathname/mpirun\ -}\!{\tt nolocal\ -}\!{\tt np\ n\ a.out}
\end{equation}
or $\s \dots\ {\tt ./a.out}$ in some installations. Here the option 
{\s\tt -np} defines the number $\s n$ of processes, $\s n=1$ if the 
{\s\tt -np} option is omitted. In our installation (\ref{MPI_run}) 
reads
\begin{equation} \label{MPI_run2}
\s {\tt \dots pathname/mpirun\ -\!-} {\tt hostfile\ hostfile.txt\ -}
   {\tt nolocal\ -} {\tt np\ n\ a.out}
\end{equation}
where the file {\s\tt hostfile.txt} contain the names (addresses) of
the participating non-local processors.  The number of processors 
(CPUs) can be smaller than the number of processes. MPI will then 
run several, or even all,  processes on one CPU. 

As in our previous Fortran programs, we include the {\r implicit 
declaration} for our real and logical (l) variables and use the Fortran 
default for the remaining integers. This is now followed by the MPI 
preprocessor directive, which is brought into the routines through 
$$ \s  {\tt include\ 'mpif.h'}\ .$$
In a properly implemented MPI environment the compiler will find the 
file {\tt mpif.h} which contains definitions, macros and function 
prototypes necessary for compiling the MPI program. The order of 
the statements matters, because {\tt mpif.h} overwrites some of our 
implicit declarations with explicit ones. All definitions which 
MPI makes have names of the form
$$\s  {\tt MPI\_ ANY\_ NAME}\ .$$
Therefore, {\bf do not to introduce any own variables with names 
of the form {\s\tt MPI\_}$\s\dots$}. In our code we use instead
{\s\tt ANY\_NAME\_MPI}$\s\dots$, when we like to emphasize a
relationship to MPI. Our usual include statements for parameter 
files and common blocks follow ${\tt include\ 'mpif.h'}$. 

Before calling MPI communication routines, MPI needs to
be initialized and it is mandatory to call
$$\s  {\tt MPI\_ INIT(IERROR) }\ .$$
{\tt MPI\_INIT} must be called only once. The integer variable {\s\tt 
IERROR} can be ignored as long as everything proceeds smoothly. It 
returns MPI error codes, which can be looked up in a MPI manual. Next, 
a process may call
$$\s  {\tt MPI\_ COMM\_ RANK (COMM,RANK,IERROR) }$$
to find out its rank. The rank is an integer, called {\tt MY\_ID} 
(for my identity) in our program. The first argument of 
{\tt MPI\_COMM\_RANK} is a {\bf communicator}.  A communicator 
is a collection of processes that can send messages to each other. 
For basic programs the only communicator needed is the one used here
$$\s  {\tt MPI\_ COMM\_ WORLD }\ .$$
It is predefined in MPI and consists of all processes running when 
the program execution begins. The second argument is the {\s integer 
rank} of the process, which takes the values $\s 0,\dots,n-1$ if there 
are {\s $n$ processes.} In a program the integer variable {\s\tt 
MY\_ID} (which denotes the rank) can be used to branch the program by 
assigning different random numbers and temperatures to the processes.

Some constructs depend on the total number of processes executing the 
program. MPI provides the routine
$$\s  {\tt MPI\_ COMM\_ SIZE (COMM,ISIZE,IERROR) }$$
which returns the number of processes of the communicator {\s\tt COMM} 
in its integer argument {\s\tt ISIZE}. In our programs we use the 
integer variable {\s\tt N\_PROC} (number of processes) to denote 
the size. 

Two important MPI routines achieve {\r point to point communication 
between processes:} 
$$\s {\tt MPI\_SEND (BUF,ICOUNT,DATATYPE,IDEST,ITAG,COMM,IERROR)}$$
and
$$\s {\tt MPI\_RECV 
     (BUF,ICOUNT,DATATYPE,ISOURCE,ITAG,COMM,ISTATUS,IERROR)}\ .$$
Differences in the arguments are {\s\tt DEST} in {\s\tt MPI\_SEND} 
versus {\s \tt SOURCE} in {\s \tt MPI\_RECV} and the additional 
{\s\tt STATUS} array in {\s\tt MPI\_RECV}. All arguments are explained 
in the following. 

\begin{enumerate}

\item {\s\tt BUF}: The initial address of the send or receive buffer.

\item {\s\tt ICOUNT}: An integer variable which gives the number of
      entries to be send.

\item {\s\tt DATATYPE}: The datatype of each entry. MPI datatypes which
agree with the corresponding Fortran datatypes are {\s\tt MPI\_INTEGER},
{\s\tt MPI\_REAL}, {\s\tt MPI\_DOUBLE\_PRECISION}, {\s\tt MPI\_COMPLEX}, 
{\s\tt MPI\_LOGICAL} and {\s\tt MPI\_CHARACTER}. Additional MPI data types 
({\s\tt MPI\_BYTE} and {\s\tt MPI\_PACKED}) exist.

\item {\s \tt IDEST:} The rank of the destination process. 

\item {\s\tt ISOURCE:} The rank of the source. Note that the sender 
and receiver can be identical. 

\item {\s\tt ITAG:} The message tag, an integer in the range
$0,\dots,32767$ (many MPI implementations allow for even larger 
numbers than $32767$), which has to be identical for sender and
receiver.  The purpose of the tag is that the receiver can figure 
out the message order when one process sends two or more 
messages (the receive can be delayed due to other processing).

\item {\s\tt COMM:} The communicator. Most often this is
{\s\tt MPI\_COMM\_WORLD}.

\item {\s\tt ISTATUS:} The return status of {\s\tt MPI\_RECV}. This
array is properly dimensiones by {\s\tt MPI\_STATUS\_SIZE}. Strange
errors can result if the corresponding dimension statement is
missing (consider to have {\s\t ISTATUS(MPI\_STATUS\_SIZE)} in a 
common block, which is propagated to all routines that need it).
Besides that {\s\tt ISTATUS} can be more or less ignored. Consult 
the MPI manual, if peculiar completions codes are encountered.

\item {\s IERROR:} The error code, an integer with which we deal 
in the same way as with the return status.

\end{enumerate}

The fields {\bf source, destination, tag} and {\bf communicator} are 
collectively called the {\bf MPI message envelope}. The receiver may 
specify the wildcard value {\s\tt MPI\_ANY\_SOURCE} for {\s\tt SOURCE}, 
and/or the wildcard value {\s\tt MPI\_ANY\_TAG} for {\tt TAG}, to 
indicate that any source and/or tag value is acceptable. However, 
there is then a danger to loose control about what it transferred by 
whom to who.

MPI code should conclude with a call to
$$\s {\tt MPI\_ FINALIZE (IERROR) }$$
which tells its process that no further MPI instructions follow. This
call is mandatory for an error free termination of MPI. 

More usefull instruction are listed in the following.

$$\s {\tt MPI\_ BARRIER(COMM,IERROR) }$$

which is normally called with the {\s\tt MPI\_COMM\_WORLD} communicator. 
Execution of the code proceeds only when all processes have reached 
the barrier, so that synchronization is enforced at this point. In 
particular for debugging a code that can be of value.

\begin{eqnarray} \nonumber
&\s {\tt MPI\_ ALLGATHER}&\s\tt\!\!\!\! (SENDBUF,ISENDCOUNT,SENDTYPE,
RECVBUF, \\ & &\s {\tt IRECVCOUNT,RECVTYPE,COMM,IERROR)} \nonumber
\end{eqnarray} 
gathers on each process information from all processes, with the
range defined by a communicator. The arguments are: {\s\tt SENDBUF}, 
the starting element of the send buffer; {\s\tt ISENDCOUNT} (integer), 
the number of elements in the send buffer; {\s\tt SENDTYPE}, the data 
type of the send buffer elements; {\s\tt RECVBUF}, the first element 
of the receive buffer; {\s\tt IRECVCOUNT} (integer), the number of 
elements in the receive buffer; {\s\tt RECVTYPE}, the data type of 
the receive buffer; {\s\tt COMM}, the communicator; {\s\tt IERROR}, 
the MPI error code.

Finally a usefull instruction, in particular for debugging purposes, is
$$\s\tt MPI\_BCAST(BUF,ICOUNT,DATATYPE,ISOURCE,COMM,IERROR)$$
which broadcasts the buffer {\s\tt BUF} from the process with rank 
{\s\tt ISOURCE} to all processes specified by the communicator 
{\s\tt COMM}. The number of elements in the buffer is given by 
{\s\tt ICOUNT}, the datatype by {\s\tt DATATYPE}, and
{\s\tt IERROR} is the MPI error code.
 
% \newpage

% \newpage
% \vpagecolor{white} \renewcommand\normalcolor{\color{pictcol}}
% \begin{figure} \begin{center}
% \includegraphics[width=0.7\textheight]{figures/test10.pdf}
% \includegraphics[width=0.7\textheight]{figures/test02.pdf}
% \caption{\small The largest cluster surface for the 10-state (left) 
% and 2-state (right) Potts models on various lattice sizes as indicated 
% in the figure (the extensions are the value of $n_{\beta}$ or e for 
% equilibrium). }
% \end{center} \end{figure}

\end{document}


