Kapitel 5 MPI

Message Passing Interface (MPI)

- Open standard, portable and versatile.

- Natively supports C, C++ and FORTRAN

Message Passing Interface (MPI)

• “Single Program, Multiple Data” (SPMD) paradigm

  • The same binary is started on all nodes in an MPI job at the same time

  • Might be started multiple times on same node

  • Processes know how to reach each other (communications channels)

  • All variables have the same name and address in every process, but are always local!

• Processes send each other messages

  Messages go from source rank(s) to destination rank(s)

  Messages contain size items of a single data type

• Actual encoding of messages on the communications channel is implementation-dependent

• Most implementations support multiple channels (e.g. IPC/shared memory between processes on the same node and TCP/IP or InfiniBand between nodes)

• Content of messages is entirely up to programmer

• Keeps the API very simple, flexible and fast But requires more work

Every process gets a random sequential index (the rank)

• Determined by the implementation, but usually the an external componenten (e.g. the batch system) assigns the ranks

• All processes are started using a wrapper command called mpirun or mpiexec

• All decisions (e.g. distribution of data) are based on rank numbers

• Sequential parts of the code are usually assigned to rank 0 (arbitrary choice)

One source, one destination

Synchronous/Blocking

• Sender calls MPI_Send(). This call only returns after destionation has called

• MPI_receive(), read and acknowledged the message

• If destination never calls MPI_receive() or crashes, sender hangs

Asynchronous/Blocking

• Sender calls MPI_Isend(), call returns immediately and sender continues with work

• Destination calls MPI_Irecv() at some point and sends back acknowledgement

• Sender can call MPI_Test() or MPI_Wait() if it wants to check/wait for an acknowledgement

• Sender must specify a valid destination rank

• Receiver must specify a valid source rank (or wildcard on destination)

• Communicators must be identical

• Tags must be identical (or wildcard on destination)

• Data type must be identical

• Receive buffer must be large enough