How to use non-blocking sends and receives to great effect

http://www.cs.ucsb.edu/~hnielsen/cs140/mpi-deadlocks.html

The send and receive calls are almost exactly the same as the blocking versions, with two differences. The first (which is conceptually important) is that the function will return immediately, regardless of whether the send or receive has finished. The second (which is important for the implementation) is that they take an MPI_Request pointer as the final parameter. This is later used to verify that the send or receive has been completed by calling MPI_Wait().


It is important to note that you can use any combination of blocking and non-blocking receives! 

Concept: Race Condition

electronics 
Computing
Filesystem

http://en.wikipedia.org/wiki/Race_condition

Summary

• Use the calls MPI_Isend() or MPI_Irecv() to initiate a non-blocking transfer.
• Always include an MPI_Wait() after a non-blocking call. The buffer you specified in the send or receive doesn't contain modifiable (in the case of sending) or even readable data (in the case of receiving) until this call completes.
• You can use a non-blocking call with a blocking call. Most of the example code does this.
• Be very careful about race conditions in the data. Don't write to data that is being sent, and don't read or write to data that is being received.