PParallelProgramming/29_apr.md

general info

Register until may 23 for an exam on campus portal

Sequential (turing-full):

• assignment x := E
• composition: S1, S2
• choice: if
• iteration: loops, reqursions, etc

Parallel

• initiation and termination
  • unsructured: fork / join
  • structured: cobegin / coend
• synchronisation
  • supervises the computational progress of individual processes
  • mitigates non determinism caused by parallelism
• communication
• orientation

libraries to use [Foster, ch.8]

MPI mpodel

• SPMD
• can be used as MIMD
  • all processes get the code for all processes
• point-to-point communications
• more anstract communication patterns
• checking for messages
• sub-groups of processes

The 6 basic functions

int MPI_Init(int *argc, char ***argv);
// turn on mpi

int MPI_Finalize();
// turn off mpi

int MPI_Comm_size(MPI_Comm comm, int *size);
// in: comm - communicator
// OUT size

int MPI_Comm_rank(MPI_Comm, int *pid);
// data is written into size / pid / etc

int MPI_Send(void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm);
// buf - adress of the buffer
// int count - number of elements to send
// datatype - datatype of elements in buffer
// dest - id of the receiving process
// tag - additional mark on the message to be send
// comm - communicator

int MPI_Recv(void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status* status);
// OUT status - usually ignore, but you can add more persice error handler

function	blocking	non-blocking	sender
synchronous	MPI_Ssend	MPI_Issend	completes together with receiver (danger of deadlocks)
buffered	MPI_Bsend	MPI_Ibsend	completes when message has been buffered (no deadlocks; explicit buffer registration/freeing necessary)
standard	MPI_Ssend	MPI_Isend	synchronous or buffered (implementation decides)
ready	MPI_Rsend	MPI_Irsend	reciever is already waiting (can reduce handshake overhead)

int MPI_Wait(MPI_Request *request, MPI_Status *status);

int MPI_Waitall(int count, MPI_Request reqests[], MPI_status *statuses);

int MPI_Test(MPI_request *request, int *flag, MPI_Status *status);

int MPI_Barrier(MPI_Comm comm); // wait for all processes to reach this point
int MPI_Bcast(void *inbuf, int incnt, MPI_Datatype datatype, int root, MPI_Comm comm); // broadcast a message from one process to all others
// INOUT: inbuf - address of the input for root and output for others
// IN: incnt - number of elements to send
// IN: datatype - datatype of elements in buffer
// IN: root - id of the sending process
// IN: comm - communicator
int MPI_Scatter(); // send a message to all processes
int MPI_Gather(); // gather messages from all processes