• Here we have used the \( MPI\_Barrier \) function to ensure that that every process has completed its set of instructions in a particular order.
• A barrier is a special collective operation that does not allow the processes to continue until all processes in the communicator (here \( MPI\_COMM\_WORLD \)) have called \( MPI\_Barrier \).
• The barriers make sure that all processes have reached the same point in the code. Many of the collective operations like \( MPI\_ALLREDUCE \) to be discussed later, have the same property; that is, no process can exit the operation until all processes have started.

However, this is slightly more time-consuming since the processes synchronize between themselves as many times as there are processes. In the next Hello world example we use the send and receive functions in order to a have a synchronized action.