A barrier is a synchronization technique that helps coordinate the progress and synchronization of several threads by ensuring that they reach a predetermined point in their execution, known as the “barrier point.”
Barrier Synchronization: Threads or processes in a barrier must halt at a synchronization point and cannot continue until all other threads/processes have done the same. This is beneficial in cases where tasks are divided among threads and subsequent operations depend on completing all preceding ones.
Implicit Barriers: Some parallel programming frameworks (such as OpenMP in Fortran) include implicit barriers. For example, with OpenMP, following a parallel do loop, the application will not resume execution on any thread until all loop iterations have been completed.
Barriers in Message Passing: Certain collective operations (such as reduction or scatter) in message-carrying systems may suggest a barrier. This ensures all communication and computation associated with these actions are completed before proceeding to the next task.
Latch versus Barrier: A latch is a specialized type of barrier. It starts in the “raised” stage, where threads can pass through. The latch is “lowered” once a predetermined number of threads have passed, and no more threads can enter or raise the latch. A count-down latch is similar, except it automatically lowers once a predetermined number of threads/processes have arrived.