Overhead present in parallel computing

  • Uneven load balance: not all the processors can perform useful work at all time.
  • Overhead of synchronization
  • Overhead of communication
  • Extra computation due to parallelization

Due to the above overhead and that certain parts of a sequential algorithm cannot be parallelized we may not achieve an optimal parallelization.