Computational Physics Lectures: How to optimize codes, from vectorization to parallelization

 

 

 

Parallelizing nested for-loops

  • Serial code

for (i=0; i<100; i++)
    for (j=0; j<100; j++)
        a[i][j] = b[i][j] + c[i][j];
    }
}
  • Parallelization

#pragma omp parallel for private(j)
for (i=0; i<100; i++)
    for (j=0; j<100; j++)
       a[i][j] = b[i][j] + c[i][j];
    }
}
  • Why not parallelize the inner loop? to save overhead of repeated thread forks-joins
  • Why must j be private? To avoid race condition among the threads