CISC 709 Contemp Comput Sys Programming

This course discusses and advocates a structured approach to parallel programming. This approach is based on a core set of common and composable patterns of parallel computation and data management with an emphasis on determinism and scalability. By using these patterns and also paying attention to a small number of factors in algorithm design (such as data locality), programs built using this approach have the potential to perform and scale well on a variety of different parallel computer architectures. A special emphasis will be put on both collective “data-parallel” patterns as well as structured “task-parallel” patterns such as pipelining and superscalar task graphs. The structured patter-based approach, like data-parallel models, addresses issues of both data access and parallel task distribution in a common framework. Optimization of data access is important for both many-core processors with shared memory systems and accelerators with their own memories not directly attached to the host processor. Extensive use of pertinent and practical examples from scientific computing will be made throughout. The programming languages used will be Python, Fortran, or C++. Both the shared and distributed paradigms of parallel computing will be covered via the OpenMP and MPI libraries.