Current postgraduate taught students
COMP60031: High Performance Computing in Science and Engineering (2007-2008)
Today's highest performance computers embody substantial amounts of parallel hardware, to the extent that the latest generation of machines harness the power of thousands of cooperating processors. The programming of such highly parallel hardware has proved to be difficult: progress has been slow and achieved mostly by "trial-and-error".
This course unit studies the base technologies for HPC and allows ``hands-on'' experience of a state-of-the-art parallel supercomputer to be gained. The course unit explores, through a combination of directed reading, lectures, group-based laboratories and group-based mini-projects, a framework for the development, analysis and performance tuning of parallel algorithms for the solution of numerical problems.
On completion of this course unit, students will:
have an understanding of the different levels of abstraction in HPC Modelling and of different parallel programming models; (A)
have an understanding of parallel performance overheads and of techniques for reducing them; (A)
be able to implement a moderately complicated application in a parallel language; (B and C)
have an understanding of some parallel numerical algorithms; (A)
be able to work effectively as a member of a group to develop parallel applications. (D)
Assessment of Learning outcomesLearning outcomes (1) and (2) are assessed by examination, in the laboratory and via the mini-project, learning outcomes (3) and (5) are assessed in the laboratory and via the mini-project, and learning outcome (4) is assessed by examination.
Introduction to HPC
Why is HPC important in Science and Engineering? Introduction to Parallel Computers and Computational Overheads.
Levels of Abstraction, Models of Computation and Parallel Overheads
Levels of Abstraction, Multiple Program Counters in Hardware; Multi-Thread Models, with Primary Sources of Overhead; Parallel Languages and Compilers; Task-Parallel versus Data-Parallel Programming Models; Further Sources of Overhead; Experimentation and Presentation of Results; Memory Architecture and Memory Access Times and Associated Sources of Overhead; Multi-Process Execution Model; Performance Tuning via Overhead Reduction; Task Scheduling; Data Partitioning and its Effect on Performance.
Restructuring for Parallel Performance
Parallelising Compilers; Loop Transformations; Data Transformations; Dependence Analysis; Compiler Strategies.
Cyclic Reduction; Iterative Algorithms (Jacobi, Gauss-Seidel and Red-Black Orderings); Divide-and-Conquer Algorithms, Adaptive Quadrature, Correct Termination.
Review of the course material and the unifying theme of levels of abstraction.
Core TextTitle: Designing and building parallel programs: concepts and tools for parallel software engineering
Author: Foster, Ian
Publisher: Addison Wesley
Supplementary TextTitle: Computer architecture: a quantitative approach (5th edition)
Author: Hennessy, John L. and David A. Patterson
Publisher: Morgan Kaufmann
Supplementary TextTitle: Parallel computer architecture: a hardware/software approach
Author: Culler, David E. and Jaswinder Pal Singh and Anoop Gupta
Publisher: Morgan Kaufmann
Supplementary TextTitle: Performance optimization of numerically intensive codes
Author: Goedecker, Stefan and Adolfy Hoisie