Parallel Programming: Options and Design

Prerequisites

This course is part of the Scientific Computing series.

This is a detailed overview of using parallelism for achieving more computation in the same amount of elapsed time, covering both "shared memory" and "distributed memory" designs. It will concentrate on principles rather than details, to help attendees make the right decision and proceed in the right direction. It is aimed at users with significant programming experience who need more computation power than can be delivered by a single thread of execution on a single CPU core, including those who already program in parallel. The course is also designed for system administrators who need to support parallel codes, by describing the range of practical options, and their strengths, weaknesses, configuration issues and other important issues.

Significant programming experience with production code, OR a moderate amount of both programming and system administration experience. Participants should have attended "Building Applications out of Several Programs", and need to use more advanced facilities.

Number of sessions: 2

• Reasons for needing parallelism and basic design
• Thread pools, client-server, CPU farms etc.
• HPC parallelism - vector, SPMD and message passing
• SIMD, MIMD, SPMD, NUMA and other terminology
• MPI, PVM, message passing and data distribution
• Shared memory, POSIX/Microsoft threads, OpenMP,
• SHMEM, BSP, Coarray Fortran and UPC
• Synchronisation, atomicity and scheduling

A presentation, followed by questions and discussions.

• The principles are equally applicable to Microsoft Windows and most other systems, but the course will be in terms of Unix. Attendees should know enough Unix to understand its concepts and terminology.
• More information may be found in http://www-uxsup.csx.cam.ac.uk/courses/Parallel

Two half day sessions

• Programming and Scripting
• Scientific Computing