NDAK21006U Data Parallel Programming (DPP)

Volume 2021/2022
Content

Data parallel programming models express parallelism declaratively (explicitly) by means of higher-order language constructs, whose rich semantics allow high-level reasoning for exploring the large space of strategies for efficiently mapping application's parallelism to hardware.  

The aim of the course is to introduce the principles and practice of parallel programming (e.g., programming using multiple hardware cores or processors in order to gain speed) in a declarative programming setting. The course focuses on deterministic programming models that are easy for humans to reason about, yet possible to be compiled to efficient code.  Potential topics of interest range from (i) compilation techniques for nested parallel programs, e.g., flattening, to (ii) locality optimizations, e.g., tiling and fusion of stencil computation, to (iii) supporting low-level data parallel programming (e.g., SPMD programming model), to (iv) optimization frameworks aimed at imperative loops with affine indexing.

The course includes current research on these topics and relies heavily on scientific papers as its source materials. The course will demonstrate the presented parallelisation strategies on applications from various domains, such as machine-learning, image-processing and finance. 

The lectures will provide an overview of approaches to parallel programming and associated analysis and code-generation techniques, and give practical instructions to writing, testing, and optimising data-parallel programs. The topics covered in the lecture will be exercised in lab assignments, consisting of programming and analysis of programs as well as questions for theoretical discussion.

Learning Outcome

Knowledge of

  • the difference between the concepts of concurrency and parallelism, and between data parallelism and task parallelism
  • strategies for optimizing parallelism and locality, programming patterns
  • different approaches to parallelism taken in various languages, with particular focus on how high-level description of parallelism may be mapped in a principled way to high-performance hardware.
     

Skills to

  • express a parallel computation in data-parallel paradigms
  • write, modify, optimize and test data-parallel programs, in different programming environments, targeting different architectures such as multi-core CPUs and GPGPUs

 

Competences to

  • identify opportunities for using data-parallel programming to parallelise algorithms
  • select a suitable programming language/dialect to implement a parallel algorithm on a given hardware platform
Literature

The course does not use a single textbook but instead provides tutorials and scientific papers available from the course pages.

The course syllabus assumes basic knowledge and programming competences in a functional programming language, which, at DIKU, can be acquired through ”Advanced programming”, or through self-study.

Academic qualifications equivalent to a BSc degree is recommended.
Lectures, in-class exercises, group work on programming and analysis assignments.
The course is equivalent to Parallel Functional Programming (PFP) NDAK14009U. It is not allowed to pass PFP and take Data Parallel Programming (DPP).

The course material is intended to present synergies with the material of Programming Massively-Parallel Hardware (PMPH). This means that there are connecting elements with said courses, but they are not pre-requisites for taking this course.

The course is identical to the discontinued course NDAK14009U Data Parallel Programming (DPP). Therefore you cannot register for NDAK21006U - Data Parallel Programming (DPP), if you have already passed NDAK14009U Data Parallel Programming (DPP).
If you are registered with examination attempts in NDAK14009U Data Parallel Programming (DPP) without having passed the course, you have to use your last examination attempts to pass the exam in NDAK21006U - Data Parallel Programming (DPP). You have a total of three examination attempts.
  • Category
  • Hours
  • Lectures
  • 28
  • Preparation
  • 15
  • Exercises
  • 60
  • Laboratory
  • 14
  • Project work
  • 83
  • Exam Preparation
  • 5
  • Exam
  • 1
  • Total
  • 206
Written
Oral
Individual
Collective
Continuous feedback during the course of the semester
Credit
7,5 ECTS
Type of assessment
Continuous assessment
Continuous assessment based on 3-4 individual assignments (that account for 40% of the final grade) and a group mini-project with individual oral defense 20-25 minutes (that account for 60% of the final grade).

The part-examinations must be individually approved. The final grade is based on an overall assessment.
Aid
All aids allowed
Marking scale
7-point grading scale
Censorship form
No external censorship
Several internal examiners
Re-exam

Resubmission of (individual) assignments and (group) mini-project and a 30 minutes oral examination without preparation in full course syllabus.

The part-examinations must be individually approved. The final grade is based on an overall assessment.

Criteria for exam assesment

See Learning Outcome.