NDAA09009U Numerical Optimisation (NO)

Volume 2026/2027
Education

MSc Programme in Bioinformatics

MSc Programme in Computer Science

MSc Programme in Physics

MSc Programme in Statistics

MSc Programme in Mathematics-Economics

MSc Programme in Actuarial Mathematics

Content

Numerical optimisation is a useful computer tool in many disciplines like image processing, computer vision, machine learning, bioinformatics, eScience, scientific computing and computational physics, computer animation and many more.  This course will build up a toolbox of numerical optimisation methods which the student can use when building solutions in his or her future studies.

This course teaches the basic theory of numerical optimisation methods. The focus is on deep understanding, and how the methods covered during the course works. Both on a theoretical level that goes into deriving the math but also on an implementational level focusing on computer science. A special focus of the course lies on empirical evaluation of the different methods and communication of the results in report form. As a result the course is very practical and there will be weekly group-based programming exercises.

During the course, we will start from the simple gradient descent algorithm and introduce more ideas to improve on this simple approach to create algorithms that are fast and reliable. The topics covered during the course are:

  • First-order optimality conditions, Karush-Kuhn-Tucker Conditions, Taylors Theorem, Mean Value Theorem.
  • Nonlinear Equation Solving: Newtons Method, etc.
  • Linear Search Methods: Newton Methods, Quasi-Newton Methods, etc.
  • Trust Region Methods
  • And many more...
Learning Outcome

Knowledge of

  • The theory of convex and non-convex optimisation
  • The theory of Newton and Quasi-Newton Methods
  • The theory of Trust Region Methods
  • First-order optimality conditions (KKT conditions)

 

Skills in

  • Applying numerical optimisation problems to solve unconstrained and constrained minimisation problems and nonlinear root search problems
  • Implementing and testing numerical optimisation methods

 

Competences to

  • Evaluate which numerical optimisation methods are best suited for solving a given optimisation problem
  • Understand the implications of theoretical theorems and being able to analyse real problems on that basis

See Absalon

The programming language used in the course is Python. It is expected that students know how to install and use Python, Numpy, Scipy and Matplotlib by themselves.

It is expected that students know what matrices and vectors are and that students are able to differentiate vector functions.

Theorems like fundamental theorem of calculus, mean value theorem or Taylor's theorem will be touched upon during the course. The inquisitive students may find more in depth knowledge from Chapters 2, 3, 5, 6 and 13 of R. A. Adams, Calculus, 3rd ed. Addison Wesley.

Academic qualifications equivalent to a BSc degree is recommended.
Mixture of study groups and project group work with weekly assignments and small lectures.
The focus is on flipped-classroom teaching.
  • Category
  • Hours
  • Lectures
  • 10
  • Preparation
  • 40
  • Exercises
  • 72
  • Project work
  • 84
  • Total
  • 206
Written
Oral
Individual
Continuous feedback during the course of the semester
Credit
7,5 ECTS
Type of assessment
Oral exam on basis of previous submission, 15 minutes (no preparation time)
Type of assessment details
The submission consists of weekly assignments. The student must submit 6 out of 7 weekly assignments completed as group work.

The oral examination is based on both the full curriculum and on the submitted weekly assignments. Students must be able to discuss key theoretical concepts as well as implementation details related to the methods covered in the weekly assignments and questions based on the curriculum.

Timely submission of the 6 required weekly assignments is a prerequisite for participation in the oral examination.

The final grade is determined based on an overall assessment.
Aid
Only certain aids allowed (see description below)

Print-outs of the students' own submitted weekly assignments.

Marking scale
7-point grading scale
Censorship form
No external censorship
Several internal examiners
Re-exam

Same as the ordinary exam.

Students who have not submitted the 6 required group assignments must submit individual assignments. The student has the option to resubmit previous group work, to fulfil the requirement of submitting the 6 assignments.

The missing assignments must be submitted no later than two weeks before the reexamination.

Criteria for exam assesment

See Learning Outcome