NMAK17008U  Sparse Learning

Volume 2017/2018
Education

MSc programme in Statistics

Content

​A sparse statistical model is one having only a small number of nonzero parameters. Examples of models and problems that will be considered in the course are: regression models, matrix decompositions and Gaussian graphical models.

The theme of the course is estimation and statistical inference with sparsity inducing penalties. Lasso and its variations are the a main examples. Sparse estimation is often achived via convex optimization, and this theory will also be treated in the course. 

In the course there will be a focus on models and algorithms, and how to apply them to real problems. Some results from the statistical theory will be touched upon, but it will not be a main part of the course. 

Learning Outcome

Knowledge:

  • The lasso family of penalties: Lasso, group lasso, elastic net, fused lasso etc. 
  • Algorithms for convex optimization: Coordinate descent and proximal gradient descent.
  • Convex duality theory.
  • Sparse regression models.
  • Sparse multivariate methods.
  • Sparse matrix decompositions.
  • Sparse graphical models.
  • Signal approximation and compressed sensing.
  • R packages for sparse learning.

 

Skills: Ability to

  • derive and implement standard optimization algorithms for convex penalized estimation.
  • use standard R packages for computing penalized estimators.
  • transform optimization problems between the penalized and constrained formulation.
  • tune methods to optimal performance.

 

Competences: Ability to

  • evaluate if sparse learning will be useful or beneficial for an application.
  • evaluate performance characteristics for algorithms and methods for sparse learning.
  • adapt standard learning methods to sparse learning methods.

 

See Absalon for a list of course literature.

Regression or Machine Learning. The course is targeted toward students in statistics or machine learning at the master's or PhD level.
Four hours of lectures and two hours of exercises per week for seven weeks.
Credit
7,5 ECTS
Type of assessment
Continuous assessment
The course evaluation consists of three individual assignments, which each count 1/3 to the total grade.
Aid
All aids allowed
Marking scale
7-point grading scale
Censorship form
No external censorship
Re-exam

Same as the ordinary exam.

If the ordinary exam is not passed, it is possible to hand in one or more of the three assignments in an improved version. The grade is based on all three assignments – equally weighted.

Criteria for exam assesment

The student must in a satisfactory way demonstrate that he/she has mastered the learning outcome of the course.

  • Category
  • Hours
  • Lectures
  • 28
  • Theory exercises
  • 14
  • Exam
  • 45
  • Course Preparation
  • 119
  • Total
  • 206