NSCPHD1299 Internationalt PhD-kursus i Deep Learning

Subject area: Deep Learning

One of the main challenges in medical image analysis, computer vision, and machine learning is to discover useful data representations (or features). A common approach has been to manually define features based on prior knowledge about the problem or by selecting representations from the literature, (e.g., Gaussian derivative features, Wavelets, or SIFT). However, manually defining or selecting features is a time-consuming process and might not generalize to the problem at hand. In recent years, a lot of research has been devoted to learning representations directly from the data. In particular, “deep learning” methods have been developed, which seek to learn multi-layer representations of the input data.

”Deep learning” refers to the recent wave of neural networks that attempt to learn a hierarchy of features of increasing complexity. This contrasts to the standard 'shallow' approach where hand-crafted features are combined with a machine learning algorithm for the task at hand. Deep learning tries to factorize the input and organize it in layers where the factors at one level are transformed and used to to obtain the factors at the next level. The goal is to learn input representations at each level such that higher level factors correspond to more complex abstractions in the input. The relationship between layers can be exemplified by the functioning of the human vision where e.g. a face (high level) consists of a nose (medium level) and that nose is constructed from some edges (low level). By distributing computations across multiple units at multiple layers, deep learning architectures are able to scale to large data and fight the curse of dimensionality by reusing lower-level concepts in higher-level units.

Deep learning builds on top of the artificial neural networks of the late 80's. The main innovation since then is in the design of new network types and a better insight in how to optimize the non-convex training functions of the many parameters of such networks. Applications of deep learning include numerous problems in vision, speech, audio, or natural language processing. The best performing contenders on several benchmark datasets are deep leaning methods, and also mayor companies like Google, Microsoft, Yaihoo, or Baidu are exploiting deep learning in their research and products.


Scientific content

The summer school will consist of 5 days of lectures and exercises. The students will be expected to read a predefined set of scientific articles on deep learning methods prior to the course. Additionally, the students should bring a poster presenting their research field (preferably with an angle towards deep learning).

The course will consist of the following parts:
- A crash course on neural networks and their implementation.
- A theoretical insight in the challenges of designing and training neural networks.
- A practical session with hands-on exercises.
- Applications of deep learning.