NSCPHD1297 Internationalt PhD-kursus i Informationsgeometri
International PhD course on Information Geometry
Subject area: Information Geometry
In Information Geometry the space of probability distributions that
can be represented by a parametrized probabilistic model is
described as a manifold, on which the Fisher information metric
defines a Riemannian structure. Through the geometry of the
Riemannian manifold of distributions, optimization and statistics
can be done directly on the space of distributions. This ability to
do statistics on probability distributions and sample from the
space of probability distributions is extremely valuable in machine
learning and optimization.
Information geometry was founded and pioneered by Shun'ichi
Amari in the 1980s, with statistical learning as one of the first
applications. Due to the nonlinear nature of the space of
distributions, the steepest ascent direction for adapting a
probability distribution parametrized by a set of real-valued
parameters (e.g., the mean and the covariance of a Gaussian
distribution) is not the ordinary gradient in Euclidean space, but
the so called natural gradient, defined with respect to the
Riemannian structure of the space of distributions. The natural
gradient is natural in the sense that it renders the adaptation
invariant under reparametrization and changing representations, and
it is closely linked to the Kullback-Leibler divergence often used
for quantifying the similarity of distributions.
The natural gradient for adapting probabilistic models has been
successfully used in all major areas of machine learning, from
supervised learning of neural networks over independent component
analysis to reinforcement learning. In this PhD course there will,
in particular, be lectures on supervised learning, reinforcement
learning andstochastic optimization. Reinforcement learning refers
to machine learning algorithms that improve their behavior based on
interaction with the environment, whereas stochastic optimization
refers to stochastic solutions to complex optimization problems for
which we do not have an analytical description. Both in stochastic
optimization and reinforcement learning, (intermediate) solutions
are best described by probability distributions. In the one case,
we consider distributions over potential actions to be taken in a
certain situation. In the other case, we consider the search
distribution describing which candidate solution to probe next.
Thus, both the learning as well as the optimization process are
best described by an iterative update of probability distributions.
Scientific content
The course will consist of 5 days of lectures and exercises. In
addition, students will be expected to read a pre-defined set of
scientific articles on information geometry prior to the course,
and write a report on information geometry and its potential use in
their own research field after the course.
The course will consist of three modules:
1. A crash course on Riemannian geometry and numerical tools for
applications of Riemannian geometry
2. Introduction to Information Geometry and its role in Machine
Learning and Stochastic Optimization
3. Applications of Information Geometry
After participating in this course, the student should
* Understand basic differential geometric concepts (manifolds,
Riemannian metric, geodesics, manifold statistics) to the point
where they can apply differential geometric concepts in their own
research
* Be able to implement basic numerical tools for differential
geometric computations
* Have a strong knowledge of information geometry and its role in
machine learning and stochastic optimization
* Be able to apply information theoretic approaches to machine
larning and stochastic optimization in their own research
* Have a basic knowledge of existing applications of information
geometry
- Kategori
- Timer
- Eksamen
- 15
- Forelæsninger
- 40
- Undervisningsforberedelse
- 10
- I alt
- 65
- Point
- 2,5 ECTS
- Prøveform
- Skriftlig aflevering
- Censurform
- Ingen ekstern censur
Kursusinformation
- Sprog
- Dansk
- Kursuskode
- NSCPHD1297
- Point
- 2,5 ECTS
- Niveau
- Ph.d.
- Varighed
- Placering
- Efterår
- Skemagruppe
- 5 days plus individual work before and after, see attached schedule.
- Studienævn
- Ph.d.-studienævn SCIENCE
Udbydende institut
- Datalogisk Institut
Kursusansvarlige
- Christian Igel (4-716f6d74486c7136737d366c73)
- Aasa Feragen (4-69697b69486c7136737d366c73)