Course homepage for Optimisation Algorithms in Statistics II (PhD course Spring 2021; 3.5 HEC)Summary
Based on the topics discussed in first part of the course, we continue with deepening the theoretical basis for stochastic optimisation algorithms. Specifically, we discuss theory around stochastic gradient ascent (including momentum and adaptive step sizes), simulated annealing, and particle swarm optimisation. Theoretical results on convergence and speed will be discussed.
We will also discuss some important optimisation algorithms which were not included in the first part. Here, quasi-Newton methods and genetic algorithms will be discussed.
We will use again implementation with R. Examples from machine learning and optimal design will illustrate the methods.
Most welcome to the course!
Lectures: March 23; Time 10-12, 13-15 (registered participants will receive link to all lectures).
Example code: no new example code, but note that there was an example code for steepest ascent in the first part of the course which might be a basis for Problem 1.3. Also a little program for log likelihood of logistic regression and its derivative was posted in the first part; it included the dataset used in Problem 1.1.
Assignment 1 (Deadline April 12).Topic 2: Particle swarm optimisation and stochastic gradient descent with momentum
Lectures: April 13; Time 10-12, 13-15.
Example code: PSO_order1_stability.r
Assignment 2 (Deadline April 26).Topic 3: Simulated annealing and genetic algorithms
Lectures: April 27; Time 10-12, 13-15.
Reading (will be updated):
Assignment 3 will be posted here (Deadline May 11).
The course is intended for Ph.D. students from Statistics or a related field (e.g. Mathematical Statistics, Engineering Science, Quantitative Finance, Computer Science). Previous knowledge in the following is expected:
The course is graded Pass or Fail. Examination is through three individual home assignments.Course literature
We will not use a central course book. Several articles, book chapters and other learning resources will be recommended.Course structure
The topics will be discussed during three online meetings with Zoom. Course participants will spend most of their study time by solving the problem sets for each topic on their own computers without supervision. The course will be held in March and April 2021.Course schedule
To register for the course, please send an email to me (frank.miller at stat.su.se) until March 9, 2021. You are also welcome for any questions related to the course.