Stochastic differential equations and deep generative models

Overview

This is a 5 credit PhD course, which welcomes students from all of Chalmers. Since it is a PhD course, the only possible grades are pass or fail. We are currently developing the course and the information below is subject to change. 

Teachers

Lennart Svensson, lennart.svensson@chalmers.se
Fredrik Kahl, fredrik.kahl@chalmers.se
Moritz Schauer, smoritz@chalmers.se

Aim

The purpose of this course is to give an introduction to stochastic differential equations (SDEs) and how they can be used to derive deep generative diffusion models. The family of deep diffusion models has demonstrated remarkable results in various applications. We focus primarily on the theoretical foundations of SDEs and deep diffusion models, emphasizing the SDE concepts important to understanding score-based and more recently proposed diffusion models. 

Content

The following is a summary of the course content:

• Brownian motion and its properties
• Itô calculus, including Itô integrals and Itô's lemma
• Stochastic differential equations, Euler-Maruyama, and solutions to simple examples
• Forward, backward, and reverse-time Kolmogorov equations
• Doob's H-transform and Girsanov's theorem
• Score-based diffusion models
• A selection of more advanced topics related to diffusion-based generative models
  
  

Schedule 

Below is a brief preliminary description (titles) of the lectures (Lx) where we also list the responsible teacher and the date. Lectures take place on Mondays and Wednesdays 10-12, except for L3 which starts at 9.00. You can import the calendar using this link. 

• L1: course introduction and a primer on relevant concepts for stochastic processes, Lennart Svensson, 31/3, room ED.
• L2: Brownian motion, stochastic differential equations, and some of their properties, Lennart Svensson, 2/4, room EE.
• L3: Itô calculus, Lennart Svensson, 7/4, 09.00-10.45, room EE. Note that we start an hour earlier.  
• L4: Forward, backward, and reverse Kolmogorov, Fredrik Kahl, 9/4, room ED
• L5: Score-based diffusion models, Fredrik Kahl, 23/4, room EE. 
• L6: Doob's H-transform and Girsanov's theorem, Moritz Schauer, 28/4, room ES51. 
• L7: Operator theory and the connection between the content of L6 and L5, Moritz Schauer, 30/4, room EF. 
• L8: Possibly a lecture on more advanced topics, all teachers, 5/5, room ED. 

Apart from this, there will be paper presentations by the students from 12/5 to 16/5. All rooms are located in the EDIT building, see https://maps.chalmers.se/ for directions. 

Learning objectives

• understand and apply the main results and techniques of stochastic calculus and stochastic differential equations in the setting of diffusion models,
• understand and apply deep diffusion models in several common applications. 

Literature and approach 

The course will use selected parts of various books and papers. The first three lectures are mostly based on Chapters 3 and 4 in 

Jazwinski, Andrew H. Stochastic processes and filtering theory, Academic Press, 1970.

The book is available online. Note that the book does not present the material using measure theory, and we will prioritize intuition instead of rigor, but without sacrificing rigor entirely. 

Examination 

Students are evaluated based on three home assignments (based on L1-3, L4-5, and L6-7, respectively) and the final paper presentations. The home assignments are submitted individually and then peer-reviewed by another student. Paper presentations are performed individually or in groups of two, depending on the complexity of the paper. 

Prerequisites

Students are expected to have a solid background in calculus, statistics, and programming. 

Course evaluation

After the course, we will send out an individual, anonymous survey. The results and suggested updates will be shared with the students who participated in the course. 

Registration 

Please register using https://forms.office.com/e/YN20H6j6W1 no later than 2025-03-10.