MVE187 / MSA101 Computational methods for Bayesian statistics Autumn 21

MVE187 / MSA101 Computational methods for Bayesian statistics Autumn 21

Course PM

This page contains the program for the course. Other information, such as learning outcomes, teachers, literature, examination, and old exams is in a separate course PM.

Program

The schedule is in TimeEdit and in the table below. Note that in the fourth week the second lecture is on Tuesday instead of Wednesday.

This time the lectures will be given in a hybrid format: You will be able to attend the live lecture, you may watch it live on zoom, or you may watch the recorded lecture later. This format is new to me, so feedback on how it works for you is very welcome. You should study the reading material listed in the schedule below before each lecture.

The recorded lectures from the 2020 edition of the course are available here, but note that there are some contents changes and some scheduling changes in the course.

There is an exercise session / computer lab each week. They will be mostly driven by student questions. You may choose to attend in person, in MVF22, or via Zoom. (We will try out this mixed format, and we will change it if it works poorly).

The second main part of the course is the obligatory assignments. There are three, and answers are submitted individually, although you are allowed to cooperate in small groups. The deadlines are in the third, fifth and seventh course weeks. Each assignment is based on material discussed at least one week before each deadline.

Guide for reading the textbooks:

  • The sections from Bishop selected in the specification of the study material below are relevant.
  • All chapters in Albert are relevant to some extent. Often concepts from the lectures are explained in more detail (and better?) with good examples. However, concepts in Albert that do not appear at all in the lectures are not important for the course. Also, Albert has a lot of focus on using R functions from the R package LearnBayes (written by Albert) that implement various Bayesian computations. Using such pre-written R functions may be very helpful for some students but for others it obscures what is going on. In our course we will not focus on using these functions.
  • Chapters 2, 3, 6, 7 and section 5.4 of Robert and Casella are relevant. Again, there are many good examples, but concepts that only appear in these chapters and not in the lectures are not important for the course.

Overview of course contents

 

Schedule

Contents Study material
Time and place
Lecture 1: Introduction to the Bayesian paradigm. Issues with classical frequentist inference.

Overheads

Literature: B2.1-B2.4 (meaning chapter 2 sections 1 through 4 from Bishop; see literature list). A2, A3, A4 (chapters 2, 3, 4 from Albert).

Make sure you know how to use R! See A1, or RC1, or any other introductory text to R. (RC means Robert and Casella)

Monday 2021-08-30, 15:15-17:00, in Pascal, or watch on zoom. RECORDING
Lecture 2: Basic computations using conjugacy. The exponential family of distributions.

Overheads

B2.1-B2.4. A2, A3, A4.

Wednesday 2021-09-01, 10:00-11:45, in ED, or watch on zoom. RECORDING(Note: there are two rooms called ED (..!..) We are in the one on the 5th floor of the EDIT building.)
Exercises / computer lab

Please have a look at exercises on beforehand:

EXAM 2019-01-08 question 1. A1.6 exercise 4.

Some exercises

A2.9: 1,4,5; A3.9: 1,3,(4). (A4.8 1,4,7).

R-code for some solutions.

Thursday 2021-09-02, 15:15-17:00, in MVF22 or join on zoom   RECORDING
Lecture 3: Using discretization. Mixtures. Some multivariate conjugacies.

Overheads

B2.1-B2.4. A2, A3, A4.
Monday 2021-09-06, 15:15-17:00, in Pascal, or watch on zoom. RECORDING. R-code used. 
Lecture 4: Inference by simulation. Monte Carlo integration. Basic simulation methods. Rejection sampling.

Overheads

B11.1 (until B11.1.6); B11.4; A5; RC2, RC3. 
Wednesday 2021-09-08, 10:00-11:45, in ED, or watch on zoom. RECORDING. whiteboards, Rcode
Exercises / computer lab

 

Exercises: EXAMS: 2019-08-29 q1;
2019-01-08 q4; 2018-10-27 q1; 2018-01-02 q3; 2017-10-21 q1; Exercises: RC 2.11, 2.12, 2.18, 2.22. EXAMS: 2019-08-29 q2; 2019-01-08 q2ab, 2018-01-02 q2, 2017-10-21 q2. 2020-08-27 q2.
Thursday 2021-09-09, 15:15-16:00 online via zoom, 16:15-17:00 in MVF22. Solutions for RC exercises (Note: All exam questions have suggested solutions on Canvas).
Lecture 5: Importance sampling and SIR. Laplace approximation. Introduction to Markov chain Monte Carlo (MCMC) methods.

Overheads

B11.1.4-5; B11.2.1-2; A5.9-10

Monday 2021-09-13, 15:15-17:00, in Pascal, or watch on zoom. RECORDING. R-code. Whiteboards
Lecture 6: MCMC. Random walk. Independent proposal. Convergence; checking convergence. Burn-in. Smart proposals.

Overheads

A6. For information: RC6-8

Wednesday 2021-09-15, 10:00-11:45, in ED, or watch on zoom. RECORDING. R-code
Exercises /computer lab

I will go through solutions for the following exercises live starting at 15:15: 

A5:13 exercises 1, 5

A6:13 exercises 1, 2

Further exercises:

EXAMS: 2020-01-08 q3; 2019-10-28 q2; 2018-10-27 q2; 2018-01-02 q8; 2020-08-27 q5; 2018-01-02 q5.

Thursday 2021-09-16, 15:15-17:00 online via zoom or in MVF22. First part: I show solutions live, NO recording. R solutions Bild1 Bild 2 Second part: Individual help.
Deadline first assignment 

See below

Friday 2021-09-17, 16:00
Lecture 7: Gibbs sampling. Hierarchical models. Slice sampling. Tips and tricks. Convergence.

 Overheads

B11.3, 11.4. A6. For information: RC6-8

Monday 2021-09-20, 15:15-17:00, in Pascal, or watch on zoom. RECORDING. R-code
Lecture 8: Hidden Markov models (HMM) and state space models (SSM). Kalman filters.

Overheads

B13.2, 13.3

TUESDAY 2021-09-21, 10:00-11:45, in ED, or watch on zoom. RECORDING. R-code used

NOTE THAT THIS LECTURE IS ON TUESDAY AND NOT ON WEDNESDAY!

Exercises / computer lab

I will go through solutions for the following exercises: A5.13: 2a;  A6.13: 3;   A10.7 6.  EXAM 2019-08-29 q5;

Additional exercises:

A9.7: 3,4;    EXAMS 2020-01-08 q4; 2019-08-29 q8; 2018-01-02 q7; 2017-10-21 q3.

Thursday 2021-09-23, 15:15-17:00 online via zoom or in MVF22. First part: I show solutions live, NO recording. R-code. Second part: Individual help.

Lecture 9 (Lecturer Umberto Picchini): Sequential Monte Carlo (particle filters) and particle MCMC for state-space models

Overheads (static) (for easier browsing)

Overheads (dynamic) (shows the particle filter animation but it is otherwie the same as the "static" version)

Chapter 7 in Särkkä

Review by Creel

RECORDING

Lecture 10 (Lecturer Umberto Picchini): More on particle MCMC and exact sampling using pseudo-marginal methods.

Overheads

Tutorial paper which comes with an R package

Optional exercises

RECORDING

Exercises / computer lab

Individual help

Thursday 2021-09-30, 15:15-17:00 online via zoom or in MVF33. Individual help.

Deadline second assignment

See below

Friday 2021-10-01, 16:00

Lecture 11: Missing data / augmented data. Hamiltonian MCMC.

Overheads

B11.5; A7; RC7.

Monday 2021-10-04, 15:15-17:00, in Pascal, or watch on zoom. RECORDING. R-code

Lecture 12: Some information theory. The EM algorithm.

Overheads

Literature: B1.6, 9.4 (9.2, 9.3); RC5.4.

Wednesday 2021-10-06, 10:00-11:45, in ED, or watch on zoom. RECORDING part 1RECORDING part 2. R code (with some problems)

Exercises / computer lab

I will go through solutions for the following exercises:

A9.7: 6;    A10.7: 5, 7.

Additional exercises:

Exercises: EXAMS 2020-08-27: q3, q7; 2017-10-21 q4, q5; A7.13: 1,2. EXAMS 2020-01-08 q5; 2019-10-28 q4; 2019-08-29 q3, q6; 2019-01-08 q3, q6; 2018-10-27 q5; 2018-01-02 q6; 2017-10-21 q6.  

Thursday 2021-10-07, 15:15-17:00 online via zoom or in MVF33. R code shown

Lecture 13: Variational Bayes

Overheads

Literature: B10.1 (B10.2)

Monday 2021-10-11, 15:15-17:00, in Pascal, or watch on zoom. RECORDING. R code used

Lecture 14: Graphical Models.

Overheads (with fixed error)

B8 excluding 8.4.3-8.4.8.

Wednesday 2021-10-13, 10:00-11:45, in ED, or watch on zoom. RECORDING. WB1 WB2 WB3 WB4 WB5 WB6 WB7

Exercises / computer lab

I will go through solutions for the following questions:

EXAM 2019-10-28 q6;  2020-08-27 q4;

Additional questions:

EXAM 2019-10-28 q5; 2019-08-29 q4; 2018-10-27 q3, q4, q6. 2018-01-02 q4.

Thursday 2021-10-14, 15:15-17:00 online via zoom or in MVF33. First part: I show solutions live, NO recording. Second part: Individual help.

Deadline third assignment

See below

Friday 2021-10-15, 16:00

Lecture 15: Applied Bayesian modelling.

Overheads.

Literature: A7, A8.  Paper and corresponding overheads.

Monday 2021-10-18, 15:15-17:00, in Pascal, or watch on zoom. RECORDING wb1 wb2

Review

Everything

Wednesday 2021-10-20, 10:00 - 11:45, in ED, or watch on zoom. RECORDING. wb1 wb2 wb3 wb4

Question time

Everything

Thursday 2021-10-21, 15:15-17:00 online via zoom or in MVF33.

 

Back to the top

Course work

To understand and learn the methods of this course, it is essential to work with examples on a computer, in addition to working with the study material and doing theoretical exercises. Our textbooks contain a large number of exercises, for both theoretical and computer solutions, and some exercises are listed above.

As an obligatory part of the course, each student must do 3 assignments. The deadlines for these are Friday 17 September 16:00, Friday 1 October 16:00, and Friday 15 October 16:00. Details about the assignments will be available from the links at the bottom of this page. Answers must also be handed in via Canvas. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.

As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.

If you have problems with getting started with the course, or for example getting started with R, please do not hesitate to contact me. I hope that we can have active communication during the course. In addition to contact at lectures and exercise sessions, you may contact me on canvas or by mail at mostad@chalmers.se. I usually answer within a day or so.

Back to the top

Course summary:

Date Details Due