MVE188 Computational methods for Bayesian statistics

Course PM

This page contains the program for the course. Other information, such as learning outcomes, teachers, literature, examination, and old exams is in a separate course PM.

Program

The schedule is in TimeEdit and in the table below.

NOTE: The earlier plan was to change to newly-written teaching material this year. However, this material was not ready in time, so this year will be taught in a similar way as earlier years, while the new teaching material will be used in 2025. In anticipation of the new material, the number of exercise classes / student workshops was increased. We will keep these extra times for now, but some classes may be cancelled if too few students show up.

This year, I will generally NOT make recordings of the lectures. You should study the reading material listed in the schedule below before each lecture. Active students with all kinds of questions are greatly appreciated.

On Wednesdays, there are student workshops: These are generally for individual or group work, with the teacher present to answer questions and assist in any way. On Thursdays, we will together go through lists of exercises, or solutions to assignments.  

An important part of the course is the obligatory assignments. There are two, and answers are submitted individually, although you are allowed to cooperate in small groups. The deadlines are in the third and sixth course weeks. Each assignment is based on material discussed at least one week before each deadline.

Note that, this year, the lectures will not be recorded. All students are encouraged to attend all lectures. However, if you cannot, you may have some benefit from recordings of last year's lectures.

A piazza forum has now been added to the course. Please sign up! If you do not have access to a mail address ending in chalmers.se, you may mail me so that I can add you manually. 

Guide for reading the textbooks:

  • Relevant material from Bishop (B): 1.6, 2.1-2.4, 8.1-8.3, 8.4.1-8.4.3, 9.2-9.4, 10.1, 11.1-11.5, 13.  
  • All chapters in Albert (A) are relevant to some extent. Often concepts from the lectures are explained in more detail with good examples. However, concepts in Albert that do not appear at all in the lectures are not important for the course. Also, Albert has a lot of focus on using R functions from the R package LearnBayes (written by Albert) that implement various Bayesian computations. Using such pre-written R functions may be very helpful for some students but for others it obscures what is going on. In our course we will not focus on using these functions.
  • Chapters 2, 3, 6, 7 and section 5.4 of Robert and Casella (RC) are relevant. Again, there are many good examples, but concepts that only appear in these chapters and not in the lectures are not important for the course.
  • Chapters 1, 4, and 7 of Särkkä (S) are relevant.
  • There will be some material appearing in lectures and in overheads that does not appear in the textbooks above.

Schedule

Contents Study material / activity
Time and place
Lecture 1: Introduction and preliminaries

B2.1 - B2.4 (meaning chapter 2 sections 1 through 4 from Bishop; see literature list). A2, A3, A4 (chapters 2, 3, 4 from Albert). Some overheads.

Monday 2/9, 15:15 - 17:00,  Pascal.
Workshop / help

A1, RC1. / Introduction to R for students who have not used R before.

Wednesday 4/9, 8:00 - 9:45, MVF33.
Lecture 2: Using conjugacies in exponential families of distributions

B2.1-B2.4. A2, A3, A4. Some overheads.

Wednesday 4/9 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

Exam 2023-08-24 question 1; 2020-10-26q1; 2021-01-05q1; 2023-01-05q1; A1.6 exercise 4. A2.9; 1,4,5; A3.9; 1,3. R code.

Thursday 5/9, 15:15 - 17:00, Pascal.

Lecture 3: Conjugacies, mixtures, discretizations. Examples

B2.1-B2.4. A2, A3, A4. Some overheads. R code.

Monday 9/9, 15:15 - 17:00,  Pascal.
Workshop / help

Do task on overhead 11 of Lecture 3. Run through and complete remaining examples of Lecture 3. R code. In addition: Exercises of your choice from our textbooks.

Wednesday 11/9, 8:00 - 9:45, MVF33.
Lecture 4:  Inference by simulation. Monte Carlo integration. Basic simulation methods. Rejection sampling. 

B11.1 (until B11.1.6); A5; RC2, RC3. Some overheads. R code.

Wednesday 11/9 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

Exercises: Based on student requests, we will go through exercises selected from lines 3 and 4 in the table of recommended study questions below.

Thursday 12/9, 15:15 - 17:00, Pascal.
Lecture 5: Importance sampling and SIR. The multivariate Normal and Laplace approximation. Introduction to Markov chain Monte Carlo (MCMC) methods.

B11.1.4 - B11.1.5.    A5.9 - A5.10.    B11.2.1 - B11.2.2. Some overheads. R code.

Monday 16/9, 15:15 - 17:00,  Pascal.
Workshop / help

Clarifications based on incoming student questions. More on examples from lectures. Individual work/help with assignment questions. 

Wednesday 18/9, 8:00 - 9:45, MVF33.
Lecture 6: MCMC. Random walk. Independent proposal. Convergence; checking convergence. Burn-in. Smart proposals.

A6. For information: RC6-8. Some overheads. R code.

Wednesday 18/9 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

We will go through the exercises in lines 5 and 6 in the table of recommended study questions below, except those designated "extra".

Thursday 19/9, 15:15 - 17:00, Pascal.
Lecture 7: More on basic MCMC. Graphical Models.

B8 excluding 8.4.3-8.4.8. Some overheads. R code.

Monday 23/9, 15:15 - 17:00,  Pascal.
Workshop / help

Individual or group work, with teacher available for help.

Wednesday 25/9, 8:00 - 9:45, MVF33.
Lecture 8: Graphical models. Hierarchical models. Gibbs sampling.

B11.3. A6. For information: RC6-8. Some overheads. R code.

Wednesday 25/9 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

Students work in groups with the exercises specified in lines 7 and 8 in the table below (excluding those marked "extra"). Then we discuss the solutions together. 

Thursday 26/9, 15:15 - 17:00, Pascal.
Deadline first obligatory hand-in

 

Friday 27/9 16:00.
Lecture 9: Hierarchical models. Hamiltonian MCMC.

 A7; RC7. B11.5. Some overheads. R code. 

Monday 30/9, 15:15 - 17:00,  Pascal.
Workshop / help

Individual or group work, with teacher available for help. 

Wednesday 2/10, 8:00 - 9:45, MVF33.
Lecture 10: Missing data / augmented data.  Slice sampling.

B11.4. Some overheads. R code.

Wednesday 2/10 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

A discussion of common issues in assignment 1. Then, students work in groups with the exercises specified in lines 9 and 10 in the table below (excluding those marked "extra"). Then we discuss the solutions together.  

Thursday 3/10, 15:15 - 17:00, Pascal.

Lecture 11:  State space models and Hidden Markov Models. Kalman filters. 

 Finishing overheads from Lecture 10. Then: Some overheads. R code.

Monday 7/10, 15:15 - 17:00,  Pascal.
Workshop / help

Individual or group work, with teacher available for help. 

Wednesday 9/10, 8:00 - 9:45, MVF33.
Lecture 12: Kalman filters. Particle filters. Viterbi. 

S chapters 1,4,7. B1.6, 9.4 (9.2, 9.3); RC5.4. Some overheads. R code.

Wednesday 9/10 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

Students work in groups with the exercises specified in lines 11 and 12 in the table below (excluding those marked "extra"). Then we discuss the solutions together.  

Thursday 10/10, 15:15 - 17:00, Pascal.
Deadline second obligatory hand-in

 

Friday 11/10, 16:00.
Lecture 13: Some information theory. The EM algorithm. Parameter inference in state space models.

B1.6, 9.4 (9.2, 9.3); RC5.4. Some overheads. R code.

Monday 14/10, 15:15 - 17:00,  Pascal.
Workshop / help

Individual or group work, with teacher available for help. 

Wednesday 16/10, 8:00 - 9:45, MVF33.
Lecture 14: Variational Bayes. Applied Bayesian Modelling  

B10.1 (B10.2). Literature: A7, A8. Some overheads. R code

Wednesday 16/10 10:00 - 11:45, Pascal.
Exercise class (demonstration/discussion of exercises)

Students work in groups with the exercises specified in lines 13 and 14 in the table below (excluding those marked "extra"). Then we discuss the solutions together.  

Thursday 17/10, 15:15 - 17:00, Pascal.
Lecture 15: Review

Review. Some overheads.

Monday 21/10, 15:15 - 17:00,  Pascal.
WRITTEN EXAM

Everything

Saturday 26/10, 8:30 - 12:30.

Recommended study questions

After the lecture whose number is on the left, you may work on the questions listed in that line.

OLD EXAMS Albert Robert & Casella Other
1

2023-08-24 question 1.

2

2023-01-05q1. 2021-01-05q1. 2020-10-26q1. 2019-01-08q1.

A1.6 exercise 4. A2.9: 1,4,5; A3.9: 1,3

some solutions

Some exercises 

solutions

3 2022-10-29q1. 2022-08-25q2. 2021-10-30q1. 2020-10-29q2. 2020-10-26q2.

A3.9: 4; solutions

Some exercises

solutions

4

2022-10-29q2. 2021-08-26q2. 2019-08-29q1; 2019-01-08q4; 2018-10-27q1; 2018-01-02q3; 2017-10-21q1.

A4.8 1,4,7. solutions

RC 2.11, 2.12, 2.18, 2.22.

some solutions

5

2022-10-29q4. 2020-10-26q5. 2019-08-29q2. 2019-01-08q2ab.

EXTRA: 2023-08-24q2. 2018-01-02q2. 2020-08-27q2.

6

2021-10-30q3. 2021-08-26q6. 2021-01-05q3.

EXTRA: 2020-01-08q3. 2018-10-27q2. 2018-01-02q8. 2020-08-27q5.

A5.13: 1,5. A6.13: 1,2. solutions
7

2023-08-24q4. 2023-01-05q4, q7. 2022-08-25q4, q5. 2022-01-05q7.

EXTRA: 2018-01-02q5. 2019-10-28q2. 

8

EXTRA: 2021-10-30q6. 2021-08-26q4. 2021-01-05q5, q6. 2020-10-29q3, q6. 2020-10-26q7. 2020-08-27q3, q7; 2017-10-21q4, q5; 2020-01-08q5; 2019-10-28q4; 2019-08-29q3, q6; 2019-01-08q3, q6; 2018-10-27q5; 2018-01-02q6; 2017-10-21q6.

A7.13: 1,2. 

9

2020-10-26q3. 2023-01-05q2. 2022-10-29q3. 2022-08-25q1. 2021-10-30q2.

 

10

2022-01-05q2. 2021-01-05q2.

A5.13: 2;  A6.13: 3;   A10.7: 6.  A9.7: 3,4; solutions 
11

 2023-01-05q3. 2022-08-25q3. 2021-10-30q4. 2020-10-29q5. 2020-10-26q4.

A9.7: 6; A10.7: 5, 7. solutions
12

 2020-01-08q4. 2019-08-29q8. 2018-01-02q7. 2017-10-21q3. 2023-01-05q6. 2022-10-29q5. 2021-10-30q7. 2021-01-05q7. 2020-10-29q7.

EXTRA: 2019-08-29q5.

13

2023-08-24q5. 2022-10-29q6. 2022-08-25q7. 2022-01-05q6. 2021-10-30q5. 2021-08-26q3.

14

2023-08-24q6. 2023-01-05q5. 2022-08-25q6. 2022-01-05q3, q8. 2021-08-26q5. 2020-10-26q6.

 

Back to the top

Course work

To understand and learn the methods of this course, it is essential to work with examples on a computer, in addition to working with the study material and doing theoretical exercises. Our textbooks contain a large number of exercises, for both theoretical and computer solutions, and some exercises are listed above.

As an obligatory part of the course, each student must do 2 assignments. The deadlines for these are Friday 20 September 16:00 and Friday 11 October 16:00. Details about the assignments will be available from the links at the bottom of this page. Answers must also be handed in via Canvas. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.

As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.

If you have problems with getting started with the course, or for example getting started with R, please do not hesitate to contact me. I hope that we can have active communication during the course. In addition to contact at lectures and exercise sessions, you may contact me on canvas or by mail at mostad@chalmers.se. I usually answer within a day or so.

Back to the top

Course summary:

Date Details Due