Course syllabus

Course PM

This page contains the program for the course. Other information, such as learning outcomes, teachers, literature, examination, and old exams is in a separate course PM.

Program

The schedule is in TimeEdit and in the table below.

During the pandemic years, the availability of recordings of the lectures has been greatly appreciated. Thus I will continue with making the lectures available live on zoom and as a recording afterwards.  However, I hope most of you will attend the physical lectures.

You should study the reading material listed in the schedule below before each lecture. Active students with all kinds of questions are greatly appreciated.

There is an exercise session / computer lab each week. Ideally these sessions are driven by student questions.  We will adapt the format so that it fits this year's group of students.

The second main part of the course is the obligatory assignments. There are two, and answers are submitted individually, although you are allowed to cooperate in small groups. The deadlines are in the third and sixth course weeks. Each assignment is based on material discussed at least one week before each deadline.

Guide for reading the textbooks:

  • The sections from Bishop selected in the specification of the study material below are relevant.
  • All chapters in Albert are relevant to some extent. Often concepts from the lectures are explained in more detail with good examples. However, concepts in Albert that do not appear at all in the lectures are not important for the course. Also, Albert has a lot of focus on using R functions from the R package LearnBayes (written by Albert) that implement various Bayesian computations. Using such pre-written R functions may be very helpful for some students but for others it obscures what is going on. In our course we will not focus on using these functions.
  • Chapters 2, 3, 6, 7 and section 5.4 of Robert and Casella are relevant. Again, there are many good examples, but concepts that only appear in these chapters and not in the lectures are not important for the course.

Schedule

Contents Study material
Time and place
Lecture 1: Introduction to the Bayesian paradigm. Issues with classical frequentist inference.

Literature: B2.1-B2.4 (meaning chapter 2 sections 1 through 4 from Bishop; see literature list). A2, A3, A4 (chapters 2, 3, 4 from Albert).

Make sure you know how to use R! See A1, or RC1, or any other introductory text to R. (RC means Robert and Casella)

Monday 2022-08-29, 15:15-17:00, in Pascal. RECORDING. Overheads. WB1 WB2 WB3 WB4 WB5
Lecture 2: Basic computations using conjugacy. The exponential family of distributions.

B2.1-B2.4. A2, A3, A4.

Wednesday 2022-08-31, 10:00-11:45. RECORDING. Overheads. WB1 WB2 WB3 WB4 WB5 WB6 WB7 WB8
Exercises / computer lab

Please have a look at exercises on beforehand:

EXAM 2019-01-08 question 1. A1.6 exercise 4.

Some exercises  

A2.9: 1,4,5; A3.9: 1,3,(4). (A4.8 1,4,7).

R code solutions

Thursday 2022-09-01, 15:15-17:00, in MVF22.

 

Lecture 3: Using discretization. Mixtures. Some multivariate conjugacies.

 

B2.1-B2.4. A2, A3, A4.
Monday 2022-09-05, 15:15-17:00, in Pascal. RECORDING. Overheads. R-code used.
Lecture 4: Inference by simulation. Monte Carlo integration. Basic simulation methods. Rejection sampling. More about priors.

 

B11.1 (until B11.1.6); B11.4; A5; RC2, RC3. 
Wednesday 2022-09-07, 10:00-11:45, in ED. RECORDING. Overheads. R-code used. WB1 WB2
Exercises / computer lab

Exercises on conjugacy: EXAMS: 2019-08-29 q1;
2019-01-08 q4; 2018-10-27 q1; 2018-01-02 q3; 2017-10-21 q1;

Exercises on random number simulation: RC 2.11, 2.12, 2.18, 2.22. EXAMS: 2019-08-29 q2; 2019-01-08 q2ab, 2018-01-02 q2, 2017-10-21 q2. 2020-08-27 q2.

R code solutions for RC exercises

Thursday 2022-09-08, 15:15-16:00: Individual help, 16:00-17:00 together, in MVF33.
Lecture 5: Importance sampling and SIR. Laplace approximation. Introduction to Markov chain Monte Carlo (MCMC) methods.

B11.1.4-5; B11.2.1-2; A5.9-10

Monday 2022-09-12, 15:15-17:00, in EE. RECORDING. Overheads. R code used. WB1 WB2 WB3
Lecture 6: MCMC. Random walk. Independent proposal. Convergence; checking convergence. Burn-in. Smart proposals.

A6. For information: RC6-8

Wednesday 2022-09-14, 10:00-11:45, in ED. RECORDING. Overheads. R code used(ish). WB1 WB2
Exercises /computer lab

A5.13: 1,5. A6.13: 1,2. EXAMS: 2020-01-08 q3; 2019-10-28 q2; 2018-10-27 q2; 2018-01-02 q8; 2020-08-27 q5; 2018-01-02 q5.

Some R solutions to Albert exercises

Thursday 2022-09-15, 15:15-16:00: Individual help, 16:00-17:00 together,  in MVF22.
Deadline first assignment 

See below

Friday 2022-09-16, 16:00
Lecture 7: Gibbs sampling. Hierarchical models. Tips and tricks. Convergence.

B11.3. A6. For information: RC6-8

Monday 2022-09-19, 15:15-17:00, in Pascal. RECORDING. Overheads. R code used. WB1 WB2 WB3
Lecture 8, Lecturer Umberto Picchini: state space models (SSM), intro to particle filters and sequential importance sampling

Book by Särkkä: chapters 1 and 7.1-7.3

Two blog posts by Umberto: one and two.

Wednesday 2022-09-21, 10:00-11:45, in ED.

slides_ssm-intro_particles.pdf

demo_sis.m [deprecated. see instead demo_sis_with_states.m]

yobs.dat

 

Exercises / computer lab

First exercises: A5.13: 2;  A6.13: 3;   A10.7: 6.  EXAM 2019-08-29 q5.

Additional exercises: A9.7: 3,4;    EXAMS 2020-01-08 q4; 2019-08-29 q8; 2018-01-02 q7; 2017-10-21 q3.

Some R code for Albert exercises

Thursday 2022-09-22, 15:15-16:00: Individual help, 16:00-17:00 together,  in MVF33

Lecture 9. Lecturer Umberto Picchini: continuation particle filters (the bootstrap filter) + coding examples and particle MCMC for state-space models

Särkkä section 7.4.

Dahlin-Schön here

blog post 1, blog post 2

Monday 2022-09-26, 15:15-17:00, in Pascal.

slides_bootstrap_static.pdf

(for easier browsing but lacks the animation)

slides_bootstrap_animated.pdf

(use this version for the particle filter animation)

demo_sis_with_states.m

demo_bootstrap.m

demo_pmcmc.m

resampling.m (invoked by demo_pmcmc)

README_Nimble_Installation.txt

demo_nimbleSMC.R

demo_nimbleSMC_ with_parameters.R

Lecture 10, Lecturer Umberto Picchini: the pseudomarginal approach to exact/approximate inference; examples and coding; if time, some elements of ABC (approximate Bayesian computation)

references for additional reading are linked in the slides. Also this article is pedagogical and covers a lot of what we have done.

Finally, my blog on pseudo-marginal methods

Wednesday 2022-09-28, 10:00-11:45, in ED.

slides_pmm.pdf

abc_slides_reduced.pdf

weibull_NOsummaries.R

weibull_summaries.R

Exercises / computer lab

We go through the exercises posed by Umberto in his two first lectures. We also go through the first assignment. (+ requests)

R code

Thursday 2022-09-29, 15:15-16:00: Individual help, 16:00-17:00 together,  in MVF22

Lecture 11: Hierarchical models. Missing data / augmented data.  Hamiltonian MCMC.

B11.5; A7; RC7.

Monday 2022-10-03, 15:15-17:00, in Pascal. RECORDING. Overheads. R code used. WB1 WB2 WB3 WB4

Lecture 12: Some information theory. The EM algorithm.

B1.6, 9.4 (9.2, 9.3); RC5.4.

Wednesday 2022-10-05, 10:00-11:45. RECORDING. Overheads. R code. WB1 WB2 WB3 WB4 WB5 WB6

Exercises / computer lab

A9.7: 6;    A10.7: 5, 7.

 

Thursday 2022-10-06, 15:15-16:00: Individual help, 16:00-17:00 together,  in MVF33

Deadline second assignment

See below

Friday 2022-10-07, 16:00

Lecture 13: Variational Bayes. Slice sampling.

Literature:  B10.1 (B10.2). B11.4.

Monday 2022-10-10, 15:15-17:00, in Pascal. RECORDING. Overheads. R code. WB1 WB2 WB3

Lecture 14: Graphical Models.

B8 excluding 8.4.3-8.4.8.

Wednesday 2022-10-12, 10:00-11:45, in ED. RECORDING. Overheads. R code (not used). WB1 WB2 WB3 WB4 WB5 WB6 WB7 WB8 WB9

Exercises / computer lab EXAMS 2020-08-27: q3, q7; 2017-10-21 q4, q5; A7.13: 1,2. EXAMS 2020-01-08 q5; 2019-10-28 q4; 2019-08-29 q3, q6; 2019-01-08 q3, q6; 2018-10-27 q5; 2018-01-02 q6; 2017-10-21 q6.

Thursday 2022-10-13, 15:15-17:00 together,  in MVF22

Lecture 15: Applied Bayesian modelling.

Literature: A7, A8.  Paper.

Monday 2022-10-17, 15:15-17:00, in Pascal. RECORDING. Overheads first part. Overheads second part. R code. WB1 WB2

Review

Everything: Please mail in requests for things to review.

Wednesday 2022-10-19, 10:00 - 11:45, in ED. RECORDING. WB1 WB2 WB3 WB4 WB5 WB6

Question time

Discussion of the second assignment. Please mail requests for questions we should go through.

Thursday 2022-10-21, 15:15-17:00, in MVF33

Written exam

Everything

Saturday 2022-10-29 8:30-12:30

 

Back to the top

Course work

To understand and learn the methods of this course, it is essential to work with examples on a computer, in addition to working with the study material and doing theoretical exercises. Our textbooks contain a large number of exercises, for both theoretical and computer solutions, and some exercises are listed above.

As an obligatory part of the course, each student must do 3 assignments. The deadlines for these are Friday 16 September 16:00, Friday 30 September 16:00, and Friday 14 October 16:00. Details about the assignments will be available from the links at the bottom of this page. Answers must also be handed in via Canvas. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.

As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.

If you have problems with getting started with the course, or for example getting started with R, please do not hesitate to contact me. I hope that we can have active communication during the course. In addition to contact at lectures and exercise sessions, you may contact me on canvas or by mail at mostad@chalmers.se. I usually answer within a day or so.

Back to the top

Course summary:

Date Details Due