MVE187 / MSA101 Computational methods for Bayesian statistics Autumn 20

Course PM

This page contains the program for the course. Other information, such as learning outcomes, teachers, literature, examination, and old exams is in a separate course PM.

Program

The schedule is in TimeEdit, but as the course is now given online, you should consult the schedule specified below.

Online learning material for the course is structured into packages, with two packages each week. The packages contain

  • recorded lectures
  • overheads used in these lectures
  • R code used in these lectures
  • references to relevant sections in our textbooks
  • lists of recommended exercises
  • written solutions to some exercises
  • recorded videos for solutions to some exercises

You are expected to go through and learn from this material BEFORE the online course meeting corresponding to each package. Thus there will generally be two Zoom meetings each week, of at most 45 minutes each, where we can all meet: 

  • You will be able to ask big and small questions related to the online learning material.
  • You may ask me to review or clarify material (if you mail me questions on beforehand I can prepare, but this is not necessary).
  • You may comment on or discuss course content or course organization.
  • I will ask you some questions, to get feedback on what you know, and to help you understand what you need to know. (This will not be part of any grading).
  • We can discuss whatever else we want to discuss in the whole group.

The second main part of the course is the obligatory assignments. There are three, and answers are submitted individually, although you are allowed to cooperate in small groups. The deadlines are in the third, fifth and seventh course weeks. Each assignment is based on material discussed the week before each deadline, or earlier in the course.

Finally, I will have "office hours" one hour each week on Zoom, where students can contact me individually or in small groups for tutoring. The office hours work as follows: You enter the "waiting room" of my zoom office, and I admit one student at a time from that waiting room, on a first-come first-served basis. Each student should limit their question time to about 10 minutes, if there is a line. The amount of office hours may be adjusted based on demand. I will also try to answer questions posed here in Canvas when I see them.

Guide for reading the textbooks:

  • The sections from Bishop selected in the specification of the study material below are relevant.
  • All chapters in Albert are relevant to some extent. Often concepts from the lectures are explained in more detail (and better?) with good examples. However, concepts in Albert that do not appear at all in the lectures are not important for the course. Also, Albert has a lot of focus on using R functions from the R package LearnBayes (written by Albert) that implement various Bayesian computations. Using such pre-written R functions may be very helpful for some students but for others it obscures what is going on. In our course we will not focus on using these functions.
  • Chapters 2, 3, 6, 7 and section 5.4 of Robert and Casella are relevant. Again, there are many good examples, but concepts that only appear in these chapters and not in the lectures are not important for the course.

List of contents of course

Schedule

Contents (preliminary)

Study material (preliminary) Zoom meeting (preliminary)
Course introduction  Monday 31/8 15:15 - 16:00
Introduction to the Bayesian paradigm. Issues with classical frequentist inference.

Lecture 1: Video1.1 Video1.2 Video1.3 / overheads

Literature: B2.1-B2.4 (meaning chapter 2 sections 1 through 4 from Bishop; see literature list). A2, A3, A4 (chapters 2, 3, 4 from Albert).

Make sure you know how to use R! See A1, or RC1, or any other introductory text to R.

Exercises: EXAM 2019-01-08 question 1.

A1.6 exercise 4. RC1.11 exercise 19.

 Wednesday 2/9 10:00 - 10:45

Video

Basic computations using conjugacy. Using discretization. The exponential family of distributions.

Lecture 2: Video2.1a  Video2.1b Video2.2  Video2.3  Video2.4 / overheads / R code

Literature:  B2.1-B2.4. A2, A3, A4.

Exercises: A2.9: 1,4,5; A3.9: 1,3,4. A4.8 1,4,7.

 Thursday 3/9 13:15 - 14:00

Video

Office hours  Thursday 3/9 15:15 - 16:00

Mixtures. Some multivariate conjugacies.

Lecture 3:

video3.1   video3.2  video3.3  video3.4 / overheads / R code.

Literature: B2.1-B2.4. A2, A3, A4.

Exercises: EXAMS: 2019-08-29 q1;
2019-01-08 q4; 2018-10-27 q1; 2018-01-02 q3; 2017-10-21 q1; 2020-08-27 q2.

 Wednesday 9/9 10:00 - 10:45

Video(I forgot to record first part of meeting)

Low dimensional Bayesian inference. Inference by simulation. Monte Carlo integration. Basic simulation methods.

Lecture 4:  Video4.1  Video4.2a  video4.2b  Video4.3a  (for missing whiteboard sharing 2:15 - 3:30 please see the discussion video to the right 7:50 - 11:15) video4.3b  Video4.4  / overheads / R code

Literature: B11.1 (until B11.1.6); B11.4; A5; RC2, RC3. 

Exercises: RC 2.11, 2.12, 2.18, 2.22, 3.13. EXAMS: 2019-08-29 q2; 2019-01-08 q2ab, 2018-01-02 q2, 2017-10-21 q2.

/ Thursday 10/9 13:15-14:00

video

Office hours  Thursday 10/9 15:15 - 16:00
Introduction to Markov chain Monte Carlo (MCMC) methods

Lecture 5: Video5.1  Video5.2  Video5.3  / overheads / R code

Literature: B 11.2; A6; RC6. 

Exercises: A6:13, exercise 1. RC exercise 6.7.EXAMS: 2020-01-08 q3; 2019-10-28 q2; 2018-10-27 q2; 2018-01-02 q8.

 Wednesday 16/9 10:00 - 10:45

video

MCMC. Random walk. Independent proposal. Convergence; checking convergence. Burn-in. Smart proposals.

Lecture 6:  Video6.1a  video6.1b

 Video6.2    Video6.3  / overheads / R code

Literature: B 11.2; A6; RC6.

Exercises: EXAMS 2020-08-27 q5; 2018-01-02 q5. RC questions 8.2, 8.8.

 Thursday 17/9 13:15 - 14:00

video

Office hours  Thursday 17/9 15:15 - 16:00
   --- Friday 18/9 16:00: DEADLINE first assignment (see below) ---
Laplace approximation. MCMC convergence. Gibbs sampling. Hierarchical models. Slice sampling.

Lecture 7:  Video7.1 Video7.2  Video7.3  Video7.4  / overheads / R code

Literature: B11.3, 11.4; A6; RC6.

Exercises: A5.13: 1,4; A6.13: 2. EXAM 2019-08-29 q5. 

 Wednesday 23/9 10:00 - 10:45

Video

Missing data. Hamiltonian MCMC. Review / overview MCMC.

Lecture 8:  Video8.1 Video8.2  Video8.3  / overheads / R code

Literature: B11.5; A7; RC7.

Exercises: EXAMS 2020-08-27: q3, q7; 2017-10-21 q4, q5; A7.13: 1,2.

 Thursday 24/9 13:15 - 14:00

video

Office hours  Thursday 24/9 15:15 - 16:00
Some information theory. The EM algorithm.

Lecture 9:  Video9.1  Video9.2  Video9.3  / overheads / R code

Literature: B1.6, 9.4 (9.2, 9.3); RC5.4.

Exercises: EXAMS 2020-01-08 q5; 2019-10-28 q4; 2019-08-29 q3, q6; 2019-01-08 q3, q6; 2018-10-27 q5; 2018-01-02 q6; 2017-10-21 q6.   RC: 5.8, 5.9, 5.10.

 Wednesday 30/9 10:00 - 10:45

video

Introduction to Graphical Models.

Lecture 10: Video10.1 video10.2 video10.3 / overheads 

Literature: B8 excluding 8.4.3-8.4.8.

Exercises: EXAMS 20-08-27 q4; 2019-10-28 q5; 2019-08-29 q4; 2018-10-27 q3, q4, q6. 2018-01-02 q4.

 Thursday 1/10 13:15 - 14:00 Because of technical problems, the meeting has been postponed to 14:15!

Video

Office hours  Thursday 1/10 15:15 - 16:00
--- Friday 2/10 16:00: DEADLINE second assignment (see below) ---
Hidden Markov models.

Lecture 11: video11.1 video11.2 video11.3 video11.4 overheads / R code

Literature: B13.2

Exercises: EXAMS 20-01-08 q4; 19-08-29 q8; 18-01-02 q7; 17-10-21 q3.

 Wednesday 7/10 10:00 - 10:45

Video

Variational Bayes

Lecture 12: Video12.1 Video12.2 / overheads / R code

Literature: B10.1 (B10.2)

Exercises: EXAM 2019-10-28 q6.

 Thursday 8/10 13:15 - 14:00

Video

Office hours  Thursday 8/10 15:15 - 16:00
Applied Bayesian modelling: Model choice and model checking.

Lecture 13: Video13.1 Video13.2 Video13.3 / overheads / R code

Literature: A7

Exercise: A7.13: 1

 Wednesday 14/10 10:00 - 10:45

Video

Bayesian model choice. A larger example.

Lecture: Video14.1 / overheads14.1 / R code  / Video14.2 / overheads14.2.

Literature: A8. Paper.

Exercise: EXAM 2019-10-28 q3.

 Thursday 15/10 13:15 - 14:00

video

Office hours  Thursday 15/10 15:15 - 16:00
--- Friday 16/10 16:00 DEADLINE third assignment (see below) ---
Lecture 15: Approximate Bayesian Computation, or "What to do when exact Bayes is impossible?"

Lecture:

abc_intro1.mp4

abc_intro2.mp4

abc_intro3.mp4

Overheads/ code

Literature: (optional!) Marin_etal_review.pdf; Marjoram_2003.pdf

 Wednesday 21/10 10:00 - 11:45. (Lecturer: Umberto Picchini)
Course evaluation discussion

 

Thursday 22/10 13:15 - 14:00 

Video

Office hours

 

 Thursday 22/10 15:15 - 16:00

 

Back to the top

Course work

To understand and learn the methods of this course, it is essential to work with examples on a computer, in addition to working with the study material and doing theoretical exercises. Our textbooks contain a large number of exercises, for both theoretical and computer solutions, and recommended exercises are listed above.

As an obligatory part of the course, each student must do 3 assignments. The deadlines for these are Friday 18 September 16:00, Friday 2 October 16:00, and Friday 16 October 16:00. Details about the assignments will be available from the links at the bottom of this page. Answers must also be handed in via Canvas. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.

As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.

If you have problems with getting started with the course, or for example getting started with R, please do not hesitate to contact me. I hope that we can have active communication during the course. If you have questions connected to the teaching material, for example to the recorded lectures, please consider asking your question during the Zoom course meetings: Most students have very similar questions. The Zoom office hours are intended for more personal help, support, and feedback.

My preferred modes of communication are:

  • Talk to me during the two weekly course meetings on Zoom.
  • Talk to me during my weekly office hours on Zoom.
  • Mail me at mostad@chalmers.se. I usually answer within a day or so.
  • Send a message here on Canvas. However, I will probably not answer immediately.
  • If you need immediate contact, you may call me at 031-772-3579.

Back to the top

Course summary:

Date Details Due