MVE187 / MSA101 Computational methods for Bayesian statistics Autumn 20
Course PM
This page contains the program for the course. Other information, such as learning outcomes, teachers, literature, examination, and old exams is in a separate course PM.
Program
The schedule is in TimeEdit, but as the course is now given online, you should consult the schedule specified below.
Online learning material for the course is structured into packages, with two packages each week. The packages contain
- recorded lectures
- overheads used in these lectures
- R code used in these lectures
- references to relevant sections in our textbooks
- lists of recommended exercises
- written solutions to some exercises
- recorded videos for solutions to some exercises
You are expected to go through and learn from this material BEFORE the online course meeting corresponding to each package. Thus there will generally be two Zoom meetings each week, of at most 45 minutes each, where we can all meet:
- You will be able to ask big and small questions related to the online learning material.
- You may ask me to review or clarify material (if you mail me questions on beforehand I can prepare, but this is not necessary).
- You may comment on or discuss course content or course organization.
- I will ask you some questions, to get feedback on what you know, and to help you understand what you need to know. (This will not be part of any grading).
- We can discuss whatever else we want to discuss in the whole group.
The second main part of the course is the obligatory assignments. There are three, and answers are submitted individually, although you are allowed to cooperate in small groups. The deadlines are in the third, fifth and seventh course weeks. Each assignment is based on material discussed the week before each deadline, or earlier in the course.
Finally, I will have "office hours" one hour each week on Zoom, where students can contact me individually or in small groups for tutoring. The office hours work as follows: You enter the "waiting room" of my zoom office, and I admit one student at a time from that waiting room, on a first-come first-served basis. Each student should limit their question time to about 10 minutes, if there is a line. The amount of office hours may be adjusted based on demand. I will also try to answer questions posed here in Canvas when I see them.
Guide for reading the textbooks:
- The sections from Bishop selected in the specification of the study material below are relevant.
- All chapters in Albert are relevant to some extent. Often concepts from the lectures are explained in more detail (and better?) with good examples. However, concepts in Albert that do not appear at all in the lectures are not important for the course. Also, Albert has a lot of focus on using R functions from the R package LearnBayes (written by Albert) that implement various Bayesian computations. Using such pre-written R functions may be very helpful for some students but for others it obscures what is going on. In our course we will not focus on using these functions.
- Chapters 2, 3, 6, 7 and section 5.4 of Robert and Casella are relevant. Again, there are many good examples, but concepts that only appear in these chapters and not in the lectures are not important for the course.
Schedule
Contents (preliminary) |
Study material (preliminary) | Zoom meeting (preliminary) |
---|---|---|
Course introduction | Monday 31/8 15:15 - 16:00 | |
Introduction to the Bayesian paradigm. Issues with classical frequentist inference. |
Lecture 1: Video1.1 Video1.2 Video1.3 / overheads Literature: B2.1-B2.4 (meaning chapter 2 sections 1 through 4 from Bishop; see literature list). A2, A3, A4 (chapters 2, 3, 4 from Albert). Make sure you know how to use R! See A1, or RC1, or any other introductory text to R. Exercises: EXAM 2019-01-08 question 1. A1.6 exercise 4. RC1.11 exercise 19. |
|
Basic computations using conjugacy. Using discretization. The exponential family of distributions. |
Lecture 2: Video2.1a Video2.1b Video2.2 Video2.3 Video2.4 / overheads / R code Literature: B2.1-B2.4. A2, A3, A4. Exercises: A2.9: 1,4,5; A3.9: 1,3,4. A4.8 1,4,7. |
|
Office hours | Thursday 3/9 15:15 - 16:00 | |
Mixtures. Some multivariate conjugacies. |
Lecture 3: video3.1 video3.2 video3.3 video3.4 / overheads / R code. Literature: B2.1-B2.4. A2, A3, A4. Exercises: EXAMS: 2019-08-29 q1; |
Video(I forgot to record first part of meeting) |
Low dimensional Bayesian inference. Inference by simulation. Monte Carlo integration. Basic simulation methods. |
Lecture 4: Video4.1 Video4.2a video4.2b Video4.3a (for missing whiteboard sharing 2:15 - 3:30 please see the discussion video to the right 7:50 - 11:15) video4.3b Video4.4 / overheads / R code Literature: B11.1 (until B11.1.6); B11.4; A5; RC2, RC3. Exercises: RC 2.11, 2.12, 2.18, 2.22, 3.13. EXAMS: 2019-08-29 q2; 2019-01-08 q2ab, 2018-01-02 q2, 2017-10-21 q2. |
|
Office hours | Thursday 10/9 15:15 - 16:00 | |
Introduction to Markov chain Monte Carlo (MCMC) methods |
Lecture 5: Video5.1 Video5.2 Video5.3 / overheads / R code Literature: B 11.2; A6; RC6. Exercises: A6:13, exercise 1. RC exercise 6.7.EXAMS: 2020-01-08 q3; 2019-10-28 q2; 2018-10-27 q2; 2018-01-02 q8. |
|
MCMC. Random walk. Independent proposal. Convergence; checking convergence. Burn-in. Smart proposals. |
Lecture 6: Video6.1a video6.1b Video6.2 Video6.3 / overheads / R code Literature: B 11.2; A6; RC6. Exercises: EXAMS 2020-08-27 q5; 2018-01-02 q5. RC questions 8.2, 8.8. |
|
Office hours | Thursday 17/9 15:15 - 16:00 | |
--- | Friday 18/9 16:00: DEADLINE first assignment (see below) | --- |
Laplace approximation. MCMC convergence. Gibbs sampling. Hierarchical models. Slice sampling. |
Lecture 7: Video7.1 Video7.2 Video7.3 Video7.4 / overheads / R code Literature: B11.3, 11.4; A6; RC6. Exercises: A5.13: 1,4; A6.13: 2. EXAM 2019-08-29 q5. |
|
Missing data. Hamiltonian MCMC. Review / overview MCMC. |
Lecture 8: Video8.1 Video8.2 Video8.3 / overheads / R code Literature: B11.5; A7; RC7. Exercises: EXAMS 2020-08-27: q3, q7; 2017-10-21 q4, q5; A7.13: 1,2. |
|
Office hours | Thursday 24/9 15:15 - 16:00 | |
Some information theory. The EM algorithm. |
Lecture 9: Video9.1 Video9.2 Video9.3 / overheads / R code Literature: B1.6, 9.4 (9.2, 9.3); RC5.4. Exercises: EXAMS 2020-01-08 q5; 2019-10-28 q4; 2019-08-29 q3, q6; 2019-01-08 q3, q6; 2018-10-27 q5; 2018-01-02 q6; 2017-10-21 q6. RC: 5.8, 5.9, 5.10. |
|
Introduction to Graphical Models. |
Lecture 10: Video10.1 video10.2 video10.3 / overheads Literature: B8 excluding 8.4.3-8.4.8. Exercises: EXAMS 20-08-27 q4; 2019-10-28 q5; 2019-08-29 q4; 2018-10-27 q3, q4, q6. 2018-01-02 q4. |
Thursday 1/10 13:15 - 14:00 Because of technical problems, the meeting has been postponed to 14:15! |
Office hours | Thursday 1/10 15:15 - 16:00 | |
--- | Friday 2/10 16:00: DEADLINE second assignment (see below) | --- |
Hidden Markov models. |
Lecture 11: video11.1 video11.2 video11.3 video11.4 overheads / R code Literature: B13.2 Exercises: EXAMS 20-01-08 q4; 19-08-29 q8; 18-01-02 q7; 17-10-21 q3. |
|
Variational Bayes |
Lecture 12: Video12.1 Video12.2 / overheads / R code Literature: B10.1 (B10.2) Exercises: EXAM 2019-10-28 q6. |
|
Office hours | Thursday 8/10 15:15 - 16:00 | |
Applied Bayesian modelling: Model choice and model checking. |
Lecture 13: Video13.1 Video13.2 Video13.3 / overheads / R code Literature: A7 Exercise: A7.13: 1 |
|
Bayesian model choice. A larger example. |
Lecture: Video14.1 / overheads14.1 / R code / Video14.2 / overheads14.2. Literature: A8. Paper. Exercise: EXAM 2019-10-28 q3. |
|
Office hours | Thursday 15/10 15:15 - 16:00 | |
--- | Friday 16/10 16:00 DEADLINE third assignment (see below) | --- |
Lecture 15: Approximate Bayesian Computation, or "What to do when exact Bayes is impossible?" |
Lecture: Literature: (optional!) Marin_etal_review.pdf; Marjoram_2003.pdf |
Wednesday 21/10 10:00 - 11:45. (Lecturer: Umberto Picchini) |
Course evaluation discussion |
|
|
Office hours |
|
Thursday 22/10 15:15 - 16:00 |
Course work
To understand and learn the methods of this course, it is essential to work with examples on a computer, in addition to working with the study material and doing theoretical exercises. Our textbooks contain a large number of exercises, for both theoretical and computer solutions, and recommended exercises are listed above.
As an obligatory part of the course, each student must do 3 assignments. The deadlines for these are Friday 18 September 16:00, Friday 2 October 16:00, and Friday 16 October 16:00. Details about the assignments will be available from the links at the bottom of this page. Answers must also be handed in via Canvas. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.
As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.
If you have problems with getting started with the course, or for example getting started with R, please do not hesitate to contact me. I hope that we can have active communication during the course. If you have questions connected to the teaching material, for example to the recorded lectures, please consider asking your question during the Zoom course meetings: Most students have very similar questions. The Zoom office hours are intended for more personal help, support, and feedback.
My preferred modes of communication are:
- Talk to me during the two weekly course meetings on Zoom.
- Talk to me during my weekly office hours on Zoom.
- Mail me at mostad@chalmers.se. I usually answer within a day or so.
- Send a message here on Canvas. However, I will probably not answer immediately.
- If you need immediate contact, you may call me at 031-772-3579.
Course summary:
Date | Details | Due |
---|---|---|