Course syllabus

Course-PM

SSY316 Advanced probabilistic machine learning lp2 HT23 (7.5 hp)

This course is offered by the department of Electrical Engineering

Course purpose

This course delves into the connections between machine learning and probability theory and statistics, unveiling a profound probabilistic perspective of machine learning problems.

The very essence of probability theory is the study of uncertainty.  Within machine learning,  uncertainty manifests in multifaceted ways--ranging from the intrinsic noise within collected data to uncertainties concerning optimal predictions based on historical data or the most fitting model to explain the data.

The fundamental tenet underscoring the probabilistic paradigm of machine learning is that learning can be thought of as the art of inferring plausible probabilistic models that describe the data emanating from a given system. These probabilistic models not only enable to make predictions and statements about observable data but also possess the capability to articulate the inherent uncertainty of those predictions.

This course will describe an extensive array of probabilistic models, suitable for a diverse spectrum of data types and tasks. It will also elucidate a wide variety of algorithms tailored for inference and learning faciltated by such models. The overarching goal of the course is to present a unified vision of machine learning, casting it in the illuminating light of probabilistic modeling and inference.

In the course the students will not only learn universal models/methods that are useful in probabilistic machine learning, but also in other areas.

Learning outcomes

After completion of this course, the student should be able to

• Explain the philosophy behind Bayesian inference
• Develop an inference algorithm using the principles of Bayesian decision theory and a given cost function
• Understand the connections between probability theory and machine learning
• Explain similarities and differences between probabilistic and classical machine learning methods
• Interpret and explain results from probabilistic machine learning
• Derive, analyze, and implement the probabilistic methods introduced in the course
• Understand how to apply several probabilistic models to data and determine the most suitable one for a given task
• Discuss and determine whether an engineering-relevant problem can be formulated as a supervised or unsupervised machine learning problem

Content

• Review of probability theory: frequentist vs Bayesian approach
• Bayesian inference, probabilistic modeling of data
• Supervised learning: Bayesian linear regression
• Linear methods for classification
• Bayesian graphical models: Bayesian networks, Markov random fields, factor graphs
• Belief propagation
• Approximate inference and learning: Monte Carlo inference (importance sampling, Gibbs sampling,  Markov chain Monte Carlo)
• Approximate inference: Variational inference and Expectation propagation
• Unsupervised learning: K-means clustering, the Gaussian mixture model, expectation maximization, principal component analysis

Course staff

• Examiner and Lecturer: Alexandre Graell i Amat (alexandre.graell@chalmers.se). Office: Room 6409, Department of Electrical Engineering
• Teaching assistants: Mehdi Sattari (mehdi.sattari@chalmers.se, office: 6323) and Javad Aliakbari (javada@chalmers.se, office: 6329)

Student representatives

• Hubert Andersson,      hubert@karse.se
• Anton Bornecrantz,   antbor@student.chalmers.se      
• Xinglu Gong,   1911562394@qq.com
• Yue Lou,   ly_6666@outlook.com
• Filip Ronnblad,   ronnblad@student.chalmers.se

Course literature

This course will follow closely the following book: 

• C. M. Bishop, Pattern Recognition and Machine Learning. Berlin, Heidelberg: Springer-Verlag, 2006.

The book is available for free download in the link: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

An errata is available in the link: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/prml-errata-1st-20110921.pdf

Additional course material

Slides for the lectures and exercises for the tutorial sessions will be made available on Canvas.

Schedule

Lectures are typically on Tuesdays, Thursdays, and Fridays 13:15-15:00.  However, there may be some exceptions. Please see the course memo for the preliminary schedule for the lectures.  Please check Canvas for updates.

Tutorials are typically on Tuesdays and Thursdays 15:15-17:00. Please check Canvas for updates.

Grading

The final score (maximum score is 100) will be decided based on the points collected via the theoretical assignments (maximum score 30), the python assignments (maximum score 35) and the final (take-home) python assignment (maximum score 35).