SSY316 Advanced probabilistic machine learning lp2 HT23 (7.5 hp)
This course is offered by the department of Electrical Engineering
This course delves into the connections between machine learning and probability theory and statistics, unveiling a profound probabilistic perspective of machine learning problems.
The very essence of probability theory is the study of uncertainty. Within machine learning, uncertainty manifests in multifaceted ways--ranging from the intrinsic noise within collected data to uncertainties concerning optimal predictions based on historical data or the most fitting model to explain the data.
The fundamental tenet underscoring the probabilistic paradigm of machine learning is that learning can be thought of as the art of inferring plausible probabilistic models that describe the data emanating from a given system. These probabilistic models not only enable to make predictions and statements about observable data but also possess the capability to articulate the inherent uncertainty of those predictions.
This course will describe an extensive array of probabilistic models, suitable for a diverse spectrum of data types and tasks. It will also elucidate a wide variety of algorithms tailored for inference and learning faciltated by such models. The overarching goal of the course is to present a unified vision of machine learning, casting it in the illuminating light of probabilistic modeling and inference.
In the course the students will not only learn universal models/methods that are useful in probabilistic machine learning, but also in other areas.
After completion of this course, the student should be able to
- Explain the philosophy behind Bayesian inference
- Develop an inference algorithm using the principles of Bayesian decision theory and a given cost function
- Understand the connections between probability theory and machine learning
- Explain similarities and differences between probabilistic and classical machine learning methods
- Interpret and explain results from probabilistic machine learning
- Derive, analyze, and implement the probabilistic methods introduced in the course
- Understand how to apply several probabilistic models to data and determine the most suitable one for a given task
- Discuss and determine whether an engineering-relevant problem can be formulated as a supervised or unsupervised machine learning problem
- Review of probability theory: frequentist vs Bayesian approach
- Bayesian inference, probabilistic modeling of data
- Supervised learning: Bayesian linear regression
- Linear methods for classification
- Bayesian graphical models: Bayesian networks, Markov random fields, factor graphs
- Belief propagation
- Approximate inference and learning: Monte Carlo inference (importance sampling, Gibbs sampling, Markov chain Monte Carlo)
- Approximate inference: Variational inference and Expectation propagation
- Unsupervised learning: K-means clustering, the Gaussian mixture model, expectation maximization, principal component analysis
- Examiner and Lecturer: Alexandre Graell i Amat (firstname.lastname@example.org). Office: Room 6409, Department of Electrical Engineering
- Teaching assistants: Mehdi Sattari (email@example.com, office: 6323) and Javad Aliakbari (firstname.lastname@example.org, office: 6329)
- Hubert Andersson, email@example.com
- Anton Bornecrantz, firstname.lastname@example.org
- Xinglu Gong, email@example.com
- Yue Lou, firstname.lastname@example.org
- Filip Ronnblad, email@example.com
This course will follow closely the following book:
- C. M. Bishop, Pattern Recognition and Machine Learning. Berlin, Heidelberg: Springer-Verlag, 2006.
The book is available for free download in the link: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf
An errata is available in the link: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/prml-errata-1st-20110921.pdf
Additional course material
Slides for the lectures and exercises for the tutorial sessions will be made available on Canvas.
Lectures are typically on Tuesdays, Thursdays, and Fridays 13:15-15:00. However, there may be some exceptions. Please see the course memo for the preliminary schedule for the lectures. Please check Canvas for updates.
Tutorials are typically on Tuesdays and Thursdays 15:15-17:00. Please check Canvas for updates.
The final score (maximum score is 100) will be decided based on the points collected via the theoretical assignments (maximum score 30), the python assignments (maximum score 35) and the final (take-home) python assignment (maximum score 35).
The syllabus page shows a table-oriented view of course schedule and basics of course grading. You can add any other comments, notes or thoughts you have about the course structure, course policies or anything else.
To add some comments, click the 'Edit' link at the top.