Course syllabus

Course-PM

TIF360 / FYM360 TIF360 / FYM360 Advanced machine learning with neural networks lp4 VT24 (7.5 hp)

Course is offered by the department of Physics

Contact details

Examiner

Giovanni Volpe giovanni.volpe@physics.gu.se 

Lecturers

Mats Granath Mats.Granath@physics.gu.se
Kristian Gustafsson Kristian.Gustafsson@physics.gu.se
Bernhard Mehlig Bernhard.Mehlig@physics.gu.se
Daniel Midtvedt daniel.midtvedt@physics.gu.se 

Lectures and TAs for Laboratories and Projects

Agnese Callegari, agnese.callegari@physics.gu.se 
Daniel Midtvedt daniel.midtvedt@physics.gu.se 
Henrik Moberg, henrik.kleinmoberg@chalmers.se  [tutors groups 1, 6, 11, 16]
Benjamin Midtvedt,  benjamin.midtvedt@physics.gu.se  [tutors groups 2, 7, 12, 17]
Jesus Pineda,  jesus.pineda@physics.gu.se  [tutors groups 3, 8, 13, 18]
Harshith Bachimanchi, harshith.bachimanchi@physics.gu.se [tutors groups 4, 9, 14, 19]
Yu-Wei Chang, yu-wei.chang@physics.gu.se  [tutors groups 5, 10, 15, 20]

Lecturers and TAs for Homeworks

Homework A: Graph neural networks
Lecturer: Mats Granath Mats.Granath@physics.gu.se
TA: Marvin Richter marvin.richter@chalmers.se 

Homework B, Playing Tetris using Reinforcement Learning
Lecturer: Kristian Gustafsson Kristian.Gustafsson@physics.gu.se
TA: Henrik Moberg, henrik.kleinmoberg@chalmers.se 

Homework C: Video frame prediction using transformers
Lecturer: Daniel Midtvedt daniel.midtvedt@physics.gu.se 
TA: Jesus Pineda, jesus.pineda@physics.gu.se

Homework D: Non-linear time-series analysis with reservoir computers
Lecturer: Bernhard Mehlig Bernhard.Mehlig@physics.gu.se
TA: Ludvig Storm ludvig.storm@physics.gu.se

Guest Lecturer

Charles Martinez, G Research charles.martinez@gresearch.com 

Dr Charles Martinez is the Academic Relations Manager at G-Research. Charles started his studies as a physicist at University Portsmouth Physics department's MPhys programme, and later completed a PhD in Phonon interactions in Gallium Nitride nanostructures at the University of Nottingham. Charles then worked on indexing and abstract databases at the Institution for Engineering and Technology (IET) before moving into sales in 2010. Charles' previous role was as Elsevier's Key Account Manager, managing sales and renewals for the UK Russell Group institutions, Government and Funding body accounts, including being one of the negotiators in the recent UK ScienceDirect Read and Publish agreement. Since leaving Elsevier Charles is dedicated to forming beneficial partnerships between G-Research and Europe's top institutions, and is living in Cambridge, UK.

Course representatives

Damodar Datta Kancharla damodark@student.chalmers.se MPCAS
Shuyue Ding  shuyueding86@gmail.com MPCAS
Erik Håkansson erik.hakansson96@gmail.com MPCAS
Max Johansson max@peter-johans.com MPCAS
Felix Waldschock felixwal@chalmers.se MPCAS

Course purpose

This course introduces students to recent developments and state-of-the-art methods in machine learning using artificial neural networks. This advanced course builds on Machine learning with neural networks (FFR135) and provides an in-depth analysis of many of the concepts and algorithms that were briefly introduced in that course, with particular emphasis on applications in the natural and engineering sciences. The goal is to become familiar with several advanced machine-learning methods, and to code them efficiently in Python using current neural-network packages. An essential part of the course are projects in deep learning, recurrent learning, and reinforcement learning.

Schedule

TimeEdit, until +2 week from now (graphic)

TimeEdit, complete schedule: (graphic),  (text)

Course literature

There is no single book that covers all the topics in the course, but we will provide Jypiter notebooks demonstrating the techniques that will be taught during the course.

Suggested reading for background: Bernhard Mehlig, Machine Learning with Neural Networks. Cambridge University Press  (2022)  www.chalmersstore.seLinks to an external site. or www.adlibris.comLinks to an external site.

Course design

Tuesday, March 19 8:00-12:00 HC1	Introductory Lectures	Bernhard Mehlig & Giovanni Volpe
Wednesday, March 20 8:00-12:00 F-T7203 F-T7204	Deep Learning Workshop Convolutional Neural Networks for Image Analysis
Friday, March 22 8:00-12:00 F-T7203 F-T7204	Deep Learning Workshop DeepDreams and Neural Style Transfer
March 22	Project Groups Announced
Tuesday, March 26 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Encoder-Decoders for Latent Space Manipulation
Wednesday, March 27 8:00-10:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop U-Nets for Image Transformation
Wednesday, March 27 10:00-12:00 FB	Guest Lecture	Charles Martinez, G Research
Tuesday, April 9 8:00-12:00 F-T7203 F-T7204 FT4011	Initial Discussion of Project Ideas First meeting with Tutor	Project TAs	Submit your project proposal before this meeting
Wednesday, April 10 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Reservoir Computing for Time-Series Analysis & Recurrent Neural Networks for Time-Series Analysis
Friday, April 12 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Self-supervised Learning
Tuesday, April 16 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Reinforcement Learning for Strategy Optimization
Wednesday, April 17 8:00-12:00 FT4011	Homework Help Session	Howework Lectures and TAs
Friday, April 19 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Introduction to Attention and Transformers for Sequence Processing & Introduction to Graph Neural Networks for Graph Data Analysis
Tuesday, April 23 8:00-12:00 F-T7203 F-T7204 FT4011	Follow-up on Project Second meeting with Tutor	Project TAs	Submit your project outline before this meeting
Wednesday, April 24 8:00-12:00 FT4011	Homework Help Session	Howework Lectures and TAs
Friday, April 26 8:00-12:00 & 13:00-17:00 FT4011	Homework Correction - First Session	Howework Lectures and TAs
Friday, May 3 8:00-12:00 & 13:00-17:00 FT4011	Homework Correction - Second Session	Howework Lectures and TAs
Tuesday, May 7 8:00-10:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Generative Adversarial Networks for Image Synthesis (1)
Tuesday, May 7 10:00-12:00 F-T7203 F-T7204 FT4011	Homework Help Session	Howework Lectures and TAs
Wednesday, May 8 8:00-10:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Generative Adversarial Networks for Image Synthesis (2)
Tuesday, May 14 8:00-12:00 & 13:00-17:00 FT4011	Homework Correction - Third Session	Howework Lectures and TAs
Wednesday, May 15 8:00-12:00 F-T7203 F-T7204 FT4011	Follow-up on Project & Check of Project Poster Draft Third meeting with Tutor	Project TAs	Submit the project poster draft before this deadline
Thursday, May 16	Final Project Poster		Submit the final project poster to be printed before this deadline
Friday, May 17 8:00-12:00 F-T7203 F-T7204 FT4011	Deep Learning Workshop Diffusion Models for Data Representation and Exploration
Tuesday, May 21 8:00-12:00 & 13:00-17:00 FT4011	Check of Project Report Draft Fourth meeting with Tutor	Project TAs	Submit the project report draft before this deadline
Wednesday, May 22 8:00-12:00 F-T7203 F-T7204 FT4011	Homework Correction - Make-up Session	Howework Lectures and TAs
Friday, May 24 13:00-17:00 Canyon (MC2)	Project Poster Presentation
Friday, June 7	Final Project Report		Submit the final project report by this deadline

Learning objectives and syllabus

Learning objectives:

Knowledge and understanding

-       Describe the different available neural network models with their advantages and disadvantages

-       Find relevant literature to keep up with this quickly advancing field

Skills and ability

-       Implement a broad range of state-of-the-art neural network models

-       Train and validate these models 

-       Optimize these models for a specific task 

-       Plan, manage and execute a small scale project in the field

-       Write a report of their results of the project

Judgement and approach

-       Critically analyse the advantages and disadvantages of the available neural network models

-       Benchmark the results of a neural network models against other models

-       Critically evaluate and discuss advances in the field of neural networks

 

Link to the syllabus on the Chalmers Studieportalen:
Study planLinks to an external site.

Link to the syllabus on the GU website:
https://utbildning.gu.se/kurser/kurs_information?courseid=FYM750Links to an external site. 

Examination form

There will be no written final examination in this course.

The examination consists of homework assignments and a group project.
Specifically, the examination is based on

• 30% homeworks (10% for each)
• 20% final project presentation
• 50% final project report

A necessary (but not sufficient) requirement for passing grade is that 5/10 points are achieved in each homework.

Homework information

The purpose of the homework problem sets is to provide you with hands-on experience of advanced deep learning techniques. You are strongly encouraged to team up and collaborate on the homework, to ask your classmates if you are stuck at some point, and to assist classmates in need of advice. But you must write and run your own code, and have your own work assessed.

Grading: Out of the four available homeworks, you need to solve and get assessed on three homeworks.
Each homework accounts for 10 points. A necessary (but not sufficient) requirement for passing grade is that 5/10 points are achieved in each homework.
Overall these three homeworks account for 30% of your total course grade (10% for each).

Assessment: Be prepared for the assessment: Generate all your figures and/or videos up front. There will be no time for running code during the assessment. Check that you have answered all questions carefully. You should be able to give reasoned answers regarding your programming choices as well as being able to discuss the implications of your results. 

Project information

Group project

The purpose of the group project work is to provide you with training in

• Developing your own small research-style project.
• Executing a collaborative project.
• Presenting your independent work in writing and as a poster.

The deliverables are a poster presentation (20% of the course grade) and a written report (50% of the course grade).

The final poster to be printed should be submitted by May 16, 2024.

The project presentation will be on May 24, 2024.

The final written report is due by June 7, 2024. No late reports will be accepted.

General information

You will be organized into groups:

• Each group will be paired with another group, and they will act as review groups for each other.  The pairing will be done between groups with the same tutor.
• Each pair of groups gets a tutor assigned and has 4 meetings, 45 minutes each, with this tutor during the course.
• The tutor’s job is to give you general advice on anything you need as best as he or she can. The tutor’s job is not to formulate your project for you, nor to debug your code.
• Should you experience problems with the group assembly, you need to take this up with your tutor as soon as possible.

To start your project you need a challenge you will address. You can get a challenge in one of the popular challenge websites, such a Kaggle, CodaLab, etc.

The first meeting with the tutor is to ensure that you’re embarking on a feasible project. Before this meeting you should:

• Identify your challenge.
• Write a two-paragraph proposal: one explaining the background of the project, possibly with references to relevant literature, and another outlining what you aim to do and why.
• Discuss your project with your review group and revise your proposal accordingly.
• Send the proposal to your tutor.
• Prepare for the meeting; prepare questions, comments, simulations, results, or whatever is relevant for a most efficient outcome of the meeting.  

The following meetings with the tutor are to ensure that you have progress and direction so that you are on course to completion. Before these meetings you should:

• Have ready an implementation of your model.
• Have a set of preliminary results from simulations.  
• Have ready the outline of a report.
• Have met with your review group to discuss model, results and report outline.
• Send the report outline to your tutor.
• Prepare for the meeting; prepare questions, comments, simulations, results, or whatever is relevant for a most efficient outcome of the meeting.  

Poster

The poster presentation of the project should clearly show what you have done.
Put the emphasis on what your problem formulation is, a general discussion about how you tackled it, what problems you had, and what your conclusions are.

Evaluation criteria for poster

Criteria that will be considered are:

• Structure: Is the poster well structured?
• Clarity: Is it clear from the poster what is being investigated?
• Format: Does the poster fit the requested format? 
• Motivation: Is it explained well why the project is important? 
• Conclusion: Are conclusions and implications clear? 
• Methods: Are the methods explained sufficiently but not excessively?
• Presentation quality: Is the poster engaging and visually appealing?
• Interactive Engagement: How effectively do the presenters engage with the audience during the presentation?

Report

The report should be written in RevTex 4.2 (see https://journals.aps.org/revtexLinks to an external site.), using the format of Physical Review X with a limit of 8 pages in double-column reprint format (including pictures and references). If needed, you can have an extra document containing an unlimited amount of Supplementary Materials.

Structure your report like a scientific article, with an abstract summarising the rationale and results of the project; an introduction shortly explaining its background and motivating why the question is interesting; methods and/or results section(s) describing your model and what you do with it; a discussion section; and ending with a conclusion. Put some work into the discussion and conclusion sections. This is where you demonstrate that you truly understand the implications of your work, including shortcomings and uncertainties. It is important that the discussion does not fall out as a simple summary of what the figures show.

Write in plain language and write enough to say what you need to say. Don't think "the report feels short, better throw in some extra figures." If you can say the same thing in fewer words, do so.

Be sure to reference any source you use. The report should be readable and understandable on its own, but there is no need to reproduce for example derivations of equations from your sources; a citation is enough.

Figures are an important part of the report, but only those that substantially contribute to your analysis should be included. Make sure that each figure is well designed with informative captions (not just "Fig. X shows how quantity A depends on quantity B"). If you find it hard to do this, you probably are not clear on why you include an figure, so cut it out.

Most importantly, use your own judgement and try to write a report you would like to read.

Evaluation criteria for report

Criteria that will be considered are:

• Structure: Is the project report well structured (see above)?
• Motivation: Does the project have a clear research question?
• Originality: How original is the research question?
• Background: Is the background of the research question well explained, including references to the literature?
• Methods: Are the methods and analysis appropriate for convincingly answering the research question?
• Execution: Does the project answer the initial or iterated research question?
• Figures: Are the figures included informative with descriptive captions?
• Format: Is the format of the report as required?
• Outlook: Are the implications of the results clearly stated?
• Contributions: Is there a contributions statement in the report? 

Note that the research question does not have to be answered in the affirmative. A negative result is equally valid as long as an analysis is carried out that properly explains the negative result and what eventually would work better. 

Calculation of final grade

You can get at most 30 points from the homework and 70 points from the project. The maximum combined score is 100.

The grade limits will be 50 points for grade 3, 70 points for grade 4, and 90 points for grade 5.

For GU the limits will be 50 points for G and 80 for VG.