Course syllabus

Teachers

Bernhard Mehlig (lecturer)  Bernhard.Mehlig at gu.se
Navid Mousavi (TA)
Ludvig Storm (TA)
Linus Sundberg (TA)

Course representatives

G. Aertgeerts glenaertgeerts at gmail.com
M. Andersson melwina at student.chalmers.se
P. Seenivasan purusothaman.seenivasan1997 at gmail.com
Y. Xuan yuner at student.chalmers.se
E. Östling emrikos at student.chalmers.se

Important

News regarding the course are posted on the discussion forum (see below). Therefore it is important that all students register there. Let me know if you have problems.

Welcome

Welcome to Artificial Neural Networks 2023. How to prepare? Login to the online system OpenTA (link on the menu on the left) to do the preparatory maths exercises. Sign up for the discussion forum and pose any OpenTA questions there.

Signup to discussion forum.

Link to discussion forum.

Link to preliminary schedule in Time Edit.

Important dates

First lectures on Tuesday August 29 in GD salen.

Deadline homework problem 1: Sept. 11 (2023)
Deadline homework problem 2: Oct. 2 (2023)
Deadline homework problem 3: Oct. 30 (2023)

Contents

1. Introduction

Part I Hopfield models
2. Deterministic Hopfield networks
3. Stochastic Hopfield networks.
4. The Boltzmann distribution

Part II Supervised learning
5. Perceptrons
6. Backpropagation
7. Deep learning
8. Convolutional networks
9. Recurrent networks

Part III Learning without labels
10. Unsupervised  learning
11. Reinforcement learning

Preliminary schedule

Link to schedule in Time Edit.

Lecture 1 (Aug. 29) 13:15 Introduction (Chapter 1, Slides) Bernhard Mehlig
Lecture 2
(Aug. 29) 15:15 Hopfield model, one-step error probability (Chapter 2)  Bernhard Mehlig
Lecture 3 (Aug. 31) 15:15 Linear algebra (repetition, summary) Bernhard Mehlig
Lecture 4 (Sept. 1) 13:15 Hopfield model (Chapter 2) Bernhard Mehlig
                                
For discussion of energy function, see separate video.
Lecture 5 (Sep. 5) 13:15 Hopfield model (Chapter 3)  Bernhard Mehlig
                                 
For derivation of mean-field theory, see separate video
                               For calculation of critical storage capacity, see separate video
Computer lab 1 (Sep. 7) 13:15 HW1 Navid Mousavi, Ludvig Storm, Linus Sundberg
Lecture 6 (Sep. 7) 15:15 Monte-Carlo simulation (Chapter 4)  Bernhard Mehlig
Lecture 7 (Sep. 8) 13:15 Boltzmann machines (Chapter 4)  Bernhard Mehlig
Exercise class 1 (Sep. 8) Exercises 2.3, 2.9, 2.10 (note erratum), 2.12, 3.4, Linus Sundberg
Lecture 8 (Sep. 12) 13:15 Simple perceptrons (Chapter 5)  Bernhard Mehlig
Exercise class 2 (Sep. 12) 15:15 Exercises 2.8, 3.5, 4.2, 4.6 Linus Sundberg
Lecture 9 (Sep. 14) 15:15 Sections 5.3, 5,5, 5.6  Bernhard Mehlig
                                For capacity of simple perceptron, see separate video.
Computer lab 2 (Sep. 15) 13:15 HW2 Navid Mousavi, Ludvig Storm, Linus Sundberg
Lecture 10 (Sep. 19) 13:15 Section 6.1, Bernhard Mehlig
Exercise class 3 (Sep. 19) 15:15 Exercises 5.4, 5.5, 5.6, 5.8 Ludvig Storm
Lecture 11 (Sep. 21) 13:15 Sections 6.2, 6.3  Bernhard Mehlig
Exercise class 4 (Sep. 21) 15:15 Exercises 4.10, 4.11, 6.2, 6.8  Bernhard Mehlig
Lecture 12 (Sep. 22) 13:15 Section 6.4, 6.5  Bernhard Mehlig
Lecture 13 (Sep. 22) 15:15 Sections 7.1 and 7.2   Bernhard Mehlig
Lecture 14 (Sep. 26) 13:15 Sections 7.3 and 7.4  Bernhard Mehlig
Exercise class 5 (Sep. 26) 15:15 Exercises 2.13, 4.4, 6.1, 6.5, 6.6 Bernhard Mehlig
Computer lab 3 (Sep. 28) 13:15 HW2 Navid Mousavi, Ludvig Storm, Linus Sundberg
Lecture 15 (Sep. 28) 15:15 Section 7.5
  Bernhard Mehlig
           
Please read Chapter 8 at home. I'll take questions on Sept. 28 and 29.
Lecture 16 (Sep. 29) 13:15 Sections 7.6 and 9.1 Bernhard Mehlig
Lecture 17 (Oct. 3) 13:15 Section 9.2 Bernhard Mehlig
Exercise class 6 (Oct. 3) 15:15 Exercises 5.11, 6.7, 7.2, 7.4 Bernhard Mehlig
Lecture 18 (Oct. 5) 13:15 Sections 9.3 and 9.5  Bernhard Mehlig
Computer lab 4 (Oct. 5)  15:15 HW3 Navid Mousavi, Ludvig Storm, Linus Sundberg
Lecture 19 (Oct. 10) 13:15   Sections 10.1, 10.2, and 10.3 Bernhard Mehlig
Lecture 20 (Oct. 10) 15:15   Advanced neural networks: from robots to ChatGPT Giovanni Volpe
Lecture 21 (Oct. 12) 13:15   Summary unsupervised learning  Bernhard Mehlig
Exercise class 7 (Oct. 12) 15:15 Exercises 7.6, 7.7, 8.1, 8.3 Navid Mousavi
Lecture 19 (Oct. 13) 13:15  Chapter 10 Bernhard Mehlig
Exercise class  8 (Oct. 13) 15:15 Exercises 9.3, 9.4,  9.5, 10.2 Bernhard Mehlig
Lecture 23 (Oct. 17) 13:15 Guest Lecture Hampus Linander
Exercise class 8 (Oct. 17) 15:15 Navid Mousavi
Computer lab 5 (Oct. 19) 13:15 HW3 Navid Mousavi, Ludvig Storm
Lecture 24 (Oct. 19) 15:15 Chapter 11  Bernhard Mehlig
Lecture 25 (Oct. 20) 13:15 Chapter 11 Bernhard Mehlig
Exercise class 10  (Oct. 20) 15:15 Exercises 10.3, 10.14 & exam preparation Bernhard Mehlig
Extra Exercise class  (Oct. 23) 10:00 (PJ salen)  Exercises 2.9, 4.2 Bernhard Mehlig

Chapters and Sections refer to the course book below.

Course book

B. Mehlig, Machine learning with neural networks, Cambridge University Press (2021).
Errata for Machine Learning with Neural Networks (October 18, 2022)

Examination

Credits for this course are obtained by solving the homework problems (solutions of examples and programming projects) and by a written examination. There are three sets of homework problems. Each of the three gives at most 3 points. The exam gives at most 15 points, resulting in a maximum of 24 points.

To pass the course,  it is necessary to obtain at least 6 points in the written exam, and to have at > 13.5 points in total.

Passing grades:
Chalmers: 3: >13.5p; 4: >17p, 5: >21.5p
GU: G: >13.5p; VG: >19.5p
ECTS: C: >13.5p; B: >17p; A: >21.5p

OpenTA

This course uses the OpenTA online system developed by Stellan Östlund and Hampus Linander for exercises, homework, and exam preparation.

Rules for homework submissions

Same rules as for written exams apply: it is not allowed to copy any material from anywhere unless reference is given. All students must write their own computer programs and submit their own solutions. All submissions are handled via OpenTA. For all homework sets, your program code must be uploaded to OpenTA.

Keep a backup of your solutions to the OpenTA questions, of your submitted PDF files as well as the answers you typed in. The system does not store your answers after December 2024. If you take a re-exam in January or August 2023 you will be asked to re-submit all answers and OpenTA scores.

Your OpenTA points are valid for the two re-exams in January and August 2024. Please contact any of the teachers if you need guidance for your exam preparation, or if you have questions about the coming re-exams. To pass the course in future academic years (for instance 2024/2025) you need to redo the OpenTA problems for that academic year.

For some OpenTA problems, you are asked to submit your answer in the form of a PDF file. It must be a single A4 page with 12pt single-spaced text, and with 2cm margins. LateX template. The page may contain at most one Figure and/or one Table with the corresponding Figure and/or Table caption, in addition to the text discussing the results shown in the Figure/Table. It is not necessary to write a full page for each problem, but you must explain/describe what you have done and clearly state your answers/results to the questions and your conclusions. When necessary, you must discuss possible errors and inaccuracies in your results. If you are asked to plot results/make graphs, you do this in a Figure with legible axis labels and tic labels. All symbols and lines must be explained in the Figure or in a caption. The Figure may consist of separate panels. Refer to them as 'left panel', 'right panel', 'bottom panel', etc. (or alternatively label them 'a', 'b',...).

In addition you need to upload a PDF file with the computer code you used to generate your results. No length restriction applies for this file.

URKUND. For each homework, concatenate the PDF files you uploaded to OpenTA for each question, and send this PDF file electronically to Bernhard.Mehlig.chalmers@analys.urkund.se. Further instructions for the email to URKUND: Subject =  [FFR135] (Chalmers) or [FIM720] (GU). Attach the PDF file with filename firstname-lastname-hw1-#.pdf for the first HW, where the counter # distinguishes different files if you submit several.  Note: it is not possible to retract files from URKUND, neither is it possible to correct your submission to URKUND. Only one submission of any PDF file, otherwise you may alert the plagiarism software. Information about the URKUND system can be found here.

Deadlines are sharp. Late submissions are not accepted.

Rules for homework resubmissions

Most homework tasks are automatically graded. For these tasks, you only get the green light from OpenTA. You can try as often as you want (before the deadline).

Some homework tasks require you to submit a one-page PDF file with your results and discussion. These are manually graded. If we find errors or problems in these solutions you will be notified (text message on OpenTA) and you are allowed to resubmit. The final deadline for resubmissions is to be determined, a few days after the exam. After that date, the system does not accept any resubmissions. If you resubmit before that deadline and your solution is correct, you get full points for that task.

Written exam

Allowed material for exam: course book, only printed by Cambridge University Press, no written annotations (underlining/highlighting allowed).

The exam covers the material presented in the lectures as well as the homework problems. Old exam questions are given in the course book. Solutions.

Old exams with solutions
October 2023
August 2023
January 2023
October 2022
January 2022
October 2020

Old exams without solutions
October 2021
January 2021
January 2020
October 2019
January 2019

Date for written exam, deadline for registration for exam. Please follow this link. Course code FFR135.

If date & time of the exam collide with another exam you must take, then you must follow the steps outlined here.

If you don't pass the exam

Your OpenTA points are valid for the two re-exams in January and August 2024. Please contact any of the teachers if you need guidance for your exam preparation, or if you have questions about the coming re-exams. To pass the course in future academic years, you need to redo the OpenTA problems for that academic year.

Changes from last year

Maximal number of points obtainable in exam/homework and minimum number of points in exam needed to pass changed compared with last year. See above for the current boundaries.

Course summary:

Date Details Due