Course syllabus

Course PM

This page contains the program of the course: lectures, exercise sessions and computer labs. Other information, such as learning outcomes, teachers, literature and examination, are in a separate course PM.

Registering for courses for PhD students: for registering, please, sent me mail to larisa@chalmers.se

PhD students taking a Chalmers Master’s course
- apply to the course by contacting the course examiner

- register to the exam:
- via Doktorandportalen (if you are a Chalmers PhD)
- by sending an e-mail to utb-adm.mv@chalmers.se (GU PhD)
- the course will be registered when the result is reported in Ladok.

Here is the link to the course Numerical methods and machine learning algorithms   for solution of inverse problems

Program

The schedule of the course is in TimeEdit.

The course will be given in the hybrid format: the theoretical lectures will be in

live or in Zoom and the computer  sessions  can be performed in computer labs

via supervision in Zoom: students can ask questions   in Zoom  as well as come to my office  during  time of  computer labs. 

We will follow schedule in the Table below, see all dates and Zoom links for lectures  and comp.labs in Time Edit.

 

Zoom link for all lectures and comp.labs:

Meeting ID: 677 1750 7274

Passcode:   644985

 

List of formulas for examination

Proposition for groups for computer labs ( I divided Chalmers/GU students separately into 8 groups, 2 persons in every group):  TMA265  and MMA600.

 

 

Lectures  and comp.ex. in 2023

Day Time  Place Remarks

MONDAYS:  Pascal

28.08 

04.09,

11.09,

18.09,

25.09,

02.10,

09.10,

16.10

13:15-15:00 Zoom/Pascal  lecture Lecture

WEDNESDAYS:

30.08,

06.09,

13.09,

20.09,

27.09,

04.10,

11.10,

18.10

13:15-15:00 MVF24, MVF25, Zoom supervision Computer exercises

THURSDAYS:  Pascal

31.08,

07.09,

14.09,

21.09,

28.09,

05.10,

12.10,

19.10

10:00-11:45 Zoom/Pascal Lecture

FRIDAYS:

01.09,

08.09,

15.09,

22.09,

29.09,

13.10,

20.10

13:15-15:00 MVF24, MVF25, Zoom  supervision Computer exercises
24.10.2023 14.00-18.00   exam Examination
?.01.2024 14.00-18.00   exam Examination

 

Grades at Examination (written examination + bonus points)

Grades Chalmers Points Grades GU Points
-   < 14 U < 14
3 14-20 G 14-25
4 21-25 VG > 25
5 > 25

Changes compared to the last occasion

There are new computer labs in Matlab and C++/PETSc:

Computer Lab. 1: Solution of least squares problem

Computer Lab. 2:  Least squares and machine learning algorithms for classification problem

Computer Lab. 3: Regularized least squares and machine learning algorithms for classification problem

Computer Lab. 4:  Principal component analysis for image recognition

Computer Lab. 5:  Solution of Helmholtz equation 

 

Deadlines for computer exercises  and homeworks:

Homeworks 1           :                    12  September

Homework 2:                                  19 September

Comp.ex. 1 :                                        3 October

Homeworks 3 and  4:                   10 October

Comp.ex. 2:                                         17 October

Comp.ex. 3, 4:                                     24 October

Comp.ex.5 :                                          30 October

 

  • Lecture 1
    Introduction and organization of the course. Introduction to linear algebra and numerical linear algebra. If this looks unfamiliar you should better consult you former literature in linear algebra and refresh your knowledges. We will concentrate on the three building bricks in Numerical Linear Algebra (NLA) : (1) Linear Systems, (2) Overdetermined systems by least squares, and (3) Eigenproblems. The building bricks serve as subproblems also in nonlinear computations by the idea of linearization. We considered example of application of linear systems: image compression using SVD, image deblurring. We introduced some basic concepts and notations: transpose, lower and upper triangular matrices, singular matrices, symmetric and positive definite matrix, conjugate transpose matrix, row echelon form, rank, cofactor.

Lecture 1

  • Lecture 2
  • IEEE system and floating-point numbers.
  • We will discuss  perturbation theory in polynomial evaluation.   See section 8.2.1 in the course book.
  • Lecture 2
  • Lecture 3
    We will discuss why we need norms: (1) to measure errors, (2) to measure stability, (3) for convergence criteria of iterative methods. Sherman - Morrison formula.  See   chapter 6 in the course book.
  • System of linear equations. Gaussian elimination, LU-factorization. Gaussian elimination (and LU factorization) by outer products (elementary transformations). Pivoting, partial and total.  See chapter 8 in the course book.
  • Lecture 3
  • Lecture 4
    We will discuss the need of pivoting and uniqueness of factorization of A. Different versions of algorithms of Gaussian elimination. Error bounds for the solution Ax=b. Roundoff error analysis in LU factorization. See  sections 8.1.2, 8.2.2, 8.2.3, 8.2.4  in the course book.
  • Estimating condition numbers. Hagers's algorithm.
  • Lecture 4
  • Lecture 5
    Componentwise error estimates. Improving the Accuracy of a Solution for Ax=b: Newton's method and equilibration technique. Convergence of the Newton's method. Real Symmetric Positive Definite Matrices. Cholesky algorithm.
  • See    sections 8.2.5, 8.3, 8.4.1 of the course book.
  • Band Matrices, example: application of Cholesky decomposition in the solution of ODE. See sections 8.4.2, 8.4.3 in the course book.
  • Example: how to solve Poisson's equation on a unit square using LU and Cholesky factorization. See section 8.4.4 in the course book.
  • Lecture 5
  • Lecture 6
    Linear least squares problems. Introduction and applications. The normal equations. Example: Polynomial fitting to curve.  See introduction to chapter 9  and sections 9.1, 9.3 in the course book.

  • QR and Singular value decomposition (SVD). QR and SVD for linear least squares problems.  Example of application of linear systems: image compression using SVD in Matlab.   See sections 9.4, 9.6 in the course book.
  • Lecture 6
  • Lecture 7
  •  Least squares and classification algorithms. 
  • See sections 9.2 in the course book and the  the paper Numerical analysis of least squares and perceptron learning for classification problems, https://arxiv.org/abs/2004.01138  for classification.
  • Machine learning algorithms for classification.
  • Lecture 7
  • Lecture 8 
  • Solution of nonlinear least squares problem.
  •  See section 9.2 in the course book.
  • SVD and Principle Component Analysis (PCA)  to find patterns and object orientation.   
  • Example of PCA.   PCA for recognition of handwritten digits. 
  • Lecture 8
  • Lecture 9
  • Householder transformations and Givens rotations. QR-factorization by Householder transformation. Examples of performing Householder transformation for QR decomposition and tridiagonalization of matrix.
  • See section 9.5.1 of the course book.
  • Givens rotation. QR-factorization by  Givens rotation. See section 9.5.2 of the course book.
  • Moore-Penrose pseudoinverse. Rank-Deficient Least Squares Problems.  See sections 9.6.1, 9.6.2 of the course book.
  • Lecture 9
  • Lecture 10
  • Introduction to spectral theory, eigenvalues, right and left eigenvectors. Similar matrices. Canonical forms: Jordan form.
    Canonical forms: Jordan form and Schur form, real Schur form. Gerschgorin's theorem. Perturbation theory, Bauer-Fike theorem. See chapter 5 of the course book.
  • Discussed algorithms for the non-symmetric eigenproblems: power method, inverse iteration. See chapter 10, sections 10.1, 10.2  in the course book.
  • Lecture 10
  • Lecture 11
    Discussed algorithms for the non-symmetric eigenproblems: inverse iteration with shift, orthogonal iteration, QR iteration and QR-iteration with shift. Hessenberg matrices, preservation of Hessenberg form. Hessenberg reduction. Tridiagonal and bidiagonal reduction. Regular Matrix Pencils and Weierstrass Canonical Form.
  • See sections 10.2, 10.3,10.4,10.5, 10.6, 10.7  in the course book. See section 5.3 for matrix pencils.
  • Lecture 11
  • Lecture 12
    Regular Matrix Pencils and Weierstrass Canonical Form. Singular Matrix Pencils and the Kronecker Canonical Form. Application of Jordan and Weierstrass Forms to Differential Equations. Symmetric eigenproblems Perturbation theory: Weyl's theorem. Corollary regarding similar result for singular values. Application of Weyl's theorem: Error bounds for eigenvalues computed by a stable method. Courant-Fischer (CF) theorem. Inertia, Sylvester's inertia theorem. Definite Pencils. Theorem that the Rayleigh quotient is a good approximation to an eigenvalue. Algorithms for the Symmetric eigenproblem: Tridiagonal QR iteration, Rayleigh quitient iteration.
  • In the course book see section 5.3 for matrix pencils  and sections 11.1,11.2  for algorithms for the symmetric egenproblem.
  • Lecture 12
  • Lecture 13
    Theorem that the Rayleigh quotient is a good approximation to an eigenvalue. Algorithms for the Symmetric eigenproblem: Tridiagonal QR iteration, Rayleigh quitient iteration, Divide-and-conquer algorithm. QR iteration with Wilkinson's shift. Divide-and-conquer, bisection and inverse iteration, different versions of Jacobi's method.
  • See sections 11.2, 11.3,11.4, 11.5 in the course book.
  • Lecture 13
  • Lecture 14
    Algorithms for symmetric matrices (continuation): different versions of Jacobi's method. Algorithms for the SVD: QR iteration and Its Variations for the Bidiagonal SVD. The basic iterative methods (Jacobi, Gauss-Seidel and Successive overrelaxation (SOR)) for solution of linear systems. Jacobi, Gauss-Seidel and SOR for the solution of the Poisson's equation in two dimension. Study of Convergence of Jacobi, Gauss-Seidel and SOR. 
  • In the course book see sections 11.5, 11.6,11.7,11.8 for algorithms for symmetric matrices and Chapter 12  for iterative methods for solution of linear system.
  • Lecture 14

Lecture 15
 Introduction to the Krylov subspace methods. Conjugate gradient algorithm. Preconditioning for Linear Systems. Preconditioned conjugate gradient algorithm. Common preconditioners. 
In the course book see sections 11.5, 11.6,11.7,11.8 for algorithms for symmetric matrices and Chapter 12  for iterative methods for solution of linear system.

Lecture 15

Back to the top

Homeworks

  • To pass this course you should do 2 compulsory home assignments before the final exam. Choose any 2 of 4 assignments presented here: 
  • Homeworks
  • Homeworks  should do  done  individually  (not in groups).
  • Sent pdf file with your assignment to my e-mail  larisa@chalmers.se
    before deadline (see the course page "Program" for deadlines for every home assignment). Handwritten home assignments can be left in the red box located beside my office.

 

Recommended exercises in the course book

 

Chapters Exercises
8 8.7, 8.10, 8.16
9 9.5, 9.7, 9.8, 9.9, 9.11, 9.12, 9.14, 9.15
11 11.1 - 11-13

 

Back to the top

Computer labs

 

Proposed division into groups for Comp.labs 

Computer labs with instructions

Reference literature:

  1. Learning MATLAB, Tobin A. Driscoll. Provides a brief introduction to Matlab to the one who already knows computer programming. Available as e-book from Chalmers library.
  2. Physical Modeling in MATLAB 3/E, Allen B. Downey
    The book is free to download from the web. The book gives an introduction for those who have not programmed before. It covers basic MATLAB programming with a focus on modeling and simulation of physical systems.

 

Back to the top

Course summary:

Date Details Due