Course syllabus

Learning dynamical systems using system identification, SSY 230, lp4 VT20 (7.5 hp)

Course is offered by the department of Electrical Engineering

Lecturer: Jonas Sjöberg, jonas.sjoberg@chalmers.se, 031-772 18 55

Assistants: Angelos Toytziaridis,  angtoy@chalmers.se, 031-772 1712

Prerequisites: Basic stochastic processes recommended corresponding to an undergraduate course. Basic knowledge about linear system theory necessary.

The course ESS101 - Modelling and simulation, gives an introduction to system identification. Those who have not taken that course are strongly advised to quickly read that part of the book which concerns system identification.

The course is about estimating models of dynamical systems using observations of the input and output signals from the system. The course covers theory explaining how practical methods (algorithms) work. Properties like model quality, and how it depends on the data is explained. Also, "model quality" can mean different things depending on the expected use of the model. Assuming that the system to be identified is also influenced by disturbances of various kinds, it is often assumed that if the number of data goes to infinity, the influence of the disturbances will disappear. In the course it will be explained when this is true, and also measures are given how fast the model improves when the number of data is increased. Such knowledge is sometimes necessary to be able to decide if the model is usable, or to what extent it can be trusted.

System identification is about estimating models from data, and an important part of this is practical experience. Hence, we will work with data and computer tools through-out the whole course. There will be three min-projects where practical system identification problems are solved. There will also be problem solving sessions where we have a mixture of theoretical problems and problems where we work with data. If you have a laptop, please, bring it to the lectures so that you can use it during the class.

The first part of the course focus on general estimation of functions using data, and we will use material from an online course Statistical learning. Note that you should not use the material directly from that course, but use the link further down. This first part of the course will be given using the "flipped classroom" technique, see further down. It covers general insights which are important in machine, learning, neural networks, data-mining, and other fields which have received recent attention. System identification has connections to these fields with the important aspect that we consider models of dynamical systems.

Literature: For the two first weeks we will use material from the online course on general function estimation, mentioned above. We will use video lectures and a freely available book (pdf). You access the online lectures and quizzes via this link 

  • Chapter 1-3 of An Introduction to Statistical Learning, Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Springer. Freely available as pdf: http://www-bcf.usc.edu/~gareth/ISL/.
  • System Identification, by Söderström & Stoica. This book is very good in explaining the basic theory. Most topics we cover in the course can be found here.
  • Alternative literature: System Modeling & System Identification, Rolf Johansson, available at Cremona (STORE).

Schedule in TimeEdit

Teaching:  To a large extent, the course will be given in a flipping-the-classroom style and it is necessary for you to understand what that is so that you can participate and benefit from the teaching.

In short, the idea behind flipping the classroom is that you should get a basic understanding of the material already before the class, such that the time with the teacher is instead focused on deepening your understanding and learning how to analyze and apply what you have learnt. There are several arguments in favor of this approach, for instance that students show up better prepared for the in-class discussions and that they have the support of their peers and a teacher when they perform higher-level tasks (analyze, apply, etc). The main argument, however, is that studies show that it improves learning.

An excellent overview on flipping the classroom can be found here: http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/. They list the key elements of flipping the classroom:

  1. Provide an opportunity for students to gain first exposure prior to class.
  2. Provide an incentive for students to prepare for class.
    You are given a strong incentive since it is mandatory to watch the videos and complete the quizzes before class. Note that the online system (see below) keeps track on which videos you have seen.
  3. Provide a mechanism to access student understanding.
    This is useful for students but also for the teacher since it provides material that enables the teacher to tailor in-class activities to what the students need.
  4. Provide in-class activities that focus on higher level cognitive learning
    In class we devote our time to active learning, where students can achieve deeper understanding. There are many different types of activities that we can devote our time to, and we will try to design activities that fit the students and the topic as good as possible. 

It will only be the first part of the course which strictly follows the flipped classroom style. After the first two weeks, there will be a mixture of overview lectures and online lectures.

Examination: Three smaller projects. Written exam at the end of the course. Two hand-in problems. Exam date. Since the exam is over Zoom this year, the procedure is different than normally. Be well-prepared and read the instructions at Chalmers Corona info to students.  The exam will consist of problems of similar type as the exams available on the course web. Additional to the written exam, there will also be a  oral exam of 15-30 minutes. Schedule for these exams will be prepared  after the written exam.

  • Make sure that each paper is clearly marked with your name, exam problem number and page number.
  • Scan or photograph your solutions. Make sure to have a good lightning and preferably use a document scanning app, e.g. CamScanner or Genius Scan.
  • Combine your exam into one pdf document
  • Submit your solutions by uploading the image files or documents via Canvas within 10 minutes after the end of the exam. Note, this is a hard limit.

Link to the exam, A Zoom meeting where you sign in, and a link in Canvas to the exam.

Ph. D students can choose when they take the exam, earliest 2nd of June 14:00, latest 5th of June. Examination time is 4 hours. Contact the examiner in advance about when you want to take the exam.

Lectures: using Zoom, the app seems to work better than using a web browser

https://chalmers.zoom.us/j/64662388652

Or Skype for Business (Lync):
https://chalmers.zoom.us/skype/64662388652

Or iPhone one-tap :
Sweden: +46850520017,,64662388652# or +46850539728,,64662388652#

Or Skype for Business (Lync):
https://chalmers.zoom.us/skype/64662388652

Student representative: Oskar Leander, Klas Svensson Qvistberg, Tommy Sy. Description of responsability

Time: Mainly Tuesdays and Fridays 9:00-11:45, with some exceptions. Red text has not yet been updated.

Meeting

   # 

 Subject

 Preparation, prior to lecture

 Literature

Deadline

24/3

online film from lecture 1

ES53

J

 1

  •  Introduction and course overview, what is system identification, and what are the challenges?
  • Discussion and work on project 1


Statistical learning, Lecture 1, look at the first lecture "
Opening remarks" and answer quiz.

slides

project 1

matlab files can be found in "project"

 

27/3

ES53

JA

 2

  • Statistical learning, estimate an unknown function, input vector, or equivalently, regressor, output vector, prediction and inference, supervized versus unsupervised learning, quality of fit, model quality, bias-variance trade-off
  • Exercises: Ch. 2: 1, 2, 3, 5, 6
  • Discussion and work on project 1

Statistical learning, Lecture 1 & 2 + Quiz

slides: lecture 1, lecture 2

The parts on classification are optional.

Chapter 1, Chapter 2, you can skip  sections 2.2.3 and 2.3  

31/3

ES53

JA

 3

 
  • Exercises: left from last time + uncertainty estimation
  • Discussion and work on project 1

 

   

3/4

ES53

JA

online film from lecture

4

Linear regression

  • multi-linear regression
  • linear regression with nonlinear regressors
  • parameter uncertainty
  • is a regressor of importance?

 

  • Exercises: Ch. 3: 1, 2, 3, 4, 5, 6
  • Discussion and work on project 1
  • Overview of next lecture topic,  slides
 

Statistical learning, Lecture 3 + Quiz

slides lecture 3, (up to slide 20)

The parts on classification are optional.

 
Chapter 3, you can skip Sect. 3.3.1, and 3.6   

15/4

EL53

Wednesday

8-10

JA

online film from lecture

 5

  •  Estimating spectra and Transfere Functions in frequency domain
  • Exercises: 3, 4, on your own: 5, 6
  • Introduction to project 2
  • Overview of next lecture topic, slides

 Read chapter, look at slides & problems

Online lectures (note, they go much deeper into the topic, than we do): lect2, lect3, lect4, lect5

Project 2

S&S:Chapter 3

project 1: 14/4

17/4

EL53

 

JA

online film from lecture

 6

Linear Regression for Dynamic Models

  • Exercises: 7, 8, on your own: 9, 10, 11
  • Discussion and work on project 2, feedback on project 1
  • Overview of next lecture topic,  slides

 Read chapter, look at slides & problems

online lect:

S&S:Chapter 4  

21/4

 

ES53

JA

online film from lecture

 7

Model Parameterization, Linear black-box models

  • Exercises: 12, 13, on your own: 14, 15
  • Discussion and work on project 2
  • Overview of next lecture topic,  slides

Read chapter, look at slides & problems

online lect:  lect

S&S:Chapter 6, 12.4    

24/4

ES53

JA

 8

Predicting the parameters: Prediction Error Methods, Maximum Likelihood

  • Exercises: 18, on your own: 16
  • Discussion and work on project 2

Read chapter, look at slides & problems

online lect: lect, (skip parts on instrumental variables)

S&S Chapter 7  

28/4

ES53

JA

online film from lecture

 9

  • Exercises: Catch up with problems
  • Discussion and work on project 2
  • Overview of next lecture topic,  slides

Suggest problems to Angelos in advance, if  U want.

 

29/4

Wednesday

8-10

online film from lecture

9.5

Model Validation

  • Exercises: 21, 22, on your own: 20
  • Discussion and work on project 2
  • Overview of next lecture topic,  slides

Read chapter, look at slides & problems

S&S Chapter 11, 12.10  

5/5

ES53

JA

online film from lecture

 10

Experiment design & Data quality, Closed loop Identification

  • Exercises: 26, 28, (25 if time) on your own: 23, 24, 28, 29
  • Introduction to project 3, Matlab demo of System Identification Toolbox

Read chapter, look at slides & problems

online lecture on closed loop identification: lect

S&S Chapter 5, 10, 12.2

Project 3

project 2: 4/5

8/5

ES53

JA

online film from lecture

 11

 

  • Exercises: Catch up with problems
  • Discussion and work on project 3, feedback on project 2
  • Overview of next lecture topic,  slides

Suggest problems to Angelos in advance.

   

12/5

ES53

JA

online film from lecture

 12

State-Space Identification, Instrumental Variable Methods, Subspace Identification

  • Exercises: 31, 32, 33, on your own: the rest
  • Discussion and work on project 3
  • Overview of next lecture topic, slides

Online lecture on subspace identification: lect

Only instrumental variables: lect

Read chapter, look at slides & problems

Chapter, article

S&S Chapter 8  

 

15/5

ES53

JA

online film from lecture

 13

Nonlinear System Identification

  • Exercises: 34, 37 if time 35, on your own: the rest
  • Discussion and work on project 3
  • Overview of next lecture topic,  slides
 Read chapter, look at slides & problems  
Hand-in problems

19/5

ES53

JA

online film from lecture

 14

Recursive System Identification

  • Exercises: 38, 39
  • Discussion and work on project 3
  • Overview of next lecture topic, slides
 
Read chapter, look at slides & problems S&S Chapter 9  

26/5

ES53

JA

15

Adaptive Control

  • Exercises: 40, 42
  • Discussion and work on project 3, show presentation to Jonas or Angelos
 
Read  book chapter, look at slides & problems Introduction chapter on Adaptive Control
Go through your presentation either with Angelos or Jonas
 

29/5

ES53

JA

16
  • Presentations of project 3
 

Look at presentations from the other groups. Give feedback in the quiz.

 

project 3: 29/5 

 

 

Course summary:

Date Details Due