Course syllabus

Jump to today

Course-PM

TIF150 / FIM780 Information theory for complex systems lp3 VT23 (7.5 hp)

Course is offered by the department of Space, Earth and Environment

 

Contact details

  • Claes Andersson (Examiner). E-mail: claeand@chalmers.se
  • Ankit Vikrant (Lectures, example classes). E-mail: ankitv@chalmers.se
  • Sofia Cvetkovic Destouni (homework, projects). E-mail: destouni@chalmers.se

 

Course overview

The course provides an understanding of fundamental concepts used to describe complex systems, in particular dynamical systems such as chaotic low-dimensional systems, self-organizing systems, and simple spatially extended systems such as cellular automata. Many of the concepts are based in information theory.

  • Basic concepts of information theory: Shannon entropy, complexity measures.
  • Information theory and statistical mechanics.
  • Geometric information theory -- randomness and complexity in spatially extended systems.
  • Information flow. The relation between microscopic and macroscopic levels.
  • Statistical models, in particular hidden Markov models.
  • Cellular automata.
  • Applications in nonlinear dynamics, computational biology, chemical self-organizing systems, and statistical mechanics.

 

Schedule

TimeEdit

Further details are given at TIF150 Lecture plan 2023.pdf

 

Course literature

The lectures will follow the presentation in:

K. Lindgren InformationTheoryOfComplexSystems.pdf 

An information perspective on complexity in dynamical systems, physics and chemistry. (Chalmers, 2014.)

 

If you want to learn more:

T. M. Cover and J. A. Thomas, Elements of information theory (Wiley, 1991).

See also: David MacKay,  Information theory, Inference and Learning (2003).

 

Problem sessions and solutions

Preliminary list of problems to be solved

  • 20 January: 2.2, 2.4, 2.6, 2.8, 2.9, 2.16
  • 27 January: 3.2, 3.3, 3.5, 3.7, 3.8 (possibly leaving one or two for the next session)
  • 9 and 10 February: 4.1, 4.2, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9
  • 21 February: 5.1, 5.2, 5.3, 5.4, 5.6, 5.8
  • 2 March: 8.1, 8.2, 8.7
  • 10 March: solving an old exam (which one to be determined)

 

Recommended textbook problems for exam

  • 2.1, 2.2, 2.16
  • 3.3, 3.7, 3.8
  • 4.2, 4.4, 4.5, 4.6, 4.9
  • 5.1, 5.2, 5.3, 5.4, 5.6, 5.8
  • 8.1, 8.2, 8.3, 8.4, 8.5, 8.7

 

Homework

Five optional homework problems are given. Some of these problems involve programming tasks. Each one gives up to two (2) extra points for the exam. Late submissions will normally not be graded. All solutions must be submitted as single PDF files in Canvas. Hand-written solutions are fine for non-programming problems, but please take care to make them legible.

The deadlines are:

  • Homework 1: Thursday 26 January, 23:59
  • Homework 2: Thursday 9 February, 23:59
  • Homework 3: Monday 20 February, 23:59
  • Homework 4: Wednesday 1 March, 23:59
  • Homework 5: Thursday 9 March, 23:59

 

Projects

Optional project work can be done in groups of 1-3 students. The project work is awarded up to 10 extra points for the exam. Further instructions are given in this file: TIF150_Project_2023.pdf  

 

Exam, grading

The exam is given on March 13, 08:30-12:30. The exact details will be shared with the students well in time.

The course is graded based on the exam score including extra points from homework (max 10 points) and projects (max 10 points). The exam gives up to 50p. Grade limits (Chalmers/ECTS): 25p for 3/E, 28p for 3/D, 34p for 4/C, 38p for 4/B, 42p for 5/A. To pass, a minimum of 20p on the written exam is required, regardless of additional points.

 

Learning objectives and syllabus

After successfully completing this course students will be able to:

  • Define and use the basic concepts of information theory: Shannon entropy, relative entropy, complexity measures based on these
  • Use information theory to characterise both cellular automata and low-dimensional chaos
  • Understand the connection between information theory and statistical mechanics
  • Use geometric information theory to characterise patterns in spatially extended systems like pictures
  • Explain how information is flowing in chemical self-organising systems exhibiting pattern formation

Link to the syllabus on Studieportalen (Chalmers).

Link to syllabus Gothenburg University.

 

Examination form

The examination will be based on a final written exam, with the possibility of extra points from homework assignments and the optional mini project.

 

 

Course summary:

Date Details Due