Course syllabus
Course-PM: Welcome to DAT550 / DIT978 Advanced Software Engineering for AI/ML-Enabled Systems LP4 VT24 (7.5 hp)
This is a literature seminar course. You will study scientific literature on a current topic related to Software Engineering for AI/ML enabled systems, write a literature survey, and present the chosen topic in front of the class.
Artificial intelligence and machine learning are more and more used in practice. However, the introduction of AI/ML components into a software system comes with new challenges and needs and changes the way the software system is engineered. This course introduces processes, practices and techniques for engineering AI/ML-enables software systems.
Contact details
Examiner: Eric Knauss (mail)
Lecturers: Hans-Martin Heyn (mail), Eric Knauss (mail)
Please contact both of us in case of questions or comments.
Organisation
This course is offered to students from the University of Gothenburg and from Chalmers. The course homepage is located in Canvas.
The course is a literature seminar, which combines reading papers, student presentations, and discussions. Students will explore one of the topics about Software Engineering for AI/ML-enabled systems in detail and gain more summative knowledge on the other topics. An individual report is the final element of the course.
Topics are assigned to student teams that typically consist of two students. Students can submit topic preferences and argue for their preferences by referring to their previous experience with certain fields in software engineering or AI/ML. Topics will be assigned based on these preferences, but also with regard to ensuring that all themes are covered.
Components
The course consists of the following components.
- Presentation (team of 2 students): Each team reads papers and presents a topic that belongs to one of the course’s themes.
- Pedagogic example, exercise, or tutorial (team of 2 students): To enrich the presentation each team provides a pedagogic example, exercise, or tutorial that enables participants to gain a deeper understanding of the presented knowledge and to train their judgment in the area.
- Teach-me video (optional, team of 2 students): In addition, teams may produce a short “teach-me” video that illustrates one specific takeaway from their studies in an engaging way. The video will help your follow students to study the content of your seminar. The video is optional to earn extra points towards the final grade.
- Discussion Leader (individual): Each student team acts as a discussion leader for another presentation. There are multiple discussion leaders per presentation.
- Attendance (individual): Attendance at the presentations of other students is mandatory.
- Individual written final report (individual): Each student submits an individual report containing two parts: Part A: A literature survey discussing the state-of-the-art of the chosen topic. Part B: A reflection on two other student presentations.
Support for seminar preparations
Every week after the lectures (Tuesday 10-11, Friday 12.30 - 13.30) we will offer guidance sessions for the teams presenting in the current or following week.
If you need support, please contact Hans-Martin Heyn (hans-martin.heyn@gu.se) or Eric Knauss (Eric.Knauss@cse.gu.se). You can also try to "drop-in" after the lecture, but then we cannot guarantee that not another group has already reserved some time for guidance.
Course literature
Initial course literature for each topic can be found in the respective Modules.
Schedule
The room schedule can be found on TimeEdit.
The student presentation schedule is here.
| Week Number | Tuesday (08:15 - 10:00) | Friday (08:15 - 11:45) | Task Due Dates |
| 12 (19.03 / 22.03) |
Introduction, General Overview of Themes => Lecture_1 - Introduction - 2024.pdf => Lecture_2 - How to read a paper_2024.pdf
|
Example of system architecture for ML enabled systems => Lecture_3 - Architecting ML Systems - 2024.pdf
|
Task A: Topic Selection |
| 13 (26.03) |
Guest Lecture Volvo Trucks In the core-ML team at the Volvo Group, we train deep learning models that target a variety of problems. Sure, all of them flow through the MLOps pipeline of dataset assembly, training, validation, deployment, etc., meaning that the pipeline itself should be well constructed and reusable. But the sheer range of applications that we work with forces us to consider other kinds of reuse. How about network architectures, training loops or, more importantly, training objectives themselves? Can they not be reused across problems? I’ll outline our efforts to promote broad spectrum reuse of deep learning code to solve various problems in automotive (software) engineering. I’ll also introduce topics that you could explore as part of an internship or thesis. |
Self-Studies: Preparation of Presentations / Eastern | |
| 14 | Self-Studies: Preparation of Presentations / Eastern | ||
| 15 (09.04 / 12.04) |
Lecture starts at 09.00
|
Lecture starts at 08.30
|
|
| 16 (16.04 / 19.04) |
Start: 08:30 |
Start: 08:30 Lecture_Technical Debt, Code Smells, Patterns, and Anti-Patterns.pdf Menti_Technical Debt, Code Smells, Patterns, and Anti-Patterns.pdf Video Technical Dept, Code Smells, and Anti-Patterns
|
|
| 17 (23.04 / 26.04) |
Start: 08:30 |
Start: 08:30 Lecture_Safe-guarding strategies and security.pdf Menti_Safeguarding strategies.pdf Video: Safeguarding Strategies and Security Informed Machine Learning - Integrating Prior Knowledge into Learning Systems https://miro.com/app/board/uXjVKPbpjoE=/?share_link_id=430511245514 |
|
| 18 (03.05) | Self-Studies: Walpurgis Night |
Start: 08:30 Lecture_Software Architecture for ML.pdf Menti_Software Architecture for ML.pdf
Lecture_DataRequirement_Maleekanya.pdf Mid-course student representative meeting |
|
| 19 (07.05) | Self-Studies: Video Preparation |
Self-Studies: Ascension Day | Task D: Teach-me video Su. 12.05 23:59 |
| 20 (Wednesday, 15.05 / 17.05) |
OBS! Wednesday 15.05 Start: 08:30 Menti_Ethics Gen AI Menti.pdf |
Start: 08:30 The Seven Deadly Sins of AI Prediction Two GPT-4os interacting MLOps |
|
| 21 (21.05 / 24.05) |
OBS: Meeting at 08:30 in the lobby of the Ericsson building Start: 08:30 Lecture_Generative_AI_for_Software_Engineering Menti_Generative_AI_for_Software_Engineering
|
Start: 08:15 (!!) Menti_llm4se_menti.pdf |
|
| 22 Exam Week |
Exam Week, no lectures |
Task F: Individual Report |
|
Learning Outcomes
After completing the course, the student should be able to:
Knowledge and understanding
- Explain processes and engineering practices for developing AI/ML-enabled systems, from requirements engineering to testing
- Explain typical roles in software engineering of AI/ML enabled systems as well as challenges in interdisciplinary teams consisting of Data Scientists and Software Engineers
- Explain typical requirements for AI/ML components, such as non-functional requirements, requirements on data, and contextual requirements
- Explain architectures and patterns for AI/ML-enabled systems
- Describe existing techniques to verify and explain decisions made by AI/ML-enabled systems
- Give an overview of recent research on SE for AI/ML-enabled systems
Skills and abilities
- Read a research paper on software engineering for AI/ML-enabled systems, present its content, and critically discuss the presented research design
- Demonstrate a software engineering approach for AI/ML-enabled systems with appropriate examples
- Assess new engineering knowledge for AI/ML-enabled systems and relate it to the knowledge presented in this course
Judgement and approach
- Judge the extent to which an AI/ML component needs to be safe-guarded
- Judge what verification methods are appropriate when developing an AI/ML-enabled system given the requirements o that system
- Judge whether a model has systematic biases and discuss the consequences of these biases
- Judge fairness and potential other ethical issues of an AI/ML-enabled system
- Judge user‘s information needs to work with an AI-enabled system
- Judge limitations of a state-of-the-art software engineering approach for AI/ML given evidence presented in research paper
Changes made since the last occasion
- Refinement of topics.
- Made video recording optional.
- Added clearer assessment criteria for each graded course moment.
Examination
- Topic Selection and Assignment: Students should form teams and submit their topic preferences by the deadline shown in the table below.
- Presentation and Pedagogic example, exercise, or tutorial: The presentation will happen on one of the following days between 8:00 and 12:00. The order of presentations will be determined by topic. The Pedagogic example, exercise, or tutorial is to be delivered as part of the presentation. Presentation slides and materials from the pedagogic example, exercise, or tutorial must be submitted one week after the presentation via Canvas.
- Discussion Leader: Leading a discussion will happen directly after the corresponding presentation.
- Teach-me video: should be submitted on Monday of the week after the presentation of the team. Exceptions are the videos for presentations in the week of May 22 – 26. Videos for these presentations should be submitted on Friday May 19.
- Attendance: Attendance to all student presentations (see dates above) is mandatory. A student may miss a maximum of two dates.
- Individual written final report: The final report is due on Friday. June 2.
| Task | Name | Description | Team | Individual | Due Date |
| A | Topic Selection | Topic Selection and Assignment: Students should form teams and submit their topic preferences | X | Su. 31. March 2024 23:59 | |
| B1 | Presentation and Pedagogic example, exercise, or tutorial | The presentation will happen on one of the following days. The order of presentations will be determined by topic after the course started. Presentation days are stated in the schedule. The Pedagogic example, exercise, or tutorial is to be delivered as part of the presentation. | X | Individual | |
| B2 | Submission of material | Presentation slides and materials from the pedagogic example, exercise, or tutorial must be submitted one week after the presentation via Canvas. | X | Individual | |
| C | Leading a discussion | Leading a discussion will happen directly after the corresponding presentation. | X | Individual | |
| D | Teach-me video | The video will help your follow students preparing the final report. |
X | Su. 12. May 2024, 23:59 | |
| E | Attendance |
Attendance to all student presentations (see dates above) is mandatory. A student may miss two of the dates. |
X | All lectures | |
| F | Report | A final report | X | Su. 02. June 2024, 23:59 |
The use of large language models in this course
For guidelines on how to use text generating large language models (LLMs) such as ChatGPT responsibly please see https://chalmers.instructure.com/courses/4215/pages/recommendations-on-use-of-generative-ai-in-software-engineering-courses?module_item_id=373207. If you use LLMs for creating your presentation or report, please add a disclaimer explaining how and for what aspects you used LLMs.
Grading
Grading will be based on individual achievements in the different project activities.
|
Lecture and Pedagogical Element |
Report |
Teach-Me Video |
|
|
3 |
The student gives a presentation that represents the theme correctly and to a reasonable degree. The presentation includes a correct pedagogical example, exercise, or tutorial. The student acts as a discussion leader. |
The student submits a sound individual written report that addresses the main aspects asked for in the report. |
% |
|
4 |
The student gives a presentation that represents the theme correctly and to a large degree, shown for example through the inclusion of releveant additional literature beyond the starting literature. The presentation includes a correct and illustrative pedagogical example, exercise, or tutorial. The student acts as a discussion leader, asking questions that show a general understanding of the presented topic. |
The student submits a sound individual written report that addresses the main aspects asked for in the report and shows the ability to critically reflect on the discussed topic. |
The student submits a teach-me video that is correct in presented content and of sufficient quality so that it can be shown to the class. |
|
5 |
The student gives a presentation that represents the theme correctly, to a large degree, and shows a depth of knowledge on the topic, shown for example through the inclusion of a significant number of releveant additional literature beyond the starting literature. The presentation includes a correct and illustrative pedagogical example, exercise, or tutorial that encourages participants to reflect critically on the topic and its limitations. The student acts as a discussion leader, asking questions that show a deep understanding of the presented topic and critical examination of the presented information. |
The student submits a sound individual written report that addresses all aspects asked for in the report and shows the ability to critically reflect on the discussed topic. |
The student submits a teach-me video that is correct in presented content, of sufficient quality so that it can be shown to the class and illustrates the information in an engaging way. |
Assessment Criteria
The assessment criteria for each graded element are summarised here: https://chalmers.instructure.com/courses/29092/pages/assessment-criteria
Grading Scheme
- Element 1 and Element 2 are Lecture and Report or vice versa.
- You must pass both elements to pass the course.
- Without the video, you cannot reach a 5.
|
Final Grade |
Element 1 |
Element 2 |
Video |
|
U |
U |
U |
%, 4 or 5 |
|
U |
U |
3 |
%, 4 or 5 |
|
3 |
3 |
3 |
%, 4 or 5 |
|
3 |
3 |
4 |
% |
|
4 |
3 |
4 |
4 or 5 |
|
4 |
4 |
4 |
%, 4 or 5 |
|
4 |
5 |
5 |
% |
|
4 |
5 |
3 |
%, 4 or 5 |
|
4 |
5 |
4 |
% |
|
5 |
4 |
5 |
4 or 5 |
|
5 |
5 |
5 |
4 or 5 |
Missed Deadlines and Revisions
Handling of missed deadlines and revisions depends on the grading component.
- Presentation: If a student fails to show up for their own presentation, a substitute literature review paper can be submitted. The deadline for the resubmission is August 21st, 2024.
- Pedagogic example, exercise, or tutorial: If a student fails to present a pedagogic element, in-depth example, or tutorial, as part of their presentation, a substitute presentation can be scheduled to "make up" for the missing aspects of their presentation.
- Teach-me video: The course can be passed without submission of a teach-me video. A re-submission is not possible.
- Discussion Leader: If a student fails to show up and act as a discussion leader for the assigned presentation, the student has one chance to compensate for that by submitting a report similiar to the report required for missed attendance (see below). The deadline for this substitution task is August 21st, 2024.
- Individual written final report: The individual report can be resubmitted on three dates: August 21st, 2024, October 23rd, 2024, and January 15th, 2025.
- Attendance: If a student misses more than 2 presentation days, the student has the chance to compensate for that by submitting a report including 2 page per lacking presentation day (i.e., if a student attended 8-X presentation days, they should pick X of the non-attended presentations). The report should discuss open questions and give constructive feedback on the presentation slides and pedagogic example, exercise, or tutorial, of the non-attended presentation. The deadline for this substitution task is August 21st, 2024.
Course summary:
| Date | Details | Due |
|---|---|---|