Course syllabus

DAT450/DIT247 Machine Learning for Natural Language Processing, 2022, period 2.

For questions about the course, please get in touch with Richard (richajo@chalmers.se).

Department of Computer Science and Engineering (CSE), Gothenburg University | Chalmers

Course content

The course gives an introduction to machine learning models and architectures used in modern natural language processing (NLP) systems.

Rapid developments in machine learning have revolutionized the field of NLP, including for commerically important applications such as translation, summarization, and information extraction. However, natural language data exhibit a number of peculiarities that make them more challenging to work with than many other types of data commonly encountered in machine learning: natural language is discrete, structured, and highly ambiguous. It is extremely diverse: not only are there thousands of languages in the world, but in each language there is substantial variation in style andgenre. Furthermore, many of the phenomena encountered in language follow long-tail statistical distributions, which makes the production of training data more costly. For these reasons, machine learning architectures for NLP applications tend to be quitedifferent from those used in other fields.

The course covers the following broad areas:

• Working practically with text data, including fundamental tasks such as tokenization and word counting;
• probabilistic models for text, such as topic models;
• overview of the most common types of NLP applications;
• architectures for representation in NLP models, including word embeddings, convolutional and recurrent neural network, and attention models;
• machine learning models for common types of NLP problems, mainly categorization, sequence labeling, structured prediction and generation;
• approaches to transfer learning in NLP.

Learning outcomes

On successful completion of the course the student will be able to:

Knowledge and understanding

• describe the fundamentals of storing textual data for the world's languages,
• describe the most common types of natural language processing tasks,
• describe the most common types of machine learning models used in modern natural language processing,
• explain how text data can be annotated for a natural language processing task where machine learning techniques are used.

Skills and abilities

• apply software libraries using machine learning for common natural processing tasks,
• write the code to implement some machine learning models for natural language processing,
• apply evaluation methods to assess the quality of natural language processing systems.

Judgement and approach

• discuss the advantages and limitations of different machine learning models with respect to a given task in natural language processing,
• reason about what type of data could be useful when training a model for a given natural language processing task,
• select the appropriate evaluation methodology for a natural language processing system and motivate this choice
• reason about ethical questions pertaining to machine learning based natural language processing systems, such as stereotypes and under-representation.

Assessment and grades

The grading uses a numerical scale (Fail or 3–5).

The course is examined by four mandatory programming assignments, as well as a self-defined project that requires the submission of a written report and an oral presentation. The assignments include some parts that are solved individually, and some parts that are solved in a group. The project is conducted in a group of 2-4 students.

Each submitted assignment, including the project, will be awarded a score, where 0 corresponds to a Fail (U) grade. For a submission be awarded minimal score (1 point), the code and report need to be clearly written, and contain only minor errors. To get a higher score (up to 10 points for each assignment and 30 points for the project), the solutions should be correct, well explained, and more insightful; in addition, some assignments may require the student to solve additional tasks to get a higher score.

If you submit a solution before the deadline that does not meet the requirements for a Pass grade, you will get some feedback and be asked to correct the most important errors within a stipulated period of time. Resubmitted solutions will receive a somewhat lower score than a submission in the first round. If you miss the deadline (or the resubmission deadline), or if the examiner believes that the solution was not an honest attempt to solve the assignment, the solution will get the grade of Fail (U). If you have received the U grade for some submitted assignment, you can submit a new attempt during next year's course.

To get the grade Pass (3) for the assignment part of the course, all assignments and the final project must be passed. The final grade will be based on the total sum of scores for all the assignments and project, using the following thresholds:

54-67 (at least 80% of the maximal): 5

40-53.9 (at least 60% of the maximal): 4

5-39.9: 3

Working in groups

Most assignments in the course are group assignments, and for these assignments it is compulsory to work in a group. If you have a very strong reason for working alone (e.g. related to your health or some other special circumstances), you can ask the examiner for an exemption. In this case, you need to include a statement in your submission that you got an exemption from the requirement to work in a group. Individual submissions that do not include this statement will not be graded.

General Rules and Policies

• The deadlines for submitting assignments are firm. Delays must be motivated before the deadlines. Unannounced late submissions will not be considered.
• It is allowed, even encouraged, to discuss the assignments during the course. Also, do not hesitate to ask if you have difficulties with the assignments, or if something is unclear.
• You must write your final solutions on your own, using your own words, and expressing them in the way you understood them yourself.
• Submitting others' work in your own name is cheating! It can lead to severe consequences, in very bad cases even suspension from studies.
• Specifically, it is prohibited to copy (with or without modifications) from each other, from books, articles, web pages, etc., and to submit solutions that you got from other persons, unless you explicitly acknowledge the sources and add your own explanations. We will be particularly watchful if exercises appear as (alleged) innocent questions in internet forums.
• You are also responsible for not giving others the opportunity to copy from your work. We will not investigate who copied from whom.