The mathematics of machine learning

Owner	Bachelor Wiskunde
Coordinator	Tim van Erven
Part of	Exchange Programme Faculty of Science, specialisation BSc Mathematics, year 1Bachelor Wiskunde, year 3Dubbele bachelor Wis- en Natuurkunde, year 3Dubbele bachelor Wiskunde en Informatica, year 3

Course manual 2021/2022

Course content

Machine learning is one of the fastest growing areas of science, with far-reaching applications. This course gives an overview of the main techniques and algorithms. The lectures introduce the definitions and main characteristics of machine learning algorithms from a coherent mathematical perspective. In the workgroups the algorithms will be implemented in Python and applied to a selection of data sets.

This course is intended to bridge the gap between mathematical theory and practical applications of machine learning. As such, there is a strong emphasis on the intuition and motivation behind algorithms. Students will learn to explain the practical behavior of algorithms in terms of their theoretical properties. Attention will also be given to computational efficiency of algorithms, which is crucial when scaling up to modern big data sets.

Taking this course will be helpful if you wish to take the Machine Learning Theory course in the master program.

Contents of this course:

• From supervised learning, we will cover both classical and state-of-the-art methods for classification and regression, with an emphasis on classification, including: linear discriminant analysis, the nearest-neighbor method, naive Bayes, decision trees and random forests, logistic regression, boosting, support vector machines, neural networks/deep learning, least-squares regression.

• From unsupervised learning, we will cover K-means clustering and its relation to the Expectation-Maximization algorithm.

• We will cover several methods for model selection: cross-validation, ridge regression and the Lasso.

• From optimization, we will cover stochastic gradient descent and its application to deep learning using the backpropagation algorithm.

Study materials

Literature

• • The Elements of Statistical Learning (ESL), 2nd edition, by Hastie, Tibshirani and Friedman, Springer-Verlag 2009.

• • Selected parts from Ch.18 of Computer Age Statistical Inference: Algorithms, Evidence and Data Science (CASI) by Efron and Hastie, Cambridge University Press, 2016.

• A small number of handouts, which will be made available via the course website.

Objectives

• The student should be able to formalise a given machine learning task in the framework of statistical decision theory.
• The student should be able to reproduce definitions of the machine learning algorithms presented in the course.
• The student should be able reason about the behavior and mathematical properties of machine learning algorithms from the course.
• The student should be able to reflect on the suitability of machine learning algorithms from the course for a given machine learning task.
• The student should be able to apply algorithms from the course to given instances of machine learning tasks.
• The student should be able to connect the theoretical properties of machine learning algorithms from the course and their practical behavior observed on a given machine learning task, and reflect on whether the practical behavior is as expected.
• The student should be able to write a clear scientific report in LaTeX that discusses the application of a machine learning algorithm to a given machine learning task.
• The student should be able to collaborate with other students to produce a scientific report.

Teaching methods

• Lecture
• Laptop seminar

Learning activities

Attendance

Programme's requirements concerning attendance (OER-B):

• Each student is expected to actively participate in the course for which he/she is registered.
• If a student can not be present due to personal circumstances with a compulsory part of the programme, he / she must report this as quickly as possible in writing to the relevant lecturer and study advisor.
• It is not allowed to miss obligatory parts of the programme's component if there is no case of circumstances beyond one's control.
• In case of participating qualitatively or quantitatively insufficiently, the examiner can expel a student from further participation in the programme's component or a part of that component. Conditions for sufficient participation are stated in advance in the course manual and on Canvas.
• In the first and second year, a student should be present in at least 80% of the seminars and tutor groups. Moreover, participation to midterm tests and obligatory homework is required. If the student does not comply with these obligations, the student is expelled from the resit of this course. In case of personal circumstances, as described in OER-A Article A-6.4, an other arrangement will be proposed in consultation with the study advisor.

Additional requirements for this course:

Conditions for sufficient participation are: submitting serious attempts for at least 60% of the homework exercises.

Assessment

Item and weight	Details
Final grade
40% Final exam
30% Midterm
30% Homework

The midterm will be about the first half of the course.
The final exam will only be about the second half of the course.
The resit exam will cover both halves; it will replace both the midterm and the final exam.

Fraud and plagiarism

The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl

Course structure

Timetable

The schedule for this course is published on DataNose.

Processed course evaluations

Below you will find the adjustments in the course design in response to the course evaluations.

Contact information

Coordinator

• Tim van Erven