Mathematics of Machine Learning

Course manual 2025/2026

Course content

Machine learning is one of the fastest growing areas of science, with far-reaching applications. This course gives an overview of the main techniques and algorithms. The lectures introduce the definitions and main characteristics of machine learning algorithms from a coherent mathematical perspective. In the exercise classes students will experiment with the algorithms in Python Jupyter notebooks, and apply them to a selection of data sets.

This course is intended to bridge the gap between mathematical theory and practical applications of machine learning. As such, there is a strong emphasis on the intuition and motivation behind algorithms. Students will learn to explain the practical behavior of algorithms in terms of their theoretical properties. Attention will also be given to computational efficiency of algorithms, which is crucial when scaling up to modern big data sets.

Taking this course will be helpful if you wish to take the Machine Learning Theory course in the master program.

Contents of this course:

• From supervised learning, we will cover both classical and state-of-the-art methods for classification and regression, with an emphasis on classification, including: linear discriminant analysis, the nearest-neighbor method, naive Bayes, decision trees and random forests, logistic regression, boosting, support vector machines, neural networks/deep learning, least-squares regression.

• From unsupervised learning, we will cover K-means clustering.

• We will cover several methods for model selection: cross-validation, ridge regression and the Lasso.

• From optimization, we will cover stochastic gradient descent and its application to deep learning using the backpropagation algorithm.

Study materials

Literature

• • The Elements of Statistical Learning (ESL), 2nd edition, by Hastie, Tibshirani and Friedman, Springer-Verlag 2009.

• • Selected parts from Ch.18 of Computer Age Statistical Inference: Algorithms, Evidence and Data Science (CASI) by Efron and Hastie, Cambridge University Press, 2016.

• A small number of handouts, which will be made available via the course website.

Objectives

• The student should be able to formalise a given machine learning task in the framework of statistical decision theory.
• The student should be able to reproduce definitions of the machine learning algorithms presented in the course.
• The student should be able reason about the behavior and mathematical properties of machine learning algorithms from the course.
• The student should be able to reflect on the suitability of machine learning algorithms from the course for a given machine learning task.
• The student should be able to understand and modify algorithms from the course when they are applied to given instances of machine learning tasks.
• The student should be able to connect the theoretical properties of machine learning algorithms from the course and their practical behavior observed on a given machine learning task, and reflect on whether the practical behavior is as expected.
• The student should be able to collaborate with other students to conduct small scale experiments with machine learning algorithms and report on the outcome.

Teaching methods

• Lecture
• Laptop seminar

Learning activities

Attendance

Attendance requirements for the program (OER - Part B):

• Active participation is expected from every student in the course component for which the student is enrolled.
• In addition to the general requirement that the student actively participates in the education, the additional requirements per component are described in the study guide. It also specifies which parts of the component have a mandatory attendance requirement.
• If a student is unable to attend a mandatory part of the program due to personal circumstances, the student must report this in writing as soon as possible to the relevant lecturer and the study advisor.
• It is not permitted to miss mandatory parts of a component if there are no personal circumstances.
• In cases of qualitatively or quantitatively insufficient participation, the examiner may exclude the student from further participation in the component or part of it. Conditions for sufficient participation are determined in advance in the study guide and on Canvas.

Additional requirements for this course:

Conditions for sufficient participation are: submitting serious attempts for at least 60% of the homework exercises.

Assessment

The midterm will be about the first half of the course.
The final exam will only be about the second half of the course.
The resit exam will cover both halves; it will replace both the midterm and the final exam.

Fraud and plagiarism

The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl

Course structure

Contact information

Coordinator

• Tim van Erven