Bayesian Statistics for Machine Learning

Studiewijzer 2024/2025

Globale inhoud

Modern machine learning methods are based on mathematical concepts, especially from probability theory and statistics. This course treats these concepts in detail, through the spectrum of the Bayesian school of thought in machine learning. This will lay the groundwork for a solid understanding of advanced machine learning methods taught in other courses.

Leerdoelen

• Understand the relationship between the frequentist and Bayesian views of probability.
• Be able to apply the sum rule, the product rule, and Bayes' rule to compute marginal and conditional probabilities distributions for discrete variables.
• Understand the mathematical definitions of sample spaces, events, how probabilities of events define joint and conditional probabilities, and of independent and dependent events.
• Be able to apply rules for computing probabilities of events using set operations, and assess whether variables are dependent or independent based on the results.
• Understand the mathematical definitions of cumulative distribution functions, probability density functions, and of random variables.
• Understand the mathematical definition of an expectation, as well as the definitions of the mean, the variance, and the covariance of a random variable.
• Be able to apply properties of expectations to derive properties of random variables, such as their mean and (co)variance.
• Understand the mathematical definitions of a Monte Carlo estimator, independent and identically distributed variables, unbiasedness, and consistency.
• Understand the mathematical statement of the central limit theorem, and understand its implications for the properties of Monte Carlo estimators.
• Be able to analyse how the amount of data affects the mean and variance of an estimator in the context of specific examples.
• Understand how a Bayesian inference can be performed sequentially, by using the posterior on preceding data as the prior for new data.
• Understand the mathematical definition of an exponential family likelihood and its conjugate prior, and the special cases of the Bernoulli likelihood and the Beta prior.
• Be able to perform calculations in models with conjugate priors to compute sufficient statistics, the joint distribution, posterior distribution, the marginal likelihood, and the predictive distribution.
• Be able to analyse how the choice of prior affect and the amount of data influence the shape of a posterior.
• Understand the mathematical definition of a maximum likelihood (ML) estimator.
• Understand the interpretation of an ML estimator as the choice of parameters most likely to yield the previously seen data, and its relationship to Monte Carlo estimators.
• Understand the mathematical definition of a Maximum a Posteriori (MAP) estimator.
• Understand the relationship between ML and MAP estimators, and the bias and variance of these estimators.
• Be able to derive ML and MAP estimates in specific example, and be able to analyse how the choice of prior affects the MAP estimate.
• Understand how the marginal likelihood can be used to perform model selection.
• Be able to compute the marginal likelihood in specific examples and be able to analyse the results of this computation to evaluate the relative merits of spefic models.
• Understand the mathematical definition of a linear regression problem.
• Be able to analyse how overfitting and underfitting can manifest in regression problems.
• Be able to compute ML and MAP estimates in regression problems and be able to interpret the results of the computation to identify overfitting or underfitting.

Onderwijsvormen

• Hoorcollege
• Werkcollege

Verdeling leeractiviteiten

Aanwezigheid

Aanwezigheidseisen opleiding (OER-B):

• Voor practica en werkgroepbijeenkomsten met opdrachten geldt een aanwezigheidsplicht. De invulling van deze aanwezigheidsplicht kan per vak verschillen en staat aangegeven in de studiewijzer. Wanneer studenten niet voldoen aan deze aanwezigheidsplicht kan het onderdeel niet met een voldoende worden afgerond.
  

Aanvullende eisen voor dit vak:

Attendance is not formally required but strongly encouraged. During class hours (hoorcollege), in-class exercises will be completed and discussed, which are graded pass/fail and count for 15% of the overall grade. During recitations (werkcollege), some number of homework problems will be explained each week. Homework assignments count for 35% of the overall grade.

Toetsing

• You must have a minimum of 5.0 in your homework and a minimum of 5.0 in your combined exams to pass the course
• All homeworks count towards the final mark.
• There is a penalty of 25% per day for late hand-ins, with a maximum cut-off of 2 days. This penalty may be waived in case of sickness, but we may need proof from the studieadviseur.
• Exams are closed book and will be written (in-person), assuming no pandemic-related developments.
• In case a student decide to make the Hertentamen, the Hertentamen will count for 50%. It still holds that a student must have a minimum of 5.0 for the Hertentamen and a minimum on 5.0 for the average of the homework.

Inzage toetsing

• Deeltoets marks will be released on Ans
• After the release of marks on Ans, students have 1 week to ask questions regarding their marks

Opdrachten

• In-class assignments are graded pass/fail and individual
• Homework assignments are graded and individual
• Individual homework grades will be released on Canvas, along with written feedback

Fraude en plagiaat

Dit vak hanteert de algemene 'Fraude- en plagiaatregeling' van de UvA. Hier wordt nauwkeurig op gecontroleerd. Bij verdenking van fraude of plagiaat wordt de examencommissie van de opleiding ingeschakeld. Zie de Fraude- en plagiaatregeling van de UvA: http://student.uva.nl

Weekplanning

Aanvullende informatie

This course will be delivered in English.

Contactinformatie

Coördinator

• Jan-Willem van de Meent