Bayesian Statistics (BSc)

6 EC

Semester 2, period 4, 5

5122BAST6Y

Owner Bachelor Wiskunde
Coordinator dr. B.J.K. Kleijn
Part of Exchange Programme Faculty of Science, specialisation BSc Mathematics, year 1Bachelor Wiskunde, year 3Dubbele bachelor Wiskunde en Informatica, year 3

Course manual 2020/2021

Course content

In frequentist statistics we assume that the data is distributed according to some unknown probability distribution. In Bayesian statistics, the data and the parameter are both treated as a random variable. Besides specifying the statistical model, the Bayesian procedure also specifies a prior distribution on the model. The data will be used as an updating mechanism for the prior resulting in the posterior distribution.

In this course we consider the consider the classical problems considering point-estimation, hypothesis testing, confidence sets and decision theory where we will describe the Bayesian and frequentist methods and compare them to each other. Furthermore, we will discuss the choice of the prior distribution, depending on both the statistical model and the intended posterior distribution.

Study materials

Syllabus

Other

Objectives

  • The student knows the definitions of the model distributions, prior distribution, posterior distribution, prior predictive distribution and posterior distribution
  • The student is able to calculate the posterior distribution given a statistical model and prior distribution
  • The student is able to calculate the following Bayesian point estimators: Maximum-a-posterior estimator and Posterior mean
  • The student is able to construct credible sets and HPD-credible sets
  • The student is able to calculate the prior odds, posterior odds and Bayes factor given a statistical model and prior distribution
  • The student knows the definitions of Loss, Risk and Bayes risk and is able to apply them for statistical decision problems 
  • The student is able to compare the Bayesian methods for point-estimation, credible sets, hypothesis testing and statistical decision theory to the corresponding methods in frequentist statistics
  • The student is able to explain the difference between subjective and objective priors
  • The student is able to calculate the Jeffreys prior for a statistical model
  • The student is able to determine whether a collection of probability distributions is a conjugate family for a given model
  • The student knows the definitions of hyperparameters and hyperpriors and is able to construct the prior for the original parameter given a sequence of hyperpriors and determine the corresponding posterior distribution
  • The student is able to calculate the ML-II estimator for a hyperparameter of a prior distribution
  • The student is able to do an analysis of the Bayesian methods from a frequentist point of view
  • The student knows the construction of the Dirichlet process prior
  • The student knows the Bernstein-von Mises theorem and its consequences

Teaching methods

    • Lectures 
    • Exercise classes

    Learning activities

    Activiteit

    Aantal uur
    Lectures

    26

    Exercise classes

    26 

    Mid-term exam

    3

    Final exam

    3

    Zelfstudie

    110

    Attendance

    Programme's requirements concerning attendance (OER-B):

    • Each student is expected to actively participate in the course for which he/she is registered.
    • If a student can not be present due to personal circumstances with a compulsory part of the programme, he / she must report this as quickly as possible in writing to the relevant lecturer and study advisor.
    • It is not allowed to miss obligatory parts of the programme's component if there is no case of circumstances beyond one's control.
    • In case of participating qualitatively or quantitatively insufficiently, the examiner can expel a student from further participation in the programme's component or a part of that component. Conditions for sufficient participation are stated in advance in the course manual and on Canvas.
    • In the first and second year, a student should be present in at least 80% of the seminars and tutor groups. Moreover, participation to midterm tests and obligatory homework is required. If the student does not comply with these obligations, the student is expelled from the resit of this course. In case of personal circumstances, as described in OER-A Article A-6.4, an other arrangement will be proposed in consultation with the study advisor.

    Assessment

    Item and weight Details

    Final grade

    0%

    Mid-term exam

    7 (35%)

    Final exam

    10 (50%)

    Re-sit exam

    3 (15%)

    Mid-term exam

    Result for mid-term exam counts for 30% towards final grade (or, optionally, for 0%, if the mid-term grade is too low).

    Result for final exam counts for 70% towards final grade (or, optionally, for 100%, if the mid-term grade is too low).

    Result for re-sit replaces all other exam results and counts towards final grade for 100%.

    Inspection of assessed work

    Contact the course coordinator to make an appointment for inspection.

    Fraud and plagiarism

    The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl

    Course structure

    Weeknummer Onderwerpen Studiestof Exercises
    1 Frequentist statistics, introduction Bayesian statistics  Section 1.1 -- 1.4 1.1, 1.2, 1.3, 1.4
    2 Prior, posterior & model distributions, Bayes's Rule  Subsec 2.1.1, app B.5  2.2, 2.3, 2.4
    3 Bayes's billiard, Bayesian view of the model, frequentist view of the posterior  Subsec 2.1.2 — 2.1.5 2.1, 2.5, 2.7
    4 Bayesian point estimators  Section 2.2 2.9, 2.10, 2.12
    5 Confidence sets and credible sets  Section 2.3  2.14, 2.15, 2.13
    6 Tests and Bayes factors  Section 2.4  2.16, 2.17
    7 Bayesian and frequentist decision theory  Section 2.5  2.18, 2.19, 2.20
    8 Mid-term exam    
    9 Subjective priors, non-informative priors Section 3.1, 3.2  Discuss mid-term exam
    10 Hierarchical priors, empirical priors Section 3.3, subsec 3.4.1 3.1, 3.2
    11 Empirical priors and estimation bias; conjugate and Dirichlet priors Subsec 3.4.2, section 3.5, 3.6 3.3, 3.4ace
    12 Dirichlet process priors and posteriors Section 6.4, app D.1, D.2 3.6, 3.7
    13 The Bernstein-von Mises theorem Section 4.1, 4.2 4.2, 4.3, 4.5
    14 Final exam     

    Timetable

    The schedule for this course is published on DataNose.

    Honours information

    There is no honours extension to this course.

    Additional information

    Recommended prerequisites: a basic understanding of frequentist statistics (as taught in any introductory statistics course, particularly, the BSc course Stochastics II), a basic understanding of measure theory and point-set topology (as in the BSc course on measure theory and the (first half of) the BSc topology course). Note, however, that measure theory and topology are applied only sparingly and explained throughout.

    Contact information

    Coordinator

    • dr. B.J.K. Kleijn