Advanced Statistics

Course manual 2019/2020

Course content

This course is about the mathematical foundations of the two big schools of statistics: Bayesian and Frequentist inference. We will study applications, differences and limitations of these approaches that are at the foundation of all of modern science.

1. Probability crash course
2. Frequentist inference
   
   1. Hypothesis testing
   2. Estimators
   3. Maximum likelihood ratio test
   4. Confidence regions
3. Bayesian inference
   1. Bayes theorem
   2. Credible intervals & priors
   3. Sampling techniques

Study materials

Literature

• G. Cowan, 'Statistical Data Analysis'. (optional)

• R.J. Barlow, 'Statistics, A guide to the Use of Statistical Methods in the Physical Sciences'. (optional)

• P. Gregory, 'Bayesian Logical Data Analysis for the Physical Sciences'. (optional)

Other

• Lecture slides, course notes.

Objectives

• Give examples for and explain difference between Frequentist and Bayesian probabilities Recall definitions of basic probability distributions (Binomial, Poisson, Normal, Chi-squared) and their properties Apply the central limit theorem and specify its limitations Apply hypothesis testing and define confidence levels and various types of inference errors
• Give examples for and explain difference between Frequentist and Bayesian probabilities
• Recall definitions of basic probability distributions (Binomial, Poisson, Normal, Chi-squared) and their properties
• Apply the central limit theorem and specify its limitations
• Apply hypothesis testing and define confidence levels and various types of inference errors
• Apply the chi-squared goodness-of-fit test to simple and composite hypothesis Apply the $$\Delta \chi^2$$ method for signal detection Know the basic definitions related to general estimators
• Apply the chi-squared goodness-of-fit test to simple and composite hypothesis
• Apply the $$\Delta \chi^2$$ method for signal detection
• Know the basic definitions related to general estimators
• Know the definition of Fisher information and Cramer-Rao, and be able to apply these definitions in simple examples Know and apply the maximum likelihood estimator (MLE) to given parametric probability distribution function Understand and apply the maximum-likelihood-ratio test to various problems Sketch the basic steps of the derivation of Wilk's theorem, know what the theorem is about and be able to apply it to simple examples; know the limitations of Wilk's theorem Know how to estimate global significance from local significance, if number of trials is given
• Know the definition of Fisher information and Cramer-Rao, and be able to apply these definitions in simple examples
• Know and apply the maximum likelihood estimator (MLE) to given parametric probability distribution function
• Understand and apply the maximum-likelihood-ratio test to various problems
• Sketch the basic steps of the derivation of Wilk's theorem, know what the theorem is about and be able to apply it to simple examples; know the limitations of Wilk's theorem
• Know how to estimate global significance from local significance, if number of trials is given
• Understand general definition of confidence region Be able to construct confidence regions using the likelihood ratio construction (if Wilks' theorem applies) Apply Neyman belt construction to simple examples Know the components of Bayes theorem, and apply them to simple scenarios
• Understand general definition of confidence region
• Be able to construct confidence regions using the likelihood ratio construction (if Wilks' theorem applies)
• Apply Neyman belt construction to simple examples
• Know the components of Bayes theorem, and apply them to simple scenarios
• Apply Bayesian model comparison and interpret the Bayes factor in terms of Jeffreys' scale Exemplify the mechanism of Ockham's razor in Bayesian inference Understand general definition of various credible intervals Give examples for non-informative and informative priors Derive priors using the maximum entropy principle, using Lagrange multipliers
• Apply Bayesian model comparison and interpret the Bayes factor in terms of Jeffreys' scale
• Exemplify the mechanism of Ockham's razor in Bayesian inference
• Understand general definition of various credible intervals
• Give examples for non-informative and informative priors
• Derive priors using the maximum entropy principle, using Lagrange multipliers
• Understand various sampling techniques (inverse transform, reject/accept sampling) Understand and apply Metropolis-Hastings MCMC
• Understand various sampling techniques (inverse transform, reject/accept sampling)
• Understand and apply Metropolis-Hastings MCMC

Teaching methods

• Lecture
• Seminar
• Self-study

The main course material will be presented in the lectures, and can be read in the course notes. The three homework exercises give the student the opportunity to test the material in practice. The homework include a significant amount of writing statistical programs in the programming language Python.

Learning activities

Attendance

Requirements concerning attendance (OER-B).

Additional requirements for this course:

The full attendance of both the lectures and homework sessions is strongly encouraged.  In the case of absence, the course coordinator should be notified.

Assessment

The final grade is given by 80% final exam and 20% homework. The exam has to be passed with a grade of at least 6.0 (50% of the points) in order to complete the course. If the exam grade is below 6.0, it will be the final grade of the course. Same rules apply to the retake.

Inspection of assessed work

Contact the course coordinator to make an appointment for inspection.

Assignments

Homework 1

• First homework

Homework 2

• Second homework

Homework 3

• Third homework

Homework exercises can be discussed in groups, but must be handed in individually.

Fraud and plagiarism

The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl

Course structure

Timetable

The schedule for this course is published on DataNose.

Additional information

Recommendend prior knowledge: Good knowledge of the material from 'Statistical Data Analysis' (Snoek) is recommended though not necessary. Many of the exercises will require the use of the programming language python.

Contact information

Coordinator

• dr. C. Weniger