MVE326/MSF100 Statistical inference principles

Jump to today

This page contains the program of the course: lectures, exercise sessions and computer labs. Other information, such as learning outcomes, teachers, literature and examination, are in a separate course PM.

Program

The schedule of the course is in TimeEdit.

The first meeting to fix the schedule of the course is on Wednesday, January 22, 8:00 in MVH:11. Timetable for further lectures will be decided together.

Lectures

Notice  [CB] = Casella and Berger, "Statistical Inference", 2nd edition 2002, Brook/Coole.

[Murphy] = K. Murphy, "Machine learning: a probabilistic perspective", 2012, MIT Press.

[HMC] = Hogg, McKean, Craig, "Introduction to mathematical Statistics", 7th ed, Pearson

[CASI] = Efron and Tibshirani. "Computer age statistical inference",  2016 Cambridge University Press. FREELY available here

Day Sections Content
22 Jan [CB]: some of the topics are found in ch. 1, 2, 4

Quick summary of basic probabilistic results: sets and events; sigma-algebras; probability functions, random variables, cum. distr. funct. (cdf), pdf/pmf, expectations, variance/covariance, moments generating function, joint and marginal distributions

formula sheet summary
24 Jan section 3.4 in CB; section 5.1, Th. 5.2.11, section 5.4, section 6.1 and 6.2 up to definition 6.2.1

unbiasedness; random samples; statistics; parametric families and exponential family; order statistics (with proof of Th. 5.4.3 in CB); data reduction and sufficiency .

assignment1.pdf due 31 January
27 Jan [CB] chapter 6 until theorem 6.2.13 more on sufficiency and examples, with proof of Th. 6.2.2 (the construction of the proof in [CB] actually precedes the theorem statement (!)); the Fisher-Neyman factorization theorem (with proof). Minimal sufficient stats and the Lehmann-Scheffe' theorem (only stated. Proof on Wednesday). Check also this.
29 Jan proved Lehmann-Scheffe' theorem. Likelihood function, likelihood principle (also strong/formal version)
31 Jan

presentations for assignment1.

Also illustrated an example for the Strong Likelihood Principle.

assignment2.pdf
3 Feb [CB] chapter 7 until theorem 7.2.10 (with proof)

Point estimation: method of moments, maximum likelihood and the invariance property (theorem 7.2.10 (with proof)).

The following gives the original proof for the invariance (not necessary, but see the proof in [CB]): Zehna_1966_invariance_mle.pdf
6 Feb [Murphy] section 11.4, except the parts crossed out. [CB]  sect. 7.2.4

Introduced two examples of "Incomplete data" models (nonlinear time series, Gaussian mixtures), the EM algorithm, its detailed theoretical construction and proof of the monotonicity property. For a nice exposition see murphy_EM.pdf
7 Feb

minutes of the midcourse meeting

presentations for assignment 2. EM_missingvalues.pdf

assignment3.pdf, em_data.dat, em_data_full.dat, storms.txt
10 Feb [CB] section 7.3 until Theorem 7.3.17 (but not corollary 7.3.15)

mean squared errors, best unbiased estimators, Cramér-Rao inequality (with proof of Theo. 7.3.9), Rao-Blackwell Theorem (only stated, proof on Wednesday)
12 Feb [CB] theorem 7.3.17 (with proof), 7.3.19 (no proof), sect. 7.3.4 without Bayesian approaches. section 8.1

proof of Rao-Blackwell theorem; examples with Rao-Blackwell; decision-theoretic approaches; hypothesis testing
14 Feb

Presentations for assignment 3;

Likelihood ratio tests (started)

assignment4.pdf
17 Feb

[CB] theorem 8.2.4, sect 8.3. For "best rejection region" (=same as UMP in [CB]) see [HMC]; for Neyman-Pearson theorem see again [HMC]

 

theorem 8.2.4 (with proof), LRT examples, type I/II errors, power function, best rejection region (or uniformly most powerful test, UMP); Neyman-Pearson lemma (no proof)
19 Feb [CB] section 8.3, 9.1

 

lemma Neymann-Pearson in presence of suff. stats. (corollary 8.3.13 with proof), monotone likelihood ratio + example, Karlin-Rubin theorem (only statement), p-values, intro to interval estimation
21 Feb [CB] example 9.2.1

presentations for assignment 4; confidence intervals via inversion of statistical test
24 Feb

[CB ] Theorem 9.2.2 with proof.

Some notes on the bootstrap.

For the interested reader: [CASI] ch. 10-11

 

correspondence between (CI) and acceptance regions; the bootstrap method(s) for finding CI via simulation and via the percentile method.

bootstrap slides;

matlab example
26 Feb

[CB] ch. 10 until section 10.1.2 excluded

checking confidence intervals coverage using simulation;

asymptotics: consistency; proof of Theorem 10.1.3; convergence in probability; consistency of MLE (Th. 10.1.6 with proof) requiring a brush-up of the (weak) law of large numbers.

coverage slides

coverage R code

mle is consistent
28 Feb

 

no presentations

assignment5.pdf
2 March

[CB] section, 5.5.3, 5.5.4, 10.1.2

efficiency and asymptotic efficiency; convergence in distribution; central limit theorem; asymptotic efficiency of the mle; delta method
6 March

 

Assignment 5 presentations

 

Back to the top

Recommended exercises

Day Exercises

 

 

Back to the top

Course summary:

Date Details Due