Stat 471 Computational Statistics

Instructor: Moo K. Chung
E-mail: mchung@stat.wisc.edu
Office:  4382 CS&S  Tel: (608) 262-1287

Lectures: MWF 1:20-2:10pm 1289 COMP S&ST
Office HR WF 2:30-4:30pm.
Requirements:  Stat/Math 309-310 or Stat 311-312 and basic understanding of computer programming. Assignment problems and project will require computer programming in matlab, S+/R or any other programming language. Most topics will be self contained.

Textbooks: Computational Statistics Handbook with MATLAB

Wendy L. Martinez & Angel R.  Martinez
Chapman & Hall/CRC, 2002
ISBN 1-58488-229-8

Advanced statistical computing course notes (300page PDF) by Robert Gray, 2001. This is for Splus/R users. If you are intending to use Splus/R for submitting homeworks and projects, I recommand you to read it as well as the textbook.

Topics covered
Data Structure: Vector and matrix data manipulation. Data input/ouput. Data format.
Statistical graphics: scatter plots,histograms, quantile-quantile(QQ) plot. visualizing bivariate data.
Random numbers: probability integral transform,  how to generate random variables, vectors, matrices and stochastic processes.
Matrix algebra: Singular value decomposition, Cholesky factorization, Generalized inverse (Psudo inverse). least-squares estimation.
Monte-Carlo methods: Monte-Carlo integral, importance sampling, Monte-Carlo inference.
Markov Chains: random walks, transition probability. Chapman-Kolmogorov equation. invariant distributions.
Markov chain monte carlo (MCMC): Metropolis-Hastings algorithms. Gibbs sampler. Bayesian inference. Bayes estimates.
Bootstrap methods: bootstraping techniques are introduced for estimation and inference problems.
Monte-Carlo optimization: maximum likelihood, EM algorithm
Smoothing: bivariate smoothing, nonparametric regression, cross-validation, kernel density estimation, simulation of Gaussian processes.

Course Evaluation
There are two options. Assignment 70% + Project 30% or Assignment 50% + Project 50%.
Assignments (70% or 50%) 5 challenging homeworks. Half theoretical, half computional.
Project (30% or 50%) Students are required to submit minimum 10 page (30%) or 15 page (50%) double  spaced and typed report excluding figures and computer codes by the last class day on topics discussed during the class (projects done in other class will not be accepted). For gradaute sudents from other department, you can do a project in your own research area after consultation.

Computer Access and Softwares
The simplest way to do homeworks and project would be to creat a computer account for statistics computers in the department. Statistics computer lab is at the first floor.

1. R http://cran.us.r-project.org/
2. Creating an account in stat computers http://www.cs.wisc.edu/csl/doc/faq/started/index.html
Lectures There are some typos and mistakes. It will be fixed when I teach this course again. :)
lecture 01 MATLAB/OCTAVE
lecture 02 integral transform method
lecture 03 accept-reject method
lecture 04 quantile-quantile plot
lecture 05 multivariate normal I.
lecture 06 multivariate normal II.
lecture 07 diagonalization
lecture 08 generalized inverse
lecture 09 least squares estimation
lecture 10 Monte-Carlo integration
lecture 11 importance sampling I.
lecture 12 importance sampling II.
lecture 13 Monte-Carlo inference I.
lecture 14 Monte-Carlo inference II.
lecture 15 bootstrap I.
lecture 16 bootstrap II.
lecture 17 bootstrap III.
lecture 18 Markov Chains I.
lecture 19 Markov Chains II.
lecture 20 Markov Chains III.
lecture 21 MCMC I.
lecture 22 MCMC  II.
lecture 23 Gibbs Sampler
lecture 24 Bayesian Inference I.
lecture 25 Bayesian Inference  II.
lecture 26 Expectation-Maximization (EM) algorithm I.
lecture 27 Expectation-Maximization (EM) algorithm II.
lecture 28 bivariate smoothing.
lecture 29 Monte-Carlo smoothing.
lecture 30 cross-validation.
lecture 31 kernel density estimation.
lecture 32 mixture models
lecture 33 Gaussian processes I.
lecture 34 Gaussian processes II.