STA376: Advanced modeling & scientific computing: Spring 2012

Prof:Sayan Mukherjee sayan@stat.duke.edu OH: Friday 1-2pm, 112 Old Chem
Class:Tu/Thu 1:15-2:30pm 025 Old Chem

Syllabus

WeekTopicHomework
I. Numerical analysisProblems
Jan 12 Overview of numerical methods and computing
Jan 17, 19 Matrix vector operations and stability analysis (Stuart and Voss, extra notes Sayan) Extra notes
Jan 24, 26 Factorizations (Stuart and Voss)
Jan 31, Feb 2 Linear systems (Stuart and Voss) hw3
Feb 7, 9 Optimization: primal and dual theory with SVMs as example
Feb 14, 21 Iterative methods
Feb 23 Gaussian process example
II. Markov chain Monte Carlo
Feb 28, March 1, 13 Markov chain theory (From Geyer)
--- Spring break (March 02-12) ---
March 15 ,20 Basic algorithms (From Geyer, Green, Neal)
March 22, 27 Diagnostics hw5
April 3,5 Convergence and complexity
III. "Modern methods"
April 10, 12 Randomized algorithms for massive data hw11
April 12, 17 Platforms for scientific computing hw13
April 24 Final projects due (due 2pm) example projects

Description

This is a course about numerical methods used in statistical computing with a focus on topics particularly relevant in Bayesian modeling. The course also will have a component that will develop ideas of use for large scale data analysis. The course can be roughly sub-divided into three sections: (a) a crash course on numerical analysis, (b) a crash course on theory and practice of Markov chain Monte Carlo, (c) two weeks on modern methods such as randomized algorithms and novel computational platforms.

It is recommended to have taken STA 290 (Modern Statistical Data Analysis), STA 215 (Statistical Inference), and STA 244 (Linear Models) before this course but they are not required. One will better appreciate the utility of the material in the course after taking the above classes. Knowledge of linear algebra, multivariate calculus, and properties on the multivariate normal distribution are assumed.

There is no text for this class. However some texts and notes of use are

  1. For the first section on numerical methods
    1. Matrix Computations by G. Golub and C. Van Loan
    2. G. Golub's lecture notes (by way of Lek-Heng Lim)
    3. Matrix Algebra from a Statistician's Perspective by D. Harville
    4. Matrix Analysis and Algorithms by A.M. Stuart and J. Voss
  2. For the second section on Markov chain Monte Carlo
    1. Monte Carlo Strategies in Scientific Computing by J.S. Liu
    2. Markov chain Monte Carlo lecture notes by C. Geyer
    3. A mathematically motivated review notes by P. Diaconis
  3. For the third section on "Modern methods"
    1. Algorithms for massive data sets by M. Mahoney
    2. Fast methods in scientific computing by G. Martinsson

Course grade is based on homework (15%), take home midterm (35%) and a final project (50%). Example final projects.

This syllabus is tentative, and will almost surely be superceded- RELOAD your browser for the current version.