Lecture Notes

Complete Lecture Notes (PDF 1.3MB)

Introduction (PDF)

  • Regression Analysis and Prediction Risk
  • Models and Methods

Chapter 1: Sub-Gaussian Random Variables (PDF)

  • Gaussian tails and MGF
  • Sub-Gaussian Random Variables and Chernoff Bounds
  • Sub-Exponential Random Variables
  • Maximal Inequalities

Chapter 2: Linear Regression Model (PDF)

  • Fixed Design Linear Regression
  • Least-Squares Estimation
  • The Gaussian Sequence Model
  • High-Dimensional Linear Regression

Chapter 3: Misspecified Linear Models (PDF)

  • Oracle Inequalities
  • Nonparametric Regression

Chapter 4: Matrix Estimation (PDF)

  • Basic Facts About Matrices
  • Multivariate Regression
  • Covariance Matrix Estimation
  • Principal Component Analysis

Chapter 5: Minimax Lower Bounds (PDF)

  • Optimality in a Minmax Sense
  • Reductions to Finite Hypothesis Testing
  • Lower Bounds Based on Two Hypotheses
  • Lower Bounds Based on Many Hypotheses
  • Application to the Gaussian Sequence Model