Optimization: Rayleigh-Ritz and Linear Algebra

This is intended as a mini intro to some practical linear algebra.

  1. Linear Algebra: Bilinear Form - A Simple Optimization Problem
    1. Definitions
    2. Inner Product
    3. Quadratic Form and the Rayleigh-Ritz Quotient
    4. A real symmetric matrix has only real eigenvalues
    5. The eigenvectors of a real symmetric matrix form an orthonormal basis
    6. Spectral decomposition of a real symmetric matrix
    7. Equality of optimization of Rayleigh-Ritz quotient
    8. argmax_{\|x\|=1} x^TAx = \lambda_{max}
  2. Analysis
    1. Dual Space
    2. Operator Norm
    3. Sets