Lecture Notes on Large-scale Unconstrained and Constrained Optimization

• Introduction and Basics of Unconstrained Optimization (basics, characterization of solutions, overview of algorithms)
• Line Search Methods (step length conditions and algorithms, global convergence conditions, and rate of convergence)
• Trust Region Methods (basic trust regions ideas, Cauchy point and minimum progress, dogleg and twodimensional subspace minimization, global convergence, local convergence)
• Linear and Nonlinear Conjugate Gradients (linear cg, rate of convergence, nonlinear cg, Fletcher-Reeves, Polak-Ribiere variant, global convergence FR CG and FR-PR CG)
• Quasi-Newton Methods, Nonlinear Least Squares, and Background of Constrained Optimization
• Interior Point Methods for Linear and Nonlinear Programming

Lecture Notes on Nonlinear Systems of Equations

• Introduction (discussion of basic issues, convergence of Newton, linesearch, stopping criteria)
• Newton with Gaussian elimination (exact Newton, finite difference Jacobian, exploiting sparsity)
• Newton-Krylov methods (soon)
• Broyden's method

Lecture Notes on Large, Sparse Eigenvalue Problems

• Introduction (basics - chapter 1 sections 1 and 2 and chapter 4 section 1)
• Perturbation Theory (chapter 1 section 3 and chapter 4 section 2)
• Overview of Methods (for this course)
• Krylov subspaces and Rayleigh-Ritz approximation (chapter 4 sections 3 and 4)
• Krylov sequence methods, Arnoldi and Lanczos methods and variants (implicitly restarted Arnoldi/Lanczos)
• Newton-based methods, Jacobi-Davidson method, Davidson's method

Useful material on Numerical Linear Algebra, based on David Watkins, Fundamentals of Matrix Computations (2nd ed.), Wiley:

• Intro
• Overview
• Chapters 1 and 2 - Solving Linear Systems of Equations
• Chapter 3 - Overdetermined Systems
• Chapter 4 - Singular Value Decomposition
• Chapter 5 - Eigenvalue Problems Part I
• Chapter 5 - Eigenvalue Problems Part II
• (more fun stuff to be added)

Large, sparse eigenvalue problems and sensitivity/accuracy of eigenvalue problems (from Summer School organized by the Materials Computation Center):

• Derivation of Methods
• Sensitivity and Accuracy