|
|
MATH 5486 / CS 5486
Numerical Analysis and
Software II
|
Numerical
Analysis and Software II
Large-Scale
Optimization and Nonlinear Systems of Equations
Many of the methods discussed in part II are
closely related to the Krylov methods discussed in part I of this course.
However, part II can be taken independently of part I.
Large nonlinear systems of equations and optimization problems arise in
the solution of nonlinear differential and integral equations, the computation
of steady states of dynamical systems, parameter estimation, the optimal design
of structures, inverse problems, and a range of other problems from science and
engineering. Nowadays, such problems are almost a standard ingredient of any
large-scale simulation or design problem. We look at several variants of
Newton’s method suitable for very large problems, and we consider extensions
that lead to robust methods. Robust methods are guaranteed to converge to a
solution or local minimum from any starting guess (under weak and reasonable
assumptions on the problem). In many cases, we will use iterative methods to
solve approximately the linear systems arising in Newton-type methods, and we
analyze the properties of these methods, conditions for convergence, and rates
of convergence. We will also consider several methods that avoid the (in
general) expensive computation of Jacobians or Hessians, such as secant methods
(like Broyden’s method) and Newton-Krylov methods.
For robustness, we consider line search and trust region methods. Although
theoretical considerations are an inherent part of the course, we focus on
practical methods for real-world applications.
This course goes substantially beyond what is taught in our 4000/5000
numerical analysis courses (which do provide a good background).
Instructor:
- Eric de Sturler (click to check what I do the rest of my time)
- Office: 544 McBryde
- Phone: 231-5279
- Email: sturler-at-vt-dot-edu
Textbooks:
1. Required textbook: Numerical Optimization, Nocedal and Wright, Springer
2. Optional/Recommended textbook: Solving Nonlinear Equations with Newton’s Method, C.T. Kelley, Society for Industrial and Applied Mathematics (SIAM), 2003
The lecture notes will cover significant additional material:
Lecture Notes for Iterative Methos (part I)
Lecture Notes for Nonlinear Systems (part IIa)
Lecture Notes on Optimization (part III) – to be developed
Other useful books on nonlinear systems of equations and/or optimization are:
- Numerical Methods for Unconstrained Optimization and Nonlinear Equations, J.E. Dennis, Jr. and R.B. Schnabel, SIAM, 1996 (originally Prentice-Hall 1983)
- Trust-Region Methods, A.R. Conn, N.I.M. Gould, and P.L. Toint, SIAM 2000
- Iterative Methods for Optimization, C.T. Kelley, SIAM 1997
- Iterative Methods for Linear and Nonlinear Equations, C.T. Kelley, SIAM 1995
Quizzes will only be given on request J
Class Projects (link will be added shortly)