Course - Continuous Optimisation HT26

• Course webpage
• Lecture notes (old):
  • 1a, Problems and solutions. Optimality conditions for unconstrained optimisation
  • 1b, Problems and solutions. Optimality conditions for unconstrained optimisation
  • 2, Problems and solutions. Optimality conditions for unconstrained optimisation
  • 3a, Methods for unconstrained optimisation. Linesearch algorithms
  • 3b, Methods for unconstrained optimisation. Linesearch algorithms
  • 4, Methods for unconstrained optimisation. Linesearch algorithms
  • 5, Steepest descent methods
  • 6a, Newton’s method for unconstrained optimisation
  • 6b, Newton’s method for unconstrained optimisation
  • 7, Quasi-Newton methods. Nonlinear least-squares and Gauss-Newton methods
  • 8, Trust region methods
  • 9a, Trust region methods
  • 9b, Trust region methods
  • 10, Optimality conditions for constrained problems
  • 11, Optimality conditions for constrained problems
  • 12, Penalty methods for constrained optimisation
  • 13, Augmented Lagrangian methods
  • 14, Interior point methods for inequality constrained optimisation problems
  • 15, Interior point methods for inequality constrained optimisation problems
  • 16, SQP methods for constrained optimisation (nonexaminable)
• Additional resources:
  • Proof of convergence of trust region methods (nonexaminable)
  • Proof of second-order optimality conditions for constrained problems (nonexaminable)
  • Mathematical background
  • Other useful resources
  • Prof. Yuji Nakatsukasa’s notes on Taylor’s theorem
  • Prof. Raphael Hauser’s continuous optimisation notes
• Lecture recordings:
  • 2024-2025
  • 2025-2026
• Other courses this term: Courses HT26^U
• My notes here are based primarily on the slides above, written by Prof. Coralia Cartis.
• See also: Course - Optimisation for Data Science HT25^U

Notes

• Notes - Continuous Optimisation HT26, Overview of results and methods^U

• Notes - Continuous Optimisation HT26, Optimisation terminology^U

• Notes - Continuous Optimisation HT26, Taylor’s theorem^U

• Notes - Continuous Optimisation HT26, Unconstrained optimality conditions^U

• Notes - Continuous Optimisation HT26, Unconstrained optimality conditions for convex problems^U

• Notes - Continuous Optimisation HT26, Linesearch methods^U

• Notes - Continuous Optimisation HT26, Steepest descent^U

• Notes - Continuous Optimisation HT26, Newton’s method^U

• Notes - Continuous Optimisation HT26, Quasi-Newton methods^U

• Notes - Continuous Optimisation HT26, Least-squares^U

• Notes - Continuous Optimisation HT26, Trust-region methods^U

• Notes - Continuous Optimisation HT26, Constrained optimisation problems^U

• Notes - Continuous Optimisation HT26, KKT conditions^U

• Notes - Continuous Optimisation HT26, Constrained optimality conditions for convex problems^U

• Notes - Continuous Optimisation HT26, Penalty methods^U

• Notes - Continuous Optimisation HT26, Augmented Lagrangian methods^U

• Notes - Continuous Optimisation HT26, Inequality constrained optimisation problems^U

• Notes - Continuous Optimisation HT26, Useful miscellany^U

Related notes

There is a significant amount of overlap with the Part B Course - Optimisation for Data Science HT25^U:

• Notes - Optimisation for Data Science HT25, Overview of results^U
• Notes - Optimisation for Data Science HT25, Terminology^U
• Notes - Optimisation for Data Science HT25, Convexity^U
• Notes - Optimisation for Data Science HT25, Steepest descent^U
• Notes - Optimisation for Data Science HT25, Steepest descent with inexact line search^U
• Notes - Optimisation for Data Science HT25, Misc^U

Problem Sheets

• Sheet 1 (solutions to A&C, additional past paper questions), redacted^?
• Sheet 2 (solutions to A&C), redacted^?
• Sheet 3 (solutions to A&C), redacted^?
• Sheet 4 (solutions to A&C), redacted^?