Optimization is one of the core components of machine learning – the essence of many ML algorithms is to build a model and learn the parameters in the model by using optimization techniques to maximize or minimize an objective function given the available data. In this module, you should understand what optimization is, how it’s used in ML, and what can be optimized. You should understand first-order optimization, higher order, and derivative-free optimization as well as concepts such as convexity, duality, momentum, etc.

This chapter is organized as follows:

2.1 Introduction to Optimization

2.2 Optimization Basics

2.3 Gradient-based Optimization

2.4 Constrained Optimization

2.5 Duality & Quadratic Programming

2.6 Other Techniques

2.7 Summary

<aside> ⬅️ Module 1: Revision

</aside>

<aside> ➡️ 2.1 Introduction to Optimization

</aside>

© 2022 — DD1420 Authors