This course introduces mathematical optimization and modeling, with a focus on convex optimization. We will use convexity as a starting point from which to consider some nonconvex problem types. The course will have a practical focus, with participants formulating and solving optimization problems early and often using standard modeling languages and solvers. By introducing common models from machine learning and other fields, the course aims to make students comfortable with optimization modeling so that they may use it for rapid prototyping and experimentation in their own work.
We'll cover simple but useful optimization algorithms such as gradient descent for when problems become too large for black-box solvers and cover majorization-maximization as a useful framework to view optimization algorithms for both convex and non-convex problems. Parallel and stochastic gradient descent will be covered in the contexts of distributed and large-scale optimization. Time permitting, we will also discuss methods to solve nonconvex models for nonnegative matrix factorization, matrix completion, and neural networks. Students should be comfortable with linear algebra, differential multivariable calculus, and basic probability and statistics. Experience with Python will be helpful, but not required.
- Varieties of mathematical optimization
- Convexity of functions and sets
- Convex optimization modeling with CVXPY
- (Stochastic) gradient descent and basic distributed optimization
- In-depth examples from machine learning, statistics and other fields
- Applications of bi-convexity and non-convex gradient descent
To view other workshop descriptions, or to get general information about the ICME Summer Workshop Series, click here.
ICME Summer Workshops are open to participants 18 years and older. If you are under the age of 18 and would like to participate, please contact email@example.com.
Questions: please contact the ICME at firstname.lastname@example.org .