Context and scope

In this lecture we focus on the important computational aspect of convexity. We will see that:

• “optimality conditions” give a simple way to verify optimality of solutions
• convex optimization problems can be solved (approximated) efficiently
• on the negative side, without convexity, even very simple optimization problems can be computationally hard
• later in this course, we will also see statistical / out-of-sample guarantees implied by convexity

Convex optimization, functional constraints

Our focus here is on convex (possibly constrained) optimization, that is:

<aside> <img src="/icons/looped-square_orange.svg" alt="/icons/looped-square_orange.svg" width="40px" />

Convex optimization, generic form:

$$ \begin{align*} \min ~& f(x) \\ \text{s.t.} ~~& x \in S \end{align*} $$

where both $f$ and $S$ are convex

</aside>

A canonical, general form for convex problems is using functional (convex) constraints:

<aside> <img src="/icons/looped-square_orange.svg" alt="/icons/looped-square_orange.svg" width="40px" />

Convex optimization, functional constraints form:

$$ \begin{align*} \min ~& f(x) \\ \text{s.t.} ~~& f_i(x) \leq 0 \quad \forall ~ i \in [m] \end{align*} $$

where $f,f_1,\ldots,f_m$ are all convex

</aside>

• Note this is a convex problem since the feasible domain is intersection of $\{x : f_i(x) \leq 0 \}$ which are the zero level sets of the $f_i$, and therefore convex