Context and scope

In this lecture we focus on the important computational aspect of convexity. We will see that:

• “optimality conditions” give a simple way to verify optimality of solutions
• convex optimization problems can be solved (approximated) efficiently
• on the negative side, without convexity, even very simple optimization problems can be computationally hard
• later in this course, we will also see statistical / out-of-sample guarantees implied by convexity

Convex analysis recap

• The notion of convexity plays a central role in optimization (we will see why a bit later)
• Here we make a quick recap of convex sets, convex functions and some basic properties
• Most proofs will be omitted and can be found in standard treatments of convex optimization

Convex sets

<aside> 💡

Definition: Convex set

A set $S \subseteq \R^d$ is convex iff

$$ \lambda x + (1-\lambda)y \in S ~,

\qquad \forall x,y \in S ~, 0 \leq \lambda \leq 1. $$

</aside>