We will now turn to consider mathematical optimization as a computational problem.

Consider the usual:

$$ \begin{aligned} \min ~& f( x) \\ \text{s.t.} ~~& x \in S \subseteq {\R}^d \end{aligned} $$

Assume that the minimum is attained at some $x^\in S*$ and denote $f^ = f(x^)$.

Can it be solved algorithmically?

• What does it mean to “solve” the problem?
• How to specify the objective $f$ to the algorithm?
• How to specify the domain $S$ to the algorithm?
• What assumptions to place on $f$ and $S$ for the problem to make sense?
• When can we say that a problem is efficiently solvable? That an algorithm is efficient?

Solution concepts

Exact solution

An algorithm solves the optimization problem exactly if it returns $\hat { x} \in \argmin_{ x \in S} f( x)$.

This is unrealistic, even for extremely simple optimization problems:

$$ \begin{aligned} \min ~~& x + \frac{2}{x} \\ \text{s.t.} ~~~& x \geq 1 \end{aligned} $$