We will now turn to consider mathematical optimization as a computational problem.

Consider the usual:

$$ \begin{aligned} \min ~& f( x) \\ \text{s.t.} ~~& x \in S \subseteq {\R}^d \end{aligned} $$

Assume that the minimum is attained at some $x^\in S*$ and denote $f^ = f(x^)$.

Can it be solved algorithmically?

• What does it mean to “solve” the problem?
• How to specify the objective $f$ to the algorithm?
• How to specify the domain $S$ to the algorithm?
• What assumptions to place on $f$ and $S$ for the problem to make sense?
• When can we say that a problem is efficiently solvable? That an algorithm is efficient?

Many of these are straightforward for discrete problems, but here our focus in on continuous problems.

Solution concepts

Exact solution

An algorithm solves the optimization problem exactly if it returns $\hat { x} \in \argmin_{ x \in S} f( x)$.

This is unrealistic, even for extremely simple optimization problems: