Penalty method

Penalty method

card Is a method for solving Optimization problem that is constrained with unconstrained $f^{\prime }$, that

1. Contains function $f$ from the original problem
2. For each constrain there is penalization element, which is zero $⟺$ constrain is met, otherwise it’s positive
3. Penalization elements are often multiplied in each step with some variable ${\nu }_k$.

Quadratic penalization

card $Q(x,v)=f(x)+\frac{\nu }{2}\sum g(x{)}^2+\frac{\nu }{2}\sum ([h(x){]}^{+}{)}^2$ where $[y{]}^{+}=max(y,0)$

Algorithm for the quadratic penalization

card

1. declare $v_0>0$, and sequence ${\tau }_k$, ${\tau }_k\to 0$
2. for $k=\{\mathrm{0..1}\}$
   1. find approx minimum $x_k$ of the function $Q(x,{\nu }_k)$ with breaking condition $||{\nabla }_{x}Q(x,{\nu }_k)||\le {\tau }_k$,
   2. if the final condition for convergence is reached: return $x_k$
   3. otherwise set ${\nu }_{k+1}>{\nu }_k$

$l_1$ penalization

card names after the $l_1$ norm $\phi (x,\nu )=f(x)+\nu \sum |g(x)|+\nu \sum [h(x){]}^{+}$