Krdytkyu

This is a rather odd request.

I only recently started studying the Lagrange Multipliers, and was given a task to create some challenging problems on them and also provide solutions. Could somebody please suggest how I could get started on this? Some example problems would be welcome!

Thanks very much!

Sure, here's 2:

a really good one to start out with is the optimization problem behind Principal Component Analysis.

$$text{max$_{bf v}$ } langle bf{v},Sigma_n bf{v}rangle$$ $$text {s.t.}$$ $$||bf {v}||_2=1$$

where $bf v$ is a vector and $Sigma_n= frac 1nsum_{i=1}^n(bf x_i -mu_n)(bf x_i - mu_n)^T$ is the Covariance matrix of the data points $bf x_i$

The answer for $bf v$ is the vector that points in the direction of the eigenvector of $Sigma_n$ corresponding to its largest eigenvalue $lambda_{max}$

Another really good application of Lagrange multipliers/ difficult problem involving Lagrange multipliers is solving for the Euler equation in Economics for logarithmic utility. This is extremely important in the theory of dynamic programming as well.

$$max sum_{t=0}^{T-1} lnc_t + lnx_T$$ $$text{s.t.}$$ $$x_{t+1} =alpha (x_t-c_t)$$

Be careful, $x_t$ shows up twice. Interpret as maximizing consumption and final wealth where $x_t$ is the wealth at period $t$ with $t=0$ corresponding to initial wealth and $c_t$ is the consumption for period $t$.

The solution is: $x_t^*=frac {T+1-t}{T+1}(frac 1{alpha})^{-t}x_0$ and $c_t^*=frac {(frac 1{alpha})^{-t}x_0}{T+1}$

These two problems include applications of lagrange multipliers to 1. a problem involving matrix calculus. and 2. a problem in which an optimal consumption path is determined (function), which has ties to optimal control and the functional optimization.

Here is a moderately complicated example, cooked up in such a way that it can be solved explicitly all the way to the end.

Given are the two points $A=(0,-7)$, $B=(-7,0)$, and the circle $gamma: >x^2+y^2=4$. Determine two points $P$, $Qingamma$ such that the quantity
$$d(P,Q):=|AP|^2+|PQ|^2+|QB|^2$$
becomes maximal, resp., minimal.

You will obtain four conditionally stationary situations $(P_k,Q_k)$: the two extremal situations and two "saddle points". The latter had to be expected, since the surface determined by the conditions is a torus.

A full solution (in German) is given on pp. 212–214 of the following pdf-file. The file contains a full chapter (pages 128–236) of a textbook for engineering students. Scroll down to page 212; there you will see the figure printed above.

https://people.math.ethz.ch/~blatter/Inganalysis_5.pdf

Here two other ones :

Exercice 1 :

Let $alphainmathbb{R}_{+}$. For all $left(x,y,zright)inmathbb{R}^{3}$, find the optimal constant $Cleft(alpharight)inmathbb{R}$
such that $$x^{3}-left(frac{z}{alpha}right)^{3}leq Cleft(alpharight)left(x^{2}+y^{2}+z^{2}right)^{3/2}$$
and precise the case of equality in term of $alpha$.

Solution :

We notice that the both sizes of the above equation are homogeneous functions of degree $3$ : it suffices to consider the problem on the unit sphere $mathcal{S}=left{ left(x,y,zright)inmathbb{R}^{3};x^{2}+y^{2}+z^{2}=1right}$.
Soient $f$ et $g$ the functions defined on $mathbb{R}^{3}$ by $$begin{cases}
fleft(x,y,zright)=x^{3}-left(frac{z}{alpha}right)^{3}\
gleft(x,y,zright)=x^{2}+y^{2}+z^{2}-1
end{cases}.$$

As $f$ is continuous and $mathcal{S}$ is compact, $f_{midmathcal{S}}$ is bounded and attain its maximum values.
For all $left(x,y,zright)inmathbb{R}^{3}$, we have
$$begin{cases}
mathrm{d}fleft(x,y,zright)=3x^{2}dx-3frac{z^{2}}{alpha^{3}}dz\
mathrm{d}gleft(x,y,zright)=2xdx+2ydy+2zdz
end{cases}$$
or in matricial notations in the basis $left(mathrm{d}x,mathrm{d}y,mathrm{d}zright)$
of $left(mathbb{R}^{3}right)^{*}$

$$begin{pmatrix}3x^{2} & 0 & -3frac{z^{2}}{alpha^{3}}\
2x & 2y & 2z
end{pmatrix}.$$

Cancelling all the $begin{pmatrix}3\2end{pmatrix}=3$ $2times2$-determinants so that $mathrm{d}fleft(x,y,zright)$ and $mathrm{d}gleft(x,y,zright)$ are linearly dependant yields :

$$begin{vmatrix}3x^{2} & 0\
2x & 2y
end{vmatrix}=6x^{2}y=0Longleftrightarrowleft(x=0right)textrm{ ou }left(y=0right).$$
$$ begin{vmatrix}3x^{2} & -3frac{z^{2}}{alpha^{3}}
2x & 2z
end{vmatrix}=6xzleft(x+frac{z}{alpha^{3}}right)=0Longleftrightarrowleft(x=0right)textrm{ ou }left(z=0right)textrm{ ou }left(x=-frac{z}{alpha^{3}}right).$$
$$begin{vmatrix}0 & 3frac{z^{2}}{alpha^{3}}\
2y & 2z
end{vmatrix}=6yfrac{z^{2}}{alpha^{3}}=0Longleftrightarrowleft(y=0right)textrm{ ou }left(z=0right).$$

Then, for all $left(x,y,zright)inmathbb{R}^{3}$ :

$(i)$ if $alpha<1$, $$fleft(x,y,zright)leqfrac{1}{alpha^{3}}left(x^{2}+y^{2}+z^{2}right)^{3/2}$$
with equality iff $left(x,y,zright)inleft{ left(0,0,-lambdaright);lambdainmathbb{R}_{+}^{*}right}$ ;

$(ii)$ if $alpha=1$, $$fleft(x,y,zright)leqleft(x^{2}+y^{2}+z^{2}right)^{3/2}$$
with equality iff $left(left(x,y,zright)inleft{ left(lambda,0,0right);lambdainmathbb{R}_{+}^{*}right} right)$
or $left(left(x,y,zright)inleft{ left(0,0,-lambdaright);lambdainmathbb{R}_{+}^{*}right} right)$ ;

$(iii)$ If $alpha>1$,
$$fleft(x,y,zright)leqleft(x^{2}+y^{2}+z^{2}right)^{3/2}$$
with equality iff $left(x,y,zright)inleft{ left(lambda,0,0right);lambdainmathbb{R}_{+}^{*}right} $.

Exercice 2 :

Let $n$,$alphainmathbb{N}^{*}$, $1leqalphaleq n$.
For all $left(x_{1},ldots,x_{n}right)inmathbb{R}^{n}$, find the optimal constant $Cleft(n,alpharight)inmathbb{R}$ such that
$$x_{1}ldots x_{n}leq Cleft(n,alpharight)left(x_{1}^{alpha}+ldots+x_{n}^{alpha}right)^{n/alpha}$$
and precise the case of equality in term of $alpha$.

Solution :

We notice that the both sizes of the above equation are homogeneous functions of degree $n$ : it suffices to consider the problem on the hypersurface $mathcal{S}=left{ left(x_{1},ldots,x_{n}right)inmathbb{R}^{n};x_{1}^{alpha}+ldots+x_{n}^{alpha}=1right}$.
Let $f$ and $g$ the functions defined on $mathbb{R}^{n}$ by

$$begin{cases}
fleft(x_{1},ldots,x_{n}right)=x_{1}ldots x_{n}\
gleft(x_{1},ldots,x_{n}right)=x_{1}^{alpha}+ldots+x_{n}^{alpha}-1
end{cases}.$$

We can assume that $x_{j}>0$ for all $1leq jleq n$ (the inequality is stronger in the case where there is an even number of $x_j<0$, which is the same that consider them all non-negative).
For all $left(x_{1},ldots,x_{n}right)inmathbb{R}_{+}^{n}$, we have

$$begin{cases}
dfleft(x_{1},ldots,x_{n}right)=sum_{j=1}^{n}left(prod_{kneq j}x_{k}right)dx_{j}\
dgleft(x_{1},ldots,x_{n}right)=alphasum_{j=1}^{n}left(x_{j}^{alpha-1}right)dx_{j}
end{cases}$$
or in matricial notations

$$begin{pmatrix}x_{2}x_{3}ldots x_{n} & x_{1}x_{3}x_{4}ldots x_{n} & cdots & x_{1}x_{2}ldots x_{n-2}x_{n-1}\
alpha x_{1}^{alpha-1} & alpha x_{2}^{alpha-1} & cdots & alpha x_{n}^{alpha-1}
end{pmatrix}.$$

We cancel all the $begin{pmatrix}n\2end{pmatrix}$ $2times2$-determinants so that $mathrm{d}fleft(x_{1},ldots,x_{n}right)$ and $mathrm{d}gleft(x_{1},ldots,x_{n}right)$ linearly dependant : for all $1leq k$, $ellleq n$ with $kneqell$,
$$begin{vmatrix}x_{1}ldots x_{k-1}x_{k+1}x_{n} & x_{1}ldots x_{ell-1}x_{ell+1}ldots x_{n}\
alpha x_{k}^{alpha-1} & alpha x_{ell}^{alpha-1}
end{vmatrix}=alpha x_{1}ldots x_{k-1}x_{k+1}ldots x_{ell-1}x_{ell+1}ldots x_{n}left(x_{ell}^{alpha}-x_{k}^{alpha}right)=0
Longleftrightarrow x_{k}=x_{ell}Longrightarrow x_{1}=left(frac{1}{n}right)^{frac{1}{alpha}}$$
with the constraint $gleft(x_{1},ldots,x_{n}right)=0$.
We get $$fleft(left(frac{1}{n}right)^{frac{1}{alpha}},ldots,left(frac{1}{n}right)^{frac{1}{alpha}}right)=left(frac{1}{n}right)^{frac{n}{alpha}}$$
and hence $$f_{midmathcal{S}}left(x_{1},ldots,x_{n}right)leqleft(frac{1}{n}right)^{frac{1}{alpha}}.$$
We extend this result by homothetie to $mathbb{R}^{n}$ :
for all $left(x_{1},ldots,x_{n}right)inmathbb{R}^{n}$, $$x_{1}ldots x_{n}leqleft(frac{x_{1}^{alpha}+ldots+x_{n}^{alpha}}{n}right)^{frac{n}{alpha}}$$
with equality iff $x_{1}=ldots=x_{n}$.