Krdytkyu

Given a positive definite matrix $A in mathbb R^{n times n}$. If $operatorname{trace}(A) = 1$, for $n geq 3$, prove that $$text{det}(A) leq frac{n^n}{(n-1)^{2n}} text{det}(I -A)^2$$ and the equation only holds when $A = frac{1}{n}I$.

What I have tried:
For any positive definite matrix $P in mathbb{R}^{n times n}$, there exists an invertible matrix $Omega$, s.t.
begin{align}
&P = Omega^{-1} Sigma Omega \
&Sigma = text{diag}(x_1, cdots, x_n) \
&x_1, cdots, x_n > 0
end{align}
We have $text{det}(P) = text{det}(Sigma) =prod_{i = 1}^n x_i$ and $text{det}(I - P) = text{det}(I - Sigma) = prod_{i = 1}^n (1 - x_i)$.
Therefore, we only need to prove $forall x_1, cdots, x_n$ with $x_1, cdots, x_n > 0$ and $sum_{i=1}^n x_i=1$, the inequality holds:
begin{align}
prod_{i = 1}^n (1 - x_i)^2 geq frac{(n - 1)^{2n}}{n^n} prod_{i = 1}^n x_i
end{align}
Assume $f(x_1, cdots, x_n) = prod_{i = 1}^n (1 - x_i)^2 / prod_{i = 1}^n x_i$ and we then find the minimum of $f$.

In order to investigate behavior of the function $f$, at the first we consider the following auxiliary problem.

Let $0<x,y,a$ and $x+y+a=1$. If $a$ is fixed then

$$h(x,y)=frac{(1-x)^2(1-y)^2}{xy}=frac{(a+xy)^2}{xy}=frac{a^2}{xy}+2a+xy.$$

A function $frac{a^2}{t}+t$ has a derivative $-frac{a^2}{t^2}+1$, so it decreases when $0<t<a$ and increases when $a<t<1$. By AM-GM inequality, $xyle frac{(1-a)^2}4$ and the equality is attained iff $x=y$. In particular, if $frac{(1-a)^2}4le a$ then $h(x,y)$ attains its minimum at $(x,y)$ iff $x=y$. It is easy to check that $frac{(1-a)^2}4le a$ iff $$age 3-2sqrt{2}=0.1715dots.$$

Consider the function $f(x_1,dots, x_n)$. Fixing the smallest $x_l$ we restrict domain of $f$ to a compact set $C$ consisting of $x$ such that ${x_jge x_l}$ for each $j$ and $sum x_j=1$. Since $f$ on $C$ is continuous, it attains its minimum at some point $x$. Let $x_i$ be the largest coordinate of $x$. Then $$x_j+x_kle frac 23(x_i+x_j+x_k)le frac 23<1-(3-2sqrt{2})$$ for each remaining distinct $j$ and $k$, so the minimality of $f(x)$ on $C$ and the properties of the function $h$ imply that $x_j=x_k=t$ for each remaining $j$ and $k$. Then $x_i=1-(n-1)t$ and

$$f(x)=r(t)=frac{((n-1)t)^2}{1-(n-1)t}left(frac{(1-t)^2}{t}right)^{n-1}=frac{(n-1)^2(1-t)^{2n-2}}{(1-(n-1)t)t^{n-3}}.$$

If $t$ tends to $0$ or to $frac 1{n-1}$ then $r(t)$ tends to infinity. So $r(t)$ attains its minimum at a compact subset of an interval $(0, frac 1{n-1})$ in some its point $s$. Then $r’(s)=0$. This easily implies that

$$left((1-s)^{2n-2}right)’ (1-(n-1)s)s^{n-3}=(1-s)^{2n-2}left((1-(n-1)s)s^{n-3}right)'$$

$$(2-2n)(1-(n-1)s)s^{n-3}=(1-s)left((1-(n-1)s)s^{n-3}right)'$$

The following cases are possible

1) $n=3$. Then $-4(1-2s)=(1-s)left(1-2sright)'$ and $s=frac 13$.

2) $n>3$. Then

$$(2-2n)(1-(n-1)s)s^{n-3}=(1-s)left((n-3)s^{n-4}-(n-1)(n-2)s^{n-3}right)$$

$$(2-2n)(1-(n-1)s)s=(1-s)left((n-3)-(n-1)(n-2)sright)$$

$$s^2(n^2-n)+s(n^2-4n+1)-n+3=0$$

$$s=frac 1nmbox{ or }s=-frac {n-3}{n-1}<0.$$

Thus anyway $s=frac 1n $. Then all $x_j$ equal to $frac 1n$ and $f(x)=frac{(n - 1)^{2n}}{n^n}$.

Thanks so much for the help from @Alex Ravsky. I also write one proof based on his ideas.

When $n = 3$, since $x_1 + x_2 + x_3 = 1$, at least one of $x_1$, $x_2$, $x_3$ is larger than $frac{1}{3}$. Without loss of generality, we assume $x_1 geq frac{1}{3}$. Then construct the function $h(x_2, x_3)$ for $x_2, x_3 > 0$ and $x_2 + x_3 = 1 - x_1$:
begin{align}
h(x_2, x_3) = frac{(1 - x_2)^2 (1 - x_3)^2}{x_2 x_3} = frac{x_1^2}{x_2 x_3} + x_2 x_3 + 2 x_1
end{align}
Using the mean value inequality, we have $x_2 x_3 leq frac{(1 - x_1)^2}{4}$. Thus if $frac{(1 - x_1)^2}{4} leq x_1$, i.e. $x_1 geq 3 - 2 sqrt 2$, then we have
begin{align}
h(x_2, x_3) geq frac{4 x_1^2}{(1 - x_1)^2} + frac{(1 - x_1)^2}{4} + 2 x_1
end{align}
and the equality holds iff $x_2 = x_3 = frac{1 - x_1}{2}$. Since $x_1 geq frac{1}{3} > 3 - 2 sqrt 2$, the condition holds. Now consider the function $f(x_1, x_2, x_3)$. Then
begin{align}
f(x_1, x_2, x_3) = frac{(1 - x_1)^2}{x1} h(x_2, x_3) geq 4 x_1 + frac{(1 - x_1)^4}{4 x_1} + 2 (1 - x_1)^2
end{align}
and the equality holds iff $x_2 = x_3 = frac{1 - x_1}{2}$ for every $x_1 geq frac{1}{3}$. Assume $p(x_1) = 4 x_1 + frac{(1 - x_1)^4}{4 x_1} + 2 (1 - x_1)^2$, then for $x_1 geq frac{1}{3}$, we have
begin{align}
p'(x_1) = frac{(1 + x)^3 (-1 + 3 x)}{4 x^2} geq 0,
end{align}
thus $p(x_1)$ attains it minimum iff $x_1 = frac{1}{3}$, i.e.
begin{align}
f(x_1, x_2, x_3) geq left(frac{(1 - frac{1}{3})^2}{frac{1}{3}}right)^3 = frac{(3 - 1)^{2 cdot 3}}{3^3}
end{align}
and the equality holds iff $x_1 = frac{1}{3}$ and $x_2 = x_3 = frac{1 - x_1}{2} = frac{1}{3}$.

Then we consider the case for $n > 3$. Similarly, without loss of generality, we can assume $x_1 geq frac{1}{n}$. We still first fix $x_1$ and define
begin{align}
h(x_2, cdots, x_n) = prod_{i=2}^n frac{(1 - x_i)^2}{x_i}.
end{align}
We then assume $x_n$ is the largest one in $x_2, cdots, x_n$. For $sum_{i neq 1, n} x_i = 1 - x_1 - x_n$, we define the following functions ($1 < j, k < n$, $j neq k$):
begin{align}
g_{j, k}(x_j, x_k) = frac{(1 - x_j)^2 (1 - x_k)^2}{x_j x_k} = frac{a^2}{x_j x_k} + x_j x_k + 2 a,
end{align}
where $a = sum_{i neq j, k} x_i$. Similarly, if $a geq 3 - 2 sqrt 2$, then $g_{j, k}$ attains its minimum iff $x_j = x_k$. If there exists $j$ and $k$ s.t. $a = sum_{i neq j, k} x_i < 3 - 2sqrt 2$, i.e. $x_j + x_k > 2sqrt 2 -2 > 0.82$. Then at least one of $x_j$ and $x_k$ is no less than $0.41$, e.g. $x_j$, then $x_n geq x_j geq 0.41$ and thus $a > x_n geq 0.41$, which is a contradiction. Therefore all $sum_{i neq j, k} x_i geq 3 - 2 sqrt 2$. Now consider the function
begin{align}
g(x_2, cdots, x_{n-1}) = prod_{i=2}^{n-1} frac{(1 - x_i)^2}{x_i}.
end{align}
We first only let $x_2$ and $x_3$ move and fix the others. To attain the minimum we only need to consider $g_{2, 3}(x_2, x_3)$ and it must be $x_2 = x_3 = t$ for any other fixed variables. Then let $x_4$ move and we need to consider $g_{3, 4}(t, x_4)$. To attain the minimum, we then have $x_3 = x_4 = t$. Repeatedly, we get $x_2 = cdots = x_{n-1} = t$, thus $t = frac{1 - x_1 - x_n}{n-2}$. Now consider $h(x_2, cdots, x_n)$, we have
begin{align}
h(x_2, cdots, x_n) geq frac{(1 - x_n)^2}{x_n} left( frac{left(1 - frac{1 - x_1 - x_n}{n-2}right)^2}{frac{1 - x_1 - x_n}{n-2}} right)^{n-2}
end{align}
%Denote the right side as $p_{x_1}(x_n)$ and it can be proved that $p_{x_1}(x_n)$ attains the minimum when $x_n = frac{1 - x_1}{n - 1}$ if $x_1 leq frac{1}{n}$ ($x_n leq 1 - frac{1}{n}$).
Since $t = frac{1 - x_1 - x_n}{n - 2}$, $x_n = 1 - x_1 - (n - 2) t$. Therefore we can consider the function
begin{align}
r (t) = frac{(x_1 +(n - 2) t)^2}{1 - x_1 - (n - 2) t} left(frac{(1 - t)^2}{t} right)^{n-2}
end{align}
Since $x_1 geq frac{1}{n}$, we have $x_n leq 1 - frac{1}{n}$. It can be proved that $r(t)$ attains the minimum at $t = frac{1 - x_1}{n - 1}$. The proof is nothing than straightforward but complicated calculation thus we omit the proof. Therefore, $x_n = frac{1 - x_1}{n - 1}$ and $x_2 = cdots = x_{n-1} = frac{1 - x_1}{n - 1}$. Eventually we have
begin{align}
f(x_1, cdots, x_n) geq frac{(1 - x_1)^2}{x_1} left( frac{left(1 - frac{1 - x_1}{n - 1}right)^2}{frac{1 - x_1}{n - 1}} right)^{n-1}
end{align}
Denote the right side as $q(x_1)$, then we have
begin{align}
q'(x_1) = frac{(n - 1) (1 - x_1)^2
left(frac{(n - 1) left(1 - frac{1 - x_1}{n - 1}right)^2}{(1 - x_1)} right)^n
(n - x_1 - 2) (n x_1 - 1)
}{(x_1^2 (n + x_1 - 2)^3}
end{align}
Since $x_1 geq frac{1}{n}$, we have $1 - x_1 > 0$, $n + x_1 - 2 > 0$, $n - x_1 - 2 > 0$, $n x_1 - 1 geq 0$. Thus $q'(x_1) geq 0$ and $q(x_1)$ attains its minimum at $x_1 = frac{1}{n}$, thus $x_2 = cdots, x_n = frac{1 - x_1}{n - 1}=frac{1}{n}$. In other words, we prove that
begin{align}
f(x_1, cdots, x_n) geq left(frac{(1 - frac{1}{n})^2}{frac{1}{n}}right)^n = frac{(n - 1)^{2 cdot n}}{n^n}
end{align}
and the equality holds iff $x_1 = cdots x_n = frac{1}{n}$.