Krdytkyu

I want to solve the following problem. Let $G$ be a finite simple group of even order greater than $2$. Let $n$ be the number of involutions in $G$. Show that $n < |G| / 3$.

Edit: Adding the information I do know about $G$, although not sure whether it is useful for this problem. I can see that $G$ is non-abelian since all abelian simple groups have prime order.

Also before in the text book before this exercise there is a proposition that states that if $t in G$ is an involution with $m = |C_{G} (t) | $, then $C_{G} (t) $ is a proper subgroup of $G$ and $ |G | leq ( frac{1}{2} m(m+1) ) ! $.

Corollary $(2I)$ in the paper "On groups of even order" by Brauer and Fowler says that if $G$ is a simple group which contains $n$ involutions and $t= frac{|G|}{n}$ then $|G| < lceil t(t+1)/2 rceil !$.

This is a simple consequence of Theorem $(2F)$ or Theorem $(2H)$ in that paper so check if something along these lines is in your book. (Which book is it by the way?)

Arguing by contradiction, suppose that $n geq frac{|G|}{3}$. Then $t leq 3$ so $|G|<720$. There are just $5$ non-abelian simple groups of order less than $720$, which are $A_5$, $A_6$, $operatorname{PSL}_2(7)$, $operatorname{PSL}_2(8)$ and $operatorname{PSL}_2(11)$, and these have $15$, $45$, $21$, $63$, and $55$ involutions respectively. In no case does $n geq frac{|G|}{3}$ hold, which is a contradiction.

Added. You can use directly Theorem $6.7$ in Rose's "A course on Group Theory" which says: Let $G$ be a group of even order with precisely $n$ involutions, and suppose that $|Z(G)|$ is odd. Let $a = |G|/n$. Then $G$ has a proper subgroup $H$ such that either $|G:H|=2$ or $|G:H|<frac{1}{2}a(a+1)$.

Now suppose that $G$ is a finite simple group with precisely $n$ involutions. Since $|Z(G)|=1$ the preceding theorem applies. Note that $|G:H| neq 2$ since otherwise $H$ is normal in $G$. In fact, the stronger claim is true (which Derek mentioned in his answer), that $|G:H| geq 5$.

Assume for a contradiction that $n geq |G|/3$. Then $a := |G|/n leq 3$, so $G$ has a proper subgroup $H$ such that $|G:H|<6$ by Thm. $6.7$, thus $|G:H|=5$ by the preceding observation. But $|G:H|=5$ is only possible if $G cong A_5$ (do you see why?) and you are given that $A_5$ has less than $60/3=20$ involutions. That is a contradiction, however, and the proof is complete.

I think the following is essentially the same argument as in the proof of the Brauer-Fowler result cited by the_fox.

Let $I$ be the set of involutions in $G$ and $m=|I|$, $n=|G|$.

For $x in G$, define $C^*_G(x) = { g in G mid g^{-1}xg = x^{pm 1} }$. Then either $C^*_G(x) = C_G(x)$, or $|C^*_G(x):C_G(x)|=2$.
Notice that, if $u,v in I$ and $x=uv notin I$, then $u,v in C^*_G(x) setminus C_G(x)$.

Now, for $x in G$, let $beta(x)$ be the number of ordered pairs $(u,v) in I times I$ with $uv=x$. Then $m^2 = sum_{x in G}beta(x)$.

To estimate $beta(x)$, we consider three cases.

If $x = 1$, then clearly $beta(x)=m$.

If $x in I$ and $uv = x$ with $u,v in I$, then $u,v in C_G(x)$, so $beta(x)$ is the number of involutions in $C_G(x) setminus {x}$, which is at most $|C_G(x)|-2$.

Otherwise, if $x ne 1$ and $x notin I$, then either $beta(x)=0$, or $beta(x) le |C^*_G(x)| - |C_G(x)| = |C_G(x)|$.

We will use the well-known result that the only finite nonabelian simple group with a proper subgroup of index at most 5 is $A_5$, and that the result is true in $A_5$.
(In fact $|I|=|G|/4$ in $A_5$.)

So, for $m in I$, we have $|C_G(x)| le n/6$, and for $m in G setminus (I cup {1})$, $|C^*_G(x)| le n/6$ and hence $|C_G(x)| le n/12$. So we get

$$m^2 le m + m(n/6-2) + (n-m-1)n/12,$$
and then, putting $k=n/m$, we have
$$n^2/k^2 le n/k + n(n/6-2)/k + (n-n/k-1)n/12= n^2/(12k) - n/k +n^2/12- n/12,$$
so $$nleft(frac{1}{k^2} -frac{1}{12k}-frac{1}{12}right) le -frac{1}{k} - frac{1}{12},$$
but the left hand side is positive for $k le 3$, so $|I| < |G|/3$, QED.