Krdytkyu

I am reading Statistics by David Freedman

Suppose the same die had been used to generate the data in tables 4A and 4C
(p. 531), rolling it first 60 times for table 4A, and then 600 times for table 4C. Can
you pool the results of the two tests? If so, how?

The author then pools the two tests as

Pooled $chi^2$ = 13.2 + 10 = 23.2, d = 5 + 5 = 10, p $approx$1 %.

My question is, why can't we sum the two table to create new frequency table like this(since same dice is used)

and then compute the $chi^2$ statistic with 5 degree of freedom?

Both methods are acceptable. Under the null hypothesis, the frequency table made by pooling the two frequency tables together can be used to compute a $chi^2_{5}$ statistic. And the method that the author suggests is also correct - under the null hypothesis the sum of the $chi^2_5$ statistics will have a $chi^2_{10}$ distribution.

Your question is an interesting example for comparing the two methods because the methods give quite different results. In Table 4A, it seems like there is significant evidence that $3$ and $4$ are the most likely and and that $1$ is the least likely. However Table 4C suggests a completely different distribution: $5$ is the most likely and $4$ is the least likely. By adding the $chi^2$ statistics together we get a significant result but the large $chi^2$ statistics are suggested by different deviations from the uniform distribution. If we create the pooled frequency table then the distribution is more uniform - the $chi^2_5$ statistic is $7.7$, giving a p-value of $0.17$.

I've done some simulations of data generated by fair dice rolls: I generate 60 dice rolls in the first data set and 600 in the second. For each generated data sets I then computed the p-value in two ways

1) pooling the two data sets and then calculating a $chi^2_5$ statistic and p-value, and

2) calculating a $chi^2$ statistic for both data sets, adding them together, and calculating the p-value using the $chi^2_{10}$ distribution.

The joint p-values from the two is shown below.

You can see that it's possible for the added p-value to be very low while the pooled p-value is quite high - one point is an extreme version of the problem that you presented, with two data sets showing very different un-uniform distributions being merged together to form a uniform distribution. Because these deviations from the uniform distributions can cancel each other out, personally I would choose the method that you suggested - pooling the data first and then comparing it to a $chi^2_5$ distribution. Also, the $chi^2$ distribution approximation is more accurate for large data sets (Pearson's $chi^2$ statistic better approximates a true $chi^2$ distribution as the sample size grows).

Here is the code that I used to generate the plot:

set.seed(1)

n1 = 60

n2 = 600

N = 10000

pooledPvals = numeric(N)

addedPvals = numeric(N)



for (i in 1:N) {

  r1 = table(sample(6, n1, replace = TRUE))

  r2 = table(sample(6, n2, replace = TRUE))



  pooledPvals[i] = chisq.test(r1 + r2)$p.value

  addedPvals[i] = pchisq(chisq.test(r1)$statistic + chisq.test(r2)$statistic,

                         df = 10, lower.tail = FALSE)

}

plot(addedPvals, pooledPvals,

     xlab = "Method 2 p-value (added)", ylab = "Method 1 p-value (pooled)")