Hypothesis Testing

There are two types of errors: Type one error is when $\mathcal{H}_0$ is rejected when $\mathcal{H}_0$ is indeed true; Type two error is $\mathcal{H}_0$ is not rejected when $\mathcal{H}_1$ is indeed true.

Example: Suppose we have a random sample $X_1,\cdots,X_n \sim N(\mu,1)$, and $\mu = { 0,1}$. So we would like to test that $\mathcal{H}_0 : \mu = 0$ and $\mathcal{H}_1: \mu=1$.

To use NP lemma, we first construct the ratio:

Then the NP lemma says that we will reject $\mathcal{H}_0$ if $\displaystyle{\frac{p(\vec x,\mu=1)}{p(\vec x,\mu=0)}} > k$ for some $k$, so we solve for $e^{n\overline{X}_n - n/2}>k$, and we get $\overline{x}_n >k^* = \frac{\ln k}{n} +\frac{1}{2}$, and this is the rejection region. Now given significance level $\alpha$, the type one error is given by

Then we use the fact that $\displaystyle{\overline{X}_n \sim N\left(0,\frac{1}{n} \right)}$ (the case when $\mathcal{H}_0$ is true), and we have

and then we can refer to the normal table to solve for $k^*$. The same idea applies for type two error, where in this case we have

Also, we have a general formula to solve for hypothesis testing. We list some cases here:

• Testing the Mean in a (Asymptotically) Normal Sample with Known Variance

Suppose we know the random sample takes the form $N(\mu,\sigma^2)$ where $\sigma^2$ is known, and we wish to test:

and we first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if:

• Testing the Mean in a Normal Sample with Unknown Variance

Suppose we know the random sample takes the form $N(\mu,\sigma^2)$ where $\sigma^2$ is unknown, and we wish to test:

and we first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if:

• Testing the Variance in a Normal Sample with Unknown Mean and Variance

Suppose we know the random sample takes the form $N(\mu,\sigma^2)$ where $\mu,\sigma^2$ is unknown, and we wish to test:

and we first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if:

• Testing the Ratio in a Bernoulli Sample

Suppose we know the random sample takes the form $Bernoulli(\theta)$ where $\theta$ is unknown, and we wish to test:

and we first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if:

• Testing the Difference in Mean of Two Bernoulli Samples

Suppose we have two mutually independent large random samples $(\geq25)$ $X_1,\cdots,X_m \sim Bernoulli(\theta_1)$ and $Y_1,\cdots,Y_n \sim Bernoulli(\theta_2)$, and we wish to test

and we first compute the value under the null hypothesis:

If $D_0 = 0$, then

If $D_0 \neq 0$, then

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if

• Testing the Difference in Mean of Two Normal Samples with Known Variance

Suppose we have two mutually independent large samples $(>25)$ $X_1,\cdots,X_m \sim N(\mu_1,\sigma_1^2)$ and $Y_1,\cdots,Y_n \sim N(\mu,\sigma_2^2)$ where $\sigma_1,\sigma_2$ are known. We wish to test

We first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if

• Testing the Difference in Mean of Two Normal Samples with Unknown Variance

Suppose we have two mutually independent large samples $(>25)$ $X_1,\cdots,X_m \sim N(\mu_1,\sigma_1^2)$ and $Y_1,\cdots,Y_n \sim N(\mu,\sigma_2^2)$ where $\sigma_1,\sigma_2$ are unknown, but we have $\sigma_1^2 = \sigma_2^2$. We wish to test

We first compute the value under the null hypothesis:

and we will reject $\mathcal{H}_0$ at significance level $\alpha$ if

Now I have listed a bunch of exercises, do it carefully, and identify which of the above $7$ cases! The values of $z,t,\chi$ can all be found in the standard table.

A general approach is that, we always make null hypothesis to be simple, i.e it is equal to something. So $\mathcal{H}_0 : \mu=550$. Also we want to see if its the weight has been reduced, so we make the alternative hypothesis to be $\mathcal{H}_1 : \mu<550$. Then according to the table, here both $\mu,\sigma$ unknown, we compute the value under the null hypothesis $\mathcal{H}_0$ first:

now, in order to reject $\mathcal{H}0$ under level $\alpha$, we need $t < -t_{\alpha, n-1}$, which according to the $t$-table we have $t_{0.05,11} = 1.796$, where we have $-1.31049 > -1.796$, hence we do not reject $\mathcal{H}_0$, so at significance level $\alpha=0.05$, we do not see any evidence that the mean weight is less than $550$ pounds.

Here, we have a similar case as the previous exercise: We’re in a normal random sample with both $\mu$, $\sigma^2$ unknown. So let $\mathcal{H}_0 : \mu = 1000$ and $\mathcal{H}_1 := \mu \neq 1000$. So again we compute the value under $\mathcal{H}_0$, we have

now in order to reject $\mathcal{H}0$ under level $\alpha$, we need $\vert t\vert > t_{\alpha/2.n-1} $ and from the $t$-table we have $t_{\alpha/2 , n-1} = t_{0.025,20} = 2.086$, and since we do have $\vert t\vert = 2.153 > 2.086$, we will hence reject $\mathcal{H}_0$ and in favor of $\mathcal{H}_1$, that is, there is evidence that the mean water usage is different from $1$ million liter per day.

In this example, we have two mutually independent random samples, and we design $\mathcal{H}_0 : \mu_1 -\mu_2 = 0$ while $\mathcal{H}_1 : \mu_1-\mu_2 \neq 0$, and in this two samples we have $\sigma_1^2=\sigma_2^2$ unknown, so we first compute the value under the null hypothesis, which is

where we compute $\displaystyle{s_p = \sqrt{\frac{9\times0.017^2 + 12\times 0.006^2}{10+13-2}} = 0.012}$, and we have

while we will reject $\mathcal{H}0$ if $\vert t\vert >t_{\alpha/2, n+m-2} = t_{0.025,21 } = 2.08$, which we see that clearly we should reject $\mathcal{H}_0$, meaning that the amount of DDT in juveniles and nestlings are not the same.

In this example, we will first compute the value under the null hypothesis $\mathcal{H}_0$:

In this case we will reject $\mathcal{H}0$ if we have $\chi^2 > \chi^2{\alpha,n-1} = \chi^2{0.05,9} = 16.918$, where we see that we do not satisfy this condition, and hence we will not reject $\mathcal{H}0$, and we conclude that at significance level $\alpha=0.05$, we do not have much information that $\sigma^2>0.0002$.

In this example, we will test $\mathcal{H}_0 : \sigma^2 = 4$ and $\mathcal{H}_1:\sigma^2\neq 4$. Note that standard deviation is $\sigma$!!!. In this normal sample with both mean and variance unknown, we first compute the value under the null hypothesis $\mathcal{H}_0$:

Under this condition we will reject $\mathcal{H}0$ if

or

Here we have $\alpha=0.05, n-1=15$, and $\chi{0.025,15}^2 = 27.48839$ and $\chi{0.975,15}^2=6.26214$, and we see that $\chi^2$ do not satisfy any of these two conditions, so we will not reject $\mathcal{H}0$, at significance level $0.05$.

Here since we have a large sample so we could use a normal approximation for this sample. We will test $\mathcal{H}_0 : \mu = 15$ and $\mathcal{H}_1 : \mu>15$. We first compute the value under the null hypothesis:

We will reject $\mathcal{H}0$ if $t > t_{\alpha,n-1} = t_{0.01,45} = 2.326$, where we will reject $\mathcal{H}_0$ under this case, at a significance level $\alpha=0.01$.

This one is similiar to the one in exercise $26$.

Here, we have a Bernoulli random sample, and we test $\mathcal{H}_0 : p = 0.1$ and $\mathcal{H}_1:p>0.1$. We first compute the value under the null hypothesis:

and we will reject $\mathcal{H}0$ if $z > z_{\alpha} = z_{0.005} = 1.645$, which means we will reject $\mathcal{H}_0$ under significance level $\alpha=0.05$.

Here, we have two Bernoulli random samples, let $X_i$ be the sample treated with Lipitor and $Y_j$ be the sample treated with placebo. We have $X_1,\cdots,X_{94} \sim Bernoulli(\theta_1)$, $\theta_1 = 0.074468$ and $Y_1,\cdots,Y_{270} \sim Bernoulli(\theta_2), \theta_2 = 0.1$, we test $\mathcal{H}_0:\mu_1-\mu_2=0$, and $\mathcal{H}_1: \mu_1-\mu_2\neq 0$. We first compute the value under the null hypothesis:

and we get

We will reject $\mathcal{H}0$ if $\vert z\vert > z_{\alpha/2} = z_{0.025} = 1.96$, and hence we do not reject $\mathcal{H}_0$, under the significance level $\alpha=0.05$.

Recommended citation: Jiajun Zhang, (2025) Hypothesis Testing