Chapter 16 Hypothesis Tests: Proportions

Interactive Example: Classical Hypothesis Test for a Proportion

In this interactive example, we examine the mechanics of a classical hypothesis test for a proportion. In the left panel, you can select a null hypothesis value \(\pi_0\) and a level of significance \(\alpha.\) The defaults are \(\pi_0=.5\) and \(\alpha=.05\), respectively. Below those sliders, you can then select a sample size \(N\) and a sample proportion \(p\). Normally, the sample size and sample proportion will be determined by the data you’re analyzing. For this exercise, think of these sliders as allowing you to ask “what if” questions. For example, “If my null hypothesis is \(\pi_0=.5\) and level of significance is \(\alpha=.05\), what would I conclude if my sample of \(N=200\) observations resulted in a sample proportion \(p=.55\)?”

As you change the values of any sliders, the graphs and the equations in the main panel will update. The left graph displays a Normal distribution for the sample proportion \(p\). The distribution is centered over the null hypothesized value \(\pi_0\). The red area in each tail is \(\alpha/2\) probability. The red area defines the rejection region. The blue dot is the value of the sample proportion \(p\) selected in the left panel slider.

Below the graphs, you’ll see the four main components of a classical hypothesis test.

State the null and alternative hypotheses.
Calculate the test statistic. Here, we use the \(Z\) transformation of our sample proportion \(p\).
Determine the critical value \(z_{crit}\) associated with \(\alpha/2\) probability in each tail of a N(0,1) distribution.
State the conclusion of the hypothesis test.

The right graph illustrates the components of the hypothesis test, but in terms of the \(Z\) statistic, rather than \(p\). The distribution here is centered over zero because \(Z \sim \mbox{N}(0,1)\). The value of the \(z\) statistic is shown as a blue dot. The \(\pm z_{crit}\) values define the rejection region and are shown in the graph as dark red lines and as numeric values.

Let’s look at four cases. First, suppose the null hypothesis is \(\pi_0=.5\) and you’ve chosen a level of significance of \(\alpha=.05\). What would you conclude if your sample of \(N=200\) observations resulted in a sample proportion of \(p=.55\)? Set the left-panel sliders to those values. The \(z\) value should be \(1.41\), the critical values defining the rejection region will be \(\pm 1.96\). And you should see that the blue dot in both graphs has moved to the right, but is not in the red (rejection) region. Since the test statistic \(z = 1.41\) is not in the rejection region, we fail to reject the null hypothesis at the \(\alpha=.05\) level of significance.

Consider the second case. Suppose everything is the same as in the first case, except now your sample produces a proportion of \(p=.58\). Change the corresponding slider to \(p=.58\). The \(z\) statistic is now 2.26, which is larger than the critical value, 1.96. Looking at the graph, you can see that the sample proportion (the blue dot) is in the red rejection region. In this case, we can reject the null hypothesis at the \(\alpha=.05\) level of significance.

What happens if we want a higher level of statistical significance? Again, keep everything as in the second case, but now let’s choose an \(\alpha=.02\) significance level. Although the \(z\) statistic has the same value, 2.26, the critical value for the rejection region has increased to \(z_{crit} = 2.33\). Where previously we rejected the null hypothesis at the \(\alpha=.05\) level, here we cannot reject the null hypothesis at the \(\alpha=.02\) level.

Finally, let’s assume the same null hypothesis, same \(\alpha=.02\) significance level, and same sample proportion \(p=.58\). However, now suppose we calculated that sample proportion based on a larger sample size. Set \(N=300\). The larger sample reduces the standard error of the proportion. In the left graph, that means the distribution of \(p\) has a smaller (or narrower) variance. In the right graph, the smaller standard error produces a larger \(z\) statistic value. In both graphs, the blue dot is now in the rejection region.

Try other scenarios — i.e., values of \(\pi_0\) and \(p\). Pay close attention to how changing the sample size \(N\) and the significance level \(\alpha\) changes the results.

Interactive Example: Level of Significance \(\alpha\) and Sample Size

This example makes a very simple point: \(\alpha\), the probability of Type I error, does not change with the sample size. The probability of Type I error is the probability of rejecting the null hypothesis \(H_0\), given that the null is true. The level of statistical significance, \(\alpha\), is something we choose when conducting a classical hypothesis test.

The graph in the main panel below shows two sampling distributions for the proportion \(p\), under the assumption that the null hypothesis \(\pi=\pi_0\) is true. The black density \(f(p_{N_1})\) is the sampling distribution for proportions based on samples of size \(N_1\). Similarly, the blue density \(f(p_{N_1})\) is the distribution for proportions based on samples of size \(N_2\). In the default setting, the null hypothesis is \(H_0: \,\, \pi = .5\), the level of statistical significance \(\alpha\) is set to .05, the first sample size \(N_1=100\), and the second sample size \(N_2=500\).

For both sample sizes, \(N_1\) and \(N_2\), the total probability in the shaded areas of the tails is \(\alpha\). If we increase \(N_2\), say to \(N_2=800,\) the total probability (i.e., the blue shaded areas) in the tails of \(f(p_{N_1})\) remains \(\alpha\).

Although increasing the sample size doesn’t affect \(\alpha\), it does affect the sampling distribution. As we saw with the Central Limit Theorem, as the sample size increases, the variance of the sample proportion \(p\) will decrease. When we have a larger sample, we are more likely (in repeated random sampling) to estimate a proportion that is closer to the population proportion than if we had used a smaller sample.

Interactive Example: Type I and Type II Error

This example allows you to explore Type I and Type II error. The graph below shows two densities. The density in black is a density based on the assumption that the null is true: \(\pi=\pi_0\). Type I error is the probability that we reject the null hypothesis when the null is in fact true. The shaded red areas under the black density represent the total probability of Type I error, which we also refer to as \(\alpha\), the level of significance.

Type II error is failing to reject the null hypothesis when it is actually false. In order to calculate a probability for this type of error, we need to assume a true population proportion \(\pi\). You can select a value of \(\pi\) in the lower part of the left-panel. Be careful to differentiate between the null hypothesis \(\pi_0\) and the population (or true) proportion \(\pi\). The blue shaded area represents the probability of Type II error, often represented as \(\beta\).

When conducting an hypothesis test for a proportion, the sample proportion is typically transformed to a \(Z\) statistic and critical values are calculated using a N(0,1) distribution. In this example, however, the distributions are shown without the transformation. Because of that, the lower \(L_{\pi_0,\alpha/2}\) and upper \(U_{\pi_0,\alpha/2}\) critical values are calculated (a) assuming the null \(\pi=\pi_0\) and (b) with an \(\alpha\) level of significance. The lower and upper critical values are displayed below the plot, as are the calculations for \(\alpha\) and \(\beta\).

In the default setting, the null hypothesis is that \(\pi = \pi_0=.5\), the level of significance is \(\alpha=.05\), the sample size is \(N=200\), and, unknown to the analyst or researcher, the population (or true) proportion is \(\pi=.6\). Take a second to look at the graph and equations. Again, the red shaded areas represent the total probability of Type I error, \(\alpha\), and the blue shaded area is \(\beta\), the probability of Type II error

There are two points you should try to understand in this example. First, what happens when we decrease \(\alpha\)? Move the slider to \(\alpha=.02\). Although \(\alpha\) decreased, notice that the probability of Type II error (the blue area) increased. Move the slider to \(\alpha=.01\) and you should again see that as we decrease the probability of Type I error, we increase the probability of Type II error.

The second takeaway concerns sample size. Return the slider for \(\alpha\) to \(.05\). The probability of Type II error, \(\beta\), should be about .187. Increase the sample size from \(N=200\) to \(N=300\). What happens? Increase it again to \(N=400\) and then to \(N=500\). As we increase the sample size, the probability of Type II error decreases.

Try other scenarios. For example, what if the true \(\pi\) is less than the null hypothesized \(\pi_0\)? Experiment until you feel comfortable with the concepts of Type I and Type II error.

Practice Session: Large-Sample Hypothesis Test for a Proportion

In this practice session, you will conduct a classical hypothesis test for a proportion. You are provided the sample proportion \(p\), the null hypothesized value \(\pi_0\), and the level of significance \(\alpha\). You will need to calculate the test statistic \(z\), find the critical value \(z_{crit}\) associated with the level of significance, and then select the conclusion: reject \(H_0\) or fail to reject \(H_0\).

Enter your answers in the appropriate boxes below and use the radio buttons to select the test conclusion. Then click Submit. You will be shown whether your answer is correct or not. Try four or five problems, or until you feel comfortable with hypothesis tests of proportions.