Topic 3.6 Notes – P-Values

What a P-Value Is

In a one-sample test for a population proportion, your hypotheses look like

• $H_0:p=p_0$
• $H_a:p>p_0$, $H_a:p<p_0$, or $H_a:p\ne p_0$

The p-value is the probability, assuming $H_0$ is true, of getting a test statistic as extreme or more extreme than the one you got.

A few parts of that definition matter a lot:

• Assuming $H_0$ is true means you pretend the true proportion really is $p_0$.
• Observed test statistic means the $z$-value from your actual sample. Once data are collected, that number is fixed.
• Null distribution means the distribution of possible test statistics you could get from many random samples if $H_0$ were true. For a one-proportion $z$-test, this is usually the standard normal distribution.

“Extreme” depends on the alternative hypothesis. In the picture here, the shaded left-tail area represents the p-value for a test with $H_a:p<p_0$.

Left-tail p-value on the null distribution

Finding the P-Value from the Null Distribution

For a one-proportion test, the observed test statistic is

$$z=\frac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}$$

You use $p_0$ in the denominator because the whole calculation is built under the null assumption.

Which tail to use

• If $H_a:p>p_0$, the p-value is the right-tail area
  $P(Z\ge z_{obs})$

• If $H_a:p<p_0$, the p-value is the left-tail area
  $P(Z\le z_{obs})$

• If $H_a:p\ne p_0$, use both tails
  $$P(Z\le -|z_{obs}|)+P(Z\ge |z_{obs}|)$$ By symmetry, this is also
  $$2P(Z\ge |z_{obs}|)$$

That absolute value is a common trap. If $z_{obs}=-2.1$, you double the area beyond $2.1$, not the huge left-tail complement of the wrong side.

Conditions for the normal null model

You need:

• a random sample or random assignment
• independence, plus the 10\% condition if sampling without replacement
• large counts under the null, so $np_0\ge10$ and $n(1-p_0)\ge10$

P-Values from Simulation

Sometimes the null distribution is simulated instead of using the normal model. Then the p-value is just the proportion of simulated statistics that are as extreme or more extreme than the observed one.

• $H_a:p>p_0$ → proportion at or above $z_{obs}$
• $H_a:p<p_0$ → proportion at or below $z_{obs}$
• $H_a:p\ne p_0$ → proportion at or below $-|z_{obs}|$ plus proportion at or above $|z_{obs}|$

In the simulated null distribution below, the observed statistic is $z=1.8$, so for a right-tailed test you count the simulated values at or above 1.8. That highlighted right-tail proportion is the simulation p-value.

Simulation p-values should be close to the model-based p-values, but not identical. Simulation has random variation.

Interpreting the P-Value and Using It as Evidence

A correct interpretation must be in context and must begin with the null assumption.

Example wording:

• “Assuming the true proportion of commuters who use public transportation is 0.40, the probability of getting a sample proportion this high or higher, just by random sampling, is 0.0418.”

That works because it includes

• the null assumption in words
• the variable and population in context
• “as extreme or more extreme”
• the correct direction from the alternative

What the size means:

• Small p-value → the result would be unusual if $H_0$ were true, so there is evidence for $H_a$
• Smaller p-value → stronger evidence against $H_0$
• Large p-value → the result is not unusual under $H_0$, so there is not convincing evidence for $H_a$

A large p-value does not prove $H_0$ is true.

Common Mistakes with P-Values

These show up constantly on tests and FRQs:

• Saying the p-value is the probability that $H_0$ is true
• Forgetting the phrase assuming $H_0$ is true
• Picking tails based on the sample result instead of the stated alternative
• For two-sided tests, doubling without using $|z_{obs}|$
• Saying “probability of getting exactly this sample” instead of “as extreme or more extreme”
• Treating statistical significance as practical importance
• Ignoring bad design or failed conditions. A tiny p-value cannot rescue biased data.