Topic 6.7 Notes – Potential Errors When Performing Tests

What Type I and Type II Errors Are

Every significance test has two parts:

• Reality: Is $H_0$ actually true or false?
• Decision: Do we reject $H_0$ or fail to reject it?

That gives four possible outcomes:

Lock in the definitions:

• Type I Error (false positive)
  You reject $H_0$ even though it’s true.
  You claim there’s an effect when there isn’t one.

• Type II Error (false negative)
  You fail to reject $H_0$ even though it’s false.
  You miss a real effect.

Students mix these up because they forget what must be assumed.

• Type I → assume $H_0$ is true.
• Type II → assume $H_0$ is false.

If a question says, “Suppose the company’s claim is actually correct…” that’s your clue $H_0$ is true.

Alpha, Beta, and Power

Significance level $\alpha$

$$\alpha = P(\text{Type I error})$$

• Set before collecting data.
• Common values: 0.05 or 0.01.
• If $\alpha = 0.05$, there’s a 5% chance we reject a true $H_0$.

So when you choose a smaller $\alpha$, you’re being more cautious about false positives.

Beta $\beta$

$$\beta = P(\text{Type II error})$$

This is the chance you miss a real difference.

Unlike $\alpha$, we usually don’t pick $\beta$ directly. It depends on things like sample size and effect size.

Power

Power tells you how good the test is at catching a false null.

$$\text{Power} = P(\text{reject } H_0 \text{ when } H_0 \text{ is false})$$

And the key relationship:

$$\text{Power} = 1 - \beta$$

High power means you’re unlikely to miss a real effect.

Here’s the tradeoff that always matters:

• Decrease $\alpha$ → fewer Type I errors
• But $\beta$ increases → power decreases

You can’t shrink both errors just by changing $\alpha$. That tradeoff shows up a lot in FRQs.

What Affects Type II Error and Power

Assume everything else stays the same. The probability of a Type II error decreases (so power increases) when:

• Sample size increases
  Larger $n$ → smaller standard error → test statistic moves farther from 0 if a real effect exists.
  This is the most common correct answer when asked how to increase power.

• Significance level $\alpha$ increases
  Easier to reject $H_0$.
  Lowers $\beta$, but raises Type I risk.

• Standard error decreases
  Less variability makes real effects easier to detect.

• True parameter is farther from the null value
  Bigger actual difference → easier to spot.

Think of it as signal vs noise. More signal or less noise means more power.

Interpreting Errors in Context

On quizzes and the AP exam, definitions alone are not enough. You must describe errors in context.

Suppose a school tests
$H_0: p = 0.40$ (40% of students prefer online classes)
$H_a: p > 0.40$

Type I Error in context

Assume $H_0$ is true (really 40%).
You reject it.

“In concluding that more than 40% of students prefer online classes when in reality only 40% do.”

Notice:

• Mentions rejecting
• States the incorrect real-world conclusion

Type II Error in context

Assume $H_0$ is false (true proportion is greater than 40%).
You fail to reject it.

“In failing to conclude that more than 40% of students prefer online classes when in reality the true proportion is greater than 40%.”

You must name the variable and situation. Generic answers lose points.

Which error is worse?

It depends on context.

• Medical screening: Missing a disease (Type II) could be worse.
• Criminal trial: Convicting an innocent person (Type I) could be worse.
• Manufacturing: Missing defective products (Type II) might be worse.

Because $\alpha$ controls the probability of a Type I error, serious Type I consequences → choose a smaller $\alpha$.

That decision is practical, not mathematical.