Significance Level (Alpha)
In short: Significance level, written alpha, is the threshold set before a study that says how small the evidence’s p-value has to be to reject the null, and it equals the Type I error rate.
AP StatisticsStatisticsAP Statistics Unit 3
What is Significance Level?
The significance level, \(\alpha\), states how much false-alarm risk you will tolerate. With \(\alpha = 0.05\), a null hypothesis that is actually true still gets wrongly rejected in about 5 of every 100 studies.
The rule is short: reject \(H_0\) when the p-value falls below \(\alpha\), otherwise fail to reject. Typical picks are 0.10, 0.05 or 0.01. Shrinking \(\alpha\) asks for stronger proof, so false alarms become rarer while real effects slip by more often. Lock in the number before opening the data.
How to spot it
- The problem says “at the 5% significance level” or gives \(\alpha = 0.01\).
- You must compare a p-value with a cutoff and write a decision.
- A critical value or a rejection region is drawn in the tail of a curve.
- The question asks about the probability of a Type I error.
The method
- Read or choose the significance level before collecting data, based on how costly a false alarm would be.
- Compute the p-value from the sample.
- Reject \(H_0\) if p-value \(< \alpha\); otherwise fail to reject \(H_0\).
- Write the conclusion in context and mention the significance level used.
Significance Level: solved example
Problem
A factory claims at most 4% of its gaskets are defective. A test of \(H_0: p = 0.04\) against \(H_a: p > 0.04\) gives \(z = 2.05\), so the p-value is 0.0202. What is the decision at \(\alpha = 0.05\) and at \(\alpha = 0.01\)?
Solution
At \(\alpha = 0.05\): \(0.0202 < 0.05\), so reject \(H_0\). There is convincing evidence that the defect rate exceeds 4%.
At \(\alpha = 0.01\): \(0.0202 > 0.01\), so fail to reject \(H_0\); the evidence is not strong enough for this stricter standard.
The same data lead to different decisions because the two levels set different amounts of tolerance for a false alarm.
Try it
Question 1
If a test is run at \(\alpha = 0.10\) and \(H_0\) is actually true, what is the probability of rejecting it?
Show the answer
0.10. The significance level is exactly the probability of a Type I error when the null hypothesis is true.
Question 2
A drug safety test uses \(\alpha = 0.01\) instead of 0.05. What is the trade-off?
Show the answer
False alarms become rarer, but a real effect is more likely to be missed (a Type II error), so power drops.
Significance Level vs. P-Value
The significance level is a fixed threshold set before the study. The p-value changes with every sample. Statements like “the p-value is 0.05” and “\(\alpha\) is 0.05” are not the same kind of claim: one summarizes this particular data set, the other describes the testing policy.
Keep going
Lessons
Worksheets and quizzes
Related words
- P-Value
- Hypothesis Testing
- Type I and Type II Errors
- Null and Alternative Hypotheses
- Confidence Interval
Whole dictionary: A–Z · AP Statistics · Statistics