What is a p-value?
A p-value is a number that helps you judge whether you saw the results you did by luck, assuming a standard rule used in statistics called the null hypothesis.
Step-by-step intuition
- Set up a null hypothesis: You start with a default idea you try to test. For example, a coin is fair (50% heads).
- Collect data: Do an experiment or look at data. Suppose you flip the coin 20 times and get 16 heads.
- Ask how surprising the data is: If the coin is truly fair, getting 16 heads out of 20 is unusual but possible. The p-value answers: "How unusual would this be if there were no real effect (under the null hypothesis)?”
- Interpret the p-value:
- A small p-value (commonly ≤ 0.05) suggests your result would be unlikely if the null hypothesis were true. This is some evidence against the null.
- A large p-value suggests the observed result could easily happen by luck if the null were true. This is not strong evidence against the null.
- Important caveats:
- It does not prove the null is true or false.
- It depends on how many tests you run and what you consider “unusual.”
- It’s a tool for decision-making, not a final verdict.
Simple analogy
Imagine you have a deck of 52 cards. If someone tells you a card drawn is a King, a p-value would be a way to measure how surprising that result is if you had not looked at the card yet. A very surprising result would have a small p-value, suggesting something noteworthy happened.
If you’re ever unsure
Ask these questions: What is the null hypothesis? What would be considered a “surprising” result here? Is the p-value small enough to matter given how many tests I did?
In short: a p-value tells you how likely your data would be if there were no real effect. A small p-value hints at an effect, but it’s not a guarantee.