What Is a P-Value? A Plain-Language Guide for Thesis Writers

Ask ten graduate students what a p-value means, and you will probably get ten different answers, most of them wrong. That is not a criticism. P-values are one of the most misunderstood ideas in all of statistics, and plenty of published researchers get them wrong too. But in a thesis defense, “the p-value is less than .05 so it’s significant” is not enough of an answer. Panels want to know whether you understand what that number actually says.

This guide explains p-values in plain language, with no formulas and no Greek letters, and shows you how to talk about them confidently in your results chapter and at your defense.

The One-Sentence Definition

A p-value tells you how surprising your data would be if there were actually no effect or no difference in the population.

That is it. A small p-value means your data would be very surprising in a world where nothing is going on. A large p-value means your data would not be surprising at all in that world.

An Everyday Analogy: The Suspicious Coin

Imagine a friend hands you a coin and says it is perfectly fair. You flip it 10 times and get 10 heads.

Could a fair coin do that? Technically yes, but it would happen only about 1 time in 1,000. That “1 in 1,000” is the p-value. It answers this question: “If the coin really were fair, how often would I see a result at least this extreme just by chance?”

Because the answer is “almost never,” you start doubting the claim that the coin is fair. You do not prove the coin is rigged. You conclude that “the coin is fair” is hard to believe given what you saw.

Now swap the coin for your thesis. The “fair coin” is the assumption of no effect (no difference between groups, no relationship between variables). Your data is the 10 flips. The p-value tells you how hard it is to believe the “no effect” story given your data.

Where the .05 Cutoff Comes From

Most theses use a significance level (alpha) of .05. If your p-value is below .05, you call the result “statistically significant.” If it is above .05, you call it “not statistically significant.”

Here is the part nobody tells students: .05 is a convention, not a law of nature. It became popular because early statisticians needed a practical rule of thumb, and .05 (roughly 1 chance in 20) felt reasonable. Some fields use .01 or even stricter cutoffs. A result with p = .049 is not meaningfully different from one with p = .051, even though one gets called “significant” and the other does not.

This is why many journals and statisticians now encourage reporting the exact p-value rather than simply writing “p < .05,” and interpreting it as a matter of degree rather than a pass or fail line.

What a P-Value Does NOT Tell You

This is where most mistakes happen. Here are the most common misinterpretations we see in thesis drafts.

“The p-value is the probability that my hypothesis is true.” It is not. The p-value is calculated assuming the null hypothesis (no effect) is true. It tells you how compatible your data is with that assumption, not how likely your own hypothesis is.

“A p-value of .03 means there is a 3% chance the result is due to chance.” This sounds right but is subtly wrong. The p-value is the probability of seeing data this extreme if there is no real effect. It is not the probability that your specific result is a fluke.

“A small p-value means a big or important effect.” No. P-values mix two things: how large the effect is and how large your sample is. With a very large sample, even a tiny, practically meaningless difference can produce p < .001. With a small sample, a large and genuinely important difference can fail to reach significance. This is why you should always report an effect size (such as Cohen’s d, eta squared, or R squared) alongside your p-value.

“A p-value above .05 proves there is no effect.” It does not. It only means your data did not provide strong enough evidence to rule out “no effect.” The effect might truly be absent, or your sample might have been too small to detect it. Absence of evidence is not evidence of absence.

“P = .000 means the probability is zero.” Software often displays .000 when the value is very small, but a p-value is never exactly zero. Write it as p < .001 in your thesis.

A Worked Example

Suppose you compare the exam scores of two groups of students: one taught with a new method and one taught the traditional way. The new-method group averages 82, and the traditional group averages 78. You run an independent samples t-test and get p = .02.

Here is how to read that:

• Null hypothesis: the two teaching methods produce the same average score in the population.
• What p = .02 says: if the two methods really were equal, you would see a gap of this size (or bigger) only about 2% of the time just from random sampling variation.
• Conclusion: because that is rare under the “no difference” assumption, you reject the null hypothesis at the .05 level and conclude there is statistically significant evidence of a difference.
• What you still need to ask: is a 4-point difference meaningful in practice? Check the effect size and the confidence interval before you call the new method a success.

A sentence you could actually use in your results chapter: “Students taught with the new method scored significantly higher (M = 82.0, SD = 6.4) than those taught traditionally (M = 78.0, SD = 6.1), t(58) = 2.41, p = .02, d = 0.63.”

How Sample Size Changes Everything

Because p-values depend heavily on sample size, the same difference can look “significant” or “not significant” depending on how many respondents you have. A 4-point difference with 20 students per group may give p = .20. The identical 4-point difference with 200 students per group may give p < .001.

That is not a flaw in your analysis. It is how the math works. It is also why your sample size justification (see our article on sample size calculation) matters so much, and why a “non-significant” result from a tiny sample should be described cautiously rather than treated as proof that nothing is happening.

Better Habits for Your Thesis

• Report exact p-values (p = .023) rather than only “p < .05,” except for very small values, which you write as p < .001.
• Always pair p-values with effect sizes so readers can judge practical importance.
• Add confidence intervals when you can. They show the range of plausible values for the effect, which is more informative than a single yes or no verdict.
• Avoid language like “highly significant” or “marginally significant.” Stick to “statistically significant” or “not statistically significant,” and describe the size and direction of the effect in words.
• Do not hunt for significance. Running many tests and reporting only the ones below .05 inflates your false-positive rate. Decide your analyses in advance and report all of them.
• Explain non-significant results honestly. “No significant difference was found” is a legitimate finding, not a failure. Discuss whether your sample size may have limited your power.

Questions Panels Often Ask

If you can answer these in your own words, you are in good shape:

1. “What does your p-value actually mean?”
2. “Why did you choose .05 as your significance level?”
3. “Your result is significant, but is it practically important? How do you know?”
4. “Your result is not significant. Does that mean there is no relationship?”
5. “Did your sample size affect this result?”

The Bottom Line

A p-value is a measure of how surprising your data would be if nothing were going on. It is a useful piece of evidence, but it is not a measure of truth, importance, or the probability that your hypothesis is correct. The strongest theses treat p-values as one part of the story, alongside effect sizes, confidence intervals, and sound reasoning about the research design.

If you want a second pair of eyes on how you are interpreting and reporting your results before your defense, that is exactly the kind of question StatsAce exists to help with.

---

Unsure how to interpret or report your statistical results? Reach out to StatsAce at statsace.com. We help Philippine graduate students and researchers present their findings clearly and defend them with confidence.