An Introduction to p-value

Learning Objectives

1. Understand what a p-value is and how it is used in statistical analysis.

2. Know how to calculate a p-value and interpret the results.

3. Understand the limitations of p-values, including the fact that they do not provide a direct measure of effect size or the probability that the results are due to chance.

4. Be able to evaluate the appropriateness of using a p-value in a particular research study.

5. Be aware of the controversy surrounding the use of p-values and the ongoing debates about their appropriateness in scientific research.

6. Understand how to communicate the results of a statistical analysis that includes a p-value in a clear and accurate way.

P-value is a statistical measure that is widely used in the field of bioestatistics. It is a probability that represents the likelihood of obtaining a result that is at least as extreme as the one observed, if the null hypothesis (the hypothesis that there is no relationship between the variables being studied) is true.

In other words, the p-value helps researchers determine whether the results of their study are statistically significant or not. A p-value of less than 0.05 is generally considered to be statistically significant, meaning that there is a less than 5% chance that the observed results occurred by chance. On the other hand, a p-value of greater than 0.05 suggests that the observed results could have occurred by chance and are not statistically significant.

However, it is important to note that the p-value is not a direct measure of the size or strength of the relationship between the variables being studied. It only provides information about the statistical significance of the results. Therefore, it is important to interpret the p-value in the context of the study and to consider other factors, such as the sample size, the magnitude of the effect, and the relevance of the results to the research question.

In addition, it is important to note that the p-value should not be used as the sole criterion for determining whether a result is statistically significant or not. Other statistical measures, such as confidence intervals and effect sizes, can provide additional information about the strength and magnitude of the relationship between the variables being studied.

Overall, the p-value is a valuable tool for assessing the statistical significance of research results in the field of bioestatistics. It helps researchers determine whether their results are likely to be true or simply due to chance, and it is an important factor to consider when interpreting and communicating the results of a study.

Controversy

There has been a growing controversy surrounding the use of p-values in scientific research, as some argue that they can be misused or misinterpreted, leading to erroneous conclusions.

One of the main criticisms of p-values is that they can be easily misused to “cherry pick” results that support a particular hypothesis, while ignoring results that do not. This can lead to a biased interpretation of the data and a distorted view of the underlying phenomenon being studied.

Another concern is that p-values are often used as a “cut-off” point, with results that have a p-value of less than 0.05 being considered statistically significant and worthy of publication, while those with a p-value greater than 0.05 are deemed not statistically significant and are often ignored. This can lead to a “publication bias,” where studies with statistically significant results are more likely to be published, while those with non-significant results are less likely to be published. This can lead to an inflated view of the reliability of certain findings, as the published literature may not accurately reflect the true state of the underlying phenomena being studied.

There have been a number of proposals for alternative approaches to p-values, including the use of confidence intervals, Bayesian statistics, and other methods. Some argue that these approaches are more robust and less prone to misuse than p-values. However, there is ongoing debate about the appropriateness of these alternatives and how they should be implemented in scientific research.

An example using R

A researcher wanted to investigate if boys and girls differ on body mass index (BMI). She collected data on five boys (2) and five girls (1).

   bmi sex
1   32   1
2   33   1
3   34   1
4   32   1
5   30   1
6   29   2
7   30   2
8   33   2
9   27   2
10  28   2

Prior to running the Student’s t-test for independent means, I will check for the assumption of equal variances between the two groups.

This can be done with the var.test() function in R. With the variance test, we want the p-value to be above 0.05, which would indicate that the variance between the two groups is not statistically significant.

If the var.test() were found to be statistically significant, the Welch’s Two Sample t-test must be used in place of the Independent-Samples t-test.

var.test(bmi ~ sex)

    F test to compare two variances

data:  bmi by sex
F = 0.41509, num df = 4, denom df = 4, p-value = 0.4153
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
 0.0432186 3.9867860
sample estimates:
ratio of variances 
         0.4150943 

Since the calculated p-value is above 0.05 (0.4153), there is no evidence the variance between the two groups is different. We can now proceed to with the Independent-Samples t-test.

t.test(bmi ~ sex, var.equal=TRUE)

    Two Sample t-test

data:  bmi by sex
t = 2.2862, df = 8, p-value = 0.05157
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
 -0.02426674  5.62426674
sample estimates:
mean in group 1 mean in group 2 
           32.2            29.4 

To conclude, the Student’s Independent-Samples t-test was used to compare the BMI mean values between boys (M = 32.2) and girls (M = 29.4). No significant difference was found between the groups, t(8) = 2.2862, p-value > 0.05.