Introduction
Statistical significance relates to the null hypothesis’ determination, which postulates that outcomes result from chance. According to Benjamin et al. (2018), a hypothesis is a presupposition or belief about the relationship between one’s data set. On the other hand, the null hypothesis, commonly denoted as H0, relates to a default assumption that no difference or variation exists between two phenomena being evaluated. In research, this concept is often identified as the computation of the null hypothesis’ likelihood of being factual in comparison to the established degree of its improbability of being true. The paper provides a comprehensive discussion of statistical significance, including its meaningfulness.
Level and Tests of Significance
The level of statistical significance is associated with definite standard values, such as 0.01 and 0.05. A 0.01 significance level implies a 1% probability of the null hypothesis being declined when it is factual after a given analysis. A 0.05 level, on the other hand, indicates that the probability of repudiating the null hypothesis when the null is true is 5% (Salkind, 2017). This conceptualization is connected with every null hypothesis’ independent test.
Tests of significance can be applied to non-identical circumstances, depending on the research question and null hypothesis structure. Examples of these questions include “do you want to look at the difference between home-schooled children scores and that of their counterparts?” Or “do you need to assess the correlation between two variables: the availability of tutors and students’ academic performance?” The two instances infer non-identical approaches but will result in a null hypothesis evaluation using particular statistical significance tests. Therefore, this concept is established on the actuality that each form of null hypothesis has a specific type of statistic.
The following are the general phases involved in statistical test application to a null hypothesis:
- Null hypothesis statement: One must have a starting point – the concept that the researcher intends to test.
- Developing the risk’s extent, type 1 error, or significance level related to the null hypothesis: Every postulation in a survey is associated with a particular magnitude of risk linked to Type 1 error. The lower the inaccuracy, the minimal the risk one is willing to assume. No hypothesis test is entirely risk-free due to the unknown nature of the interrelation between variables.
- Choosing the most effective test statistic: Every null hypothesis is linked to a specific test statistic.
- Analysis of the test statistic’s value – commonly referred to as the obtained value: It is typically identified as an outcome of a given statistical test.
- Assessment of the integer required for declining the null hypothesis utilizing the proper critical value table for the specific statistic: Every statistical analysis, together with the assumed risks and the selected population size, has a critical value attached to it. If the null hypothesis is erroneous, the test statistic is likely to generate this minimal value.
- Contrasting between the critical and obtained value: This phase involves comparing the figure generated from the computed test statistic with the critical value one would hope to achieve by chance only (Salkind, 2017).
Whenever the obtained value is substantially higher in comparison to the critical one, the null hypothesis is rendered unacceptable. On the contrary, the null hypothesis may be deemed as the most suitable elucidation in cases where the critical value is higher than the obtained one. If one can account for the difference acquired, then the origin of this variance is linked to chance or an uncontrollable element.
How Results of a Study Can Be Significant but Not Meaningful
Statistical significance typically demonstrates the effect of a chance of a survey’s findings; it is based on the conceptualization of objective probability. Meaningfulness, on the other hand, highlights the value of the results – it underscores the level to which the relationships and differences reported in research are pertinent to the measured concept. For instance, in an intervention survey conducted by Riemann and Lininger (2015), the outcomes attained a statistical significance P value of.047, i.e., P =.047; nonetheless, the variation in the improvement of range-of-motion – a variable in the study – was recorded at 1.10. From this example, the meaningfulness of the outcome relies on whether a 1.10 increase in motion range complements the added time used to conduct the myofascial release before stretching, instead of the attained P-value – the relationship between variables.
When establishing the meaningfulness of a survey’s findings, it is crucial to determine the assessment tools’ reliability and the category of candidates involved in the research. According to Riemann and Lininger (2015), unreliable approaches to evaluating the measures of an outcome usually decrease the probability of achieving a finding’s efficacy, irrespective of the attained P-value. Furthermore, results from a study involving a broader sample size may not be generalized in other populations due to the likelihood of yielding minimal effect sizes – the statistical conception used to analyze the strength of the interrelation between two elements on a numeric scale. Its analysis may reveal a statistical significance level, but trivial relationships may exist, thereby undermining its meaningfulness. Salkind (2017) also supports this argument by indicating that the “usefulness” of statistical significance relies on a research’s conceptual base. Such approaches as the effect size and confidence levels can also enhance meaningfulness. From the above analysis, the study finding’s usefulness depends on factors other than statistical significance.
Conclusion
Statistical significance is a great tool in legitimizing experiments and studies. However, the data used in its analysis may be inaccurate or non-impartial since respondents in a survey could present false or incorrect information. Therefore, when assessing critical concepts, statistical significance in its entirety should not be used as a basis for decision making. Study outcomes can be statistically significant but lack meaningfulness concerning their applicability or usefulness.
References
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E. J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., Cesarini, D., Chambers, C. D., Clyde, M., Cook, T. D., De Boeck, P., Dienes, Z., Dreber, A., Easwaran, K., Efferson, C.,… Johnson, V. E. (2018). Redefine statistical significance. Nature Human Behaviour, 2(1), 6-10.
Riemann, L. B., & Lininger, M. (2015). Statistical primer for athletic trainers: The difference between statistical and clinical meaningfulness. Journal of Athletic Training, 50(12), 1223-1225.
Salkind, N. J. (2017). Exploring research (9th ed.). Pearson Education UK.