Introduction
The criteria for scientific research using quantitative methods are essential to conducting the objective investigation. The characteristics of the quantitative study are the validity and reliability of the obtained data. These aspects are relevant to the description of the criteria because they allow scholars to generate legitimate knowledge without mixing facts with their perception. Horne (2018) and Bhattacherjee (2012) discuss internal and external validity and reliability requirements in their texts. For example, Horne (2018) claims that there are no wholly controlled experiments in social sciences, but the level of valid and reliable information is still prevalent. The quasi-experimental design allows scholars to conduct a reliable investigation using the control groups and the experiment.
Validity and Reliability
The research paradigm is essential in choosing the validity and reliability criteria. For example, as Denzin and Lincoln (2018) claim in their investigation, the positivist and the postpositivist research supposes the appeal to external and internal validity. Constructivist research, in turn, focuses on such criteria as confirmability, transferability, credibility, and trustworthiness of the data (Denzin & Lincoln, 2018).
The feminist research paradigm is centered around such measures of the proper investigation as gender, accountability, dialogue, and living experience (Denzin & Lincoln, 2018). The queer theory features the criteria of deconstruction and reflexivity, while cultural studies emphasize the importance of such issues as subjectivity, praxis, and cultural practices (Denzin & Lincoln, 2018). Therefore, the criteria for reliability and validity depend significantly on the sphere of knowledge, and the choice of the subject determines the selection of the particular methods to test the reliability of the data and conclusions.
The formal quasi-experimental design in quantitative research supposes the division of the respondents into comparison groups, which allows scientists to achieve the criteria of validity and reliability. Among the types of comparisons used in this method of investigation are switching replications with nonequivalent groups and regression discontinuity that suppose the combination of reliable factual data and a certain amount of creative information obtained during the study.
It is possible to illustrate the division of the respondents into comparison groups using the following ideas articulated by Horne. He writes that the quasi-experimental design depends on the choice of the comparison group and the control group (Horne, 2018). Selecting the respondents based on similar cases from the experimental groups is necessary to construct a comparison group (Horne, 2018). For example, it is not right to compare drivers to children in the kindergarten classroom based on their response to pedagogical innovations.
Therefore, people from the comparison group should share the dominant characteristic that unites them. It is vital both for the natural and field experiments because it shows that the respondents in the comparison groups have common characteristics that apply to the research goals. Conducting the case study and driving the participants into the comparison groups are connected with the possibility of making overgeneralized and oversimplified conclusions (Harrison et al., 2017). It negatively affects the quantitative research results, doubting their validity and reliability. Horne (2018) writes about the importance of not ignoring complexities that arise during the experiment and case study to obtain objective information that can be used in subsequent research.
Data analysis is another vital component of achieving reliability and validity in quantitative research. It is not entirely correct to state that the quantitative data analysis focuses only on processing the numeric data, while the textual information is analyzed in the qualitative format. This claim is especially controversial in social sciences, where content analysis is among the most critical parts of the investigation and combines quantitative and qualitative research methods (Horne, 2018). Evaluating primary data reliably and validly is stable and predetermined, as scholars claim, making scientific investigation justified and foregrounded.
It is possible to describe the significant steps in quantitative data analysis using the pattern described in the article by Horne. He writes that if the researchers evaluate the notion similar to political literacy, they need to operationalize this concept. It can be done by listing the quiz items, gradually collecting the required data, and analyzing the respondents’ scores (Horne, 2018). These characteristics reflect the respondents’ political literacy level and can be used for the subsequent quantitative analysis. Therefore, the first step is conceptualization, and the second is the notion’s analysis (Horne, 2018). It is possible to interview to understand the respondents’ thoughts concerning political literacy, and the identification of the concepts will follow this step of conceptualizing data.
Therefore, conceptualization and analysis are inseparable, and scholars start from the moment of gathering data collection. Quantitative data makes the research substantial and reliable, adding numerals and objective information to the analysis. At the same time, articulating the concepts that require the support of quantitative data is the result of qualitative research. It allows us to hypothesize that using qualitative and quantitative information is a simultaneous process, and it is impossible to use them separately.
The appropriate generalization of the obtained quantitative information is the vital component of the research that makes it valid and reliable. For instance, the collected data should feature an evident connection to the community group, demographic group, community in general, or professional union. In other words, the numerical information is meaningless without the clearly articulated research question that the quantitative data supports because it does not endorse any hypothesis (Horne, 2018). For this reason, it is especially critical to generalize the information the scholar obtains to ensure that it corresponds to the topic and supports the research question.
Evaluation of the validity and reliability of the data is the last component of measuring the quantitative data. As Horne (2018) claims, there are two levels of evaluation in the research design. The first level supposes the general description of the invention and operationalizing the variables and the concepts used in the study. These holistic claims concerning the value of the investigation and its possible application in practice can be made in all cases. The second level is the evaluation of the study’s internal validity, which focuses on the truthfulness of the claims and observations that the research features. It is necessary to appeal to the causal claims or interferences while evaluating the study’s internal validity. These characteristics show that the choice of the control and comparison groups is adequate, the data obtained from them is reliable, and the experiment is invalid in general.
In general, the concept of reliability shows whether the scholar would have the same investigation results if other researchers repeated the experiment. Sometimes, the researchers evaluate the reliability of their experiments using theoretic conclusions, especially in social sciences. For instance, face validity determines whether the measures used in the investigation seem valid and reliable. Content or construct validity, in turn, appeals to the overall analysis of the information presented in the study.
For example, when the scholar applies the conclusions to the claims about the more general demographic group, these claims should be supported by the shared background concerning these issues. These types of validity measurements are informal and intuitive. Operational validity, in its turn, is the formal way to assess the obtained data, and it supposes the comparison of the used variables with the research goal (Horne, 2018). Therefore, both formal and informal methods of evaluating validity and reliability are usually applied in the study evaluation to test its scholarly merit.
From the formal point of view, these should be scales to measure the quantitative data’s reliability and validity. As Bhattacherjee (2012) writes, the concepts in social sciences have psychometric properties that determine whether they accurately represent the described situation or context. The peculiar issue is that a measure can be not valid but reliable simultaneously, which means that the scientist uses the initially wrong concept to experiment, but from the proper perspective, their evaluation is adequate. The opposite situation is widespread when the scholar uses the appropriate construct, but the way they measure it is wrong and does not correspond to the scientific investigation (Bhattacherjee, 2012). For this reason, it is equally essential to ensure that the concepts are valid and that the methods of their measurement and evaluation are reliable from the beginning of the experiment.
It is possible to create reliable measures using several scientific methods, including inter-rater reliability, test-retest reliability, and split-half reliability. Bhattacherjee (2012) claims that inter-rater reliability features the consistency between several independent observers who study the same concept. The researchers should unite their efforts to determine reliable ideas during the pilot study to ensure the subsequent validity and reliability of the results (Bhattacherjee, 2012). Test-retest reliability means that the results obtained concerning the same variable are checked several times during the study to ensure that the initial conclusions are correct (Bhattacherjee, 2012). Split-half reliability supposes the division of the concept into two equal parts and measuring their consistency separately (Bhattacherjee, 2012). The results of both pieces should show the same results in a reliable and valid investigation.
Conclusion
Summing up, there are various types of validity and reliability, as the discussion shows. The researchers need to avoid controversial personal opinions in evaluating these criteria because the study’s author can make significant mistakes. It is especially critical when the concepts used in the research are measured using reliable methods of scientific inquiry, but they are not valid by their essence, and vice versa. Therefore, it is necessary to recheck the concepts and the measurements used in the study and to ask for their objective evaluation to avoid possible errors. As the discussion shows, quasi-experimental research is one of the most famous examples of quantitative analysis in educational discourse. It allows scholars to obtain numerical data from the comparison groups to apply these claims to more generalized populations and the educational process.
Reference List
Bhattacherjee, A. (2012) ‘Social science research: Principles, methods, and practices’, Textbooks Collection, 3. pp. 1-159.
Denzin, N. K., Lincoln, Y. S. (2018) The SAGE handbook of qualitative research. Thousand Oaks, CA: Sage.
Harrison, H., Birks, M., Franklin, R. and Mills, J. (2017) ‘Case study research: Foundations and methodological orientations”, Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 18(1), pp. 1-17.
Horne, C. S. (2018) A quick, free, somewhat easy-to-read introduction to empirical social science research methods. Open Educational Resources.