The article selected for the appraisal investigates the benefits of web-based motivational interventions for the reduction of adolescent substance use. The results of the randomized control trial (RCT) conducted by Arnaud et al. (2016) suggest that the intervention group demonstrated a considerable reduction in drinking frequency and quantity compared to the control group. The following JBI critical appraisal demonstrates that the article is not appropriate for making a clinically-based decision due to its limitations.
The main weakness of the study is the lack of a blinding procedure ensuring the internal validity of the results. Elimination of bias via blinding is the component of the JBI checklist, but the study’s participants “were not blinded to random allocation” and were aware of their assigned group (Arnaud et al., 2016, p. 5). The three-fold strategy, including printed promotional materials and an open-access landing page, was used to avoid bias when recruiting participants from diverse backgrounds (Arnaud et al., 2016). The minimization of selection bias contributes to the article’s strength, but the lack of a single- or double-blinded protocol involves the risk of detection bias leading to outcome measurement errors.
Another limitation of the study is the high dropout rate for the planned follow-up assessment. Follow-up assessment is an essential component of the critical appraisal tool, and the unexpected decrease in the number of participants impacted the validity of the findings. The authors admit that “any approach to missing data imputation such as the EM method employed in this study could be compromised” as a result of significant dropout rates (Arnaud et al., 2016, p. 16). The follow-up differences between the groups were not adequately described, lost statistical power, and increased the risk of attrition bias. Therefore, the study’s failure to meet the critical components of the JBI tool means that the article cannot be used for clinical decision-making.
Follow-up Questions
- The appraisal of the study identified the risk of detection bias, which might complicate the use of the article for clinical decision-making. What is the impact of detection bias on the validity of the results?
- The authors of the article reported significant dropout rates during the planned follow-up assessment. How could the researchers prevent the unexpected decrease in the number of participants?
Reference
Arnaud, N., Baldus, C., Elgan, T. H., De Paepe, N., Tonnesen, H., Csemy, L., & Thomasius, R. (2016). Effectiveness of a web-Based screening and fully automated brief motivational intervention for adolescent substance use: A randomized controlled trial. Journal of Medical Internet Research, 18(5), 1–22.