What question did the systematic review address?
The research question addressed by the researchers was “Is there a difference in effective BCTs associated with the initiation and maintenance of change?” (Samdal, Eide, Barth, Williams, & Meland, 2016, p. 2). The review addressed the question specifically; however, further development of focus was needed to define the target population. The two research objectives were clear and focused on the population, intervention, and outcomes of interest.
Is it likely that all relevant studies (published and unpublished) were identified?
The review includes an adequate description of eligibility criteria, search methods, and data extraction tools used in the gathering of material. The search for relevant studies was detailed: the authors describe using appropriate sources, such as MEDLINE, PsycInfo, and EMBASE, as well as university libraries and specific peer-review journals (Samdal et al., 2016), which is sufficient scope of the search for the studies in this case.
However, the search strategy was based on previous systematic reviews on the topic (Samdal et al., 2016). This is a limitation, particularly as there is no evidence of the authors assessing its effectiveness and reliability. Hence, there is a possibility that not all relevant studies were included in the analysis.
Were the criteria used to select articles for inclusion predetermined, clearly stated, and appropriate?
The eligibility criteria are outlined by the reviewers in a separate paragraph. The initial criteria used to filter the search results were: “published, peer-reviewed, randomized and cluster randomized controlled trials (RCTs) of behaviour change interventions providing baseline and/or follow-up data at minimum 12 weeks after randomization” (Samdal et al., 2016, p. 3). Further criteria included interventions that promoted diet and/or PA and applied behavior- and/or cognitive behavioral strategies (Samdal et al., 2016). There were also criteria regarding age and BMI of subjects, original language of the paper, and measured outcomes (Samdal et al., 2016).
Overall, the criteria are clearly stated and appropriate to the research goals. However, the review would further benefit from and explanations of why certain criteria were applied and how do the chosen criteria compare to the ones used in previous meta-studies.
Were the included studies sufficiently valid?
The criteria applied by the authors to the search for relevant studies ensured that all items were of appropriate scholarly quality and thus could be used in the review. The studies included both randomized and cluster randomized controlled trials; these types of trials are viewed as sufficiently valid for both large-scale and small-scale studies of medical interventions. Furthermore, the use of cluster randomized controlled trials means that a higher number of similar results had to be produced to have statistical power, which adds value to the individual studies included.
Nevertheless, the reviewers do not describe whether or not the methodological quality of the studies was thoroughly assessed, which is a major drawback of the review. No indication of subject follow-ups is given.
Were studies selected and data extracted by 2 or more individuals?
Yes, the selection of studies was carried out by two individuals: Gro Beate Samdal and Eivind Meland (Samdal et al., 2016). The authors mention that there was a high degree of agreement between the two researchers: “There was an 85% agreement whether to 1) include, 2) exclude or 3) carry out a full text evaluation” (Samdal et al., 2016, p. 3). Moreover, there was also a set procedure for disagreement, which was consistent throughout the search and analysis – all disputes between the researchers during the search or coding were to be set by group discussion (Samdal et al., 2016, p. 3).
Were the results similar from study to study? Reasons for the heterogeneity explored?
The results yielded by the analysis are presented in forest plots, which allows us to see the differences and similarities in results. Overall, the results of the studies were relatively homogenous, with few exceptions where the results achieved were significantly different. The heterogeneity is more prominent in short-term results than in long-term effects of the interventions studied. The authors did not perform a chi-square test to estimate the differences.
However, they addressed the possible reasons for the variations in results: “In particular, several BCTs that facilitate self-regulation of behaviour explain intervention effects, e.g. the BCTs goalsetting of behaviour and self-monitoring of behaviour” (Samdal et al., 2016, p. 9). The authors also outline the particular BCTs that they believed to be accountable for the differences in ineffectiveness. For instance, facilitating self-regulation and sustained positive motivation, as well as MI, SDT and other autonomous-based interventions are believed to be more effective in the long term (Samdal et al., 2016).
Was the risk of bias adequately assessed?
The reviewers used independent assessment and coding to determine the risk of bias. There is no mention of likelihood ratio analysis being applied to the range of possible test results, as the studies with a high risk of bias were not excluded from the review. Several different interventions were studied in the review, however, the overall estimated effect sizes were modest (0.19-0.41).
Was a meta-analysis used?
The reviewers used meta-analysis, as well as forest plots, to examine the results obtained by the studies. The results are relatively precise, with appropriate estimates used to determine the overall effects of interventions in the individual studies. The CI was 95% for both long-term and short-term results, which is a satisfactory value.
Are the populations studied similar to my targeted population?
Were all important outcomes considered?
The authors do not provide a rationale for the choice of target population criteria; however, the results of the study can be applied to the general adult population struggling with obesity, as the BMI value satisfies the diagnostic requirement. The main outcomes of the study for further obesity interventions are discussed, although briefly.
Are the benefits worth the costs and potential risks?
The behavior change techniques (BCTs) are generally regarded to be low-risk interventions, which is why the benefits outweigh their costs and potential risks. The authors do acknowledge some of the risks related to the wrongful application of these techniques in counseling, however, the detailed risk and benefits analysis was not completed.
References
Samdal, G., Eide, G., Barth, T., Williams, G., & Meland, E. (2016). Effective behaviour change techniques for physical activity and healthy eating in overweight and obese adults; Systematic review and meta-regression analyses. International Journal of Behavioral Nutrition and Physical Activity, 14(42), 1-14.