Program Background
The New Moves is a program designed to help adolescent girls to adopt healthy lifestyles. The program is designed to promote a healthy lifestyle through the development of helpful habits in the target populations. The majority of the focus students are from low-income families since these individuals tend to face the problem of excessive weight more often. The program started in 1996 in St. Paul Minnesota School District. In 2000 to 2001, an experimental study to assess the feasibility of implementing the program was conducted. The positive results of the study led to first implementation of New Moves program in 2005. The main purpose of the program is to address the psychological issues that obese girls encounter, such as low self-esteem, social isolation, stress, anxieties and depression (Neumark-Sztainer, 2005). In the current program, the stakeholders include the medical professionals, researchers, educators, the government, parents, and the obese girls.
The targeted population is 356 girls aged between 11and 18. The funders include National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and National Center for Research Resources (NCRR) in conjunction with the ministry of health nutrition department. The program is administered during the physical education classes and should be carried out for nine months. There are individual coaching sessions that normally take 15-20 minutes; In addition to support nutrition classes. The program is based on a logic model that integrates trainers, food promotion, information about healthy lifestyle, physical activities, equipment, intellectual and capacity building processes.
The intended outcome is to make the girls fit, look good and motivated to continue with healthy lifestyles. This is achieved by increased physical exercise, counseling and motivational talks. The Heath Belief Model (HBM) forms the theoretical concept for the program. The theory stipulates that health behavior is determined by a personal belief and the strategies that are adopted to reduce the health risks or disease occurrence. For instance, in the New Moves the concept enhances peer support, goal setting and personal factors such as body image. This is in line with extensive formative and summative researches carried out in accordance with Social Cognitive Theory that have shown that the program improves the socio-environmental factors which help girls to adopt healthy lifestyles (Neumark-Sztainer et al., 2010). These changes results in increased physical activities, healthy eating and desired weight control behavior. Figure 1 below is the logic model for the program. Earlier evaluation studies carried out about New Moves rated it as an effective intervention for obese girls (Msila & Setlhako, 2013).
Evaluability Assessment
Based on the program theory, secondary data analysis, and interviews with key participants such as the health instructors, the program has been progressing as planned. The implementation of a program includes exploration, installation, initial implementation, full implementation and program sustainability (Msila&Setlhako, 2013). The New Moves program is in the sustainability stage, as the program has already been fully implemented.Sustainability of the program depends on the ability of the program to adapt to changes and priorities, and to attract continued funding.
The stability of the program is strengthened by the intended outcome. New Moves is designed in a participatory approach to help the girls to adopt enjoyable physical activities they can engage outside the class, and provides a supportive environment. The participatory nature of the program and the anticipated health needs contributes to its stability and reduces attrition rates. The New Moves has been received well by the girls, teachers, and parents. The girls are not coerced to participate in the program. In addition, the program has led to weight reduction among girls and boosted the girls’ confidence (Stame, 2004). The positive results achieved justify the costs for the evaluation to help program implementers determine the factors that contribute to success or failure (Stame, 2004). The data sources included primary sources, such as quantitative data obtained by measuring the weight of the participants on a weekly basis, and qualitative interviews. In addition, secondary sources of data were used, such as information on how similar programs have performed.
Purpose of Evaluation
The evaluation is intended to determine the impact of the program on the girl participants. It will help identify how the girls have responded to the program in terms of weight management practices and lifestyle changes. According to Fitzpatrick, Worthenand Sanders (2003), the findings of an evaluation help in establishing “what works” and “what does not work”. Thus, the findings will be used to establish the benefits of the program. The evaluation will also determine the training skills among the staff and their ability to deliver the services. The results will be applied to showcase the effectiveness of the program.
The parties interested in the results include the funders, the government agencies in charge of planning, the parents, civil society, the program implementers and United Nations agencies that deal with health and nutrition. In addition, the participating girls will need to know the progress of the program. Evaluations serve as tools to improve programs (Van & Brown, 1997). Thus, the information obtained can be used to streamline the future implementation of the program.
The Evaluand and Evaluation Purpose Statement
In the healthcare setting, evaluation is the critical assessment and judgment to determine effectiveness of a project. The design of a good evaluation should start with the identification of the priorities and information needs of the key stakeholders (Msila &Setlhako, 2013). The evaluation will be used to appraise the process or recruiting the participants to the program, the number of participants, the extent the participants have achieved the set goals, the effects of the program on the status of obesity, behavioral, knowledge, and lifestyle changes.
Evaluation Questions
The evaluation questions provide the specifications that help in the determination of the factors that are used to judge the progress of the program. They were decided based on the preferences of the stakeholders, usefulness and the uncertainties in the program implementation.
- How many participants were recruited into the program?
The first question is a program process question. In the logic model, the question can be answered by the boxes for people and funders. This question is used in order to examine the stakeholders in the program and expected operation of the program. - To what extend did the program result in the expected changes?
The second question relates to the expected outcomes (program outcome). This question is used to probe the contribution of the program in changing the knowledge, behavior, health and the general practices of the students. Thus, it links the outcome to the expectation of the stakeholders. Therefore, in the logic model, this question relates to various outcomes such as gaining knowledge about healthy lifestyles and the subsequent arrows that lead to decreased rate of students’ obesity. The other three questions are about the factors that facilitate the change and how they interlink to produce change. - To what extent did the program decrease obesity?
The third question attributes the outcomes to the program. This is critical in ascertaining that the changes obtained did not result from other external factors. The outcome can be ascertained by measuring the body weight and body mass index (BMI). - Which behavioral changes can be attributed to the New Moves program?
The fourth question links the program process and the outcome. The question assesses how the programs processes lead to the expected change in lifestyle. The question probes the behavioral changes that can be attributed to the New Moves Program. Therefore, to get the answers to the questions, four parts of data will be measured. This will include multiple comparison groups, attendance of each participant’s activities and doses, frequency of attending the program and the log of doses for the three activities, self-report assessment. - To what extent does gaining knowledge about healthy lifestyle help to effect positive change?
The fifth question links the program activities to the outcome. The question is designed to examine specifically how the sessions contribute to the behavioral change. This is best ascertained through the indicators of lifestyle changes. The indicators for a healthy lifestyle include healthy eating habits, the level of physical exercise and self-esteem. In this case, self-reporting will be used as the strategy for data collection. Table 1 is a summary of the questions and data collection methods.
Appropriateness of the Questions
The process/outcome questions are the most appropriate for both the formative and summative evaluation. According to Weiss (1997), the questions are normally framed based on the feasibility of the program. This helps in determining whether the program is within the scope in terms of budget allocations and time. On the other hand, the outcome evaluation questions are used to investigate the consequences of the intervention. The outcome evaluation explores whether the intervention program has led to positive changes as outlined in the goals and objectives of the program. In addition the outcome evaluation questions are used to check whether the program is operating within the set timelines and budget. For example, assuming that the New Moves Program requires a budget of $10,000 to be used in program execution for a period of nine months, the process and outcome questions will be used to audit whether the funds were used for the planned activities. Secondly, the questions will investigate whether the program processes and outcomes are being achieved within the set times.
Evaluation Design and Methods
In order to answer the questions, both the quantitative and qualitative approaches will be used. In the case of qualitative approaches, the participants will be interviewed in order to determine their perceptions and attitudes about the program. On the other hand, the quantitative approaches will entail the use of measurement tools such as weighing scales and tape measures to determine the and height of the students. The numerical data obtained will be used to calculate the BMI of the students and to determine the trends in weight gain or loss. Therefore, the quantitative approach will be used to determine the weight changes over the nine months.
The evaluation design that that will be used is experimental design. Adolescent girls who are obese will be recruited and divided into two groups. One group will be the experiment group and the other one a control group. The experiment group will be subjected to the New Moves activities such as the exercise regimes, counseling, dietary changes and motivation. The control group will not be involved in the listed activities. However, the girls will attend the normal P.E classes during the nine months of the program execution. At the end of the intervention, the results from the experiment group will be compared with the results from the control group.
Methods of Data Collection
The evaluation will rely on primary sources of data. The data collection will include individual interviews, use of structured questionnaires, observations, measurements and computation of BMI, self-assessment reports and action research. The evaluation will rely on both the primary and secondary sources. The reliance on the primary sources of data will ensure that the information collected directly relates to the program. For example the measuring the heights and weight to investigate the changes in weight will provide unbiased data. In addition, the observation of the behaviors of the girls will provide first hand information on whether the program is making any positive changes or not. The presence or absence of change will best be determined by the comparisons between the control and the experiment group.
Table 1 is a clarification of the data collection strategies in relation to the evaluation questions.
Table 1: Questions and Data Collection Strategy
The clients for the New Moves program will be involved in the evaluation process through a voluntary participatory manner. A written consent will be obtained from the clients. The willingness to participate will ensure that that the findings are objective.
Answering the Evaluation Questions
The data collected will be categorized into qualitative and quantitative data. The data will then be analyzed. This will provide the descriptive statistics that relate to the program process and outcome. Based on the obtained results, inferences will be drawn to answer the specific evaluation questions.
Options of Designs and Data Methods
The study designs that can be used in the program include randomized experiments, quasi experiments and the non experiment design. The viable option for the evaluation of the New Moves is the experimental design. This is because the random selection provides an equal chance for any study participant to be either in the control or intervention group (Fitzpatrick et al., 2003). Thus, randomization will be based on chance. This will ensure that the lurking variables are distributed at chance levels. This is important because it reduces the personal bias that may result due to personal motives. Complete randomization will be carried out in order to divide the students into the two groups. The randomization will entail use of a computer program to generate the random numbers. The names of the obese girls will be assigned random numbers. The numbers will then be subjected to MINITAB, a computer program that is used to generate random samples.
The options for data methods include the use of primary or secondary data sources. Primary methods of data collection include the use of interviews, questionnaires, observations and focused group discussions. The secondary methods include the use of stored or recorded data. In the case of the New Moves, both the primary and secondary data collection methods provide a good option for gathering the evaluation data.
Limitations
The program is expected to run for nine months. In the course of the program, the school administration may alter the timetable. As a result, the projected time may be affected. Thus, some training sessions may not be completed within the nine months. The other challenge may relate to the budgetary allocations. The costs of the services such as traveling expenses are affected by economic changes. As a result, the cost of some items may increase and hence the program may run out of funds. In relation to the personnel, the busy schedules of the health professionals may not allow them to adhere to all training sessions. In addition, some personnel may drop out of the program. Based on the program design and purpose, there is an assumption that there will be no limitation in relation to data collection and political interferences.
In order to overcome the challenges, the program is tailor-made to fit the needs of the girls. Therefore, time issues can be addressed based on an individual basis. The budgetary projections are made based on factors such as foreseeable inflations. Thus, possible price changes are planned at the onset of the program. In order to ensure program continuity, the program will have many trainers. This implies that a trainer who departs will be replaced immediately.
References
Fitzpatrick, J., Worthen, B., & Sanders, J. (2003).Program evaluation: Alternative approaches and practical guideline. New York: Pearson Education Inc.
Msila, V., & Setlhako, A. (2013).Evaluation of Programs: Reading Carol H. Weiss. Pretoria: Research Publishing.
Neumark-Sztainer, D. (2005).Preventing weight-related problems in girls. Minneapolis: Guilford Press.
Neumark-Sztainer, D., Friend, S., Flattum, C., Hannan, P., Story, M., Bauer, K., & Petrich, C. (2010). New Moves-preventing weight-related problems in adolescent girls: a group- randomized study..American journal of preventive medicine, 39(5), 421-432.
Stame, N. (2004). Theory-based evaluation and types of complexity. Evaluation, 10(1), 58-76.
Van, V., & Brown, K. (1997). Evaluability assessment: A tool for program development in corrections. Cincinnati: University of Cincinnati.
Weiss, C. (1997).Evaluation: Roles for the Evaluator. Upper Saddle River, NJ: Prentice Hall.