Chapter 16. Statistics are used in a variety of ways in the process of evaluation. Which statistics to choose depends on the level of measurement used for coding the phenomena of interest (nominal or ordinal). The chapter concerns the selection of the statistical techniques. The first part discusses descriptive and inferential statistics that use the technique of generalizing from samples (following the four principles: reasonably known and identifiable population of interest; using a sampling technique that can be calculated; the size of sample appropriate to the relative size of population examining the sample to ensure its relativity to the whole population (Newcomer and Wirtz, 2004).
The authors continue with selecting a statistical confidence level, using a confidence interval to convey results, testing statistical significance for nominal- and ordinal-level variables. Measures of association in nominal-level variables are phi squared, Cramer’s V, contingency coefficient, Goodman and Kruskall’s Tau, and Lambda. Ordinal-level variables are Spearman’s r, measures of concordance – Goodman-Kraskal Gamma, Kendell’s T, Stuart’s T, Somers’ D (Newcomer and Wirtz, 2004).
Selecting appropriate statistics is also a complicated process directed at the achievement of multiple targets:
No matter which analytical technique is selected, both the statistic used to assess statistical significance and the magnitude of an effect or strength of the relationships analyzed should be reported (Newcomer and Wirtz, 2004).
Criteria for selection are question-related, measurement-related are audience-related. Sorting measures or units is also important as well as reporting statistics appropriately: it includes identifying contents of all tables and figures clearly and indicating the use of decision rules in analysis, consolidating analyses and avoiding abbreviations, providing basic information about the measurement of variables, presenting appropriate percentages, presenting information on statistical significance clearly, presenting information on the magnitude of relationships clearly and using graphics to present analytical findings (Newcomer and Wirtz, 2004).
Chapter 17. The author signifies the importance of regression:
Correlation and regression are extraordinarily powerful tools that find frequent use in evaluation and applied research (Berger, 2004).
The author of the chapter explains the multiple regression model, compares two groups on two examples (testing the difference between two group means and comparing treatment effects in two groups, controlling for pretest). He also gives out the SPSS syntax for analysis of change as a significant element of regression analysis.
One more element is the mediation analysis with regression –
Mediation analysis can help us understand how programs work and guide development and modification of programs to make them more effective (Berger, 2004).
In the context of discussing mediation analysis with regression the author considers tests of significance of mediation and gives an example of estimating and testing mediating effects, the issues that are also discussed are presenting mediation analyses and presenting results from regression analyses in a table. Other issues the author turns his attention to are categorical variables, correlation and causation, multicollinearity, interactions, centering continuous predictor variables, non-linear relationships, outliners, missing data, power analysis, and sample size, stepwise versus a hierarchical selection of variables (Berger, 2004).
The concluding section of the chapter is an admission of the fact that estimates and tests of effects of individual variables may be misleading. The solution is “to include sample size with each analysis and a measure of effect size” (Berger, 2004). The last tip the author gives is to prefer the manual evaluation of data to obtain more efficient results and not to use computerized methods.
Bibliography
- Berger, D.E. (2004). Using Regression Analysis. In J.S. Wholey, H.P. Hatry, K.E. Newcomer (Eds), Handbook of practical program evaluation (2nd ed., pp. 417-438). San Francisco: Jossey-Bass.
- Newcomer, K.E. and P.W. Wirtz (2004). Using Statistics in Evaluation. In J.S. Wholey, H.P. Hatry, K.E. Newcomer (Eds), Handbook of practical program evaluation (2nd ed., pp. 417-438). San Francisco: Jossey-Bass.