Handbook of Practical Program Evaluation

The second edition of Handbook of Practical Program Evaluation offers managers, analysts, consultants, and educators in government, nonprofit, and private institutions a valuable resource that outlines efficient and economical methods for assessing program results and identifying ways to improve program performance. The Handbook has been thoroughly revised. Many new chapters have been prepared for this edition, including chapters on logic modeling and on evaluation applications for small nonprofit organizations. The Handbook of Practical Program Evaluation is a comprehensive resource on evaluation, covering both in-depth program evaluations and performance monitoring. It presents evaluation methods that will be useful at all levels of government and in nonprofit organizations.

Figures, Tables, and Exhibits xi

Preface xvii

The Editors xix

The Contributors xxiii

Meeting the Need for Practical Evaluation Approaches: An Introduction xxxiii
Kathryn E. Newcomer, Harry P. Hatry, Joseph S. Wholey

Part One: Designing Performance Monitoring Systems and Evaluation Studies 1

1. Using Logic Models 7
John A. McLaughlin, Gretchen B. Jordan

2. Evaluability Assessment 33
Joseph S. Wholey

3. Implementation Evaluation 63
Arnold Love

4. Performance Monitoring 98
Theodore H. Poister

5. Quasi-Experimentation 126
Charles S. Reichardt, Melvin M. Mark

6. Using Randomized Experiments 150
Robert G. St. Pierre

7. Meta-Analysis, Systematic Reviews, and Research Syntheses 176
Robert F. Boruch, Anthony Petrosino

Part Two: Practical Data Collection Procedures 205

8. Trained Observer Ratings 211
John M. Greiner

9. Using Surveys 257
Kathryn E. Newcomer, Timothy Triplett

10. Using Expert Judgment 292
Harvey A. Averch

11. Role Playing 310
Margery Austin Turner, Wendy Zimmermann

12. Using Focus Groups 340
Robert Goldenkoff

13. Collecting Data in the Field 363
Demetra Smith Nightingale, Shelli Balter Rossman

14. Using Agency Records 396
Harry P. Hatry

Part Three: Analyzing Evaluation Data 413

15. Qualitative Data Analysis 417
Sharon L. Caudle

16. Using Statistics in Evaluation 439
Kathryn E. Newcomer, Philip W. Wirtz

17. Using Regression Analysis 479
Dale E. Berger

18. Cost-Effectiveness and Cost-Benefit Analysis 506
James Edwin Kee

Part Four: Getting Evaluation Results Used 543

19. Pitfalls of Evaluation 547
Harry P. Hatry, Kathryn E. Newcomer

20. Managing Evaluation Projects 571
James B. Bell

21. Writing for Impact 604
George F. Grob

22. Using Organizational Report Cards 628
William T. Gormley Jr.

23. The Use of Evaluation by Nonprofit Organizations 649
Mary E. Kopczynski, Kathleen Pritchard

24. Other Issues and Trends in Evaluation 670
Harry P. Hatry, Joseph S. Wholey, Kathryn E. Newcomer

Name Index 685

Subject Index 693

Joseph S. Wholey is a professor of public administration at the University of Southern California.

Harry P. Hatry is a principal research associate and director of the Public Management Program at the Urban Institute in Washington, D.C.

Kathryn E. Newcomer is director of the School of Public Policy and Public Administration at the George Washington University.

The second edition of Handbook of Practical Program Evaluation offers managers, analysts, consultants, and educators in government, nonprofit, and private institutions a valuable resource that outlines efficient and economical methods for assessing program results and identifying ways to improve program performance.

The Handbook has been thoroughly revised. Many new chapters have been prepared for this edition, including chapters on logic modeling and on evaluation applications for small nonprofit organizations.

The Handbook of Practical Program Evaluation is a comprehensive resource on evaluation, covering both in-depth program evaluations and performance monitoring. It presents evaluation methods that will be useful at all levels of government and in nonprofit organizations.

This important book covers:

• A variety of evaluation designs, including designs for ongoing monitoring of program results and designs for in-depth evaluation of program performance
• Selection of data collection procedures that help evaluators obtain relevant information in an efficient manner
• Methods for analyzing evaluation data
• Steps for increasing the usefulness and actual use of evaluation results by policymakers, managers, their staff members, and other stakeholders in order to improve programs and inform policy decisionmaking
• Selection and training of evaluators, quality control of the enterprise, standards and ethics in evaluation work, the creation of incentives for undertaking program evaluation, and future trends in program evaluation

Praise for the First Edition

"I am better in my job because I read the Handbook of Practical Program Evaluation, and without question, the analysts I supervise would benefit from reading this handbook."
—John A. Mahone, Virginia Department of Planning and Budget