Jordan- Irbid
Are Performance Improvement
Professionals Measurably Improving
Performance? What PIJ and PIQ Have to Say About the Current Use of
Evaluation and Measurement in the
Field of Performance Improvement
´pez and Hillary N. Leigh
Ingrid Guerra-Lo
T
he ability to prove that performance improvement professionals have made a measurable contribution to their clients and the field remains uncertain (Kaufman & Clark, 1999). Clark and
Estes (2000) noted that highly regarded research groups who surveyed performance improvement solutions found ‘‘a huge gap between what we think we accomplish and what scientific analyses say we accomplished’’ (p. 48). Here are some of the findings cited by
Clark and Estes (2000) from the work of the National
Academic of Sciences and the National Research
Council and other independent research groups:
~
~
~
~
Measurement and evaluation are at the core of reliably improving performance. It is through these central mechanisms that performance improvement professionals are able to demonstrate the true worth of their efforts. However, the true value of the contributions they make is inconclusive. This article presents a content analysis of 10 years’ worth of Performance Improvement and Performance
Improvement Quarterly articles as an initial data point to be used for professional reflection and further exploration into the intentions and practices of performance improvement practitioners.
Scientific studies of training found training interventions often leave participants worse off than before the training intervention (more confused, less able to remember important information, less able to use their work-related knowledge effectively).
More than half of organizational change initiatives are quickly abandoned. Kirkpatrick’s level one evaluation, the most commonly used method for evaluation, often gives about as much inaccurate information as it does accurate information, including the perception that
the
Professionals Measurably Improving
Performance? What PIJ and PIQ Have to Say About the Current Use of
Evaluation and Measurement in the
Field of Performance Improvement
´pez and Hillary N. Leigh
Ingrid Guerra-Lo
T
he ability to prove that performance improvement professionals have made a measurable contribution to their clients and the field remains uncertain (Kaufman & Clark, 1999). Clark and
Estes (2000) noted that highly regarded research groups who surveyed performance improvement solutions found ‘‘a huge gap between what we think we accomplish and what scientific analyses say we accomplished’’ (p. 48). Here are some of the findings cited by
Clark and Estes (2000) from the work of the National
Academic of Sciences and the National Research
Council and other independent research groups:
~
~
~
~
Measurement and evaluation are at the core of reliably improving performance. It is through these central mechanisms that performance improvement professionals are able to demonstrate the true worth of their efforts. However, the true value of the contributions they make is inconclusive. This article presents a content analysis of 10 years’ worth of Performance Improvement and Performance
Improvement Quarterly articles as an initial data point to be used for professional reflection and further exploration into the intentions and practices of performance improvement practitioners.
Scientific studies of training found training interventions often leave participants worse off than before the training intervention (more confused, less able to remember important information, less able to use their work-related knowledge effectively).
More than half of organizational change initiatives are quickly abandoned. Kirkpatrick’s level one evaluation, the most commonly used method for evaluation, often gives about as much inaccurate information as it does accurate information, including the perception that
the
References: Andrasik, F. (1979). Organizational behavior modification in business settings: A methodological and content review. Journal of Organizational Behavior Management, 2, 85–102. Balcazar, F. E., Shupert, M. K., Daniels, A. C., Mawhinney, T. C., & Hopkins, B. L. (1989). An objective review and analysis of ten years of publication in the Journal of Organizational Behavior Management Clark, R., & Estes, F. (2000). A proposal for the collaborative development of authentic performance technology Cowan, D. A. (1990). Developing a classification structure of organizational problems: An empirical investigation Culig, K. M., Dickinson, A. M., McGee, H. M., & Austin, J. (2005). An objective comparison of applied behavior analysis and organizational behavior management research. Dickinson, A. M. (2000). The historical roots of Organizational Behavior Management in the private sector: The 1950s–1980s Duriau, V. J., Reger, R. K., & Pfarrer, M. D. (2007). A content analysis of the content analysis literature in organizational studies: Research themes, data sources, and methodological refinements Eisenhardt, K. (1998). Decision making and all that jazz. In V. Papadakis & P. Barwise (Eds.), Strategic decisions Frederiksen, L. W., & Johnson, R. P. (1981). Organizational behavior management. Progress in Behavior Modification, 12, 67–118. Frederiksen, L. W., & Lovett, S. B. (1980). Inside organizational behavior management: Perspectives on an emerging field Guerra, I. (2003). Key competencies required of performance improvement professionals. Hopkins, B. L., & Sears, J. (1982). Managing behavior for productivity. In L. W. Frederiksen (Ed.), Handbook of organizational behavior management (pp Kaufman, R. (2006). Change, choices, and consequences: A guide to Mega thinking and planning Kaufman, R., & Clark, R. (1999). Re-establishing performance improvement as a legitimate area of inquiry, activity, and contribution: Rules of the road Kaufman, R., & Thomas, S. (1980). Evaluation without fear. New York: New Points/Franklin Watts. Klein, J. D. (2002). Empirical research on performance improvement. Performance Improvement Quarterly, 15(1), 99–110. Marker, A., Huglin, A., & Johnsen, L. (2006). Empirical research on performance improvement: An update. Performance Improvement Quarterly, 19(4), 7–22. Nutt, P. (1979). Calling out and calling off the dogs: Managerial diagnoses in organizations. Nutt, P. (2006). Intelligence gathering for decision making. Omega, No. 25, 604–622. O’Hara, K., Johnson, C. M., & Beehr, T. A. (1985). Organizational behavior management in the private sector: A review of empirical research and recommendations for further Pounds, W. (1969). The process of problem finding. Industrial Management Review, 1969, 1–19. Rummler, G. A., & Brache, A. P. (1995). Improving performance: How to manage the white space on the organization chart (2nd ed.) Scriven, M. (1967). The methodology of evaluation. In R. Tyler, R. Gagne, & M. Scriven (Eds.), Perspectives on curriculum evaluation Stufflebeam, D., & Shinkfield, A. (2007). Evaluation theory, models, and applications. San Francisco: Wiley.