An Analysis of Predictive Validity at Colleges
Is NSSE Messy? An Analysis of Predictive Validity
David DiRamio and David Shannon
Paper presentation for the Annual Meeting of the American Educational Research Association April 10, 2011 in New Orleans, LA
Contact Information: David DiRamio, Ph.D. Associate Professor Administration of Higher Education 4096 Haley Center Auburn University, AL 36849 Office (334) 844-3065 E-mail: diramio@auburn.edu David Shannon, Ph.D. Professor Educational Psychology 4028 Haley Center Auburn University, AL 36849 Office (334) 844-3071 E-mail: shanndm@auburn.edu
Running Head: Is NSSE Messy? 4‐5‐11
Is NSSE Messy? An Analysis of Predictive Validity Each year at colleges and universities across the nation, senior administrators and governance officials (i.e. trustees) gather to hear an annual report, typically presented by institutional research staff, detailing how their own school scored on the National Survey of Student Engagement (NSSE). The venerable NSSE, with its five benchmarks and peer comparisons, provides empirical evidence about participation in the activities and programs that institutions provide for students’ learning and development. The individual annual NSSE report will detail for stakeholders how well (or poorly) the institution is engaging its students. The premise that student engagement is a proximal measure, or “proxy,” for learning is well supported in the literature (Angelo & Cross, 1993; Astin, 1993; Kuh, Kinzie, Schuh, & Whitt, 2005). NSSE is highly regarded as an instrument for measuring factors pointing to student learning and success. However, do NSSE scores actually correlate with academic success outcomes that are on the minds of senior officials? Others have investigated similar issues related to NSSE (Gordon, Ludlum, & Hoey, 2008; Pascarella & Seifert, 2008; Swerdzewski, Miller, & Mitchell, 2007), but ours is a different approach. In this study we investigated the relationship between NSSE and two of the outcomes measures that typically
David DiRamio and David Shannon
Paper presentation for the Annual Meeting of the American Educational Research Association April 10, 2011 in New Orleans, LA
Contact Information: David DiRamio, Ph.D. Associate Professor Administration of Higher Education 4096 Haley Center Auburn University, AL 36849 Office (334) 844-3065 E-mail: diramio@auburn.edu David Shannon, Ph.D. Professor Educational Psychology 4028 Haley Center Auburn University, AL 36849 Office (334) 844-3071 E-mail: shanndm@auburn.edu
Running Head: Is NSSE Messy? 4‐5‐11
Is NSSE Messy? An Analysis of Predictive Validity Each year at colleges and universities across the nation, senior administrators and governance officials (i.e. trustees) gather to hear an annual report, typically presented by institutional research staff, detailing how their own school scored on the National Survey of Student Engagement (NSSE). The venerable NSSE, with its five benchmarks and peer comparisons, provides empirical evidence about participation in the activities and programs that institutions provide for students’ learning and development. The individual annual NSSE report will detail for stakeholders how well (or poorly) the institution is engaging its students. The premise that student engagement is a proximal measure, or “proxy,” for learning is well supported in the literature (Angelo & Cross, 1993; Astin, 1993; Kuh, Kinzie, Schuh, & Whitt, 2005). NSSE is highly regarded as an instrument for measuring factors pointing to student learning and success. However, do NSSE scores actually correlate with academic success outcomes that are on the minds of senior officials? Others have investigated similar issues related to NSSE (Gordon, Ludlum, & Hoey, 2008; Pascarella & Seifert, 2008; Swerdzewski, Miller, & Mitchell, 2007), but ours is a different approach. In this study we investigated the relationship between NSSE and two of the outcomes measures that typically
References: Angelo, T., & Cross, K. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco, CA: Jossey-Bass. Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass. Gordon, J., Ludlum, J., & Hoey, J. J. (2008). Validating the NSSE against student outcomes: Are they related? Research in Higher Education, 49, 19-39. Kuh, G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J. (2005). Assessing conditions to enhance educational effectiveness: The inventory for student engagement and success. San Francisco, CA: Jossey-Bass. Lipka, S. (2009, March 6). Facing recession, colleges push 3-Year degrees. The Chronicle of Higher Education, 55(29), A19. Litwin, M. S. (Ed.) (2002). How to assess and interpret survey psychometrics. Sage: Thousand Oaks, CA. Pascarella, E. T., & Seifert, T. A. (2008). Validation of the NSSE benchmarks and deep approaches to learning against liberal arts outcomes. Paper presented at the Association for the Study of Higher Education, Jacksonville, FL. Pedhauzer, E. J. (1997). Multiple regression in behavioral research. Orlando, FL: Harcourt Brace & Co. Schmidt, P. (2005, April 8). On-time graduates to get cash rewards. The Chronicle of Higher Education, 51(31), A33. Swerdzewski, P., Miller, B. J., & Mitchell, R. (2007). Investigating the validity of the National Survey of Student Engagement. Paper presented at the Northeastern Educational Research Association, Rocky Hill, CT. 11