applied psychology
Journal of Applied Psychology
1973, Vol. 58, No. 1, 116-121
EFFECTS OF ITEM FORMAT ON ITEM DISCRIMINATION
AND DIFFICULTY
ARTHUR L. DUDYCHA 1 AND JAMES B. CARPENTER 2
Purdue University
The tests and measurement literature is replete with multiple-choice item construction rules, most of which have little or no empirical verification. Item stem orientation (positive/negative), structure (open/closed), and the presence or absence of an inclusive alternative were systematically investigated as to their effects on the item properties of difficulty and discrimination. The data were obtained from 1,124 introductory psychology students taking a regularly scheduled test. The results indicated that item difficulty was critically affected by all three independent variables with no interactions and that item discrimination was affected by the use of an inclusive alternative with a significant interaction between orientation and stem structure. Some suggestions were made with respect to effective item-writing strategies.
As a person sits poised to write a typical achievement or classroom test, he may well be troubled by three points: (a) Item-writing rules of thumb flourish (41 distinct rules were found in a cursory survey), (b) Empirical support of these rules is as conspicuously absent as the rules are present (Ebel, 1951;
Wesman, 1971). (c) The best laid test plan can be severely undermined by structurally defective items, which can capriciously inflate or attenuate test scores.
The specific problem of interest here is that of determining the relationships between some multiple-choice item formats and the item properties of difficulty and discrimination.
Admittedly, test validity and reliability are the critical parameters of a final test product, but their fair assessment crucially depends upon the principles of item and test construction.
The multiple-choice item is, most often, comprised of a stem or lead, followed by one or more
1973, Vol. 58, No. 1, 116-121
EFFECTS OF ITEM FORMAT ON ITEM DISCRIMINATION
AND DIFFICULTY
ARTHUR L. DUDYCHA 1 AND JAMES B. CARPENTER 2
Purdue University
The tests and measurement literature is replete with multiple-choice item construction rules, most of which have little or no empirical verification. Item stem orientation (positive/negative), structure (open/closed), and the presence or absence of an inclusive alternative were systematically investigated as to their effects on the item properties of difficulty and discrimination. The data were obtained from 1,124 introductory psychology students taking a regularly scheduled test. The results indicated that item difficulty was critically affected by all three independent variables with no interactions and that item discrimination was affected by the use of an inclusive alternative with a significant interaction between orientation and stem structure. Some suggestions were made with respect to effective item-writing strategies.
As a person sits poised to write a typical achievement or classroom test, he may well be troubled by three points: (a) Item-writing rules of thumb flourish (41 distinct rules were found in a cursory survey), (b) Empirical support of these rules is as conspicuously absent as the rules are present (Ebel, 1951;
Wesman, 1971). (c) The best laid test plan can be severely undermined by structurally defective items, which can capriciously inflate or attenuate test scores.
The specific problem of interest here is that of determining the relationships between some multiple-choice item formats and the item properties of difficulty and discrimination.
Admittedly, test validity and reliability are the critical parameters of a final test product, but their fair assessment crucially depends upon the principles of item and test construction.
The multiple-choice item is, most often, comprised of a stem or lead, followed by one or more
References: ADKINS, D. C. Construction and analysis of achievement tests. Washington, D.C.: U.S. Government Printing Office, 1947. multiple-choice item construction principles. Educational and Psychological Measurement, 1959, 19, 171-179. McGraw-Hill, 1967. Base, Texas: Author, 1963. WAKEFIELD, J. A. Does the fifth choice strengthen a test item? Public Personnel Review, 1958, 19, Education, 1971. (Received February 14, 1972)