Steps in Item Analysis
Steps in Making
Item Analysis
________________________________________________________________
1. Score each answer sheet, write score total on the corner * Obviously have to do this anyway
2. Sort the pile into rank order from top to bottom score
( 1 minute, 30 seconds tops)
3. If normal class of 30 students, divide class in half * Same number in top and bottom group; * Toss middle paper if odd number (put aside)
4. Take “top” pile, count number of students who responded to each alternative * Fast way is simply to sort piles into “A”, “B”, “C”, “D” // or true/false or type of error you get for short answer, fill-in-the-blanks
Or set up on spread sheet if you’re familiar with computers ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 C 1 D 1 O*= Keyed Answer |
* Repeat for lower group
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 2 C 1 D 1 O*= Keyed Answer |
* This is the time consuming part but not bad, can do it while watching TV, because you’re just sorting piles
________________________________________________________________
THREE POSSIBLE SHORT CUTS HERE (STEP 4) (A) If you have a large sample of around 100 or more, you can cut down the sample you work with * Take top 27% ( 27 out of 100 ); bottom 27% ( so only dealing with 54, not all 100) * Put middle 46 aside for the moment * Larger the sample, more accurate, but have to trade off against labour; using top 1/3 or so is probably good enough by the time you get to 100; -- 27% magic figure statisticians tell us to use * I’d use halves at 30, but you could just use a sample of top 10 and bottom 10 if you’re pressed for time * But it means a single student changes stats by 10% * Trading off speed for
Item Analysis
________________________________________________________________
1. Score each answer sheet, write score total on the corner * Obviously have to do this anyway
2. Sort the pile into rank order from top to bottom score
( 1 minute, 30 seconds tops)
3. If normal class of 30 students, divide class in half * Same number in top and bottom group; * Toss middle paper if odd number (put aside)
4. Take “top” pile, count number of students who responded to each alternative * Fast way is simply to sort piles into “A”, “B”, “C”, “D” // or true/false or type of error you get for short answer, fill-in-the-blanks
Or set up on spread sheet if you’re familiar with computers ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 C 1 D 1 O*= Keyed Answer |
* Repeat for lower group
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 2 C 1 D 1 O*= Keyed Answer |
* This is the time consuming part but not bad, can do it while watching TV, because you’re just sorting piles
________________________________________________________________
THREE POSSIBLE SHORT CUTS HERE (STEP 4) (A) If you have a large sample of around 100 or more, you can cut down the sample you work with * Take top 27% ( 27 out of 100 ); bottom 27% ( so only dealing with 54, not all 100) * Put middle 46 aside for the moment * Larger the sample, more accurate, but have to trade off against labour; using top 1/3 or so is probably good enough by the time you get to 100; -- 27% magic figure statisticians tell us to use * I’d use halves at 30, but you could just use a sample of top 10 and bottom 10 if you’re pressed for time * But it means a single student changes stats by 10% * Trading off speed for