C8057 Reliability Analysis Paper
C8057 (Research Methods II): Reliability Analysis
Reliability Analysis
Measures of Reliability
Reliability: the fact that a scale should consistently reflect the construct it is measuring.
One way to think of reliability is that other things being equal, a person should get the same score on a questionnaire if they complete it at two different points in time (test-retest reliability. Another way to look at reliability is to say that two people who are the same in terms of the construct being measured, should get the same score. In statistical terms, the usual way to look at reliability is based on the idea that individual items (or sets of items) should produce results consistent with the overall questionnaire.
The simplest way …show more content…
Kline (1999) notes that although the generally accepted value of 0.8 is appropriate for cognitive tests such as intelligence tests, for ability tests a cut-off point of 0.7 if more suitable.
He goes onto say that when dealing with psychological constructs values below even 0.7 can, realistically, be expected because of the diversity of the constructs being measured.
However, Cortina (1993) notes that such general guidelines need to be used with caution because the value of alpha depends on the number of items on the scale (see Field, 2005 for details). Alpha is also affected by reverse scored items. For example, in our SAQ from last week we had one item (question 3) that was phrased the opposite way around to all other items. The item was ‘standard deviations excite me’. Compare this to any other item and you’ll see it requires the opposite response. For example, item 1 is ‘statistics make me cry’. Now, if you don’t like statistics then you’ll strongly agree with this statement and so will get a score of 5 on …show more content…
None of the items here would substantially affect reliability if they were deleted. The worst offender is question 10: deleting this question would increase the alpha from .823 to .824. Nevertheless this increase is not dramatic and both values reflect a reasonable degree of reliability.
Dr. Andy Field
Page 3
2/15/2006
C8057 (Research Methods II): Reliability Analysis
Finally, and perhaps most important, the value of Alpha at the very bottom is The Cronbach’s alpha: the overall reliability of the scale. To re-iterate we’re looking for values in the magnitude of .7 to .8 (or there about) bearing in mind what we’ve already noted about effects from the number of items. In this case alpha is slightly above .8, and is certainly in the region indicated by Kline, so this probably indicates good reliability. As a final point, it’s worth noting that if items do need to be removed at this stage then you should re-run your factor analysis as well to make sure that the deletion of the item has not affected the factor structure!
RELIABILITY
ANALYSIS
-
Reliability Analysis
Measures of Reliability
Reliability: the fact that a scale should consistently reflect the construct it is measuring.
One way to think of reliability is that other things being equal, a person should get the same score on a questionnaire if they complete it at two different points in time (test-retest reliability. Another way to look at reliability is to say that two people who are the same in terms of the construct being measured, should get the same score. In statistical terms, the usual way to look at reliability is based on the idea that individual items (or sets of items) should produce results consistent with the overall questionnaire.
The simplest way …show more content…
Kline (1999) notes that although the generally accepted value of 0.8 is appropriate for cognitive tests such as intelligence tests, for ability tests a cut-off point of 0.7 if more suitable.
He goes onto say that when dealing with psychological constructs values below even 0.7 can, realistically, be expected because of the diversity of the constructs being measured.
However, Cortina (1993) notes that such general guidelines need to be used with caution because the value of alpha depends on the number of items on the scale (see Field, 2005 for details). Alpha is also affected by reverse scored items. For example, in our SAQ from last week we had one item (question 3) that was phrased the opposite way around to all other items. The item was ‘standard deviations excite me’. Compare this to any other item and you’ll see it requires the opposite response. For example, item 1 is ‘statistics make me cry’. Now, if you don’t like statistics then you’ll strongly agree with this statement and so will get a score of 5 on …show more content…
None of the items here would substantially affect reliability if they were deleted. The worst offender is question 10: deleting this question would increase the alpha from .823 to .824. Nevertheless this increase is not dramatic and both values reflect a reasonable degree of reliability.
Dr. Andy Field
Page 3
2/15/2006
C8057 (Research Methods II): Reliability Analysis
Finally, and perhaps most important, the value of Alpha at the very bottom is The Cronbach’s alpha: the overall reliability of the scale. To re-iterate we’re looking for values in the magnitude of .7 to .8 (or there about) bearing in mind what we’ve already noted about effects from the number of items. In this case alpha is slightly above .8, and is certainly in the region indicated by Kline, so this probably indicates good reliability. As a final point, it’s worth noting that if items do need to be removed at this stage then you should re-run your factor analysis as well to make sure that the deletion of the item has not affected the factor structure!
RELIABILITY
ANALYSIS
-