Questions of Reliability
When answering the question of why there are different ways to estimate reliability, it is necessary to provide a definition for the word reliability.” Reliability refers to the consistency of a measure. A test is considered reliable if we get the same result repeatedly. For example, if a test is designed to measure a trait (such as introversion), then each time the test is administered to a subject, the results should be approximately the same. Unfortunately, it is impossible to calculate reliability exactly, but it can be estimated in a number of different ways.” (Cherry, K, about.com guide)
There are different methods for estimating reliability because there are different factors to measure the reliability of. There are four main classes of reliability estimates, with all measuring reliability in a different way. These are: 1. Test-Retest Reliability 2. Inter-Rater or Inter-Observer Reliability 3. Parallel-Forms Reliability 4. Internal Consistency Reliability
The type of reliability to be used would, in part, depend on what is being measured. It could be a skill, a talent, or an aptitude, to name a few. “Two of the primary criteria of evaluation in any measurement or observation are: 1. Whether we are measuring what we intend to measure. 2. Whether the same measurement process yields the same results” (QMSS e-lessons)
Test-retest reliability would consist of comparing the results of the same test, given to the same group of people, on two separate occasions. This method is mostly used to measure things that remain fairly constant, such as intelligence or math abilities. Inter-rater reliability would consist of having two different raters evaluate the same group of people on the same occasion and comparing the results of one rater to the other. Internal consistency reliability is mainly used to measure how inter-correlated the measured items are. This reliability would consist of repeating, essentially the same question, worded
There are different methods for estimating reliability because there are different factors to measure the reliability of. There are four main classes of reliability estimates, with all measuring reliability in a different way. These are: 1. Test-Retest Reliability 2. Inter-Rater or Inter-Observer Reliability 3. Parallel-Forms Reliability 4. Internal Consistency Reliability
The type of reliability to be used would, in part, depend on what is being measured. It could be a skill, a talent, or an aptitude, to name a few. “Two of the primary criteria of evaluation in any measurement or observation are: 1. Whether we are measuring what we intend to measure. 2. Whether the same measurement process yields the same results” (QMSS e-lessons)
Test-retest reliability would consist of comparing the results of the same test, given to the same group of people, on two separate occasions. This method is mostly used to measure things that remain fairly constant, such as intelligence or math abilities. Inter-rater reliability would consist of having two different raters evaluate the same group of people on the same occasion and comparing the results of one rater to the other. Internal consistency reliability is mainly used to measure how inter-correlated the measured items are. This reliability would consist of repeating, essentially the same question, worded
References: (Cherry, K, about.com guide) retrieved on 2/6/13 from http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm (QMSS e-lessons) retrieved on 2/6/13 from http://ccnmtl.columbia.edu/projects/qmss/measurement/validity_and_reliability.html http://www.ablongman.com/graziano6e/text_site/MATERIAL/reliability.htm http://www.socialresearchmethods.net/kb/reltypes.htm