Evaluating Survey Worthiness
Evaluating Survey Worthiness
Surveys are used to collect data. Nearly every day, you read or hear about survey or opinion poll results in newspapers, on the Internet, or on radio or television. To identify surveys that lack objectivity or credibility, you must critically evaluate what you read and hear by examining the worthiness of the survey. First, you must evaluate the purpose of the survey, why it was conducted, and for whom it was conducted.
The second step in evaluating the worthiness of a survey is to determine whether it was based on a probability or nonprobability sample. You need to remember that the only way to make valid statistical inferences from a sample to a population is through the use of a probability sample. Surveys that use nonprobability sampling methods are subject to serious, perhaps unintentional, biases that may make the results meaningless. Even when surveys use random probability sampling methods, they are subject to potential errors.
There are four types of survey errors:
• Coverage Error - It occurs if certain groups of items are excluded from the frame so that they have no chance of being selected in the sample. Coverage error results in a selection bias. If the frame is inadequate because certain groups of items in the population were not properly included, any random probability sample selected will provide only an estimate of the characteristics of the frame, not the actual population.
• Nonresponse Error - Not everyone is willing to respond to a survey. In fact, research has shown that individuals in the upper and lower economic classes tend to respond less frequently to surveys than do people in the middle class. Nonresponse error arises from failure to collect data on all items in the sample and results in a nonresponse bias. Because you cannot always assume that persons who do not respond to surveys are similar to those who do, you need to follow up on the nonresponses after a specified period of time. You
Surveys are used to collect data. Nearly every day, you read or hear about survey or opinion poll results in newspapers, on the Internet, or on radio or television. To identify surveys that lack objectivity or credibility, you must critically evaluate what you read and hear by examining the worthiness of the survey. First, you must evaluate the purpose of the survey, why it was conducted, and for whom it was conducted.
The second step in evaluating the worthiness of a survey is to determine whether it was based on a probability or nonprobability sample. You need to remember that the only way to make valid statistical inferences from a sample to a population is through the use of a probability sample. Surveys that use nonprobability sampling methods are subject to serious, perhaps unintentional, biases that may make the results meaningless. Even when surveys use random probability sampling methods, they are subject to potential errors.
There are four types of survey errors:
• Coverage Error - It occurs if certain groups of items are excluded from the frame so that they have no chance of being selected in the sample. Coverage error results in a selection bias. If the frame is inadequate because certain groups of items in the population were not properly included, any random probability sample selected will provide only an estimate of the characteristics of the frame, not the actual population.
• Nonresponse Error - Not everyone is willing to respond to a survey. In fact, research has shown that individuals in the upper and lower economic classes tend to respond less frequently to surveys than do people in the middle class. Nonresponse error arises from failure to collect data on all items in the sample and results in a nonresponse bias. Because you cannot always assume that persons who do not respond to surveys are similar to those who do, you need to follow up on the nonresponses after a specified period of time. You