Scaling Study

Scaling Study
Sonja Sievers
RES/320
October 18, 2010
Jen Burns

Scaling Study

This paper is study of the efficiency of setting up a survey questionnaire. A short questionnaire of 5 questions was distributed to 11 selected participants. The participant’s answers were calculated and analyzed. The two different types of questions that was used in the survey where the rating and ranking scale. The participants were asked to compare a new product on a 4-point rating scale without comparing it to another product. The other portion of the survey was made up of ranking scale questions were the participant was asked to make a comparison and determine what rank-order the importance of the product is in taste, price, and convince. The validity and reliability of the questionnaire was controlled by selecting the participants. This was a useful approach to evaluate the efficiency of the survey. For example, if the product was a famine product you would choice female participants that were at the age that they would use the product. The survey would not be given to males or females between the ages of 1-12. The data would be useless and a waste of time. One of the questions asked in the survey was “Would you use this product?” and “If yes how many times?” Although this question may be reliable is may not be valid. The reason for this is a standardized questionnaire that is used again and again in several of survey has a high reliability, but poor validity. Establishing survey validity is not as straightforward as reliability and usually requires evidence from several sources. Careful wording, format, and content can reduce the surveys own unreliability. When measuring the validity of the survey I choice three basic approaches. The first approach was content validity. If the content of the survey has affiliation with the content that is being studied, then the survey has content validity. For example, a Manager Evaluation Survey for department managers should have content (such as knowledge, skills, and qualifications) relevant to the jobs of department managers. Skills that pertain to a receptionist would not be appropriate in a manager evaluation survey for managers. The second approach would be predictive validity. This form of validity comes from the products ability to predict the future outcome of sales. If a questionnaire is developed to assess the products potential to succeed, the results of the survey should be able to predict which product will be successful. The predictive validity of the product is shown in the correlation between the scores from the survey and the products success. The third approach is construct validity. Key (1997), “This form of validity derives from the correlation between the survey and another process that measures the same construct” (Reliability and Validity, para. 3-6). The Direct Product Model (DPM) is well-established test used to analyze the results of multiple data. A survey “developed to assess the same characteristics would have construct validity if the scores from the new instrument correlated highly with the scores”(Key, 1997) from the DPM. There were several things that the new product survey needed to address. In order to complete a successful survey the company needs to understand which benefits are most important to the customers. Which features are essential in delivering the product’s guarantees? Indentify what the customer’s needs are. I would also look what the customer was looking for in advertising and placement factors. Only after evaluating and accessing the customer’s needs was I able to come up with what type of survey would give me the largest amount of reliable data. When selecting which type of survey to use in the study there were several issues to consider. There is not easy or clear way to make the decision of the context that the survey will need. Clearly, there may not be one approach that is best, it may be several approaches. Trochim (2006), “You may have to make tradeoffs of advantages and disadvantages. There is judgment involved. Two expert researchers may, for the very same problem or issue, select entirely different survey methods. But, if you select a method that isn’t appropriate or doesn’t fit the context, you can doom a study before you even begin designing the instruments or questions themselves” (Administrative Issues, para. 5).

References Key, J. P. (1997). Reliability. Oklahoma State University. Retrieved from http://www.okstate.edu/ag/agedcm4h/academic/.../newpage 18.htm Trochim, W. m.k. (2006). Selecting the Survey Method. Research Methods Knowledge Base. Retrieved from http://www.socialresearchmethods.net/kb/survsel.php

References: Key, J. P. (1997). Reliability. Oklahoma State University.  Retrieved from http://www.okstate.edu/ag/agedcm4h/academic/.../newpage 18.htm Trochim, W. m.k. (2006). Selecting the Survey Method. Research Methods Knowledge Base.  Retrieved from http://www.socialresearchmethods.net/kb/survsel.php