Following Question

MARIKINA POLYTECHNIC COLLEGE
Mayor Chanyungco St., Sta. Elena
Marikina City

STUDENT’S NAME: RITCHELL S. GALAS
TITLE: GUIDELINES ON THE FORMULATION OF QUESTIONS
INTRODUCTION
All rules regarding the formulation of questionnaire questions – no matter who has created them and where they are to be found – have one crucial disadvantage: they only have limited use. Of course, they are more or less suitable as “general guides” which can point you in a general direction, but their importance usually diminishes when it comes to formulating specific questions for specific questionnaires. It is then necessary to rethink each formulation and, while the rules regarding the formulation of questions can offer some support or be helpful, they do not excuse you from reassessing their validity and effectiveness for each question.

So when you are developing your questionnaire questions you should pay attention to the following “ten rules for formulating questions”. You should also note that they are not fixed, irrevocable rules which can be followed blindly. The majority of the rules leave room for interpretation indeed – as you will see – they often contradict each other so they should not always be fully observed. You should view the “10 rules” critically when you are asking yourself whether your specific questions are “good”.

It is obvious why questions need to be “good“, meaning methodical and technically flawless, as poor questions lead to poor results and no weighting or analysis process in the world can make good results out of poor data
Survey tip 1: Use simple, unambiguous terms, which can be understood by all respondents in the same way!
The concept of a question being understood by all respondents in the same way is vitally important when carrying out standardized questionnaires. The chances of achieving this aim depend on whether the questions are simply and clearly (or rather unambiguously) formulated. But whether a question is "simply" or "unambiguously" formulated very much depends on the people you intend to ask. Certain formulations may well be simple and understandable for an economics professor, but that does not mean they will be understood by other people. Let’s take the following question as an example:

"How do you think the business cycle in Germany will develop between now until the end of 2001 compared to the current situation? Very positively, positively, negatively, very negatively or will it stay the same?"

While the aforementioned economics professor should (hopefully!) be able to understand and answer this difficult question, the same should not be expected of everyone. Is the term "business cycle" familiar (and used in the correct way)? Is the scale understandable? The following formulation would be better:
"Do you think it is necessary to have a family in order to be really happy or do you think you can be just as happy living alone?"

Anyone who thinks this question is simple and unambiguous should show it to twenty people and ask them what they understand by "family" and "really happy" (you can send me the twenty different definitions by email!).

However, you shouldn’t be too strict about this as it would hardly be possible to formulate a questionnaire. Instead let’s try to formulate questions which are "reasonably" simple and unambiguous:

Survey tip 2: Avoid long and complex questions!
Long and complex questions run the risk of being incomprehensible and confusing the participant, they can contain terms which are redundant or superfluous and/or – perhaps even unintentionally – contain too many different stimuli. Although the question of how complex a questionnaire may be depends to a large extent on the target group of the questionnaire, the difference between a long and complex question and a short and simple question is obvious.
Let’s consider the following example:
"As you know, some people are quite politically active, while other people often don’t have time or aren’t interested in taking an active part in political stuff. I will now give you a list of things that people do. Tell me in each case how often you personally do stuff or how often it happens. (Followed by a list with the answer categories often – sometimes – rarely – never.) Firstly, how often do you have a political discussion?"
Apart from the sloppy use of the English language ("political stuff", "a list of things.., that people do"), information is provided which isn’t necessary in order to answer the question ("As you know...", "other people..."); furthermore it could also be unclear what is meant by having a "political discussion". Where does this discussion take place? In public or among friends? With the family or in the pub?
Let’s try it this way:
"How often do you take part in public political discussions? Often, sometimes, rarely or never?"
It is simpler, clearer, shorter and less complex, although the answer categories could be reconsidered.
Survey tip 3: Avoid hypothetical questions!
Hypothetical questions are those which ask the respondent to put themselves in imaginary situations. Whether or not they are able to do this depends in part to the extent the person has already dealt with the hypothetical situation and how far the hypothetical situation is close to or removed from reality.

It is possible to deal with the situation if it is hypothetical but perfectly conceivable as the respondent already has some experience of the issue. Anyone could answer the following question.
"Let’s assume you have won a million pounds on the lottery – would you then give up work or would you keep working?"

The question is easy to answer because everyone has thought about "what would happen, if..." But imagine that you wanted to test the hypothesis that young people nowadays would rather have immediate professional and financial success, even if it were on shaky ground, than accept reduced, but perhaps more secure, job prospects.

So we ask a young person the following question:
"Imagine you are married and have a 16 year old son who wants to give up his apprenticeship in order to become a professional footballer. Would you support him or would you advise him to complete his apprenticeship?"
Even though the actual alternative in this question – interrupt or terminate the apprenticeship – could be quite true to life for young people, the requirements formulated in the questions – married, having a son with sporting ambitions who has begun an apprenticeship – create an abstract set of general conditions for which people at a young age will not have formed an opinion. The question is too hypothetical.
Survey tip 4: Avoid stimuli and negatives!
The question...
Do you like listening to music by Chopin?

contains a single concrete stimulus, which is "listening to music by Chopin", and can be answered immediately with "yes" or "no", depending on whether or not you like listening to music by Chopin. If a second stimulus were to be added, creating a double stimulus, the question becomes confusing. The question...

Do you like listening to music by Chopin and Wagner? is not in itself absurd because Chopin and Wagner are both great masters of classical music. However, the question is not about classical music, but rather about quite different types of composer. This is why the question cannot be answered clearly, if, for example, you like music by Chopin, but don’t like listening to music by Wagner. "Chopin yes, Wagner no" could be a possible correct and conceivable answer. As the respondent may only answer with a "yes" or "no" we do not know if "yes" means that they like listening to Chopin and Wagner or if they are just reacting to one of the stimuli with a "yes" – if so, which one are they reacting to? If you want to ask about both masters there is no other option but to ask two questions:
"Do you like listening to music by Chopin?" with the answer categories "yes" and "no" and
"Do you like listening to music by Wagner?" also with two answer categories "yes" and "no"
A double stimulus is a "technical" error when formulating questionnaires. A further "technical" error which often occurs can be questions with a linguistically negative formulation (Please note: not to be confused with negative content and linguistically positive items such as "I hate my job" or "I don’t like doing favors for others").

Survey tip 5: Avoid assumptions and suggestive questions!
Assumptions in the formulation of questions can lead to the respondents being unable to answer properly, because while they agree with the actual statement they do not agree with the part of the sentence which includes the assumption (this also applies to questions which contain explicit premises which you must agree with before you get to the "actual" question. For example:
"Does the lack of respect schoolchildren have for their teachers, in your opinion, influence everyday teaching methods in schools?"

In the formulation of this question it is declared to be correct that schoolchildren have a lack of respect for their teachers.
This may even be factually correct. However, how should the respondent react if they do not share this clear assumption? If the respondent does not agree that schoolchildren have a lack of respect for their teachers they won’t (be able to) answer the question.

Suggestive questions have the disadvantage of forcing the respondents into a corner and reducing their freedom to answer the questions. Here is an example:
"Leading scientists believe that car exhaust emissions can hinder the growth of children. Do you think this view is correct or do you think it is wrong?"

Other comparable types of phrases are "Most people..." or "It has been well documented that..." or "as is well known...".
Phrases such as these either lead to the respondent lacking the courage to contradict the authority of the statement or the majority of "the others", so they adapt their answer. It can also lead to people reacting to the restrictions when answering the question by quite deliberately going against the opinion of the experts or the majority. In both cases the actual opinion is not given, rather the respondent adapts or rejects the general opinion. So the suggestion should be removed from the formulation of the question and instead you should ask: "Do you think the statement, car exhaust emissions can hinder the growth of children is right or wrong?"

Survey tip 6: Avoid questions which target information to which many respondents probably don't have access!
As with the first rule, the implementation of rule 6 is also largely dependent on the target group you want to question. Answering the question...
"Have there been measures in your community with regard to implementing the Local Government Act 2000?" should (hopefully!) not be a problem for local politicians, but for the "average person" it is barely possible to answer it, not only because of the term "Local Government Act 2000" but also because they will not have had to deal with this question until now. They will probably not have the information at their disposal which is necessary to answer the question (what is the Local Government Act 2000? how does it affect the community?) The problem is intensified for general knowledge questions. The question...
"Can you name the leader of Liverpool City Council?" would not be answered correctly by everyone in Liverpool. The answer is (as of: April 2009) "Warren Bradley" and a country-wide survey would reveal even less correct answers. So, for questions which involve facts or knowledge you should always bear in mind the extent to which the target group would have access to the information in order to be able to answer the question sufficiently. If there is any doubt it is better to leave the question out, unless the actual point of the question is to establish knowledge or lack of knowledge.

Survey tip 7: Use questions with a clear temporal reference!
If you want to use questions involving facts or opinions within a certain period of time in the past, present or future you must define the time frame. It is important that the related temporal reference is clear.

Clearly ambiguous formulations such as "... recently...", ".. in the near future..." or”. Previously.." and also "... during your childhood...", are completely at the discretion of the respondent and what they understand by them: "recently" can refer to the last three weeks or it can also mean the last three years

The following types of formulation are also ambiguous: "during the last week" (does it refer to the calendar week or the previous seven days before the survey?) or ".. when you were 16 years old" (does this refer to the beginning of the 16th year of your life or the end or something else?).
On the other hand, unambiguous formulations can use specific dates or times as an "anchor", for example: "... since the 1st of April", "..on your 16th birthday", ".. up until the 31st of January." or ".. in the last three working days...". Of course the use of such dates or times does not guarantee that the respondent will actually be able to confine their answers to these periods of time, but they do specify the time in which the answer should be confined.

Survey tip 8: Use answer categories which are exhaustive and disjointed (free from overlap)!
Answer categories are disjointed if each person can assign one of them with absolute certainty (provided of course that multiple answers are not allowed). In the case of the question...
"How many lectures on the subject of "healthy living" have you attended so far in the year 2000?" with the answer categories "none" – "one lecture" – "two to five lectures" – "five or more lectures?" those who have attended exactly five lectures will have difficulty answering the question as they could choose both the third and the fourth answer categories.

The following answer categories are disjointed: "none" - "one lecture" - "two to four lectures" - "five or more lectures"?
Answer categories are not exhaustive if the answer which a specific person would like to give is not covered by the answer categories. Example:
"How many hours do you spend developing questionnaires in a normal working week?" none at all – less than three hours – three to less than five hours – between five and ten hours?"

Anyone who is fortunate enough to spend more than 10 hours developing questionnaires in a normal working week would not be able to give a correct answer. That’s why the scale should be changed:

none at all – less than three hours – three to less than five hours – five to less than 10 hours – 10 or more hours.
Anyone thinking this appears somewhat wooden is absolutely correct. In this case you could forego the answer categories and ask an open-ended question. This also applies to the previous question about the "healthy living" lectures.

Survey tip 9: Make sure that the context of a question does not have an (uncontrolled) effect on the answer!
We now come to the rule which in practice causes the most difficulties and is least controllable. The fact that questions and their corresponding answers can have an effect on follow-up questions is indisputable and fully proven. But which questions have an effect on follow-up questions? The answer is that we can often only speculate about this when formulating questionnaires. Only a pretest or in the worst case scenario only the data of the questionnaire provide information, provided that mechanisms (e.g. different versions of the questionnaire with different preliminary questions inserted before the question you are interested in)have been built into the questionnaire, which can control context effects.

We can demonstrate this with an example. In order to do this we must make an exception and give a reference, the following example can be found in Schwarz & Bless (1992)1:
The question...

"What is your general opinion of the CDU (Christian Democratic Union)?"

with an answer scale from 1 = "very negative" to 11 = "very positive"

we obtain quite different average values depending on the preliminary question:

A) "Do you know which position Richard von Weizsäcker holds outside party politics?" - average value 3.4
B) Preliminary question without political content – average value 5.2
C) "Do you know which party Richard von Weizsäcker has been a member of for over 20 years?" - average value 6.5

Depending on whether the well-known and popular former President of the Federal Republic of Germany Richard von Weizsäcker is excluded from the CDU(variation A) or included in the CDU (variation C) the CDU is rated with or without Weizsäcker – with a remarkably different result. The example is persuasive, but it is elaborately constructed and cannot be generalized. Context effects in questionnaires can be anticipated by reflection, but can only be proven by systematic tests.

Make sure when designing your questionnaire that some questions do not influence other questions; in case of doubt a systematic test is advisable.

1: Schwarz, Norbert & Herbert Bless (1992): Assimilation and Contrast Effects in Attitude Measurement: An Inclusion/Exclusion Model. P. 72 - 77 in: Advances of Consumer Research 19

Survey tip 10: Define unclear terms!
Terms are unclear if they are not understood at all by respondents or not understood by all respondents in the same way. The problem of unclear terms is similar to the problem of simple terms in rule 1and the problem of the availability of information in rule 6. Basically there is only one solution: the strategy is not to simplify the question (sometimes it just isn’t possible) or to adapt the question to the target group (this would make the term unclear), but rather to define the unclear term. Let’s look at the following question:

"At what age do you think the andropause begins for men?"

There would be no problem with the question if we were to direct it at a random sample of andrologists (medical specialists for the treatment of male sexual diseases). For everyone else the term andropause would have to be explained. This can be done, according to the random sample, for example, by comparing it to the menopause or even more generally:

"The Andropause is a development in the ageing process of men, which is comparable to the menopause in women. What do you think...".

Or even more clearly:

"The term andropause describes the process of hormonal changes to men, which can affect their emotions and sex life. It is comparable to the female menopausal process, also known as the change. What do you think...". This example ends our short introduction to the problems of creating survey questions. As we said at the beginning, general rules can be extremely useful for the development of a questionnaire, but they must always be adapted to the specific situation of a specific survey and the specific questionnaire. So pay attention to the “10 rules of formulating questions" when developing your questionnaire questions, but view them critically when asking yourself if your specific questions really are “good”.