Most problems with questionnaire analysis can be
traced back to the design phase of the project.
Well-defined goals are the best way to assure a good
questionnaire design. When the goals of a study can be
expressed in a few clear and concise sentences, the
design of the questionnaire becomes considerably easier.
The questionnaire is developed to directly address the
goals of the study.
One of the best ways to clarify your study goals is to
decide how you intend to use the information. Do this
before you begin designing the study. This sounds
obvious, but many researchers neglect this task. Why do
research if the results will not be used?
Be sure to commit the study goals to writing. Whenever
you are unsure of a question, refer to the study goals
and a solution will become clear. Ask only questions that
directly address the study goals. Avoid the temptation to
ask questions because it would be "interesting to
As a general rule, with only a few exceptions, long
questionnaires get less response than short
questionnaires. Keep your questionnaire short. In fact,
the shorter the better. Response rate is the single most
important indicator of how much confidence you can place
in the results. A low response rate can be devastating to
a study. Therefore, you must do everything possible to
maximize the response rate. One of the most effective
methods of maximizing response is to shorten the
If your survey is over a few pages, try to eliminate
questions. Many people have difficulty knowing which
questions could be eliminated. For the elimination round,
read each question and ask, "How am I going to use
this information?" If the information will be used
in a decision-making process, then keep the question...
it's important. If not, throw it out.
One important way to assure a successful survey is to
include other experts and relevant decision-makers in the
questionnaire design process. Their suggestions will
improve the questionnaire and they will subsequently have
more confidence in the results.
Formulate a plan for doing the statistical analysis
during the design stage of the project. Know how every
question will be analyzed and be prepared to handle
missing data. If you cannot specify how you intend to
analyze a question or use the information, do not use it
in the survey.
Make the envelope unique. We all know how important
first impressions are. The same holds true for
questionnaires. The respondent's first impression of the
study usually comes from the envelope containing the
survey. The best envelopes (i.e., the ones that make you
want to see what's inside) are colored, hand-addressed
and use a commemorative postage stamp. Envelopes with
bulk mail permits or gummed labels are perceived as
unimportant. This will generally be reflected in a lower
Provide a well-written cover letter. The respondent's
next impression comes from the cover letter. The
importance of the cover letter should not be
underestimated. It provides your best chance to persuade
the respondent to complete the survey.
Give your questionnaire a title that is short and
meaningful to the respondent. A questionnaire with a
title is generally perceived to be more credible than one
Include clear and concise instructions on how to
complete the questionnaire. These must be very easy to
understand, so use short sentences and basic vocabulary.
Be sure to print the return address on the questionnaire
itself (since questionnaires often get separated from the
Begin with a few non-threatening and interesting
items. If the first items are too threatening or
"boring", there is little chance that the
person will complete the questionnaire. People generally
look at the first few questions before deciding whether
or not to complete the questionnaire. Make them want to
continue by putting interesting questions first.
Use simple and direct language. The questions must be
clearly understood by the respondent. The wording of a
question should be simple and to the point. Do not use
uncommon words or long sentences. Make items as brief as
possible. This will reduce misunderstandings and make the
questionnaire appear easier to complete. One way to
eliminate misunderstandings is to emphasize crucial words
in each item by using bold, italics or underlining.
Leave adequate space for respondents to make comments.
One criticism of questionnaires is their inability to
retain the "flavor" of a response. Leaving
space for comments will provide valuable information not
captured by the response categories. Leaving white space
also makes the questionnaire look easier and this
Place the most important items in the first half of
the questionnaire. Respondents often send back partially
completed questionnaires. By putting the most important
items near the beginning, the partially completed
questionnaires will still contain important information.
Hold the respondent's interest. We want the respondent
to complete our questionnaire. One way to keep a
questionnaire interesting is to provide variety in the
type of items used. Varying the questioning format will
also prevent respondents from falling into "response
sets". At the same time, it is important to group
items into coherent categories. All items should flow
smoothly from one to the next.
If a questionnaire is more than a few pages and is
held together by a staple, include some identifying data
on each page (such as a respondent ID number). Pages
often accidentally separate.
Provide incentives as a motivation for a properly
completed questionnaire. What does the respondent get for
completing your questionnaire? Altruism is rarely an
effective motivator. Attaching a dollar bill to the
questionnaire works well. If the information you are
collecting is of interest to the respondent, offering a
free summary report is also an excellent motivator.
Whatever you choose, it must make the respondent want to
complete the questionnaire.
Use professional production methods for the
questionnaire--either desktop publishing or typesetting
and keylining. Be creative. Try different colored inks
and paper. The object is to make your questionnaire stand
out from all the others the respondent receives.
Make it convenient. The easier it is for the
respondent to complete the questionnaire the better.
Always include a self-addressed postage-paid envelope.
Envelopes with postage stamps get better response than
business reply envelopes (although they are more
expensive since you also pay for the non-respondents).
The final test of a questionnaire is to try it on
representatives of the target audience. If there are
problems with the questionnaire, they almost always show
up here. If possible, be present while a respondent is
completing the questionnaire and tell her that it is okay
to ask you for clarification of any item. The questions
she asks are indicative of problems in the questionnaire
(i.e., the questions on the questionnaire must be without
any ambiguity because there will be no chance to clarify
a question when the survey is mailed).