 Developing
New Data Collection Tools
4.4 Modify existing questions to
suit respondents
Consider both the ability and willingness of respondents
to answer the questions, in terms of, e.g.,
It is important that questions are interpreted the
same way by all respondents, so they are all answering
the same question —the one you meant.
- language, reading level and specialized vocabulary
- knowledge
- cultural experience, attitudes
- community context
Language and vocabulary
Translation
When language comprehension is a problem and a written
questionnaire is necessary:
- translation of questions into the most familiar
language for informants is a good option but may introduce
new
errors.
- a helpful check is to have a second person translate
questions back into the first language (without
seeing the original)
- compare results of re-translation and discuss
implications of any differences.
Other options
Options followed by youth programs
- have an oral interview conducted by a staff person
or volunteer who speaks the same language
- children often act as translators for parents in
community programs
- someone else may be able to explain the questionnaire
at the time it is being completed
- errors may creep in because there is no way of
ensuring the explanations are sufficiently correct
or complete
Inappropriate vocabulary
Inappropriate language may have a negative effect
on respondents’ willingness to answer a question
- possible problems in vocabulary include:
- jargon or specialized meanings
- convoluted phrases open to more than one interpretation
- juvenile language that undermines the serious intent
of the questions
- culturally offensive terms
Generally a grade 4-6 vocabulary level is considered
simple enough without seeming too juvenile. (Available
software programs can determine vocabulary level.)
Knowledge
View each question from the viewpoint of the respondent,
e.g., youth participating in the program, parent, teacher,
volunteer or tutor.
Clarify whether a question should ask about attitudes
or about behaviour and which is more likely to provide
the most meaningful information. Examples of behavioural
change can be a strong indicator of program effectiveness.
Reporting on behaviour may also be less subjective
or influenced by a respondent’s support for
the program. Questions about observed behaviour can
be asked of several sources to build a composite picture
and strengthen data.
- Can each question be answered easily, without
much mulling over and second-guessing?
- Is it equally easy for all informants to answer
the question?
- Does the respondent have the knowledge to answer
the question?
- Does the question ask about a concept that needs
a definition or some example?
- Does the question ask about something outside
the informant’s sphere of experience, e.g., asking
parents about how children are behaving in the classroom
or teachers about conditions or behaviour in a child’s
home?
- Is the question culturally appropriate or sensitive
to cultural, religious or ethnic differences?
For example, the terms synagogue, mosque or simply “religious
services” might be included to expand a question
about church attendance.
- Does the question ask for secondhand information?
Need an example?
For example, “How did your child/student enjoy
the program?” really means,
“How do you think they enjoyed the program?” The
reliability of the response could vary depending on
the parent or teacher’s level of knowledge and
relationship with the child. Firsthand information
can be obtained by reworking the question, e.g., “Has
your child/student ever expressed any feelings about
the program? Yes/ no If yes, what did they say?”
Community context
Words may assume different meanings in different
contexts, providing stumbling blocks to confuse data
analysis. Specific terms may be unknown in a different
jurisdiction. It is helpful to check terms with a similar
audience to the informants and convert them to local
usage. Obvious examples are U.S. compared to Canadian
usage, in currency, terms such as food stamps vs. food
banks or government programs such as the U.S. National
School Lunch Program.
Questions should be unambiguous. Less obvious problems
may be arise from questions that are inappropriate
for the local context. Asking a general or open-ended
question allows a fuller range of responses that
can be classified later. Focus groups or more general
discussions
with respondents before writing a questionnaire also
helps determine the full range of options to be considered.
Need an example?
For example, a researcher inquiring about relationships
with non-school activities and violence asked youth in a particular
area about membership in non-school clubs. Since there
were few non-school clubs available in the area, the number
of
positive responses led the researcher to suspect that
some youth had considered gangs to be non-school clubs! It
can
be useful to add examples. In the above case, adding, “e.g.,
Boy Scouts, 4-H, Boys and Girls clubs” might have given
the researcher more confidence in the data. What if youth
also have access to athletic clubs for golf, tennis, swimming,
curling or gymnastics? Is membership in those clubs any different
from membership in the YM/YWCA, which respondents would likely
not consider a “club”? Beginning with a more general
question, then asking for specifics is another way of avoiding
ambiguity.
For example, “Do you belong to a non-school group or
club where you spend time outside school? If yes, please
specify what it is.”______________________________.
|
