 Working
through the Data
6.4 Organize qualitative data in relation to
evaluation questions
Asking a few important questions of the data, and
doing that analysis carefully, is the best use of time
and resources for evaluation
The prime motivation for data collection and analysis
is to learn more about what is going on in programs
so they can be improved. As in other elements of community
programming, it is important to set priorities for
what can be accomplished within budget and resource
limits.
After reading through at least some of the qualitative
data collected, start to think about ways of relating
the data to evaluation questions and indicators.
Become familiar with the range of responses and ideas
raised. Data will raise questions that seem important
to try to analyze, ideas that may be generalizable
across respondents, (extent to which responses reflect
the whole group or a specific subgroup.) By organizing
and coding qualitative data, a researcher can compare
responses among sub-groups (e.g., by gender or other
known demographic data) or among responses and variables.
- Looking first for information needed to answer
questions raised in the planning stage:
- increases the speed of handling data and interpreting
results
- avoids the risk of being overwhelmed
by reams of data, charts and tables
- Develop priorities for which questions are
most important to the original intent of the evaluation.
The disadvantage of a narrow focus is that you can
fail to notice unexpected outcomes.
Questions to ask while reviewing the data:
- What information do the responses give about
what the evaluation was assessing?
- What ideas
are common to responses?
- How can
responses be grouped to say something
meaningful about the program— meaningful to
people in the program, people running
the program and the broader community?
- Do responses say anything
surprising?
- Do
responses from one question support
or contradict those from other questions and from
other data sources data.
Working through qualitative data: An example
A community recreation centre offering a recreation/homework
program might ask parents an open-ended question: “Why
did you enroll your child in this program?” and
obtain the following responses:
a) to give him something to do instead of video games
at home
b) to keep her out of mischief till I get home from
work
c) to relieve my mother who looks after the baby
d) to learn English more
e) access to a computer for schoolwork
f) for homework help
g) to make sure he does his homework before dinner
h) to get more physical exercise
i) the teacher suggested it to help with homework
j) to make friends
k) he thought it would be fun instead of staying
home
l) for the sports activities
m) an opportunity to socialize with other children
n) to get her away from the television
First, think about the reasons for asking this question.
Perhaps programmers developed the program to keep latch-key
children off the streets while their parents worked.
They advertise the program as providing a positive
alternative to hanging out on the street. Only response
(b) specifically supports the contention that the program
helps latch-key children. Two other responses (c) and
(g) might be categorized as ‘parents wanting
adult supervision for child,’ which is similar,
but clearly those children would not be out on the
street without the program. The level of interpretation
to be used with the data is set by the quality and
amount of data collected, as well as original discussions
with stakeholders about what they want to learn from
the evaluation.
Programmers want to assess the fit between what
they offer and parents’ needs in order to better
serve the community. They also wish to find out what
elements
attract participants so they can emphasize those
in advertising and presentations to increase their
numbers.
With those questions in mind, programmers decide
how best to categorize the data to focus the analysis.
A category like ‘seeks productive activity for
child’ would fit almost all the responses but
tells little about how to improve the fit with parents’ needs
or attract more children. Looking at the responses
with those objectives in mind, one might group and
code the responses as 1. “to provide academic
help” (d,e,f,i), 2. “to provide supervision” (b,c,g),
3. “for outgoing social activity” (j,k,m,),
4. “to increase physical activity” (h,l).
Responses (a) and (n) do not provide enough information
to put them in either of the last two categories though
one might interpret them as fitting in one or the other.
They could go into a catch-all category 5. “other”,
or “alternative to passive activity at home.”
An alternative, using less interpretation, is to
collapse fewer responses, giving each a separate
coding number except for those with almost identical
wording.
That would mean collapsing only (f) and (i), ‘homework
help ’ and
(j) and (m) ‘to make friends.'
|
