Preparing the Groundwork
for
Evaluation
1.2
Define the evaluation purpose and scope
Communities are
coming to understand that steps taken to
help children’s
development can have long lasting consequences
for their health, happiness and
success as adults. Evaluations that look at the effects
of programs launched to help youth can provide data
to validate programming efforts and directions and
guide programming change. Some types of evaluation
do not consider program effects.
What is the evaluation
purpose? What is to be evaluated?
This toolkit provides guidance for
extending evaluation to include outcomes.
At the outset
of planning, understand the different types
of evaluations, what you and others
expect
to achieve and set parameters accordingly
to avoid later
confusion and encourage a shared purpose
among stakeholders.
Types of evaluations include: service
evaluation, process evaluation, organizational
review, outcome
evaluation.
- service evaluation (also called program
evaluation) looks at service
delivery,
- what is delivered
to participants
- how it compares
with what the organizers promised would
be done
- customer service –quality
of participants’ experience
and satisfaction
- process
evaluation looks at how/how well the
program is delivered
- what relationships
are involved
- how well they
operate
- addresses weaknesses,
e.g., what is not being done and why
- an organizational
review looks at organizational set-up
for service delivery
- roles and responsibilities
of staff
- resources
- performance
- training
needs
- outcome evaluation looks beyond service delivery
- to assess a program’s
value to participants,
in terms of
- what benefits
participants and others gained
from program,
- what changes may
have occurred in them (knowledge,
skills, attitudes and behaviour)
- possible ripple
effect in the family, classroom
and/or
community
- basic questions
include:
- Is the program
meeting the needs of participants?
- Why might
it not be?
- Is the strategy
working as planned?
- Is the program
reaching the target audience?
For whom and
why is the evaluation being done?
Motivating
Factors
Motivations may
include:
- funders’ requirements
- public relations
- to fulfill partnership
demands or
- for a research
agenda.
Evaluations of
existing programs may also be called for
- to
encourage sustainability
OR
In response to
- specific questions
raised in the community
- observations raised
by staff, both positive and negative
Intended
Benefits
- trustworthy information
that will be meaningful to staff, volunteers
and families
- a transparent
process
- greater feeling
of input from community stakeholders
- community
support
- data may allow
stakeholders to consider how programming
benefits can
repay present, or offset future, costs
What
should be included?
“Be sure
that whatever you do is useful to
the project itself, not
just an add-on to
keep funders happy. Think: what do we who
are delivering the program want to know to
help us get smarter faster?
Build on that.”
– Dr. Fay Martin, Executive Director,
Family Services of Haliburton County
Whatever
initiated the evaluation will dictate to some extent the
questions asked Building-block data for evaluation
- description of program activities,
times, duration, venues
- participant data:
registration, full name, contact information
- volunteer data: application data,
training material
Record what and how programming was delivered in order to learn more about what types of activities and strategies can be most effective.
- program materials
- costs (actual
and donated)
- outcomes data (See Step 2)
Monitoring
program costs
Monitoring and regulating costs
are aspects of accountability: determining
how much a program costs to deliver benefits
to the community.
Consider the total cost
of remounting a program.
Track
actual spending and the value of donated goods and
services, e.g.,
- resources
- facilities
- volunteer time
- discounts and
subsidized costs
Programs will not be copied and operated in other areas, despite evidence of effectiveness, if they are too expensive for most communities to run
Donated contributions reflect
the value of staff and
community support to your program:
- useful knowledge to aid
fundraising
- show areas of vulnerability and
strengths that could
have a bearing on sustainability
|
