Hire from within for your best evaluation team
One of your funders wants to see hard data, and another set of stakeholders needs touching stories that aren’t just cherry-picked anecdotes. You know you should evaluate your program. In fact, you genuinely want to know what’s working in your program and what needs to be tweaked. But you have a slightly queasy feeling in your stomach because despite your best intentions, you simply don’t have the time or expertise to conduct extensive program evaluation. Here’s the good news: you may already have the resources to do program evaluation right in your office.
Take a minute to list all your key staff, volunteers and hands-on-type board members.
Here’s the next step: See if any of them match any of these descriptions:
Clear thinker: someone who really “gets” the program.
Go getter: someone who doesn’t mind face-to-face interaction with perfect strangers and who knows how to just be aggressive enough to engage people without turning them off.
Accurate and tolerates tedium: a person who doesn’t mind doing slightly monotonous work and does it accurately.
Sees number patterns: someone who can look at a set of numbers and see patterns. (You know these people when you see them in action.)
Sees comment patterns: this person can look at a set of comments and see what they have in common.
Good writer: a person with writing skills.
Champion: a person who believes in your program and has the ear of your stakeholders.
Here are five other requirements for people who help you with evaluation:
- They have to be able to commit some amount of time to their work and maintain that commitment.
- They have to be willing to put aside their own opinions about the program.
- They have to be teachable.
- They have to be able to maintain confidentiality.
- They have to understand the limits of their evaluation job so they don’t run amuck and make outlandish suggestions to your board just because they’re involved in evaluation.
Now, here’s a simple outline of a soup-to-nuts evaluation process:
Phase One: Planning for Evaluation (a two-step process)
Step 1: Develop a Logic Model (a 1-page description of your program’s activities, intended outcomes and how you think they relate to each other)
Step 2: Develop an Evaluation Plan (a list of the evaluation tasks you’ll do, such as surveys, focus groups, interviews, etc., that will help you understand your progress toward your intended outcomes)
Phase Two: Doing Evaluation (a six-step process)
For each evaluation task you do,
Step 1: Design the survey (for example)
Step 2: Distribute and collect it
Step 3: Enter the data
Step 4: Analyze and synthesize the data
Step 5: Write up a report
Step 6: Use the findings
Here are six hints about who can help with each task:
- Clear thinker can help you with your logic model.
- Go getter can help you distribute and collect surveys. With training, he/she can also conduct interviews (if he/she is removed enough from the program to be impartial).
- Accurate-And-Tedium-Tolerant member is just the person you need to do data-entry.
- People who can see number and comment patterns will LOVE helping you with data analysis.
- The writer can help you write your report.
- And when it’s time to use your data, you’ll need your champion.
When you recruit people with these characteristics, keep in mind that people (staff and volunteers) love doing what they’re good at in service of things they believe in. And most people, especially teachable ones, like learning new things. So, while they’ll be doing you a favor, you’re also giving them a great opportunity.
Here are three caveats.
- There’s an additional skill-set which is very specialized: facilitation. If you have a facilitator on your team, lucky you. Put him or her to work running a focus group (if he or she is removed enough from the program to be impartial).
- If you have stakeholders who should be involved but don’t have any of the characteristics listed above, it may be a good idea to find an appropriate role for them so they don’t feel left out.
- Even with all these great resources, it’s handy to bring in an evaluation expert who can help you anticipate costs and timing and help with designing your evaluation plan and instruments.
Yes, there’s a lot to think about. But you are already far ahead of where you were 10 minutes ago! Even if you’re not ready to jump into evaluation yet, you can be confident you won’t be diving in alone when it’s time to take the plunge.