Creating a new habit: Incorporating program evaluation into your daily operations
You are already busy enough. In fact, you’re busy running your programs. You don’t want to steal time away from actually doing the work and spend it on evaluation. Let’s face it: evaluation takes staff time, some expertise and money.
At the same time, you know that evaluation is at the very least a necessary evil. I’ve been hearing this comment repeatedly, “More and more funders are demanding information about outcomes, not just outputs.” And, in your heart of hearts, you know it can be a force for good if you use it to improve your program.
So, what to do? Simply start asking yourself two key questions on a regular basis. For any and all of your programs, new or ongoing, ask yourself:
What are we trying to achieve with this program?
What will I see and hear that will indicate to me whether we’re achieving what we want to achieve?
Let’s take an example from my home life. If I were planning a yard sale and I asked myself the first key question (What did I want to achieve?), I would tell you that I wanted to
a) get rid of all my useless stuff,
b) make a little spending money,
c) accomplish (a) and (b) without driving myself nuts and wearing myself out in the process.
If you then asked me the second key question (What would I see and hear that would indicate that I was achieving my desired outcomes?), I would say that at the end of the day, I would see an empty yard and a full cash box, and I would not be exhausted from the process. As far as quantifiable outcomes, I might tell you that 80% of my stuff would be gone from the yard and there would be $50 in the cash box.
So, what is the significance of these two key questions, and why are they so powerful when it comes to incorporating program evaluation into your day-to-day operations? Simply because program evaluation is, above all, more than using surveys and interviews and focus groups to measure outputs, outcomes and impacts. Program evaluation is a mindset. Program evaluation is a manner of thinking in evaluative terms. When you start asking yourself about achievements and predicting what you will see and hear that will help you understand your achievements, you are thinking about–-no, you are actually doing–program evaluation.
At this point you may be wondering, “What good is a mindset when my funders are asking for numbers and touching stories about how we are changing lives? And more numbers?”
There are two advantages of having and practicing a mindset:
With a mindset of evaluative thinking, you are on the right track to the reliable numbers and valid touching stories that your funders want.
To start practicing right away, ask yourself the two key questions in the following situations:
at a staff meeting as the season’s work begins.
at a board meeting when considering which program(s) to cut and which to grow.
with a funder who shows you an RFP for a potential program which may be a good fit for your organization.
with your colleagues in social situations while you’re brainstorming ways to change the world for the better.
by yourself on a weekend when you can’t help wondering why a particular program feels stale and another one animates you whenever you think about it.
I guarantee that if you begin asking yourself these questions on a regular basis (this means at least a couple of times each week), evaluation practices will naturally follow. Without any additional drudgery on your part, you’ll find yourself doing things like…
debriefing programs with your staff by asking specific questions such as, “What did you see and hear, and what does that tell you about whether we’ve achieved what we wanted to achieve?” These are even more powerful than the excellent common question, “What worked and what didn’t work and what should we do differently next time?”
building a budget for a proposed program that allocates 11 or 12 hours per week of your program director’s time instead of 10 hours, so that he or she has some additional time to think evaluatively.
designing and distributing quick surveys at your events. There’s a strong possibility that you already do this, and it’s equally likely that your current survey isn’t asking the questions that get at what you really want to know. Once you’ve asked the two key questions about any program, you’ll naturally refer back to those questions and your surveys will become significantly more useful.
Before you know it and without any sense of resistance or undue burden, you’ll be doing exactly what you need to do to make your funders happy and gather useful information to improve your program.
So, start incorporating program evaluation into your day-to-day operations today. Place the two key questions on your computer desktop so that you’ll see it every day. Then, take a minute to focus on one of your programs and begin your new habit by asking yourself, “What are we trying to achieve with this program?”
If you would like to participate in an online Google Doc forum to help build the habit of asking yourself the two key questions, go to http://www.maggiemiller.org/, then go to the Links Page, then follow the link to the Google Doc.