Posts Tagged ‘outcomes’

Creating a new habit: Incorporating program evaluation into your daily operations

You are already busy enough. In fact, you’re busy running your programs. You don’t want to steal time away from actually doing the work and spend it on evaluation. Let’s face it: evaluation takes staff time, some expertise and money.

At the same time, you know that evaluation is at the very least a necessary evil. I’ve been hearing this comment repeatedly, “More and more funders are demanding information about outcomes, not just outputs.” And, in your heart of hearts, you know it can be a force for good if you use it to improve your program.

So, what to do? Simply start asking yourself two key questions on a regular basis. For any and all of your programs, new or ongoing, ask yourself:

What are we trying to achieve with this program?

What will I see and hear that will indicate to me whether we’re achieving what we want to achieve?

Let’s take an example from my home life. If I were planning a yard sale and I asked myself the first key question (What did I want to achieve?), I would tell you that I wanted to

a) get rid of all my useless stuff,

b) make a little spending money,

c) accomplish (a) and (b) without driving myself nuts and wearing myself out in the process.

If you then asked me the second key question (What would I see and hear that would indicate that I was achieving my desired outcomes?), I would say that at the end of the day, I would see an empty yard and a full cash box, and I would not be exhausted from the process. As far as quantifiable outcomes, I might tell you that 80% of my stuff would be gone from the yard and there would be $50 in the cash box.

So, what is the significance of these two key questions, and why are they so powerful when it comes to incorporating program evaluation into your day-to-day operations? Simply because program evaluation is, above all, more than using surveys and interviews and focus groups to measure outputs, outcomes and impacts. Program evaluation is a mindset. Program evaluation is a manner of thinking in evaluative terms. When you start asking yourself about achievements and predicting what you will see and hear that will help you understand your achievements, you are thinking about–-no, you are actually doing–program evaluation.

At this point you may be wondering, “What good is a mindset when my funders are asking for numbers and touching stories about how we are changing lives? And more numbers?”

There are two advantages of having and practicing a mindset:

Without it, program evaluation is pure tedium, and besides, you’re probably doing it wrong and thus wasting your resources.

With a mindset of evaluative thinking, you are on the right track to the reliable numbers and valid touching stories that your funders want.

To start practicing right away, ask yourself the two key questions in the following situations:

at a staff meeting as the season’s work begins.

at a board meeting when considering which program(s) to cut and which to grow.

with a funder who shows you an RFP for a potential program which may be a good fit for your organization.

with your colleagues in social situations while you’re brainstorming ways to change the world for the better.

by yourself on a weekend when you can’t help wondering why a particular program feels stale and another one animates you whenever you think about it.

I guarantee that if you begin asking yourself these questions on a regular basis (this means at least a couple of times each week), evaluation practices will naturally follow. Without any additional drudgery on your part, you’ll find yourself doing things like…

debriefing programs with your staff by asking specific questions such as, “What did you see and hear, and what does that tell you about whether we’ve achieved what we wanted to achieve?” These are even more powerful than the excellent common question, “What worked and what didn’t work and what should we do differently next time?”

building a budget for a proposed program that allocates 11 or 12 hours per week of your program director’s time instead of 10 hours, so that he or she has some additional time to think evaluatively.

designing and distributing quick surveys at your events. There’s a strong possibility that you already do this, and it’s equally likely that your current survey isn’t asking the questions that get at what you really want to know. Once you’ve asked the two key questions about any program, you’ll naturally refer back to those questions and your surveys will become significantly more useful.

Before you know it and without any sense of resistance or undue burden, you’ll be doing exactly what you need to do to make your funders happy and gather useful information to improve your program.

So, start incorporating program evaluation into your day-to-day operations today. Place the two key questions on your computer desktop so that you’ll see it every day. Then, take a minute to focus on one of your programs and begin your new habit by asking yourself, “What are we trying to achieve with this program?”

Resources:

If you would like to participate in an online Google Doc forum to help build the habit of asking yourself the two key questions, go to http://www.maggiemiller.org/, then go to the Links Page, then follow the link to the Google Doc.

See also:

Level Best

Leap of Reason

Leave a reply

The grant proposal: one document – several audiences

No matter how sophisticated your grant seeking process is or your foundation relationships are, you seldomly have the chance to ask program officers or foundation board members to tell it like it is. And if they do, how often will the answer be filtered for their own purposes? I asked former foundation CEO and featured author, Martin Teitel, about the proposal screening process.

CausePlanet: What would grant seekers find most surprising about how their proposals are handled once submitted?

Martin Teitel: It’s often the case that incoming proposals are moved up the staff hierarchy, from bottom to top. So the people who are most distant from actual decision-making do the greatest amount of screening. Picture the process as funnel-shaped: proposals are rejected, in many cases, at each level as they move along. This fact is one of the reasons writing proposals is so difficult: you have to entice the first readers, so you can stand out from the throng. But the same document then needs to later impress a steely-eyed program officer who will push hard against the details. And the proposal might eventually have to wow a foundation board. One document – several distinct audiences. Writers of successful proposals should give themselves great big pats on the back for making it through this thicket. And by the same token, people who worked hard for a long time, only to have their proposal rejected by a form letter, should try to not take it personally, because getting through the proposal mill is a thorny combination of chance and arcane skill.

You can read the complete interview in our Page to Practice summary feature of “The Ultimate Insider’s Guide to Winning Foundation Grants” by former foundation CEO Martin Teitel this week at CausePlanet. Or, you can learn more about this book and others at www.emersonandchurch.com.

CausePlanet subscribers: Don’t forget to register for the author interview on Wednesday, August 29 at 11 a.m. CST.

See also:

The Foundation: A Great American Secret; How Private Wealth is Changing the World

Leap of Reason: Managing to Outcomes in an Era of Scarcity

Level Best: How Small and Grassroots Nonprofits  Can Tackle Evaluation and Talk Results

Leave a reply

Request free copies of “Leap of Reason” for your board and funders

Rarely do I have the opportunity to tell my readers they can request free print copies of books we feature. This is one of those opportunities. I asked Leap of Reason author, Mario Morino, about his advice for making the case for overhead support in our CausePlanet interview. His answer will give you glimpse of what the rest of the book delivers.

CausePlanet: What advice do you have for nonprofit leaders who want to make a case for overhead support so they can engage in more meaningful information gathering to drive relevant outcomes?

Mario Morino: Great question. I don’t want to sound self-serving, but I would encourage them to write to us at info@leapofreason.org for free print copies of Leap of Reason they can distribute to their boards and key funders. Here are some relevant passages from the book they might want to bookmark and highlight for their key stakeholders:

Page 2: “The cold reality is that in our present era of unsustainable debts and deficits, our nation simply will not be able to justify huge subsidies for social-sector activities and entities without more assurance that they’re on track to realize results. Public funders—and eventually private funders as well—will migrate away from organizations with stirring stories alone, toward well-managed organizations that can also demonstrate meaningful, lasting impact.”

Page 41: “The magnitude of the combined hit—greatly reduced funding and increased need—will require organizations to literally reinvent themselves. Incremental responses will be insufficient. I agree wholeheartedly with Dr. Carol Twigg, President and CEO of the National Center for Academic Transformation, who concludes, ‘We will have to produce significantly better outcomes at a declining per-unit cost of producing these outcomes, while demand for our services will be increasing.’”

Page 42: “We need to be much clearer about our aspirations, more intentional in defining our approaches, more rigorous in gauging our progress, more willing to admit mistakes, more capable of quickly adapting and improving—all with an unrelenting focus and passion for improving lives. It’s no longer good enough to make the case that we’re addressing real needs. We need to prove that we’re making a real difference.

Email info@leapofreason.org for your free copies of Leap of Reason or learn more by visiting our summary library.

See also:

Level Best

Nonprofit Sustainability

Leave a reply

Look at “what and why” instead of “how”

Leap of Reason is a bold and wise look at a persistent problem in the nonprofit sector by one of our leading philanthropists. Managing to outcomes requires nonprofit leaders to take a candid look at what and why they measure instead of how. No one is left out of the equation in Morino’s analysis. Whether you represent government, business, or nonprofit, you’ll find Morino’s insights deeply provocative. While it’s impossible to predict how dismantled our economy will be in the coming years, we can ensure nonprofits are more durable than ever by making our outcomes indispensible through purposeful and enlightening measurement.

In our CausePlanet interview, I asked Mario Morino about the set of conditions organizations must possess before they can successfully manage to outcomes. Here’s what he had to say:

CausePlanet: You explain the real challenge in managing to outcomes is that organizations need a set of prerequisites: an engaged board, leadership with conviction, clarity of purpose and a supportive performance culture. These conditions appear to be best tackled at the top. Have you seen boards and CEOs successfully self-diagnose their level of engagement or conviction?

Mario Morino: I agree with your premise. The top of the organization must value high performance and lead the way on the changes required to get there. That’s not to say you can’t get an initial spark from elsewhere in the organization. I’ve seen that happen a number of times. But if the top leadership doesn’t help to kindle that spark, leading by its own example, then the fire for performance will die out quickly.

And yes, I have seen boards and CEOs self-diagnose their challenges and make the leap of reason! I’ve seen it up close quite a few times. For example, I saw this at the Lawrence School in Northeast Ohio, where I serve on the board and as an advisor, and some know me as “the parent from hell.” Lou Salza, a brilliant, passionate new headmaster and a highly committed board chair, Susan Karas, led a fundamental rethink and reinvention. I describe Lou’s role in Lawrence’s transformation in my recent speech, “Relentless: Investing in Leaders Who Stop at Nothing in Pursuit of Greater Social Impact” What I should have also pointed out was the important role Susan played and what happens when you have this kind of passionate, focused leadership leading the charge.

I’ve also seen rethinking and reinvention in organizations that did not have an infusion of new leadership, such as:

Camden Coalition of Healthcare Providers
Friendship Public Charter School
Maya Angelou Public Charter School
Roca
Saint Luke’s Foundation
Share Our Strength
Year Up
Youth Villages
The SEED School and others.

Watch for more highlights of our interview with author and philanthropist, Mario Morino, next week.

Leave a reply

Managing outcomes requires a leap of reason

Rather than take a blind leap of faith based on intuition or anecdotal information, author Mario Morino asks us to take a leap of reason when managing to outcomes. Why? Because our causes deserve it and social markets increasingly will fund only the nonprofits that demonstrate results.

Unfortunately, nonprofits aren’t great at managing to outcomes for a variety of reasons Morino explains in Chapter One of this book (request a free copy below).

Similar to other professions, nonprofit leaders aren’t rewarded for good management and consequently, have an acute shortage. Funders generally don’t provide financial support in order to make the leap to managing outcomes. Admittedly, nonprofits are cautious of managing to outcomes because they fear the information will be used against them rather than constructively for them. Among those who do try managing to outcomes get lost in the how to measure rather than the what and why.

I asked Morino about the benefit of good information in our interview:

CausePlanet: Your book claims the vast majority of nonprofits do not have the benefit of good information and tools to manage desired outcomes. Furthermore, you stress the importance of collecting better data to determine where you’re headed, chart a logical course, redirect when necessary and compete for funding. If some nonprofits are guilty of overmeasuring, why is there a disconnect with outcomes?

Mario Morino: There are some nonprofits that overmeasure, often because they are pushed by their many funders to provide a lot of data that help their funders check compliance boxes but don’t help the nonprofits themselves to navigate, learn and improve. And there are many nonprofits that undermeasure or don’t measure at all, perhaps because their leaders and board members are not asking all the hard questions they should be asking.

But I don’t want to dwell on the measurement part of this story. Measurement is a tool of effective management, not an end in itself. The macro point of Leap of Reason is that as a society, we’re not making nearly enough progress toward solving our big social and environmental challenges, and we desperately need to find better ways of encouraging, supporting, and rewarding high performance in our social and public organizations. In this era of scarcity, investing in high performance is not a luxury. It’s a necessity.

Join us next week for more of Morino’s author interview and our discussion of the prerequisites necessary for managing to outcomes.

Request a free copy of Leap of Reason for you, your board and/or your association by emailing info@leapofreason.org or visit our Page to Practice™ summary feature of Leap of Reason.

Leave a reply

Hire from within for your best evaluation team

One of your funders wants to see hard data, and another set of stakeholders needs touching stories that aren’t just cherry-picked anecdotes. You know you should evaluate your program. In fact, you genuinely want to know what’s working in your program and what needs to be tweaked. But you have a slightly queasy feeling in your stomach because despite your best intentions, you simply don’t have the time or expertise to conduct extensive program evaluation. Here’s the good news: you may already have the resources to do program evaluation right in your office.

Take a minute to list all your key staff, volunteers and hands-on-type board members.

Here’s the next step: See if any of them match any of these descriptions:

Clear thinker: someone who really “gets” the program.

Go getter: someone who doesn’t mind face-to-face interaction with perfect strangers and who knows how to just be aggressive enough to engage people without turning them off.

Accurate and tolerates tedium: a person who doesn’t mind doing slightly monotonous work and does it accurately.

Sees number patterns: someone who can look at a set of numbers and see patterns.  (You know these people when you see them in action.)

Sees comment patterns: this person can look at a set of comments and see what they have in common.

Good writer: a person with writing skills.

Champion: a person who believes in your program and has the ear of your stakeholders.

Here are five other requirements for people who help you with evaluation:

  1. They have to be able to commit some amount of time to their work and maintain that commitment.
  2. They have to be willing to put aside their own opinions about the program.
  3. They have to be teachable.
  4. They have to be able to maintain confidentiality.
  5. They have to understand the limits of their evaluation job so they don’t run amuck and make outlandish suggestions to your board just because they’re involved in evaluation.

Now, here’s a simple outline of a soup-to-nuts evaluation process:

Phase One: Planning for Evaluation (a two-step process)

Step 1: Develop a Logic Model (a 1-page description of your program’s activities, intended outcomes and how you think they relate to each other)
Step 2: Develop an Evaluation Plan (a list of the evaluation tasks you’ll do, such as surveys, focus groups, interviews, etc., that will help you understand your progress toward your intended outcomes)

Phase Two: Doing Evaluation (a six-step process)

For each evaluation task you do,

Step 1: Design the survey (for example)
Step 2: Distribute and collect it
Step 3: Enter the data
Step 4: Analyze and synthesize the data
Step 5: Write up a report
Step 6: Use the findings

Here are six hints about who can help with each task:

  1. Clear thinker can help you with your logic model.
  2. Go getter can help you distribute and collect surveys. With training, he/she can also conduct interviews (if he/she is removed enough from the program to be impartial).
  3. Accurate-And-Tedium-Tolerant member is just the person you need to do data-entry.
  4. People who can see number and comment patterns will LOVE helping you with data analysis.
  5. The writer can help you write your report.
  6. And when it’s time to use your data, you’ll need your champion.

When you recruit people with these characteristics, keep in mind that people (staff and volunteers) love doing what they’re good at in service of things they believe in. And most people, especially teachable ones, like learning new things. So, while they’ll be doing you a favor, you’re also giving them a great opportunity.

Here are three caveats.

  1. There’s an additional skill-set which is very specialized: facilitation. If you have a facilitator on your team, lucky you. Put him or her to work running a focus group (if he or she is removed enough from the program to be impartial).
  2. If you have stakeholders who should be involved but don’t have any of the characteristics listed above, it may be a good idea to find an appropriate role for them so they don’t feel left out.
  3. Even with all these great resources, it’s handy to bring in an evaluation expert who can help you anticipate costs and timing and help with designing your evaluation plan and instruments.

Yes, there’s a lot to think about. But you are already far ahead of where you were 10 minutes ago! Even if you’re not ready to jump into evaluation yet, you can be confident you won’t be diving in alone when it’s time to take the plunge.

See also:

Leap of Reason: Managing Outcomes in an Era of Scarcity

Level Best: How Small and Grassroots Organizations Can Tackle Evaluation and Talk Results

 

 

Leave a reply

Welcome! Please provide your log-in information below.
Forget your password?
Enter your email or user name and your log-in information will be sent to the email on file.