Evaluation frameworks offer starting point for grantmakers
There is growing interest in both strategic grantmaking and evaluation, as foundations and nonprofits become progressively more aware of how focused attention in these areas can serve their mutual best interests. Organizations such as the W. K. Kellogg Foundation and the United Way have been trailblazing this path since the late 1980s. Indeed, many grantmakers and nonprofits have made considerable strides towards demonstrating accountability and public benefit, but there’s still a long way to go, and there are some significant trends worth watching.
One is that foundations increasingly want not only to assess the effectiveness of the programs they support, but also to understand the broader community impacts resulting from their efforts. This type of inquiry goes beyond looking at the performance of one grantee or a cluster of grantees; rather, it aims to understand grantmaking strategy in its entirety.
Other foundations ask their grantees to conduct evaluations and report on their work, but rarely reflect on these results themselves in order to improve their own programs. Indeed, there is inherent difficulty in measuring community-level change, and foundations are wise to settle for recognizing their contributions to public benefit, rather than seeking attribution. Wishing to “improve – not prove,” funders are learning to adopt more systems-oriented views that consider factors such as socio-economic and political climate, as well as what other foundations are, or aren’t, doing.
Two frameworks for evaluation
To this end, two frameworks that are used to underpin much of the work of philanthropy – “Logic Models” and “Theory of Change” provide a base for evaluation plans. These frameworks link resources to results – they clarify goals, inform strategy and are important framing tools for both foundations and
nonprofits at two levels: programmatic and organizational.
Speaking at the Conference of Southwest Foundations 2008 Annual Conference, Peter Frumkin, director of the RGK Center for Philanthropy and Community Service at the University of Texas in Austin, explained that Logic Models first build a case for why one believes funding choices will lead to desired results. “What these models all share in common are causal claims about value creation: If we have these resources, we can do these activities; if we do these activities, we can generate this volume of work; if we do this work, we can achieve this level of outcomes; and if we have those outcomes, we’re going to have this social impact. All these models have in common an attempt to causally, clearly depict value creation.”
Used at the program level, Logic Models further support strategic planning and evaluation efforts. According to Frumkin, Logic Models “drive planning, guide implementation, connect to performance measurement, improve internal alignment and commitment, and secure external support. As funders, you want to have a clear claim about how you are going to make a difference. The second use of philanthropic logic models is to use them at the grantee level to help them clarify their work.”
Theory of Change
At the same conference, Nancy Csuti, director of Research, Evaluation & Strategic Learning at The Colorado Trust, presented her organization’s recent evolution through a Theory of Change process: “For about 20 years, the foundation did initiative-based grantmaking. They picked an area, researched it, developed the idea, and selected grantees who were going to work in that area. At the height of this effort, there were 25 different initiatives, each with lots of different grantees. There were advantages to this approach – it spread funding across the state, it addressed lots of different issues and we were able to respond to emerging issues.”
However, one disadvantage, according to Csuti, was that after conducting a communications audit of grantees and key leaders in the community to find out the impact of its grantmaking, the foundation discovered its efforts were incongruent and its ultimate goal unclear. As a result, the foundation has now articulated one clear overarching goal, which is “Access to Health for all Coloradans by 2018.” Future initiatives will now be tied more distinctly to that goal, and grantees will need to articulate within their applications how their work is going to lead to the goal of Access to Health. According to Csuti, “The Theory of Change built the case that funding is going to make the change we want to see.”
Evaluating strategies still evolving
Theory of Change and Logic Models also serve as a constructive base for evaluation design. According to the National Center for Charitable Statistics, over 900,000 nonprofit and religious organizations deliver many of our public services today – and many of these, through government or other funders’ requirements, are charged with monitoring and measurement. Unfortunately, many nonprofits and foundations have been scared off from more meaningful evaluation due to the rigor of “randomized control trials” and similarly severe research methodologies. Though far less stringent evaluation designs have been acknowledged as important and valid ways to collect and analyze data, the field on the whole is still a mystery to most. Yet, particularly in today’s economic climate, trends towards demonstrating efficiency and effectiveness are only likely to strengthen as funders continue to seek measurable results and return on investment.
There is no single way to conduct an evaluation or to measure outcomes. Rather, there is a full array of methodologies and tools suitable for various situations. It is strongly advised that foundations themselves seek expert advice and offer technical assistance to their grantees. In truth, grantmakers have been somewhat slow to accept evaluation as a value-added component of their work. According to Csuti, “I really want to encourage you [foundations] to examine your grantmaking and what you are doing – because in the busy-ness of just doing this work every day, critical thinking often is the thing that gets shoved to the bottom of the list. Most of us are used to doing what is urgent, not what is vital.”
colourbox.com, workforceplanning.com.au, gettingsmart.com
This article was originally published at CausePlanet on 7/16/09.