Recently, Charity Navigator asked members of its Advisory Panel to provide commentary on an important question: “What can donors do to gauge the effectiveness and results of charities?” I chose to answer this question in the context of social service charities since that is where my expertise lies and will address this question here.
Despite over a decade of discussion about clear deliverables and outcomes in the social service sector, there is still a lack of clarity about effectiveness. The first problem, and an important one, is that tracking effectiveness is expensive. Most small and mid-size nonprofits do not have the resources to undertake rigorous outcome research, and few donors are willing to pay for it.
Even among the larger nonprofits able to invest in robust research, the question is often asked, “should we be spending money on research at a time when so many in our community need so much more than what’s available?” It is an important question and one that we should never stop asking ourselves.
We come to this work to meet human need and to make the world a better place for all of us. We should strive to direct the highest percentage of every dollar invested as a direct benefit to those we serve. We certainly need more of almost everything.
But we cannot let the “needing more” and “doing more argument” become an easy excuse for not measuring and knowing effectiveness. Our donors and funders have a right to know if our interventions are truly making the world a better place or if they are merely maintaining the status-quo. More importantly, the children, youth and families we serve have a right to interventions that produce a positive impact in their lives.
Another reason we fail is because we are unwilling to identify and to state univocally what anticipated effectiveness is. Maybe it is the fear of embarrassment and failure. Maybe it is because we are unwilling to draw a cause and effect relationship between our activities and the end result. Or maybe it is because we have no idea what effectiveness looks like.
In all of these conditions, it is easier to simply describe the quantity of what we deliver than to risk describing a bottom-line expectation of effectiveness. Let me give you an example from the foster care and juvenile justice services sector.
Both foster care and juvenile justice are temporary interventions aimed at improving child/family safety and functioning during a defined service period. But describing the quality and quantity of the interventions provided during the temporary service period, while educational, does not inform us of effectiveness. Is the number of home visits by a culturally sensitive worker important? Yes, home visits are very important. But, more important is what happens once those home visits end. The nonprofit’s willingness to clearly identify the desired end result is the first step to measuring effectiveness. There is no question that in foster care and juvenile justice, the only dependable gauge of effectiveness remains the children/families’ functioning during the months and years following the service phase–namely, long-term behavior, adjustment and community integration during the post-service phase.
At The Children’s Village, we too struggle with this issue of effectiveness. But we have come to the conclusion that, while we cannot always do rigorous outcome research following standard research protocols, almost every intervention and service offered can and should clearly identify the results we seek to achieve, and we must collect data to measure the result.
This commitment led to us establishing an outcomes committee that works with and supports staff in this effort. And each year we publish an outcomes report for our donors. While there is always failure in every program, and sometimes entire interventions are deemed ineffective, knowing the rate of failure and pinpointing where the highest failure is occurring helps us reassess and refine the offerings.
Government can and should play a role too. Today, much of what we do is mandated and wrapped in layers of costly and often counter-productive regulation. Compliance with regulation drives the process, and the assessment of effectiveness is more-often-than-not described in numbers served and activities completed. Nonprofits are forced to spend an inordinate amount on compliance and the conversation of bottom-line effectiveness is lost in the demands of regulation compliance.
Government must find ways to reduce costly regulation. More importantly, government must begin to describe bottom-line expectation of effectiveness and concurrently, incentivize effectiveness over rote regulation compliance.
Sophisticated donors and funders interested in long-term efficacy and the impact of their funds invested in this crucial service continuum must request that charities provide follow-up data on clients served. But, be realistic in the expectations too. You don’t want an organization to spend a large share of its budget tracking outcomes.
Incremental change is what we need, more nonprofits committing to bottom-lining the expectation and willing to track sensible outcomes and more donors and funders willing to fund these efforts.
–Jeremy Kohomban is CEO of The Children’s Village, which provides residential and community services to children and families in New York.
