Nonprofit, You Can Get Much More Utility From Your Data

SPAINTER_VFX/SHUTTERSTOCK

I settled into the uncomfortable silence in front of about 150 nonprofit and government leaders in education. I had asked a question, one that I thought would generate the positive energy needed to carry the training forward: Imagine that money was no object. You have access to any data you want about the outcomes of your programs and those of other agencies who do similar work. What things would you want to know from that data? Participants shuffled papers and looked at their tables. After a few very long minutes of silence, they began to offer possibilities in their discussion groups.

At that moment, I realized that the data problem for many nonprofit and government agencies is not just about resources (although program evaluation is underfunded); it is also about knowledge and imagination. There are generally three ways that nonprofits use their data: compliance uses, negotiated uses and directive uses. Compliance uses are when nonprofit and government agencies collect and report data because they are mandated to — it’s a reactive use. Negotiated uses are when agencies collect and report outcome data to communicate to external stakeholders. In contrast, directive uses are when agencies collect data to hold themselves accountable and to learn.

The reason for collecting the data determines the type and quality of data that you collect. If you are collecting data for compliance purposes alone, you do what you’ve been told you have to do. If you’re collecting it for negotiated uses, the data is designed to show your agency in the most favorable way possible. These two uses aren’t bad — if you are required to collect and report outcome data you should keep doing that and it’s important to let donors hear testimonials from clients. But these uses of data won’t help your agency get better or increase your capacity.

Test your agency

Instead, directive uses help your organization learn and improve client outcomes. If you aren’t sure if your agency is collecting data for directive uses, try answering the following questions. Does the program outcome data you collect:

• influence the direction of management decisions?
• provide key information needed for strategic planning?
• show up in employee performance evaluations?
• influence budgeting decisions?
• influence how you interact with clients?

If you answered with an emphatic “YES!” to the above questions, you are using data for directive purposes.

In my own research, when nonprofits have systems that allow them to set program goals and then evaluate against those goals, there are remarkable benefits. Everything from board governance to external communication capacity improves.

Once you begin on this journey of directive data use, there are immediate benefits. Those benefits can be even further enhanced through three data practices:

Invest time in designing a robust data management system.

Be consistent in your data collection.

Develop benchmarking collaborations.

The first of these practices is perhaps the most mundane, but, I’ve discovered over the years of working with nonprofit and government agencies, it’s the least done. When collecting data consider your data management system. Robust data management systems include the following elements:

1. Every client has a unique identification code. All the data about that client, including which programs they participated in, their attendance at those programs, surveys they completed, and demographic data, should all be tied to that participant code. Note, this means that all the program evaluation data needs to be entered into a single database, not stored in various databases that aren’t linked.
2. Collect data about what happened to who. Systematic data about what clients received what services or participated in what programs, who ran those sessions, and for what duration need to be collected and entered.
3. Follow-up after clients leave. When possible, long-term outcome data, including data about clients after they matriculated from the program, should be entered into the system. This may involve surveying alumni or doing checks in administrative records for past clients periodically.

When you collect this type of data you can answer questions like:

• Which service providers’ clients have the best long-term outcomes?
• When a client participates in two programs concurrently, do they do better than clients who only participated in one program?
• How often do clients have to attend the program to receive benefits from it?
• How has our client population changed over time and how has that affected the outcomes of our program?

Imagine the possibilities.

Second, be consistent in your data collection practices. Sometimes program evaluation and data collection can fall by the wayside as other priorities come up. However, when evaluations are done at inconsistent times during programs or different measures are used year to year, it hampers your ability to draw conclusions about what’s working and not working in your program. In the most robust programs, leading indicators of the progress you hope your agency is making should be collected along the way. This allows you to identify clients who might need greater levels of intervention.

Third, and finally, to get the most out of your data, don’t go it alone. Find other agencies that are willing to share their data so that you can benchmark your progress and learn from others. Kate Cooper and I have written about the Chicago Benchmarking Collaborative before as one example of such a data-sharing system.

Data, collected for directive uses and in the right way, can be one of the most powerful tools for organizational improvement that nonprofit and government leaders have at their disposal. Take a minute and answer the question for yourself: Imagine that money was no object. You have access to any data you want about the outcomes of your programs and those of other agencies who do similar work. What things would you want to know from that data?

Michelle Shumate (@ProfShumate) is a professor at the School of Communication at Northwestern University and the founding director of the Network for Nonprofit and Social Impact, a lab dedicated to answering the question: How can nonprofit networks be rewired for maximum social impact?