For all the talk about overhead expense ratios being a bad way to decide if a charity is any good, there aren't many tools available that help donors do better evaluations. There are organizations like Philanthropedia, GiveWell, Root Cause, New Philanthropy Capital and GreatNonprofits, which offer information, but as a group they only cover a tiny sliver of organizations. Charity Navigator is overhauling their rating system in order to move beyond overhead expense ratios, but even they only rate 5,500 of the literally million-plus nonprofits in the country.
So what to do if you are wondering if your local after school tutoring program or a nearby homeless shelter is worthy of support? Last month, I published a piece in the Chronicle of Philanthropy that offered five questions that donors can ask nonprofits, but I intentionally designed the questions to apply to a huge range of organizations.
A great new tool has now been launched by GiveWell. Designed as a Do-It-Yourself Evaluation kit for the every day donor, GiveWell offers cause specific questions that donors can pose to nonprofits they are considering supporting.
Rather than trying to design some sort of quantitative system of analysis, GiveWell rightly recognizes that the average donor can learn a great deal about a nonprofit by asking a series of important, but relatively simply questions.
An example:
Issue Area: K-12 Education
GiveWell's Overview of Issue Area
Example Organizations: Children's Scholarship Fund, KIPP, Teach for America
Questions:
- What do you do to improve K-12 education? What is your relationship with the school? Do you work within it or outside it?
- What academic literature on education – particularly randomized controlled trials – exists on the type of intervention you are conducting?
- Who is targeted by your activities? What are the requirements for participation? In the case of over-subscription, how do you determine who gets in?
- Have you done a randomized controlled trial of your program? If not, are you planning to? If not, why not?
- Have you collected any systematic data on student satisfaction / retention? How happy are students with the program? What do they wish were improved? How often do they drop out and for what reasons do they drop out?
- To what extent do you stay in touch with students? Have you systematically collected (and can you share the reports on) information regarding any outcomes, particularly test scores & graduation rates?
- Have you tried to assess the impact of your program on later life outcomes, compared to how participants would have done without the program?
- How much has been spent on this program? How many students have been served?
- How would your activities change if you had more revenue than expected? Less? Would more revenue translate directly into more students served, and up to what point?
GiveWell is generally focused on assessing the evidence of impact: the strength of information about program results and the degree to which they prove that the program is working. I tend to be more focused on assessing organizational performance: the strength of the information about how the nonprofit operates and the degree to which that information indicates that they are performing at a high level. What I like about GiveWell's questions is that they can be used to at least begin to explore both dynamics.
Personally, I think that unless a donor has any specific expertise in the issue area, it is less important that they try to determine if the answers to these questions prove that the nonprofit's programs work and more important that they assess their overall interaction with the nonprofit around these question. Did the nonprofit seem prepared to address the questions or did they feel like they would need to do special work to find your answers? When they answered negatively to questions about the availability of evidence, did they attempt to convince you that those reports were too expense/not important, or did they share an enthusiasm for obtaining that sort of evidence over time?
The current GiveWell tool kit includes issue specific questions to ask organizations across a range of almost 100 different areas both domestically and internationally.
You can explore the new offering here.
While the took kit is designed for donors, I think the questions are highly useful for nonprofits as well. Organizations within any of the issue areas covered by GiveWell should consider sharing the questions with their management team and board and discussing the extent to which they feel they can answer them. Ideally, board members and management team members should all be able to answer these questions on their own, since most of them get at core dynamics at the nonprofit rather then seeking statistical details.
