Lessons from the Evaluability Assessment of the Teagle Foundation’s College-Community Connections Initiative 

By Kristin Black (April 2016)

How do we know when an initiative is ready to pursue a rigorous evaluation? More and more funders, policymakers, and program leaders recognize the value of high-quality evidence. But to make good use of a program evaluation, initiatives must contend with a set of fundamental questions first. Some of these are about the initiative itself: What outcomes does it seek to affect? Are daily activities in line with long-term goals? Others are about the purpose of the evaluation: What do initiative stakeholders hope to learn? Who is the audience for evaluation findings?

One way to answer these questions is to undertake an “evaluability assessment”—a process designed to determine the current potential for a rigorous evaluation, and to provide recommendations for how an organization, or a group of partnering organizations, can prepare for a successful evaluation in the future. 

This report describes an evaluability assessment conducted by the Research Alliance for NYC Schools on behalf of the Teagle Foundation. The Foundation contracted the Research Alliance to explore the feasibility of a variety of approaches to evaluating its College-Community Connections (CCC) initiative, which funds partnerships between community-based organizations and universities, aimed at promoting college access and success among talented but underrepresented students in NYC.

The report highlights challenges and lessons in five areas that are typical of an evaluability assessment:

1. Articulate Goals and Outcomes: What long-term goals and interim outcomes does the initiative seek to affect? What implications do these goals have for an evaluation of the initiative’s impact on students?
2. Assess Past Learning and Identify Current Evaluation Needs: What do we already know about the initiative, based on previous program reviews? What do those findings suggest about evaluation needs for the future? 
3. Describe Current Operations: What are the current structures and elements found among initiative partners? What do these suggest about the most promising designs for an evaluation?
4. Develop Mechanisms for Organizational Learning: Will stakeholders be able to use evaluation findings productively? What new mechanisms for feedback might improve organizational learning?
5. Explore Evaluation Strategies: What is the current potential for a rigorous experimental evaluation? What evaluation structures best meet the initiative’s needs?

We believe the insights that emerged from our work with Teagle and its CCC partners will be informative for other initiatives—especially those working to support college access—that are interested in evaluating their efforts.