Using principles-focused evaluation with a complex, multi-grantee initiative

Background

Evaluators have a growing number of approaches and methods available to meet the needs of increasingly complex initiatives. In evaluating a chronic disease prevention initiative that included state-level and local work across nearly two dozen communities, we partnered closely with our clients to create an evaluation plan, theory of change, and logic models. There was a LOT to evaluate! Despite the usefulness of some of these traditional approaches, we realized the complex nature of the program required another approach. This is when we introduced and dove into principles-focused evaluation in partnership with our client.

Approach

In a principles-focused evaluation framework, stakeholders develop a guiding framework embodied in principles that cross cut individual program components. The principles are evaluated, rather than the individual program components. This fit the situation because the primary evaluation challenge was to meaningfully define and measure how nearly two dozen grantees were working in different ways toward shared goals, and because the deliverables were flexible. A principles-focused evaluation also fit well because discussions with the program staff to co-construct the health equity statement laid the groundwork to develop their principles. To determine the principles, we began by advising our client that principles are guiding, not directive and that we needed language that was good enough to put into practice, but not necessarily perfect. We engaged in multiple rounds of review with the principles and made sure there was a shared understanding of the intent of each principle.

Our evaluation team then explored these guiding evaluation questions:

1. To what extent are the identified principles meaningful to those meant to follow the principles?
2. To what extent are the principles adhered to in practice?
3. If adhered to, to what extent do the principles guide followers to the results they hope to achieve?

Results

Moving to a principles-focused evaluation allowed our client to see how the evaluation and program informed each other and how they can use the evaluation to make decisions moving forward. They brainstormed ways to put the principles into practice, including:

1. Adding principles to the program website
2. Integrating principles into documents
3. Highlighting principles in webinars and trainings
4. Using principles as a guide for RFPs, contracts, and meetings

Because the principles-focused evaluation resonated with the client, there were opportunities for the evaluation to have broader use and involved more stakeholders. We were invited to share evaluation results with our client’s grantees, rework the evaluation of the coalitions, and host a full day in-person session with grantees.  The health equity statement, which is reflective of the program principles, is now displayed on all of the program’s documents, alongside the program mission, vision, and logo.