Organizational Learning & Evaluation Capacity Building
Jennifer Heettner, MPP
Director, Monitoring and Evaluation
Peter G. Peterson Foundation, United States
Arun Karpur, MD, MPH
Director, Monitoring, Evaluation, & Research
Peterson Center on Healthcare
New York, New York, United States
Sarah Berk, MPH
Associate Director, Monitoring, Evaluation, & Research
Peterson Center on Healthcare, United States
Location: White River Ballroom A
Abstract Information: As M&E leads within two philanthropic organizations, we are highly cognizant about the careful allocation of precious resources in service of M&E implementation and impact. As we work to build organizational M&E capacity within grantmaking, our colleagues often look to us for guidance about M&E intensity and frequency. To address these questions systematically, we have developed a a tool and process to tailor M&E to project needs. The goal is to provide consistency, collaboration, and transparency in our M&E planning conversations with colleagues and grantees to enhance strategic decision-making around use of organizational resources for M&E.
M&E planning conversations and resource allocation can be a difficult process that is impacted by a variety of organizational dynamics. Our goal in building this tool was to create a way to maintain focus on the M&E needs of the project in a way that is agreed upon by all stakeholders. To do this we started by engaging our grantmaking colleagues to identify project attributes that affect how much M&E is warranted.
Our colleagues’ experience and expertise with M&E varies, so using M&E language (or jargon) to conceptualize or describe what attributes might be relevant was too limiting. To have open-ended conversations, we asked them to tell us their stories about grantee work, project development and design, project implementation and reporting and how they understood defining and measuring potential impact. By using narrative to describe the work, our colleagues could share with us on their terms, through their framing. We then took what they shared with us and defined a set of attributes based on organizational and grantee frameworks and common organizational language.
We organized the attributes based on different dimensions, including:
- Project types and purposes
- Range of potential impact (i.e., for grant target audience, grantees, and other organizations) including risks and opportunities
- Required M&E outputs (and their applicability) for next steps like scaling or modifications
- Levels of project complexity (e.g., implementation requiring multiple partners, or across multiple geographical areas)
- Levels of confidence/rigor required to inform decisions
We had a separate team (our in-house design team) facilitate reflection sessions to test and gather feedback on the attributes. We then developed a rubric to map these attributes to different degrees of M&E, with associated roles and responsibilities for internal program staff, M&E teams, and grantees. This provides a structure for collaborative M&E planning and decision-making.
In this Skill-Building Workshop we will share the process and tool with attendees and facilitate experimentation with this tool. Attendees will work in small groups to apply the tool so they can build their understanding about how a similar approach might support their organization’s strategic decision-making about M&E.
Relevance Statement: As results-based management is becoming a norm in grantmaking, philanthropic organizations are leveraging M&E functions to ensure accountability and yield strategic insights. At the same time, organizations are also leveraging M&E as a tactic to advance their strategies that rely on multi-stakeholder engagement. Resultingly, decisions of what gets evaluated and how much gets evaluated are conditioned on a core set of project features that can communicate practical and strategic value. Surfacing these attributes and operationalizing evaluative criteria informing intensity and frequency of M&E can support strategic decision making and meaningful resource allocation. While attempts to create a taxonomy of project attributes have been suggested - a toolkit piloted and tested in our organizational context provides opportunity for other organizations to emulate and scale. Resource constraints can influence M&E decisions in a variety of ways –including the way M&E activities are designed and implemented within a project or the way M&E resources are allocated across projects. Often, M&E professionals are asked to weigh-in on how resources should be allocated for M&E within a project. However, their recommendations may then meet resistance when the costs (both time and money) are evaluated against other organizational priorities and the original rationales are not clearly understood or shared by all stakeholders. Subsequent conversations and decisions may then be affected by factors that are inconsistent or disconnected from the original project needs and goals.
The goal of our tool was to create a simple approach to support a more consistent approach to M&E decision-making that can be readily documented and applied across various projects. We aimed to create something that could be replicated, but also bespoke to our organization’s own frameworks and language and not dependent on our colleagues having a uniform understanding of M&E.