Government Evaluation
Terell Lasane, n/a
Assistant Director
US Government Accountability Office
Washington, District of Columbia, United States
Terell Lasane, n/a
Assistant Director
US Government Accountability Office
Washington, District of Columbia, United States
Gergana Danailova-Trainor, n/a
Senior Economist
US Government Accountability Office
Washington, District of Columbia, United States
Valerie Caracelli, n/a
Senior Social Science Analyst
US Government Accountability Office
Washington, District of Columbia, United States
Lawrence Crockett, n/a
Analyst
US Government Accountability Office
Washington, District of Columbia, United States
Location: Grand Ballroom 9
Abstract Information: The Center for Evaluation Methods and Issues (CEMI) is located in the Applied Research and Methods Mission Team (ARM) at the US Government Accountability Office ( US GAO). CEMI operates to improve the quality of evidence used to inform federal decision-making. CEMI supports this objective by leading and contributing to studies that identify the challenges and achievements of agencies carrying program evaluation work. Testimonial evidence is a major source of information from which these issues are identified. In this session, the authors will examine the importance of systematically obtaining testimonial evidence through a survey administered to a probability sample of federal managers. Survey methodology is a form of gathering stories from informed government stakeholders who can provide relevant, valid, and reliable reports. The author will present key findings from the Federal Managers’ Survey and report how stories help to build important relationships and gain valuable insights to what increases the likelihood of evaluation evidence use.
Relevance Statement: In keeping with AEA’s Conference theme “The Power of Story” the Center for Evaluation Methods and Issues (CEMI) has produced numerous studies that highlight the role that evaluation evidence plays in federal decision-making. CEMI, located in the United States Government Accountability’s (GAO) Applied Research and Methods (ARM) Team operates as one of six ARM centers that ensure that research done at GAO is undertaken with rigor and sound methodological design and analysis. CEMI has provided leadership and insights about how the field and practice of evaluation can continuously improve to foster greater appreciation for the unique value add of evaluation. With a staff diverse in training and skills, CEMI staff partner with mission teams to clarify the invaluable role evaluation plays in creating good government. In order to achieve this goal, key evaluation leaders and stakeholders in the federal government are often surveyed and their perspectives are sometimes used to inform recommendations about how evaluation evidence can be optimally deployed.
GAO serves at the pleasure of Congress. Congress sends requests to GAO mission teams to answer questions about government performance and compliance to important statutory requirements of government accountability. Using principles for good evaluation embraced across the evaluation field (i.e., transparency, independence, ethics, relevance and rigor), CEMI provides criteria to help advise Engagement Teams about the extent to which evaluation evidence is adhering to those principles and meeting mandates set by legislative directives.
Recently, CEMI partnered with the Strategic Issues (SI) mission team to gather insights from federal managers (i.e., GS13, GS14, GS15, and Senior Executive Series equivalents). The CEMI team revised a survey (the Federal Managers Survey) that has been administered in the past to track how federal managers are using various types of evidence in order to comply with the Government Performance and Results Modernization Act (GPRAMA 2010). CEMI staff revised the program evaluation section in the most recent administration of the survey to gauge managers’ use of evaluation evidence and to identify correlates with this use. With data collected between July 2020 and December 2020, we surveyed 3993 federal managers from a population of 150,447 civilian managers at 24 major federal agencies to ask for their views on the availability and use of performance information, practices that can promote the use of this information, and their agencies’ data-driven reviews. The results of that online survey includes managers’ perspectives on how these various factors correlate with use.
In this presentation, a member of that research team will discuss the key findings from that analysis and will discuss how this methodology, relying on testimonial evidence, is akin to these managers sharing stories about their experiences with evaluation and how these stories can help evaluators better understand and improve the quality of evidence used in federal decision-making