Evaluation Analyst E3 Alliance- Austin Community College, United States
Abstract Information: In this Ignite presentation, attendees will learn about the implementation and efficacy of an Utilization Focused Evaluation (UFE) (Patton, 1978, 2013) on a framework for improving literacy in elementary schools. While a formative evaluation was initially conducted on this literacy framework, further evaluation was needed to better understand its implementation, utility, and efficacy. Additionally, barriers to evaluation, such as timing and participant fatigue (Patel et al., 2020; Smith, Thrupp, & Barrett, 2019), were present. Attendees will be presented with how UFE ultimately addressed these challenges, its applicability to new contexts, and key insights into UFE in education. This presentation is well connected to the 2023 Evaluation Theme: The Power of Story because it highlights the need for dynamic methods to elevate constituent participation and voice in evaluation design. Additionally, this proposal is aligned with AEA's Respect for People Guiding Principle and Interpersonal Evaluator Competency due to its co-design approach. This evaluation engaged educators, school and district administrators, and researchers as members of a steering committee who defined specific metrics to ensure they found value in the work and resulting findings. The steering committee utilized the UFE Checklist (Patton, 2002) and this proposal represents the work up to Step 12 of 17 (i.e., Simulate Findings). The co-design process centered understanding steering committee members’ perspectives, developing research questions, forming an evaluation model, identifying methods, and mocking-up a final report. Implementing an UFE elicited participation and engagement and this presentation will highlight how evaluation methods can address challenges and resituate constituent voice.
Relevance Statement: This presentation is well connected to the 2023 Evaluation Theme: The Power of Story because it highlights the need for dynamic methods to elevate constituent participation and voice in evaluation design. Additionally, this proposal is aligned with AEA's Respect for People Guiding Principle and Interpersonal Evaluator Competency due to its co-design approach. Works Cited: Patel, S. S., Webster, R. K., Greenberg, N., Weston, D., & Brooks, S. K. (2020). Research fatigue in COVID-19 pandemic and post-disaster research: causes, consequences and recommendations. Disaster Prevention and Management: An International Journal, 29(4), 445-455. Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills. Patton, M. Q. (2002). Utilization-focused evaluation checklist. Utilization-Focused Evaluation (UFE) Checklist. Patton, M. Q. (2013). The roots of utilization-focused evaluation. Evaluation roots: A wider perspective of theorists, 293-97. Smith, M., Thrupp, M., & Barrett, P. (2019). Contemporary pressures on school-based research: A cautionary tale for school leaders. Journal of Educational Leadership, Policy and Practice, 34(1), 44-64.
