Graduate Student and New Evaluators
Jill Hendrickson Lohmeier, Ph.D.
Professor
University of Massachusetts Lowell, United States
Jill Hendrickson Lohmeier, Ph.D.
Professor
University of Massachusetts Lowell, United States
Meg Johnson, M.S. (all pronouns)
Research Assistant
University of Massachusetts Lowell
Andover, Massachusetts, United States
James Fargnoli, M.Ed.
Ph.D. Student
University of Massachusetts Lowell, United States
Nadia Sahila, M.Ed.
Ph.D. Student
University of Massachusetts Lowell, United States
Golnar Fotouhi, M.A. (she/her/hers)
Ph.D. Student
University of Massachusetts Lowell
Lowell, Massachusetts, United States
Abstract Information: In order to develop reflective practice as students, members of three related research projects will engage in a reflective panel on their research and evaluation experiences as graduate students in a doctoral research and evaluation program. This panel serves as an educational opportunity for panelists as well as for audience members to further their understanding of the student experience. This panel addresses the need for training evaluators in reflective practice as identified by practitioners (Galport & Azzam, 2017) and in the American Evaluation Association (AEA) competencies (King & Stevahn, 2020). All members of the panel completed a two-course series on program evaluation in 2023. The first course focused on program evaluation theory and models while the second course centered on developing skills as practitioners and researchers of evaluation. During the second course, the professor assigned students to teams of two or three members. The teams worked with a consenting evaluand of the students’ choosing. Concurrently with the program evaluation, the teams also completed a research on evaluation project. The professor from both courses will facilitate the panel discussion. This panel will draw on the recent student research projects conducted by the panelists as part of their graduate work. The studies include (1) how graduate students make sense of teamwork in evaluation through a student evaluation project, (2) the effects of survey modality on data quality for evaluation and, (3) participants’ experiences with the AEA’s student case competition. The students will reflect on and share the story of their research, including how their research developed, their experience conducting research as graduate students, future directions, and the impact that their research experiences had on their learning. Reflection in a work-based context promotes creativity, experimentation, and skill development (Helyer, 2015). Through reflecting together, the panel will also draw on the connections between their research projects to weave a story of conducting research as students and developing skills relevant to work in the field of evaluation. Additional anticipated discussion points include a brief introduction to the projects, reflections on teamwork, challenges, skill development, and advice for graduate student researchers. The students will also discuss how their evaluation and research projects built a deeper understanding of the differences and similarities between evaluation and social science research, both of which are fundamental components of the students’ degree program. Program evaluation instructors, professional evaluators, and graduate students will find this panel relevant to understanding ways in which the necessary and desirable skills, competencies, and values, can be developed for evaluation students.
Relevance Statement: This panel tells a co-constructed story of how a group of graduate students are developing their skills and understanding of evaluation and research in evaluation through a graduate class, an evaluation competition, and student research projects. While each event stands alone, together they represent several facets of the students’ education in evaluation. Each of these experiences is interwoven with the other. For example, the students’ participation in the evaluation competition is a learning experience that is also the topic of a student research project. The experiences represented in this panel took place within a 6-month period in the spring of 2023. Multiple panel members shared each of the experiences represented. The facilitator was in a supervisory role for each of the experiences as the instructor for the graduate class, the coach for the evaluation competition, and the principal investigator for the student research projects. The need for evaluator training has grown with the demand for professional evaluators (LaVelle, 2020). Practitioners identify skill gaps in university-trained evaluators, including team management and practical research design (Galport & Azzam, 2017). These skills are also addressed in the AEA competencies (King & Stevahn, 2020). In addition to training high-quality evaluators, it is important to understand how evaluators are trained to conduct research in evaluation. The link to a body of systematic evidence on its practice marks evaluation as distinct from fields that operate by ritual-based practices (LaVelle & Donaldson, 2015; Pavalko, 1988). This panel explores the training of practitioner- and research-based evaluation competencies through the stories of student evaluators and researchers.
Presenter: Jill Hendrickson Lohmeier, Ph.D. – University of Massachusetts Lowell
Presenter: Golnar Fotouhi, M.A. (she/her/hers) – University of Massachusetts Lowell
Presenter: Linda P. Dart-Kathios, M.A. (she/her/hers) – University of Massachusetts Lowell
Presenter: James Fargnoli, M.Ed. – University of Massachusetts Lowell
Presenter: Nadia Sahila, M.Ed. – University of Massachusetts Lowell
Presenter: Meg Johnson, M.S. (all pronouns) – University of Massachusetts Lowell