University-Based Center
Erin Berical, MS.Ed. (she/her/hers)
Assistant Director
The Center for Human Services Research
Albany, New York, United States
Margaret Gullick, PhD (she/her/hers)
Director
The Center for Human Services Research
Albany, New York, United States
Location: Room 204
Abstract Information: Ascertaining the skills of a job candidate, and evaluating the performance of an employee, both involve understanding the story of the person and their experiences, whether in their previous or current position. These processes can be inconsistent and subjective, potentially resulting in inadequate hires and dissatisfied staff. To address this gap, one evaluation group has developed a series of materials to better assess applicants and staff members in a more consistent, objective manner. These tools include a standardized rating system for job applicants, directing hiring committee members to review for critical skills and experiences; and scenario-based interview questions aimed at uncovering how the applicants’ background and experiences have shaped how they might respond to the type of tasks and problems frequently encountered in our projects and allowing assessment of whether their thinking would meet staffing needs. Additionally, a “role definition matrix” was developed, creating explicit expectations around skills and responsibilities in key domains for each researcher title. This tool is also used at all staff members’ formal annual evaluations and informal quarterly reviews, prompting conversations about professional development, promotion, or support needs. This session will share more specific examples of the materials developed and offer recommendations and tools to help attendees create their own with a goal of better evaluating an applicant’s “story” and thus hiring and developing qualified, successful employees.
Relevance Statement: Ascertaining the skills of a job candidate, and evaluating the performance of an employee, both involve understanding the story of the person and their experiences, whether in their previous or current position. These processes can be inconsistent and subjective, potentially resulting in inadequate hires and dissatisfied staff. To address this gap, we developed a series of materials to better learn the story of applicants and of staff in a more consistent, objective manner. First, reviewing application materials can be difficult, with hiring committee members unsure of what to prioritize or distracted by writing style or grammar. In response, we utilized a rating system directing review to critical experiences (e.g., developing research or evaluation plans, quantitative data analysis, presenting findings to wider audiences), with a consistent rating scale (e.g., 0=not addressed, 1=limited experience, 2=extensive experience). This system helped reviewers focus on the components most important in determining qualification for the role. Second, interviews require delving past surface-level questions to understand not only a candidate’s history, but how they approach a problem and think through solutions. Typical questions (e.g., “what is your experience working on a team”) might allow candidates to share some relevant experiences and speak to their strengths, but do not illustrate how they might perform in a new environment. To address this, we created a set of scenario-based questions reflecting the sort of situations a candidate might encounter in our Center, allowing the interview committee to better assess how the candidate might approach their work if hired. Candidates are presented with a plausible framing scenario and asked what they might consider at each stage of a project, including critical questions to ask a funder when first setting up an evaluation, methods of gauging client satisfaction, and steps for dealing with conflict between team members, as informed by their previous experiences. Importantly, questions did not have one “right answer:” multiple reasonable solutions could be given, and subsequent questions were relatively independent from the specifics of their answers. This framework assisted the hiring committee in assessing how each candidate’s story informed their thinking. And last, once a hire is made, they need to be able to flourish within their role, and hopefully develop beyond it, which requires a clear understanding of what each role requires. A “role definition matrix” was developed for the three major, hierarchical research positions to create explicit expectations around skills and responsibilities in domains such as project leadership, data analysis, supervision, etc. Both staff and supervisors use this matrix at formal annual evaluations and informal quarterly reviews to determine if the skills shown and work performed are in line with the staff member’s title. These reflections then prompt conversations about areas for professional development, promotion opportunities, critical role expectations, and needed supports. This session will share more specific examples of the materials developed and offer recommendations and tools to help attendees create their own with a goal of better evaluating an applicant’s “story” and thus hiring and developing qualified, successful employees.