Needs Assessment
Hsinling Hung, PhD (she/her/hers)
Director of Evaluation and Assessment
VCU Center on Health Disparities, United States
Hsinling Hung, PhD (she/her/hers)
Director of Evaluation and Assessment
VCU Center on Health Disparities, United States
Yi-Fang Lee, Professor
Professor at Department of Industrial Education
National Taiwan Normal University, United States
Yi-Fang Lee, Professor
Professor at Department of Industrial Education
National Taiwan Normal University, United States
James Altschuld, Ph. D.
Professor Emeritus
The Ohio State University, United States
Location: Room 105
Abstract Information: Surveys are frequently employed in needs assessment (NA) to collect information about gaps (the needs). Double-scale Likert-type instruments are routinely used for this purpose; however, the quality of such measures has not been seriously examined. Evaluators are well versed in survey methodology, but are less aware of scaling techniques as applied to the NA. Attention to survey scaling challenges is vital to the NA survey development process. Findings and suggestions on different scales for NA survey will be presented. Topics in the workshop are brief overviews of NA scaling concerns, and scaling techniques for obtaining trustworthy NA results. A hands-on activity will be provided for writing survey questions and identifying proper scales for the NA survey. Discussion and comments during the session will be encouraged.
Relevance Statement: Surveys are a means of data collection in research and evaluation. How a survey is designed affects the quality of data gathered. For example, when designing a NA survey, the assessor has to consider the language used, the order and content of questions, survey scales, and a host of other factors. The Likert scale seems to be a default for many NA surveys but there are other options. One method often used is a survey with double scales to measure “what is” and “what should be” conditions. Likert 4- or 5- point scales are most common used with the assumption that the discrepancies determined from them truly denote key areas or difficulties to be resolved. But, does this premise really hold? Concern about the validity problem of the double-scale NA survey has been raised in a recent publication. Although Likert scale in double-scale format is seen often, there are other alternatives that might be better and lead to meaningful and trustworthy information for deriving discrepancies. Based on the above thinking, this workshop is designed to introduce the factors affecting the quality of needs data from surveys with double scales, to offer suggestions for improvement in double-scale survey design, and to discuss implications for what we do as assessors. Since many evaluators are not familiar with NA, this workshop begins with very brief overviews of NA, the design of double/multiple scales surveys, especially regarding factors affecting their validity. Suggestions for improvement based on the literature and the presenters’ past work will be included in the presentation. Then, a hands-on activity will be offered to enhance attendees’ understandings and skills. This workshop offers insights on how to best design a double-scale NA survey and the opportunity to discuss issues in doing so. Thus, this workshop addresses the competence and systematic inquiry of the AEA guiding principles as well as the evaluators’ methodology competency.