Skip to main content
Evaluation 2023 Main banner
Icon Legend
Presentation Icons
PD Workshop
PD Workshop
Virtual Offering
Virtual Offering
Presidential Strand
Presidential Strand
Plenary Session
Plenary Session
BOF
BOF
Ignite
Ignite
Back

Paper

Graduate Student and New Evaluators

Stories to Challenge the Evaluation Practice

Application of Weight and Sum Methodology to Product Evaluations: Key Lessons Learned

Friday, October 13, 2023
10:15 AM – 11:15 AM ET
Location: Grand Ballroom 1
Abstract Information: Product evaluation is often not considered in the training of evaluators. There is much to be learned from product evaluation that carries over to other evaluands. This paper summarizes the experiences and lessons learned by a group of graduate students who engaged in the evaluation of a range of products as part of their graduate training. The paper highlights steps taken to identify criteria of merit & worth, the design of instruments, sources of evidence, the creation of rubrics, and the use of weight and sum to establish summative scores to inform decisions. Key lessons learned include: (i) it is important to identify the users and uses of a product evaluation; (ii) use rubrics to ensure accuracy and fidelity of scoring; (iii) importance scores can be determined using multiple sources and methods; and (iv) weight and sum is especially appropriate for product evaluation because it results in a clear recommendation for consumers or client. Finally, the paper underlines that while everyone engages in product evaluation, formal product evaluation differs from the daily decisions made by consumers.