2 - Theory and Practice: Evaluation in the Museum Context

Abstract Information: Evaluation has been positioned as a transdiscipline: it stands independently as a discipline while at the same time having application as a methodological or analytical tool in other disciplines. Because evaluation is practiced in a wide array of contexts, it is important to build a strong knowledge base of how different evaluation theories are enacted in practice across contexts. Museums may likewise be considered transdisciplinary, concerning “...a variety of intentions, disciplines, practices, and motivations” (Miller, 2017), all with a focus on informal learning. Recent critiques have suggested a need for increased rigor in museum evaluation work. To extend that inquiry, this paper considers the application of fundamental evaluation theory in museum evaluation. If we consider evaluation to be a transdiscipline, it follows that its theory should likewise have application across contexts. Findings from this work will have Implications for the rigor, quality, and usability of evaluations in museums. Miller, S. (2017). The Anatomy of a Museum: An Insider's Text. John Wiley & Sons.

Relevance Statement: Evaluation has been positioned as a transdiscipline, meaning that – like statistics, accounting, or design – it stands independently as a discipline while at the same time having application as a methodological or analytical tool in other disciplines (Scriven, 2008). As such, evaluation is practiced in a wide array of contexts. Considering King’s (2003) suggestion for a context-sensitive approach to evaluation theory generation, it is important to build a strong knowledge base of how different evaluation theories are enacted in practice across contexts. This paper considers museum evaluation as one such example of situating evaluation practice in context. Museums may likewise be considered transdisciplinary. Martin (2017) describes museums and their collections as covering “...a variety of intentions, disciplines, practices, and motivations” (pg. 6). While Martin limits his description of museums to those with traditional, permanent collections – think history, art, natural history, science, and similar museums – when considering a focus on informal education, this description may be expanded to include children’s museums, zoos, aquaria, botanic gardens, and similar sites. Recent critiques have suggested a need for increased rigor in museum evaluation work. Fu, et al. (2016) examined summative evaluation methods used in informal science education, much of which happens in museums. They found strong reliance on self-report measures and non-experimental designs, limiting the claims of impact that can reasonably be made. Grack Nelson & Tranby (2015) likewise examined reporting trends in evaluation reports. They concluded that the evaluation field lacks guidance on what to include in evaluation reports and proposed a set of guiding questions for evaluators preparing reports. Using reports the same repositories as Fu, et al. and Grack Nelson & Tranby, the current author is examining what role theory plays in evaluations conducted in museums and other informal learning contexts. This paper will present those findings and consider, per King’s (2003) suggestion, an approach to evaluation theory generation sensitive to the context of museums. This paper has relevance to the understanding and application of evaluation theory. If we consider evaluation to be a transdiscipline, it follows that its theory should likewise have application across contexts. Additionally, with evaluation being practiced in museums by both internal and external evaluators, such an inquiry has the potential to illuminate needs that, if met, would raises the rigor, quality, and usability of their work. Fu, A. C., Kannan, A., Shavelson, R. J., Peterson, L., & Kurpius, A. (2016). Room for rigor: Designs and methods in informal science education evaluation. Visitor Studies, 19(1), 12-38. Grack Nelson, A., & Tranby, Z. (2015, September). Reporting with an evaluator audience in mind: A BISE synthesis paper. King, J. A. (2003). The challenge of studying evaluation theory. New directions for evaluation, 2003(97), 57-68. Miller, S. (2017). The Anatomy of a Museum: An Insider's Text. John Wiley & Sons. Scriven, M. (2008). The concept of a transdiscipline: And of evaluation as a transdiscipline. Journal of MultiDisciplinary Evaluation, 5(10), 65-66.