Research, Technology & Development Evaluation
Henrik Fosse, PhD (he/him/his)
Head of Analytics
Novo Nordisk Foundation
Birkeroed, Denmark
Rasmus Jensen, MSc
Head of Data
Novo Nordisk Foundation
Hellerup, Italy
Katrine Iversen, MSc (she/her/hers)
Impact Manager
Novo Nordisk Foundation, Hovedstaden, Denmark
Gert Balling, PhD
Senior Impact Partner
Novo Nordisk Foundation
Hellerup, Italy
Location: Grand Ballroom 8
Abstract Information: We provide practical experience from a large private funder of, primarily, public research concerned with implementing a systemic evaluation practice that meets several challenges of large organisations, including monitoring large portfolios of grants and learning from experience, as well as effectively communicating the impact of funding of public research activities. The 2030 strategy of the Novo Nordisk Foundation is mission-oriented and addresseses actions to be taken now to solve grand challenges within health and environmental sustainability. This implies taking risks and funding more mission-driven initiatives than previously. Learning from practice is even more crucial than before, and the Foundation must actively engage with stakeholders at all levels. To meet these requirements the Foundation has introduced Impact Management as an overarching management concept, which provides a unified understanding of the purpose of evaluation and delivers a versatile data environment that can be used by the Foundation, our grantees and their institutions, and is also available to the general public via a public dashboard. Presentations in this session provide examples and learnings from systemic approaches to evaluation practice, and data collection. It demonstrates how systemic use of the evaluator's toolbox can have profound impact on applicants storytelling in submitted proposals, provide extensive possibilities for using and visualizing data for multi-level purposes, and for collaborative evaluations with e.g. recipient institutions.
Relevance Statement: This session is particularly interesting for internal and external evaluators working within/with organizations that are aiming to introduce, or to improve, an evaluation culture that systematically addressesses multi-level measurement, evaluation and learning, and also provides a basis for effectively providing public transparency. We are especially focused on sharing how evaluation practices influence storytelling for and towards different stakeholders from grantees to institutions and public national and international stakeholders. From a technical, methodological point of view, the presentations build on examples that include advanced modelling practices in many different domains, including e.g. econometrics and topics in data science such as text mining and machine learning. The domain of the Novo Nordisk Foundation is primarily research funding. From our experience, working closely with other notable international private Foundations (but also public funders) in the public research funding landscape, evaluation practices in funding bodies are often fragile or decentralized and not governed by general evaluation guidelines. This poses a potential challenge to learning organisations that must prioritise funding in more ways than one: First, scientific grant managers are hired as topic matter experts and are rarely trained or experienced evaluators of programmes or complex projects. Topic matter is very important when working with highly advance research proposals, but in the research funding landscape this must be accompanied by a firm understanding of, e.g., science-of-science or behavioural theories and how incentives work with or against the intended purpose. Second, evaluation skills are also useful when designing grant instruments as this involves systematic thinking in the lines of intervention logic and theory of change. Using these tools to highlight critical points in intervention designs is very effective. We have developed an effective workshop-based process that provides a logic model to be included in grant applications, providing grant managers with basic understanding of evaluation principles. We work with grant applicants to ensure the quality of project success criteria that underpin the pathway to (long-term) outcomes or impact. This process early on strengthens the relations and trust between the Foundation and the grantee and the grantee's understanding of the purpose of the data collection, which may stretch beyond the critical information need to assess success of the grantees project but serves a greater purpose for the foundation. The presentations will each provide examples that together provide a picture of how attention to evaluation and careful data collection and planning can provide a powerful base for more or less disaggregate use for powerful stories.
Presenter: Rasmus V. L Jensen, MSc – Novo Nordisk Foundation
Presenter: Gert V. Balling, PhD – Novo Nordisk Foundation
Presenter: Katrine Z. Iversen, MSc (she/her/hers) – Novo Nordisk Foundation
Presenter: Henrik B. Fosse, PhD (he/him/his) – Novo Nordisk Foundation