Lesbian, Gay Bisexual and Transgender Issues
Carlos Romero, MBA
Executive Producer
Apex Evaluation
Albuquerque, New Mexico, United States
Location: White River Ballroom I
Abstract Information: Unleashing the power of story requires a dramatic shift in the business model of evaluation - one that gives currency to positionality. This is based on the premise and belief that story does not merely serve as an enhancement that is essential to evaluation. Rather, all evaluation is the act of storytelling. Equitable evaluation could be described as producing truer stories based on a credible body of evidence. Credibility that comes not from data, but from the testimony of witnesses closest to the ground. Positionality, a term used in the equitable evaluation movement, refers to the lived experience of evaluators – the kind you’ll find in the "diversity" TIGs (LGBT Issues, Feminist Issues, Indigenous Peoples, and Latinx). Evaluators from these diverse and marginalized backgrounds have one thing in common, which Edgar Villanueva sums up best: “Those most excluded and exploited by today’s broken system possess exactly the perspective and wisdom needed to fix it." These are the voices that should be leading the storytelling of evaluation. This requires activation, influence, and power in ways the status quo does not support. I will examine the inherent problems in the business model of evaluation including the problematic currency of academic credentials, methodological expertise, and institutional capacities that perpetuate the status quo and its power structures, preventing positionality from assuming the leadership role the story deserves. This will involve dissecting the wickedness of both the supply and demand in the field of evaluation, including the systems-power bind, which will be explained and illustrated with stories. Specifically, I’ll point out how the workforce diversity being offered by the big institutions of evaluation is woefully insufficient for activating positionality. Finally, I’ll make the case for more than traditional, incremental change. Then, I’ll explore a novel idea for total transformation of the economic model using two thought-provoking analogies, Hollywood studios and organic foods, and involving three recalibrations: 1) Taking on new categories of roles (executive producer, producer, director, crew, cast) that better describe how practice and positionality come together to ensure that all the tools and methods of our trade are in service of the story and not the other way around; 2) ensuring that procurement and payment practices are designed to find and place talent in these new roles and compensate them appropriately in relation to their contribution; and 3) creating conditions for better and limitless collaboration across individuals and organizations including community voices. The recalibrations of the studio model balance power dynamics and eliminate the zero-sum nature and mutual exclusivity that prevent meaningful collaboration and cut off most of the talent needed to produce more authentic, more compelling, more impactful stories. Finally, I’ll turn to the organic food industry as a guide to inject standards and accountability for equitable evaluation that will purify both demand (funders) and supply (practitioners). This is an appeal to early adopters from both camps who are committed to the vision of equitable evaluation and longing for a better way to do business together.
Relevance Statement: I’ll start by using my positionality (LGBT, Latino, overcame SDOHs) as privilege to say that story is not essential to evaluation. Evaluation itself is storytelling. Always. This notion comes from Counting (Stone) which posits that “every number is born of subjective judgments, points of view, and cultural assumptions.” Stone teaches us how numbers are stories, evoking Brené Brown’s adage “stories are just data with a soul.” Holding this truth is particularly important if your work intersects the trifecta of wicked problems: poverty/income inequality, health/education disparities, and racism/discrimination. See, wicked problems are associated with fast thinking which, according to Daniel Kahneman (Thinking Fast and Slow) is the jumping-to-conclusions part of our brain where assumptions and biases go without scrutiny. Fast thinking has no time for context or perspective which, according to David Epstein in Range is where we are likely to find solutions to wicked problems. The remedy is slower thinking that challenges dangerous assumptions, exposes harmful biases, and reveals new stories that give context and meaning to data. Then I introduce Systems Learning as a universal and systematic approach for (a) building a body of evidence that inextricably links data with the context of testimony from credible witnesses - another way to think of storytelling; and (b) using a simple model of systems thinking (Cabera) to facilitate slower thinking. Systems Learning elevates a diversity of perspectives, prioritizing those closest to the work and those most affected by the work as Edgar Villanueva suggests in Decolonizing Wealth. Systems Learning is optimized for equity and learning and can serve as a universal, actionable mission for equitable evaluation. This movement needs a unifying mission that is (1) actionable - it tells us what to do; (2) clear and simple – unambiguous and doable; (3) repeatable – but iterative and emergent; and (4) universal – everyone can do it together. These missional attributes are also qualities of a good game, and every good game involves an economy to incentivize the desired outcome. Here, I pivot from practice to business, explicating how the economy of the evaluation game is not optimized for producing more authentic, compelling, and impactful stories. Think about the widespread support from both funders and practitioners for the vision of equitable evaluation which centers on things like “evaluation as tool for advancing equity, diversity of teams, appropriate methods, revealing drivers of inequity, empowering those more affected to shape and own how evaluation happens, etc.” A compelling vision to be sure. But if we don’t know what to do, (see missing mission above) then how can we buy and sell it at any scale to make a difference? What’s holding us back? I’ll illustrate how the current economy is optimized for status quo and fast thinking, with misplaced market incentives for academic credentials, methods, and institutional capacities at the expense of positionality. I’ll explicate a Systems Learning Studio solution using two thought-provoking analogies, concluding that our industry is ripe for what John Elkington calls Green Swan transformation – system-wide breakthroughs for exponential progress.