What Have We Learned About Learning in the Past 10 Years?

As a member of the evaluation community, and in my work as an advisor to foundation clients including Kaiser Permanente’s Community Benefit, the Kresge Foundation, and the Walton Family Foundation, I think about trends shaping the field, and how they are showing up in practice.

Last week, I was invited by the American Evaluation Association’s president Kathryn Newcomer to present at the start of this year’s conference, “Learning and Action.” Ten years ago when I was the president of the American Evaluation Association, I chose the conference theme “Evaluation and Learning,” to bring attention to the role learning plays in evaluative thinking and practice. This year, David Bernstein and I discussed what we have learned about learning in the past 10 years; the following are some of my reflections from our conversation.

The evaluation profession and practice has grown considerably in the U.S. and around the world in the past decade. In that time 3 movements have emerged that are influencing evaluation practice:

1. Our understanding of the importance of systems thinking, systems change, and complexity. One of my favorite definitions of systems change comes from the work of the Social Innovation Generation in Canada. They define systems change as “shifting the conditions that are holding the problem in place.” As evaluators who are working in the sphere of social systems change, we need to look for patterns, types and strengths of relationships, unexpected changes or consequences, how things morph and evolve as they occur, as well as gaps and momentum within the systems we are studying.
2. A deepened understanding, acknowledgment, and realization that we need to learn about and bring a diversity, equity, and inclusion lens to our work. We are learning that social and environmental systems cannot change unless we recognize the inequitable structures, policies, and practices that hold so many social problems in place. This increased understanding about racial inequity is showing up in our work as clients are increasingly asking us about our experiences and training; the diversity of our staff and boards; how we reflect the voices, history, and cultural context of the communities in which we conduct evaluation; and how we address power in our work. As a field, are being challenged (in a good way) to learn about white privilege and power; what it means to work within communities of color; what it means to have real and authentic stakeholder and community engagement; our own biases; the critical importance of disaggregating data; and of course, being humble.
3. Learning is also showing up at the Intersection of Strategy and Evaluation. In the last decade, many evaluators have been working with clients to help them understand how strategy informs what gets evaluated, and how evaluation informs strategy design and implementation processes. Thanks to Patti Patrizi, Michael Quinn Patton, Jewlya Lynn, Tanya Beer, Julia Coffman, and others, we are increasingly helping our clients learn about the ways in which evaluation can be an integral part of a strategy’s development, as well as its implementation and impact. Those working at this intersection have put learning at the heart of their work—they have designed and implemented evaluations that articulate, clarify, and challenge assumptions, expectations, and outcomes of the strategy. They are posing new kinds of evaluation questions that are strategic, that capture the complexity of social change, and are intended to inspire action. They have willingly (some would say, gleefully!) blurred the lines between these two endeavors believing that they are intertwined, and that evaluators adopting this approach are always creating learning processes that support experimentation, innovation, and confidence in the work, through a continuous process of individual, group, and organizational learning activities.  

Often, we find our clients wanting to partner with FSG’s Strategic Learning and Evaluation practice to help craft program and organization level learning agendas, to help diagnose and support the development of learning cultures, and to develop a cadre of learning tools to facilitate learning.

To give a sense of how these interests have been manifested, here are a few examples of FSG’s recent evaluation client work. We partnered with Kaiser Permanente’s Community Benefitorganization on a 2-year journey to create a Measurement and Evaluation for Learning and Outcomes (MELO) system. Over 200 Community Benefit colleagues participated in the process to create the MELO strategy (theories of change, learning questions, outcomes, and indicators) and since then, many have been using this tools and processes to help improve their work.

We worked with the Walton Family Foundation, which undertook a 9-month effort to develop new practices and systems for supporting learning and strategic planning within the foundation—all with the goal of better understanding the foundation’s progress and impact of their efforts. This has involved developing an overarching framework that describes its various functions, including developing strategic learning questions and learning plans for each of the foundation’s programmatic teams, refreshing its vision for the unit, and renaming the function from Evaluation Unit to Strategic Learning and Evaluation Department (SLED).

In 2014, the Kresge Foundation started on its organizational learning journey. Over the last 3 years, this has involved hiring Chera Reid, as Director of Strategic Learning, Research, and Evaluation, engaging each of its 6 programmatic teams in a process to develop a strategic learning and evaluation system, designing and facilitating team learning retreats whereby they reflect on and discuss what they are learning from data and their experiences, enhancing its program operations, IT,  and communications infrastructure to support learning and evaluation, and are building learning capacity with their staff and Board.

Recently, Chera shared a note that, “data galleries are taking off at Kresge! We’ve used them to have discussions about advancing equity across our grantmaking, and most recently, with our Board to help showcase the impact of our Health program grantmaking.” We are thrilled when we hear about clients using learning tools to reflect on and engage in dialogue about their work—its progress, effects, and impact.

Some of our favorite tools for helping social change organizations facilitate learning include: Marilyn Darling’s “Emergent Learning: A Framework for Whole-System Strategy, Learning, and Adaptation”, and her work with After Action Reviews, Spark Policy Institute’s various toolkits for social innovators, including the developmental evaluation toolkit and tools for integrating an equity lens, and FSG’s recent Guide to Facilitating Intentional Group Learning.

It’s wonderful to see the growing commitment within the evaluation field to use evaluation to accelerate learning—I can’t wait to see what the next decade holds.

Learn more about FSG's evaluation practice >