Meet Our #OpenForGood Award Winner: An Interview with Veronica Olazabal, Director of Measurement, Evaluation and Organizational Performance, The Rockefeller Foundation

This post is part of the Glasspockets’ #OpenforGood series done in partnership with the Fund for Shared Insight. The series explores new tools, promising practices, and inspiring examples showing how some foundations are opening up the knowledge that they are learning for the benefit of the larger philanthropic sector. Contribute your comments on each post and share the series using #OpenForGood. View more posts in the series.

The Rockefeller Foundation advances new frontiers of science, data, policy, and innovation to solve global challenges related to health, food, power, and economic mobility. In this interview, Veronica Olazabal shares insights with GlassPockets' Janet Camarena about how the foundation’s practices support learning and open knowledge.

GlassPockets: Congratulations on being one of our inaugural recipients of the #OpenForGood award! The award was designed to recognize those foundations that are working to advance the field by sharing what they are learning. Can you please share why you have prioritized knowledge sharing at the Rockefeller Foundation and how this practice has helped you to advance your work? Or put another way, what is the good that has come about as a result?

Veronica Olazabal: We are excited to be an inaugural recipient of the #OpenForGood award! As you may be aware, since its founding more than 100 years ago, The Rockefeller Foundation's mission has been “ to promote the well-being of humanity throughout the world.” To this end, the Foundation seeks to catalyze and scale transformative innovation across sectors and geographies, and take risks where others cannot, or will not.

While often working in new and innovative spaces, the Foundation has always recognized that the full impact of its programs and investments can only be realized if it measures - and shares - what it is learning. Knowledge and evidence sharing have been core to the organization's DNA dating back to its founder John D. Rockefeller Sr., who espoused the virtues of learning from and with others—positing that this was the key to "enlarging the boundaries of human knowledge." You can imagine how this, in turn, resulted in transformational breakthroughs such as the Green Revolution, the eradication of Yellow Fever and the formalization of Impact Investing.

GP: Your title has the word “evaluation” in its name and increasingly we are seeing foundations move toward this staffing structure of having staff dedicated to evaluation and learning. For those foundations that are considering adding such a unit to their teams, what advice do you have about the structures needed to create a culture of learning across the organization and avoid the creation of one more silo?

VO: Learning is a team sport and to that end, an evaluation and learning team should be centrally positioned and accessible to all teams across a foundation. At the Rockefeller Foundation, the Measurement and Evaluation team engages with both the programmatic and the impact investing teams. We see our role as enablers of good practices around impact management and programmatic learning -- often working with teams in early stage design support, through start-up, implementation and exit. We also work collaboratively with others at the Foundation such as our grants-management and data teams to ensure the “right” M&E data is being captured throughout our grantee’s lifecycle.

Yet, I will be the first to say that building a culture of learning by continuously reaching “over the fence” is a lot of work and might be challenging for a small team, which is the reality for most foundations. Benchmarking data produced by the Center for Evaluation Innovation (CEI) and the Center for Effective Philanthropy (CEP) lands most M&E teams at foundations at around 1.5. So, capacity for culture change is clearly a challenge. My suggestion here is to source evaluation and learning talent that balances the hard technical chops with the softer people skills. I believe you truly need both and if an organization optimizes for one over the other, might experience a series of false starts. A good place to start in sourcing evaluation talent is the American Evaluation Association (AEA).

GP: As you heard during the award presentation, one of the reasons the Rockefeller Foundation was selected to receive this award is because of your commitment to sharing the results of any evaluation you commission, before you even know the outcome. This pledge seems designed to not let negative findings affect your decision about whether or not to share what your learned. We often hear that foundation boards and leaders are worried about reputational issues with such sharing. What would you say to those leaders about how opening up these pain points and lessons has affected Rockefeller’s reputation in the field, and why it’s worth it?

VO: In 2017, The Rockefeller Foundation was pleased to be the first to make all of its evaluations available to IssueLab as part of #OpenForGood. But to the Foundation, being open goes well beyond passively making information available to those seeking it. Being truly open necessarily involves the proactive sharing of lessons so that others can be aware of and leverage from the things that we are learning. To that end, we regularly author blogs, disseminate evaluation reports and M&E learnings via digital channels, and – perhaps most importantly – share back evaluation results with our grantees and partners – so that evaluation is more than a one-way extractive exercise.

"Being truly open necessarily involves the proactive sharing of lessons so that others can be aware of and leverage from the things that we are learning."

Taking sharing one step further, earlier this year, The Rockefeller Foundation adopted a new Data Asset Policy aimed at making the data that we collect as part of our grantmaking freely available to others who could use it to effect more good in the world. The policy is grounded on two core principles: 1) that the data we fund has incredible value for public good and that these assets can serve as fuel for better decision-making; and 2) we commit ourselves to being responsible stewards of these data, which means prioritizing privacy and protection, especially of those individuals and communities we seek to serve. Moving forward, this opens up the ability to amplify our learning even further and in even more innovative ways.

GP: A concern we often hear is that a funder creating a culture of learning leads to an increased burden on grantees who are then asked for robust evaluations and outcomes measures that no one is willing to pay for. Does Rockefeller include funding for the evaluations and reporting or other technical assistance to mitigate the burden on grantees?

VO: Having had the experience of being both a funder and a grantee, I know this is a real barrier to enabling robust learning cultures and evidence-informed decision-making. For this reason, at The Rockefeller Foundation we approach resourcing in a few different ways:

• First, through embedding resources for evaluation and learning into individual grantee budgets and agreements from the start. This type of funding enables grantees to generate the type of data they need for their own decision-making, learning and reporting.
• We also often work in a consortia model where we commission an evaluation and learning grantee separately to synthesize learnings across groups of grantees and provide technical assistance as needed. This approach helps decrease the reporting burden for “implementation” types of grantees as it generates what is it the Foundation would like to learn (which could differ from what the grantees and their clients find useful). Here is an example from our Digital Jobs Africa portfolio generated through this evaluation and learning model.
• Finally, we have also at times, and upon request, seconded our own M&E staff to grantees and partners to help build their M&E muscle and enable them to measure their own impact. While this is rare, we are seeing this request more and more and hence why we value both technical expertise and relationship management skills.

GP: Learning is a two-way street and foundations are both producers and consumers of knowledge. Let’s close this interview with hearing about a noteworthy piece of knowledge you recently learned thanks to another foundation or organization sharing it, and how it helped inform your own work.

VO: There are many opportunities to learn from others. In my current role, I am in continuous engagement with colleagues in similar roles at other philanthropies and regularly meet before or after convenings organized by CEP, GEO and AEA. In addition, as part of my work on the Fund for Shared Insight which is a funding collaborative working to make listening to end-users the norm, my philanthropy colleagues and I often exchange on where we all are in our personal and institutional learning journeys.

Finally, as part of a W.K. Kellogg Foundation-funded Lab for Learning, The Rockefeller Foundation was most recently among a cohort of 15 foundations that took part in a year-long series of convenings to address systemic barriers to learning. Participation here required us to experiment with ideas for supporting learning in our own settings and then sharing our experiences with the group. Through this engagement, we learned about how others were building learning habits in their foundations (written about in Julia Coffman’s post here). More specifically, the measurement and evaluation team was able to introduce Making Thinking Visible and Asking Powerful Questions in our early stage support to program teams to push thinking about assumptions and concrete dimensions of the work. This engagement then helped to structure the foundations of a learning agenda (e.g. theory of change-like tool with clear outcomes, hypotheses, assumptions and evidence) that would be used to anchor adaptive management and continuous improvement once the program strategy rolled out.

--Veronica Olazabal & Janet Camarena