Measurement Tension: Is it Possible to Impact Meaningfully, Given the Realities of Time and Control?
Is it possible to measure impact meaningfully, given the realities of time and control?
One famous story concerns evaluations of the federal Head Start pre-school program. An early study showed that the gains children made through the program disappeared after two to four years. But a more recent evaluation tells a different story. “Forty years later,” explained the director of evaluation at a major foundation, “the Head Start participants are statistically different. What looks like it washed out at one period of time has turned out to be very important later down the line.” Longitudinal studies are often prohibitively expensive, though — and of course they take time that is often simply not available.
The executive director of a youth program explained that his organization is held accountable by the city council — the program’s funder — for outcomes over which the school district actually has much more control. The city council knows this but has no formal control over the schools. “Sometimes,” he noted ruefully, “those who end up accountable have the least power.”
Takeaways are critical, bite-sized resources either excerpted from our guides or written by Candid Learning for Funders using the guide's research data or themes post-publication. Attribution is given if the takeaway is a quotation.
This takeaway was derived from Making Measures Work for You.