Define your impact
Are you making the difference you target?
by Elena Bondareva
I bet you are hearing more and more about impact: impact models, impact reporting, social impact indicators, and the like. Building on our extensive work in this, today’s installment of the Change-maker’s Handbook sets out to demystify the concept and leave you with some actionable insights.
Jargon
Like with any groundswell occurring simultaneously around the world, this topic is fraught with jargon: social impact, social return on investment, the “s” in ESG, etc. While some will argue for the absolute correctness of one term over another, after 8 international moves and work on 5 continents, I am not one of them. For me, it is not what it says but what it does. Pick the words that resonate with your people and focus on the content, I’d say.
Why does impact matter?
In basic barter economies, one can only make a living by creating value that is both in demand and obvious, such as horseshoes, clothes, or midwifery. The subjective assessment of value is a product of a more complex equation: salt or imported silks commanded premium prices because they claimed status. Today, neither price nor popularity mark genuine value. Sadly, a significant proportion of economic turnover has the opposite impact: plastic bags, sugar and trans fats in foods, and disposable fashion made in sweatshops clog our arteries and landfills while leaving no real benefit behind.
As our life gets more complex, it gets harder to trace the impact of any action.
Faced with this uncertainty, both government and the third sector – industry wasn’t being held accountable yet – settled for activities-based models. If there was a correlation between a child’s access to a tablet or laptop and that child’s performance in school, an organization may have focused on distributing tablets. But how much of a difference did that tablet make in a child’s education if the child (a) had no wi-fi, (b) was living through adverse childhood experiences (ACEs) like domestic violence or substance abuse, or (c) brought home the only device the entire family depended on for income, social connectivity, and entertainment? Collectively, we did not even know to care. We settled for reporting on outputs, not outcomes.
However rational and well-intentioned, failure to close the loop often leads to ineffective activity that sinks immense amounts of capital with no return. Sadly, failing to evaluate outcomes leads to underinvestment in what really matters in society, but that’s not the worst of it. Too often, public and non-profit funds have fueled a greater dependency on such funds or even made things worse. The false sense of impact created by output-focused reporting ultimately breaks public trust.
Through Dead Aid (2010), Dambisa Moyo shone the first floodlight on the flaw of the entrenched non-profit model: its celebration of throughput (e.g., of blankets or meals) rather than of breaking cycles of disadvantage. There are undoubtedly times to deliver bottled water, blankets, and canned food but this approach should rarely extend beyond emergency response.
Q: If an input doesn’t readily cause an outcome, how do we know if we are making the difference we target?
I have spent hours coaching leaders on this topic; how could I assess if my time was worthwhile? How do you – as an engineer, manufacturer, an elected leader, or a non-profit leader — determine whether the strategies, plans, budgets and pitches you sweat over actually matter?
A: You define and measure impact.
What is “impact”?
Simply put, impact is “the end” in the “means to an end.” It is the result, not the effort.
Sports analogy: while a lot of attention is devoted to what players do on, say, the soccer field, it is the goals that decide the game.
What is “social impact”?
“Social impact” has become a shorthand for social, environmental, and economic outcomes, acknowledging that they tend to converge. Social impact is distinct from the ROI or other metrics meant to capture the direct benefit to the organisation.
Can all value be measured?
Not yet.
Furthermore, some thought leaders argue that not all of impact should be measured. Even as we collectively pursue more accurate and accountable impact models, many pursuits are worthy of our best effort regardless.
What is an impact model?
An impact model is an accounting system that articulates the relationships between inputs (e.g., the tablets), outputs (e.g., time spent learning, literacy and numeracy levels, graduation rates), and impacts (e.g., whether that child ended up breaking out of the intergenerational cycle of disadvantage).
An impact model anticipates, monitors, and reports on net impact. It makes impact easy to understand and act upon. In this way, an impact model is like a financial model or a budget, except that it tracks social, environmental, and economic gains.
The most effective impact models expand on financial models to track impact as a function of financial input. This allows reporting on “social impact return on investment” (SIROI).
Elements of an impact model (quantified and tracked over time):
The problem. E.g., children raised in poor families have lower educational attainment.
The activity. E.g., the bouquet of targeted programs, incl. tablet distribution.
Outputs. E.g., improved literacy and numeracy. This is where most of even best-practice measurement stops because this is often the boundary of direct control. However, even if there is a strong correlation between an indicator and the ultimate change, stopping here may create a false sense of impact (i.e., “impact washing”) as well as miss the data crucial for truly closing the loop.
Impact. I.e., the targeted change, such as a child’s improved quality of life.
The benefits of an impact model
Declutter your operating model and align your activities and teams like never before.
Dramatically increase effectiveness in global or intersectional priorities.
Reset what is possible, reaching further than ever before.
Enhance public trust and approval due to clear and compelling reporting.
For your idea, consider:
What change do you want to create?
What are the 2-3 key indicators you would use to measure it?
How would that data be collected?
How do they shape your idea, investment pitch, financial model?
Please let me know whether you are finding this helpful and what you’d like to hear more of.