In the realm of social impact projects, measuring outcomes is crucial to understand the effectiveness and lasting impact of initiatives. This article explores how CLUSTER, supported by the Wealth Inequality Initiative by Julius Baer Foundation, leverages outcome measurement to bridge the gap between the high-end design industry and the informal market in Egypt.
By pairing informal artisans with creative designers, CLUSTER aims to foster collaboration and narrow the wealth gap. Let's dive deeper into the process of measuring results with our impact evaluation and measurement expert, Beatrice Scarioni from Dripstone Impact Consulting, and understand the importance for achieving long-term social change.
Beatrice, can you explain the concept and process of outcome measurement in the context of CLUSTER?
“Measure impact” has become a mantra for creating social change. Claims about making a difference are no longer sufficient; evidence of how much difference you’re making vis-à-vis your project participants (in other words, the artisans and craftspeople operating in Egypt’s capital Cairo) is now required.
Outcome measurement systematically analyses the broader effects emerging from a project’s outputs, the direct results stemming from the project’s activities. In the specific context of CLUSTER, we focus on what type of skills the artisans have acquired after participating in the skills-building workshops working with the designers. As a second step, we transform outcomes into key performance indicators (KPIs) that we will measure over a period of two to five years after a project’s implementation. This will also allow CLUSTER to understand how a project is creating social change, thus indicating its effectiveness in having a lasting impact.
Outcome measurement is described as a seven-step process. Can you outline these steps and elaborate on the importance of data collection, analysis and sharing of results?
Establishing a Baseline: the foundation of measuring change. To evaluate the impact of any activity, it is essential to establish a baseline, capturing the state of affairs before the project is initiated. For the CLUSTER project, baseline data could include the number of current clients for each informal craftsperson or the artisans' existing skill sets. This data provides a reference point for measuring change and progress over time.
Impact Canvas: mapping the path to intended outcomes. The impact canvas serves as a framework that outlines the project's planned work, activities, outputs and outcomes. It helps identify key metrics and assumptions to assess whether the project is on track to achieve its long-term goals. The canvas also integrates baseline metrics, allowing stakeholders to track progress and align their efforts with desired outcomes.
Turning Outcomes into KPIs: To quantify the desired changes over a specific period, outcomes are transformed into KPIs. For example, in CLUSTER, an outcome could be the acquisition of additional skills by craftspeople. The associated KPIs would measure the types of skills gained, such as proficiency in metal and woodwork or enhanced communication and collaboration abilities. These indicators enable stakeholders to track progress and measure the project's success.
Data Collection: the power of evidence. Collecting relevant data is crucial to monitor and measure KPIs over time. CLUSTER utilizes a mix of qualitative and quantitative data collection methodologies, such as one-to-one interviews and surveys with the craftspeople. This data provides evidence to assess whether the project is achieving its intended outcomes and sheds light on progress or potential obstacles faced by the artisans.
Analysing Data: unveiling insights and influences. Analysing the collected data involves comparing it with the established baseline metrics. This analysis helps stakeholders understand the intended and unintended changes brought about by the project. By examining both quantitative and qualitative data, insights can be gained into internal and external influences, successes and challenges. For CLUSTER, analysing the data might reveal improvements in product quality and sales processes due to the artisans' acquired skills, or external factors like inflation rates affecting their businesses.
Sharing Results: enabling collaboration and decision-making. Sharing actionable insights both internally and externally is vital to ensure the project's intended outcomes are achieved. Internally, the data points inform the project's future lifecycle, guiding planning processes and procedures. Externally, the results are shared through factsheets and progress reports integrated into the Julius Baer Foundation's digital collaboration platform. This transparency enables project partners and the Foundation to gain comprehensive insights into the project's development and achievements.
Incorporating Takeaways: enhancing impact and decision-making. Integrating the actionable insights into the project's intended impact is essential to maximise its effectiveness. When evidence demonstrates the project's intentions and realistic achievements, project partners can mobilise resources and explore future grant opportunities. The Julius Baer Foundation benefits from evidence-based decision-making, assessing the impact of current donations and ensuring grants have a targeted and meaningful impact on project participants.
"Measuring impact has become a mantra for creating social change. Statements about making a difference are not longer sufficient. Evidence is key"
What are the key challenges and considerations in implementing outcome measurement for CLUSTER?
In the context of CLUSTER the current economic situation in Egypt – with currency devaluation, a large informal market and lower employment rates - does present some challenges that can potentially hinder the outcome measurement. As a result, some of the artisans and craftspeople might be forced to close their activities. On the other hand, thanks to the connections and exchange between the artisans and the designers, the former group can expand their activities and reach a potentially higher number of clients to whom they can sell their artisanal creations in the future.
There needs to be a perfect balance between quantitative and qualitative approaches in outcome measurement. Can you explain how CLUSTER addresses this balance and leverages existing tools, such as progress reports and fact sheets, for data collection and analysis? Are there any specific challenges related to the socio-economic context or to ensuring representation of all genders in the data collection process?
As a result of Egypt’s fragile economic situation and large informal market, it is challenging for CLUSTER to collect quantitative data. On the other hand, by leveraging CLUSTER’s progress reports we focus on qualitative data that is collected via 1-1 interviews and surveys with the artisans which are conducted approximately every quarter.
Regarding the representation of gender in the data collection process, it is also worth noting that for CLUSTER’s first set of activities (carpentry) the artisans are all male whilst the designers are predominantly female. However, for CLUSTER’s activities around papyrus, the majority of artisans are female and the data collection process will focus entirely on women.
How does outcome measurement help assess the effectiveness and social impact of the project?
Although it is a time-consuming process, meticulous outcome measurement benefits all stakeholders. Partner organisations gain a clearer understanding of their initiatives’ progress – or of any unintentional adverse effect. Donors receive more comprehensive, data-based information on how their contributions are managed and with what effect. The Foundation obtains evidence-based criteria for its grant-making decisions.
Summing up
CLUSTER is an example of the power of outcome measurement in driving social change and integration between sectors. By systematically analysing outcomes and measuring results, stakeholders gain valuable insights into the effectiveness and impact of the project. Such insights enable evidence-based decision-making, foster collaboration and ultimately contribute to narrowing the wealth gap and promoting sustainable development in Egypt's creative design industry and informal market.
Beatrice Scarioni is the Founder and Managing Partner of Dripstone Consulting, a purpose-driven consultancy specialized in impact assessment and measurement that works across the private, public and non-profit sector to advance collective social impact and sustainability.
Beatrice has over a decade of experience working for impact-driven organizations. She has been advising and mentoring start-ups throughout Switzerland as a trainer for Innosuisse, the Swiss Innovation Agency, whose role is to promote science-based innovation in the interests of industry and society in Switzerland, where she teaches impact measurement and impact analytics targeted to impact ventures.
Susan Hermann is the responsible digital marketing and communications specialist for the Wealth Inequality Initiative.
Susan has more than 10 years of experience in digital communication and marketing in Peru, Spain and Switzerland. She has a master's degree in International Marketing and Business Management from the Rey Juan Carlos University and the EAE Business School in Madrid, Spain. In recent years, she has held specialized marketing positions in Switzerland, her main role being to generate a positive impact on organizations by taking her to the world of philanthropy and Foundations.
Hand in hand with the Wealth Inequality Initiative, Susan seeks to generate an impact on the reduction of inequalities and identify the community factor to generate a thought of change makers.
About Cluster
See how the Julius Baer Foundation and Cluster's BICEIA Design- Craft-Brand programme aims to narrow the wealth gap by fostering collaboration among high-end design industries and micro enterprises in the informal economy.
