Monitoring and Impact Measurement
At MEDA, measurement matters. We don’t wait until the end of our projects to evaluate success. From the moment a project begins, we gather data to understand whether our interventions are achieving the results we expected, what we can learn, and which approaches need to be refined to help our clients reach their true potential.
MEDA strives for results. We use a Results-Based Management (RBM) approach and tools to collect and manage robust data related to our intended results. From project design to implementation all the way through to wrap-up, we focus on feedback loops to learn how each project can best contribute to our clients’ economic empowerment and prosperity.
Sustainability is important to us, and we want to see our clients continue to grow their incomes and operate successful businesses even after we leave. We measure key business performance indicators among past clients three years after our projects close and use this information to continually improve how we create business solutions to poverty.
Our measurement tools and approaches:
In many of MEDA’s projects, staggered entry of clients allows us to manage and improve interventions over the course of the project based on early results. Clients enter the project in a series of cohorts, which begin receiving interventions at different times within the life cycle of the program. A rolling baseline methodology allows us to use a “before and after” one-group design. The logic of this design is simple yet effective. Program participants are assessed before they work with MEDA and then again throughout their interactions with us.
Using baseline information collected with the entrance of each cohort, the project has a series of data collection points offering a “rolling” perspective on the pre-MEDA socio-economic conditions of clients. By comparing data collected at each of these points, a “rolling” control group is established.
Additionally, by comparing data at a single point in time from cohort 1 after one year to that of cohort 2 at baseline, we have an ongoing perspective on the progress of the project. In this way, we can establish a quasi-control group analysis. This technique mitigates ethical challenges related to traditional control group studies, whereby control respondents are not given access to the benefits of working with MEDA.
MEDA uses surveys to establish cause and effect relationships in the lives of our clients. We sample our entrepreneur and business clients and ask questions on variables to measure the effectiveness of the project activities, as well as key performance indicators.
Surveys are comprehensive, attributable, and allow our teams to gather the data they need to assess key indicators related to the business success and wellbeing of our clients. Usually conducted annually, they allow us to measure project results over time.
Mobile data collection
Mobile data collection is the use of mobile phones or tablets for data collection in rural and inaccessible places. Two main advantages include data available online in real-time and data errors are reduced.
MEDA works with mobile data collection platforms that allow offline data entry, allowing us to collect data from rural clients. We customise surveys to collect specific data needed by our teams, such as income data, enterprise performance and capacity, business metrics, photographs, GPS coordinates, and even photos.
Having this data at our fingertips faster allows MEDA to quickly pivot and make decisions based on realities on the ground.
Focus group discussions
Focus group discussions (FGDs) are interviews, but of 6-10 people at the same time in the same group. FGDs are designed to elicit qualitative data.
FGDs consist of diverse groups of people whose reactions are studied to understand the impact of MEDA’s work and measure performance indicators.
Questions are asked in an interactive group setting where participants are free to talk with other group members. During this process, the researcher either takes notes and records the vital points he or she is getting from the group. Researchers select members of the focus group carefully for effective and reliable responses.
Most significant change stories
The most significant change (MSC) story approach is a form of participatory monitoring and evaluation. MEDA involves stakeholders at different levels to discuss the sorts of changes expected and to review which are the most important. This method is valuable for learning about what types of unanticipated changes may be occurring in a project.
At its core, MSC is a simple process of asking our clients what changes they have seen in their lives since working with MEDA, and why these are important. MSC is an effective monitoring tool because it can occur throughout the program cycle and provides information to help people to manage the program.
It can evaluate project efforts because it provides data on impact and outcomes to assess the performance of a program. It is not based on pre-defined indicators but instead involves a systematic process of selecting the most significant stories, chosen by a panel of designated stakeholders.
- EDGET client stories doc http://meda.org/market-systems-publications/218-growing-entrepreneurs-ethiopian-stories-of-change
- EDGET Parboiling Case Study http://meda.org/market-systems-publications/224-edget-rice-parboiling-case-study
- GROW Market Actor Case Study http://meda.org/women-and-youth-publications/grow-greater-rural-opportunities-for-women
- Sarona Values Report https://www.saronafund.com/responsible-investment/values-reports/
- UHDP Most Significant Change Stories http://meda.org/market-systems-publications/259-uhdp-most-significant-change-stories-ukraine
- YouthInvest Business Case for Serving Youth http://meda.org/women-and-youth-publications/youthinvest/youthinvest-english/11-business-case