Lessons Learnt from Rwanda: Digitisation of Ultra-Poor Graduation (UPG) Monitoring and Evaluation
By: Julie Byun, Technical Advisor, Global Livelihoods
In August 2020, World Vision launched a year-long learning initiative to attain lessons from World Vision Rwanda’s (WVR) pilot use of Last Mile Mobile Solutions (LMMS) technology in the monitoring and evaluations (M&E) of the Ultra-Poor Graduation (UPG) programmes. Six months into the pilot, despite the COVID-19 pandemic and its subsequent challenges, the initiative has brought World Vision (WV) one step closer to how LMMS Solutions can be best used for UPG and where the technology adds the most value.
According to Aimable Nsengiyumva, Resilience and Livelihoods Technical Programme Manager for World Vision Rwanda, “I was excited by the findings, which did a thorough job of containing key lessons that we need to consider for the next phases of UPG implementation and monitoring.”
What is Last Mile Mobile Solutions (LMMS) Solutions?
LMMS is a mobile information management platform system for humanitarian response, which provides numerous solutions, including Digital Identity, Beneficiary Registration, Distribution in Kind, Cash-based Programming, Activity & Attendance Tracking, Flexible Forms and Management Insights & Analytics.
As part of the Flexible Forms Solution, data collected from LMMS can be merged with Kobo Toolbox, an open source data collection software, through the Unique Identification Number, or through simply scanning the unique bar code on the LMMS ID Card. Power BI can also be integrated to join information from other sources.
For example, household registration data on LMMS can be merged with routine household survey data collected by KoBo Toolbox, which can then be visualised using Power BI. The diagram below depicts the different functions available from the three systems that form LMMS Solutions integrated technology.
Why does UPG need a digital solution?
In UPG, households in extreme poverty are identified using a multi-stage targeting process and a package of multi-sectoral interventions is provided to the households for 24 months. Every month, a household coach conducts a monthly visit to monitor the household’s progress and mentor them according to their status to help them stay on track towards the agreed end-goal of ‘graduation’ towards a sustainable pathway out of extreme poverty.
Therefore, UPG’s need for a digital monitoring & evaluation solution should:
1. Collect, store, manage a vast amount of household data in an online database
2. Ensure data integrity by allowing ubiquitous access of database from different geographic locations
3. Track household’s longitudinal data and provide accurate data-based coaching to UPG participants
4. Rapidly make sense of the vast data to see the overall progress of the project and household
5. Adaptively manage project implementation strategy based on analysis results
6. Remove hassle for head of household coaches to carry/use physical notebooks
7. Safely secure UPG households’ personal data
8. Empower head of household coaches with digital skills
LMMS and UPG: what we learned from World Vision Rwanda’s experience
All in all, as the LMMS Solutions are not technology bespoke to UPG, it cannot satiate every conceivable need of UPG programmes, however, there are many perks to using it. Below are four major learnings that we have observed from this pilot so far:
#1. LMMS Solution’s ability to track longitudinal data is ideal for programmes like UPG, which works with specifically targeted households throughout the entire project cycle.
#2. As many are already familiar with systems, such as KoBo Toolbox, adopting LMMS Solutions’ integrated technology will be a smoother process with easier transition in comparison to adopting an entirely new system. This is also one of the reasons why World Vision Rwanda decided on LMMS Solutions for UPG.
#3. Although World Vision Rwanda is not yet at this stage, LMMS Dashboard Solution is anticipated to significantly reduce the burden on data analysis as it provides the analytics service, which converts collected data into dashboards ready for analysis. This will reduce time needed to run statistical software by implementers and allow more time to utilise data to make informed programmatic decisions. We will continue to follow World Vision Rwanda’s journey on UPG baseline and monthly household visits to elicit learnings on the ease of analysis using LMMS Dashboards Solution and how these can contribute towards adaptive management.
#4. While LMMS analytics function will allow project staff to effectively use and analyse data, it is unlikely that the technology can be used by household coaches to provide on-site personalised coaching based on data. This is because navigating three different systems (LMMS, Kobo Toolbox, Power BI) may be challenging for people with lower digital literacy andt he dashboards on the Power BI app are only accessible with an internet connection. A more likely scenario is that coaches will be responsible of collecting data using KoBo Toolbox questionnaires, and the analysis will be conducted by project staff and cascaded down to coaches during management meetings.
How World Vision Rwanda is using LMMS Solutions in their UPG Monitoring & Evalutation
World Vision Rwanda’s pilot has been instrumental in mapping out how the three systems can be used in the different stages of Monitoring & Evaluation. Below is a diagram of how World Vision Rwanda is using (and planning to use) LMMS Solutions for their UPG M&E cycle. The diagram lists the process in chronological order, starting from the formation of Monitoring & Evaluation framework to the collection of endline data.
Overall, we are encouraged by the pilot thus far and, as we gain insights, feel that LMMS could be a good option for UPG digital monitoring and evaluation. We are looking forward to launching the anticipated results at the initiative’s end date in August 2021.
For more information about this pilot or World Vision's Ultra-Poor Graduation project model, please contact WV’s Global Livelihoods Team at livelihoods@wvi.org.