How we use MEL at Kiron
How can an NGO ensure that it remains committed to its mission and accountable for its decisions? At Kiron we use a framework called MEL. MEL stands for Monitoring (or Measurement), Evaluation and Learning, and represents a set of steps, activities, methodologies and a mindset. We use MEL to continuously understand and learn in order to improve the impact we achieve with and for the Kiron students.
Since the beginning, Kiron’s research and learning analytics teams have built tools and processes to understand platform performance and helped us adapt our programs. We are now dedicating the roles of a MEL officer and a data analyst to build and run a MEL framework that reflects Kiron’s new collective impact model and continues to support Kiron and its partners in creating impact for our learners.
MEL at Kiron
We are building our MEL system on the basis of a Theory of Change (TOC), which contains the high level goals we want to achieve with and for Kiron’s students and the intermediate outcomes which are needed as preconditions to reach those goals.
Theories of Change typically come in the form of diagrams and are widely used in impact-oriented fields such as development cooperation, social change and humanitarian work. They are strategic tools, can become quite complex (just like the project they are depicting) and on the other hand have a strong applicability to the operational level of activities and outputs.
One of Kiron’s high level goals is to contribute to SDG 8 so that refugees and underserved communities engage in meaningful and decent work or self-employment. The outcomes we think can lead to these goals are (among others) that “Students gain the skills & recognition needed to succeed in education and employment” as well as “Kiron students are able to organize their entry to the job market”. The above mentioned outcomes are within Kiron’s sphere of influence. Systems level and political changes certainly have a profound impact on achieving SDG 8. We watch those closely but they lie outside our direct sphere of influence.
So how do we go about setting up and implementing our MEL framework at Kiron?
We collect relevant data (mostly quantitative such as course/session attendance) and we monitor it closely and continuously. We know what’s relevant from the pathways that were established in the Theory of Change: the outcomes and activities that are crucial for reaching the higher level goals. Not all aspects within the Theory of Change need to be constantly measured though as the system can quickly become very complex. Measurement data already provides insights into what works and what doesn’t, for example, if attendance rates are low, action can be taken to improve it.
Different measures (i.e. data sources and collection techniques) can be used for one single outcome, which is typical for more high level outcomes. E.g. for our aforementioned “Kiron students finish courses in preparation for local or remote job markets: we can employ analytics data such as course completion, qualitative student feedback as well as learning assessments to get a more complete picture. At the Evaluation stage we evaluate if outcomes were achieved and assess the profoundness of change. Evaluation goes beyond measurement.
How are we doing with our higher level goals? What worked and what didn’t? What should we change on the strategic level regarding general direction? Should we change or discontinue certain programmatic aspects? The Theory of Change can be adapted to include those learnings. While we certainly learn all the time, changes to the program and MEL setup should only be made in longer cycles of 6-12 months. This way we can properly base our decisions on big enough data sets.
Kiron is currently re-building its Measurement, Evaluation and Learning framework for the new collective impact model. We are developing a strategic approach to the different types of data and their purposes. By setting up the framework with the goal of supporting our impact partners in collecting meaningful data, we can understand if their programs are impactful and to help them make decisions to improve the setup for their learners.
Impact partners will soon be able to access measurement data as discussed above (e.g. completion or attendance data) through their customized MEL dashboard. A flexible survey system built into the CIC platform for structured or semi-structured learner surveys is on our development roadmap and Kiron can already support partners in collecting learner data from other sources (e.g conducting focus discussions).
Our vision is a collective of like-minded organizations and people where we work and learn together about successful approaches and where we share these learnings in order to bundle our energy for the sake of our learners. Our ultimate goals are as codified in the SDGs 4 and 8:
- Refugees and underserved communities engage in meaningful and decent work or self-employment
- Refugees and underserved communities have access to an inclusive and equitable quality education and lifelong learning opportunities
If we want to be able to reach these goals we need to first provide relevant and viable opportunities for refugees and underserved communities while changing the systems of unequal opportunities. We welcome you to join us and moreover to build the collective, the learning platform and the MEL framework together. Please contact our Partnerships Lead Lily at [email protected] if you want to find out more.