MSC Method & Results

First a little bit of background, after results and the lessons learned!

What is Most significant Change? 

The most significant change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded (collection of significant change (SC) stories) and in analysing the data. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program.   It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole. And it contributes in strengthening organization learning and communication through the analysis process, when the staff comes together to discuss about the perceived MSC or the not perceived (Answering to the question: “what went wrong?”) 

Why collecting stories?

•          People tell stories naturally - indigenous 
•          Stories can deal with complexity and context 
•          People remember stories 
•          Stories can carry hard messages /undiscussables 
•          But stories not known for accuracy/truth  

Why the analysis process  
•          Analysis for discussions/communication
•          For learning: elaborate information and transform it in practical lessons
•          For improving our work/our organization work: increase efficetiveness in our activities
•          To create bridges among far places and different cultures/experiences
•          To understand numbers
•          To discover new important element difficoultly perceived by normal M&E tools (especially if qualitative) 

Why MSC is relevant for us? 
•          To complete qualitatively the quantitative M&E analysis
•          To improve planning (having a better understanding of what is going on in the field)
•          To help explain a point to a farmer (can be used as KM material)
•          To recruit new participants
•          To help explain a point to another member of staff
 

The SCAMPIS MSC technique

In our case, the SCAMPIS project, the technique was extremely relevant to complete the participatory M&E process started in 2010[1]. The methodology  was adopted by SCAMPIS in Guatemala and India, developed in a participatory way with IFAD Scampis M&E responsible. The tool consisted in the implementation of 4 steps:

•          Stories collection
•          Field level analysis and selection / second field level analysis and selection
•          National level Analysis (Hq) (only analysis no more selection)
•          Cross-countries analysis

The methodology have been reshaped and modified to better match: specific needs and context,  limited availability of resources (budget, time).
The main goal for Scampis MSC were:
i) understanding M&E numbers,
ii) knowledge and learn about (unexpected) farmers perception of the project/technology,
iii) fill the gaps between field reality and higher top management or donor,
iv) involve all staff in a coherent process of discussion and reflection of the work conducted.

Stories collections[2]

The methodology has been reshaped from the original MSC (Davis and Dart) in order to adapt to specific goal and context.
Collected by kids (age between 13-16 years) who will contemporaneously, as real reporters, film, photograph and ask questions to the project beneficiaries. A training for the kids was provided at the beginning of the day to break the ice, practice the use of the technologies and understand the reason why of the activity as well the questions for the adults.
Why working with youth? Kids are curious, they can easily learn how to use the technology, they will let the interviewed feel  comfortable (elder or senior people are not advisable as they would influence to much the interview). They can at the best preserve the “ethic” of the MSC stories collection that is: DO NOT INFLUENCE THE ANSWERS, LET COME OUT THE REAL STORY FROM THE INTERVIEWED. [3] Finally the see how important they are for their community!
                                                             
Figure 1 Stories collection India                                                               

 

 

 

 Figure 2 Stories collection Guatemala

  

 

 

 

 

 

Various levels of analysis

The analysis at various level was conducted using brain storming modality reflecting on each story form 4 different point of view:
  1. Who is the interviewed and what I understood of his specific life experience (general)
  2. MSC perceived and shared
  3. Difficulties and risks perceived
  4. New ideas (hidden stories)
After each analysis an overview of the findings have been discussed within the group. During the first stories analysis, the staff selected the “best”10 MSC stories, and during the second analysis among this 10 stories 6 have been chosen as most representatives.  The videos were translated and subtitled.[4] After the national analysis a cross-country analysis have been implemented (Guatemala national staff analysed the Indian video and vice versa).


SCAMPIS MSC Results 

The videos were analysed at 3 level: field staff level, national staff and international (cross-countries) level. 
Among all the videos (46) 6 videos for Scampis Guatemala and Scampis India have been identifies as Most Significant and representative of the project.
The videos were analysed and discussed. The results of those analysis are presented briefly in the two schemes below.

Lessons Learned about MSC use for Scampis

  1. Limits of the  analysis process were mostly due to time constrains and end of the activities in the field, no longer staff employed and involved in the project):
  2. Cross-country analysis is effective learning and knowledge tool used at national level and local level.
  3. Qualitative information storage: even if the MSC community suggested various different tools for appropriate storage and elaboration of the info accumulated (text, photos, videos) there has not been time to learn to use them, so has been used a simple excel programme.
  4. The strength of the tool adopted showed strengthens on the major areas of monitoring, evaluation, learning and communicating
  5. Organization development/learning: improve dialogue, trust and democratic discussion. Moreover it can fill the gaps between top management and field reality
  6. Project implementation: Activities implementation & Strategy adaptation: if expected results are not satisfied, the staff will understand it quick and can modify the approach
  7. M&E: Understanding and contextualize quantitative M&E  (it is compatible)
  8. Effectiveness:  Involve project beneficiaries  in improving project effectiveness.
  9. Communication: Improve internal and external communication
  10. Sustainability: it can be easily implemented and managed by communities and staff (low costs).
  11. Evaluation: It can be used for ex-post evaluation

[1] Video on Scampis M&E and MSC http://www.youtube.com/watch?v=LbjJ4If9yUI
[3] Additional information on how kids have been trained and involved in the stories collection, the central role they played, opportunities, pitfalls and lessons learned will be shared separately.





Useful bibliography






    Nessun commento:

    Posta un commento

    Iscriviti a: Post (Atom)