Most Significant Change

The method is well-suited in contexts where conventional monitoring and evaluation tools may not provide sufficient data to make sense of impacts and foster learnings (e.g. complex projects that produce diverse and emergent outcomes, projects focused on social changes, large projects with multi stakeholder levels, etc.)

The method has the purpose of providing a structure for learning from project experiences, by providing discussion categories and a template to collect input (knowledge sharing and documenting).

The process provides a simple means of making sense of a large amount of complex information collected from many participants across a range of settings, as well as identifying unexpected changes. It is a participatory form of monitoring that does not require expert knowledge or skills.

It consists in collecting stories from various stakeholders on changes occurring during the project, and is easy to implement and communicate across cultures. It can deliver a rich picture of what is happening and can be used to monitor and evaluate bottom-up initiatives that do not have predefined outcomes.

It provides a useful method to generate knowledge and facilitate improvements in particular, and it can also assist in fostering a shared vision, building staff capacity in impact evaluation, helping stakeholder steering committees act, etc.

MSC is a participatory method that encourages collective learning. It does not require expert knowledge and is simple to implement. However, it can be time-consuming however and a short in-house training might be useful for relevant stakeholders to understand the method. It is important to ensure stories representativeness that stakeholders from different hierarchies level of the project are involved.

Most significant change is a participatory method that involves many project stakeholders in telling/collecting stories, deciding the sorts of change to be recorded and in analysing the data.

The MSC process is relatively simple in practice, and a trial and error approach can be used. However, understanding the method is a frequent stumbling block so training people in MSC can be useful. In-house training takes 1-3 days and can be led by external consultant or an internal M&E specialist.

MSC occurs during a predefined period of time (e.g. 6 months or 1 year) after which a document is produced with all the stories selected of change by upper levels over the period in each domain.

The method is quite time-consuming, depending on the nature of the project and intended uses of the process. Hours will be needed to collect the stories, transcribe and analyse them. It is important to allocate enough time for the story collection phase for various stakeholders to enter into a meaningful dialogue about what was happening in the field.

Regular meetings must also be held to select the most significant stories; these will take about 1.5 hours depending on the number of domains and selection process.

The frequency of SC story collection can vary greatly depending on projects, from biweekly to yearly. Usually, reporting will be more regular at the start of implementing this M&E technique (e.g. every few weeks or monthly initially) and then evolve to a lower frequency as the process continues (e.g. quarterly selection). Lengthening reporting period (after several months or a year) will help reduce the time commitment. The whole process will become quicker and more streamlined as time goes.

The MSC process involves the collection of significant change stories from the field and the selection of these stories by a pre-determined panel of various stakeholders who first try to identify impact, and then regularly discuss the value of these stories of change.

A full implementation of the MSC method includes 10 steps:

Step 1: Starting and raising interest – introducing stakeholders to MSC, fostering interest and commitment to participate.

Step 2: Defining the domains of change – identifying the change domains to be monitored involves the identification of broad domains (e.g. changes in people’s lives, in the sustainability of activities…) by selected stakeholders. These domains are to be left loose (vs indicators) to be then defined by actual users.

This step is not essential; stories can be collected and analyzed as a group without being categorized, although using domains helps group the stories into more manageable lots which can then be more easily analysed.

Step 3: defining the reporting method – deciding how frequently to monitor changes taking place in the defined domains.

Step 4: collecting SC stories – a simple question (such as “what do you think was the most significant change that took place during the last month”) should be asked to people most directly involved. Respondents are encouraged to report why they consider the change to be most significant. There are many ways to identify and document SC, such as active searching via fieldwork, interviews, group discussions, or having participants write stories directly.

Step 5: selecting the most significant of the stories – stories (1-2 pages) are then analysed and filtered up through the projects’ authority levels, in a systemic and transparent way.

Step 6: feeding back the results of the selection process - it is important to record and communicate the criteria/reasons used to select a story at all levels so that each round of collection-selection is informed by previous ones and information is fed back to project managers.

Step 7: verification of stories – stories can be verified to ensure they have been reported accurately and honestly. This also provides an opportunity to gather more detailed information about these perceived significant events.

Step 8: quantification – this step can take place either when an account of change is first describe (with quantitative or qualitative information) or it can also be useful to quantify the extent to which MSC identified in one location have taken place in others within a specific period for instance.

Step 9: secondary analysis and meta-monitoring – the monitoring system itself is then monitored (e.g. analysing how often different types of changes are reported, who participated and how they affected the contents, …). MSC can be a rigorous process without secondary analysis but it deeper analysis of all stories can be useful (using thematic coding, against a logic model, analysis of selection criteria, differences between selected and not selected stories, etc.). Meta-monitoring is simple and strongly recommended to monitor: the number of stories and variation over time, whose stories are selected and whose aren’t, or the outcome of the stories (how many generated recommendations and how many of these were then acted on), among others.

Step 10: revising the system – the final step if to revise the design of the MSC process (e.g. changing domains, reporting frequency, types of participants…), taking into account what was learned from using the technique and analysing its use.

When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact.

MSC is an emerging technique and many adaptations have already been made. It is currently being used to evaluate hundreds of initiatives internationally, from assessing single projects to the evaluation of programmes in large organisations, such as the GIZ for the FORCLIME project in particular.

Davies, R. & Dart, J. (2005). The ‘Most Significant Change’ (MSC) Technique: A guide to its use. https://www.mande.co.uk/wp-content/uploads/2005/MSCGuide.pdf

Lennie, J. (2011). The Most Significant Change technique: A manual for M&E staff and others at Equal Access. Retrieved from https://www.betterevaluation.org/sites/default/files/ EA_PM%26E_toolkit_MSC_manual_for_publication.pdf

https://evaluatingadvocacy.org/doc/The-Most-Significant-Change-Technique.pdf

https://mande.co.uk/2010/lists/updated-msc-bibliography/