Education Innovation in Sierra Leone: What the Sierra Leone Education Innovation Challenge is teaching us, and what it can teach us in the future

This blog was originally written by: Fergal Turner (Evidence and Learning Advisor, Save the Children UK), Esther Elliot-Nyuma (Advocacy Campaigns, Communication and Media Director, Save the Children Sierra Leone), Lydia Kamara (Director - SLEIC, Save the Children Sierra Leone)

The second round of the Sierra Leone Education Innovation Challenge (SLEIC) evaluation, conducted in May 2024 and published in October 2024, presents a compelling narrative of progress. The results indicate significant learning gains, particularly in mathematics, where students have achieved performance levels equivalent to an additional year of schooling in a top-performing African education system[1]. This reinforces the programme’s potential to drive meaningful educational transformation.   

Save the Children is one of five providers participating in the Innovation Challenge. Our lot covers three districts (Bo, Kenema and Kailahun) and 65 schools. The headline results for our lot mirror those of the programme nationally.  The results of the evaluation indicate a strong probability that our interventions over the last two years have led to children in our schools learning significantly more than they would have in control schools. 

Unpacking our Results

While the overall results give us a positive indication impact, when we look more closely, the picture becomes more complicated:

1. In Maths, results in our schools improved significantly more than in control schools. In English, scores in all schools fell, but in our schools, they fell significantly less than in control schools. This means we had a significant positive effect on learning in both maths and English. 
2. In both subjects our treatment effect was apparently much stronger for girls than for boys.
   1. In maths girls scored lower at baseline than boys in all schools. By the end of year two these differences had disappeared in our schools but persisted in control schools. 
   2. In English, boys and girls had similar scores at baseline. By the end of year two girls had fallen behind in control schools, but in our schools no gap emerged. 
3. In Maths our treatment effect was strongest for learners with the lowest overall scores. This aligns the positive results for girls who had lower baseline scores – though it is not possible to say which is responsible for the visible effect (i.e., are our interventions specifically benefitting girls who happen to have lower scores, or students with lower scores who happen to be girls). An alternative theory is that our focus on the use of manipulative’s in mathematics has greater benefits for simple skills, supporting younger or further behind learners more. 
4. Our learning trajectories are not linear. In maths learners in lower classes made are learning more, and have gained more since baseline. In English there is a consistent dip in performance for learners in primary four. 

While there is a wealth of other useful data points in the evaluation report – these have been the starting point which we have used for further reflection. 

The Education Outcomes Fund has (rightly) emphasised the sense of pride, and potentially relief which comes along the results. They show that, two thirds of the way through the programme, the theory appears to be holding true – that significant learning gains can be achieved at a cost per child which is potentially scalable and sustainable. 

The indication is positive. It appears that what we are doing in schools is leading to improved scores in Maths and English. It also appears that what we are doing is reducing achievement gaps between boys and girls in both maths and English. But what does this change look like from the perspective of teachers, learners and communities? How can we test our theories about what is leading to change, and how can we build on this for the final year of implementation? What have been the internal and external factors which have facilitated successful programme delivery? Which elements could be institutionalised in local and national systems and practices, and what conditions would need to be in place for their success at scale? (How) has the outcomes funding model changed the way we work? 

What are we doing now?  

We are approaching the end of the third and final year of SLEIC implementation, with the final evaluation being conducted in May 2025. Since receiving the evaluation data, we have worked to make use of the data we have to make decisions about how to adapt and deliver our activities for the two remaining terms. 

To do this, we needed to fill in some of the gaps in our understanding. 

The evaluation data gives us a snapshot of what is happening in schools, but it doesn’t give us a full picture of what is changing, why it is changing and what we can do to build on and maintain progress. Data on learning outcomes is a road-sign not a road-map – it alerts us to something we should be paying attention to, but we need to dig deeper to know where it leads us. 

To do this we have spent time socialising and reflecting on the evaluation data with all of those involved in programme delivery. This had included teachers and head teachers, our programme officers and the government School Quality Assurance Officers they work with, and finally our own programme implementation team. For each of these groups the questions have been the same:

• Do these findings reflect your experience in our schools? 
• What may be causing the trends we see? 
• What (if anything) should we be changing to respond to the findings? 

Emerging insights from these discussions range from reflections on the placement and administrative burdens of senior teachers, to the encouragement of the use of the book banks we are providing schools. Looking at the differences in programme effects for boys and girls, school level stakeholders theorised that our engagement with parents to support their children’s learning may have a disproportionate effect for girls, by reducing their burden of household chores and making more time for learning at home. 

Some of this has immediately fed into our planning for the final two terms of implementation, and other act as questions for further exploration as part of our learning agenda for the SLEIC programme. 

Learning from SLEIC

A central aim of SLEIC is to demonstrate affordable models for school improvement which can be institutionalised and scaled by the government of Sierra Leone. As an organisation we are also interested in the potential for understanding and scaling cost-effective interventions to other contexts. 

Adapted from: https://media.nesta.org.uk/documents/making_it_big-web.pdf

Of course, evidence on the potential of is crucial for this, but institutionalising and scaling an intervention will depend on far more than demonstrating effectiveness locally. 

Understanding what this change looked like for those involved, and what internal (to Save the Children) or external (education system, local context) factors facilitated or impeded change will help us give meaningful direction on what elements of our interventions could be institutionalised locally or nationally, and how it could be done. 

EOF is working with the What Works Hub for Global Education on a process evaluation to capture this learning across the five providers as part of the first pillar of their learning agenda. 

Internally our own learning agenda aims to capture as much implementation knowledge as possible before the end of the programme. For our learning as an organisation, this will supplement the rigorous analysis of implementation processes and drivers of change which the What Works Hub will capture through their research.

In addition to the data from EOF’s evaluations, and our own monitoring activities, we are bringing together qualitative data from a sources including focus-group discussions at school (children, teachers and head-teachers) and community (school management committees and mothers’ clubs) level, and reflection workshops with programme teams and partners. 

We hope that this intentional process of reflection will allow us to better replicate successful activities in other contexts and will support the scale up of our work in Sierra Leone. 

Learning from our Engagement with Outcomes Funding

In addition to this we have another question. If the programme delivers cost-effective improvements to learning outcomes, how much of our success was due to the outcomes funding approach? 

Our hope for working with EOF is that the model of Outcomes Based Financing (OBF) will facilitate us in being more focused on outcomes, and the flexibility and evidence will give us a foundation to be adaptive and innovative in our approach. 

As is this the first collaboration we have worked on with EOF, we are interested in understanding:

1. Were these intended benefits of the OBF model for the SLEIC manifested in practice for us? 
2. As an organisation, what about our ways of working facilitated or hindered the outcomes-funding approach? 
3. Does engagement in OBF lead to sustainable improvements in our ways of working?  
4. Does OBF represent a cost-effective approach to achieving impact for children?

These questions form part of a wider OBF learning agenda, being delivered by Save the Children UK through the Innovation Hive. We see SLEIC as an unmissable opportunity to deepen our understanding of outcomes funding in Education, and the opportunity it may present for us. 

This aligns closely with the third pillar of EOF’s learning agenda for SLEIC, for which they are working with Ecorys to understand the impacts of the OBF model on providers. Our own exploration will have a more self-reflective and future focused agenda, focusing on how the model worked for us, what we did (or should have done) to maximise its benefits, and what we can learn from SLEIC for future engagement in OBF and with EOF. 

As always (and as it should), the evaluation data has given us answers that become more questions. It is hugely encouraging to see SLEIC learning outcomes moving in the right direction, and we hope that we can maintain this momentum and deliver transformational results for children in Sierra Leone. 

We also hope that we can be a valuable learning partner for the Government of Sierra Leone and EOF, sharing what we are learning to support sustainable change for schools and children, and to help deepen our understanding of whether, and how outcomes-based financing can help us to become a more accountable, outcomes focused and cost-effective organisation.

Where to from here? 

Between now and the end of the programme we have three main activities:

1. As we approach the end of the final term, we are gathering qualitative data from schools and communities through focus group discussions. We will analyse this against our learning framework and will use it to contextualise what we have learned from our monitoring data. 
2. As implementation of the programme ends, we will run a series of in-depth reflection workshops with programme staff and partners to reflect on what we have learned from implementation. This will capture what we have done, why decisions were made, and what internal and external factors facilitated programme delivery. 
3. Once the results of the final evaluation are published, we will reflect on the extent to which that we have learned from programme implementation compliments what we learn from the final results. At this stage we will also consolidate our reflections on the outcomes funding modality and how we believe in influenced our ways of working and results.

By the end of 2025 we will consolidate what we have learned into a single learning report, which can complement the work done across the three pillars of EOF’s learning agenda, and more widely to the evidence base of what works to improve learning outcomes, and on the effectiveness of outcomes based financing to support impact. 

Between now and then we are looking at opportunities to use what we are learning to support the scaling process, both at the local level, in collaboration with our district level partners, but also at the national level, identifying opportunities to share what we are learning with the Ministry of Basic and Senior Secondary Education (MBSSE), as well as with EOF and other SLEIC providers.