Outcomes-Based Instruments Work: Learning from the World’s Largest Education Impact Bond

Recent years have seen an increase in outcomes-based contracts as a means to deliver social impact with quality, equity, and efficiency. Unlike traditional grants, where payment depends on conducting specific activities and investing in certain inputs (such as training teachers and purchasing learning materials), payments in outcomes-based contracts depend on meeting pre-agreed outcomes. The concept has seen some success in piquing the interest of philanthropists and impact funders; the world’s largest education bond is a good case in point. The Quality Education India Development Impact Bond (QEI DIB) offers valuable insight into how to design and improve outcomes-based instruments for success.

In any development impact bond, a third-party investor takes the risk of failure from the outcome funder (typically a philanthropic organization), losing money if the program does not meet outcomes – but earning a return if it is successful.

Between 2018 and 2022, the QEI DIB supported four education partners in running diverse interventions – EdTech labs focused on adaptive learning, learning centres for out-of-school children, remedial education, and teacher and principal development. The programme, which aimed to improve literacy and numeracy skills for around 2,00,000 primary school children in India, exceeded its outcome targets over the four-year period. Partner organisations more than doubled the pace of learning for students who participated in the intervention compared to those who did not and significantly closed the learning gap caused by the pandemic.

The QEI DIB brought together a broad coalition of private, not-for-profit, and development sector partners to elevate learning outcomes across India. UBS Optimus Foundation played the role of the investor in QEI DIB and worked closely with Dalberg, the design partner and performance manager. Designing a program of this scale entailed forming an anchor consortium (of funders and investors), selecting education partners, developing a framework to measure outcomes, and structuring the finances required for the instrument. Performance management requires working closely with education partners to ensure that their on-ground activities run smoothly from start to finish. This way, the education model can be continuously developed based on what is working, to deliver student learning outcomes. 

Overall, the QEI DIB programme provides useful lessons for outcomes-based financing and designing instruments for the Indian education ecosystem at large.

Setting up an Outcomes-Based Education Programme

Considering multi-provider and multi-funder approaches mitigates risks and directs more funds towards programme costs

Different interventions drive learning outcomes in varied ways, each with their own strength and value. Education partners can choose between student vs teacher-focused, digital vs physical models, or academic vs socio-emotional learning. A multi-provider approach acknowledges this diversity, allowing partner organisations to test which models are the most effective and how these learnings can be applied across programmes.

A multi-funder approach can increase funding for the outcomes. It can also reduce risks for a single-outcome funder, whilst giving funders the opportunity to share learnings as they transition from paying for inputs to paying for outcomes. This is particularly important as outcomes-based financing is at an evolving stage and requires significant changes at an operational level from funders.

All in all, future outcomes-based programmes should consider adopting a diversified approach to reduce execution risks. Internal execution risks that a partner may face such as delayed hiring or external execution risks such as school closures due to COVID require activities to be adapted. Multiple funders and education partners increase the likelihood of better learning results and implementing those learnings across programmes. Albeit sometimes challenging to set up, the benefits of these programmes outweigh the initial groundwork.

Ensure that scale and sustainability are strongly intertwined at the design stage

Outcomes-based instruments should be designed keeping in mind the scale and sustainability of the outcomes. Programme designers can ensure that improvements take place at a systemic level by engaging governments upfront and embedding specific interventions within the public sector.

A useful example is the Ghana Education Outcomes Project (GEOP), which provides remedial interventions for approximately 70,000 out-of-school children and additional support to primary school teachers to improve learning outcomes for around 100,000 students. In this case, the Ghana government is one of the outcome funders, ensuring that it has a say in the programme’s design. This also enables it to use any key learnings and implement those in the wider education system. Similarly, under the QEI DIB, district and local-level government relations were established by each education provider to implement interventions at state schools successfully.

A wide range of investors participate in outcomes-based instruments when investment terms appear attractive

Certain outcomes-based instruments like impact bonds require the participation of investors, who assume the risk of non-performance. Currently, several impact bonds have seen participation from foundations and philanthropic funders, who typically seek to drive impact rather than look for market-based returns. Considering the time and money spent by investors on activities like design and management, the actual financial returns are sometimes lower than what is agreed upon.

However, the QEI DIB has shown that the impact returns in outcomes-based instruments can be substantial. For instance, the outcomes were approximately 50% higher for the same organisations that had previously implemented similar programmes through grant settings. The students’ pace of learning in partner organisations was also 2.5 times higher compared to students in non-participating schools. In order to encourage greater participation, the investment terms need to be made more attractive.

Financial returns for investors can increase when there is outperformance on social outcomes. In the case of QEI DIB, the investor returns were capped at an 8% Internal Rate of Return, regardless of the magnitude level of outperformance. This may make it less attractive from an investor perspective. After all, in a DIB, outcome funders make a payment only if the agreed-upon outcomes are met at the end of the program. This is different from a typical grants-making role where funding would be provided upfront based on activities or inputs).

The return an investor in an outcomes-based instrument makes is linked to how well the investor manages the capital requirements for the programme. Where the outcome funders should be fully focused on the price per outcome, the investor should focus on the delivery and funding. Some outcome funders tend to determine how capital is used or deployed, which becomes a serious hurdle for investors.

To attract a wider range of participants in the outcomes financing space, it is important that outcomes funders are fully committed to driving down the price per outcome and achieving more outcomes. This will allow programme implementers and investors to design the most suitable programmes.

Evaluating an Outcomes-Based Education Programme: Looking Ahead

Structure evaluation milestones to allow adequate time for demonstrating outcomes

While there are merits in frequent evaluations such as obtaining an early indication of performance or generating robust data, there are significant risks as well. Apart from increasing the cost, frequent evaluations may result in interventions not having enough time to show results. This is especially true of the education sector. Interventions that build the capacity of government officials or principals need longer lead times to show results in students’ learning outcomes, as compared to direct ‘inputs’ or interventions such as changing the syllabus for students or procuring e-boards for teachers. 

Ensure outcomes evaluation frameworks do not disincentivise outperformance

The evaluation framework should be carefully designed to ensure there are no disincentives for high performance. For instance, if the outcome targets for a particular period use the levels reached in the previous evaluation as the baseline, outperformance in a previous period would imply the need to improve on a significantly high baseline. Under the QEI DIB, education partners achieved close to 300% of outcome targets in the first two years. However, in the final years, the levels reached before were used as the baseline. This resulted in incredibly difficult targets, not just because of the high base effect but also because schools in India were shut down for a significant period in the final years of the programme. A longer-term evaluation focused on the achievement of cumulative outcomes can help mitigate this.

Account for potential external influences while selecting indicators and designing the evaluation framework

There is a need to carefully consider any positive or negative influence that unpredictable events such as the pandemic can have on outcomes. In the QEI DIB, student learning and enrolment rates were used to determine outcome payments. Enrolment in state schools increased during COVID-19, which resulted in partners in such schools overachieving on enrolment targets, despite not being directly responsible for it. Similarly, another education partner running Affordable Private Schools in overcrowded urban areas was negatively impacted by the pandemic, as families migrated back to their hometowns. In the QEI DIB, a 15% margin for enrolment dampened some of the effects of over or under-achievement. However, enrolment beyond 85-115% of the range still affected learning performances. Accounting for such indicators that can be impacted by external factors would be crucial for future outcomes-based programmes.

The Quality Education Impact Development Impact Bond’s learnings have heavily influenced the design and structure of ongoing programmes such as the ‘Back to School’ Outcomes Fund Development Impact Bond and Skill India Impact Bond. Learning the lessons across a wider range of interventions can support better design and evaluation of outcomes instruments by combining strong programmatic interventions with well-designed outcomes-based financing structures.

However, to implement this approach successfully, organisations must be willing to continuously adapt to the ever-changing contexts and needs of the sector. While we are already seeing a lot of traction for outcomes-linked funding in India, there is still the next set of hurdles to overcome. The first is to overcome regulatory barriers that can allow for outcomes to be procured by government entities and CSR funding. Then, we need to build a robust pipeline of on-ground providers – as the QEI DIB did with education – that have a strong focus on outcomes and can nimbly adapt to continue improving. Finally, building more reliable and readily-available outcome-based data can reduce the costs of evaluation. More reliable government survey data, such as the ongoing state and national level learning assessments, can incentivize funders to adopt more outcomes-based programmes to benefit India’s development sector.

Featured image of students at an intervention school in Delhi where SARD ran a remedial learning program, courtesy Dalberg and the QEI DIB.