Toward an Equal Start: Closing the Early Learning Gap for Roma Children in Eastern Europe: Evidence from a Randomized Evaluation in Bulgaria

Inequalities in development begin early in life. Children growing up in poor households often don’t get the nutrition and early stimulation they need for healthy development. Good preprimary programs can help poor children catch up, but many children are unable to enroll or attend regularly. This evaluation studied whether addressing financial and non-financial barriers can be effective for promoting early school access among poor, mainly Roma children in Bulgaria and promote greater Roma inclusion in European society more broadly.

Research area: Early Childhood Nutrition, Development, and Health

Country: Bulgaria

Evaluation Sample: 6,000 poor families in 236 settlements

Timeline: Completed

Intervention: Tuition Waivers, Conditional Cash Transfers, Community Outreach, Information

Collaborators: Elise Huillery, Sciences Po; Paul Gertler, University of California Berkeley; Eugenia Volen, Trust for Social Achievement; Joost de Laat, University of Utrecht

Partners: America for Bulgaria Foundation; Trust for Social Achievement; Open Society Institute, Sofia; Bulgarian Ministry of Education and Science, Central Committee on Research Ethics 

World Bank Task Team Lead (TTL): Plamen Nikolov Danchev

Challenge

Bulgaria’s population of 7.3 million includes some 700,000 to 800,000 Roma, the majority of whom are poor and live in vulnerable conditions. Roma children in particular suffer multiple disadvantages and few complete secondary school. The education gap between Roma and other children starts early; while more than 75 percent of all Bulgarian children aged three to six attend a preprimary program, many Bulgarian Roma children don’t go to preprimary school and they often live in homes without books or toys. Bulgaria recently passed a law making two years of preprimary education compulsory for all children, usually starting at age five. While fees have been lifted for children aged 5 and 6, in practice, parents are often asked to contribute to cover various fees, which can harm enrollment from poor families. For children ages 3 and 4, preprimary programs aren’t free, which is an impediment especially for poor families.

Intervention and Evaluation Details

Intervention

The Sofia-based Trust for Social Achievement, which financed the intervention, implemented a program with 20 civil society organizations that waived any preprimary fees for Roma children and gave some families a small stipend to encourage enrollment. A community outreach program sought to teach families about the importance of preschool.

Evaluation Design

The study took place in 236 settlements that had at least 25 households with children ages 3 to 6 years. The communities were randomly assigned to either receive one of three financial packages—removal of fees, or removal of fees with a monthly voucher worth about US$5 or US$13 for regular attendance— or be part of the control group and receive no incentives. The communities were then randomly assigned again so that half of each group were offered the community outreach meetings.

Eligibility

Poor households with children ages 3 to 6 years.

Results

Removing the costs of preprimary programs increased enrollment by 19 percent and improved attendance by 24 percent. Offering an additional cash incentive didn’t have any impact above removing school fees. Community outreach, however, wasn’t effective. But while more Roma children went to school, they didn’t show the same developmental progress as the Bulgarian children; and in some cases, Roma children in the program did slightly worse than children who didn’t go to school.

Impacts

The evaluation results have helped focus attention by policymakers and development groups in Bulgaria on the challenges of making preprimary programs work for all children. The Open Society Foundation now is funding an advocacy project to encourage officials to remove all fees. As part of this, an inter-institutional working group, chaired by the government’s Ombudsman office, will propose to the government an operational plan to remove all fees for kindergarten around the country.

*Additional impacts on policy decisions and program design

• The baseline results revealed a 70 percent preprimary enrollment rate for Roma children in Bulgaria – a figure significantly higher than expected and an increase from a 2011 survey, which found a 40 percent enrollment rate. The Ministry of Education and Science is using the data to better understand which government policies have helped bring about the improvement—as well as how to strengthen those policies even further.

• The government will continue tracking the children, specifically focusing on school readiness and examining the impact of three years of preprimary attendance. SIEF researchers shared their data sets so that the government can use the sample for a longitudinal study of children that tracks both the treatment and control groups.

Next Steps

Based on the results, the Trust for Social Achievement is partnering with the government and local organizations to test new approaches for improving preprimary education in Bulgaria and promoting access to all. A new curriculum that promotes and recognizes diversity will be evaluated with World Bank technical support. The evaluation also includes a component to give Roma parents information on what they can do at home to stimulate children’s development.

Evaluation Details

The Open Society Institute in Sofia established a list of 750 settlements (towns, villages or cities) that each had at least one neighborhood inhabited by a majority of Roma. The study took place in 236 settlements that had at least 25 households with children of preschool ages three to six.  While all households with preschool age children were eligible to receive the intervention, 25 were randomly sampled in each settlement to be part of the study, for a sample size of nearly 6,000 households. At a public meeting organized in Sofia in June 2014, the 236 communities were randomly assigned to one of four financial treatment groups described above, and to one of two outreach groups.

Prior to this public lottery, a baseline survey was conducted among all the households and kindergartens. Over the course of the 2014-2015, the Open Society Institute is carrying out unannounced attendance checks. At the end of the school year, with the support of Save the Children, the Open Society Institute conducted the International Development and Early Learning Assessment test of cognitive and socio-emotional development among the children as well as a follow-up questionnaire with parents and preschools. School administrative data was reviewed as well. Researchers also asked about social interactions between Roma and non-Roma, as well as general attitudes about inclusion.