Challenge
The state of São Paulo’s education system is hampered by under performance and poor quality. In an effort to improve the quality of education, the Secretariat of Education introduced a program of interactive content and pedagogies. The Virtual Classroom project was intended to create a public-private partnership (PPP) with industry to manage the development and deployment of new interactive content and technology in the state secondary schools. With an estimated cost of over US$7 billion this project represented the largest technology and education project in the world. Moreover, a PPP of this magnitude had never been designed in the education sector.
Solution
To assess the costs and risks of this program, the Government of São Paulo requested World Bank support in reviewing private sector proposals for the modeling of the project’s PPP arrangement. The Bank’s Brazil Reimbursable Advisory Service(RAS) -- São Paulo Support to Education PPP Project provided the feedback for these proposals, analyzing risks, creating an evaluation framework, and developing a financial model.
At various stages of the process between August and November 2012, the Bank team provided input on the development of the technical model that emerged from the nine private sector proposals, including:
- Refinement of objectives.
- Key gaps in the model and options for filling those gaps.
- Monitoring and evaluation options.
- Intellectual property issues on content.
- Integration of content with learning management and content management systems.
- Advantages of outsourcing data management.
- Importance of a clear communication strategy and roles and responsibilities for project’s various components.
The Bank team provided a comprehensive review of the risks associated with the project. Along with the identification and classification of risks, the team provided potential mitigation strategies and provided input on which risks should be retained by the Government and which risks should be allocated to the private sector. Through the value of money analysis[1] the Bank team assigned probabilities and costs to these risks to model the financial viability of the project from a risk-cost perspective.
The team also provided a series of inputs to support the development of the monitoring and evaluation framework, the institutional arrangements for this framework, the integration of the framework into the financial model for definition of payment structures, and an outline of an impact evaluation for the project.
Finally, the team engaged in a series of steps to identify key aspects of the financial model and then develop and refine the model based on client feedback. The key deliverables for this aspect of the project included:
- An outline of categories for the financial model.
- A tool to compare the characteristics of the proposed financial proposals, assess the feasibility of the project's financial structure, and identify the key investment categories.
- A macro tool to run a number of simulations on the financial model.
- A final model and report on the simulation results.
- An economic evaluation of the project.
Results
The analysis was a major factor in the Government’s decision not to move ahead with the Virtual Classroom project. In addition, the engagement created a number of important results that would apply to other countries considering a large information and communications technology (ICT) and education-related project.
The PPP’s focus on outputs and outcomes instead of inputs was a very interesting model for service delivery payment and in the education sector would have been the first of its kind. The team also incorporated innovative concepts around how to leverage the project for Big Data opportunities and personalized learning. Also, the solutions around open content, ownership and privacy of data generated through the project presented new opportunities to develop digital content and software markets in Brazil.
The intensive process of risk identification, risk mitigation, risk value, and risk allocation in the context of a PPP was unique to the education sector. Some of the risks that had not been considered by the client included:
- Challenges in the proposed model around delayed implementation and payment to the private sector partner.
- The ten year time frame for the PPP.
- The potential low impact on learning outcomes (the importance of placing more focus on student engagement and satisfaction).
- Intellectual property and digital content concerns.
- The challenge of “technology lock” with the initial choice of learning and content management systems.
Some of the innovative approaches to monitoring and evaluation included transparency and openness in sharing data to engage the public and increase accountability of the private sector partner; Big Data opportunities to use data more effectively to both analyze the education system as a whole and individual learning progress; citizen participation in data and accountability; and the use of dashboards for real time data access.
The technical quality was very high and broke ground with new thinking about the challenges and opportunities of PPPs in education. The client was very engaged throughout the whole process. The team delivered all three outputs on time and included additional outputs such as an economic analysis and a legal analysis that were above and beyond the scope of the terms of reference.
Bank Group Contribution
The total cost to the Bank was around US$310,000.During this six-month intensive engagement, the Bank positively influenced a multi-billion dollar and multi-year project. The team drew on experience and resources from across the Bank including Human Development and Education Network, the Transport Sector, Development Economics (DEC), International Finance Corporation (IFC), the ICT group (TWICT), and the Infrastructure Policy Unit.
Partners
The Bank team worked with the Government team to develop and implement the financial model and to build the Government’s own capacity to run various simulations to test assumptions in the model. The main partners were the Secretariat of Education and the Secretariat of Planning and Regional Development.
The team helped the Government improve its understanding of the challenges and opportunities of digital content, particularly from a legal perspective, including issues around open source options, privacy, and Big Data opportunities. The associated risks, costs and benefits of this design approach were well fleshed out at the end of the engagement. Through this intense engagement, the team members at the Secretariat of Education strengthened their ability to design a first-of-its-kind PPP for education.
Moving Forward
The Government of São Paulo fully incorporated the Bank team’s inputs into the development of the model. While this project was initially on a very fast track to implementation, the Government decided to withdraw the original technical proposal to rethink its content and scope, such as the overall pedagogical model, the locus of teacher training, objectives for digital content, and review of risks and risk allocation.
Beneficiaries
The main beneficiaries are the Government of São Paulo and the state’s Secretary of Education, who indicated that through this engagement he has a clearer perspective on the areas of education that could be outsourced to partners outside of Government and has a clearer and stronger understanding of the costs and risks of these partnerships.
Secretary of Education Herman Voorwald, credited the risk analysis as one of the primary factors in the decision to postpone the project:
“... the request to postpone the project was made by the state Secretariat of Education, which highlighted the following as the principal motivations for rethinking the initiative: ‘the risk analysis of the project carried out by the World Bank, the consolidations of a new internal organizational structure—which presented a new context for the execution of projects—as well as new evaluations about the Projects basic assumptions’’ [2]
[1] Value for Money Analysis compares the expected financial impacts of the PPP approach with the expected financial impacts of a traditional public delivery approach.
[2] See website, http://www.pppbrasil.com.br/portal/content/estado-de-s%C3%A3o-paulo-arquiva-projeto-de-ppp-com-foco-em-tecnologias-educacionais-interativas.