Terms of Reference 
Independent Completion Report for the
Australia Indonesia Basic Education Program 

A.	Summary
1.	The Australian Agency for International Development (AusAID) will undertake an evaluation to assess the achievements of the Australia Indonesia Basic Education Program (AIBEP) 2006-2010.  The evaluation will review and analyse what the program has achieved, what has worked, what did not and why.  This analysis will provide important lessons to inform and shape early implementation of the next phase of Australia’s support to the education sector in Indonesia through the joint EC-AusAID Education Sector Support Program (ESSP).  The evaluation report will be drafted in accordance with AusAID’s guidelines on Independent Completion Report (ICR) preparation.

B.	Context 
2.	Australia has a long history of engagement with Indonesia’s education sector.  Education currently constitutes over half of Australia’s aid program.  AIBEP has been the largest single country sector program in the agency’s history, a measure of the importance of Indonesia’s human development to Australia.  AIBEP has a budget allocation of AUD387.6million (consisting of both AUD200million loan and AUD187.6million grant fund) and implemented by two contractors, the Managing Contractor Program Management (MCPM) and Contractor Strategic Advisory Services (CSAS).  MCPM focused on the access, quality and governance aspects of the program (Pillars 1, 2 and 3), while C-SAS provided technical assistance to improve the Government of Indonesia (GoI) capacity in strategic planning, sector monitoring and resource mobilisation to better addressing priority needs in education (Pillar 4).

3.	AIBEP has been supporting Indonesia to achieve its Education for All (EFA) and Millennium Development Goals (MDGs) by targeting the provision of universal access to nine years of basic education (six years of primary and three years of junior secondary) for all girls and boys.  The Program is financed through Indonesian systems, enabling the Government of Indonesia (GoI) to direct key aspects of the partnership - school site selection, policy dialogue and procurement.  AIBEP has worked closely with the GoI to improve access to education by funding the construction of over 2000 schools and to support the improvement of education quality and governance at school, district, provincial and national levels.  (refer Appendix 1 - AIBEP Design Summary).

4.	AusAID is making progress in ensuring that aid delivery in the Indonesian education sector is consistent with the Paris Declaration on Aid Effectiveness and with the Jakarta Commitment.  AusAID’s next phase of support will be delivered through the new Education Sector Support Program (ESSP).  It is fully aligned with the education strategic priorities outlined in Indonesia’s five-year Medium Term Development Plan (2010-2014) and education strategic plan (Renstra 2010-2014). ESSP will be delivered primarily through Indonesian government systems.  Australia’s approach to supporting education improvements in Indonesia has been evolving from contractor-managed activities parallel to the Government’s to greater alignment with and use of Indonesian government systems.  Presently 60% of Australia’s education assistance is delivered through Indonesian government systems.  It is envisaged that an increased proportion of Australian resources will be channelled through the country’s systems over the next five years.  

5.	Assessment of AIBEP implementation has been undertaken through the ESSP design process and the MCPM/AIBEP Schools Survey Results (in 2008 and 2009).   This independent evaluation will make use of design process evaluation findings and undertake further analysis.  The Evaluation Team will focus primarily on analysing data and evidence already available.  Collection of new data will only occur where necessary and will be limited to the key evaluation questions. The process of AusAID’s ESSP design is in its final stages.  This evaluation will therefore provide additional valuable lessons to inform, influence, and shape the preparation and early implementation phases of ESSP.

6.	With heightened attention given to accountability and aid effectiveness in the scale-up of Australian education assistance over the next five years – and increased delivery through government systems - this independent evaluation is critical in providing an evidence-based assessment of the performance and effectiveness of Australia’s current investment in Indonesia’s basic education sector and its impact, so that these can contribute to design and the operational planning of the next phase of Australia’s development aid in education sector. 

C.	Background on key issues of AIBEP implementation

7.	The evaluation team will consider, inter alia, the following key implementation issues: 

i)	Impact and sustainability of new schools built

AIBEP’s Infrastructure Component improved access to basic education by constructing over 2,000 junior secondary schools (1,500 general and 500 Islamic schools), in areas that are poor, remote and neglected.  An assessment by the ESSP’s design team found that building schools does increase equitable access; each classroom built under AIBEP resulted in 30-40 new enrolments and students having to travel considerably less distance to school.  On the other hand, AIBEP surveys assessing the impact of school construction found that some schools have lower than predicted enrolments.  

There have been some positive trends in terms of sustainability.  All schools constructed to date are operational, with government providing teachers and financing.  Ministry of National Education (MoNE) has adopted AIBEP’s manual on Asset Management and Maintenance for all GoI schools.  MoNE officials have also been trained to implement the Construction Reporting System (CRS) and Complaint Handling System (CHS) developed under AIBEP and have allocated resources to operate these systems.  Nevertheless it remains to be seen whether the above systems will be fully adopted/integrated.

Factors that may undermine sustainability include:

•	Surveys indicate that selection of location and school type has been less than ideal and in some instances has resulted in less than optimal use of schools.  There has been little non-formal enrolments in AIBEP constructed schools since GoI has yet to implement its non-formal education strategy.
•	Targeted districts under AIBEP generally do not have sufficient involvement (possibly due to lack of resources and capacity) in implementing and monitoring school construction and ensuring problems are fixed.
•	While officials of Ministry of Religious Affairs (MoRA) have been highly committed to the program, the absence of a construction program and thus construction management expertise in the ministry presented significant operational challenges, including the fact that the schools constructed were largely for private foundations outside of direct GOI management.  In this context, the evaluation team will be required to also review the impact and the sustainability of madrasah constructed under AIBEP.  

The AIBEP construction component is a significant investment, and AusAID needs to ensure that program benefits are sustainable.  The evaluation team will assess the impact of the school construction and look at the prospects for long-term sustainable returns on AusAID’s investment, taking into consideration the sustainability of the physical infrastructure; whether the district governments have the capacity and ownership to support these schools; and whether the above CRS and CHS systems put in place by AIBEP will continue to be used by GoI after the program concludes.  The evaluation team will also assess whether the quality of the buildings constructed meet Indonesian building standards and are of comparable quality to schools built by GoI during the same time period. 

ii)	Working through government of Indonesia systems 

Most of the infrastructure component is loan-funding.  The AIBEP differs from a more traditional development partner program in that implementation is managed by the Indonesian Government, not by AusAID or its contractors.  AusAID’s role is that of a financier who monitors the activities taking place to ensure that funds are being spent appropriately, and provides support as necessary.

Working through government systems has presented opportunities for AusAID to learn about GoI systems and where additional safeguards and improvements are required.  In 2009 underspent loan funds were used by MoNE to construct 60 additional schools and add facilities to already constructed AIBEP schools exclusively using government systems (i.e. without a managing contractor) for constructing schools, making this component more directly an earmarked budget support.  The modality used to deliver the unspent loan funds therefore presented opportunities to pilot increased use of government systems and to test the assumption that it is more cost effective than using a parallel system.  

However, the program’s use of government systems (for MoNE construction) has presented the following challenges:

•	Despite a sustained increase of central and district education budget allocations and spending for the education sector, recent budget allocation for MoNE have impacted on the ability of the government to manage construction issues (including CDC performance) that if not urgently addressed will impact on the transparency, accountability, and quality of construction. 
•	Management of complaints received through the AIBEP complaints handling system relies on government investigation and resolution of complaints, and the capacity (technical and resource) of MoNE and MoRA to do so.
•	Inability of many district governments to perform their role in monitoring and supervising construction, due to lack of budget, capacity, and perceptions that the program is centrally managed and implemented.

Therefore, the evaluation team will examine and consolidate lessons (to include previous and future challenges and opportunities) on working through government’s systems under AIBEP.  As AusAID moves towards increased use of local government systems, these lessons will feed into ESSP implementation and also inform the broader agency 

iii)	Effectiveness and cost-effectiveness of capacity building for new principals and district education personnel
GOI does not have a formal policy on principals’ induction program and provides very little training for principals of new schools.  New schools receive no budget allocation until two years after the school begins operation.  Each district is responsible for dissemination of the guidelines on school planning, budgeting and school based management to the schools.  
AIBEP filled this gap with its capacity building activities under the Whole District Development and Whole School Development (WDD/WSD) component.  These activities addressed the vacuum of capacity building and ongoing development and quality improvements support for the school’s managers (Principals and District Education Officers).  Although too early to see the long term impact, there is confidence that the WSD/WDD inputs have achieved their objective of providing a foundation for the initial operations of the new schools.  
There is a need for the evaluation team to investigate the cost-effectiveness and sustainability of this AIBEP sub-program.  The team will analyse what capacity-building gains have been made through the WSD/WDD induction process and whether this school/district capacity building was cost effective. These findings will help inform capacity-building activities proposed under ESSP. 

iv)	Impact of support for analytical and technical assistance services

AIBEP has supported GOI policy formulation, such as the GOI Mid Term Assessment of the education sector for Bappenas (National Planning Agency) and TA to develop the five-year Strategic Plans for MONE and MORA.  AIBEP has been supporting GoI in conducting research and analysis to improve the technical quality of education data and enhance the analytic capacity of GoI.  AIBEP has repeatedly been requested to provide analysis to assist GoI’s high level policy making.  GoI has called for similar assistance for the future, and AusAID will do this through the new ESSP Analytical and Capacity Development Partnership (ACDP).  The ACDP is a facility established within the ESSP to provide for good quality analysis and research that will support GoI education policy making and organisational capacity development.  

The evaluation team will investigate how AIBEP’s analytical and technical services have contributed to government policy making and organisational development.   The evaluation team should consider the key question of whether the $20 million spent on analytical and technical services (including improving the education quality assurance system) has the intened influence to impact on GoI strategic policy development in the education sector.

8.	The above implementation issues shall be considered and serve as the basis of the key questions presented below, particularly the implications of the above factors that will feed into ESSP implementation and to the broader AusAID learning.

D.	Objective of the Evaluation
9.	The objectives of this evaluation are to: 1) evaluate the extent to which AIBEP achieved its objectives; 2) assess Australia’s impact on education sector development; and 3) provide lessons learned that will inform and shape the early implementation of the ESSP.

E.	Scope of the Evaluation

10.	The ICR will assess the program’s overall performance by assessing against eight evaluation criteria: relevance, effectiveness, efficiency, impact, sustainability, gender equality, monitoring and evaluation, and analysis and learning.  Standard Evaluation Questions to guide ratings on these criteria are provided (Appendix 2).  

11.	The key questions that the evaluation team shall focus on examining are:
(i)	whether AusAID’s loan of $200 million to finance 1570 new public schools (under the Ministry for National Education) and whether the grant provision of $60 million to finance 505 private madrasah (under the Ministry for Religious Affairs) have had the impact that was expected at design, and examine the prospects for long term sustainable returns on the investment 
(ii)	Lessons learned in working through the government systems (MoNE public schools), including the use and effectiveness of  the Construction Development Consultants (CDCs)
(iii)	whether the $35 million investment in new school/madrasah induction processes (and district capacity building) was cost effective in design and achieved the anticipated outcomes
(iv)	whether the $ 20 million spent on analytical and technical services (including improving the education quality assurance system) had provided the expected impact on GoI thinking in strategic and policy development in the education sector.
12.	The below guiding questions are provided as additional reference to the evaluation team when considering the evaluation criteria specific to relevance, effectiveness, efficiency and gender.

Relevance
•	Were the objectives of the program appropriate and do they still remain relevant?
•	If the development context has changed over the life of the project, how have such changes impacted on the program?
•	How has BEP contributed to the Indonesian education sector in relation to other development partners’ contributions over the period?

Effectiveness
•	To what extent were the stated end-of-program outcomes achieved:
-	AIBEP program outcomes (in the design document)
-	Social equity (pro-poor)
-	Inclusive education outcomes		
-	Paris Declaration outcomes
•	What did AIBEP contribute to GOI’s broader EFA and MDG goals?
•	What is the likelihood of sustainability for these outcomes after the program, particularly those relating to district governments capacity to sustain the capacity building support?  Are there any areas of investment that will not be sustained?
•	What were the facilitating or inhibiting factors to achieving (or not achieving) these outcomes
•	What have been the implications of these factors in terms of achieving (or not achieving) anticipated end-of-program outcomes? 
•	What was the quality of key program deliverables (infrastructure component, capacity-building, and TA support for policy development/dialogue)?

Efficiency 
•	How effective were the program’s M&E arrangements?
•	How effective were the program’s governance arrangements (technical and managerial oversights)?
•	How effective were the program’s strategic and annual planning processes? 
•	Was risk monitoring and management integrated into design and implementation?
•	Did key program interventions give value for money?

Gender
•	To what extent were the anticipated end-of-program outcomes improved and/or achieved gender equality, including access, benefits and decision-making?
•	What was the reach and coverage of key program deliverables (number and type of beneficiaries - e.g. women and other vulnerable groups)?

13.	The structure of the report will be informed by the Independent Progress Report/Independent Completion Report (IPR/ICR) template.  Ratings will use the standard AusAID six-point scale outlined in the IPR/ICR template (Appendix 3). 

14.	The final report will provide findings for assessment of the overall program, lessons learned and recommendations related to future investments in the sector, particularly which will feed into ESSP implementation and to the broader AusAID learning.  The primary audience of the ICR will be AusAID’s Management and the key counterpart of the Government of Indonesia, i.e. MoNE, MoRA and Bappenas (Indonesia’s National Planning Agency).
F.	Evaluation Method and Processes
Roles and Responsibilities of AusAID:
15.	AusAID’s staff at Jakarta Post and in Canberra will provide oversight of the overall evaluation process.  Counsellor, Education, is the Evaluation Delegate and the Senior Program Manager, Education is Evaluation Manager. Other AusAID staff involved will be from the Performance and Quality Unit, Jakarta, and in Canberra, the Performance and Information Section and the Education Thematic Group (ETG).  The AusAID’s Indonesia Education Section will provide for management of inputs and quality oversight of method and process. 
16.	The Evaluation Manager is responsible for day-to-day management of the evaluation process.  This includes: initial and ongoing consultation with the partner government, other development partners and stakeholders; contracting and briefing the evaluation team; consultation with the Evaluation Delegate on approval of the Evaluation Plan; arranging team itineraries and making arrangements for fieldwork and for the Stakeholder’s Workshop; provide written approval of deliverables (the Evaluation Plan, Aide Memoire, Independent Completion Report); implementing the management response and Learning and Dissemination Plan. Other staff members (Education Advisor, AIBEP Activity Managers, and Unit Managers) will provide technical input in reviewing the Evaluation Plan, Aide Memoire and the draft and final Independent Completion Report (ICR).
17.	The Jakarta Post’s Performance and Quality Unit will contribute to ensuring that this evaluation is of high quality, timely and that findings feed into initiatives and the preparation/implementation of the ESSP.  The Unit will participate in the following activities: review mission meetings, attend inception briefing, and presentation of initial findings and stakeholders’ workshops; review and provide feedback on the draft Evaluation plan, Aide Memoire, the draft and final Independent Completion Report.  If required, the Unit will also provide briefing on AusAID’s requirements for the Evaluation Team. 
18.	Canberra’s Performance and Information Section, the ETG and AusAID’s Education Advisor will provide additional support to the Jakarta Post’s Team, including participating in the inception briefing and presentation of initial findings (via video-conference); reviewing and providing feedback on the aide memoire, the draft and final Independent Completion Report. 
Roles and Responsibilities of the Evaluation Team:
19.	The Evaluation Team will consult key stakeholders within AusAID, key GoI counterpart ministries (such as Ministry of National Education and Ministry of Religious Affairs) and beneficiary communities.  Field visits should be conducted in consultation with relevant GoI representatives and with in liaison with AusAID and the AIBEP’s managing contractors. The Evaluation Team is expected to work collaboratively with GoI, and to share information and analysis at appropriate junctures. Regular briefings with AusAID, and the GoI should be scheduled.  When feasible, update sessions to share and discuss methodology and preliminary findings are to be conducted during the mission.
20.	A review of key documents relating to AIBEP should be conducted by the team at the commencement of the assignment, including an appraisal of the design documents, progress/performance reports and monitoring data, in order to determine the information that is already available and to inform preparation of the Evaluation Plan/Methodology. Key documents will be provided by AusAID to the Evaluation Team at commencement of the assignment.  

21.	The Team will participate in the following process activities:

•	In-country Inception Briefing: the team will attend a briefing with AusAID in Jakarta at the commencement of their in-country inputs with the view to presenting, discussing and finalising the Evaluation Plan. 

•	Field Visits Report: The Evaluation Team is to carry out up to two field visits of not more than a total of five working days in duration.  The Team Leader will provide a justification and allocate responsibility amongst the team members for each field visits as part of the Evaluation Plan.  The number of field trips and locations are to be negotiated during the inception phase of the assignment.  Summary reports of the field visits will be submitted to AusAID in electronic form two days after completion of the field trip.  The field visit report will include references to stakeholders consulted and will also form the basis for reimbursement of any associated costs.

•	Presentation of initial findings to AusAID: the Evaluation Team will present its initial finding in a draft Aide Memoire, to be presented and discussed at a meeting involving staff in the Jakarta Post (the activity managers, the ICR evaluation delegate, ICR evaluation manager, the Performance and Quality Unit) and staff in Canberra (Performance and Information Section, The Education Thematic Group and AusAID’s Principal Education Advisor).

•	Stakeholders Workshop: the Evaluation Team will conduct at least one in-country stakeholder workshop to present the team’s preliminary findings and discuss a draft outline of the ICR.  The purpose of the workshop is to seek feedback/validation of the preliminary findings from a wider audience beyond AusAID.  Participants in the workshop will include the managing contractors, GoI counterparts, AusAID representatives and other development partners working in the education sector, as appropriate.

22.	The Evaluation Team is expected to deliver the following key outputs:

•	Evaluation Plan: This plan will outline the scope and methodology of the evaluation (including: a method for assessing the impact of the program, a process for information collection and analysis, and identification of any challenges in achieving the evaluation objectives); terms of reference for each team member and allocation of tasks among team members; key timings; a consultation schedule identifying key stakeholders to be consulted and the purpose of the consultations; activities/research to be undertaken; and a draft schedule of field visits (up to two visits per team member).  As part of the Evaluation Plan, the Team should also present a draft outline of the ICR report for AusAID’s consideration.  The AusAID Evaluation Delegate will oversee the final clearance of this plan.

In developing this plan, the team is expected to conduct consultations with the AIBEP managing contractors and AusAID staff (Canberra and Jakarta).  The Team Leader will have up to six days to prepare an Evaluation Plan from commencing the assignment, while other team members input will be not more than four days. S/he will submit the Plan to AusAID two weeks before commencing in-country inputs by 22 February 2010.  AusAID will review and provide approval of the Evaluation Plan not later than 5 March 2010.

•	Aide Memoire: The Team Leader will submit a draft Aide Memoire (maximum 5 pages) on key findings not later than eighteen days after commencement of the team in Indonesia.  The draft Aide Memoire will be prepared in reference to the Aide Memoire for Evaluation template (refer Appendix 4). The Team will prepare the draft Aide Memoire and submit to AusAID for review two days prior to broader circulation to other key stakeholders (such as GoI).  AusAID will be responsible for the translation of the draft Aide Memoire into Bahasa Indonesia for wider dissemination within two work day of receiving the draft Aide Memoire.  The team will present the draft Aide Memoire at a Stakeholders’ Workshop (see process activities above), prior to it being finalised.  

•	Independent Completion Report: the Team Leader will have up to five days to write and submit the draft ICR (25 pages in length, excluding annexes).  The draft shall be submitted five days after the Stakeholder Workshop while the Team Leader is expected to still be in Indonesia so that s/he can approach AusAID and other stakeholders with any follow-up queries. This report will be submitted to reviewers who will provide comments on the strengths and weaknesses of the draft and recommendations for its finalisation.  AusAID will provide feedback to the Evaluation Team within three weeks upon receipt of the draft report from the Team Leader.  This timeframe is to allow for feedback from a member of the ODE’s Review Panel, Education Thematic Group’s Principal Education Adviser and GoI stakeholders.  AusAID will provide consolidated comments on the draft report to the Team Leader, who will then submit the Final ICR up to four working days later (which will be done from home-country).  Other team members may have up to two days of input in the finalisation of the ICR.

G.	Duration of the evaluation

23.	It is envisaged that three consultants are to be procured, as specified in the Section H. 

24.	The assignment will start on the 15 February 2010.  For this evaluation, the team will have seven day work week in-country.  There will be a total of up to ten home-based days and up to 26 days of input in Indonesia (for the Team Leader).  Other team members will have less work days, but total effort will not be more than 30 person days, subject to negotiation of work plan during the inception phase.  The assignment is expected to have a total of up to seven weeks intermittent work (an overall duration of approximately fifteen weeks), and be completed by the week of 3 May 2010 (i.e. the submission date of the Final Independent Completion Report).  The Team will commence in mid-February to complete inputs associated with the inception phase (i.e. the Evaluation Plan).  Total work days and the completion dates will vary according to each position.

H.	Specification of the Evaluation Team

25.	The Evaluation Team members must be independent of the aid activity, and should not include any members who have had past involvement with the activity.  The focus for personnel on the team will therefore be on bringing together expertise on monitoring and evaluation, as well as an education and infrastructure specialists, in order to provide broader knowledge on program-based approaches. The team will comprise of the following members:
a)	Monitoring and Evaluation (M&E) Specialist/Team Leader

The M&E Specialist will have a strong background and experience in evaluation methods and processes, previous proven skills and experience in conducting review and performance evaluation, and demonstrated ability to draw on international best practice to inform the mission. The Specialist will possess very high analytical skill, an ability to gather and interpret data and information and write constructive, informative reports. The M&E Specialist will have a forward-looking perspective in terms of looking for lessons and implications to inform future programming. 

The Specialist will preferably have a sound knowledge of AusAID corporate policy on quality reporting system and business process for aid delivery; conversant with AusAID development assistance procedures/regulations and policies.  S/he will have substantive knowledge of evaluating large-scale loan programs and high familiarity with the Paris Declaration on Aid Effectiveness.  S/he will have working knowledge and familiarity of cross cutting issues such as public financial system and anti-corruption issues, gender, partnership, together with an understanding of Indonesia social and political context (Indonesian language skills desirable).  S/he has a high level of professionalism and commitment to delivery of results and excellent report writing skills (in English). 

The Specialist will also perform the role of team leader. They will effectively utilise the expertise of each team member in meeting the Terms of Reference and contractual obligations. As team leader, the Specialist will be ultimately responsible for delivering a quality evaluation report. Thus, team leadership skills are also essential.  

The M&E Specialist/Team Leader will be responsible for the following outputs: drafting and submitting an Evaluation Plan; drafting and finalising the Aide Memoire; presenting preliminary findings at an internal AusAID meeting and the Stakeholder Workshop; and drafting and finalising the Independent Completion Report.  S/he will lead the evaluation process, including participating in the inception briefing; assigning tasks and responsibilities among the team members; conducting site visits; presentation of initial evaluation findings in a draft Aide Memoire; and conducting a Stakeholders Workshop.

b)	Education Specialist 

The Education Specialist will be a senior expert with comprehensive knowledge of the implementation of development cooperation in the education sector, and preferably the Indonesian education system.  The specialist will have proven experience in analysis of education sectors with focus on EFA and MDG accomplishment and experience in the evaluation of major donor-funded education programmes covering basic education access, governance and quality in a decentralised system. The Specialist will have in-depth involvement in participating/leading in evaluations of international development partner assistance (with particular focus on reviews of the loan programs) and have demonstrated ability to write assessment reports in English.  Working knowledge of social evaluation methodology is highly desirable.

The Education Specialist will be responsible for assisting the Team Leader in the preparatory work before conducting the in-country part of the evaluation in Indonesia; analysing the findings of the evaluation; participating in meetings and field visits and other events as specified by the Team Leader; and actively contributing towards the quality and the delivery of the required outputs.  The Specialist will focus on providing advice on technical issues, contributing knowledge on the education sector development context, and providing relevant briefings and/or papers as requested and/or agreed with the Team Leader. 

c)	
Infrastructure Specialist

The Infrastructure Specialist will have relevant experience in planning design and implementation of social infrastructure. S/he will have suitable experience in building construction and project management and previous experience in conducting reviews and evaluations of infrastructure projects. S/he will possess good analytical skills, an ability to communicate with teachers and school management committees, gather and interpret data and information and write constructive, informative reports. 

The Infrastructure Specialist will have a sound knowledge of appropriate standards of design and quality requirements for educational facilities built in Indonesia or similar countries. S/he will have substantive knowledge on appropriate methods of implementation by community based construction and activity implemented through government systems. S/he will have a high level of professionalism and commitment to delivery of results and an excellent report writing skills (in English). 
S/he will be responsible for providing short term input to evaluate and appraise the appropriateness, quality and value for money of infrastructure projects delivered under AIBEP. S/he will review (but not be limited to) whether the infrastructure project:

•	Meets functional requirements; 
•	Meets quality requirements; 
•	Delivered on time; 
•	Delivered on budget; 
•	Represents value for money; 
•	Contributed to capacity building of local community. 

I.	Key Documents: 
26.	A review of key documents relating to AIBEP should be conducted by the team at the commencement of the assignment, including an appraisal of the design documents, progress/performance reports and monitoring data, in order to determine the information that is already available and to inform preparation of the Evaluation Plan/Methodology. Key documents will be provided by AusAID to the Evaluation Team at commencement of the assignment.  AusAID will provide all team members with the relevant documents as preparation of the literature review and document appraisals (refer Appendix 5).

Appendix 1
Australia-Indonesia Basic Education Program 
(AIBEP) Design Summary
JULY 6, 2006

1.	INTRODUCTION
1.1.	The Australia-Indonesia Basic Education Program
The Government of Indonesia has made a commitment to achieve Education for All by 2010.  To reach this goal, the Ministries of National Education (MoNE) and Religious Affairs (MoRA) have developed strategic plans  that accord top priority to increasing access to basic education, improving quality and standards, and strengthening education governance and accountability mechanisms. The A$355 million Basic Education Program (BEP) has been designed to support these Government of Indonesia (GoI) priorities. 

1.2.	Design and Preparation History 
The BEP Design was developed between September 2005 and April 2006.  The Australia-Indonesia Basic Education Program Design Proposal, dated 6 October 2005, was shared with GoI, and senior representatives of MoNE and MoRA took part in the peer review on 24 October 2005. The final Australia-Indonesia Basic Education Program Design Document, dated 16 May 2006, was provided to GoI on 17 May 2006.  This document is a summary of that final Design.  However, small but significant changes in Program targets, the financing plan, financial channelling modalities and the presentation of the logframe have been inserted to ensure consistency with the BEP Legal Agreements, to reflect implementation procedures set out in the draft Program Manual , and to make the design logic more apparent.

2.	RESPONDING TO KEY DEVELOPMENT CHALLENGES
2.1.	Conducive Policy Environment
Indonesia’s policy environment is conducive to effective and sustainable education reform. Education is an essential component of the Medium Term Development Plan (2005–2009) and macro-economic stabilisation and public expenditure reforms are designed to direct an increased share of resources to education.  Parliament has approved an increase in the MoNE development budget, including the introduction of functional, professional and special area incentives for teachers and direct operational budget support to all schools.  Primary and junior secondary school fees for the poorest students have been eliminated, offset by expansion of operational budgets for all school/madrasah providing grade 1 to 9 instruction. Public dissemination of junior and senior secondary school examination results has taken place. Wide-ranging public information campaigns on new entitlements and responsibilities for school directors, school committees and parents have helped to start the process of strengthening education governance and accountability.  

These measures offer opportunities for increased policy harmonisation across all education providers. BEP, for which both MoNE and MoRA will be implementing agencies, will contribute Australian resources that complement GoI’s. The Design identifies strategic and program priorities where Australian assistance has a comparative advantage which will help GoI optimise the impact and effectiveness of its own education spending.

2.2.	Increasing Access 
Access to education in Indonesia, as measured by net enrolment rates (NER), has stagnated over the past five years.  NERs are 92%-94% for primary education and 60%-63% for junior secondary education. The poorest socio-economic quintiles continue to be under-represented at the junior secondary level and beyond. At district level, junior secondary NERs vary by as much as 64%. District per capita education spending has also varied from Rp 1,200 up to 550,000 per annum. 

Annual primary school drop-outs are around 686,000, including 81,000 from madrasahs, which generally serve students from the poorest backgrounds. Every year some 500,000 primary graduates do not continue on to junior secondary. It is estimated that 1.75 million children aged 13-15 are not in junior secondary schools or madrasahs.  

GoI’s priority for overcoming these access barriers is the construction of new junior secondary schools/madrasah, especially in districts with low enrolment. One-roof schools (combined primary and junior secondary schools on the same site) were introduced in 2005 to help reduce primary-level drop-out and repetition and increase transition to junior secondary education. This arrangement reduces distance to school and travel costs. It is expected to help increase the number of girls attending quality junior secondary schools/madrasah, in areas where female enrolment is low.  Construction of new junior secondary schools/madrasah, as close to home as possible, is designed to reduce the opportunity costs of school attendance. For girls and the disabled, neighbourhood schools/madrasah, with appropriate facilities (e.g. separate water/sanitation) will reduce social and cultural barriers to quality education. 

2.3.	Increasing the Quality and Governance of Education
GoI has identified priorities for improvements in the quality of education.  The first is ensuring that all schools/madrasah have qualified teachers and instructional materials. Only around 50% of primary school teachers have the minimum professional qualifications. At junior secondary level the proportion of qualified teachers ranges from 34% to 60%, depending on the district.  Among junior secondary subject specialists in madrasahs, the levels are considerably lower.    

GoI also recognises that more efficient use of existing resources is essential, especially to increase the share of non-wage education spending.  Key areas it is addressing are: small school sizes; the lack of incentives for teacher postings to rural or disadvantaged areas; asset maintenance (an estimated 58% of existing primary school buildings are dilapidated); and parallel rather than harmonised non-formal education, which makes only limited use of school facilities. 

The Government is addressing these constraints. It recognises that professional development programs for teachers are an urgent priority.  Approaches to teacher recruitment, management and deployment to rural schools are being reviewed. School and teacher performance monitoring and quality assurance systems are also being revised. The independent Board of National Education Standards was established in 2005 and new minimum service standards have been issued.  New regulations for the use of school operational budgets focus on instructional materials.  A financing strategy for school rehabilitation, particularly primary schools, is a key feature of the RENSTRA 2005-2009.  

Implementing the national decentralisation policy within the education sector is another focus for reform.  MoNE and MoRA performance assessments note that central and district responsibilities are insufficiently aligned with planning and management functions.  School committees, granted more powers under decentralisation, have been slow to exercise greater authority. A survey of MoNE schools indicated that only around 20% of these committees met at least once a month. Organisational change is underway in both Ministries, but institutionalising decentralised management at all levels, particularly in MoRA which remains centrally run, will be a continuing challenge. 

3.	DESIGN RATIONALE 
3.1.	Options
The design mission screened three options for Australian support against the following key considerations: (i) consistency with reform priorities, (ii) predictable, immediate and verifiable policy impact, (iii) complementarity with (not supplementary to) Government funding and (iv) evident capacity for program implementation.  The options were:	

1)	Basic Education Quality Improvement Program: block grants to schools/madrasah for primary school rehabilitation, selective construction of additional facilities and supply of instructional materials. Community-based program management through existing Government systems.
2)	Basic Education Access Expansion/Equity Program: targeted assistance to new junior secondary school/madrasah facilities, focusing on the least served, low enrolment districts. Community-based program management through existing GoI systems.
3)	Basic Education Sector Development Program: an extension of option 2, underpinned by a rolling process of policy engagement and dialogue (through a jointly agreed policy action matrix). Designed to facilitate increased effectiveness of Government spending on education, to optimise access and improve quality and governance systems.
The design mission recommended option 3 because it assures high impact and enables a policy-led financing partnership between the two Governments.  This BEP design proposal was endorsed at senior officials level by both Government of Australia (late 2005) and GoI (early 2006).

3.2.	Reasons for the Choice
This option was chosen because it is the most effective way of helping GoI achieve its objective of providing universal access to 9 years of high quality basic education. (Of course, AusAID funds alone cannot achieve this objective; the responsibility and the bulk of the financial burden remains with GoI.) Under this option, the policy dialogue process will provide a much greater opportunity for influencing policy development for primary quality improvement (through systems interventions) and more directly support expansion of junior secondary opportunities as part of a broader sector approach. In addition, this option provides a mechanism for developing a rolling program approach which will help focus both GoI and other donor resources alongside AusAID support for achieving GoI education sector policy priorities and targets over the period 2005-2009 and beyond.

An assessment of the comparative effectiveness of community based and contractor managed approaches to infrastructure development was undertaken. The mission concluded that a community based approach should be adopted for the design, consistent with recent evaluations in Indonesia,.The main reasons in favour of community based approaches were: (i) overall construction costs are roughly 50% lower, meaning twice as many facilities can be provided, (ii) less fiduciary risk than a centralised procurement and management process, (iii) increased likelihood of strong community commitment to facilities construction and maintenance, (iv) greater likelihood of crowding in community contributions in terms of labour and additional construction funds and (v) more opportunities for community level employment and income generation. These findings were confirmed by consultation with MoNE specialists and World Bank staff and verified during field missions to recently completed community-based school construction sites. 

3.3.	Capacity Building Needs
Institutional assessments of MoNE and MoRA conducted as part of BEP preparation conclude that organisational structures and functions have responded only slowly to financial management and audit reform programs.  Capacity to implement these key systemic reforms, especially accounting and reporting responsibilities, needs to be strengthened.  A rolling program of technical and advisory support is needed, especially in MoRA. 

The institutional assessments also highlighted a need for better alignment between central organisational structures and obligatory functions. Central MoNE and MoRA authorities have issued a range of new regulations but capacity to monitor and support the implementation of these regulations is variable. In particular, central Ministries are only slowly adopting more results oriented planning and monitoring systems.   In particular, strengthening mechanisms and incentives that link policy development with district level service planning and management, possibly through matching conditional grant mechanisms, is a priority. 

The assessments indicate that MoNE can effectively implement the BEP infrastructure program.  With some additional safeguards, its financial management and oversight systems should be  sufficiently robust. For MoRA, which has limited experience of community based construction, immediate capacity building is critical. 

4.	BEP: A SECTOR SUPPORT PROGRAM
4.1.	Objective and Outcomes
The objective of the BEP is: in partnership with Government and other donors, to contribute to improved equitable access to higher quality and better governed basic education services, especially in targeted, disadvantaged areas.  The four pillars of the BEP and their anticipated outcomes are:  

i.	expanded equitable access – community-led construction of about 2,000 fully operational new and one roof junior secondary schools and madrasah (the precise number will depend on the geographical location and the type of school); the creation of at least 330,000 additional formal school places and a significant number of additional non-formal school places; and the net enrolment rate of junior secondary school students from the poorest 20% of households increases from 49.7% to 65%. 

ii.	improved quality and internal efficiency – sub-contracted site supervision of construction to professional standards; better systems for management of: school assets; teachers; quality and availability of instructional materials; school and student performance. (Whole School Development Program.)

iii.	capacity development for governance of education services – strengthened systems for financial and performance planning and monitoring. 

iv.	increased resource mobilization in the education sector – increased volume and share of MoNE/MoRA spending is directed towards basic education, particularly in poor and under-serviced districts.  
For a full description of Program components and anticipated outcomes see the logframe at Attachment A. 

4.2.	Financing Plan 
Under the umbrella of the Australia-Indonesia Partnership, BEP brings together A$355 million of funding from two Australian Government/AusAID sources: the Australia Indonesia Partnership for Reconstruction and Development (AIPRD) and the Australia Indonesia Development Cooperation Program (AIDCP).  From A$200 million of loans and A$100 of grants – governed by the Program Loan Agreement and the Project Loan and Grant Agreements - AIPRD will fund expanded equitable access (BEP school/madrasah building program) and site supervision and asset management systems. AIDCP will fund other aspects of improved quality and internal efficiency, and capacity development for governance of education services.   BEP funds are indicatively allocated to MoNE and MoRA on a 75%-25% basis respectively.  The sources and breakdown of BEP financing are set out in the following table.   

Basic Education Program (BEP) Proposed Financing Plan	$A Million 	
AIPRD FUNDING 		
Program Preparation - AIPRD grant funds	3	
Expanded Equitable Access:
MoNE School Building Program - AIPRD loan funds
MoRA Madrasah Building Program – AIPRD loan funds
MoRA Madrasah Building Program - AIPRD grant funds
MoRA Madrasah Building Program 2006-07 - AIPRD grant funds	195
5
52
8	
Improved Quality and Internal Efficiency (site supervision and asset management systems) - AIPRD grant funds 	25	
Advisory, activity management, M&E and audit  - AIPRD grant funds	12	
Sub Total – AIPRD 	300	
AIDCP FUNDING		
Improved Quality and Internal Efficiency (management of: teachers; quality and availability of instructional materials; school and student performance. Whole school development program.) - AIDCP grant funds	31	
Capacity Development for Governance of Education Services - AIDCP grant funds	9	
Advisory, activity management, M&E and audit - AIDCP grant funds	15	
Sub Total – AIDCP	55	
TOTAL PROGRAM PREPARATION & IMPLEMENTATON	355	

4.3.	GoI Contribution
GoI’s contribution to achieving BEP outcomes will include funding for: 

a)	operational costs for all new schools and madrasah constructed, including textbooks and educational material, learning media and sporting equipment;
b)	operational costs for each one-roof school constructed, commencing six months after the date that construction of the one-roof school is completed;
c)	rehabilitation of the primary school of each one-roof school constructed to ensure that such primary schools meet the Republic of Indonesia’s minimum building standards; 
d)	land acquisition, building permits and excavation works.
e)	recruitment and deployment of better-qualified teachers to the new schools/madrasah, 
f)	operational support costs for the system development aspects of the Program and 
g)	staff, office and other administrative costs necessary for the efficient implementation of the BEP.
In addition, GoI will endeavour to rehabilitate all public primary schools within the catchment area of the new public junior secondary schools created under the BEP.

4.4	Infrastructure Targeting and Selection Criteria
BEP infrastructure will target poor, remote and underserved districts with low overall enrolment rates and under-representation of girls at junior secondary level.  Sustained unmet demand for junior secondary facilities.  This will be verified by school mapping and site visits.  Selection criteria will include: (i) evidence of community commitment to manage facilities construction, (ii) evidence of community capacity to undertake agreed management and accounting responsibilities, (iii) undertaking by district or Islamic foundation, through a Memorandum of Agreement, to provide operational funds, including staff deployment, learning materials and operational budgets to the new school or madrasah. 

4.	 Policy Dialogue Process
A key objective of the program is to help lead and initiate high level policy and strategic dialogue between GoI, donor agencies and other key stakeholders in the education sector. The key mechanism for strengthening these partnership development processes will be the Policy Action Matrix (See Attachment B). Anticipated outcomes of this policy dialogue process will be: i) adoption of joint partnership principles between GoI and donors through a formal education sector working group of donors, ii) adoption of a rolling program approach which allows more effective sequencing and policy responsiveness to GoI and donor financing of the sector and iii) gradual refinement of the initial policy action matrix as a basis for greater harmonisation and coordination of support for policy and strategy implementation.
This policy dialogue process will help ensure that increased resource mobilisation for the sector is translated into improved sector outcomes. Key performance indicators for successful policy dialogue will include: sustained increases in central and district education budget allocations; and greater complementarity and reduced substitution of GoI funding by donor and other sources. 
5.	IMPLEMENTATION ARRANGEMENTS
5.1.	Management Entities
The Ministry of Education (MoNE) and the Ministry of Religious Affairs (MoRA) will be the joint executing agencies for the BEP and will be responsible for its implementation and outcomes.  The Secretary Generals of MoNE and MoRA will chair a Program Steering Committee (PSC) that includes a senior representative of the Ministry of Finance, BAPPENAS, and AusAID.  The PSC will provide the policy and strategic direction to the BEP and will approve Annual Work Programs and budgets.  

MoNE and MoRA Director Generals will chair a Program Coordination and Management Committee (PCMU)  which will be responsible for the formulation of the Annual Work Programs and disbursement schedules for pillars 1-3 and management of pillar 4, including the policy action matrix dialogue and annual programming adjustments.   Day-to-day implementation of the BEP will be undertaken by a Program Management Unit (PMU) in each of MoNE and MoRA, assisted by Technical Committees (MORA) or Sub-PMUs (MONE).  

These GoI entities will be supported by three AusAID-funded contractors: Managing Contractor Program Management (MCPM); Contractor Strategic Advisory Services (CSAS) and Independent Audit Contractor (IAC).  MCPM will facilitate implementation of the infrastructure program and activities under P2 and P3 including capacity development of MoNE and MoRA systems.  CSAS will advise on sectoral policy and the annual revision of initial compliance; annual financial acquittal systems and internal controls compliance.
 	
5.2.	Financial Management 
The infrastructure component will be funded through the AIPRD loan and grant agreements.  $252 will be dispersed through the GoI community construction model - as modified for the purposes of the BEP – and funds will be directly dispersed to school communities from the Indonesian Ministry of Finance (MoF).  MoF will report to AusAID quarterly on eligible expenditure and progress made against these loan and grant funds. 

The MCPM will manage a trust fund of $73 million which will include: A$ 8 million of AIPRD infrastructure grants for MoRA, to be used in 2006-07 while MoRA builds it capacity to efficiently and transparently manage infrastructure funding.  MCPM will sub-contract infrastructure site supervision and asset management systems strengthening through another $25 million of AIPRD grants.  The remaining $40 million will be used for capacity building in education quality and governance through Pillars 2 and 3 of the BEP.  MCPM will manage these funds in a manner consistent with Commonwealth Procurement Guidelines and will use a range of appropriate disbursement modalities.  Full details of BEP financial management and reporting arrangements are set out in the draft Program Manual.

5.3.	Monitoring and Evaluation
BEP monitoring and evaluation will be coordinated by the PCMU and principally use existing GoI agencies and capacities.  Key implementers will be the Planning Bureaux and Inspectorates General of MoNE and MoRA and considerable resources will be available, through MCPM and CSAS, to strengthen monitoring and evaluation capacity in both Ministries, right down to district and school or madrasah levels.  Because of its nature, the BEP will involve a range of different types of monitoring: longer-term benefit monitoring, sector and program level outcome and output monitoring, and annual performance assessment to inform the rolling program approach.  A priority on mobilisation will be for MCPM and CSAS to work with MoNE and MoRA to develop a full monitoring and evaluation regime for BEP encompassing base line studies, poverty and gender impact assessments and operational research.
Because of the volume of BEP funds, for the infrastructure program particularly, extensive risk identification and mitigation measures, including an institutional integrity plan and anti-corruption measures, are being developed in concert with GoI and will be set out in the Program Manual. 
  
6.	NEXT STEPS
6.1.	  Immediate
A key step in making the Project Loan and Grant Agreements effective (and making disbursements possible) is agreement by both Governments on this Design Summary, the BEP Program Manual, and the summary of the Indicative Annual Work Program 2006-2007.  A summary of the Indicative Annual Work Program and a revised draft Program Manual will be provided to GoI by 10 July with a view to achieving loan effectiveness by 14 July. 

6.2.	July-September 2006
In parallel to achieving loan effectiveness, the MCPM will work in conjunction with MONE and MORA to finalise the Annual Work Program and the Policy Action Matrix for 2006 and 2007.