Independent Evaluation of 
Australia Indonesia Basic Education Program (AIBEP)
MANAGEMENT RESPONSE

Prepared by: 	Zullia Saida, AIBEP Evaluation Manager, Education Section, Jakarta Post
Through     :	Danielle Heinecke, Director, Education Section, Jakarta Post
Approved by: 	Jacqui de Lacy, Minister Counsellor	Date Approved: 12 October 2010

Aid Activity Objective

The overall objective of AIBEP is to improve equitable access to higher quality, better governed and better resourced basic education services, especially in targeted disadvantaged areas, in partnership with Government of Indonesia (GoI) and other development partners. 
Aid Activity Summary

Aid Activity Name		
AidWorks initiative number	•	ING632 (loan) and ING133 (AIPRD grants) for the Infrastructure Component (Pillar 1) 
•	ING529 (DCP grants) for Technical Assistance Component (Pillar 2, 3 & 4)	
Commencement date	April 2006	Completion date	June 2010	
Total Australian $	AUD 387.6 million, contributing to 20% of Indonesia’s national budget allocation for the education sector (roughly Rp 209.5 trillion or equals to USD 23 billion in FY2009)	
Total other $	AUD 16 million (in 2009/10, per MCPM-AIBEP Activity Completion Report.  AIBEP did not collect data on counterpart funding during the first 30 months of the program, but MCPM reported that GoI contributed space, staff and funded events such as WSD/WDD trainings and Renstra workshops). 	
Delivery organisation(s)	MCPM –Cardno Acil
CSAS – GRM International	
Implementing Partner(s)	Government of Indonesia: Bappenas (the National Development Planning Agency); MoNE (Ministry of National Education); MoRA (Ministry of Religious Affairs)	
Country/Region	Indonesia, Asia	
Primary Sector	(Basic) Education	

Overview of the Aid Activity
The Australia Indonesia Basic Education Program (AIBEP) commenced in April 2006 and completed on 30 June 2010.  It is now been extended to 30 December 2010 (and additional three months for audit activity).  Its objective is to support the Government of Indonesia in improving equitable access to higher quality and better governed basic education services in the targeted, disadvantaged areas. It has four key areas of focus: expanded equitable access; improved quality of education services; capacity development for governance of education services; and increased resource mobilization in the education sector, including policy advice, research and sector monitoring. 
The program comprised of both loans and grants totalling AUD387.6million (A$200 million loan from the Australia Indonesia Partnership for Reconstruction and Development-AIPRD, and A$187.6 million in grants from the Australia Indonesia Development Cooperation Program and AIPRD).  Loans and grants were provided for school construction through government systems; grants for school and district level capacity development for delivery of education services, policy advice and institutional and organizational development.  The program was delivered using government systems through the Ministry of National Education (MoNE) and the Ministry of Religious Affairs (MoRA). Program management, monitoring, advice and capacity building was provided by two contractors, one focusing only on research and policy advice.  An independent audit contractor provided audit services.
Independent Evaluation Summary
Evaluation Objective: 
The objectives of the evaluation as per the terms of reference (TOR) were to: (i) evaluate the extent to which AIBEP achieved its objectives; (ii) assess Australia’s impact on education sector development; and (iii) provide lessons learned that will inform and shape the early implementation of the Education Sector Support Program (ESSP).  

Overview of the Evaluation:
The evaluation assesses the program’s overall performance against AusAID’s/DAC eight evaluation criteria.  As requested by AusAID, emphasis had been on the criteria relating to effectiveness, efficiency, impact and sustainability.  AusAID’s four key evaluation questions (as specified on page 5 of Attachment A: Terms of Reference for the Independent Completion Report of the Australia Indonesia Basic Education Program) cut across the four pillars and have been addressed within each evaluation criteria and pillar assessment. 
The findings presented are largely based on qualitative assessments including interviews with key stakeholders and informants and analyses of secondary sources of data and information for assessment and verification.  There had been limited opportunities for primary data collection and fieldwork, precluding development and testing of rigorous questions and surveys. Lack of comparative data has limited opportunities for cost comparisons, cost effectiveness and efficiency analysis. 
The in-country visit was conducted between 10 March-6 April 2010 and the draft ICR was submitted on the 6 April 2010.  The ICR Team includes Graham Walter (M&E Specialist, Team Leader); Fabia Shah (Education Specialist); and Russ Streader (Infrastructure Specialist).
Management Response
Overall, the report is comprehensive and provides practical recommendations and lessons to inform and guide future programming. Business Unit generally agrees with the key recommendations, and partially agrees with the ICR Team’s recommendation that AusAID’s agenda should not overly influence a program design, which inhibits integration into a coherent program (i.e. recommendation Five).  The ICR Team recommendations and Business Unit responses are laid out in the next section.
Recommendation One
Recommendation: 
Working through GOI systems is highly recommended. In any further program, AusAID needs to balance the needs for its own safeguards with the resultant costs, when determining what components can be fully implemented through GOI systems.
Response: Business Unit agrees with the recommendation.  

Actions: Business Unit have documented the experience and lessons from AIBEP in working through the GOI system.  Business Unit will work closely with AusAID/Jakarta Post’s Public Financial Management (PFM) Unit to analyse further Indonesia’s fiduciary risk assessment, its education sector financing and fund channelling arrangements, prior to the implementation of the next phase of Australia’s future support for Indonesia’s education sector. 
Recommendation Two
The community based construction model is regarded as both effective and appropriate and is strongly supported. However, more diligence is needed in site selection, with justification being provided in cases where school site selection does not meet the specified criteria.
Response: Business Unit agrees with the recommendation.
Actions: Business Unit has already initiated discussions with MoNE and placed a staff member in MoNE to facilitate the on-going dialogue on site selection criteria.
Recommendation Three
The program approach with its flexibility is recommended, but the overall goal, intermediate and immediate outcomes, and priority areas of focus for the Program should be clearly specified at design to avoid strategic drift. Flexibility should largely relate to adjustment of approaches in delivering a component, timing of any component, and possible dropping or addition of a component if the education environment changes. 
Response: Business Unit agrees with the recommendation. 
Actions: Business Unit will conduct ongoing regular policy dialogue between GOI and GOA regarding sub-sector priorities, to maintain focus on specific education sector’s outcomes and targets throughout the implementation of the next phase of Australia’s future program support for Indonesia’s education sector.  This dialogue will allow for the flexible adjustment of program approaches when and if the education environment shifts.
 Recommendation Four 
A holistic approach is needed for a program such as AIBEP, with interdependent components developed as one system. Training of teachers, principles, and District supervisors and coordinators should be one cohesive system. Capacity building at District level (WSD and WDD, including training in financial management, planning and monitoring, EMIS and FMIS) should be part of a holistic system running from central levels, through Provinces and Districts to schools. This requires consideration of inclusion of all appropriate components, including buildings, equipment, school teaching and learning materials and training, if not being provided by others.
Response: Business Unit agrees with the recommendation.
Actions: Business Unit has derived from the GoI medium-term development plan and its five-year education strategic plan to form the basis for the next phase of Australia’s future program support for Indonesia’s education sector.  With GOI explicit direction and endorsement, Australia’s contribution will support selected GOI education sub-sector programs, ensuring a holistic approach cross sectorally, vertically integrated and nationwide in coverage (at the central and sub-national governments of the provincial and district levels).  Capacity building and analytical support will form the basis of this approach.   
Recommendation Five
AusAID’s own concerns and agenda should not overly influence a program design, particularly when there is apparently no buy-in by government on specific concerns, which then inhibits integration into a coherent program.
Response: Business Unit partially agrees with the recommendation.  
AusAID is strongly committed to the key principles of Paris Declaration, where Indonesia should lead its own development policies, strategies and coordination.  As an important development partner, AusAID commits to align support towards Indonesia national development strategies.  
At the same time, Australia’s bilateral program must be consistent with its national development principles and aid policies relating to gender, inclusion, environmental conservation, anti-corruption and cost-effective development aid management and other key basic values that may involve advocacy and conditionality related to its development assistance.  
Many of these issues raised were actively addressed within AIBEP, for example, incorporation of inclusive education and gender mainstreaming principles; improvements towards infrastructure design that incorporates universal accessibility and environmental concerns; improving mutual accountability and anti-corruption measures and improved cost effective aid management through the strengthening of existing GoI systems.  A case in point is the efforts in building an appreciation and understanding of universal gender mainstreaming concepts and strategy, which was strongly intertwined in the cultural and religious aspects of the Indonesian society.  AIBEP has been successful in moving this issue forward without forcing negative reactions, which culminated in the 2008 issuance of the Ministerial Regulation on Mainstreaming Gender in Education.
The ICR has indicated that working through AIBEP, the GoI has embraced AusAID policy (use of recipient government systems, infrastructure guidelines, anti-corruption initiatives, gender policy and others) within an appropriate operational framework and a genuine culture of partnership with the GOI counterparts. These values and key issues are negotiated with GOI based on an understanding of where there are opportunities for reform: it is not universally supported without consideration of interest within GOI.  
Actions: 
AusAID is aware that success in policy dialogue is based on an appetite for changes within partners. Decisions about which issues are pursued are always based on a close assessment of the reform environment.
Recommendation Six
For a follow on education program, quality improvement programs are more likely to be sustainable if activities and financing are focused on the school and district level in a decentralised context such as Indonesia’s. 
Response: Business Unit agrees with the recommendation.
Actions: Business Unit will focus on supporting a decentralised education governance under the next phase of Australia’s future program support for Indonesia’s education sector.  The implementation of this support will involve supporting the provinces and districts as the beneficiaries of staff capacity development program as well as engaging them as the management units responsible for the delivery of education program activities.   
Recommendation Seven
Focus on school and district capacity building through programs such as WSD and WDD should be continued. Continued capacity building support for other activities under Pillar 2, including LPMP capacity building, is needed.
Response: Business Unit agrees with the recommendation.
Actions: AusAID’s future program support for Indonesia’s education sector will provide for establishing and make operational a national system for strengthening the leadership and management skills and capacity of school managers and districts’ education officials.  The program will also provide for good quality analysis and research that will support GOI education policy-making and organisational capacity development.
Recommendation Eight
Capacity expansion should focus on meeting the district and national needs, linking provision of junior secondary schools needs of satellite primary schools and higher education, not on number of schools to be built.
Response: Business Unit agrees with the recommendation. 
Actions: Under Australia’s future program support for Indonesia’s education sector, AusAID will conduct policy dialogues with GOI to understand and map the infrastructure needs of the sub-national governments.  AusAID will also examine whether rebuilding of schools nationwide, and extending facilities by adding laboratories and libraries, would have a greater impact than expanding the number of schools.
Recommendation Nine
The Annual School Survey should be continued, with appropriate levels of capacity building support.
Response: Business Unit agrees with the recommendation.
Actions: The annual sample survey of AusAID-financed schools under AIBEP will be continued in the next phase of Australia’s future support to Indonesia’s education sector, building on and providing continuity with surveys undertaken in 2008 and 2009 under AIBEP.  This survey will include an impact assessment of the school-based management trainings under AIBEP.
Recommendation Ten
Further evaluation is needed of the availability of teaching and learning materials.
Response: Business Unit agrees with the recommendation.
Actions: The annual sample survey of AusAID-financed schools under AIBEP will be continued to include a limited assessment of the provision of teaching and learning materials to new schools.
Recommendation Eleven
Further studies are needed of financing options for schools, including addressing the issue of maintenance.
Response: Business Unit agrees with the recommendation.
Actions: AusAID has agreed with MoNE’s proposal that studies on costs and financing be conducted in future, building upon other similar studies focusing on issues of financing of the service delivery, conducted by other development partners and think tanks, such as the USAID, World Bank and SMERU. 
Recommendation Twelve
An assessment should be undertaken in a sample number of schools in different geographical locations to assess ways in which gender policies have been incorporated within school management plans and their implementation.
Response: Business Unit agrees with the recommendation.  
Actions:  The annual sample survey of AusAID-financed schools under AIBEP will be continued to include a limited assessment on the integration of GOI’s gender policies into schools and districts development plan.  In addition, a separate review on MoNE’s progress in implementing the Ministerial Regulation No. 84/2008 on Mainstreaming Gender in Education at school and district levels will be conducted under the next phase of Australia’s future support to Indonesia’s education sector 
Recommendation Thirteen
Analytical support provided through Pillar 4 is vital and much valued by GoI, and needs to be continued under any new program.
Response: Business Unit agrees with the recommendation.
Actions: GOI and AusAID have agreed to continue AusAID’s provision of analytical support to GOI, with additional capacity to respond to ad-hoc requests for technical assistance that focuses on capacity building of GOI staff, building on AIBEP lessons.
Recommendation Fourteen
Given that AIBEP is only nearing completion and outcomes are yet to be realised, a post-evaluation could be undertaken in 3 years time to assess the continued impact of AIBEP’s access, quality and governance support.
Response: Business Unit agrees with the recommendation.
Actions:  Business Unit will include a post evaluation of AIBEP, as part of the evaluation plan of the next phase of Australia’s future support to Indonesia’s education sector.