Review of Australian Assistance to the 2009 Elections in Indonesia

Final Report

 October 2009

Submitted to:

The Australian Agency for International Development (AusAID)

Prepared by:

Eric Bjornlund 
Democracy International, Inc.
Bethesda, Maryland USA

Rustam Ibrahim
Jakarta, Indonesia 

Michael Collins
AusAID
Canberra, Australia

Completion Review Support to the 2009 Elections

ABN 62 921 558 838

INDEPENDENT COMPLETION REPORT

 
Aid Activity Summary
Aid Activity Name	Completion Review Support to the 2009 Elections	
AidWorks initiative number		
Commencement date		Completion date		
Total Australian $	$6,200,000 	
Total other $		
Delivery organisation(s)		
Implementing Partner(s)	UNDP, IFES, AEC, The Asia Foundation, Partnership for Governance Reform 	
Country/Region	Indonesia	
Primary Sector	Democracy and Governance	

Acknowledgments
The review team is grateful for assistance provided by several individuals and organisations throughout the evaluation process. The willingness of representatives of the KPU and the Government of Indonesia, including from the Ministry of Home Affairs and Bappenas, to accommodate the schedule of the review team was essential. Representatives of the governments of Canada, the Netherlands and the U.S. provided invaluable input from other donor countries. Representatives of Cetro and JPPR provided the rich perspective of Indonesian civil society. Both the flexibility and availability of implementing partners, including the Australian Electoral Commission, IFES, TAF, Partnership for Governance Reform, Puskapol UI, and UNDP, enabled the team to conduct a comprehensive review of AusAID’s assistance within a challenging time frame. 
The support of AusAID’s in-country mission was essential to the success of the review team. The team would particularly like to thank Rosyidah Handayani, whose diligence and attention to detail prevented any logistical challenges from clouding the review team’s focus. We appreciate the confidence and cooperation we have received from Ade Ganie and Emily Rainey.  The team also appreciates the assistance of Jed Ober, Democracy International Program Assistant, who participated in some interviews and team meetings in Jakarta and assisted with the research, including review of relevant documents. 

Contents
Aid Activity Summary
Acknowledgments
Contents
List of Abbreviations and Acronyms
Executive Summary
Introduction
Evaluation Findings
Evaluation Criteria Ratings
Lessons Learned and Recommendations
APPENDIX A: BIOGRAPHICAL INFORMATION ON TEAM MEMBERS
appendix b: list of people interviewed
appendix c: evaluation methodology
APPENDIX D: TERMS OF REFERENCE

List of Abbreviations and Acronyms

AEC			Austalian Electoral Commission
AIP			Australia Indonesia Partnership
AusAID		Australian Agency for International Development
Bawaslu		Election Supervisory Body
CETRO		Center for Electoral Reform
CSO			Civil Society Organization
DPS			Temporary Voter Lists
GoI			Government of Indonesia
ICR			Independent Completion Review
IFES			International Foundation for Election System
JPPR			People’s Voter Education Network
KPU			National Indonesia Election Commission
MDP			Multi-Donor Program
LP3ES		Institute for Social and Economic Research, Education and 
			Information
MOHA		Ministry of Home Affairs
KPUD			Regional Election Commission
NDI			National Democratic Institute for International Affairs
NGO			Non-Governmental Organization
NU			Nahdhatul Ulama
Partnership		Partnership for Governance Reform
Pilkada		Local Election
Puskapol 		Center for Political Studies
TAF			The Asia Foundation
ToR			Terms of Reference
UI			University of Indonesia
UNDP			United Nations Development Program
YAPPIKA		Civil Society Alliance for Democracy
Executive Summary 
An Independent Completion Review team conducted a qualitative review and assessment of the appropriateness, efficiency and effectiveness of Australia’s support for the 2009 election process in Indonesia. AusAID channeled this support through the UNDP Multi-Donor Electoral Program, the Australian Electoral Commission (AEC), the International Foundation for Electoral Systems (IFES), The Asia Foundation (TAF), and the Partnership for Governance Reform (Partnership).
Evaluation Findings
The team assessed the program’s performance against evaluation criteria established by the Terms of Reference (ToR): relevance, efficiency, effectiveness, impact, sustainability, monitoring and evaluation, gender equality, and analysis and learning. 
Relevance. In general, Australian assistance to the elections was relevant and appropriate. The assistance responded to the expressed priorities of the Australian Government, the Indonesian General Elections Commission (Komisi Pemilihan Umum, KPU), and Government of Indonesia (GoI). Working within the context of these priorities, the AusAID program responded directly to specific election management and voter education needs. Nevertheless, according to many election stakeholders and analysts, consistent with much discussion in the media and elsewhere, the election and the election management were of significantly lower quality than previous elections in Indonesia. Moreover, largely because of the choices of the GoI, neither the AusAID program nor the overall foreign assistance effort addressed several of the most fundamental challenges to the elections, including problems with the voter registration lists, vote count compilation and the process of constituting the current KPU.
Effectiveness. In general, AusAID programs were effective in contributing to strengthening the aspects of election administration and management that had been mutually agreed were areas where assistance could be provided. Programs aimed at supporting voter education may have added value locally to the civic discourse on public participation in the election process. While some projects achieved their specific objectives, others did not succeed in delivering key portions of their programming within a timeframe to be effective or within the constraints of an agreed-upon budget. The inability to effectively address such key issues as voter registration and results compilation limited the ultimate effectiveness of the assistance of AusAID  and of the greater donor community.  
Efficiency. Judging from the comments of AusAID partners, the level of funding for the project seemed largely appropriate. At the same time, there was considerable uncertainty about future AusAID’s electoral assistance. Officials involved in several AusAID-supported activities, including Partnership, Elections-MDP and the AEC, mentioned the expectation of future assistance under the pending longer-term AusAID election support program. 
Impact.  Rigorous, scientific impact evaluation of AusAID’s election program in Indonesia or of particular activities would require the design and implementation of an evaluation methodology from the beginning of the program. Nevertheless, we can infer from discussions with a range of stakeholders that AusAID’s support to the KPU contributed to building the capacity of the commission’s staff at both the national and provincial levels and that the AusAID program contributed to increased involvement of women in politics and to increased voter registration. But the inability of the overall election assistance to address some critical shortcomings of the process limited its broader impact on the overall quality of the election process.
Sustainability. By responding to GoI and KPU expressed priorities, AusAID designed its program in a way that, because of government buy-in, should be sustainable. At the project and activity level, there was also considerable focus on sustainability. By focusing on training, capacity building and organisational reforms, the project was designed to have lasting effects. But, as one Indonesian government official argued, Indonesia and its international partners still need to pay additional attention to the problem of how to institutionalise the election process and avoid addressing the same types of election-related issues every five years. 
Gender Equality. Generally, AusAID’s implementing partners took gender equality into account in their programming. In particular, voter education efforts addressed the concerns and needs of women and marginalised groups.
Monitoring and Evaluation. Some efforts were made to monitor and evaluate project activities, but in general it seems there was relatively little consideration of monitoring and evaluation in the design of programs. Monitoring and evaluation efforts often appear to have measured program outputs rather than outcomes. 
Analysis and Learning. A number of interviewees suggested that AusAID itself made a genuine attempt to understand the nuances and complexities of the challenges to democratic elections in Indonesia and often gathered information and analysis from partners. Likewise, program implementers generally demonstrated serious attention to analysis and learning. The program in support of the 2009 elections, however, could have more effectively drawn on the experience of election support in 2004. 
Conclusion/Lessons Learned and Recommendations
The review team offers the following recommendations:
•	Timing. Support to Indonesia’s electoral systems should be provided throughout the five-year electoral cycle, not just in the lead-up to election day.
•	Civil Society Engagement. AusAID should consider increasing engagement with Indonesian CSOs involved in election-related technical assistance, voter education and advocacy.
•	Donor Coordination, Paris Declaration on Aid Effectiveness and Jakarta Commitment. 	AusAID should engage the Government of Indonesia in dialogue about the role of a robust and active civil society in a democracy; it should support the role of civil society organizations in the democratic process generally and as stakeholders in decisions about the proper role and use of foreign assistance.
AusAID should ensure programmatic accountability (as opposed to and in addition to financial accountability) for any funds provided through a multidonor mechanism.
•	Engaging Core Areas of Concern. While taking account of political realities, AusAID should consider supporting efforts to address critical areas of concern in the election process, such as voter registration, vote tabulation and the electoral framework. 
AusAID should engage the GoI in dialogue about priorities for effective foreign assistance.
•	Expanding Visibility. AusAID should seek to improve the visibility of its efforts to support democratic elections in Indonesia.
•	Monitoring and Evaluation. AusAID should require more robust monitoring and evaluation systems that can inform program performance reporting. 
AusAID consider means to improve impact evaluation. 

Evaluation Criteria	Rating (1-6)	
Relevance	5	
Effectiveness	4	
Efficiency	4	
Sustainability	4	
Gender Equality	5	
Monitoring & Evaluation	3	
Analysis & Learning	5	
Rating scale: 6 = very high quality; 1 = very low quality. Below 4 is less than satisfactory.

Introduction
ICR Background
Australia provided $6.2 million to support Indonesia’s parliamentary and presidential elections in 2009. This support was delivered through five channels: (1) the UNDP Multi-Donor Elections Program (Elections-MDP) ($3.6 million), (2) the Australian Electoral Commission (AEC) ($780,000), (3) the International Foundation for Electoral Systems (IFES), (4) The Asia Foundation (TAF), and (5) the Partnership for Governance Reform (Partnership). The amount of support for IFES, TAF and Partnership together was $1.6 million.
Evaluation Objectives and Questions
The Independent Completion Report (ICR) team selected by AusAID conducted a qualitative review and assessment of Australia’s support for the 2009 election process in Indonesia. The review is intended to inform future assistance.
The review and this report follow the AusAID framework for an Independent Completion Report. That framework calls for the team to review the program in accordance with established evaluation criteria: relevance, efficiency, effectiveness, impact (if feasible), sustainability, monitoring and evaluation (M&E), gender equality, and analysis and learning. To further refine this process, the review team’s terms of reference (ToR) call for priority attention to the following:

•	Program objectives
•	Management and implementation of individual activities
•	Appropriateness (relevance) and links between activities and impact 
•	Level of funding and any significant gaps 
•	Consistency of activities with the objectives outlined in the project proposals and set out in the AIP Country Strategy 2008-2013
•	Monitoring arrangements of the activity and broader outcomes (monitoring and evaluation)
•	Lessons for:
a)	a longer term program of support over the entire electoral cycle; 
b)	in the lead up to a similar large scale election event in 2014; and
c)	to broader democratic governance programs

These questions are addressed in the context of the evaluation criteria called for in the general framework for an AusAID Independent Completion Review.
Evaluation Scope and Methods
To conduct its review, before its arrival in Jakarta, the team first developed a review plan and methodology in collaboration with AusAID. The team received from AusAID and reviewed relevant documents, including AusAID strategy and planning documents, project proposals and reports from implementing partners, interim evaluations, newsletters, and public activity reports. From 13 to 18 September 2009, the team conducted semi-structured and open-ended interviews in Jakarta and, via telephone, with an AEC representative in Australia. The team met with representatives of AusAID and its implementing partners, including UNDP, AEC, IFES, TAF and the Partnership for Governance Reform. The team also met with commissioners and the secretary general of the KPU, representatives of the Government of Indonesia, civil society groups, election analysts, and aid officials from Canada, the Netherlands, and the U.S. 
The team has discussed the evaluation criteria for each activity and the overall program as well as its key findings as summarised in this completion report. The team has followed a methodology and list of questions drawn from specific and general evaluation questions in its ToR, supplemented and modified by discussions with representatives of AusAID.  
Independent Completion ReviewTeam
The ICR team comprises Eric Bjornlund, President of U.S.-based Democracy International and an expert on elections and democracy in Indonesia; Rustam Ibrahim, former executive director of the highly regarded Indonesian research organisation LP3ES (Lembaga Penelitian, Pendidikan dan Penerangan Ekonomi dan Sosial, Institute for Economic and Social Research, Education and Information) and board member of KID (Komunitas Indonesia untuk Demokrasi, the Indonesian Community for Democracy); and Michael Collins, Policy Officer, Indonesia Policy and Analysis Section, at AusAID.
The team has drawn on its knowledge of the history and current status of electoral institutions and processes and of democratic development in Indonesia. Mr. Bjornlund has brought direct knowledge of and experience with Indonesian elections and election assistance over the last 15 years, including deep engagement in national elections in 1999, 2004 and 2009 as well as local elections (Pilkada) beginning in 2005, and is familiar with the election assistance of various donors in Indonesia and many other countries. Mr. Ibrahim, during a long career as a social scientist, analyst and civic activist, has studied and played an important role in election-related, civil society and other development programs in Indonesia. Mr. Collins has deep expertise in Indonesia as well as critically important knowledge of AusAID processes, review requirements and similar matters. 
Evaluation Findings
As discussed in the following sections, the team has assessed the program’s performance against evaluation criteria established by the ToR: relevance, efficiency, effectiveness, impact, sustainability, monitoring and evaluation, gender equality, and analysis and learning. We address each of these criteria in turn and, for each, address both AusAID’s comprehensive election assistance program and, in most cases, the specific activities of implementing partners.
Relevance
In general, Australian assistance to the elections was relevant and appropriate. The assistance responded to the expressed priorities of the Australian Government, the KPU, and the GoI. The KPU and the GoI identified their priorities through the Electoral Program Coordination Steering Committee process involving Bappenas (the National Planning Board), the KPU, the Elections Oversight Agency (Bawaslu), Ministry of Home Affairs (MoHA), Ministry of Finance, Ministry of Foreign Affairs, UNDP (Elections-MDP) and donors. Working within the context of these priorities, the AusAID program responded directly to specific election management and voter education needs. 
Despite these efforts and those of other donors, however, according to many election stakeholders and analysts, consistent with much discussion in the media and elsewhere, the election and the election management were of significantly lower quality than previous elections in Indonesia, and neither the AusAID program nor the overall foreign assistance effort addressed the most significant problems with the process. In this broader sense, the program was much less relevant than would have been desirable. Among other things, the GoI through the Steering Committee either did not invite or did not permit AusAID or other donors to work in some of the most politically sensitive areas, such as issues surrounding the voter list and results tabulation. In addition, Australian assistance could have placed greater emphasis on efforts to “build civil society capacity in analysing and monitoring the quality of election process” and on collecting information about the outcomes of CSO activities, which were called for in the AIP Country Strategy 2008-13. These issues are discussed further below.
a.	Support for Election Administration and Management 
Indonesia has implemented open, competitive elections since 1999. In 2009, however, the country still faced ongoing issues in electoral administration and management. AusAID addressed weaknesses in election administration through its support for the UNDP’s Elections-MDP, the AEC and Partnership.   
Elections-MDP. Through the UNDP Elections-MDP, Australian funding supported the development of electoral procedures and administration, especially regarding election procurement, along with support for public relations and operational training for election officials on voting and counting. AusAID was the largest Elections-MDP donor. 
The focus on procurement responded to a distinct weakness exposed by the 2004 election process. The program obtained the services of the two short-term procurement specialists who were assigned to the KPU Secretariat. The attention to training to ensure the effective implementation of procedures introduced by the new Regulation 35/2008 on voting and counting also was very relevant; it responded to shortcomings at the polling station level in the 2004 elections. 
The focus on public relations, although based on a flawed analysis of the reasons for the poor public image of the KPU, as discussed below under Analysis and Learning, was also relevant to a real need. The Elections-MDP supported the capabilities of the KPU media centre, primarily through an external company that provided technical support. It also supported a communications bureau and helped the KPU to develop a media strategy, including regular press conferences, press releases, newsletters, posters, leaflets and public gatherings. According to the UNDP, this helped to improve transparency and public understanding. 
In its quarterly report in June 2009, though, the UNDP admitted that the design of the Elections-MDP project flowed from a “wrongly made assumption” about the capabilities of the election management body: “Owing to the celebration of the 2004 successful elections, in the project document it was assumed that Indonesia had reached sufficient institutional maturity to manage the electoral planning and logistics.” That this assumption was incorrect should have been foreseeable after the experience with local elections, the late approval of the legal framework, and the flawed selection process for the KPU. Because of this incorrect assumption, the UNDP program was less relevant to the real election administration needs than it might have been.
AEC.  The activities of the AEC also contributed to supporting the goal of a well-administered election. The AEC’s assistance to the development of manuals and training modules helped to address the stated needs for training of polling station and other election officials. In fact, the chair of the KPU requested that AEC assistance be continued past its initial completion date based on its relevance to the needs of the organization. The KPU secretary general added specifically that the KPU particularly valued its relationship with the AEC, as a peer organisation.
Partnership. Partnership responded to key needs of the KPU in restructuring the KPU secretariat’s internal organisation and managing its human resources. Partnership’s assistance to election management focused on helping the KPU to implement the mandate of Law 22/2007 to restructure the Secretariat General’s office from ten to seven bureaus. By facilitating the development of job descriptions, Partnership aimed to build a strong foundation for the re-organisation of the KPU. The program also began to develop a process and criteria for KPU staff evaluations, although such evaluations were not actually carried out. 
KPU representatives told the team that these activities supported by Elections-MDP, AEC and Partnership were appropriate to their needs.  
b.	Support for Voter Education
Several interviewees suggested that voter education was important and much needed. 
IFES. IFES support for voter education programs (national socialisation and voter information) in selected provinces, including AusAID priority regions, responded to AusAID’s interest in supporting voter education. These programs directly addressed the expressed priority of AusAID in the AIP strategy to “facilitate better public understanding of, and participation in, democratic processes.” As part of the National Socialisation and Public Information program, IFES produced and broadcast a number of public service announcements to increase public awareness about the election process with the goal of greater public participation in the elections. This included information for voters about changes in election procedures and messages about the value of participation. One public service announcement provided voter education and supported the election administration simultaneously by educating voters on the integrity of the KPU. 
By working with local CSOs to deliver programming, for example with Bali Sruti on the Youth Socialisation Roadshow in Denpasar, IFES’s programming contributed to AusAID’s objective of enhancing the engagement of civil society in the election process. 
TAF/JPPR. Through TAF, AusAID supported JPPR (Jaringan Pendidikan Politik untuk Rakyat, the People’s Voter Education Network) to create public service announcements and talk shows and to air radio advertising to inform voters. TAF said the objective of JPPR was to “encourage citizen participation” but did not make the case for why this was a particular priority or how the activities could make a significant impact in this regard. 
TAF and JPPR have conducted election projects since 1999, and TAF representatives said they and JPPR want to go beyond election-related activities. “What has the country gotten from elections?” asked one person rhetorically. Even with all the experience with elections and the focus on process and procedures, they argued, the country still does not get those procedures right. It is important, they said, to move beyond procedures because people “want to believe in the process.” Rather they want to integrate elections with real problems and to monitor the performance of elected officials and conduct other postelection activities. 
This focus on local participation and on moving from procedural to substantive concerns led them to the idea of citizen forums, where citizens or members of the community could meet and discuss election procedures and issues. However valuable they may be for the participants, however, such forums have very local impact and are thus of arguable broader relevance.
None of the partners clearly identified specific problems that voter education was intended to address or provided an analytical basis for the targeted audiences, geographical focus or particular messages of voter education activities. Some did not distinguish between efforts designed to inform voters about the process, such as the procedures for voter registration and balloting, and broader voter and civic education messages, such as encouraging voters to consider candidates’ policy positions and integrity in making their choices. IFES’s documents suggest voter education messages were developed based on problems identified by public opinion research, but others did not suggest they were responding to specific issues identified through opinion research or by other means. TAF, JPPR and IFES suggested their efforts included messages to encourage citizen participation, but there was no real explanation of why this should be a donor priority. Several interviewees suggested that education efforts should address broader democracy messages, but there was little suggestion that anyone has thought deeply about whether and how voter education can deepen support for democracy or have broader impact.
c.	Visibility and Branding
AusAID representatives specifically asked the team to consider the extent to which recipients of assistance were aware of where they were receiving assistance from and whether they recognised the nature and extent of Australian contribution. 
AusAID’s implementation partners in the assistance to election administration—Elections-MDP, the AEC and Partnership—all appeared to fully understand the extent and limits of AusAID assistance, even though there may have been some overlap between AusAID-funded activities and those funded by other donors. As the beneficiary of much of the assistance from Elections-MDP, AEC and Partnership, the KPU was fully aware that it was receiving substantial support from AusAID. Moreover, the secretary general specifically understood that Partnership assistance on restructuring the secretariat was supported by AusAID and that assistance to the development and distribution of manuals came from AEC and from AusAID, through the UNDP (Elections-MDP). Unlike the secretary general, some of the members of the KPU may not necessarily know exactly what AusAID has supported, but they appeared to be aware that there has been substantial AusAID support.
With respect to voter education activities, it is much less clear how much JPPR and its partners are aware of the extent or nature of AusAID’s support. While the JPPR secretariat leadership is aware that TAF support to it comes from AusAID, there is reason to question whether organisations in the JPPR network have such awareness. For one thing, there are a number of layers of administration between AusAID and the recipients, as the assistance flows from AusAID through TAF and then through the JPPR secretariat before it reaches the implementing organisations. For another, projects seemed extraordinarily decentralised and beyond any meaningful oversight of JPPR (meaning the secretariat in Jakarta) or TAF. (The team, however, did not have the opportunity to meet with JPPR implementing organisations.) Similarly, IFES is fully aware of the support it receives from AusAID, but the NGOs and other implementing partners involved in particular AusAID-supported voter education activities in the regions may not be. In these cases, we can reasonably speculate, the Australian origins of the assistance may be obscured.
d.	Absence of Broader Relevance of Election Program 
As previously noted, AusAID’s assistance was relevant in that it responded to expressed priorities and specific needs. The AusAID program, however, did not address several of the most fundamental challenges to the elections, including problems with the voter registration lists and vote count tabulation and the flawed process of constituting the current KPU. These problems flowed in part from a flawed legislative framework, and negative trends were largely apparent from the experience with Pilkada, including major problems with the voter registry. The KPU’s lack of capacity, which was apparent from the dysfunctional selection process for the current KPU members, also contributed to many problems with the elections. In the view of many informed observers, these shortcomings—and the inability of the KPU to effectively address them—meant that the 2009 elections, as compared to previous elections, were in many respects a step backwards in the consolidation of democracy in Indonesia. Although international experts, donors and others, including the AEC and other AusAID partners, identified these shortcomings in advance as serious threats to the integrity of the election process, the GoI and the Steering Committee did not invite international assistance to address these areas.
Moreover, in comparison to previous elections, there was considerably less civil society involvement in the election process in 2009. A technical committee chose 26 civil society organisations (CSOs) to implement voter education programs. This committee was managed by the UNDP but strongly influenced by MoHA policy to only support CSOs formally registered with MoHA. There was little support for organisations that desired to work separately from the government or KPU or to pursue different priorities in supporting the election process. Donors do not appear to have challenged the control by Bappenas, MoHA and the GoI more generally over the process of choosing NGOs to receive foreign funding. Although Bappenas did organise consultation and coordination meetings with some civil society representatives, no representatives from civil society were directly involved in the Steering Committee process. 
The government’s responsibility for overseeing foreign assistance to the elections is consistent with the emphasis on local ownership in the Paris Declaration on Aid Effectiveness and the Jakarta Commitment. Nevertheless, the current approval process is cumbersome and, by giving the government such a strong voice in the details of international assistance programs, unduly constrains the nature of donor support for CSOs. This does not seem consistent with the spirit of the Paris Declaration. Nor does it seem consistent with the stated goals of Indonesia’s Second Medium-Term Development Plan (2010-2014), which prioritises among other things the development of democracy through strengthening the role of civil society and political parties in the life of the country. 
In general, Australian assistance to the 2009 elections in Indonesia contributed to the objectives set out in the AIP Country Strategy 2008-2013 Pillar 3, “to strengthen the capacity, accountability and responsiveness of legal, democratic and oversight institutions and processes.” The assistance, however, did not target the Strategy’s objective to “build civil society capacity in analysing and monitoring the quality of election processes… ” as it did not support Indonesian domestic election monitoring or civil society policy analysis. 
Effectiveness
The framework and TOR call for the review to consider whether the program achieved its objectives. In general, AusAID programs were effective in contributing to strengthening the aspects of election administration and management that had been mutually agreed were areas where assistance could be provided. Programs aimed at supporting voter education may have added value locally to the civic discourse on public participation in the election process. But while some projects achieved their specific objectives, others did not succeed in delivering key portions of their programming within a timeframe necessary to be effective or within the constraints of an agreed-upon budget.  Moreover, the ultimate effectiveness of AusAID’s assistance, and the assistance of the greater donor community was limited by the inability to address the key issues that were most critical to the success of the election process.
a.	Support for Election Administration and Management 
Elections-MDP. The UNDP’s Elections-MDP proposed general objectives, and AusAID contributed to the first of these: supporting election administration and management. There were several subactivities. 
First, Elections-MDP provided two short-term procurement specialists to support the KPU procurement staff at both the central and provincial levels. Through these experts, Elections-MDP provided training and technical assistance to KPU’s procurement process, and it channeled the production of more than 90 different procurement forms and bidding documents. According to the UNDP, this assistance “has greatly contributed to overcome[ing] the fears and the paralyzing effect” on the KPU staff of the stringent, new national procurement regulation, Keppres (Presidential Decree) 80/2003. With the support of Elections-MDP, the KPU was able to comply with 99 percent of its procurement deadlines. This technical assistance and training for the procurement process was effective because it ensured compliance with procurement rules and good practice, enabled the KPU to procure necessary election commodities at the least expensive price given the necessary quality and within established deadlines, and helped participants to understand the new rules.
Second, to facilitate training of election officials, Elections-MDP conducted simulations to test the new regulation on voting and counting, Regulation 5/2008, and it raised awareness about the need to produce further regulations to clarify the marking of the ballots, which resulted in Regulation 13/2009. It also organised KPU and  KPUD training sessions on the new regulations on voting and counting at the national level and in seven provincial capitals and conducted a second series for the presidential election. With technical input from the AEC, Elections-MDP produced, printed and distributed more than 650,000 election manuals and VCDs for officials at the polling stations and subdistrict election committees. It supported a new polling station manual for the presidential election. There were problems with the distribution of hard copies, but the manual was uploaded to the KPU’s website,. 
Third, to assist the KPU’s media and public relations, Elections-MDP supported a KPU media centre. It hired a private company to provide technical assistance to the media centre and to help formulate the KPU’s new media, communications and public relations strategies. The KPU improved its communication through a monthly newsletter, donor meetings and a webpage, among other things. As part of the process of revalidating the voter registry, the KPU used the media, including public service announcements on radio and texts on television, to encourage citizens to verify the lists. 
Fourth, with AusAID funds, Elections-MDP also supported the development and implementation of regulations for local elections. It sponsored a study on how to improve the legal framework for local executive elections (pilkada). 
Due to identified gaps and to funding and time constraints, Elections-MDP was unable to conduct the entire project as initially planned. Accordingly, it “revise[d] the scope of the project to reinforce electoral management and planning.” Elections-MDP representatives anticipate a second phase, which will emphasise the “electoral project cycle approach.”
AEC. The chairman of KPU said that AEC and Partnership assistance was effective in drafting KPU regulations, conducting voter education, and producing manuals. AEC support addressed the training needs of the KPU through a training needs analysis and by assisting the KPU in developing their training program. The main focus of AEC support was the production of procedure manuals for polling station officials. Unfortunately, as noted above, while the AEC produced apparently high quality manuals, in part because of delays in the KPU’s approval of the draft manual submitted by the AEC, the UNDP was unable to distribute many of them in time to be valuable.  
Partnership. Poor planning limited the effectiveness of the Partnership program in particular. Initially, Partnership defined two components of its program: developing job descriptions and assisting the KPU in facilitating a re-staffing process based on such job descriptions. Partnership consultants produced the job descriptions, but the KPU lacked the capacity to implement the recommended changes. By its own account, Partnership expected additional, follow-on funding from AusAID to ensure that its efforts to restructure the KPU bureaucracy were effectively implemented. Although Partnership officials believed that both program components were necessary to the ultimate success of compliance with Law 22/2007, as is clear in its proposal, it failed to effectively allocate funds or plan accordingly to match this need.   AusAID did refer to the funding provided to Partnership as “interim support” and expectations of additional funding were not entirely unreasonable, but Partnership could have increased the effectiveness of its program by designing it in a way that was not entirely dependent on such funding, especially in the near term. 
Nevertheless, the KPU reported that the activities conducted by Partnership to assist the KPU in developing a manual on staff allocation and identifying the need to increase capacity was somewhat beneficial and will help them manage similar processes in the future. 
b.	Support for Voter Education
IFES. IFES’s voter education programs arguably contributed to meeting the objective of enhancing public awareness of the election process, as articulated in the AIP strategy. In the locations in which they were carried out, the IFES voter education activities facilitated a better public understanding of the election process and engaged selected civil society groups and academic institutions in promoting positive civic participation messages and providing information regarding the election process, such as how to check voter registry information. 
TAF/JPPR. AusAID funds supported the national secretariat of JPPR from March to August 2009 rather than for specific programs. As discussed above, JPPR, in turn, conducted various activities to inform voters and encourage participation in the process. 
It is unclear how effective the TAF/JPPR voter education activities were. One person made the argument that, to be effective, programs should “work with organisations that actually exist” rather than trying to create new organizations or mechanisms to share voter education messages. This point is well taken. JPPR said its strength is to promote democracy through religious teachings and to draw on its affiliation with NU and Muhammadiyah. Nevertheless, it is hard to imagine how local citizen forums, for example, can have any macro-level or sustained effect. Although they may have some intrinsic value at the local level and for the people involved, these activities seem pretty amorphous and ad hoc, and are not designed to have any concrete or larger impact. This is not to say that JPPR voter education programs are not worth supporting, but rather that support for JPPR efforts would benefit from greater structure, more concrete objectives, and increased attention to broader impact and monitoring and evaluation. 
Efficiency
a.	Use of Time and Resources
The efficiency of the program’s use of time and resources was mixed. On the positive side, judging from the comments of AusAID partners, the level of funding seemed largely appropriate. On the other hand, there was considerable uncertainty about future iterations of Aus-AID’s electoral assistance. Officials involved in several AusAID-supported activities, including Partnership, Elections-MDP and the AEC, mentioned the expectation of future assistance under the pending longer-term AusAID election support program. As noted above, the effectiveness of some activities appeared to depend on such additional funding. 
Moreover, under Law 22/2007, the KPU cannot receive foreign assistance directly; the government must receive and manage such assistance. This requirement complicated assistance to election management, by creating a much larger, more complex authorizing environment and interposing the GoI into the relationship between donors and foreign technical advisors on the one hand and the KPU on the other. Unlike previous elections, not only were donors unable to provide financial assistance directly to the KPU, but foreign technical advisers formally could not have a direct relationship with the KPU; rather they had to work out the details of their assistance with the government rather than with the intended clients. 
b.	Value for Money
The ICR Framework asks about value for money and financial variations. Interviewees generally did not criticise the level of funding or complain about any funding gaps, except that Partnership complained about the lack of follow-on funding. Specific activities seem to have been funded at reasonable levels, although AEC, IFES and Partnership believed more funds could have been allocated for training purposes. Generally, judging from the interviews, the funding was appropriate to the activities, and there was no suggestion or evidence that increased funding would have improved project performance. 
It is our view, however, that the overall program was not designed for optimal value for money because, as discussed elsewhere, it did not target the most important challenges to the consolidation of an effective election process in Indonesia. More timely follow-on funding also might have increased the impact of earlier investments. 
c.	Responsiveness to Changing Needs
Managers of each of the activities were responsive to changing needs. TAF and JPPR, for example, responded to the need to encourage citizens to check the voter register between the legislative and presidential elections. Partners working on election management adjusted the nature of their training to respond to actual concerns, especially between the legislative and presidential elections; because of problems during the legislative elections, for example, the training for the presidential round paid special attention to the recapitulation forms (tally sheets). Likewise, several implementers credited AusAID itself for its flexible response to problems such as those posed by currency fluctuations and new challenges to the election process.
d.	Delays in Implementation
The activities carried out with AusAID funds largely began relatively late in the process. This resulted partly from delays in the election process itself. Necessary new election laws were not approved until 2007. The new KPU was appointed only in late 2007. 
Moreover, within the Australian government, for various reasons, approval for the project came later than would have been ideal. Many implementers complained that the activity suffered considerable delays in approval, which resulted in relatively short time for preparation and implementation late in the process. Given the many constraints on the process, the review team does not mean to assign blame for such delays, only to point out their effect on the program’s efficiency.
Delays in the approval and implementation of the program hampered its effectiveness. For example, the AEC produced apparently high quality manuals, but many were not distributed in time to be valuable. An AEC representative explained to the review team that the timing of their deployment affected their ability to meet their goals. The team was not able to determine whether implementers could have prevented or minimised the effects of these delays on their particular programs.   
e.	Staffing
In general, the implementing organisations appeared to have appropriate staffing resources. None of the interviewees emphasised any particular problem with the level or appropriateness of staffing resources. Some, however, suggested that the Elections-MDP staff did not effectively manage its relations with donors and were too focused on providing operational support to the KPU rather than providing critical strategic advice. 
f.	Risk Management 
The team did not note any problems with overall risk management of the activity. Indeed, the team would have preferred that the overall program exhibit a greater willingness to take chances in trying to address some of the more fundamental problems in the management of the process or in coordinating its assistance with the GoI. 
Australian partners avoided undue risk, which did protect them from criticism and perhaps protected their ability to work on election issues in the future. The AEC, for example, chose not to engage in issues related to the vote tabulation process for the presidential election in part because it perceived an undue risk of being blamed for failures, which in turn might complicate possibilities for future assistance. In contrast, after IFES helped to set up a pilot program for rapid transmittal of election results by SMS (text messaging), although not with Australian funding, IFES found itself the subject of political attacks, considerable controversy and a legal challenge in the constitutional court. 
g.	Support for Election Management and Administration
Elections-MDP. Elections-MDP reports that it was recruiting late in the process, until February 2009, just shortly before the legislative election day. Timing was also a problem with the Media Centre. As the UNDP put it, “The timely finalisation, printing and distribution of the electoral manuals and resource books for KPU and Bawaslu officials was a significant challenge.”
AEC. The AEC programs were designed to minimise risks and maintain the flexibility necessary to address emerging needs. They were also designed with an understanding that the total amount of money being allocated by Australia to assist the elections was small in comparison to the scale of an Indonesian election. The AEC determined that training and manual development could have significant “multiplier effects” and thus were appropriate activities given the level of funding. 
Partnership. Partnership’s program to develop job descriptions was efficient in managing the funds which were allocated. However, its inability to allocate funds in a manner that would ensure the value of its first intervention was clearly inefficient. 
h.	Support for Voter Education
It is unclear how efficient or cost-effective AusAID-supported voter education efforts were. IFES’s voter education activities took advantage of opportunities to attract free media, but there is little evidence of how many people saw or participated in them. JPPR completely turned over its leadership in March 2009, while the election process was going on. This was unfortunate timing, given that JPPR is an election-related organisation.
Impact
Rigorous, scientific impact evaluation of AusAID’s election program in Indonesia or of particular activities would require the design and implementation of an evaluation methodology from the beginning of the program. Even if such an approach were possible for an election assistance program, it is not possible to design such an evaluation approach after the program has taken place. Among other things, a baseline would be essential. Moreover, the short timeframe for this program makes it harder to judge program impact. In addition, within the UNDP Elections-MDP it is difficult to attribute particular assistance efforts in a way that allows for an evaluation of AusAID impact. 

Although it is difficult to make links between activities and impact, we can make inferences about the overall project and individual activities. A number of interviewees stated, for example, that the AusAID program may have contributed, at least in locations in which the program was active, to increased involvement of women in politics and to increased voter registration. 

AusAID’s continued support to the KPU contributes to building the capacity of the commission’s staff at both the national and provincial levels. As some implementers point out, however, “much ground needs still to be covered and technical assistance in the fields of electoral management, planning and logistics should be increased during the second phase to foster increasing ownership and professionalism of the target institutions.”

As discussed elsewhere, the inability of the overall election assistance to address some critical shortcomings of the process limited its broader impact on the overall quality of the election process.
Sustainability
At a macro level, the Paris Declaration and Jakarta Commitment are intended to reinforce the importance of local ownership and sustainability of development programs. By responding to GoI and KPU expressed priorities, AusAID has designed its program in a way that, because of government buy-in, should be sustainable.  
At the project and activity level, there was also considerable focus on sustainability. By focusing on training, capacity building and organisational reforms, the project was designed to have lasting effects. But, as one Indonesian government official argued, Indonesia and its international partners still need to pay additional attention to the problem of how to institutionalise the election process and avoid addressing the same types of election-related issues every five years. 
In addition, in failing to support Indonesian civil society policy analysis and advocacy, the program missed an opportunity to build Indonesian capacity to engage in ongoing constructive dialogue on electoral reform. 
Elections-MDP. The UNDP’s efforts on procurement were expressly intended not only to move forward an effective procurement process in 2009 but also to train people and establish procedures for the future. The manuals themselves contributed to capacity building at the polling station level and helped to strengthen institutional memory. 
The ultimate sustainability of the UNDP program was contingent on follow up funding to support the program’s second phase. Funding for the second phase of the project is “the only possible chance to ensure long-term capacity building and thus . . . ownership by our partners and beneficiaries of the ability to conduct elections,” according to the UNDP project report.
Elections-MDP conducted training sessions, workshops, seminars and meetings intended to build capacity, which should have benefits in the future. Elections-MDP also coordinated with local government, supported CSOs and worked through Indonesian private organisations, for the media centre and an aid implementation study, for example. These efforts inherently build local capacity and thus support the goal of sustainability. 
AEC. Rather than directly training officials or producing manuals, the AEC sought to build Indonesian capacity by involving the KPU in preparing manuals and training modules. By focusing on training and manual development for polling station officials, AEC staff members facilitated building a “corporate memory” among poll workers that could last through multiple electoral cycles. AEC officials expressed to the review team a conscious effort to move away from directly providing assistance to building the capacity of Indonesians to provide this assistance in the future. 
IFES. IFES workshops focused on increasing women’s participation in the political process in a sustainable manner by adopting short-run and long-run strategies that included developing a network of stakeholders. By identifying women interested in political participation, introducing them to party officials, and developing a network of women candidates and interested stakeholders, the program set the stage for longer-term capacity-building programs.
IFES’s partnership with University of Indonesia Puskapol (Center for Political Studies) to develop a long-term electoral system and political processes curriculum was inherently sustainable. In meeting with UI-Puskapol staff members, the evaluation team was convinced that their desire to continue pursuing such a curriculum is genuine. Through joint meetings with political and electoral specialists from universities throughout Indonesia, the program created a network of academics committed to the development of a standardised electoral process curriculum. 
TAF/JPPR. By working with the organisations that already exist—the member organisations of the JPPR network, which also belong to the massive Muhmmadiyah or NU networks—the program had the potential to be more sustainable. By investing in JPPR, theoretically the knowledge is going to organisations that will continue to exist and serve as advocates for democracy and effective elections and as a constituency for good governance. But in practice the program was too loose and not well defined. JPPR members were vague in articulating the problems their programs were addressing. 
TAF representatives claimed that local citizen forums were one of the program’s most successful activities, but they took place only one time, with no follow up.
Partnership. Partnership’s program attempted to address key problems in the long-term sustainability of the KPU’s capacity to effectively manage the electoral process. By clarifying the roles and responsibilities of Secretariat General staff members, the program had the potential to lay a foundation on which the KPU could develop its organisation in the future. 
On the other hand, Partnership failed to ensure the sustainability of its efforts in support of the restructuring of the KPU secretariat. While it produced useful job descriptions, subsequent implementation depended on further funding, which was not guaranteed. Nor did Partnership involve enough KPU staff members in the process to ensure appropriate follow up. The KPU reportedly lacks capacity and funds to follow up on Partnership efforts, although the KPU did report it plans to use these job descriptions. Partnership expressed its belief that the KPU secretariat believed in the value of the exercise. 
Gender Equality
Generally, AusAID’s implementing partners took gender equality into account in their programming. The various activities undertaken provided mixed opportunities to actively promote gender equality, but the outcome of improved electoral processes benefitted the entire Indonesian population, both men and women. 
Elections-MDP. Elections-MDP developed a database of Indonesian women political candidates. It also arranged for its regional gender advisor to visit Jakarta and conduct a review of the project to gauge its effectiveness at incorporating this cross-cutting theme. Elections-MDP promoted women’s participation in its workshops, meetings and training sessions. In fact, the UNDP reported that as many as 21 percent of the participants were women. Elections-MDP organised a national gathering for women candidates and meetings in 20 cities in eight provinces, although it is not clear that these activities were supported by AusAID funds, even though they are reported to AusAID. 
AEC and Partnership. Although the review team did not have an opportunity to review the AEC-developed manuals, such manuals were reportedly sensitive to gender issues. Activities that targeted the management of the election and building the capacity of the KPU did not target gender concerns per se. For example, Partnership’s work to assist with restructuring of the KPU was unable to promote gender equality because the make-up of the existing KPU workforce constrained organisational changes. While the reorganisation provided an opportunity for gender balance and equality to be addressed in the re-staffing process, it is unclear if Partnership was cognisant of gender issues in the development of job descriptions and its approach to the reorganisation of the KPU. While the gender balance of KPU staffing could have been directly addressed in the re-staffing of the KPU and the implementation of the developed job descriptions, it was not apparent in conversations with Partnership staff members or in a review of their final completion report that they were focused on the issue. However, of the five expert consultants commissioned by Partnership to provide technical assistance to the KPU, three were female. 

IFES and TAF/JPPR. Voter education efforts were intended to focus on concerns and needs of women in particular, along with marginalised groups. The Women’s Socialisation Program of IFES held in five provinces, which included workshops to encourage the participation of women in electoral management bodies, was one of the strongest examples of promoting gender equality. The workshops helped to address gender equality in three important areas of the election process: electoral management, women’s representation, and the perspective of civil society on women’s participation in the election process. Likewise, TAF and JPPR encouraged participation of women in citizens’ forums and produced a brochure focusing on women in politics. JPPR explored how to encourage the participation of women and other marginalised people in its voter education activities. The review team, however, did not have the opportunity to assess the actual impact of these efforts.  
Monitoring and Evaluation
Some efforts were made to monitor and evaluate project activities, but in general it seems there was relatively little consideration of monitoring and evaluation in the design of programs. Monitoring and evaluation efforts often appear to have measured program outputs rather than outcomes.
Elections-MDP.  Elections-MDP established a number of indicators to track the program’s progress, including the number of manuals produced and distributed.  It also contracted with Yappika (Aliansi Masyarakat Sipil untuk Demokrasi, Civil Society Alliance for Democracy) to conduct an aid implementation study.
AEC and IFES. The absence of indicators for some activities made it difficult to measure their effectiveness. While observers did report seeing the manuals in use on election day, for example, the AEC had no mechanism for determining how many polling stations used AEC-produced manuals. IFES did not present a monitoring and evaluation framework. While ratings reports were available for public service announcements broadcast on television, the reports only gauge saturation of programming and do not really measure penetration, much less impact. 
Partnership.  Partnership established seven indicators as part of a monitoring and evaluation system to be measured before and after its program. As part of this system, it conducted personal competency assessments of 71 KPU employees to determine the level of competency of current KPU staff. In its completion report, Partnership cited the use of evaluation questionnaires, but at the time of the report’s release the questionnaires had not been returned and were not included in documents provided to the review team. Partnership also proposed indicators for both components of the project, developing the job descriptions and facilitating the re-staffing process of the KPU based on the new job descriptions. 
TAF/JPPR. TAF hired Yayasan Peran to do program monitoring, and both JPPR and TAF staff members visited program sites to monitor the activities. On the other hand, JPPR and TAF do not appear to have robust or systematic information about the activities of organisations in the JPPR network. TAF stated that the objective was to encourage citizen participation and suggested in conversations with the review team that the indicators included the increase in the number of registered voters and whether citizens had checked voter lists. TAF representatives admitted that there had been no attempt to collect numerical or quantitative data.
Bappenas was planning, as of the date of the preparation of this report, to complete its own assessment of the assistance by October 2009. 
Analysis and Learning
A number of interviewees suggested that AusAID itself made a genuine attempt to “know the terrain” and often gathered information and analysis from partners. Likewise, program implementers generally demonstrated serious attention to analysis and learning. The program in support of the 2009 elections, however, could have more effectively drawn on the experience of election support in 2004. As stated above, foreign assistance failed to address a number of key problems with the election process, including some problems identified well before election day. 
As mentioned above, the UNDP justified the attention to public relations based on its analysis that “The bad image of the KPU in the media was mostly due to the lack of institutional capacity and manpower in the field of public relations and media analysis.” This analysis seems flawed at best. The public criticism of the KPU was largely based on the questionable process of selecting the commissioners, its unsteady performance and the series of serious problems with the elections, such as the problems with voter registration. Even though those problems could not be reasonably or wholly blamed on the KPU, they colored the public perception of the commission. The problem was not primarily one of poor public relations.
The AEC draws on its long-term engagement in Indonesia and has made efforts to understand electoral challenges and the needs of the KPU. The AEC had an important platform for providing assistance to the KPU, given that the organisation had provided similar assistance to the commission in both 1999 and 2004. The AEC’s extensive prior experience working with its Indonesian counterpart undoubtedly informed its work in support of the 2009 election cycle. The AEC assistance was also informed by a 1 July 2008 joint review of the KPU’s previous training plan as well as a 19 August peer review of the draft design of Australian assistance to the Indonesian election process. 
A national baseline survey on voter awareness issues, funded by AusAID, informed IFES’s development of public service announcements. This survey was used to determine both the locations most in need of voter education and the style of messages that would be most effective. The survey identified television broadcasting as the most effective method of informing voters on the election process. Overall, the survey showed the need for such voter education programs and informed both the design and implementation of programs, which contributed to their ultimate effectiveness. In contrast, other IFES voter education activities and those of JPPR evidently were not grounded in any empirical or analytical research. 

Evaluation Criteria Ratings

Evaluation Criteria	Rating (1-6)	
Relevance	5	
Effectiveness	4	
Efficiency	4	
Sustainability	4	
Gender Equality	5	
Monitoring & Evaluation	3	
Analysis & Learning	5	
Rating scale:
Satisfactory	Less than satisfactory	
6	Very high quality	3	Less than adequate quality	
5	Good quality	2	Poor quality	
4	Adequate quality	1	Very poor quality	

Lessons Learned and Recommendations
Timing
•	Support to Indonesia’s electoral systems should be provided throughout the five-year electoral cycle, not just in the lead-up to election day.
After the 2004 elections, there was little international engagement, funding or technical assistance for election matters until late in the electoral cycle leading up to 2009. Finalisation of the legislative and organisational framework for the Indonesian election itself was largely delayed until 2007. Undoubtedly, this delay and uncertainty created difficulties for donor support to the elections. 
A number of AusAID stakeholders voiced concern over the uncertainty and slow pace of AusAID’s program design processes. These comments focused especially on the design of the next phase of Australia’s support to Indonesia’s electoral systems and the relationship between the ‘interim’ support for the 2009 elections and this longer-term design. The short timeframe for implementation of the interim program limited the scope of the AEC’s engagement, and Partnership stated that concrete implementation of its KPU job descriptions depended on further funding under the next AusAID elections support program. Elections-MDP partner organisations likewise raised the issue of slow implementation.
An important lesson for future programming, therefore, is that support to Indonesia’s electoral systems should be provided throughout the five-year electoral cycle, not just in the lead-up to election day. If AusAID decides to continue engagement in the election sector, it is important to avoid initiating that engagement too late in the election cycle or too close to election day. The institutionalisation of democracy should be the central aim of longer-term support for elections in Indonesia.
Civil Society Engagement
•	AusAID should consider increasing engagement with Indonesian CSOs involved in election-related technical assistance, voter education and advocacy.
Some GoI representatives told the team they highly value the role of Indonesian CSOs, especially in the areas of voter education and technical assistance to electoral organisations. Likewise, stakeholders and observers stated that civil society played a vital role in the elections and could have had greater impact with increased funding and scope to work. 
In its future programming AusAID should consider increasing engagement with Indonesian CSOs in order to bolster their capacity to provide technical assistance to the KPU and the legislative process, to educate the public and to advocate for needed changes. CSOs have important comparative advantages, such as extensive networks, the ability to harness domestic expertise, and the potential to mobilise political support for reform. 
In addition to carrying out voter education efforts commissioned by the KPU or the government, CSOs in a free society should have the autonomy to work outside the direction and control of the government and to pursue their own priorities. 
AusAID should support the increased engagement of civil society organisations in future elections on the basis of compliance with the Jakarta Commitment. Among other things, Australia could support the participation of experts from Indonesian CSOs in future national coordination mechanisms for election assistance.  The Jakarta Commitment calls for “developing a new partnership paradigm” and “expanding dialogue to include new actors.” It calls specifically for increased engagement with civil society. By supporting civil society engagement in the election process and in the broader democratic process in Indonesia, AusAID will be furthering the stated goals of the GoI in the Jakarta Commitment to develop “a spirit of genuine partnership, engagement and joint responsibility” with “civil society, private sector and traditional development partners.” By specifically focusing on civil society engagement, AusAID could contribute to filling a gap in meeting the specified priorities of the Jakarta Commitment. 
Donor Coordination, Paris Declaration on Aid Effectiveness and Jakarta Commitment 
•	AusAID should engage the Government of Indonesia in dialogue about the role of a robust and active civil society in a democracy; it should support the role of civil society organizations in the democratic process generally and as stakeholders in decisions about the proper role and use of foreign assistance.
•	AusAID should ensure programmatic accountability (as opposed to and in addition to financial accountability) for any funds provided through a multidonor mechanism.
Foreign assistance to Indonesia’s 2009 elections was generally consistent with the major themes of both the Paris Declaration on Aid Effectiveness and the Jakarta Committment.  AusAID and AusAID’s implementing partners operated consistent with local ownership of Indonesia’s development strategy. This local ownership, however, was too consolidated within specific parts of the Indonesian government and not properly shared across the range of local stakeholders in the election process. For example, no experts from Indonesian civil society were invited to take part as members of the national Steering Committee. There was a broad consensus from implementing partners and other foreign donors that the Ministry of Home Affairs exercised disproportionate influence in the Steering Committee approval process. This dangerous imbalance is inconsistent with the spirit of a multistakeholder approval process. Given that an election project could not proceed without the clearance of the Steering Committee, allowing one ministry to be, in effect, the ultimate arbiter of civil society engagement is not healthy for the long-term democratic balance of Indonesia.
Moreover, donors and implementers were insufficiently assertive in challenging GoI constraints on foreign assistance or MoHA influence over the project approval process. Sharp limits on the involvement of Indonesia civil society organisations or efforts to subject CSOs to government control are inconsistent with the spirit of The Paris Declaration and Jakarta Commitment. A robust and active civil society is critical to the success of democracy in any society. If the current imbalance is not addressed, it could prove to be a serious challenge to the long-term consolidation of democracy in Indonesia.
 From the donor perspective, the need to maintain a positive relationship with the GoI is an understandable political reality and a high priority, but this relationship is not incompatible with strengthening Indonesia’s democratic balance. Foreign donors should not be excessively concerned about being “too political.” The Paris Declaration and Jakarta Commitment do not prevent the international community from supporting broader engagement of civil society organisations in election issues, policy analysis and advocacy. AusAID should engage the GoI in dialogue about the role of a robust and active civil society in a democracy. 
In addition, the greater donor community expressed frustration with the Elections-MDP’s ability to address key election problems. Some donor representatives also would have preferred more substantive communication and consultation between Elections-MDP and donors. In fact, Elections-MDP itself admitted that “appropriate management direction in communicating and meeting the expectations of the donors was a bit challenging.” Some donors, including AusAID, earmarked their contributions to the multidonor fund for specific purposes, while others did not. Whether this was a contributing factor to the lack of harmonisation is unclear, but in considering future contributions to similar implementation mechanisms, AusAID should be cognisant of this reality. 
Engaging Core Areas of Concern
•	While taking account of political realities, AusAID should consider supporting efforts to address critical areas of concern in the election process, such as voter registration, vote tabulation and the electoral framework. 
•	AusAID should engage the GoI in dialogue about priorities for effective foreign assistance.
There is broad consensus among donors, implementers, media and most election stakeholders that there were fundamental problems in the administration and management of the 2009 Indonesian elections, including problems with voter registration and vote tabulation processes and flaws in the process of selecting and constituting the KPU. Moreover, these and other problematic issues flowed from a flawed legislative framework for Indonesian elections. Foreign donors largely avoided working in these areas, due in part to GoI sensitivities. 
AusAID should consider whether there are appropriate mechanisms through which it can support efforts to address problems with voter registration and vote tabulation and to improve the electoral framework more generally. Civil society organisations able to engage constructively in dialogue and to provide technical assistance addressing these issues should be supported in their efforts. The international community should engage the GoI to preempt efforts to constrain effective foreign assistance. The value of engagement in the election sector might be in question if AusAID and other donors are unable to address key problems with the legal framework (e.g., the law on election management), voter registration, and results count compilation.
Expanding Visibility
•	AusAID should seek to improve the visibility of its efforts to support democratic elections in Indonesia.
AusAID should also consider adopting a more conscious strategy to improve the visibility of its efforts to support democratic elections in Indonesia. Although AusAID was the largest donor to the 2009 elections, there seems to be greater awareness of the assistance efforts of the UNDP and the U.S. Agency for International Development. AusAID should consider a stronger set of branding requirements as well as, perhaps, a more conscious public relations and media strategy.

Monitoring and Evaluation
•	AusAID should require more robust monitoring and evaluation systems that can inform program performance reporting. 
•	AusAID consider means to improve impact evaluation. 
Monitoring and evaluation problems were generally about poor design, rather than poor implementation. In future assistance to Indonesian elections, AusAID should consider including a more robust M&E framework by requiring implementers to submit M&E plans that define key quantitative and qualitative indicators that can be used to track program performance. Implementers should be required to track these defined indicators with M&E checklists throughout the life of the program and to discuss their M&E efforts in program completion reports. 
Rigorous impact evaluation of AusAID’s election program in Indonesia could potentially establish the effects of the program in comparison to what would be observed in the absence of the program. Impact evaluations, including evaluations using randomized, quasi-experimental and similar designs, have the potential to improve assessment of the effects of programs by controlling for factors in the external environment that might affect the results of a program. Such a design would require comparison or control groups, or random assignment of assistance across groups, It would also need to be planned from the beginning of a project and included in the program design. Impact evaluations generally must include three fundamental elements: (1) relevant and measurable indicators of project outcomes that correspond to hypothesized causal relationships; (2) data collection before, during, and after program implementation to measure program impact over time; and (3) nontreatment control group(s). By identifying control groups, taking measurements on important indicators for the control and participant groups both before and after the program, and using randomization or other statistical techniques to minimize unintentional bias in the results, AusAID and its partners might be able learn much more the impact of their programs and how those programs can be improved.

APPENDIX A: BIOGRAPHICAL INFORMATION ON TEAM MEMBERS
ERIC BJORNLUND 
Eric Bjornlund is co-founder and president of Democracy International, a U.S.-based consulting firm specializing in international democracy and governance assistance founded in 2003. A lawyer and development professional with two decades of international experience, Mr. Bjornlund has designed and managed democratic development programs in 35 countries in Africa, Asia, Europe and Eurasia, and the Middle East. From 1989 to 2000, Mr. Bjornlund worked for the National Democratic Institute for International Affairs (NDI) in various senior positions, including as Senior Associate and Asia Director. 
In Indonesia, Mr. Bjornlund began working on domestic election monitoring, civil-military and civic education programs in the mid-1990s. From 1999 to 2000, he served as NDI Country Director in Indonesia and oversaw a comprehensive program in support of Indonesia’s transition to democracy, including domestic and international election monitoring, constitutional and law reform, anticorruption, political party development, legislative process, decentralization and regional autonomy, civil-military relations, and civil society advocacy. In 2004, as Field Office Director for The Carter Center in Indonesia, he designed and directed comprehensive election monitoring program for Indonesia’s first direct presidential elections. In recent years, in addition to the current review for AusAID, Mr. Bjornlund has conducted a series of assessments and evaluations in Indonesia for USAID, including an assessment of political party assistance (2002), an assessment of local elections, including elections in Aceh (2005), a local government program evaluation (2006), an assessment of election administration and election dispute resolution mechanisms (2007), and a broad democracy and governance assessment and strategy (2008) as well as a recent evaluation of the Indonesian programs of the Netherlands Institute for Multiparty Democracy. He has served as a technical expert and senior advisor for Democracy International’s four multiyear programs in Indonesia: the Democratic Reform Support Program, the Local Government Support Program (including support for local elections in Aceh and other provinces), the Serasi Conflict Mitigation Project, and Annual National Opinion Polling for USAID. 
Mr. Bjornlund has written and spoken extensively about transitional and postconflict elections, democratization, legal reform and international democracy promotion. He is author of Beyond Free and Fair: Monitoring Elections and Building Democracy (Wilson Center Press and Johns Hopkins University Press, 2004) as well as numerous book chapters, articles, essays and reports. Mr. Bjornlund has testified on many occasions before the U.S. Congress as well as before the United Nations. He has spoken at conferences and universities throughout the world and has appeared often on television and radio programs as an expert commentator.  
For four years earlier in his career, Mr. Bjornlund practiced corporate and international law at Ropes & Gray in Boston, Massachusetts, one of the largest law firms in the United States. He holds a Juris Doctor from Columbia University in New York, New York; a Master in Public Administration from John F. Kennedy School of Government at Harvard University in Cambridge, Massachusetts; and a Bachelor of Arts magna cum laude from Williams College in Williamstown, Massachusetts.
a.	

RUSTAM IBRAHIM
Rustam Ibrahim is Senior Research Associate and Member of the Board of Directors of Lembaga Penelitian, Pendidikan dan Penerangan Ekonomi dan Sosial (LP3ES - Institute for Economic and Social Research, Education and Information) a well-known national NGO and research institution in Jakarta. He started his career at LP3ES in 1976, holding various positions in the organisation including Executive Director (1993 to 1999), Senior Advisor (1999 to 2000) and Senior Research Associate (2000 to present). He was also one of the founders of YAPPIKA, the Civil Society Alliance for Democracy, an Indonesian nongovernmental organization. From 1998 to 2004, he was Board Chairman and now serves as a Senior Advisor.  Currently, Mr. Ibrahim also serves as a member of the Governing Board of Komunitas Indonesia untuk Demokrasi (KID - Indonesian Community for Democracy), an educational institution that carries out adult education for promotion of democracy in Indonesia. In 2004 he took a lead role in forming the Indonesian Working Group on CSO Accountability and is chair of the group. 
Mr. Ibrahim began working on promotion of democracy, election monitoring and voter education in 1996. In 1996 he acted as Project Director of the program to increase the effectiveness of LP3ES and other selected NGOs that promote democracy. The program consisted of several activities such as public opinion polling and human rights training. In 1999, as Executive Director of LP3ES he managed the UNDP-funded Block Grant Program which provided financial support to 37 local NGOs in conducting voter education and election monitoring activities during the 1999 election. In 2004, as Vice Chairman of Jurdil Pemilu, an alliance between LP3ES, NDI and Forum Rektor to conduct election monitoring and voter education, he led the introduction and presentation of a vote quick count system, a tool to verify the integrity of the vote counting process.
Mr. Ibrahim wrote the Civil Society Report entitled Indonesian Civil Society 2006: (Still) a long journey to a civil society, based on research conducted by YAPPIKA. Together with Hans Antlov and Peter van Tuijl, Mr. Ibrahim wrote an article entitled “NGO Governance and Accountability in Indonesia: Challenges in a Newly Democratizing Country” in Lisa Jordan & Peter van Tuijl, NGO Accountability: Politics, Principles & Innovations (London, Sterling: Earthscan 2006). He also conducted several other research and survey activities and authored reports including, among others: Survey on Civil Society Sector and Activities of NGOs in the Fields of Environmental Issues and Education in Indonesia (conducted for JCIE, Japan, 2003), Governance, Organizational Effectiveness and the Nonprofit Sector, Country Report (presented for APPC Conference in Manila, Philippines, September 5-7, 2003; written with  Abdi Suryaningati and Tom Malik); The National Directory of  Civil Society Resource Organizations (CSRO): Indonesia (for The Synergos Institute, New York, 2000); Create an Effective of Non-Profit Board: Case Study of the Indonesian Family Planning Organizations (for PBSP, Philippines, 2000), Dian Desa: A Case Study in Earned Income Strategies for Financial Sustainability (for The Synergos Institute, 2001). 
Mr. Ibrahim has also written several articles and papers on the subject of democracy, civil society and NGO legitimacy and accountability, including: Indonesia NGO Agenda Toward the Year of 2000 (ed., 1995), The New Order Political Format: Reconsidered (ed. 1995) and The Strategy to Build Civil Society (ed., 1999). 
Mr. Ibrahim completed a degree in Political Science and acquired his postgraduate diploma from the Institute of Social Studies (ISS) in The Hague, Netherlands (1984). He also attended Training on Civil Society, Democracy and Development, Institute of Social and Economic Culture (ISEC),  Boston University, Boston, MA, 1994 and Public Opinion Polling, Chapel Hill, North Carolina, 1992
MICHAEL COLLINS
Michael Collins is a Policy Analyst at AusAID’s Indonesia and East Timor Branch in Canberra. In that role, he has responsibility for analysis of democratic governance, climate change and environmental programs in Indonesia, as well as political and development issues in Papua and Aceh.
From 2007 to 2009 Mr. Collins worked as an analyst of Indonesian political, economic and social affairs at the Australian Government’s Office of National Assessments (ONA). Previously he helped administer the Australian National University’s international student exchange program at the university’s International Education Office.
Mr. Collins holds a Bachelor of Arts / Bachelor of Asian Studies (Specialist) degree from the Australian National University, and is currently undertaking a Master of Strategic Affairs, also at ANU.
  
APPENDIX B: LIST OF PEOPLE INTERVIEWED 

Government of Indonesia 

-	National General Election (KPU)
o	Commissioners
Prof. Dr. H.A. Hafiz Anshary, Chairman
Sri Nuryanti, Member
Endang Sulastri, Member
I Gusti Putu Artha, Member
Prof. Dr. Syamsul Bahri
Andi Nurpati Baharuddin

o	Secretary General Office
Suripto Bambang Setyadi, Secretary General
Binsar Siagian, Head of Training & Development Division

-	Bappenas (National Development Planning Agency)
o	Rd. Siliwanti, Director, Directorate for Political Affairs and Communications
o	Otto H. Hadi

-	Ministry of Home Affairs, Directorate of Political Culture Development
o	David Yama, Section Head of Political Legislation

Implementing Partners

-	Australia Electoral Commission
o	Michael Maley, Director International Services

-	UNDP
o	Irman G. Lanti, Program Manager, Deepening Democracy (Electoral Support, Parliamentary Development and Democracy Assessment, Democracy Governance Unit
o	Fida Nasrallah, Chief Technical Adviser Elections-MDP

-	The Asia Foundation
o	Sandra Hamid, Ph.D, Senior Director for Programs
o	Nathalia Warat, Program Officer-Elections

-	IFES
o	Adam Schmidt, Country Director in Indonesia
o	Anhar Djamal, Senior Project Coordinator
o	Admira Salim

-	The Partnership for Governance Reform
o	Utama Sandjaja, Advisor, Democratic Governance

-	JPPR
o	Daniel Zuchron, National Coordinator, National Secretariat
o	Masykurudin, National Secretary, National Secretariat
-	PUSKAPOL UI
o	Sri Budi Eko Wardani, Executive Director
o	Yolanda Panjaitan, Research Officer

Donors

-	CIDA: Renaldy B. Martin, Program Officer (Development)
-	Dutch Embassy: Wachid Ridwan, Political Assistant
-	USAID: Christopher A. Miller, Democracy and Governance Officer, Office of Democratic Governance.  

Others

-	CETRO
o	Refly Harun, Senior Researcher

-	Prof. Dr. Ramlan Surbakti, Former Vice Chairman of KPU; Advisor, Partnership

APPENDIX C: EVALUATION METHODOLOGY

Review of Australian Assistance to the 2009 Indonesian Elections

Proposed Evaluation Methodology (Draft)

September 6, 2009

Evaluation Approaches
Rigorous impact evaluation of AusAID’s election program in Indonesia would require the design and implementation of an evaluation methodology from the beginning of the program. As a recent report by the National Research Council of the U.S. National Academy of Sciences puts it, “by using comparison or control groups, or random assignment of assistance across groups or individuals,” impact evaluations seek to establish “the effects of policy interventions relative to what would be observed in the absence of such interventions.” As in medicine or public health, for example, randomized control trials represent the gold standard in program evaluation methods because they provide a measure of the independent effects of a particular project or activity. Obviously, such an approach is not possible after the fact of an election assistance program.

Nevertheless, it is possible to conduct a process evaluation to assess how and why the program unfolded as it did and to draw qualitative inferences about program effects. It is also possible to address the perceptions of project participants, sometimes called a participatory evaluation. Using these approaches, we will seek to assess, to the extent we reasonably can, (1) program impact, and (2) program management and implementation. First, to try to assess the impact and success of the program, the evaluation team will examine available data, including information from project monitoring and evaluation efforts, and existing project reports. We will also conduct structured interviews with program participants and stakeholders. To the extent time permits, we will also conduct limited interviews with informed nonparticipants; such interviews can serve some of the functions of a control group. Similarly, to assess project management and implementation, we will conduct semi-structured interviews and review program management and performance documentation.

Following are the priority questions from the “Terms of Reference of the Review of Australian Assistance to the 2009 Indonesian Elections” (TOR):

Assessing Program Impact

1.	What were the specific program objectives, both for the overall program and for each activity? Were the objectives of the program appropriate and achievable?
2.	How well did the program and its components meet their objectives?
3.	Was the program’s focus on strengthening electoral administration and fostering civic engagement with the electoral process appropriate? How successful was the institutional support provided? Can any links be made between activities and impact?

Evaluating Program Management and Implementation

1.	How effective were the management and implementation of particular activities?
2.	How appropriate was the level of funding? 
3.	Were there any significant gaps in targeting assistance?
4.	To what extent did the activities relate and contribute to the objectives set out in the AIP Country Strategy 2008-13? Were the activities consistent with the objectives outlined in the project proposals? 
5.	What lessons can be drawn from the 2009 assistance program in the implementation of:
a.	A longer-term program of support over the entire electoral cycle
b.	In the lead-up to a similar large-scale election event in 2014
c.	Broader democratic governance programs?

Data Collection and Analysis  
The review team will use a combination of methods for collecting relevant information for this evaluation, including a desk study of relevant documents and semi-structured and open-ended interviews in Indonesia (and, via telephone, in Australia). 

To conduct its desk study, the team will review relevant documents about the program, including strategy and planning documents, project reports, interim evaluations and any public reports. These documents include:

•	BAPPENAS: An Assessment Report of International Donor Support For the 2009 Election, January 2009
•	The Asia Foundation: Promoting Civic Participation in the 2009 Elections, March 2009
•	IFES: Indonesia Strengthening Electoral Process (ISEP) – Final Program Implementation Report, April 2009
•	UNDP Elections Multi-Donor Programme to Support Democratic Elections in Indonesia: Quarterly Progress Report on AusAID’s Contribution (January-March 2009), April 2009
•	IFES: Indonesia: Technical Support on Public Information/ Socialization Activities for the 2009 Elections, May 2009
•	The Partnership for Governance Reforms for Indonesia: Project Final Report: Strengthening Capacity of the General Election Commission (SCALE), May 2009
•	AEC Mid Term Report, May 2009
•	AusAID: Monitoring Report Australia’s Support to the 2009 Elections, June 2009
•	UNDP Elections Multi-Donor Programme to Support Democratic Elections in Indonesia: Second Quarterly Progress Report on AusAID’s Contribution “Supporting the Efforts to Improve the Quality of Indonesia’s Elections”, June 2009
•	UNDP Elections Multi-Donor Programme to Support Democratic Elections in Indonesia: Quarterly Progress Report on AusAID’s Contribution (April-June 2009), July 2009

In conducting the desk study, the team will draw on its knowledge of the history and current status of electoral institutions and processes and of democratic development in Indonesia. The desk study will enable the team to better understand the program’s context and approaches.

The evaluation team will then convene in Indonesia to conduct interviews and collect additional information about the project. Upon its arrival in country, the team will meet with representatives of AusAID to agree on the final workplan and schedule of interviews and develop a more detailed understanding of AusAID’s goals for the evaluation. The team will conduct semi-structured interviews guided by the questionnaire/list of questions below, modified as needed during the course of the research. These interviews will allow the evaluation team to develop an understanding of project nuances and challenges.

AusAID has already suggested interviewees who can provide substantive feedback on the project and its impact. These interviewees are expected to include representatives of AusAID and of its implementing partners, including the UNDP, the AEC, IFES, The Asia Foundation, and the Partnership for Governance. It will also include program participants from the KPU, civil society groups and other stakeholders. To the extent possible, the team will also meet with political and election analysts, such as journalists, academics or other observers, as well as diplomats and aid officials from other governments and intergovernmental organizations. 

Evaluation Criteria Ratings
The Terms of Reference ask the evaluation team to rate on a scale of 1 to 6 the program’s relevance, effectiveness, efficiency, sustainability, gender equality, monitoring & evaluation, and analysis & learning. The team will work together to determine these ratings and will strive to be as objective as possible. The team will use the process of making these ratings as an aid to its analysis of the overall program. 

Team Members and Division of Responsibilities
Each team member will be involved in and contribute to the entire evaluation. At the same time, however, each team member will focus on a part of the overall program, based on his particular strengths and experience. We will begin with a division by each of the project’s three activity areas. Specifically, Eric Bjornlund will focus on the AusAID contribution to the UNDP Multi-Donor Electoral Progam. Rustam Ibrahim will address Civil Society Engagement in the Electoral Process. Michael Collins will focus on the support of the Australian Electoral Commission to the KPU. 

Illustrative Questions
The following are illustrative questions structured around the central topics in the priority questions included in the TOR. The evaluation team will use these questions, as and when appropriate, to provide an element of structure and commonality to information derived from interviews and to facilitate analysis of that information. The team will finalize the list of questions in consultation with AusAID. 

Program Objectives (Appropriateness and Impact):
Country Level
1.	Did the project contribute to the larger goals for improved democratic governance laid out by in the Country Plan and supported by other AusAid programming?
2.	In what ways did the program contribute to those goals? How can we identify that impact?
3.	To what extent does the interviewee believe that international assistance in general and AusAID assistance in particular has assisted or benefitted the KPU or other program beneficiary in performing their tasks?
4.	What future assistance for Indonesian elections from the international community is necessary, desirable or appropriate?
Program Level
1.	What were the overall program goals? Were they reasonable given the political and institutional context in which they were implemented? 
2.	Were individual program activities devised to contribute to overall program goals?
3.	To what extent was AusAID assistance relevant to the needs of the KPU (or other beneficiary organization)?
4.	Was overall program impact measurable in a meaningful way?
Activity Level
1.	How well did specific activities work? How effective were they at achieving their stated results?
2.	Were any activities significantly more effective than others? Were those activities’ goals in any way different (e.g., more achievable, more measurable, etc.) from those of other activities?
Management and Implementation:
1.	Did the program carry out all scheduled and proposed activities and meet goals and benchmarks laid out in the proposal and/or workplan?
2.	Were there any significant delays in the project schedule? Were the project(s) implemented in sufficient time to be effective?
3.	Did the number and variety of implementing partners, and subsequent institutional environments, assist or impede the achieving of program objectives?
4.	Do you have any concerns or criticisms of the implementation of the project?
Level of Funding/Gaps in Assistance:
1.	Were stated program/activity goals reasonable given the level of funding?
2.	Were there any obvious areas for potential electoral assistance that were not being covered by international donors? Did the lack of such assistance undermine the efficacy of the program?
3.	Were program funds attributable to particular assistance efforts within the UNDP Multi-Donor Electoral Program in a way that allows for an evaluation of AusAID impact?
4.	Was there significant overlap between AusAID funded activities and those funded by other donors?
Monitoring and Evaluation:
Program Outcomes 
1.	Were impact (or other) evaluation methodologies considered when designing the program monitoring and evaluation plan?
2.	Did the monitoring and evaluation plan measure program outcomes or outputs? What outcomes or outputs did it measure?
3.	Did program-level indicators feed into country-level indicators in a meaningful and intuitive way?
Activity Outputs
1.	Were appropriate activity-level indicators identified to measure outcomes and performance?
2.	Did activity-level indicators feed into program-level indicators in a meaningful and intuitive way?
Lessons Learned:
1.	What are some strengths and weaknesses of the support provided by AusAID and its implementing partners through this program?
2.	What types of assistance were most effective? What institutions are most receptive to assistance and effective in using it to realize positive impact?
3.	Overall, was the division of assistance among multiple partners and activities helpful in realizing program impact or detrimental to it?

APPENDIX D: TERMS OF REFERENCE

Terms of Reference
Review of Australian Assistance to the 2009 Indonesian Elections

Background

Australia provided $6.2 million to assist Indonesia prepare for and hold Parliamentary and Presidential elections in 2009.  Australia’s support to Indonesia’s 2009 elections aimed to support the Government of Indonesia’s priority of the “implementation of a democratic, honest and just general election in 2009”.  Support for the 2009 elections is also framed by the Australia Indonesia Country Strategy 2008-13 Pillar 3, “to strengthen the capacity, accountability and responsiveness of legal, democratic and oversight institutions and processes”. This funding builds on Australia’s support for democratic elections in Indonesia over a decade, including $15 million in 1999 and $12 million in 2004.  For the 2009 elections Australia has been supporting:

•	UNDP Multi-Donor Electoral Program (E-MDP) (AUD $3.6 million)
o	Australian funding was used to support the development of electoral procedures and administration; operational training for election workers; assisting the Indonesia General Election Commission (KPU) media and public relations, including help to establish an elections results centre; and supporting the KPU to develop regulations for local elections. 
o	The UNDP MDP is the main body coordinating donor assistance for the elections.

•	Australia Electoral Commission (AEC) (approx AUD $780,061)
o	The funding has built on the established relationship between the KPU and the AEC. The AEC is providing technical assistance to the KPU, primarily through the development of election day manuals and KPU electoral training modules.

•	Increased civil society engagement in the electoral process through international and national NGOs (approx AUD $1,618,820). 
o	This assistance has been channelled through IFES, the Asia Foundation and through the Partnership for Governance reform with a focus on;
•	developing voter education public service announcements;
•	assisting in the design and testing of ballot designs;
•	supporting the restructure of KPU Secretariat General; and
•	disseminating information about new electoral procedures.

AusAID has managed the agreement with these implementing partners separately through a series of requests for proposals. A broader program monitoring and evaluation framework is not available to provide with comprehensive information on the overall quality of the support. However, the Government of Indonesia (with support from UNDP) established a monitoring and evaluation system to assess all international assistance to Indonesia for the 2009 Elections. The first assessment report in January 2009 was very positive regarding Australia’s contribution to the 2009 Elections.  A second ‘mid-line assessment of support to the 2009 elections’ evaluation is currently taking place again with support from the UNDP.  

Details of all donor assistance to Indonesia for the 2009 election are provided at Attachment 2. Australia will be providing further longer term assistance over the entire electoral cycle to Indonesia from 2009 – 2014.

Review Objective

The objective of this review is to assess the appropriateness, efficiency and effectiveness of Australia’s support to the 2009 elections and inform future assistance in this area.

Scope

The IPR/ICR will assess and rate the program’s performance against the evaluation criteria of relevance, efficiency, effectiveness, impact (or potential impact), sustainability, monitoring and evaluation, gender equality and analysis and learning.  The ratings will be based on the standard AusAID six-point scale, as outlined in the IPR/ICR template (see Attachment 4). Standard evaluation questions to guide the evaluation team in forming these ratings are at Attachment 5. 

Although the evaluation team must be able to provide an assessment and rating of the evaluation criterion above, the team should give particular priority to examining the following questions:

•	How well did the program and its components meet their objectives? Were the objectives of the program appropriate?

•	How effective was the management and implementation of individual activities?  

•	How appropriate was the program in contributing to the conduct of the 2009 election given that our support has been focused on strengthening electoral administration and fostering civic engagement with the electoral process?  Can any links be made between activities and impact, in particular around institutional support provided?

•	Indonesia has implemented elections since 1999. However, the country still faced ongoing issues in electoral administration and management in the 2009 elections. Therefore how appropriate was the level of funding in support of the 2009 elections – were there any significant gaps in targeting assistance?

•	Were the activities consistent with the objectives outlined in the project proposals, and to what extent did the activities relate and contribute to, the objectives set out in the AIP Country Strategy 2008-2013?
•	What were the strengths and weaknesses of monitoring arrangements of the activity and broader outcomes level? 

•	What lessons can be taken from 2009 assistance in the implementation of;
d)	a longer term program of support over the entire electoral cycle; 
e)	in the lead up to a similar large scale election event in 2014; and
f)	to broader democratic governance programs?

Duration and Phasing

The independent assessment is expected to commence in August and be completed no later than October 2009. Expected phasing of the review is as follows:

a)	Literature/document review, Australian consultations (if necessary) and development of methodology – 7 days
b)	Indonesian mission – 10 days, including travel time (not including weekends)
c)	Preparation of Draft Independent Review Report – 5 days
d)	Completion and revision of final Independent Review Report – 4 days.

Outputs

1)	Methodology

The team will develop a methodology setting out how the review team intends to collect information on the review, undertake consultations and ensure the final report reflects good research practice. It will include a list of evaluation questions that will be answered in response to the TORs: methods for sampling, collecting and analysing data; and allocation of specific areas of responsibility for the evaluation team during the data collection as well as for reporting. The team leader is responsible for producing the methodology and will seek input from other members as necessary. The methodology should be submitted to AusAID one week prior to the in-country mission. 

2)	Pre-departure briefing

The team will attend a pre-departure briefing with AusAID in Jakarta to discuss issues for the Review, to present the methodology and to provide feedback/input for proposed itinerary and meetings schedule.

3)	Aide Memoire

On the completion of the mission, an Aide Memoire (maximum 5 pages) on key findings will be presented to the Director for Political Affairs & Communication of the State Ministry of National Development Planning/National Planning Agency (BAPPENAS) and AusAID.

4)	Draft Independent Review Report

A draft Independent Review Report will be submitted by the Review team within 10 days of the completion of the mission to Jakarta. The draft report must include draft ratings against AusAID Quality at Completion Report ratings. This report will be submitted to stakeholders who will provide comments on the strengths and weaknesses of the draft and recommendations for its finalisation. AusAID will provide feedback to the Review Team within 4 weeks of the submission date. 

5)	Final Independent Review Report

The Review team will consider stakeholder comments in preparing the Final Report (maximum 25 pages), which will be submitted within 1 week of receiving final comments from AusAID.  The report will be published at AusAID’s discretion.

If required, the Review Team will discuss the team’s preliminary findings in a workshop with key stakeholders. Participants in the workshop may include the program team, beneficiary representatives, key counterpart officials, relevant AusAID staff, members of any Technical Advisory Group or other advisory mechanism, and other donors working in the sector or country. The evaluation team can be asked to document its initial findings into an Aide Memoire to use as a basis for these discussions. 

Personnel

Review Team members must be independent of the aid activity, and should not include any members who have had past involvement with the activity.

The review will draw on international electoral experience in order to assess the overall mix of activities and the nature of their impact in the broader effectiveness of the election assistance. The focus for personnel on the team will therefore be on bringing together expertise on international and Indonesian electoral / political systems, as well as civil society in order to provide broader knowledge on comparative electoral systems and donor support.

The Independent Review Team will comprise the following members:

a)	International Elections Expert

The International Elections Expert will have a strong background in comparable international election systems, administration and processes. Knowledge of Indonesia’s elections system is a must. The Expert will possess very high analytical skills, an ability to distill information and write constructive, informative reports that are not overly lengthy or jargonistic. The Expert will have a forward-looking perspective in terms of looking for lessons and implications to inform future programming. S/he will have strong previous experience in evaluation and the ability to draw on international best practice to inform the review. Team leadership skills, together with an understanding of Indonesian politics and culture are essential (Indonesian language skills desirable).  

b)	Indonesian Elections Expert

The Indonesian Elections Expert will have a strong background in elections systems, administration and processes. The Indonesian Expert has strong knowledge of both the institutional electoral systems and processes as well as civil society’s engagement and efforts around electoral reform, voter awareness and education. Experience in participating in reviews of international donor assistance and an ability to write assessment reports in English is highly desirable.

Neither the international nor Indonesian elections experts should have had any involvement in assistance provided by Australia for the 2009 elections.

c)	AusAID Representative

The AusAID representative will have a strong understanding of Australia’s broader assistance to Indonesia, particularly in the area of democratic governance, and will have no prior engagement with the program. He/she will ensure that the review meets AusAID evaluation requirements. The AusAID representative will have understanding and knowledge of the Paris Declaration on Aid Effectiveness to provide analysis on harmonisation and alignment with partner government systems and the effectiveness of Australian support in this sector. He/she will take direction from the team leader on attendance at meetings with stakeholders, particularly where the performance of AusAID will be discussed.

Attachment 1

List of Key Documents provided by AusAID

o	BAPPENAS: An Assessment Report of International Donor Support For the 2009 Election, January 2009

o	The Asia Foundation: Promoting Civic Participation in the 2009 Elections, March 2009

o	IFES: Indonesia Strengthening Electoral Process (ISEP) – Final Program Implementation Report, April 2009

o	UNDP Elections Multi-donor Programme to Support Democratic Elections in Indonesia: Quarterly Progress Report on AusAID’s Contribution (January-March 2009), April 2009

o	IFES: Indonesia: Technical Support on Public Information/ Socialization Activities for the 2009 Elections, May 2009

o	The Partnership for Governance Reforms for Indonesia: Project Final Report: Strengthening Capacity of the General Election Commission (SCALE), May 2009

o	AEC Mid Term Report, May 2009

o	AusAID: Monitoring Report Australia’s Support to the 2009 Elections, June 2009

o	UNDP Elections Multi-donor Programme to Support Democratic Elections in Indonesia: Second Quarterly Progress Report on AusAID’s Contribution “Supporting the Efforts to Improve the Quality of Indonesia’s Elections”, June 2009

o	UNDP Elections Multi-donor Programme to Support Democratic Elections in Indonesia: Quarterly Progress Report on AusAID’s Contribution (April-June 2009), July 2009

Attachment 2

Reviews of Indonesia’s 2009 Elections

Who	Review	Objectives	Details	
Government of Indonesia, Bappenas.

Supported by UNDP	Mid-line assessment on International Assistance in Election 2009	1.	To compare the baseline result with the midline assessment result;
2.	to compare the agreed work plan with the activities in the field
3.	to ensure effectiveness of international assistance
4.	to gather data that can enable the Coordination Team to follow up any findings and to settle issues that may arise from activities supported by international assistance	This is a mid-term assessment of all donor assistance to 2009 Elections.  In Jan 2009, elections MDP conducted a baseline assessment on the international assistance in support of the 2009 election.	
Independent UNDP assessment	Assessment of UNDP MDP assistance	UNDP is looking to undertake a  mid-term review of assistance	To assess UNDP MDP assistance to date – details still being discussed	
KPU review of 2009 Elections	KPU Internal Review	Collaborating with the KPU in the preparation of the ToRs for their internal institutional evaluation exercise. The review will focus on the lessons learnt and how to take them for the second phase.	Details to come – 
Also AEC review (?).	
AusAID assessment of Australian assistance	Review of Australian Assistance to 2009 elections – covered by these Terms of Reference	•	To assess the appropriateness and effectiveness of 2009 elections support to inform the longer term program; and
•	Assess the mix and nature of assistance in the leadup to the 2009 election events themselves, to inform possible future election event support in 2014.	Review covered by these Terms of Reference	
Partnership for Governance Reform	Electoral Integrity on Process and Result	•	To review process and implementation of Elections 2009.
•	To provide strategic recommendations for the implementation of local elections 2010 and strengthening electoral legal framework. 	Funded by DANIDA as part of their support on Promoting the Integrity and Quality of 2009 Elections.	

Attachment 3

List of Proposed People to Meet

GOI:

-	National General Election (KPU)
o	Commissioners
o	Secretariat General
o	Local KPUs

-	Bawaslu (Election Supervisory Body)

-	Bappenas (National Development Planning Agency)
o	Ibu Siliwanti, Director for Political Affairs and Communications
o	Otho H. Hadi

-	Ministry of Home Affairs, Directorate for Political Culture

Implementing Partners:

-	Australia Electoral Commission
o	Trefor Owen
o	Michael Maley

-	UNDP
o	Rizal Malik, Team Leader, Democratic Governance Unit
o	Irman G. Lanti, Program Manager, Deepening Democracy
o	Fida Nasrallah, Technical Adviser E-MDP

-	The Asia Foundation
o	Jeremy Gross
o	Nathalia Warat

-	IFES
o	Adam Schmidt
o	Anhar Djamal
o	Admira Salim

-	The Partnership for Governance Reform
o	Utama Sandjaja
o	Maria Radjali

-	JPPR
o	National Secretariat
o	Local coordinators

Donors:

-	CIDA: Renaldy Martin
-	DFID: Smita Notosusanto
-	Dutch Embassy: Wachid Ridwan
-	Spanish Embassy

DFAT:
o	Steve Barraclough