Development Evaluation Policy

Introduction

The Department of Foreign Affairs and Trade (DFAT) delivers Australia’s international development program. The aim of the program is to promote Australia’s national interests by contributing to a stable, prosperous and resilient Indo-Pacific region. Robust evaluations play a vital role in ensuring we deliver a high-quality development program. They assist us by:

• Providing evidence and lessons to underpin our development spend.
• Supporting an organisational culture focused on innovation and continuous learning.
• Reinforcing public confidence in government by credibly demonstrating the achievements of the development program.
• Strengthening our program management as well as the knowledge and skills of our staff.
• Assisting us to meet our accountability obligations under the Public Governance, Performance and Accountability Act 2013, which requires DFAT to assess and report on its financial and non-financial performance.

What is evaluation?

We define evaluation as the systematic and objective assessment of an ongoing or completed investment, program or policy.^[1] It is an in-depth process which takes place on a periodic basis. Evaluation aims to provide credible evidence which can inform major program management and policy decisions and highlight important development lessons.

Evaluation is distinct from, but related to, monitoring. Monitoring is a continuous process which examines whether an investment, policy or program is on track to achieve its intended results. In DFAT, monitoring is characterised by activities such as dialogue with partners, monitoring visits to field sites, and the collection of data on key deliverables. Evaluation will generally use data gathered through monitoring as one source of evidence.

Evaluation in DFAT

Evaluations are initiated and managed by program areas, such as country and sector programs. Each program undertakes an annual process to identify and prioritise the evaluations they can use to account for and improve their work. Programs have flexibility to determine the highest priority issues their evaluations should focus on.

DFAT develops an annual Development Evaluation Plan that is approved by the Secretary and published on the DFAT website.

On occasion, program areas may wish to undertake a rapid management review to help inform immediate decisions required on individual investments. Rapid management reviews are similar to evaluations but involve less time and resources and are generally less rigorous. The requirements of this Evaluation Policy do not apply to rapid management reviews.

Evaluation use

Use is the driving force behind our evaluations. All our evaluations should be commissioned and conducted to maximise the use of evaluation findings and recommendations to improve our work.

To ensure evaluations are useful we focus on three areas: prioritisation; quality; and systems which facilitate use.

Prioritisation

We focus evaluations on the highest priority issues in our programs. Focusing on areas where there is the greatest demand for evidence and improvement will ensure evaluations add value and are used. This includes the need for accountability, particularly for large investments.

Priority evaluation topics may include areas where there are significant evidence gaps that need to be filled so we can make better decisions; issues that pose significant risks to the effectiveness of our work; or interventions which are high priority for the Australian Government and our partners.

Quality

For evaluation findings to be used they must be credible and robust. For this reason, all of our evaluation products and processes will be high quality. Quality is achieved in the following ways:

• We take advantage of both independent and internal perspectives. Evaluation teams should be led by an independent person who is not directly involved in program management to ensure the findings are objective. At the same time, evaluation teams should include DFAT staff to the extent possible. This will ensure evaluation teams understand our context and have insight into whether evaluation recommendations are appropriate and feasible. It will also ensure our staff have strong ownership of, and build their capacity in, evaluation.
• When conducting evaluations, we engage early with our partners, including partner governments and implementing partners, to the extent possible. We will ensure they participate in evaluation design and implementation, and that they understand our quality and publication expectations. Where we choose to undertake joint evaluations or allow evaluations to be led by one of our development partners, we will ensure that our quality, management response and publication requirements can be upheld.
• Evaluations fulfil our quality expectations, as outlined in our Monitoring and Evaluation Standards. Our evaluations are also conducted ethically. Evaluation teams should adhere to the requirements of DFAT’s Ethical Research and Evaluation Guidance Note and the Australasian Evaluation Society’s Guidelines for the Ethical Conduct of Evaluations.

Systems that facilitate use

Our systems and processes facilitate the use of evaluations. For example:

• We provide management responses to all evaluations. These outline whether we agree with the recommendations, and how and when recommendations will be implemented.
• All evaluations and management responses will be published on the DFAT website. This should be done within three months of an evaluation report being completed.
• We ensure evaluation findings can be easily accessed.
• We link design and evaluation. Our systems will ensure the design of our strategies, programs and investments takes into account findings from evaluations. Designs should also consider how appropriate data can be collected to ensure high quality evaluations can be conducted.
• We have senior management oversight of evaluations.

Roles and responsibilities

The Program Enabling Division is responsible for overseeing this policy and associated guidance; providing support to program areas to prioritise and conduct program evaluations; and compiling, reviewing, publishing and reporting against the annual Development Evaluation Plan.

The Aid Governance Board provides oversight of Australia’s development investment portfolio. It is responsible for ensuring evaluation findings are used to inform the development strategies and investments it approves.

The DFAT Secretary approves the annual Development Evaluation Plan and reviews progress against it.

DFAT program areas (including country/regional, global and sector programs) are responsible for identifying, completing and publishing evaluations as specified in the annual Development Evaluation Plan. First Assistant Secretaries are responsible for deciding which program evaluations will be conducted. Senior managers (e.g. Assistant Secretaries and Minister Counsellors) sign off on management responses.