Australian NGO Cooperation Program (ANCP)
Monitoring and evaluation
Monitoring Evaluation and Learning Framework
The Monitoring Evaluation and Learning Framework (MELF) provides the evidence base to demonstrate the effectiveness of the ANCP in achieving results in economic growth and poverty reduction. The MELF consists of the following elements:
- common reporting formats, one for annual planning (due 30 June); and another for reporting performance and results against a common set of indicators (approximately 90). The performance report includes a financial acquittal (due 30 September)
- meta-evaluations which focus on the quality and range of outcomes in the implementation of ANCP activities
- thematic reviews which provide an opportunity to explore particular topics around development effectiveness in greater depth and demonstrate impact of ANCP against sectoral themes
- an annual program of in-country visits to monitor effectiveness, compliance and alignment with government priorities.
Stories of Significant Change
In 2019, ANCP NGOs trialled the use of Stories of Significant Change (SSC) to demonstrate long-term development outcomes not currently captured through annual reporting. SSCs are short, retrospective and evidence-rich narratives that describe a significant ANCP ANGO or project achievement that aligns with an ANCP intermediate outcome. SSCs will be used to:
- Report on ANCP progress as part of DFAT's annual reporting process
- Support lesson sharing during ANCP's Annual Learning Event
- Generate communications products for DFAT, ANGO and external audiences
- Inform selection of topics for deeper analysis such as thematic reviews.
Data Systems Validation Review
The ANCP Data Systems Validation Review was undertaken in response to a recommendation from the 2015 evaluation of ANCP by the Office of Development Effectiveness (ODE). Key findings of this review are:
- Overall there is relatively robust level of rigour in the output data provided by accredited Australian NGOs (ANGOs) to DFAT through the online portal, ANCP Online. Whilst ANGOs demonstrate variation in their monitoring and evaluation (M&E) systems, they have all been through accreditation and therefore all have quality assurance systems in place that meet the Australian Government's minimum expectations. Larger NGOs and those with country offices and field teams tend to have greater resources for M&E allocated to sampling, triangulating, in-country missions, and external evaluations.
- A number of lessons arise from the review, leading to the recommendations outlined below. The lessons and recommendations are about the ANCP as a program, and implementation will vary between ANGOs.
Key recommendations of this review are:
- DFAT, in consultation with ANGOs, to review the ANCP Program Logic and Monitoring, Evaluation And Learning Framework (MELF), specifically to:
- increase the emphasis on qualitative outcome indicators that demonstrate instances of significant change,
- reduce the total number of indicators,
- improve reporting against the Sustainable Development Goals (SDGs), and
- provide more detailed guidance on indicators, where appropriate, whilst maintaining ANCP's focus on flexibility and working within partner systems.
- DFAT to further facilitate the ANCP Monitoring, Evaluation and Learning (MEL) Reference Group, ensuring that it is convened on a regular basis.
- DFAT, in collaboration with the ANCP MEL Reference group, to revise the disaggregation of data. This could include clearer definitions for disability, and reviewing the need for disaggregation of urban/rural.
- ANGOs to allocate, as appropriate to their situation, more ANCP funding (up to 10%) to M&E to improve the quality of data coming from local partners. DFAT may consider encouraging ANGOs to allocate, at a minimum, 4% of ANCP funding to M&E, to ensure that partners are sufficiently investing in these areas, including management information systems (MIS).
- ANGOs to increase the quality assurance of data provided by their project partners. This could include building capacity of local partners; having increased resources available at country, regional or HQ level; and including data quality assurance/capacity assessments as part of the in-country monitoring missions.
ANCP Aid Program Performance Report
Each year the performance of the ANCP in achieving its objectives is assessed. The results of that assessment are provided in an Aid Program Performance Report (APPR).