Lao PDR Road Maintenance Project 2

INH714

INDEPENDENT COMPLETION REPORT

Charles M. Melhuish
Transport Economist and Policy Specialist
Independent Consultant

April 2010

Aid Activity Summary
Aid Activity Name	Lao PDR Road Maintenance Project 2	
AidWorks initiative number	INH714	
Commencement date	4 February 2008	Completion date	31 December 2010	
Total Australian $	2,800,000 (2.5%)	
Total other USD	1,000,000 (ADB loan:1.5%); 24,392,100(World Bank: 37%); 4,800,000 (Japanese PHRD: 7%); 11,060,000 (SIDA: 17%); 23,500,776 (Road Maintenance Fund-Lao Government: 35%)	
Delivery organisation(s)	The World Bank	
Implementing Partner(s)	The Ministry of Public Works and Transport	
Country/Region	Lao PDR/South East Asia	
Primary Sector	Transport	

Author’s Details
	Mr. Charles M. Melhuish is an independent transport economist and policy specialist based in the Philippines. He has more than 38 years experience in transport sector assessment in developing countries comprising 10 years with a firm of consultants, 24 years with the Asian Development Bank in Manila, Philippines and 4 years as an independent consultant. He can be contacted at charles.melhuish@gmail.com . 

Contents
Executive Summary	1
Introduction	4
Evaluation Findings	10
Evaluation Criteria Ratings	23
Conclusion and Recommendations	24

List of Abbreviations

ADB		:	Asian Development Bank
AusAID	:	Australian Agency for International Development
DPWT		:	Department of Public Works and Transport (Provincial)
GMS	:	Greater Mekong Subregion
ICR		:	Independent Completion Report
IDA		:	International Development Association
LRN		:	local road network
MPWT		:	Ministry of Public Works and Transport
NDF		:	Nordic Development Fund
NRN		:	national road network
PTI		:	Public Works and Transport Institute
RMF		:	road maintenance fund
RMP		:	road maintenance program
RMS		:	road management system
Sida		:	Swedish International Development Cooperation Agency
URMF		:	unified road management framework
Executive Summary

This is an independent completion report (ICR) for the Second Phase of the Road Maintenance Program (RMP2) in Lao People’s Democratic Republic (PDR) which was supported by the Australian Agency for International Development (AusAID). The RMP2 was approved in 2004 and was cofinanced by the World Bank/International Development Association, Swedish International Development Cooperation Agency, Nordic Development Fund, Asian Development Bank and Japan. Later in 2007, the government requested the AusAID to support the RMP2, and following a review of the RMP2, an AusAID grant was approved in November 2008.  The primary objectives of the RMP2 were to (i) scale up the financing mechanism to a sustainable level, (ii) operate efficient systems to manage road assets in all 17 provinces; (iii) expand institutional capacity at both central and local levels, and (iv) control heavy vehicle overloading. 

Given the relatively small amount of the AusAID funding and considering that it was only approved towards the end of the RMP2, this evaluation has focused on the overall program since it is not possible to disaggregate the effects of AusAID resources from the overall program. The ICR consultant has assessed the RMP2 successful in all areas. It has achieved its primary objectives and provides a good platform to achieve sustainable road maintenance in the future. The only objective that was not achieved concerns control of overloading.

Relevance

The focus of the RMP2 was highly relevant to the country’s needs and met the strategic priority of the government in attaining sustainable preservation of road assets. Support for road maintenance provided resources where there was a major resource gap and with the majority of roads being gravel surfaced its condition was highly dependent on good maintenance practices to keep roads open in the wet season. The AusAID support was in line with both its country and subregional assistance strategies. The former includes a focus on rural development including support for infrastructure to provide improved access to social services such as schools, hospitals and markets. In the subregion, infrastructure support to promote trade is a core pillar and a portion of the resources was earmarked to maintain a major east-west trade route linking Vietnam with Thailand. The RMP2 had also focussed on supporting and building country systems and the program was implemented at the central and local government levels through the Ministry of Public Works and Transport (MPWT).

Effectiveness

From a broad perspective, the RMP2 was effective because it achieved its main objectives. It has improved the condition of the road network particularly at the national and provincial levels. The benchmarks for roads in good condition over the 5-year program have been exceeded. The RMP2 has also been successful in mobilizing additional finance for road maintenance through continued increases in the fuel levy. The target of raising LAK100 billion was reached a year ahead of target and by 2009, revenues have further doubled. 

The third RMP2 objective was to enhance the capacity of addressing road maintenance at the central, provincial and local levels. Despite relatively weak institutions, the RMP2 has achieved considerable progress in developing the systems, processes, and procedures required to implement and manage road assets devolved to the provinces. While substantial additional capacity building is required, the RMP2 has provided a platform from which future institutional strengthening can be launched. 

However, truck overloading is the only objective that is not met. Providing weighing scales alone will not solve the problem. A more focused analysis on the systems and procedures, and application of good governance are required to obtain a better understanding of the problem prior to implementing remedial measures. 

Overall, the MPWT demonstrated effective leadership in implementing the RMP2 and ensuring that the major civil works components were completed on time and within budget. The success of the RMP2 will need to be continued to further strengthen the institutions, particularly in the provinces and districts. 

Efficiency

The RMP2 was implemented as scheduled, completed on time, and within the expected budget ceilings. Part of the efficiency of the RMP2 was due to the partnership approach adopted by the various donors. The various experiences of the donors were brought together to maximise the impact of the assistance and support to the different areas of road maintenance under the RMP2. In particular, this had a significant impact on delivering capacity building where the various donors who had previous experiences in different areas of road maintenance enabled them to better focus on those areas that needed strengthening. The RMP2 also built upon the success and lessons learned from the first phase and this reduced the risks and increased the efficiency of the second phase. The RMP2 was also cost effective and a sample of roads that were subject to ex-post economic analysis demonstrated high economic returns to the investment which compared favourably to those at appraisal.

Impact	

While it is too soon to evaluate the impact of the RMP2, there are a number of areas where the impact is likely to be important. First, it has highlighted the importance of road maintenance and has established it as a core business activity of the MPWT. Second, it has mobilised a significant amount of resources from road users for road maintenance. While the road management fund (RMF) can only currently support 30% of road maintenance needs, the RMF will continue to grow with expected increases in the fuel levy, together with expansion of motorization in the economy. Third, further impact has been the strengthened country systems that have been supported by the RMP2, many of which have been at the local level. The use of standard bidding documents, and government systems and procedures has strengthened the implementation and management of road maintenance contracts. Last, other areas of potential impact include the greater awareness of safeguard policies through the creation of an environment and social safeguards operations manual and the identification of the need to strengthen financial management and control issues in the future.

Sustainability

Significant emphasis was placed on sustainability during the RMP2 implementation. From the technical sustainability point of view, the main focus must be on improving the quality of maintenance works. This will require better oversight, quality control and contract management by the MPWT and the provincial public works departments; and improved performance by contractors. Part of the technical quality outcome results from limited resources allocated for the intended works and attempting to achieve “too much” with the available budget. Resource availability is a problem given the size of the network and annual budget, particularly if donors do not provide support. To avoid widespread deterioration in the network condition, a core network should be defined which should be highly maintained, with the rest of the network receiving the remaining available resources. This will ensure that the core road network will continue to provide economic and social benefits, and support countryside development. In regard to financial sustainability, it is important to maintain the growth in the RMF revenues that underpin road maintenance activity. Action is needed to manage the RMF more transparently and to ensure that the funds disbursed are well used and will generate high economic returns. Better communication with policymakers and road users is required. While significant success has been achieved in building capacity of the institutions involved, more effort is required to continue the development of skills for managing, administering, and implementing road maintenance at central and, particularly, local levels. The RMP2 has provided the platform for future progress. 

Gender Equality

Since the RMP2 did not incorporate a gender component, the ICR did not assess gender equality.

Monitoring and Evaluation

While the RMP2 established a results-based matrix, and set benchmarks and targets, the monitoring and evaluation (M&E) process needs to be better institutionalised within the MPWT oversight framework. The MPWT needs to strengthen its M&E procedures and processes not just to examine ad hoc projects and programs supported by donors but to assess how the institution is performing continuously. This will provide valuable feedback to demonstrate to policymakers that resources allocated for the road sector are being well used and delivering results. The M&E is an important management tool to provide self-assessment and feedback on successful and unsuccessful areas.

Analysis and Learning

The RMP2 demonstrated excellent learning by building upon the success and knowledge attained under the first phase. The program included the use of technical audits undertaken annually and designed to identify program outcomes that could be improved. While the use of these audits was innovative and highlighted many important issues, it is less apparent whether the issues identified earlier were solved promptly. The RMP2 would have benefited from a formal mechanism to implement the technical audit findings. 

Using the standard AusAID evaluation criteria, the RMP2 has been rated by the ICR as follows:
Evaluation criteria: Rating (1-6)
Relevance—5
Effectiveness—4
Efficiency—5
Sustainability—4
Gender equality—N/A
Monitoring and evaluation—4
Analysis and learning—4
Rating scale: 6 = very high quality; 1 = very low quality. Below 4 is less than satisfactory.
1.	Introduction

Background

The Lao Democratic People’s Republic (Lao PDR) has a population of 5.9 million (2007 data) and a land area of 236,800 km2. The population density of 24 persons/km2 is one of the lowest in the region and the majority of the population (65%) lives in rural areas. The country has vast natural resources including forestry, minerals and hydroelectric power. Agriculture is the major sector of the economy. It is mostly mountainous and does not have access to the sea. Although landlocked, Lao PDR is in the centre of the Mekong region bordered by Thailand, Vietnam, Southern China, Cambodia and Myanmar. It is thus in close proximity to the subregion’s population and market of almost 300 million. While steady economic growth over the past decade has significantly reduced poverty and improved the social indicators, Lao PDR had a GNI per capita of US$580 in 2007 and remains one of the world’s poorest countries. The incidence of poverty varies by geographic areas and ethnicity with rural poverty estimated at 38%. It is estimated that 29% of the population live on less than US$1 per day and 74% on less than US$2 per day. Empirical evidence from recent studies indicates a strong correlation between access to basic infrastructure services and the incidence of poverty.

When the project was formulated in 2004, the government’s socioeconomic development policy rested on three pillars: economic growth, sociocultural development and environmental preservation. The National Growth and Poverty Eradication Strategy of January 2004 identified roads as a key subsector for reduction of poverty in the poorest districts. Roads and road transport is the dominant mode of transportation in Lao PDR carrying more than 60% of freight traffic and 90% of passenger traffic. In the 15 years prior to appraisal in 2004, a high percentage of the Government’s public investment program had been devoted to rebuilding the road system. Of a total road system of 31,209 km, the national road network (NRN) comprised 7,141 km of which 53% was paved, and 73% was in good condition. However, the local road network (LRN) of provincial, district and rural roads, comprised 24,068 km of which only 3% was paved. Access constraints were significant with 26% of national and 34% of provincial roads experiencing closures for more than 3 months per year.

In 2000, the Ministry of Public Works and Transport (MPWT) embarked on a comprehensive program to strengthen its road maintenance capacity and capability. This program known as the first phase of the Road Maintenance Program (RMP1) was supported by external donors led by the World Bank and included the Swedish International Development Cooperation Agency (Sida) and the Nordic Development Fund (NDF). The RMP1 succeeded in initiating a new financing mechanism called the road maintenance fund (RMF); developed an appropriate road management system to help allocate funds and prioritize the maintenance needs of the road assets; initiated a local road management system; initiated control of heavy vehicle overloading; and implemented maintenance on substantial portions of the road networks. Partnership between the various donors was considered important to achieve its goals and objectives, with each donor focussing to different aspects of road maintenance activities under one umbrella.

While the RMP1 attained its objectives, it was recognised that the benefits would only be maintained if further assistance was invested both for road maintenance civil works as well as capacity building and institutional strengthening of sector institutions. Thus, the second phase of the RMP (RMP2) was established on a sound foundation and success of the RMP1 and continued to support similar activities that were previously implemented successfully. It was approved in 2004 and was also supported by a number of partners including the World Bank/International Development Association (IDA), Sida, NDF, Asian Development Bank (ADB) and Japan through the provision of a technical assistance grant. The AusAID was requested by the Government of Laos to join the program in late 2007 and the AusAID assistance was approved in November 2008. 

Program Objectives and Phases

The goal of the RMP is the sustainable preservation of road assets through the development and implementation of financing and road management systems. The program which utilises the World Bank’s adaptable program loan instrument provides for a 3-phase approach to attain its objectives. The RMP1 focused on initiating the program and developing its maintenance concept. This phase established a sustainable financing framework based on (i) user cost recovery and management of a dedicated fund; (ii) development of a road management system for national roads and local roads, and pilot application in four provinces; initial implementation of road maintenance programs under modified procedures; and introduction of improved control on heavy vehicle overloading. 

Under the RMP2, the focus was to (i) scale up the road financing mechanism to a sustainable level to meet the majority of road preservation needs; (ii) operate a system to manage road maintenance efficiently; (iii) implement sustainable level of financing and maintenance of the NRN; (iv) expand the institutional capacity at central and local levels in the public and private sectors in managing the LRN to all 17 provinces; and (v) control and monitor the operation of heavy vehicle overloading.

Project Components

The RMP2 has three components comprising preservation of the road networks, capacity building and project administration.

(i)	Component A: Preservation of the Road Networks (US$54.2 million): This component adopted a programmatic approach to support a slice of four annual work programs for periodic and routine maintenance on the NRN countrywide and on selected areas of the LRN in all 17 provinces. It also aimed to strengthen the quality of implementation. In a shift of focus from RMP1, the RMP2 apportioned half of the resources to maintain local roads with particular emphasis on provincial roads due to their poor condition. It also contributed to substantial improvements in the quality of access to all districts, especially the 47 poorest districts.

A1. National Roads: Support for the periodic maintenance and rehabilitation program at an average level of US$6.3 million a year by supplementing the resources of the RMF which is recovering an increasing portion of costs from road users and resources from other donors.

A2. Local Roads: Support for the periodic maintenance and rehabilitation of about 30% of the provincial road network at an average level of US$4.10 million a year with costs shared by the local government, the RMF, and external funding.

A3. Implementation Support: Assistance to central and provincial departments for improving the quality of implementation of the road preservation programs, including strengthened supervision of technical aspects and monitoring of safeguards aspects especially at the provincial level.

(ii)	Component B. Capacity Building (US$9.86 million): Designed to support the strengthening of the transport sector capacity in two core business functions, namely, management and financing of road infrastructure; and management of road transport and safety as well as the general institutional capacity at both national and sub-national levels.

B1. Road Management and Financing: The establishment and operation of appropriate procedures at central and provincial levels for planning, budgeting and implementing maintenance works programs on a) national roads and bridges; b) local roads and bridges (provincial, district and rural roads); and c) support for scaling up the RMF to a sustainable level of cost recovery and determining appropriate cost-sharing arrangements at the local level.

B2. Road Transport Management and Safety: Improvements to road transport administration procedures and systems, expanding the heavy transport management program of overloading control through the operation of new load control facilities built under the RMP1, and program development and implementation of road safety improvements.

B3. Institutional Capacity Building: Strengthening the capacity and performance of the MPWT and provincial DPWTs through a) improvements to the organisation and business processes for upgrading key business functions including safeguards, and for effective performance under a decentralised structure, b) human resource development including results-oriented learning and training programs, c) information technology and communications for upgrading the communication and computerisation of central and provincial offices, d) facilities for provision of basic office facilities and transport to support operations at district level, e) strengthening the transport business sector including construction and transport industries and consultant capacity, and f) environmental and social safeguards capacity enhancing baseline information on construction resources, community right-of-way and related activities.

(iii)	Component C. Project Administration (US$0.85 million): This component was designed to support efficient program administration through the three activities.

C1. Project Monitoring: Strengthening project management, reporting, monitoring of project impacts and safeguards, and preparation of future sector programs.

C2. Incremental Operating Costs: Modest budget support for project-related activities at central and provincial levels.

C3. Auditing: Undertaking financial and technical audits to ensure the quality of project implementation.

Funding Gap

During the midterm review of the project, it was noted that limited funds were left to cover road maintenance and other outstanding components of the project. There were no funds from the World Bank to finance the maintenance of national roads in 2007/2008 and national and provincial roads in 2008/2009. The funding shortfall was attributed to (i) the road works originally proposed at appraisal were to be from an IDA amount of US$28 million but this was subsequently reduced due to IDA financial envelope constraints. The first year program in 2005/2006 had been designed for the larger availability and utilised a high proportion of the RMP2 allocations; and (ii) a portion of the resources was also used to cover components that were originally identified for the NDF funding but this source of funding did not materialise as the fund was wound down and subsequently closed. Although the level of revenue generated by the RMF had increased faster than originally anticipated, the financing gap for the national and provincial roads in 2008/2009 was estimated to be US$7 million. Given the financial constraints of the program, the government requested an urgent support from AusAID to provide resources to help fill the financing gap. 

Specific Activities Financed by AusAID 

Under the circumstances, AusAID, in 2008, agreed to provide A$2.8 million (equivalent to US$2.5 million) to bridge part of the financing gap. The AusAID contribution was to support the implementation of the periodic maintenance of the national and provincial roads program in 2008/2009.

A.	National Road Component. A national road component covering three key national roads that required urgent maintenance and rehabilitation were identified as well as a number of bridges that had to be deferred due to a funding shortfall the previous year. The total cost was estimated at US$1,570,000, 70% of which would be financed by the AusAID contribution amounting to US$1,100,000. The individual components were as follows:

A1. Spot Improvements to National Road 9 (US$620,000). Road 9 is the east west corridor linking Thailand with Vietnam which was rehabilitated in 2003 using resources from ADB and Japan. One stretch of this road between km 54 to 106 had serious pavement failures due to a combination of heavy truck traffic and technical problems associated with its cement treated base course. The failures posed a major constraint to this newly established international route and without remedial spot improvements, it was expected that portions of the road would totally collapse.

A2. Rehabilitation of National Road 1C (US$450,000). Connecting National Road 13N at Pakmong junction in Luang Prabang province to Road 6 at Phoulao in Houaphan province, Road 1C has a total length of 349 km and was built more than 20 years ago to a paved macadam standard. The road deteriorated badly due to a shortage of funds for maintenance. A substantial portion of the road had recently been rehabilitated leaving a 44 km stretch with a gravel pavement. Under the proposed maintenance program, it was proposed to undertake spot improvement work at locations where the road was in particularly poor condition. 

A3. Periodic Maintenance of Road 11. (US$350,000). The proposed works on this road include the resealing of the section between Sanakham to Nasak in Vientiane province.

A4. Maintenance of Bridges (US$150,000). Due to a resource shortfall in 2007/2008, the maintenance of a number of bridges was deferred. This component will urgently undertake the needed maintenance on a number of priority bridges on national roads.

B.	Periodic Maintenance of Provincial Roads. The funding of periodic maintenance for provincial roads would continue the program of earlier years which primarily focused on the regravelling of gravel roads. It was anticipated that the funding would cover a total of about 600 km of roads covering 16 provinces. The roads were selected using the provincial road maintenance management system which is the planning tool developed with Sida assistance to prioritize maintenance requirements for provincial and district roads. The program was expected to cost US$1,710,000, of which 70% would be financed by the AusAID contribution amounting to US$1,200,000.

C.	Supervision Services (US$80,000). To provide necessary oversight for the conduct of the works, one local consultant was required to supervise the spot improvement of Road 9, one local consultant to assist Luang Prabang province supervise Road 1C and three local consultants to supervise the provincial roads periodic maintenance. For the small scattered works in other provinces, supervision was to be handled by provincial engineers. This arrangement continued the implementation arrangements that had been established for provincial and district roads and had been assessed to have worked well.

D.	Contingencies (US$100,000). The use of the contingencies required obtaining a letter of no-objection from the World Bank prior to their use.

Implementation

The allocation of the grant indicates that it would cover 70% of the contract amount for civil works and 100% of supervision consultants (Appendix 1). These proportions are the same as those applied to the World Bank loan funds. All contracts were procured on the basis of national competitive bidding using a post-qualification process and standard bidding documents as had been previously agreed with the World Bank. The implementation of the civil works commenced approximately three months after the grant signing and contracts for the periodic maintenance of national and provincial roads were mostly awarded in the first quarter of 2009. Two contracts were awarded for national roads with contract periods of 75 days and 120 days, respectively. They were signed in March and May 2009, and completed in December 2009 following their defect liability period. One contract for bridge repair was signed in March 2009 and was completed one month later. Due to movements in exchange rates and the rise in the value of the Australian dollar, it was agreed that these surplus funds could be used to cover periodic road maintenance in 2009/2010 provided that contracts were completed by 30 June 2010, the closing date of the RMP2. Thus, a further contract for the spot rehabilitation of Road 9 was awarded in March 2010 which is scheduled for completion in June 2010. The four contract packages were awarded at a total cost of LAK10,573,805,450. Details of the national roads contracts are provided in Appendix 2.

For the periodic maintenance of provincial roads, a total of 17 contracts were awarded to 15 provinces and to the national capital, Vientiane, covering a total of 362 km. All contracts, except for Vientiane, were for regravelling, while the contract for Vientiane was for rehabilitation. A further contract was awarded for the maintenance of four bridges in Xiengkhoung province. The various contracts were awarded between February and May 2009 with civil works construction duration of between 2 and 5 months, and were all successfully completed as scheduled. Due to grant savings, a further 3 contract packages were tendered in 2009/2010, covering 72.5 km in three provinces. At the time of the ICR, two of these contracts were ongoing with scheduled completion prior to the grant closing date. Details of the contracts for the periodic maintenance of provincial roads are given in Appendix 3.
  
1.2	Evaluation Objectives and Questions

The stated objectives of the Independent Completion Report (ICR) were to:

(i)	Participate, on behalf of the AusAID, as a full team member in the joint World Bank/Sida/Japan Implementation Completion Mission including site visits and meetings with the MPWT and other organizations, as appropriate.

(ii)	In addition to contributing to the World Bank completion evaluation of the RMP2, report separately to the AusAID on the effectiveness of the RMP2 aid modality with a particular focus on: 

-	Did the MPWT show effective leadership of the RMP2?
-	How effective was the single comprehensive program and budget framework in achieving the RMP2’s project design objectives?
-	Did the planned processes for coordination and harmonisation of donor procedures for reporting, budgeting, financial management and procurement work well?
-	Did efforts to increase the use of local systems for program design and implementation, financial management, monitoring and evaluation achieve good development impacts?

(iii)	Evaluate key achievements of project objectives in each component against milestones outlined in the Lao PDR program performance assessment framework.

(iv)	Based on Mission findings, prepare recommendations on key priority areas for future potential Australian support in the Lao PDR transport sector. 

These objectives were achieved by seeking the perspectives of relevant stakeholders concerning the program’s relevance; effectiveness; efficiency; impact; sustainability; monitoring and evaluation; gender equality; and analysis and learning. These dimensions of performance were drawn from AusAID’s ICR criteria which are based on the standard development assistance committee evaluation criteria.

It should be noted that since the AusAID contribution to the RMP2 provided only a small portion towards the overall program, of approximately 3.4% of the total cost, and that the timing of the contribution was in the last year of the 5-year program, it is not possible to separate the outcomes of the AusAID assistance from the impacts of the overall program which was supported by a multiplicity of donors. Indeed, the RMP2 was the continuation of the earlier successful RMP1 and the effectiveness of the program needs to view the overall program holistically rather than as a number of individual components supported by different donors. 

1.3	Evaluation Scope and Methods

In undertaking the evaluation, meetings were held with several donor organisations who had a role in the maintenance program. The persons met included the Minister of MPWT and heads of the various MPWT departments and divisions that were associated with the project. Field visits were undertaken to the provinces of Khammouane, Savannakhet, Saravan and Champasak in the south of Lao PDR to observe samples of works ongoing and completed under earlier years of the project. The second objective of the field visits was to discuss issues of relevance with provincial and district staff who were responsible for identifying annual works programs and implementing periodic and routine road maintenance programs. A meeting was also held with the Ministry of Finance to discuss issues concerning the operation of the RMF and use of other development revenue sources to support road maintenance requirements in the future. In addition, meetings and discussions were held with the various donors who had contributed to the RMP2 including the World Bank, Sida, and ADB. Finally, meetings were also held with the AusAID officers who were responsible for oversight of the AusAID-funded portion of the RMP2.

At the conclusion of the ICR Mission, an Aide Memoire was prepared by the mission members under the leadership of the World Bank and it was discussed at a wrap-up meeting that included the key MPWT officials. Broad agreement was reached on the Mission’s findings and conclusions. A copy of the Aide Memoire is attached in Appendix 5.

1.4	Evaluation Team

The evaluation team comprised Charles Melhuish, an independent Transport Economist and Policy Specialist. The evaluation is based on a field mission to Lao PDR between 1 to 12 March 2010 during which time, meetings were held with the MPWT and other relevant government agencies. The ICR Mission was undertaken in conjunction with the World Bank’s ICR Mission and Sida’s end of project Mission. Charles Melhuish has had no previous involvement with the RMP2. The World Bank and Sida team members comprised staff who were associated with implementation of the program for the past several years. While this report has benefitted from discussions and dialogue with staff of the co-funding institutions and the government, its content is wholly the responsibility of the author, Charles Melhuish. 

2.	Evaluation Findings

This section reports the ICR findings of the AusAID grant within the overall context of the RMP2 . Given the relatively small size of the grant to the overall program and given that it only covered civil works in the final year of the RMP2, it is not considered realistic to evaluate the performance of the grant in isolation of the program. The program was the second phase of a 9-year program to introduce and institutionalise road maintenance practice in the MPWT. The RMP2 was designed to build on the success of the RMP1 and was undertaken as a partnership between the government and several donors led by the World Bank. The evaluation was against the five Development Assistance Committee (DAC) criteria comprising relevance, effectiveness, efficiency, impact and sustainability and including AusAID’s additional three criteria of gender equality, monitoring and evaluation, and analysis and learning. Since the results framework of the RMP2 did not contain gender or social development issues, the ICR has not evaluated this criteria. 

2.1	Relevance

Relevance is concerned with the alignment of the project objectives with priorities of the Government of Lao PDR and the AusAID, and the extent to which these objectives addressed a recognised need. The ICR assessed the relevance of the RMP2 program as good quality (5/6).

Meeting Government Priorities

The rehabilitation of the road network has been a high priority of the government since the early 1980s and the vision for the road sector was to make the road network passable all year round between the capital, the provinces and the districts in order to support production, distribution, provision of services, national integration and security. A high proportion of government’s investment had been in road rehabilitation in the two decades preceding the approval of the “Strategic Directions for the Development of the Road Sector” which was adopted by the MPWT in 2000. This new strategy recognised the need to focus on sustainable preservation of road assets through the development and implementation of financing and road management systems. It was under this strategy that the RMP1 was initiated in 2000 and continued under the RMP2. 

The RMP was developed as a 9-year two-phase program. The first phase, the RMP1, succeeded in (i) initiating the new road financing system, (ii) developing an appropriate road management system to help budget and prioritize the maintenance needs of the road assets, (iii) initiating a local road management system, (iv) introducing control of heavy vehicle overloading, and (v) implementing maintenance on substantial portions of the road network. While the RMP2 was designed to build on the success of the RMP1 by raising cost recovery-based financing closer to a sustainable level, enhance the road management systems and extend them to cover the whole country, and expand the coverage of heavy vehicle loading control. The two phases were required to introduce the new systems and approaches to managing road assets and then mainstream and institutionalise them into the MPWT.

Under the RMP1 and the RMP2, the MPWT has achieved substantial success in developing the road asset management systems and processes that are required to manage and maintain road assets efficiently and effectively. However, despite the success, the process still requires additional support and it is envisaged that it will take another 10 years before the systems that have been developed will reach its sustainability. This will only occur when sufficient financial resources are available, supported by well informed institutions and personnel who will be able to plan and manage the use of resources adequately. Within the context of the road sector, the RMP was highly relevant to the needs and requirements of the sector. After considerable investment in rehabilitating the network, it was important that a system was established that would maintain the assets and keep them in good useable condition so that the benefits of the road network continue to support economic and social development. 

Funding an Important Resource Gap

In 2008, there were insufficient financial resources to continue the periodic and routine maintenance programs that had been successfully implemented under the RMP2.  With the acknowledged funding gap, the government requested AusAID to participate in the RMP2 to help fill the financial resource gap for another year until the envisaged completion date of RMP at the end of 2009 and continued support under a follow-on assistance package was approved.  
 
Under the circumstances prevailing at the time, the participation of AusAID in the RMP2 helped fill a critical gap in resources due to the fact that the World Bank/IDA resources had been utilised faster than expected and were also smaller than originally anticipated due to a 35% reduction in the availability of overall IDA allocations. Within this context, the AusAID resources were highly relevant as the demand for periodic and routine road maintenance was urgently required to keep roads in good condition and continue the gains made in institutional strengthening of sector institutions and capacity building.

Relevance to Country and Subregional Assistance Strategies

The involvement of AusAID in the RMP2 was both relevant and in line with the country’s assistance strategy as well as with Australia’s strategy to promote subregional integration and cooperation. A major pillar of the country assistance strategy is a focus on rural development to help reduce rural poverty. Under this theme, a strategic objective is to provide greater access to rural infrastructure, including support for roads, to provide improved access to social infrastructure such as schools, hospitals and markets. The RMP2 supported the maintenance of roads in rural areas including those at the national, provincial and district levels. Moreover, the program resources incorporated support for capacity building and strengthening of road maintenance institutions, including those at provincial and district levels, which will help build the capability of local road administrators to better identify maintenance needs and prioritization of resource allocations in the future.

At the subregional level, one of the identified subprojects concerned the periodic maintenance of Road 9. This important national road links Thailand with Vietnam which was originally rehabilitated by ADB and Japan resources to provide the first subregional road link designed to support trade and commercial development between the three countries in the GMS. A portion of Road 9 had been badly deteriorated due to a combination of heavy truck flows and poor construction quality, which has severely affected vehicle operating costs and eroded nascent subregional trade. The periodic maintenance program has restored a substantial length of the damaged portion of the road which continues to generate increased international passenger and freight traffic.

Support for Country Systems
  
The RMP2 was wholly implemented using country systems. The procurement and disbursement systems used by the MPWT have been promoted under a number of externally-funded projects especially those of the World Bank and ADB. In this respect, the RMP2 was designed to be fully implemented using existing internal resources and administrative structures, and no special unit was created to manage the project. Thus, RMP2 fully demonstrated relevant use of partner government systems and provided a platform to build good institutional practices for road sector management.

2.2	Effectiveness

A major test of effectiveness is to determine whether the activity has achieved its intended objectives. The ICR assessed the relevance of the RMP2 program as adequate (4/6).

A measure of the effectiveness of the program is to assess how well it has achieved its objectives. Both the project appraisal documents of the World Bank and AusAID set out the following objectives: 

(i)	attains improvement in the condition of national and local road networks;
(ii)	mobilizes financial resources for road maintenance in relation to needs;
(iii)	enhances the capacity for dealing with road maintenance activities at the central, provincial and local levels; and
(iv)	reduces the incidence of overloading of heavy vehicles. 

Condition of the Road Network

During appraisal, a results framework and monitoring indicators were established for key components of the project. The MPWT, in producing the government’s ICR, has prepared the results of the monitoring of the program over the 5 years of its implementation. The results of this exercise are provided in Appendix 6 which indicate that the monitoring indicators have exceeded the parameters established during appraisal of the RMP2. At appraisal, the key baseline figures were 73% of national roads and 23% of provincial roads were in excellent to good condition and by the end of the program, they were to achieve 80% and 30%, respectively. The national roads have exceeded their benchmark by attaining 83% at the end of the 5-year period and provincial roads reached 51%. Overall, the program was expected to support 2,000 km of periodic maintenance of national roads and the same length of provincial roads. The actual achievements were 1,973 km of national and 2,313 km of provincial roads. The slight underachievement for the national total and overachievement of provincial roads was due to the greater focus placed on provincial roads in the latter half of the program due to the acute need to improve provincial road infrastructure. 

Mobilization of Financing for Road Maintenance

A key component of the RMP was the mobilization of resources to support and sustain road maintenance. The RMF was a core component created under the RMP which largely raised revenues through a fuel levy by LAK34 billion in 2004 which was the baseline indicator. Under the program, the key indicator was to increase this figure to at least LAK100 billion a year by September 2008. However, due to continued annual increases in the levy of LAK50 per litre, the income generated by the RMF reached LAK115 billion in 2007, a year ahead of the expected schedule. Indeed, in  2009/2010, the income is expected to reach LAK264 billion. The steep increase in RMF income is not only due to increases in the fuel levy but is also attributable to the increase in the vehicle fleet and motorization as more vehicles are travelling further. While there are still issues concerning shortages of funding for financing road maintenance, the benchmark indicators for the program were attained ahead of the prescribed schedule.

Capacity Enhancement

The third objective of the program was to enhance the capacity to deal with road maintenance activities at the central, provincial and local road networks. For this activity, there were no benchmarks to indicate or measure the program’s success. Capacity building under the program focussed on three components as indicated in previously in the project description. 

The first component dealt with road management and financing and as noted above, the financing component had a prescribed benchmark. The road management aspects concentrated upon the establishment and operation of appropriate procedures at central and provincial levels for the planning, budgeting and implementing of maintenance works programs. A technical assistance grant helped the MPWT enhance the road management systems by developing a tool for strategic planning and identification of 3-year rolling road maintenance needs. Part of the assistance aimed to integrate the two systems under the unified road management framework (URMF). A review of the system indicates that the principles of the planning process are well known and the system is managed by the Public Works Transport Institute (PTI). The only system drawback is that road condition data is only collected once every 2 years and traffic data once every 5 years. An effective computerised road planning tool requires accurate data to provide meaningful outputs that can be used to prepare road maintenance plans. Data collection needs to be undertaken more frequently so that information that is used in the analysis represents the actual field conditions. 

In a developing economy, such as Lao PDR, traffic is growing at a fast pace and the types of vehicles using the network are also changing rapidly. Due to the fragile nature of a high proportion of the road network conditions, they are also susceptible to change quickly and often, differences between conditions pertaining in the wet and dry seasons exist. Several senior policymakers exhibited some doubts on the effectiveness of the URMF because it frequently provides results that do not match actual conditions. It is thus important that more attention is given to data collection so that the road management tools use information that represents actual conditions of roads and produces outputs that are realistic.

While the planning system still requires support to improve its usefulness and knowledge by users at the national and local levels, significant gains have already been made to build the capability and capacity of maintenance planning at all tiers in the MPWT. Road maintenance planners and policymakers at the national and local levels are knowledgeable about the RMS and the provincial road management system but substantive capacity building is required to extend and deepen the knowledge of practitioners. At the highest levels, this will require skills to use the output to demonstrate “what if” scenarios to illustrate to policymakers the impact of different funding availability on network conditions or different investment strategies. At the lower ranks, greater knowledge is required on the technical aspects of road planning and budgeting and how to use the planning tools to maximise and optimise use of resources.

A second component concerns the development of a vehicle registration and driver licensing system. Despite several problem areas, the majority of which are due to poor information technology issues rather than poor system development, the envisaged system has been developed and pilot-tested. The system will begin its operation in May 2010 at the Thong Pong Agency serving Vientiane. Extension to other areas will take time due to cost and capacity of information and technology infrastructure and support in many provinces.

A subcomponent of this capacity development is the focus on road safety audit. Under the RMP2, some road safety audit training was undertaken, particularly on roads that had specific resources earmarked for this purpose. While key staff are knowledgeable on road safety audit techniques, there are insufficient financial resources budgeted to mainstream this activity and to conduct it regularly. The road safety fund has been created using 5% of the RMF income but as yet, it is not operational. Priority needs to be given to making it operational and allocating a portion of the resources for road safety audit purposes.

The third component on capacity enhancement focussed on building the capacity of the MPWT. Under the RMP2, a total of 61 programs and courses were conducted involving 2,424 staff at the central and provincial level over the 5-year period. The training covered a wide range of skills and needs from management and administration courses, technical courses such as HDM-4, materials testing and environmental and social management, external master degree courses, training for new recruits to the ministry, public administration, English language training, report writing and computer training. These courses have raised the needed skills of the MPWT in line with technical and administrative requirements. However, there are several capacity building components that were not taken up, particularly those relating to procurement, project, and contract management. These latter skills are in short supply and are key to the implementation performance of the MPWT in the long term particularly since the emphasis is on devolution to the provinces and priority should be given to developing these skills in the future. 

Heavy Vehicle Load Control

The fourth objective of the RMP2 was to reduce the overloading of heavy trucks. The official statistics demonstrate that there has been significant success in reducing truck overloading from a high of 90% in 2001 to 27% in 2004, and 12.7% in 2009. The MPWT’s draft ICR report suggests that the official statistics might not corroborate. In actual practice, weighbridge stations operate only during the day time and are weighing fewer trucks than in earlier years. However, from the information made available and discussions with the Department of Transport (DOT), the data obtained from the weighbridges looks rather circumspect. The 2009 data indicate relatively high levels of overloading in the northern part of the country with over 50% in Vientiane, and between 25% and 40% in the provinces to the north of Vientiane. This contrasts markedly with the weighbridge stations in the southern provinces which record less than 1% of trucks as overloaded. This is very surprising given the load capacity of many modern trucks and the relatively low axle load limits of 9.1 tons per axle. While it is possible that the southern provinces have much better enforcement control, this is not corroborated by visual inspection of trucks and trailers, and an actual demonstration of weigh station operations undertaken during the field visit. The DOT indicated that the discrepancies in data are due to malfunctioned equipment in the southern provinces. While the equipment observed in the field was in poor condition and probably did not weigh correctly, it is likely considered that overloading control system is weakly enforced and the management systems used need to be strengthened to reflect better governance controls. It is also noted that on certain Greater Mekong Subregion (GMS) roads, the permitted axle load has increased from 9.1 tons per axle to the GMS standard of 11 tons per axle, giving rise to a dual axle weight on some routes that could lead to misclassification or different interpretation as to whether a vehicle is overloaded.

Program Management

The terms of reference was specifically tasked to confirm whether the MPWT, the executing agency, demonstrated effective leadership in the RMP2. Based on a review of documents and processes, site inspections of sample maintenance works, and discussions with the MPWT staff in various departments and divisions of the organizations, it is considered that the MPWT have demonstrated a high degree of leadership in implementing the program and achieving the majority of its intended objectives. The program did not involve a separate project management unit but used the existing MPWT administrative structure and processes to implement the program. Given that this complex maintenance program involved many small contracts, scattered throughout the country, and was implemented through a decentralised system via provincial public works departments (DPWT) required skilled leadership by the MPWT to make sure that the program will be implemented on time and within budget. In this respect, and with the knowledge that the skills base and capacity of the organisation is still developing, the ICR considers that the MPWT demonstrated remarkable leadership.
 
Supporting this success was the adoption of the single comprehensive road maintenance plan and accompanying budget framework to identify prioritised works, the unified bidding process to select and award contracts, and appropriate disbursement procedures. While there remain some issues with this process, the procedures generally worked well and were efficiently managed. A major positive aspect of the program was the partnership between the donors which enabled common documents to be used in the bidding process which, in turn, significantly eased the implementation of maintenance programs regardless of the source of funding. 

2.3	Efficiency

Efficiency is concerned with implementation performance against time and budget parameters and cost effectiveness. The ICR assessed the efficiency of RMP2 as good (5/6).

Partnership Approach

A good feature of the RMP2 was a large number of donors supported the program. The partnership approach demonstrates to government that all donors have a common aim to build a maintenance system that is effective by adopting good practices, using appropriate knowledge and techniques, and delivering a high level of benefits to its users and beneficiaries. The adopted approach also ensured that the skills that had been imparted in earlier assistance at the various road hierarchies were brought together under one umbrella and further enhanced. Thus, previous skills and knowledge were not lost and wasted but were incorporated within the new developed systems.

The availability of local funds is an often quoted constraint to sustainable maintenance. The creation and expansion of the RMF has been (and will continue to be) the major pillar of the RMP2 success. While the RMF does not yet generate sufficient funds to implement a sustainable maintenance program, it provides vital support to donor contributions and will in time (currently expected by 2015) provide sufficient resources to cover the needs of the national road network. The RMF, therefore, provides a useful support mechanism to donor assistance for road maintenance programs and provides comfort that the government takes road maintenance seriously by having a facility that will be able to provide the resources required for sustainable road maintenance in due course.

Capacity Building

The RMP2 provided considerable support to building the capacity needed for the local institutions to undertake road maintenance activities. While the emphasis was on building capability at both the national and provincial levels, it is recognised that reaching an adequate level will take significantly longer. Nevertheless, under the RMP2, good progress was made with building capacity at the national and provincial levels, particularly with the development of the road maintenance management system. Systems are now in place that are capable of prioritising 3-year rolling programs and identifying annual works programs and the future focus needs to build upon a greater understanding of the system and its potential applications as well as improved data collection and analysis. 

Generally, progress has been made in institutional strengthening and human resources development during the implementation of the RMP2. Several MPWT and DPWT staff have attended trainings, mostly on technical aspects. Assessment of the human resources capacity development has been conducted which indicated that capability has improved in many areas. Based on this analysis, a 5-year human resource plan has been formulated which will soon be submitted to the Public Administration Civil Servant Authority who will then decide whether the recommended increase in staffing complement is justifiable. The MPWT’s Department of Planning is currently preparing a 10-year strategic human resources development plan covering 2011–2020. As for the organizational reform, a revised mandate and role for each department in the MPWT and the DPWT has been conducted with the view to increase the effectiveness and efficiency of the units and promoting greater transparency in managing the sector.

While increased capacity is being achieved, the MPWT now needs to pay more attention to staffing skills when rotating staff to different and/or new assignments. Often, knowledge skills are thinly spread and staffing rotation can leave significant gaps in knowledge that are difficult to fill. It would be prudent to make sure that sufficient back-up skills are available prior to rotating staff.. Efforts need to be directed at broadening the knowledge base to make sure that work units have sufficient skills among staff.

Cost Effectiveness

An important aspect of road sector investment is to ensure that economic and social returns are gained by the country and society from the investment. In late 2009, the World Bank carried out an ex-post economic analysis of a number of national road contracts that were implemented in the first year of the RMP2 (2004–2005). 

The economic benefits were derived from savings in road user costs, including savings in fuel, vehicle maintenance and travel time, calculated over the life cycle period of road maintenance and expressed in present value terms. They were computed using the Highway Development and Management Model (HDM-4) adapted and calibrated to the Lao PDR network. The ex-post economic evaluation used the same methodology that was adopted during appraisal, but using actual costs of road maintenance and the traffic growth on the project roads from 2004 and 2008. The measured traffic growth over the 4 years was 6.7% per year which was less than the estimated 8.6% growth at appraisal. The results of the analysis demonstrate that the investment in national road periodic maintenance has produced high economic returns which are broadly in line with original expectations (Appendix 7). For the 27 road sections, totalling 635 km, the ex-post economic internal rate of return is 35.2% which compares favourably with the 39.1% estimated at appraisal.

While no ex-post economic analysis has been undertaken of provincial roads, it is likely that they would also demonstrate high economic returns. At appraisal, the economic rate of return of investments in provincial roads was higher than that on national roads.

2.4	Impact
	
Impact is concerned with significant and lasting changes–both intended and unintended–that have been affected by the project. At the current stage of the project, which has yet to be formally completed, it is difficult to ascertain the actual impact that has been generated by the RMP2. This is only likely to be measured several years after the program is completed.

Notwithstanding the above, the RMP2 has fostered some significant achievements, most notably:

(i)	It has highlighted the importance and need to focus on road maintenance.
(ii)	A critical need is the mobilization of resources for maintenance and the creation of the RMF has facilitated this requirement.
(iii)	The implementation of the program using the MPWT’s systems and procedures has strengthened the use of country systems.
(iv)	The program has strengthened the promotion and use of safeguard policies.
(v)	It has raised the need to further strengthen fiduciary issues.

The relative merit of each of these achievements is discussed in turn.

Highlighted the Importance of Road Maintenance

The two-phase program over a 9-year period has highlighted the need and importance of addressing road maintenance. The program has established maintenance activity as an important core business of the MPWT and has also established systems and procedures to identify and prioritise needs, plan 3-year rolling programs and prioritise annual works programs. These activities are now mainstreamed in the MPWT business processes and have been replicated at the provincial and district levels. 

The impact of institutionalising road maintenance activity is expected to have a long term benefit to increasing the sustainability of the road network and the delivery of economic and social benefits to road users and beneficiaries. The maintenance of the asset base will provide large returns to the economy and facilitate access to markets and social services such as schools and health centres which are vital for development in the countryside and the attainment of the Millennium Development Goals, a core development objective. 

Mobilized Resources for Maintenance

Perhaps the greatest impact of the RMP is that it has created a mechanism to raise revenues from road users to undertake periodic and routine road maintenance. The RMF has had a significant impact on revenue generation and in its first year the income totalled LAK16 billion (US$1.9 million). In 2008/2009, the RMF raised LAK207 billion (US$24.4 million) and it is estimated that by 2015, income will reach LAK605 billion (US$71.2 billion), which is a significant milestone. The RMF resources provide counterpart funding requirements for many of the donor-assisted road maintenance programs and being an off-budget fund the resources are not affected by government budgets and delays in availability. This has also helped with the smooth implementation of the RMP2.

Strengthened Country Systems

With the RMP implemented through country systems, it has supported the development of processes and procedures in the MPWT and the DPWT. The impact of using country processes is likely to have been greater if a separate project implementation unit had been the modality of implementation. The decision to use country systems to implement the RMP has strengthened the MPWT and built capacity in many of its department and divisions. Since road maintenance is a function devolved to the provinces, the RMP2 has also strengthened the local systems in all 17 provinces.

Promoted Use of Safeguard Policies

The MPWT has had considerable experience in implementing road sector projects under various donor safeguard policies. Often, the policies of donors vary and are not entirely consistent with one another. Under the RMP2, significant progress was made to harmonise safeguard requirements in the road sector. In this respect, in addition to the training of central and provincial staff in safeguard requirements, the MPWT prepared an environment and social safeguard operation manual which is intended to be adopted as a MPWT policy document and will be used as a part of the country system process in the future. The manual is currently being pilot-tested as a draft working document and the intention is to finalise it once experience has been gained. 

The impact of the RMP2 has been to raise the awareness of provincial and district staff of the need to review safeguard issues prior to undertaking civil works contracts. Since the RMP2 operated in all 17 provinces, this has provided a broad base for information dissemination in the road sector and the impact has been widespread.

Identified the Need to Strengthen Fiduciary Issues

With the procurement and implementation of road maintenance devolved to the provinces, the need to strengthen financial management and internal audits has been raised. While the implementation of the RMP2 has been undertaken appropriately, the review of financial management processes and procedures has highlighted a number of recommendations to strengthen controls and audit functions. Overall, there are country level risks that need to be handled at the macro scale outside the individual project level. These risks were jointly reviewed by donors and an action plan to address the issues is being implemented. At the MPWT, the major focus will be on reducing implementation risks and improving system control functions. Required measures to strength the control functions include the need for additional training and capacity building at central and provincial levels, strengthen the newly created Department of Inspection’s internal control functions and systems, improve external audit functions of the State Audit Office, improve reporting and monitoring to ensure that payments are linked to physical works, and strictly monitor anomalous activities in the procurement process.

2.5	Sustainability

Sustainability concerns the likelihood that project benefits will endure after funding has ceased. The ICR assessed the sustainability to be adequate (4/6).

Accomplishment and Issues

The RMP2 has accomplished many achievements because it has expanded the coverage of maintenance activities on both the national and local road networks, strengthened the processes and procedures within the institutions responsible for managing the country’s road inventory, and mobilized additional resources under the RMF to more effectively maintain the road assets. 

Despite the success of the RMP2, several issues remain unresolved which could undermine the sustainability of the program. These issues concern the:

(i)	adherence to road specifications and standards, and maintaining good technical quality of civil works;
(ii)	timing of fuel levy increases and attainment of full cost recovery for road maintenance;
(iii)	need for the RMF to operate more transparently by communicating the benefits of road maintenance and use of the funds to road users and policymakers rather than just managing the financial aspects of the fund;
(iv)	resources are insufficient  for data collection and analysis, together with inconsistency in the quality of the data collected and the reliability of the output of the models and analysis generated by the processes; and
(v)	adjustments and changes to the MPWT organizational structure regarding safeguard responsibilities that require additional capacity development.

Technical Sustainability

Over the 5-year program, significant progress has been made on the amount of maintenance activity that has taken place on the national road network. Maintenance of national roads is now moving towards using performance-based 3-year contracts. The MPWT places high priority on maintenance of rehabilitated national roads and it is likely that their sustainability is high. The focus in the future will be to further develop and improve the maintenance contracting system and make it more efficient through the use of more outcome-based indicators, covering longer lengths of road and using longer contracting periods.  However, with respect to the provincial road network, there remain high risks that the sustainability of those roads maintained under the project may not provide adequate access or service to road users and beneficiaries in the future. The quality of maintenance works implemented under the RMP2 is below expected standards in many areas. A review of the sustainability issues of provincial roads has suggested a number of practices that need to be developed to improve performance, namely:

(i)	improve and increase follow-up by central government to undertake independent checks and audit the work that is being undertaken; 
(ii)	increase quality control of the civil works–implement a quality assurance system–to implement measures that will ensure that the works undertaken will last longer;
(iii)	increase funding allocation for provincial (and local) roads;
(iv)	adjust the standard parameters in the provincial road management system to provide a lower road standard that is more affordable;
(v)	reduce the size of the maintainable road network to focus on a core network that is affordable;
(vi)	unmaintainable roads should be properly reconditioned before being reclassified as “maintainable”. The current system encourages provinces to extend the maintainable network without improvement or reconditioning the existing infrastructure; and
(vii)	gravel roads with high traffic volumes, especially heavy trucks, should be given priority for sealing.

These findings indicate that emphasis needs to be placed on addressing key issues relating to sustainability. Perhaps the most sensitive concerns the development of a core network. Despite the substantial increase in resources generated by the RMF, it will be several years before sufficient resources are available to support adequate maintenance for provincial and district roads. Given the current level of funding availability there is a need to prioritise the use of resources to make sure that important assets continue to exist and provide benefits to users and beneficiaries. Identification of a core network would help to prioritise resource allocation and maximise the benefits gained from maintenance activities.

During the ICR, another issue that was highlighted concerns the delayed payments to contractors, causing increases in the cost of maintenance work by raising working capital from local banks. In addition, workers are often not fully remunerated until the contractor is paid and this adversely impacts the rural labour that is engaged to work on road maintenance contracts. 

The condition of the LRN, consisting of district and rural access roads which are largely unpaved, remains a concern particularly since the maintainable portion of the network is less than 30% in most provinces. These roads are largely unpaved and the bulk of the network is classified as unmaintainable. Efforts have focussed on periodic maintenance with resources allocated for regravelling but little effort is applied to routine maintenance particularly the reshaping of the pavement and cleaning of drainage facilities. Financial constraints are significant and maintenance efforts are often restricted to a cycle of regravelling every 3 to 4 years or more. As some roads are maintained, others drop to unmaintainable classification in a cycle of rebuilding the roads. This strategy is certainly unsustainable and efforts need to be applied to break the cycle. As noted above, the resource shortfall is expected to remain for many years and thus a new strategy needs to be adopted that maximises the use of available funding to good effect. At this tier of the network, greater selectivity needs to be applied to maintain a core network that caters to district administrative needs and supports local transport requirements. While it is recognised that additional resources will become available through Nam Theun 2 revenues, it will be necessary to ensure that these resources are used effectively, and not to provide accessibility to small numbers of villages that might only provide intermittent benefits in the short run as this will aggravate the already acute maintenance funding position.

Overall, the sustainability of the road network remains fragile and depends on the continued mobilization of resources through the fuel levy to the RMF. Priority is rightly given to maintaining national roads and this needs to be continued and further developed. However, the sustainability of provincial, district and local roads will require efficient handling of limited resources for many years. Road maintenance policy needs to become increasingly strategic in balancing available resources for improvement, rehabilitation and maintenance across the network. Consideration needs to be given to prioritising support for a core network at the district level and below to ensure adequate accessibility is maintained to key centres of population and economic activity. 

Financial Sustainability

As has been previously highlighted, the RMP2 has been markedly successful in mobilizing additional resources for road maintenance. The original program target of LAK100 billion was met as early as 2006 and by 2009, had more than doubled to LAK207 billion. Revenues are expected to continue to increase through a combination of increases in the fuel levy, the increase in the vehicle fleet and increasing vehicle kilometres driven each year. These increases will help augment the domestic resource availability of road maintenance funding and thus expand the maintenance works undertaken. However, it can be expected that as the RMF revenues increase, the amount of donor financing for maintenance will decrease as donor contributions are channelled into other development efforts and perhaps, other sectors. Thus, pressure on maintenance funding will continue. By 2015, it is forecast that the RMF revenues will reach a level where all national road needs will be met but it will be many years before RMF revenues will be able to cover total maintenance needs of the provincial, district and local roads.

Despite its success, the RMF also has issues that could affect its long term existence. At present, the RMF operates solely as the manager and financial intermediary allocating its funds for periodic and routine maintenance in accordance with its mandate. It has no involvement with the planning process and does not measure whether its expenditures and disbursements have positive impacts and outcomes. In addition, decisions to commit the RMF on various externally-assisted projects are often made without involving the RMF administrators and there is little planning on the availability and use of funds. A further issue is that the RMF are only committed to civil works and are not available for other priority tasks associated with road maintenance. Thus, data collection, which is a key input to the road asset management process, is not supported by the RMF. Consideration for using RMF to collect and analyse the information required to identify maintenance priorities and allocate funding needs to be reviewed since this would improved the outcome of the road maintenance planning process and lead to more effective use of resources.

A further issue concerns the limited information that is available to policymakers and road users on the use of the funds. While the RMF manages its resources well in accordance with government financial practice, and publishes its results annually in the media including local newspapers, it does not undertake any analysis on the impact and effective use of its funds. As a result, policymakers and road users are not aware of what RMF accomplishes and achieves with its funds. This is especially important since the road condition in many areas do not display significant and sustained improvement. Given that the RMF is an off-budget fund, it is necessary for policymakers and road users are informed of its accomplishments including assurance that the funds were put to good use. Both these actions will help remedy resistance to future increases in the fuel levy and enlighten the public on the major achievements that have been made by the RMF. In the long term, the independence of the RMF is considered essential to ensure that dedicated resources remain available for the preservation of road assets. 

Institutional Sustainability

The institutional strengthening and capacity building under the RMP2 is considered to be sustainable but several of the institutions involved will require continued support and capacity building over the next several years particularly those at the provincial and district levels. A significant contributor to building institutional sustainability was the fact that the program was implemented using the existing MPWT organizational structure and institutional arrangements and did not involve using a separate project implementation unit. This resulted in greater focus on the performance of individual units in the organization performing their functions and this assisted in building their capacity and capability. The RMP received strong support from the leadership of the MPWT and this has helped to both make the program a success as well as institutionalise road maintenance as a necessary and important part of road sector management. The program was also implemented through a decentralised approach with the individual provinces responsible for procurement and implementation of the civil works in their jurisdiction. This has helped create better accountability for the works undertaken and eventually will improve quality outcomes. Given the general weaknesses in provincial institutions, implementation was a challenge but has resulted in greater responsibility, knowledge and learning at the local level. It is recognised that additional efforts will be required to build capable and knowledgeable provincial institutions but the RMP has provided the platform for future progress. 

2.6	Gender Equality

The RMP2 did not incorporate a gender component. As such, the ICR has not assessed gender equality associated with the project.

2.7	Monitoring and Evaluation

Monitoring and evaluation (M&E) concerns the extent to which adequate arrangements were put in place to ensure accountability, enhance decision making, and promote learning. The ICR rated the quality of the M&E arrangements for RMP2 as adequate (4/6).

As noted earlier, the RMP2 incorporated a results framework and monitoring program which established a set of indicators. With the program being implemented by the MPWT the role of monitoring was the responsibility of the Project Monitoring Division in the Department of Roads. The semi-annual progress reports included the completion of the project performance indicators which facilitated the recognition of progress vis-a-vis the benchmark baseline indicators. While this framework identified the expected physical results of the program, its focus was entirely on physical road maintenance achievements and their costs. Its major weakness was that it did not cover potential social impacts such as gender, HIV/AIDS and other social development and cross-cutting issues.

As discussed in the effectiveness section, the RMP2 is considered to have fully met its primary objectives. The results indicators have been met or exceeded the target and the accomplishment over the 5-year program has slightly exceeded expectations.

As noted below in the analysis and learning section, the inclusion of annual technical audits facilitated the review and monitoring of the physical aspects of the RMP2 which identified a number of technical aspects which, if implemented, could have improved the quality of the overall accomplishment. Given the institutional weaknesses of both the public sector institutions involved with road maintenance and the local road construction sector, the adoption of technical audits was innovative. While the outputs of the technical audits appear reasonable and realistic, it is not possible, however, to ascertain their effectiveness. Given the common findings from one year to the next, it does not appear that serious consideration was given to the findings and recommendations to rectify the problems and issues raised. The process would have been improved if there had been a mechanism to formerly assess and adopt the recommendations of the technical audit findings.

The financial monitoring of the RMP2 provided the required information on time. Audits were conducted by private firms hired by the MPWT and the audit opinions did not raise any accountability issues. The RMP2 maintained an appropriate financial management system over the duration of the program and the identified shortcomings were related to systematic issues concerning weaknesses in internal controls for certifying civil works progress. While works were certified by the engineering teams responsible for procurement and contract management, there was no independent check to verify physical works progress against payments made.

2.8	Analysis and Learning

Analysis and learning concerns the extent to which relevant technical analysis and continuous learning under the RMP2 was carried out. The ICR rated the RMP2 adequate (4/6) in this regard.

Building on Previous Success

A major strength of the RMP2 is that it was built on the results and successes of the RMP1. The initial phase of the RMP set trigger criteria for progressing to the second phase which were designed to demonstrate that the government was committed to taking road maintenance seriously and that the program was being successfully implemented. The criteria required that a minimum amount of funding be allocated for road maintenance to cover at least 35% of total needs; that the funds were allocated for priority stretches of the road network identified using the newly developed RMS; and that the implementation of the program was satisfactory with at least 60% of the loan disbursed. In combination, these criteria ensured that progress was being attained on Phase 1 and that would help ensure the success of a second phase.

Improvements to the Planning and Budgeting Process

During implementation, there was continuous improvement of the decision-support planning tools that were earlier developed to select and prioritise network maintenance needs and preparation of a 3-year road maintenance plan with annual work plans. The tools were ultimately combined into the URMF which brings together the data and information used for identifying priorities for both the national and provincial roads into one database such that optimised allocations can be made for their maintenance. The use of URMF is expected to provide improved analysis of road conditions and the economic benefits associated with different types of maintenance options and result in improved identification of maintenance requirements. 

Use of Technical Audits

An important analysis and learning component was the inclusion of the technical audit component which was designed to enhance quality of road maintenance works as the program progressed. The technical audits were required to be conducted annually by an independent auditor who was engaged to assess the project components and identify shortcomings or areas that would enhance improvement. This included comment on the physical output and quality, technical issues and the effectiveness of the MPWT and donor procedures. During implementation, a total of four technical audits were conducted which identified a number of areas where improvements could be made, particularly to increasing the quality of road maintenance works. The key recommendations were:

(i)	Roads need to be maintained to the “as-built” standard and not to a lower standard as was frequently observed.
(ii)	The primary focus for road maintenance should be to ensure the quality of the works are controlled and meet the requirements of the contract.
(iii)	The project and contract management capacity of the provincial authorities needs to be strengthened.
(iv)	Bidding documents and especially the specifications must be bilingual (English and Lao).
(v)	Bidding should be competitive and unit rates based on actual cost of local materials.
(vi)	Department of Roads staff should be involved in monitoring procurement and execution of the works.
(vii)	A central registration system for contractors should be reinstated and strengthened.
(viii)	Ensure availability of sustainable funds to cover future maintenance activities.

While the technical audits were a useful addition to project monitoring, analysis and learning, there is little evidence that the MPWT used the feedback to redress the problems since the key findings noted above were recurrent themes that were mentioned in successive audit reports. There appeared to be no mechanism to take up the findings of the technical audits which would have improved the usefulness of the exercise.

Lessons in Environment and Social Safeguards

In addition, the RMP2 undertook a review of environmental and social safeguard issues based on previous lessons and experience in this area and prepared an environmental and social operations manual. This manual provides a practical framework for identification and mitigation of environmental and social impacts in the road sector. The manual describes the safeguard screening criteria for road works, the environmental and social guidelines for impacts assessment and mitigation measures as well as monitoring aspects. The social aspects cover social assessment, ethnic group development plan, resettlement plan, gender and HIV/AIDS. The manual will provide a useful support tool for future road sector projects.

3.	Evaluation Criteria Ratings

Overall, the ICR has assessed the RMP2  as a successful program for it introduced road maintenance activities into the MPWT and institutionalised good practices. While there remain several problems with current practices, particularly with respect to the quality of the works at the local level, the RMP2 has introduced new methods and processes necessary to build adequate quality. The ICR has not evaluated the AusAID assistance as a separate component since this is not possible given the fact that AusAID resources contributed a small proportion (3.4%) of the overall program focusing on civil works. A key objective of the program was to build the needed capacity to identify, prioritise and implement road maintenance requirements while also providing resources to implement civil works. 

This successful program was built on a partnership between the government and several donors working together. The road maintenance requirements of Lao PDR remain very large and at its current level of development, the country faces severe shortage of financial resources to allocate for maintenance. The RMP has been instrumental in mobilizing resources from road users through a fuel levy and after almost 10 years of operation. it currently raises LAK264 million a year, about US$31 million, sufficient to cover only 30% of road maintenance requirements. It is estimated that with continued increases in the fuel levy, there will be sufficient resources for the maintenance of all national and provincial roads until 2015 but it will take many years before sufficient resources will be available to cover the total road network.

While the contribution of AusAID was small and came at the end of the RMP2, it was nevertheless important in the context of completing the program. The involvement helped cover a critical portion of the financing gap for national and provincial road maintenance in the 2008/2009 program, a year marked by a severe worldwide economic downturn. It, thus, helped sustain good maintenance practices over this period and complete the RMP2 program as it was originally envisaged.

Evaluation criteria: Rating (1-6)
Relevance—5
Effectiveness—4
Efficiency—5
Sustainability—4
Gender equality—N/A
Monitoring and evaluation—4
Analysis and learning—4

Rating scale:
Satisfactory
6—Very high quality
5—Good quality
4—Adequate quality
Less than satisfactory
3—Less than adequate quality
2—Poor quality
1—Very poor quality

4.	Conclusion and Recommendations

Overall, the ICR assessed that the RMP2 performed well and achieved its primary objectives. The RPM2 was a two-step initiative covering 9 years. The RMP1 introduced the RMF concept which has been tremendously successful in raising resources from road users through a fuel levy for road maintenance. The RMF has augmented donor assistance for road maintenance and will, in time, if the fuel levy continues to be increased, mobilise sufficient resources to cover the entire road maintenance needs. By 2015, revenues are expected to be sufficient for the national and provincial road networks.  Beyond 2025, resources will be sufficient to cover all roads. The RMP1 also introduced the concept of the RMS to identify road maintenance requirements, prioritise them and prepare budgets for annual works programs. The RMP2 built on this success and expanded the system nationwide scaling up operations to cover all national, provincial and local roads. During the RMP2, emphasis was also placed on building the capacity of the institutions at the central and provincial levels.

Despite the success of the RMP, several areas need to be further developed and improved for road sector management. A key concern is the quality of work that results from the process, particularly for provincial and district roads. The maintenance activities do not always result in acceptable quality and emphasis needs to be placed on adopting measures that will improve the quality. In addressing this issue, greater focus needs to be placed on improving the planning system by making sure that road data and information used for planning inputs is relevant and up-to-date. Data collection on road conditions and traffic needs to be done more regularly and this will require allocation of additional financial resources. Several of the specifications for work requirements need to be better identified to ensure that the roads are maintained to the “as-built” standard and materials of the expected quality are used. Costs used for estimating requirements need to be specific to location and not “broad-based” provincial costs. Contract documents should be more specific to facilitate oversight and ensure that the works are completed according to specification. 

For nationally-bidded projects, the contract documents should be in the local language to enable contractors to understand. Efforts need to be taken to encourage local contractors to implement quality assurance systems that will better ensure the quality of civil works construction. An independent audit check should be undertaken to cross-check the payments made relate to completed works. At the province, the oversight function of contract managers should be strengthened to both ensure better quality of works and accountability of the supervision engineer. All of the above recommendations can be achieved by reviewing job skill requirements, modifying the technical portions of contract documents, and implementation of further training and capacity building programs to develop skills of public sector staff and contractors’ staff.

On the national roads, the MPWT has introduced 3-year performance-based contracts for periodic maintenance. The performance of these contracts should be reviewed to identify any shortcomings. Areas where improvements can be expected include moving towards contracts that are more outcome-based rather than output-based and extending the contract period to reduce fixed costs.

The involvement of the AusAID in the RMP was relatively small and occurred at the end of the RMP2. The resource funding was vital at a time when the RMP2 was short of funds for periodic road maintenance. While the funds were disbursed very efficiently and effectively as systems were already developed and the absorptive capacity of the sector is high, AusAID was not able to provide assistance to build capacity or strengthen the institutions due to the timing. To achieve these development objectives, it is necessary to participate in project and program appraisals at an early stage where donors can discuss sector needs and how to address them under the new project. Indeed, for the multilateral donors participation by AusAID is often vitally important since the funds are provided as a grant and can often be more effectively used to undertake important capacity building and institutional strengthening initiatives which are areas that governments are often reluctant to use loan resources.

The RMP2 demonstrated excellent partnership between the government agency, MPWT and several donors working together to achieve better road maintenance. The system worked smoothly and was facilitated by the use of standard bidding documents and use of country systems. The fact that the program was implemented through the MPWT structures and administrative procedures rather than a separate implementation unit resulted in significant improvements in the MPWT capacity and building a maintenance system and procedures that  work. While there are many further improvements that are required, as noted above, the RMP2 has provided a platform on which to base future improvements and development. Perhaps most important of all is that the RMP has not only introduced road maintenance to the country but has institutionalised it into MPWT’s practices and processes. This has been a remarkable achievement given that road maintenance is a devolved activity under the responsibility of provincial authorities.  

One area that was not incorporated in the RMP concerns the gender and social impacts of road maintenance programs. Given the diverse impact between road maintenance on a national compared with a provincial or local road it would be useful to identify the impacts of such investments on communities and gender. 

Possible Areas for AusAID Future Support

The terms of reference for the ICR require identification of areas where potential future support could be provided. It should be noted that this list is not comprehensive since it is only based on the information that was obtained when reviewing the RMP.

Road maintenance will be a continuing area of concern since there remains a significant resource gap particularly for provincial, district and local roads. This funding gap will be exacerbated by the withdrawal of Sida which has supported this sector for several years. There will also be few resources to support the technical and capacity development of local institutions which is also urgently required. These local road hierarchies have a significant impact on the success of various poverty-oriented programs as road access is a vital requirement to complement development in various sectors. Even with modest support, there will be insufficient resources to meet full requirements and the government should consider prioritising its limited resources by developing “core” road networks and focusing the bulk of resources on those roads rather than maintaining a large length of roads that is currently unaffordable. Adoption of such a policy will facilitate resource allocation to priority areas with important growth potential and ensure that good access is maintained to district centres and other locations with essential infrastructure such as schools and health centres. 

A second area where external resources can provide good impact is road safety activities. Lao PDR has a poor and deteriorating road safety record, the worst in the Association of Southeast Asian Nations, and requires assistance to redress the situation. Following support from ADB, a road safety action plan was developed and approved but implementation was impeded by shortage of funds. The government has recently approved a road safety fund using a portion of the RMF but to date, this has yet to be made operational. The AusAID could provide resources subject to the operation of the fund to deliver cross-cutting road safety programs such as road safety audit and black spot treatments, education programs to support safety at schools and health related programs to support development of trauma facilities, ambulance services and data collection based on health sector records. Another option would be to support thematic programs such as helmet wearing, speed management and school safety. 

A third area of possible support would be to help develop the GMS trade through the various transport-related investments in the GMS transport facilities including road development and cross border facilities. Since many of these investments are large this type of support might be preferably provided through cofinancing support to multilateral donor programs. Lao PDR, being landlocked country, needs open borders to facilitate its international trade and because it is centrally located, many international trade routes pass through the country. The strategic position of the country suggests that its transport network could be a core conduit in the flow of international trade in the subregion.