Annex G: Implementation Status of the Recommendations in 2004 Evaluation

Recommendations	Status	
1.	The respective roles, responsibilities, authorities and accountabilities of TPRC and IMF with respect to PFTAC should be more clearly enunciated. A formal governance charter defining the role, duties, delegations and reporting obligations of PFTAC in relation to IMF, TPRC and individual donors, clients and other stakeholders should be drafted by PFTAC and submitted to the IMF and TPRC for endorsement.	Poor (1): Nothing was done beyond repeating standard material in the FY2009/11 Project Document. No action taken to develop a formal governance charter. In practice, there have not been major problems that suggest that there is an urgent need to have such a charter. In practice, there have not been major problems in this area. 	
2.	 PFTAC should undertake, in conjunction with key stakeholders (including donors), a situation analysis (i.e. analysis and scenario planning) to identify challenges within its region that need to be addressed in its strategic plan and new project document.	Poor (1): A formal situation analysis of the challenges in the region was not prepared. 	
3.	PFTAC should adopt a more comprehensive strategic planning process and presentation format that covers: (i) situation, financial and stakeholder analysis; (ii) vision, values and mission statements; (iii) objectives, strategies, functions and action timetables; and (iv) minimum guarantees of service, risk management devices, and key performance indicators.	Good (3): Country Notes are prepared and regularly updated. A vision, values and mission statement is available for PFTAC. For TAs objectives and timetables are defined. Service delivery standards in terms of responses to PICs are defined in Para 49 of the FY2009/11 Project Document.	
4.	PFTAC should develop, possibly with the help of an outside consultant, a risk management plan that identifies and ranks key risks in terms of their probability and impact and comes up with strategies for mitigating or managing them. This risk management plan should be incorporated within PFTAC’s comprehensive strategic plan and the new project document.	Poor (1): No action taken. The lack of risk management plan has not had a noticeable adverse impact on PFTAC’s operations.	
5.	PFTAC should undertake activity costing of each of its functions (something that will become increasingly important if PFTAC obtains a significant increase in funds) to identify the breakdown in expenditure: (i) between the four functions or programs of Public Financial Management, Tax and Customs Policy and Administration, Banking Regulation and Supervision and Economic and Financial Statistics; and (ii) between the three forms of technical assistance or outputs consisting of (a) short-term ad hoc advice often consisting of one-off assistance involving less than one month, (b) longer-term reform assistance often consisting of repeat visits and lasting many months or years, and (c) capacity building consisting of short-term and longer-term personnel training and development within each of the four functional areas.	Poor (1): Weaknesses in IMF’s financial management system for RTACs and the fact that PFTAC does not have online access to current expenditure data prevented PFTAC from taking action in this area.	
6.	Again if PFTAC is able to get significant incremental funds, PFTAC should submit its budget on a line item, program and output basis to the TPRC and report its annual financial results on this basis on its website.	Poor (1): Weaknesses in IMF’s financial management system for RTACs and the fact that PFTAC does not have online access to current expenditure data prevented PFTAC from taking action in this area.	
7.	PFTAC should measure what shares of its TA goes on (i) reactive ad hoc advice, (ii) pro-active reform assistance, and (iii) capacity building such as extended courses, short-term workshops, overseas attachments and on-the-job training.	Poor (1): Weaknesses in IMF’s financial management system for RTACs and the fact that PFTAC does not have online access to current expenditure data prevented PFTAC from taking action in this area.	
8.	PFTAC should articulate a TA framework and strategy for helping the PICs adopt a relevant reform vision and use gap analysis to devise jointly agreed strategies and action plans upon which future PFTAC and donor TA can be based. Such a framework should also accommodate short-term ad hoc advice and ongoing capacity building. This framework and strategy should also recognize that progress cannot be made in all 15 countries simultaneously. Rather one or a few pilots should be developed and then rolled out to other PICs as with PC/Trade.	Modest (2): PFTAC’s limited and over stretched resources limit its ability to develop broad frameworks at the strategic level in most countries. More use could have been made of the concept of piloting an approach in one or a few pilots and then replicating it in other PICs.	
9.	In order to satisfy the demand for ad hoc advice, reform assistance and capacity building the organizational structure should be expanded as follows: (i) while it would not be useful to duplicate within PFTAC the economic research work of IMF Headquarters economists, PFTAC should examine the need for periodic employment of specialist advisors in specific fields of concern to the PICs as a means of complementing the economic research of IMF Headquarters economists ( e.g. in public debt compilation, measurement of taxation expenditures, construction of macroeconomic frameworks for PIC multi-year budgets etc). (ii) PFTAC should introduce a new tier of locally recruited experts to support the resident advisors. This should be piloted with one or two experts to start with and if successful further recruits could be sought if sufficient funds were available. The proposed local experts should be mature professionals who have research and policy advisory experience as employees of finance and economic ministries in PICs. Such local experts should be employed on 3 – 6 month contracts with renewal subject to satisfactory performance. (iii) PFTAC should use more external consultants to provide additional TA, especially capacity building, in the PICs.	Poor (1): The functional areas in which PFTAC provided assistance has remained as they were when PFTAC was established in 1993. PFTAC did not offer macroeconomic assistance or introduce a new tier of locally recruited experts to support the Resident Advisors. The budget for short term experts increased but it was only used consistently in the fiscal areas. Either very little or no use was made of short term experts in the statistics or financial sector supervision areas.  This is an important recommendation and is largely reiterated in this evaluation report.	
10.	PFTAC should develop a detailed budget of the additional funds that would be required to cover these additional resources and this budget should be presented to the donors for additional funding. 	Modest (2): Donors provided additional funding for short term experts.	
11.	PFTAC should expand the terms of reference of the resident advisor on banking regulation and supervision to cover also insurance and superannuation. Consideration should also be given to PFTAC’s appropriate future involvement in AML/CFT activities. 	Modest (2): The previous financial sector supervision advisor had sufficiently broad experience to provide advice on insurance and superannuation fund supervision. The current advisor is more narrowly focused on bank supervision. This evaluation reiterates the need for PFTAC to be able to respond to requests for assistance related to the supervision of insurance and superannuation funds. PFTAC’s AML/CFT activities declined because within IMF responsibility for this area shifted from MCM to the Legal Department in 2006 and other donors are active in this area.	
12.	PFTAC should place greater emphasis on helping PICs develop the capacity to plan and roll out long-term reform strategies and help the PICs get additional funding from other regional donors.	Modest (2): PFTAC’s limited and over stretched resources undermined its ability to be involved in long term capacity building at the strategic level. While the Evaluation Team identified some excellent examples of PFTAC working with governments and other donors to mobilize the funding to support the implementation of its recommendations, more use should be made of this approach going forward. 	
13.	PFTAC should maintain a registry of all PIC projects to which it contributes significant TA.	Good (3): PFTAC could readily provide the Evaluation Team with information on the TA projects that it supported. 	
14.	The following information should be furnished for each project: (i) client, purpose, stages, timetable and expected outcomes; (ii) PFTAC’s contribution in terms of tasks, time, costs and outputs; and, (iii) Project outcomes relative to original expectations.	Good (3): Although there are issues related to the remote use of TAIMS, it provides a framework to organize and store such information.	
15.	Each project’s success should be rated on a scale of 1 to 5 on its completion or suspension.	Good (3): TAIMS provides a consistent IMF wide rating scale. While it is a four point rating scale, identical to what was used for this evaluation, this is not a material difference for this recommendation.  	
16.	The results of this performance reporting should be published on PFTAC’s website along with the main lessons learned from the experience. 	Poor (1):  The ratings of individual TAs are not publicly available on PFTAC’s webpage.	
17.	PFTAC should develop a minimum guarantee of services statement to reinforce its high reputation and to help manage and thereby contain the burgeoning expectations of both clients and donors.	Excellent (4): These service delivery standards are stated in Para 49 of the FY2009/11 Project Document.	
18.	PFTAC should develop a formal feedback mechanism for capturing the extent of client satisfaction with its individual field services (i.e. significant ad hoc services, reform assistance and personnel capacity building). For instance at the end of each assignment the client could be asked to answer a simple questionnaire that invites a score out of five for (a) satisfaction with the project’s terms of reference, (b) the personal conduct of the advisor, and (c) the usefulness of the technical advice. 	Good (3): One such survey was undertaken and the results were reported to TPRC and in the FY2009/11 Project Document. The response rate was low. PICs should have been more diligent in responding to the questionnaire. 	
19.	PFTAC should exchange information with other RTACs (CARTAC, AFRITAC East, and AFRITAC-West) on processes, systems, structures, strategies and measurements that have proven useful for planning, delivering and reviewing technical assistance within the FEM area.	Poor (1): Information exchange between the RTACs is weak and should be improved, particularly a more structured exchange of information between PFTAC and CARTAC.	
20.	Coordinators of all RTACs should meet annually in Washington, DC to discuss their mutual challenges, how they are tackling them and lessons to be learned from their experiences. This forum could also be used for exchanging each RTAC’s strategic plan, work plans, progress reports and application models.	Modest (2): The RTAC Coordinators met in Washington in December 2008. More such meetings should take place.	
21.	 A copy of this Evaluation Report should be given to the Coordinators of each of the other RTACs as it is the first example of an evaluation of a mature RTAC. 	Not Rated: The Evaluation Team is not aware of what action was taken on this recommendation.	
22.	The FEMM or TPRC should commission PFTAC to undertake a review of the relative merits and potential cost savings of introducing shared regional service centers, especially for smaller states, in the public financial management areas such as accounting, reporting, budgeting, financial information systems and auditing. Funding and partial staffing for such an exercise should be sought from the ADB and/or other donors.	Poor (1): No action was taken to implement this recommendation.	
23.	While no change is recommended at this time, the IMF in conjunction with the TPRC should consider at a future review whether PFTAC’s name should be changed to PICTAC to conform with the nomenclature for other regional RTACs, more accurately define the region to which it applies (PICs) and to facilitate expansion of its functions to cover economic, not just financial matters.	Not Rated. The time frame for action was not specified other than to indicate that it was not expected in the short term.  	
24.	A new project document should be prepared for the next phase of PFTAC. This document should incorporate the results of many of the recommendations made above.	Good (3): The FY2009/11 Project Document includes several references to the findings of the 2004 Evaluation. Although action was not taken to implement all of the areas noted, the 2004 evaluation findings were clearly mentioned in the Project Document. 	
Note: The implementation status of the recommendations was rated on a four point scale: Excellent (4); Good (3); Modest (2) and Poor (1)	
 Source 2009 PFTAC Evaluation