Report on the Audit of Cost Recovery Process Improvement (CRPI) Initiative - Phase 1 - March 2015
6. Findings, Recommendations and Management Action Plan
This section provides the findings and recommendations resulting from the audit work carried out. While the audit was conducted based on the lines of enquiry and audit criteria identified in the planning phase, this report is structured along the following main findings:
- Roles and Responsibilities;
- Reporting and Monitoring;
- Oversight; and
- Client Satisfaction.
For conclusions by audit criterion, please refer to Appendix A.
Based on the audit work performed and our professional judgment, the risk associated with each finding was rated using a three-point scale. The risk ranking (high, moderate, and low) is based on the level of potential risk exposure we feel may have an impact on the achievement of Justice Canada’s objectives, and is indicative of the priority Management should give to the recommendations associated with that finding. The following criteria were used in determining the risk exposure level:
Controls are not in place or are inadequate.
Compliance with legislation and regulations is inadequate.
Important issues are identified that could negatively impact the achievement of program/operational objectives.
Controls are in place but are not being sufficiently complied with.
Compliance with central agency/departmental policies and established procedures is inadequate.
Issues are identified that could negatively impact the efficiency and effectiveness of operations.
Controls are in place but the level of compliance varies.
Compliance with central agency/departmental policies and established procedures varies.
Issues identified are less significant but opportunities that could enhance operations exist.
Cost recovery is an important source of funding for the Department, providing $330.2M in 2013-2014, or approximately one-third of the operating budget. The Cost Recovery Process Improvement (CRPI) Initiative was initiated in August 2010 to examine and re-engineer Justice’s end-to-end cost recovery processes. The resulting changes to the process came into effect on April 1, 2012. The CRPI is comprised of five business processes as follows:
- Memorandums of Understanding (MOUs), including Annex Bs that support planning and forecasting;
- Reimbursable Disbursements;
- Invoicing; and
- Accounts Receivable Settlements.
Throughout the duration of the CRPI Initiative, active monitoring and oversight was provided by the CRPI Project Committee (January 2011 - July 2012). Following the implementation of this initiative, the CRPI Project Committee was disbanded. While responsibility for an initial suite of system changes was delegated to a Change Advisory Board, lead responsibility for various elements of the cost recovery process was dispersed to the respective business owners throughout the organization.
Given the magnitude of this initiative and the criticality of the revenues generated through cost recovery, effective governance processes and practices are required to ensure the Department continues to achieve the expected results of the CRPI Initiative. As outlined in the CRPI Project Charter, this initiative was designed to:
- Promote consistent business practices across the Department;
- Smooth out cash flows so they are received on a more predictable basis throughout the fiscal year;
- Address client Departmental expectations;
- Achieve greater policy compliance; and
- Respond to prior audit findings and recommendations.
6.1.1 Finding 1: Roles and Responsibilities
Cost recovery roles and responsibilities are documented and have been communicated across the Department. Interviews conducted with departmental staff indicated that cost recovery roles and responsibilities were clear and well understood.
Roles and responsibilities for each of the five elements of the cost recovery process are clearly documented in the CRPI Supplementary Documentation. This is a key document that is readily available on the Department’s intranet and was referenced in the suite of guidance used to train departmental staff on the new CRPI. Cost recovery reporting and monitoring roles and responsibilities are also documented in the CRPI Supplementary Documentation, with specific reference to in-year cost recovery reporting and monitoring through the Financial Situation Reporting (FSR) process.
The Department’s Accountability Framework for MOU Related to the Provision of Legal Services elaborates further on roles and responsibilities with respect to the MOUs and Annex Bs development, negotiation and monitoring processes. The CRPI Project Close-Out Report clearly assigns to Management and CFO Sector the responsibility for the support, maintenance and troubleshooting of the financial system-related elements of cost recovery.
Interviews with staff responsible for the various elements of the cost recovery process indicated Departmental staff have a sound awareness and understanding of their roles and responsibilities. A number of interviewees indicated that they find the CRPI support provided by the Management and CFO Sector to be timely and very helpful.
6.1.2 Finding 2: Reporting and Monitoring
Processes and practices are in place and operating effectively to support cost recovery reporting and monitoring at the client level. However, the frequency and nature of cost recovery reporting does not support active monitoring and oversight at the strategic or corporate level.
MOUs and Annex Bs
The Treasury Board Common Services Policy requires departments providing mandatory common services to “develop, in consultation with client departments and agencies, meaningful and visible standards of service and performance for the delivery of mandatory services.” The Department of Justice has an MOU Framework in place that provides the overarching structure and expectations with respect to the development, negotiation and approval of client agreements (i.e. umbrella MOUs often covering more than one year and accompanying Annex Bs that are negotiated annually). This Framework includes the expectation that “the status of MOU implementation will be monitored and shared with senior management at least once a year.” This Framework also specifies that by April 1st each year, the financial codes necessary for cost recovery (i.e. Interdepartmental Settlement codes) are to be obtained from clients and reflected in Annex B.
At the client level, the Finance and Planning Branch of Management and CFO Sector maintains an MOU repository. The Branch periodically produces an Excel spreadsheet that reflects the status of MOUs and Annex Bs by client and portfolio. Once the associated information is validated by related branches and portfolios, the spreadsheet is posted on the Department’s intranet site.
At the Department level, the Business Practice and Intelligence Branch of Management and CFO Sector produced a high level status report for the 2012-2013 fiscal year. This reporting, tabled at Management Committee (MC) in September 2013, reflected the status of MOUs and Annex Bs by portfolio. However, the reporting did not include any accompanying client information. While this reporting may have technically fulfilled the requirements of “annual MOU status reporting” as outlined in the MOU Framework, the nature of it was at such a high level that it did not provide MC with meaningful insight to clearly link the absence of required Annex Bs to client billings, particularly in relation to the Department’s top billers. In addition, the tabling of this 2012-2013 report after year end, with no corresponding in-year MC reporting, was not optimal to support senior management in the timely identification and management of significant or systemic issues in this area.
There was no MOU status reporting for 2013-2014 or to date for 2014-2015. In the first quarter of 2014-2015, there was a 15% reduction in cost recovery revenue from approximately $36M at June 30, 2013 to approximately $30.6M at June 30, 2014. This was primarily the result of the less than timely receipt of signed Annex Bs and the receipt of Interdepartmental Settlement (IS) codes that facilitate the timely collection of cost recovery revenue. This issue came to light via the Department’s external Quarterly Financial Report. Regularized strategic reporting that clearly links cost recovery revenue with the status reporting on Annex Bs and receipt of IS codes would provide timely, relevant information to support MC in their monitoring role.
Interviews indicated that resource constraints resulted in the absence of any MOU status reports being developed or communicated to MC since 2012-2013. The Business Practice and Intelligence Branch and CFO Sector have indicated that work is underway to strengthen MOU monitoring and oversight processes for 2015-2016 including the tracking, escalation and status reporting to Portfolios, Branches and MC.
As previously stated, expectations for cost recovery monitoring and reporting, including in-year cost recovery reporting and monitoring through the Financial Situation Reporting (FSR) process are outlined in CRPI Supplementary Documentation.
Each month the Finance and Planning Branch (FPB) develops Cost Recovery Billing Reports and posts them on the Department’s intranet. This reporting reflects actual cost recovery billings by portfolio and by client for the related month as well as total year-to-date billings. As noted in interviews, this reporting supports LSUs in monitoring and discussing in-year billings with clients, reviewing and updating client forecasts, and planning for the next fiscal year.
On a quarterly basis, the FPB updates the consolidated revenue forecast for the Department and communicates this information to MC via the Financial Situation Reporting (FSR) process. However, we noted that the nature of the cost recovery information reflected in these reports varied from quarter to quarter. For example, the updated revenue forecast reflected in the October 2014 FSR included a one line adjustment that is based on a historical trending of forecasting accuracy. For the corresponding quarter of the preceding year (i.e. October 2013), the revenue forecast reflected a brief summary by portfolio as well as client details for major reductions in forecasted revenue.
In addition, the FPB prepares the monthly FSRs and the year-end update report. Monthly FSRs and year-end reporting for 2013-2014 provided an explicit comparison of actual cost recovery revenue versus the forecast of the prior year. The reporting included the identification of material differences between actual and forecasted cost recovery revenue for the period and the key clients whose delayed billings were the main reason for the difference. The 2014-2015 FSRs include a similar bar chart comparison of actual cost recovery for the current and prior year. However, there was no inclusion of a comparison against forecast nor was there a clear identification of, or information on, the variance between the current and prior year’s actual cost recovery revenue, either for the month or year-to-date.
The absence of a clear comparison of actual cost recovery revenue against forecast, particularly for the Department’s top 20-30 clients, makes it a challenge for MC to readily monitor the timeliness of cost recovery cash flows and address any related issues that may arise.
Management and CFO Sector should review and refine its in-year and year-end cost recovery reporting and monitoring information and processes to better support effective strategic monitoring and oversight of the cost recovery system.
Management Action Plan
The Management and CFO Sector will update the “Accountability Framework for Memorandum of Understanding related to the Provision of Legal Services” and will implement a process for monitoring and reporting starting in Fiscal Year 2015-16, which will address concerns with respect to the frequency and the nature of the reporting to Management Committee.
The Management and CFO Sector will provide the Management Committee with a consolidated comparison of the actual cost recovery revenues versus forecast revenue in the Financial Situation Report (FSR) in greater details, as well as obtain information suggested by Management Committee starting with the June 2015 FSR process.
Office of Primary Interest:
Management and CFO Sector
July 31, 2015 - Obtaining Management Committee members’ input and guidance on the information they need through the FSR process.
March 31, 2016 - Remainder of the MAP
6.1.3 Finding 3: Oversight
There is effective oversight of some elements of the cost recovery system. Opportunities exist to refine the oversight of cost recovery to support the continued achievement of the expected outcomes that the system was designed to deliver on.
The design and implementation of the CRPI Initiative included mechanisms to support oversight. This included a CRPI Project Committee, co-chaired by two Departmental executives, which oversaw the design and implementation of this initiative and a Change Advisory Board responsible for overseeing an initial suite of changes to the cost recovery system.
Following the implementation of the CRPI Initiative, it was expected that strategic oversight for cost recovery, a regime that generates revenue to cover approximately one-third of the Department’s budget, would shift to an existing Departmental committee. Management Committee (MC) is the Department’s decision-making forum. As outlined in its Terms of Reference, the committee is responsible for “legal service delivery, horizontal management, operational and accountability issues affecting the Department.” Given the horizontal nature of CRPI and the criticality of cost recovery revenue to funding the Department’s operations, MC plays a key role in the strategic oversight of cost recovery.
To support the Management Committee in its overall role, we noted that between April 2013 and October 2014, members of this Committee received and reviewed the following cost recovery reporting:
- Financial Situation Reports (FSRs) are tabled at MC throughout the year and a Year-End Financial Update is tabled once at MC a few months following year end;
- Annual Cost Recovery Analysis report for the two fiscal years that provided an analysis of total cost recovery revenues (NVA) and the associated costs, both in dollar and percentage terms for the Department as a whole; and
- A high level MOU and Annex B status report for the 2012-2013 fiscal year entitled Report to Management Committee on MOU Completion Status for 2012-13.
The above reports provide some information on cost recovery. However, the nature and frequency of the reporting does not readily support senior management in effectively exercising its oversight role in this area. For example, with little to no reporting to MC on the status of MOUs and Annex Bs, MC may not be able to identify or address significant and/or systemic issues on a timely basis.
While the Department is reviewing and considering changes to the funding model, effective oversight will continue to be imperative to ensure the efficient management of an area that is highly decentralized and material to Departmental operations.
Management Committee should review and refine the nature and frequency of Management Committee’s review and discussion of critical elements of the cost recovery system to ensure continued achievement of the benefits derived from the investment in the Cost Recovery Process Improvement Initiative.
Management Action Plan
The Management Committee will be provided with an opportunity to discuss reporting requirements on CRPI as part of the June 2015 FSR process. At a minimum, Management Committee will be provided with the new information outlined in the management action plan for recommendation #1.
Office of Primary Interest:
March 31, 2016
6.1.4 Finding 4: Client Satisfaction
Ad hoc client feedback is sometimes received by LSUs. However, no formal or systematic mechanism is in place to solicit and capture client satisfaction information with respect to CRPI.
Since 2006, the Department has undertaken a survey every three years to solicit feedback from clients. The primary focus of this survey is to gain insight into the quality, accessibility, responsiveness, usefulness and timeliness of legal services provided. To date, the survey has not encompassed cost recovery, including client invoicing and administrative burden.
The integration of LSUs in major client departments provides for a close working relationship between Justice lawyers and the client. This working relationship can provide an informal mechanism for clients to provide feedback on the cost recovery system, including the changes flowing from CRPI.
The implementation of the Billing Manager Review process and the Final Quality Assurance (QA) Review and Approval process supports improved accuracy of client invoices. Interviews with those involved in this process pointed to a reduction in the number of questions and concerns expressed by clients following the issuance of an invoice in a post-CRPI world. This is interpreted as an indication that client satisfaction has improved with respect to the accuracy of invoices.
Ad hoc verbal feedback received by LSUs indicates that the implementation of monthly billing under CRPI has improved client satisfaction with respect to the timeliness of invoicing.
Anecdotal information provided by LSUs indicates that while clients appreciate the regularized monthly billing, the lack of CRPI-generated information to readily support the amounts billed continues to be problematic. For example, the cost recovery module of the Integrated Financial and Material System (IFMS) does not provide any details to support Full Time Equivalent (FTE) billing. Hourly billing information provided by this system is effectively a ‘data dump’ of the time slips for the billing period, without any summary by lawyer and/or by file. The lack of FTE billing information and the sheer length of the accompanying hourly billing details (i.e. it is often well in excess of 100 pages) compromises the quality of the invoice and on its own would be administratively burdensome and challenging for clients to ascertain the nature and reasonableness of the services being billed.
Through interviews and document reviews, we found that some LSUs have developed their own tools and reports, outside of the cost recovery module of IFMS, to better meet clients’ invoicing needs and expectations. For example, Canada Revenue Agency’s LSU has developed tailored templates and associated monthly reporting to better meet their client’s needs with respect to forecast monitoring. This enables clear and readily understandable substantiation of the associated legal services billings. As another example, Transport Canada’s LSU has developed a database to readily track, analyze and report on the legal services they provide.
As previously noted, the absence of a systematic or formalized mechanism to solicit and respond to feedback from clients impairs Justice Canada’s ability to ensure that the investment in CRPI has resulted in improved client satisfaction with respect to invoicing and administrative burden. As a result, management is not able to effectively assess the continued achievement of the related CRPI outcomes and ensure the efficient design and implementation of invoice reporting templates and tools that meet clients’ needs and expectations.
Management and CFO Sector should leverage existing mechanisms to solicit and respond to feedback from client departments with respect to invoicing and the administrative burden of the cost recovery process.
Management Action Plan
The Legal Services Review Wave 2 proposed changes will address issues identified by client departments with respect to invoicing and the administrative burden of the cost recovery process.
In accordance with section 5.3 of the respective MOUs between Justice and clients, the Legal Services Unit Heads will continue to be encouraged to informally review and discuss the quality of legal services with clients relative to the service standards and the operational indicators in order to identify opportunities for service improvement.
Office of Primary Interest:
Management and CFO Sector
April 30, 2016
- Date modified: