Public Law Sector Evaluation

4. Key Findings

4.3. Performance – Demonstration of Efficiency and Economy

The Treasury Board’s 2009 Policy on Evaluation defines efficiency as production of “a greater level of output ... with the same level of input or, a lower level of input with the same level of output,” and economy as the achievement of expected outcomes using the minimum amount of resources required (Treasury Board of Canada Secretariat, 2009). Applying these definitions to the PLS, an analysis of its efficiency and economy considers the ability of the PLS to manage the demand for legal services and the degree to which the legal services provided are cost efficient.

4.3.1. Trends in Expenditures, Resources, and Resource Use

During the period covered by the evaluation, the PLS appears to have successfully managed its expenditures. As discussed in Section 2.2, while PLS expenditures rose between 2007–08 and 2008–09, since then they have decreased by $1 million. Between 2007–08 and 2011–12, the PLS expenditures rose by just over 3% (see Table 14). Trends in expenditures differ, however, depending on the section. Expenditures for JLT showed a decrease between 2007–08 and 2011–12, while CAILS, HRLS, and (to a lesser extent) IPLS expenditures increased (Table 14).

Table 14: Total expenditures by section, by year, 2008–12 ($ millions)
  2007-08 2008-09 2009-10 2010-11 2011-12
CAILS (see note) 2.8 2.6 3.2 3.3 3.6
HRLS (see note) 2.8 2.8 3.1 3.1 3.1
JLT 1.7 2.8 2.1 1.6 1.6
ILAPS (see note) 1.2 1.1 1.1 1.5 1.6
Business Administration Centre 0.6 0.6 0.9 1.3 1.2
IPLS 0.9 0.9 1.0 1.0 1.1
OLLS 0.8 0.8 1.1 1.0 1.0
JACTPS 0.9 0.9 0.8 0.9 0.9
Business Administration Centre Policy 0.3 0.6 0.5 0.5 0.5
ADMO 1.0 0.5 0.5 0.5 0.4
IALS (transferred to Transport DLSU) 0.2 0.8 0.2 0.2 0.2
PILS (amalgamated with CAILS in June 2011) 0.6 0.6 0.7 0.6 0.1
PLPS (see note) 1.2 1.2 1.2 1.0 -
Total 14.8 16.3 16.3 16.4 15.3

Note: Part of PLPS was amalgamated with ILAPS in 2010–11. The rest of PLPS was then divided between CAILS and HRLS in 2011–12.

The section expenditures do not sum to the total due to rounding.

After a rise from the previous year in 2008–09, total human resources have also decreased annually between 2008–09 and 2011–12, which is mostly attributable to a decrease in LA FTEs. FTE trends differ from section to section, with JLT experiencing the most pronounced decrease and CAILS the most pronounced increase, the latter due largely to the PLS reorganization (see Table 15).

Table 15: Human resources (FTEs) by section, by year, 2008–12
  2007-08 2008-09 2009-10 2010-11 2011-12
CAILS (see note) 25.2 26.4 27.4 28.7 31.8
HRLS (see note) 27.5 27.9 27.3 28.1 29.2
JLT 17.6 21.8 17.0 15.0 16.6
ILAPS (see note) 11.3 9.6 8.9 12.4 13.7
OLLS 7.9 8.4 10.0 9.1 9.1
JACTPS 8.3 8.7 7.9 7.5 8.9
IPLS 4.9 6.4 6.1 6.7 7.7
Business Administration Centre 4.3 3.8 5.1 6.9 6.8
ADMO 4.2 4.0 4.0 3.6 4.3
IALS (transferred to Transport DLSU) 1.5 3.0 2.0 2.0 2.0
Business Administration Centre Policy 2.9 3.4 3.1 1.5 1.2
PILS (amalgamated with CAILS in June 2011) 5.6 6.0 5.4 5.0 0.8
PLPS (see note) 12.3 13.0 11.4 8.8 -
Total 133.6 142.4 135.7 135.2 132.1

Note: Part of PLPS was amalgamated to ILAPS in 2010–11. The rest of PLPS was then divided between CAILS and HRLS in 2011–12.

The section FTEs do not sum to the total due to rounding.

As with expenditures and human resources, the trends on resource use, as reflected in the number of hours spent on actively managed files, has varied by section between 2007–08 and 2011–12. While one section experienced a slight decline (3% for HRLS), Footnote 18 most sections have had workload increases ranging from 6% for ILAPS to 73% for IPLS (see Table 5). JLT experienced a substantial decline in hours (26%) along with a decline in staffing and expenditures, but this does not capture the pressures on JLT. As discussed in the following section, a challenge for the PLS will be to maintain its expenditures in the face of workload pressures, to address the unique situation of JLT, and to manage the potential effects of the upcoming move away from cost recovery.

4.3.2. Managing Demand for PLS Services

Sufficiency of resources

Demand management, to a large degree, is affected by the sufficiency of resources available. The evaluation found that, in general, the materials, equipment, and technological resources available to PLS counsel are sufficient; overall, PLS counsel are able to respond to requests and produce high-quality work with current resources. The vast majority of PLS counsel who responded to Public Service Employee Surveys in 2005, 2008, and 2011 agree that they have the materials and equipment they need to do their work (in all three years, close to 90% of PLS respondents agreed or strongly agreed), and that material and tools provided (including software and other automated tools) are available in the official language of their choice (in all three years, almost all PLS respondents agreed and close to three quarters strongly agreed –– see Table 16 and Table 17). Other lines of evidence support this. Key informants representing the PLS generally agreed that they have sufficient resources to manage their work, and DLSU and other Justice counsel pointed out that, for the most part, resource issues are not preventing PLS counsel from being able to respond to requests in a timely manner or be responsive to clients’ needs. As discussed in Section 4.2.1, multiple lines of evidence show that PLS counsel respond even to urgent requests in a timely manner, suggesting that the demand for PLS services is being managed appropriately.

Table 16: PLS respondents’ opinions on materials and equipment
I have the materials and equipment I need to do my job.
Survey year Agree Neither agree nor disagree Disagree Don’t know or N/A
2005 96% N/A 4% 0%
2008 93% 0% 6% 1%
2011 91% 3% 7% 0%

Source: Public Service Employee Surveys for 2005, 2008, and 2011

Note: Summation of percent values for rows may not equal 100% due to rounding.

N/A cells indicate data not available in the survey reports. The 2005 survey used a slightly different scale from the 2008 and 2011 surveys. In 2005, respondents could select “strongly agree, mostly agree, mostly disagree, strongly disagree, don’t know, or not applicable.” In 2008 and 2011, respondents could select “strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, strongly disagree, don’t know, or not applicable.”

Table 17: PLS respondents’ opinions on language availability of materials and tools
The materials and tools provided for my work … are available in the official language of my choice.
Survey year Agree Neither agree nor disagree Disagree Don’t know or N/A
2005 96% N/A 4% 0%
2008 93% 3% 4% 1%
2011 94% 2% 3% 2%

Source: Public Service Employee Surveys for 2005, 2008, and 2011

Note: Summation of percent values for rows may not equal 100% due to rounding.

N/A cells indicate data not available in the survey reports. The 2005 survey used a slightly different scale from the 2008 and 2011 surveys. In 2005, respondents could select “strongly agree, mostly agree, mostly disagree, strongly disagree, don’t know, or not applicable.” In 2008 and 2011, respondents could select “strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, strongly disagree, don’t know, or not applicable.”

Although resources for the Sector overall appear to be sufficient, the evaluation results raised some section-specific concerns. While most sections do not seem to be experiencing major resource issues, evidence indicates that JLT is experiencing unique resource challenges. As discussed in Sections 4.2.1 and 4.3.1, unlike other PLS sections, JLT has experienced a substantial rise in the number of hours spent on actively managed files over the evaluation period. In addition, JLT has experienced the most pronounced decrease, particularly during the last four years, in both expenditures (by 41%) and FTEs (by 24%) (see Table 14 and Table 15). As JLT does not use cost recovery, the effects of decreases in A-Base funding and cuts to DFAIT (JLT’s partner organization) have been particularly noticeable.

A comparison of JLT to other PLS sections based on hours spent on actively managed files demonstrates the magnitude of the resource challenges facing JLT. As Table 18 shows, hours per staff member (counsel and paralegals) for JLT is approximately two times more than several PLS sections.

Table 18: Hours spent on actively managed files per staff by section, 2011–12
  Total number of staff
(counsel and paralegals)
Total hours Hours per staff member
(counsel and paralegals)
JLT 13 28,123 2,163
IPLS 5.8 8,606 1,484
JACTPS 6.6 8,803 1,334
OLLS 7.2 8,628 1,198
HRLS 23.6 26,228 1,111
ILAPS 11.8 11,280 956
CAILS 25.8 25,377 984

The unique pressures on JLT are supported by multiple lines of evidence. Documents and interviews (both with key informants and case study participants) identified resource limitations experienced by the Section. It appears that resource issues are having an impact on JLT’s ability to manage the demand for its services. According to evaluation stakeholders, clients and DLSUs can experience long waits for JLT advice. Although there is no evidence to suggest that JLT is not being consulted when needed, evaluation participants mentioned, on occasion, trying to avoid the need to consult with JLT.

The potential impact of upcoming changes to financing and billing procedures

In January 2013, along with changes to legal services rates for 2013–14, the Department of Justice announced that, in the upcoming year, the PLS will no longer charge for services on a per-usage basis; instead, the costs associated with the provision of all PLS services will be included in the legal service rates. This upcoming move away from cost recovery for the PLS is expected to have implications for efficiency (see Section 4.3.3); however, it may also have implications for the Sector’s ability to manage the demand for services.

Part of the reason for the move away from cost recovery is to ensure that clients’ decisions about whether to consult with the PLS will not be affected by their willingness or ability to pay. As the evaluation results have shown, cost recovery has, to some extent, affected decisions about when to engage PLS services. It is unclear what impacts incorporating PLS services into legal service rates will have on the demand for services. Considering the complexity of the cost recovery system for the PLS (where some sections charge and some do not, and some clients are charged while others are not), eliminating cost recovery for the PLS will, presumably, make it easier for sections to manage finances; however, potential impacts on requests for services may create other demand-management challenges.

4.3.3. Cost-Efficient Provision of PLS Services

The Law Practice Model

The time period for this evaluation follows the Strategic Review of programs and expenditures conducted by the Department of Justice in 2008. Based on the recommendations of this review, the 2009 Budget review resulted in a commitment to achieving a savings of $12.5 million by 2011–12 through Treasury Board reductions to the Department’s A-Base funding. These cost savings were to be realized, in large part, through implementation of the Law Practice Model (LPM), a strategy that emphasizes matching counsel to the risk and complexity level of files in order to achieve efficiencies in legal operations.

The evaluation revealed some concerns about the appropriateness of the LPM for the PLS, considering the expert nature of PLS services. The Strategic Review found potential within the Department overall for junior counsel to take on more of the work that senior counsel were doing. Therefore, throughout the Department, implementation of the LPM focused recruitment and hiring of more junior (LA1) level counsel, and initiated a temporary pause on staffing at higher levels. As the expert nature of PLS work limits the potential for increased assignment of work to more junior counsel or entry-level staff, evaluation stakeholders raised concerns about pressures for more PLS work to be undertaken by less experienced counsel.

In general, stakeholders consulted for the evaluation indicated that the assignment of PLS counsel to files is appropriate. However, due to limited data, the evaluation could not assess the extent to which the LPM is affecting the PLS, and could not substantiate either perceptions of the appropriate assignment of counsel to files or concerns raised about the LPM. In theory, the appropriateness of file assignment can be measured, to some extent, by comparing the level of counsel assigned to files with the files’ level of risk and/or complexity. However, in iCase, very few PLS files were assigned a risk or complexity level, and a relatively large proportion of files did not specify the level of lawyers assigned to work on them. The lack of availability of this data means that, currently, no objective indicators related to file assignment are available. Some stakeholders pointed out that PLS files are assigned based more on urgency than either risk or complexity, as a simple request that requires a fast turnaround may need to be completed by senior counsel. The urgency of requests, however, is also not currently tracked or captured in iCase (which tracks files, not requests). Some sections are tracking more detailed data on requests, but the PLS does not have sector-wide information. Future evaluations would benefit from the development and implementation of performance indicators related to file assignment that are appropriate to the PLS.

The cost efficiency of PLS services

Evaluation results indicate that the PLS operates in a cost-efficient manner. Evaluation stakeholders strongly perceive PLS services to be cost efficient, and believe that the PLS counsel conduct their work efficiently. Key informants, case study interviewees, and focus group participants pointed out that engaging experts is a cost-effective practice. PLS counsel, with their in-depth knowledge of specific areas of public law, can prepare opinions on complex matters of public law using less time and resources than DLSU lawyers or litigators who are less familiar with these issues. Training sessions, practice groups, publications issued by specific sections, and Justipedia were all identified as beneficial information-sharing tools or structures that help to improve efficiency by reducing duplication of work and providing Justice counsel with access to information needed to work effectively with the PLS. Footnote 19 Some stakeholders specifically described PLS services as “good value for money.”

These opinions are supported by documented evidence of measures taken by the PLS to improve the efficiency of services, as described in Section 2.1. This reorganization was designed to better serve the needs of the Department and to improve efficiency by providing a single point of contact for matters that were previously overlapping between sections. Furthermore, the approaching move away from the cost recovery model for PLS services (mentioned in Section 4.3.2) is expected to simplify billing processes and allow for streamlining of some supporting internal operations.

It should be noted that, while the usefulness of Justipedia was highlighted by stakeholders, some do not feel that Justipedia is used as much as it could be –– particularly by Justice counsel outside of the PLS. Contrary to this impression, however, data on Justipedia usage (both contribution to the database and use of the information within the database) show widespread use. PLS is not the primary contributor or user. Several portfolios (Tax Law Services; Public Safety, Defence, and Immigration; and Business and Regulatory Law) make substantial contributions to Justipedia. In addition, the regional offices and portfolios all indicate significant use of the database.

This department-wide use of Justipedia is important for the PLS, as its use by other counsel is considered a way to help manage demand for PLS legal services; counsel can review previous PLS opinions on related issues before determining whether there is a further need to consult PLS, or to assist them in focusing the legal issue they need PLS to address. PLS counsel generally found Justipedia to be very useful and an improvement over the Legal Opinions and Precedence On-line Retrieval System that preceded it. However, some limitations and potential improvements were mentioned (e.g., lack of access to secret documents; a potential need for more frequent and systematic contributions).

The role of clients in the efficiency and economy of legal services

Managing the demand for legal services and improving the efficiency and economy of the delivery of those services are joint responsibilities of the PLS and the Justice counsel and clients with whom PLS counsel work. Consequently, the evaluation explored areas where the PLS and Justice counsel and clients could work together to improve the efficiency and economy of legal services.

One important factor influencing the efficiency of PLS services is the extent to which the PLS is engaged by clients or DLSU counsel early in files. When PLS counsel are engaged at an early stage, they are able to identify the potential for legal risks and help to mitigate these risks (and their associated costs). In addition, early engagement allows PLS counsel to respond more quickly to requests that occur at later stages in the file –– particularly when the same PLS counsel are involved throughout the life of a file. As mentioned in Section 4.2.1, file review and case study results show relatively early engagement of the PLS (in general, before decision making occurs); however, stakeholders (primarily key informants and case study participants) identified that where engagement of the PLS in files has been delayed (due to cost recovery or other factors), this has had an impact on efficiency. Although cost recovery soon will not apply to the PLS, it may be necessary to ensure that other cost-reduction initiatives do not result in unanticipated pressures to delay engagement of specialized legal services. In addition, and probably more likely, the removal of cost recovery may result in increased demand for PLS legal services. The Sector will need to track and manage any increase carefully to ensure that the quality and timeliness of its legal services are not affected.

Another client- or DLSU-driven factor affecting the efficiency of PLS services is the quality of requests for services. Generally, the more comprehensive and specific the request for PLS services, the more quickly and effectively PLS counsel can respond to requests and produce advice that meets client expectations. By contrast, incomplete or ambiguous requests require more consultations and discussion to ensure that the needs of clients are effectively communicated and understood and that misunderstandings about the type or comprehensiveness of advice required are avoided.

Evaluation stakeholders (focus group participants, in particular) mentioned that the quality of legal requests has been improving. Not only has the increased availability of information through resources such as Justipedia allowed Justice counsel to prepare more detailed and informed legal service requests, but the cost recovery model has caused some DLSU counsel and litigators to conduct more background and policy analysis upfront in an effort to save costs. Footnote 20 However, despite these observations, stakeholder consultations and the file review indicate that requests still vary in quality and comprehensiveness; requests were overly broad or not in line with PLS roles and responsibilities in several of the files included in the file review.

Date modified: