Public Law Sector Evaluation

Evaluation Matrix

Relevance
Evaluation Issues and Questions Indicators Methods and Data Sources

1. Is there a continuing need for Public Law Sector (PLS) services?

  1. To what extent has the demand for PLS services changed, or is expected to change?
  • Level of demand for PLS legal services
  • Changes in demand for PLS legal services

Document review

  • Government of Canada , Justice, PLS and Section documents
  • iCase reports
  • Justice Canada Client Feedback Survey

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Focus groups with clients

2. To what extent do the objectives and activities of the PLS align with the priorities and objectives of the Department of Justice and the Government of Canada overall?

Degree of alignment of PLS services with Department of Justice mandate and Government of Canada priorities

Document review

  • Government of Canada, Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Policy/program staff at Justice

3. To what extent are the legal services provided by PLS consistent with federal roles and responsibilities?

Consistency of the role of the PLS with federal roles and responsibilities when working with other levels of government, international governments and stakeholders

Documents

  • Government of Canada, Justice, PLS and Section documents
Performance
Evaluation Issues and Questions Indicators Methods and Data Sources

4. To what extent are PLS outputs being achieved?

Evidence of the achievement of outputs for the following activities:

  • policy advice
  • legal and legal policy advice
  • litigation services
  • negotiation of international instruments
  • continuing legal education/training
  • support of the Cabinet and parliamentary process

Document review

  • Justice, PLS and Section documents
  • iCase reports

5. To what extent are PLS services/support meeting the needs of Departmental Legal Services Units (DLSUs)? Other clients?

  1. Are there areas for improvement?
  2. Are there other departments or agencies that require PLS services but are not accessing them? If so, what are the implications of this?
  • Extent to which PLS is consulted by other Justice counsel and/or District Attorneys
  • Quality/usefulness/timeliness of legal and policy services provided
  • Factors affecting the decision to use PLS services among clients and other Justice counsel
  • Potential risks of not seeking PLS services on Justice’s ability to deliver legal services and on the Government of Canada to achieve its goals
  • Evidence where not seeking PLS advice has had an impact – either positive or negative - on Justice, a client department, or the government as a whole

Document review

  • Justice, PLS and Section documents
  • Justice Canada Client Feedback Survey

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Focus groups with clients

Case studies

6. To what extent is PLS achieving its expected outcomes?

Q6A. (Immediate: Departments/agencies and Ministers have access to timely, consistent, coherent advice)

  • Extent to which clients indicate that advice is timely and coherent
  • Extent to which advice provided by Justice is consistent across the federal government

Document review

  • Justice, PLS and Section documents
  • Justice Canada Client Feedback Survey

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials

Case studies

Focus groups with clients

Q6B. (Immediate: Government litigators and decision-makers are aware and informed of policy options, legal risks, and legal options)

  • Extent to which PLS is consulted on major litigation and policies/legislative initiatives
  • Extent to which decision-makers are aware of policy options, legal risks, and legal options

Document review

  • Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials

Case studies

Focus groups with clients

Q6C. (Immediate: Effective advocacy of Government of Canada’s position)

  • Evidence of consultations, including PLS, to develop whole-of-government approach to legal and policy issues
  • Extent to which the Government of Canada’s position is consistently communicated in domestic and international contexts, including courts

Document review

  • Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials

Case studies

File review

Q6D. (Immediate: International bodies and other governments are made aware of Canadian positions)

  • Level and nature of involvement of PLS counsel in multi-lateral/bilateral meetings where the Government of Canada has an interest

Document review

  • Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials

Q6E. (Immediate: Domestic stakeholders are aware of Canada’s international obligations)

  • Extent to which PLS has made domestic stakeholders aware of the Government of Canada’s rights and obligations under international instruments (e-mails, meeting minutes, documents, legal opinions etc.)

Document review

  • Government of Canada, Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Case studies

Q6F. (Immediate: Justice counsel receive training and are kept current on public law principles and trends)

  • Level of training provided by PLS
  • Extent to which PLS training activities (training, conferences, tools) have met trainees’ needs
  • Extent to which clients are able to identify legal/policy contexts in which PLS advice is advisable
  • Extent to which tools/websites meet the needs of Justice counsel

Document review

  • Justice, PLS and Section documents
  • CLE evaluation forms
  • Justice Canada Client Feedback Survey

Interviews

  • PLS management/counsel
  • Justice counsel

Focus groups with clients

Q6G. (Immediate: Government officials are aware of the law and of their legal obligations)

  • Extent to which government officials know when to consult DLSU counsel on public law issues
  • Extent to which DLSU counsel consult with PLS before providing legal advice

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Case studies

Focus groups with clients

Q6H. (Immediate: Confidence in PLS staff as experts)

  • Level of client rating on ‘expert level of knowledge’ (measure on Client Satisfaction Survey)
  • Level of satisfaction among Justice counsel regarding PLS services
  • Extent to which PLS counsel are invited to participate in meetings, conferences, seminars

Document review

  • Client Satisfaction Survey

Interviews

  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Focus groups with clients

Q6I. (Immediate: Ministers have the necessary advice to make timely informed decisions)

  • PLS briefs Minister in preparation for Parliamentary Committees
  • Level of satisfaction with PLS services

Interviews

  • Justice counsel
  • Senior government officials

Q6J. (Intermediate: The Government of Canada has an appropriate policy and legal framework that reflects domestic and international obligations)

  • Extent to which PLS is able to provide advice to ensure that Canada’s laws and policies are consistent with domestic and international obligations

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Justice policy/program staff

Q6K. (Intermediate: Justice provides coherent [and consistent] legal, legal policy and policy advice to the Government of Canada)

  • Extent to which Justice “speaks with one voice”
  • Extent to which legal, legal policy and policy advice is considered by decision-makers
  • Evidence of consultations, including PLS, to develop whole-of-government approach to legal, legal policy and policy issues

Document review

  • Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Case studies

Focus groups with clients

File Review

Q6L. (Intermediate: The federal government is able to manage and mitigate its legal risks based on an integrated whole-of-government approach)

  • Extent to which PLS counsel contribute to the assessment of legal risk and the development of options
  • Extent to which that District Attorneys (DAs) involve Justice counsel, including PLS, at an early stage in the assessment of risk and development mitigation strategies

Document review

  • Government of Canada, Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Case studies

Focus groups with clients

7. To what extent is PLS able to manage the demand for legal and policy services?

  1. To what extent does PLS have sufficient and appropriate resources to meet the demand?
  2. To what extent are DAs sharing in this responsibility?
  3. Are there any other factors that have an impact on the delivery of PLS services?
  4. To what extent does the PLS role as a central agency impact its ability to manage the demand for services?
  • Sufficiency of resources (human, financial, technical, competencies, training) to meet demand for services
  • Extent to which DAs plan/budget for the use of PLS services
  • Other factors that have an impact on the delivery of legal services by PLS
  • Impact of non-funded work done by PLS for Justice

Document review

  • Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel

Case studies

Focus groups with clients

File review

8. To what extent are PLS services provided in a cost-efficient manner?

  • Evidence that PLS makes use of best/promising practices
  • Extent to which PLS shares information with Justice counsel (for example, through practice groups/website tools)
  • Extent to which PLS has made efforts to reduce the costs of providing services
  • Evidence of strategies to reduce costs (for example, knowledge management)

Document review

  • Government of Canada, Justice, PLS and Section documents

Interviews

  • PLS management/counsel
  • Justice counsel
  • Policy/program staff at Justice

Focus groups with clients

File review

9. Are the appropriate management structures and processes in place to support the effective delivery of services?

Appropriateness of governance structure to support the delivery of PLS services (including role of the National Legal Advisory Committee)

Interviews

  • PLS management/counsel
  • Justice counsel
  • Policy/program staff at Justice

10. Are there better or alternative ways of achieving the same results?

Evidence of alternative models for the delivery of public law services in other national governments

Interviews

  • PLS management/counsel
  • Justice counsel
  • Policy/program staff at Justice

Focus groups with clients

11. To what extent are PLS roles and responsibilities clearly defined, communicated and understood by Justice counsel?

  • Evidence that roles and responsibilities are clearly defined and communicated
  • Extent to which requests are consistent with PLS roles and responsibilities
  • Evidence that PLS counsel are involved in activities that are outside the role of PLS
  • Degree to which Justice counsel understand the roles and responsibilities of PLS

Document review

  • Justice, PLS and Section documents
  • Client Satisfaction Survey

Interviews

  • PLS management/counsel
  • Justice counsel
  • Senior government officials
  • Policy/program staff at Justice

Focus groups with clients

File review

12. Are there any best practices or lessons learned in the delivery of PLS services/supports?

Evidence of best practices and lessons learned

Interviews

  • PLS management/counsel
  • Justice counsel
  • Policy/program staff at Justice

Case studies

Focus groups with clients

Date modified: