Public Law Sector Evaluation

Data Collection Instruments

Evaluation of the Public Law Sector Key Informant Interview Guide for Public Law Sector Counsel and Management

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). The evaluation includes interviews with those working within the PLS, with representatives of the Department of Justice, and with representatives of other government departments who are familiar with the work of the PLS.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

The following questions refer to the PLS, but we understand that you may not have direct knowledge of the PLS as a whole. If that is the case, please respond for your section within the PLS.

For brevity’s sake, the questions refer to PLS legal services or advice. Please respond based on the work you do for the PLS, including litigation, litigation support, legal advice, policy and legal policy advice, and negotiations.

In addition, some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007-2011, so please consider your experiences during those years in your responses.

Introduction

1. Please describe briefly your current roles and responsibilities within the PLS. Will you be responding to the following questions for PLS as a whole or for your section only?

Relevance

2. Over the last five years, have you observed any changes in the demand for PLS services? In your response, please consider the volume, type of legal issues, complexity, legal risk level, and any other characteristics of your work. How has the PLS responded to these trends? [MATRIX Q1]

3. How does the work of the PLS align with the strategic outcomes of the Department of Justice? [MATRIX Q2]

4. How do PLS services align with the priorities of the federal government? [MATRIX Q2]

Performance — Effectiveness

5. Based on your experience, is the PLS generally consulted by client departments or other areas of Justice (Departmental Legal Services Units, regional offices, etc.) when it should be? In your response, please consider the extent to which the PLS is consulted on major public law issues that affect the areas in which you work, and whether the request for assistance from the PLS is timely and appropriate. [MATRIX Q5 and Q6B, Q6G]

6. How effective are consultations among the PLS, Justice, and other federal departments and agencies in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? What, if anything, could be improved, and how might the PLS support any improvements? [MATRIX Q6C and Q6K]

7. What factors affect the willingness or ability to consult with the PLS among clients and/or other areas of Justice (e.g., cost, awareness/understanding of when it would be advisable to consult the PLS, perception of usefulness/desirability of obtaining PLS advice)? What are the potential risks to client departments, the Department of Justice, and the Government of Canada when the PLS is not consulted? [MATRIX Q5 and Q6G]

8. One of the outcomes identified for the PLS is that client departments/agencies and Ministers have access to timely, consistent, and coherent advice. To what extent is the PLS able to achieve this outcome? What, if any factors, affect the PLS’s ability to provide timely, consistent, and coherent advice? [MATRIX Q6A]

9. When situations arise where the PLS advice differs from the advice of lead counsel on a file, how is that usually resolved? What structures are in place within PLS and Justice generally to promote consistency in its advice? How well do these structures work? [MATRIX Q6A]

10. Please describe how/if the PLS and Justice counsel work together to identify and assess legal risks, and how/if they work together to develop legal and/or policy options to manage or mitigate those risks. In your opinion, how effective is this collaboration? [MATRIX Q6B and Q6L]

11. Please describe the level and nature of the PLS’s involvement in multilateral/bilateral meetings of international bodies. Based on your experience, how effective is Canada at communicating a consistent position and making international bodies and other governments aware of Canada’s position? What, if anything, could be improved and how might the PLS support any improvements? [MATRIX Q6C and Q6D]

12. What is the PLS’s role in ensuring that domestic stakeholders are aware of Canada’s rights and obligations under international instruments? In your opinion, are these efforts successful? Why or why not? [MATRIX Q6E]

13. What is your opinion of the current training and other resources (e.g., tools, website, conferences) provided to Justice counsel and other clients on public law principles and trends? To what extent are training needs being met? Are there any gaps? [MATRIX Q6F]

14. In your opinion, is the PLS recognized within Justice and by other departments and agencies as the public law experts within the Government of Canada? What is the basis for your opinion? [MATRIX Q6H]

15. In your view, how well does the briefing process work (e.g., in terms of providing senior managers, the Deputy Minister, and the Minister with the necessary advice to make timely, informed decisions)? Please explain. [MATRIX Q6I]

16. In your experience, to what extent is the advice provided by the PLS considered by other Justice counsel when developing legal strategies and legal advice given to clients? [MATRIX Q6K]

17. To what extent does the advice appear to inform decisions made by Ministers and client departments/agencies? [MATRIX Q6K]

18. Are there any barriers to PLS advice being considered by client departments or agencies in the decision-making process? What factors make it more or less likely that PLS advice will be considered? [MATRIX Q6K]

Performance — Efficiency and economy

19. Does the PLS have adequate resources (e.g., human, financial, technological) in place to support its work? How does the PLS manage resource challenges? [MATRIX Q7]

20. In your opinion, what role do client departments/agencies play in improving the effectiveness, efficiency, and economy of the PLS legal services? To what extent are they fulfilling this role? [MATRIX Q7]

21. In your view, what factors affect the ability of PLS to manage the demand and/or cost of its legal services? In your response, please consider:

  • any PLS efforts to adopt or implement best practices;
  • any strategies undertaken to reduce overall costs (such as knowledge management strategies, etc.) and/or the costs of providing legal services;
  • efficiencies achieved through PLS information-sharing activities (such as practice groups, online tools, etc.);
  • any non-funded work done by the PLS for Justice. [MATRIX Q7 and Q8]

22. What factors contribute to or constrain the PLS’s ability to provide timely, high quality, cost effective legal services? How might these factors be addressed? [MATRIX Q10]

23. In your view, does the current governance structure within Justice (e.g., the National Legal Advisory Committee) adequately support the PLS in its delivery of services? What, if any, barriers to service delivery have the PLS encountered? [MATRIX Q9]

24. In your experience, are requests for PLS services typically in line with PLS roles and responsibilities? Are you aware of any requests made to the PLS for services that go beyond the Sector’s mandate? If so, how were these requests addressed? Are there ways in which the PLS could communicate its roles and responsibilities more clearly to Justice and other clients? [MATRIX Q11]

25. In your view, what (if any) best practices or lessons learned have emerged from the delivery of PLS services and supports over the past five years? [MATRIX Q12]

26. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector Key Informant Interview Guide for Justice Counsel (outside of the Public Law Sector)

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). The PLS is comprised of seven specialized legal advisory/policy and policy sections and one secretariat. The sections are: Constitutional, Administrative and International Law Section; Human Rights Law Section; Information Law and Privacy Section; Judicial Affairs, Courts, and Tribunal Policy Section; Trade Law Bureau; International Private Law Section; and Official Languages Law Section. In addition, PLS includes the International Aviation Law Secretariat.

The evaluation includes interviews with those working within the PLS, with representatives of the Department of Justice, and with representatives of other government departments who are familiar with the work of the PLS.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

The following questions refer to the Department of Justice and the PLS, but we understand that you may not have direct knowledge of the Department of Justice or the PLS as a whole. If that is the case, please respond for your section within Justice and the PLS section(s) with which you have worked.

For brevity’s sake, the questions refer to PLS legal services or advice. Please respond based on the type of assistance you receive from the PLS, including litigation, litigation support, legal advice, policy and legal policy advice, and negotiations.

In addition, some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007-2011, so please consider your experiences during those years in your responses.

Introduction

1. Describe the level and nature of your involvement with the PLS. Which PLS section(s) have you worked with the most?

If you have worked with more than one section and any of your responses depends on one specific PLS section, please specify to which section you are referring in your answer.

Relevance

2. Over the last five years, have you observed any changes in the Department of Justice’s demand for, and use of, PLS services? In your response, please consider the volume, type of legal issues, complexity, legal risk level, and any other characteristics. How has the PLS responded to these changes? [MATRIX Q1]

3. Based on your experience working with the PLS, how do PLS services support the Department of Justice’s strategic outcomes? [MATRIX Q2]

4. How do PLS services align with the priorities of the federal government? [MATRIX Q2]

Performance — Effectiveness

5. When do you/your area of Justice consult with the PLS? Based on your experience, is the PLS consulted when it should be? In your response, please consider the extent to which the PLS is consulted on major public law issues that affect the areas in which you work. [MATRIX Q5 and Q6B, 6G]

6. Have you experienced any situations that limited your ability to use PLS legal services when you thought it was appropriate (e.g., client refusal, budgetary constraints)? If yes, please describe your experiences. In your opinion, what, if any, impact did this have on the progress or outcome of the file? [MATRIX Q5 and Q6B, Q6G]

7. How effective are consultations among the PLS, Justice, and other federal departments and agencies in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? How effectively are differences in legal opinions resolved and through what process? What, if anything, could be improved, and how might the PLS support any improvements? [MATRIX Q6C and Q6K]

8. What factors affect your area’s willingness or ability to consult with the PLS (e.g., cost, awareness/understanding of when it would be advisable to consult the PLS, perception of usefulness/desirability of obtaining PLS advice)? What are the potential risks to client departments, the Department of Justice, and the Government of Canada when the PLS is not consulted? [MATRIX Q5 and Q6G]

9. One of the outcomes identified for the PLS is that client departments/agencies and Ministers have access to timely, consistent, and coherent advice. To what extent is the PLS able to achieve this outcome? What, if any factors, affect the PLS’s ability to provide timely, consistent, and coherent advice? [MATRIX Q6A]

10. What structures are in place within PLS and Justice generally to promote consistency in advice? How well do these structures work? [MATRIX Q6A]

11. Please describe how/if the PLS and Justice counsel work together to identify and assess legal risks, and how/if they work together to develop legal and/or policy options to manage or mitigate those risks. In your opinion, how effective is this collaboration? [MATRIX Q6B and Q6L]

12. Please describe the level and nature of the PLS’s involvement in multilateral/bilateral meetings of international bodies. Based on your experience, how effective is Canada at communicating a consistent position and making international bodies and other governments aware of Canada’s position? What, if anything, could be improved, and how might the PLS support any improvements? [MATRIX Q6C and Q6D]

13. What is the PLS’s role in ensuring that domestic stakeholders are aware of Canada’s rights and obligations under international instruments? In your opinion, are these efforts successful? Why or why not? [MATRIX Q6E]

14. What is your opinion of the current training and other resources (e.g., tools, website, conferences) provided to Justice counsel and (if you are aware) client departments/agencies on public law principles and legal and judicial trends? To what extent are training needs being met? Are there any gaps? [MATRIX Q6F]

15. In your opinion, is the PLS recognized within Justice and (if you are aware) by other departments and agencies as the public law experts within the Government of Canada? What is the basis for your opinion? [MATRIX Q6H]

16. In your view, how well does the briefing process work (e.g., in terms of providing the Minister, Deputy Minister, and Associate Deputy Minister with the necessary advice to make timely, informed decisions)? Please explain. [MATRIX Q6I]

17. In your experience, to what extent is the advice provided by the PLS considered in the legal strategies pursued and decisions made by Ministers, Justice counsel, and client departments/agencies? Are there any barriers to advice being considered in the decision-making process? What factors make it more or less likely that advice will be considered? [MATRIX Q6K]

Performance — Efficiency and economy

18. Based on your experience, does the PLS have adequate resources (e.g., human, financial, technological) in place to support its work? How does the PLS manage resource challenges? [MATRIX Q7]

19. In your opinion, what role do client departments/agencies or other areas in Justice that use the PLS play in improving the effectiveness and efficiency of the PLS legal services? Does your area of Justice have any processes to ensure that PLS services are used efficiently, such as prioritizing requests or ensuring that duplication of legal requests is avoided? [MATRIX Q7]

20. In your view, is the provision of PLS services cost-efficient? Why or why not? [MATRIX Q8]

21. Does your area of Justice obtain legal services from the PLS that are not subject to cost recovery? Please describe that work in terms of the nature and extent of the legal services being requested that are not cost-recovered. [MATRIX Q7]

22. What, if any, suggestions do you have for improving the efficiency or cost-effectiveness of legal services provided by the PLS? [MATRIX Q10]

23. In your view, does the current governance structure within Justice (e.g., the National Legal Advisory Committee) adequately support the PLS in its delivery of services? What, if any, barriers to service delivery have the PLS encountered? [MATRIX Q9]

24. In your experience, have the roles and responsibilities of the PLS been clearly defined and communicated to Justice counsel like yourself and (if you are aware) to client departments/agencies? Are these client groups aware when they should consult with the PLS? Are there ways in which the PLS could communicate its roles and responsibilities more clearly to Justice and other clients? [MATRIX Q11]

25. Based on your experience working with the PLS, what would you identify as best practices or lessons learned in the delivery of PLS services? [MATRIX Q12]

26. In general, how satisfied are you with the services you have received from the PLS? [MATRIX Q6H]

27. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector Key Informant Interview Guide for Client Departments

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). The PLS is comprised of seven specialized legal advisory/policy and policy sections and one secretariat. The sections are: Constitutional, Administrative and International Law Section; Human Rights Law Section; Information Law and Privacy Section; Judicial Affairs, Courts, and Tribunal Policy Section; Trade Law Bureau; International Private Law Section; and Official Languages Law Section. In addition, PLS includes the International Aviation Law Secretariat.

The evaluation includes interviews with those working within the PLS, with representatives of the Department of Justice, and with representatives of other government departments who are familiar with the work of the PLS.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

The following questions refer to the PLS, but we understand that you may not have direct knowledge of the PLS as a whole. If that is the case, please respond for the PLS section(s) with which you have worked.

In addition, some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007-2011, so please consider your experiences during those years in your responses.

Introduction

1. The PLS provides litigation, litigation support, legal advice, policy and legal policy advice, and support for international negotiations. Please describe the level and nature of your involvement with the PLS. Which PLS section(s) have you worked with the most?

Please base your responses to the following questions on your experiences with these PLS sections. If you have worked with more than one PLS section and any of your responses depends on one specific PLS section, please specify to which section you are referring in your answer.

Relevance

2. Over the last five years, have you observed any changes in your department or agency’s demand for, and use of, PLS services? In your response, please consider the volume, type of legal issues, complexity, legal risk level, and any other characteristics. How has the PLS responded to these changes? Has your department or agency tried to reduce or manage the level of demand for PLS legal services and, if so, how? [MATRIX Q1 and Q7]

Performance — Effectiveness

3. When does your department or agency consult with the PLS? Generally, do you consult directly with the PLS or through your Departmental Legal Services Unit? [MATRIX Q5 and Q6B, 6G]

4. Based on your experience, is the PLS consulted when it should be? What situations, if any, have limited the use of the PLS legal services (e.g., cost, lack of awareness/understanding of when it would be advisable to consult the PLS, perception of usefulness/desirability of obtaining PLS advice)? In your opinion, what, if any, impact did this have on the progress or outcome of the file? [MATRIX Q5 and Q6B, 6G]

5. How effective are consultations among the PLS, your representatives, other Justice counsel representing your department or agency, and (as applicable) other relevant government departments in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? What, if anything, could be improved? How might the PLS support any improvements? [MATRIX Q6C and 6K]

6. What factors affect your department or agency’s willingness or ability to consult with the PLS (e.g., cost, awareness/understanding of when it would be advisable to consult the PLS, perception of usefulness/desirability of obtaining PLS advice)? What are the potential risks to your department or agency and the Government of Canada if the PLS is not appropriately consulted? [MATRIX Q5 and Q6G]

7. One of the outcomes identified for the PLS is that client departments/agencies have access to timely, consistent, and coherent advice. To what extent is the PLS able to achieve this outcome? What, if any factors, affect the PLS’s ability to provide timely, consistent, and coherent advice? [MATRIX Q6A]

8. Please describe any involvement the PLS has in helping your department or agency identify and assess legal risks, and develop legal and/or policy options to manage or mitigate those risks. In your opinion, how effective is this involvement of the PLS? [MATRIX Q6B and Q6L]

9. Please describe any work that your department or agency does with the PLS concerning multilateral/bilateral meetings of international bodies. Based on your experience, how effective is Canada at communicating a consistent position and making international bodies and other governments aware of Canada’s position? What, if anything, could be improved and how might the PLS support any improvements? [MATRIX Q6C and Q6D]

10. Please describe any work that your department or agency does with the PLS in ensuring that domestic stakeholders are aware of Canada’s rights and obligations under international instruments? In your opinion, are these efforts successful? What, if anything, could be improved and how might the PLS support any improvements? [MATRIX Q6E]

11. Has the PLS provided your department or agency with any training on public law principles and trends and/or on how to identify situations where the PLS should be consulted? How satisfied are you with any training received? To what extent are training needs being met? Are there any gaps? [MATRIX Q6F]

12. In your opinion, is the PLS recognized by your department or agency as the public law experts within the Government of Canada? Why or why not? [MATRIX Q6H]

13. In your experience, to what extent is the advice provided by the PLS considered in the legal strategies pursued and decisions made by your department or agency? Are there any barriers to advice being considered in the decision-making process? What factors make it more or less likely that advice will be considered? [MATRIX Q6K]

Performance — Efficiency and economy

14. In your opinion, what role do client departments/agencies have in improving the effectiveness, efficiency, and economy of the PLS legal services that they use? [MATRIX Q7]

15. In your view, is the provision of PLS services cost-efficient? Why or why not? [MATRIX Q8]

16. What, if any, suggestions do you have for improving the efficiency or cost-effectiveness of legal services provided by the PLS? [MATRIX Q10]

17. Is your department or agency aware when it should consult with the PLS? Are there ways in which the PLS could communicate its roles and responsibilities more clearly to clients? [MATRIX Q11]

18. Based on your experience working with the PLS, what would you identify as best practices or lessons learned in the delivery of PLS services? [MATRIX Q12]

19. In general, how satisfied are you with the services you have received from the PLS? [MATRIX Q6H]

20. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector Case Study Guide for the PLS

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). One aspect of the study is an in-depth review of ten files to provide detailed information on the broad spectrum of services provided by the PLS. Each case study includes interviews with key PLS staff and client representatives (which may be representatives of the client department/agency and/or of the Departmental Legal Services Unit), as well as a review of the file, which is conducted by a PLS counsel.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

For brevity’s sake, the questions refer to PLS legal services or advice. Please respond based on the type of work conducted by your section, including litigation, litigation support, legal advice, policy and legal policy advice, and negotiations.

In addition, some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007–2011, so please consider your experiences during those years in your responses.

1. What legal services did your section provide on this file and what was your role? [MATRIX Q4]

2. If you are aware of it, please describe the events leading up to and the process used to engage your section in this file. In your opinion, were services requested and engaged in a timely manner? Why or why not? [MATRIX Q5, Q6L, and Q6G]

3. How often and for what purpose was your section consulted on the file? Did you consult directly with client departments/agencies or with other areas of Justice (e.g., regional or Departmental Legal Services Unit counsel)? [MATRIX Q5, Q6B, and Q6G]

4. Did any situations arise where the advice provided by your section differed from the advice of the lead counsel? How was that resolved? [MATRIX Q6A]

5. To what extent was the advice provided on this file by your section considered in the development of the legal strategies and the advice given to the client? To what extent did your section’s advice on this file inform the decisions made by the client? In your opinion, what factors affected whether the advice was considered? [MATRIX Q6G and Q6K]

6. In your opinion, how effective were these consultations in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? [MATRIX Q6C and Q6K]

7. Were you involved in the identification, assessment, and communication of legal risk on the file? Please describe your involvement, including whether other Justice counsel were involved. Were you satisfied with the level of your involvement and how legal risk was assessed and communicated to the client? [MATRIX Q6B and Q6L]

8. Please describe your involvement in developing legal and policy options to manage or mitigate legal risks. Did you collaborate/consult with other Justice counsel and/or the client department/agency? Were you satisfied with the level of your involvement and how/whether your advice on managing legal risk was used? [MATRIX Q6B, Q6L, and Q6K]

9. Were you involved in the briefing process as part of this file? How well does that process work? [MATRIX Q6I]

10. In general, do you think the federal government’s legal position was clearly and consistently communicated to the necessary stakeholder groups within government as well as to external stakeholders (i.e., international bodies, provincial or territorial governments, private sector, academic legal community)? Why or why not? What role did your section have in communicating the government’s legal position? [MATRIX Q6C, Q6D, and Q6E]

11. In your opinion, were the services on this file provided by your section in a timely, coherent, and consistent manner? Why or why not? [MATRIX Q6A]

12. In your view, were sufficient resources available within your section to meet the demand for services on this file? Were you aware of any other resources challenges (human, financial, technical, competency, training) in your work on this file? How were these challenges managed? [MATRIX Q7]

13. Are there any other factors that affected your ability to deliver legal services? [MATRIX Q7]

14. In your opinion, was this file handled in a cost-effective manner? What, if anything, could have been done differently by your section, the PLS, Justice, and/or other stakeholders to improve timeliness or to reduce costs? [MATRIX Q8]

15. In your view, what, if anything, could have been done differently to better serve clients’ needs? [MATRIX Q5]

16. Were any “lessons learned” from work on this file? Can you identify any best practices that emerged from the delivery of services and supports in relation to this file? Please explain. [MATRIX Q12]

Conclusion

17. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector Case Study Guide for the Justice Counsel (outside the Public Law Sector)

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). One aspect of the study is an in-depth review of ten files to provide detailed information on the broad spectrum of services provided by the PLS. Each case study includes interviews with key PLS staff and client representatives (which may be representatives of the client department/agency and/or of a Departmental Legal Services Unit), as well as a review of the file, which is conducted by a PLS counsel.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

The following questions refer to the PLS, but we understand that you may not have direct knowledge of the PLS as a whole. If that is the case, please respond for the PLS section(s) with which you have worked. In addition, some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007–2011, so please consider your experiences during those years in your responses.

1. With which section of the PLS did you work on this file?

Please base your responses to the following questions on your experiences with this PLS section.

2. Please describe the nature of the services that the PLS provided on this file. What was your role? [MATRIX Q4]

3. If you are aware of it, please describe the events leading up to and the process used to engage the PLS in this file. In your opinion, were PLS services requested and engaged in a timely manner? Why or why not? [MATRIX Q5, Q6L, and Q6G]

4. How often and for what purpose was the PLS consulted on the file? Did the PLS primarily consult within Justice, or did it also consult directly with the client? [MATRIX Q5, Q6B, and Q6G]

5. Did any situations arise where the PLS advice differed from the advice of the lead counsel? How was that resolved? [MATRIX Q6A]

6. To what extent was the PLS advice provided on this file considered in the development of the legal strategies and the advice given to the client? To what extent did the PLS advice provided on this file inform the decisions made by the client? In your opinion, what factors affected whether the PLS advice was considered? [MATRIX Q6G and Q6K]

7. In your opinion, how effective were the consultations with the PLS in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? [MATRIX Q6C and Q6K]

8. Was the PLS involved in the identification, assessment, and communication of legal risk on the file? Please describe its involvement. Were you satisfied with the level of its involvement and its advice on assessing and communicating legal risk to the client? [MATRIX Q6B and Q6L]

9. To what extent was the PLS involved in developing legal and policy options to manage or mitigate legal risks? Were you satisfied with its level of involvement and its advice on legal and policy options to manage or mitigate legal risk? [MATRIX Q6B, Q6L, and Q6K]

10. Did the PLS assist with the briefing process as part of this file? If yes, were you satisfied with the PLS support that you received? [MATRIX Q6I]

11. In general, do you think the federal government’s legal position was clearly and consistently communicated to the necessary stakeholder groups within government as well as to external stakeholders (i.e., international bodies, provincial or territorial governments, private sector, academic legal community)? Why or why not? What role did the PLS have in communicating the government’s legal position? [MATRIX Q6C, Q6D, and Q6E]

12. In your opinion, were PLS services on this file provided by the PLS in a timely, coherent, and consistent manner? Why or why not? [MATRIX Q6A]

13. In your opinion, were appropriate PLS counsel assigned to this file (considering the years of experience of the counsel and the level of complexity of the file)? Did the PLS assign adequate resources to undertake the work required? Please explain any capacity issues you encountered. [MATRIX Q7]

14. In your opinion, was this file handled in a cost-effective manner? What, if anything, could have been done differently by PLS, Justice, and/or the client department/agency to improve timeliness or to reduce costs? [MATRIX Q8]

15. In your view, what, if anything, could the PLS have been done differently to better serve your needs or the clients’ needs? [MATRIX Q5]

16. Were any “lessons learned” from work on this file? Can you identify any best practices that emerged from the delivery of PLS services and supports in relation to this file? Please explain. [MATRIX Q12]

Conclusion

17. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector Case Study Guide for Client Departments/Agencies

The Department of Justice has hired PRA Inc. to conduct an evaluation of the Public Law Sector (PLS). The PLS is comprised of seven specialized legal advisory/policy and policy sections and one secretariat. The sections are: Constitutional, Administrative and International Law Section; Human Rights Law Section; Information Law and Privacy Section; Judicial Affairs, Courts, and Tribunal Policy Section; Trade Law Bureau; International Private Law Section; and Official Languages Law Section. In addition, PLS includes the International Aviation Law Secretariat.

One aspect of the study is an in-depth review of ten files to provide detailed information on the broad spectrum of services provided by the PLS. Each case study includes interviews with key PLS staff and client representatives (which may be representatives of the client department/agency and/or representatives of a Departmental Legal Services unit), as well as a review of the file, which is conducted by a PLS counsel.

The information we gather through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA and the Department of Justice Evaluation Division. You will have an opportunity to review our written summary of the interview and make any corrections or additions.

Some questions may not be applicable to the work you do. Please let us know and we will skip that question.

The evaluation focuses on 2007–2011, so please consider your experiences during those years in your responses.

1. With which section of the PLS did you work on this file?

Please base your responses to the following questions on your experiences with this PLS section.

2. The PLS provides litigation, litigation support, legal advice, policy and legal policy advice, and support for international negotiations. Please describe the nature of the services that the PLS provided on this file and what was your role? [MATRIX Q4]

3. If you are aware of it, please describe the events leading up to and the process used to engage the PLS in this file. In your opinion, were PLS services requested and engaged in a timely manner? Why or why not? [MATRIX Q5, Q6L, and Q6G]

4. How often and for what purpose was the PLS consulted on the file? Did the PLS primarily consult within Justice, or did it also consult directly with your department/agency? [MATRIX Q5, Q6B, and Q6G]

5. Are you aware of any situations where the PLS advice differed from the advice of the lead counsel? If yes, how were the conflicts resolved? How comfortable were you with the advice you were ultimately given in these situations? [MATRIX Q6A]

6. To what extent did the PLS advice provided on this file inform the decisions made by your department/agency? In your opinion, what factors affected whether the PLS advice was considered? If you cannot respond specifically for the PLS, please respond generally for the advice you received from Justice. [MATRIX Q6G and Q6K]

7. In your opinion, how effective were the consultations with PLS in ensuring a consistent legal position across government and developing a whole-of-government approach to public law issues? [MATRIX Q6C and Q6K]

8. Were you satisfied with the risk assessments you received and how legal risk was communicated to your department/agency? Why or why not? To your knowledge, what role did the PLS have in identifying, assessing, and communicating legal risk? [MATRIX Q6B and Q6L]

9. Were you satisfied with the advice on legal and policy options to manage or mitigate legal risk provided to your department/agency? Why or why not? To your knowledge, what role did the PLS have in developing options and suggesting strategies for managing or mitigating legal risks? [MATRIX Q6B, Q6L, and Q6K]

10. To your knowledge, what, if any, briefing or reporting was done on this file? If none occurred, please explain why. If briefing occurred, did the PLS support the briefing process within your department/agency? Were senior managers/officials in your department/agency made sufficiently aware of this file? Please explain. [MATRIX Q6I]

11. In general, do you think the federal government’s legal position was clearly and consistently communicated to the necessary stakeholder groups within government as well as to external stakeholders (i.e., international bodies, provincial or territorial governments, private sector, academic legal community)? Why or why not? What role did the PLS have in communicating the federal government’s legal position? [MATRIX Q6C, Q6D, and Q6E]

12. In your opinion, were PLS services on this file provided by the PLS in a timely, coherent, and consistent manner? Why or why not? [MATRIX Q6A]

13. In your opinion, were appropriate PLS counsel assigned to this file (considering the years of experience of the counsel and the level of complexity of the file)? Did the PLS assign adequate resources to undertake the work required? Please explain any capacity issues you encountered. [MATRIX Q7]

14. In your opinion, was this file handled in a cost-effective manner? What, if anything, could have been done differently by PLS, Justice, and/or your department/agency to improve timeliness or to reduce costs? [MATRIX Q8]

15. In your view, what, if anything, could the PLS have been done differently to better serve your department’s/agency’s needs? [MATRIX Q5]

16. Were any “lessons learned” from work on this file? Can you identify any best practices that emerged from the delivery of PLS services and supports in relation to this file? Please explain. [MATRIX Q12]

Conclusion

17. Do you have any other comments?

Thank you. We greatly appreciate your participation.

Evaluation of the Public Law Sector File Review

Note: We are using the term “file” broadly. Some of you will be reviewing an entire file, while others will be reviewing discrete portions of files (e.g., a particular legal issue, round of negotiations). This decision should have been made and communicated to you in advance. Please contact your section’s member of the Evaluation Working Group (or, if she or he is unavailable, Joan Remsu) if you have any questions about the scope of the review you should be conducting for a particular file.

Time required to complete template (hours):

Overview [File characteristics and Matrix Q4]

Respond to this section for the file or, if applicable, the portion of the file you are reviewing with the possible exception of Q7. If you are reviewing a portion of the file, please describe in Q7 the nature of the file as a whole and then the section of the file that has been chosen to be the subject of the file review.

1. Assigned File Code Number:

2. Date file opened/work started: Date file closed/work ended:

3. Public Law Sector (PLS) Section (choose one):

  • CAILS
  • HRLS
  • ILAPS
  • JLT
  • IPLS
  • OLLS
  • JACTPS
  • IALS

4. Lead counsel

  • PLS
  • Regional office
  • DLSU
  • Other

5. Case type:

  • Litigation
  • Litigation support (i.e., PLS is not lead)
  • Legal advice
  • Policy advice
  • Legal policy advice
  • Negotiation

6. Client:

  • Justice
  • Other government department/agency

6A. Which area of Justice?

6B. Which other government department/agency?

7. Legal issue and brief description of the nature of the file and the selection within the file that is being used for the file review. Do not reference any information that would waive solicitor-client privilege. In your description, please include any cross-cutting issues (across PLS and/or the Department of Justice) and what factors make the file complex or high risk.

7A. How would you describe the complexity level of the file? (choose one):

  • Low
  • Medium
  • High
  • Unable to assess

Engagement of the PLS [Matrix Q5, Q6G, Q6L]

Question 8 is for litigation files only

8. When was the PLS first engaged on this file? (Check all that apply)

  • Before other areas of Justice (e.g., DLSU, regional office) provided advice to client on public law issues
  • Before initial legal risk assessment was made
  • Before client made any decisions based on legal options and/or litigation strategies
  • After the Claim or Action was commenced
  • Before a Defence or Response was filed on behalf of the Crown
  • Before other critical moment in file. Please specify
  • PLS was not engaged until late in the file. Please explain.
  • Don’t know/can’t tell

Question 9 is for litigation support and advisory files.

9. When was the PLS first engaged on this file/the request for advice received? (Check all that apply)

  • Before other areas of Justice (e.g., DLSU, regional office litigators, or Legislative Services) provided legal advice/services on public law issues
  • Before initial legal risk assessment was made
  • Before client made any decisions based on legal options and/or litigation strategies
  • Before other critical moment in file. Please specify
  • PLS was not engaged until late in the file. Please explain.
  • Don’t know/can’t tell

Question 10 is for policy and legal policy files.

10. How was the PLS first engaged on this file/the request for advice received? (Check all that apply)

  • From regular or routine work in a particular subject matter
  • In anticipation of a court decision
  • Following a court decision
  • At the request of the Justice Minister
  • At the request of a client (central agency, DLSU, client department)
  • At the request of a Justice or PLS colleague as part of an ongoing file
  • At the request of Legislative Services Branch in drafting legislation or regulations
  • As a member of a committee, practice group, or working group in Justice
  • As a member of a committee or working group in a client department
  • From stakeholder or media pressure
  • Other — please explain.
  • Don’t know/can’t tell

Information from file [Answer the following questions based on documents in the file]

11. What was the seniority level of the initial lead PLS counsel:

  • LA0
  • LA1
  • LA2A
  • LA2B
  • LA3A
  • LA3B

12. Did the lead counsel change?

  • Yes – GO TO Q13
  • No – GO TO Q14

13. What was the seniority level of the final PLS counsel:

  • LA0
  • LA1
  • LA2A
  • LA2B
  • LA3A
  • LA3B
  • LA3C
  • Can’t tell

14. Was the PLS given a financial limit for fulfilling the request for legal services? [Matrix Q7]

  • Yes
  • No
  • Unable to assess
  • Not applicable, non-funded work for Justice

15. Did the PLS miss any client-imposed deadlines? [Matrix Q6A]

  • Yes
  • No
  • Unable to assess

16. (If yes to Q15) How many times? [Matrix Q6A]

17. (If yes to Q15) For what reason(s) were client deadlines missed (please include if an explanation to the client is available in the file and note if extensions were always or usually requested prior to missing deadlines)? [Matrix Q6A]

18. Did the PLS miss any court deadlines? [Matrix Q6A]

  • Yes
  • No
  • Unable to assess

19. (If yes to Q18) How many times? [Matrix Q6A]

20. (If yes to Q18) Were additional court procedures required (e.g., motions)? [Matrix Q6A]

  • Yes
  • No
  • Unable to assess

21. Did the PLS counsel consult with the Departmental Legal Services Unit? Consultations can include oral/written updates or oral/written discussions of legal risk or possible strategies, options, and approaches to the file. [Matrix Q5, Q6B, Q6G]

  • Yes
  • No - GO TO Q25
  • Unable to assess – GO TO Q25

22. Approximately how often did the PLS consult with the Departmental Legal Services Unit? (please check the most appropriate choice):

  • At least weekly
  • Bi-weekly
  • Monthly
  • Less than monthly
  • Only at critical times in the file
  • Unable to assess

23. Is there documentation in the file that shows what the consultations with the Departmental Legal Services Unit were about? (Check all that apply. If none apply, GO TO Q25.) [Matrix Q5, Q6B, Q6C, Q6G]

  • Identifying and assessing legal risk
  • The potential impact of legal risk
  • Ensuring consistent approach across government
  • Potential legal options
  • Potential litigation strategies
  • Seeking policy direction
  • Sharing information
  • Other
  • Unable to assess

24. (If you have identified any categories listed in Q23) What evidence is there in the files?

25. Did the PLS counsel consult directly with the client department/agency? Consultations can include oral/written updates or oral/written discussions of legal risk or possible strategies, options, and approaches to the file. [Matrix Q5, Q6B, Q6G]

  • Yes
  • No - GO TO Q29
  • Unable to assess – GO TO Q29

26. Approximately how often did the PLS consult directly with the client department/agency? Consultations can include oral/written updates or oral/written discussions of possible strategies, options, approaches to the file (please check the most appropriate choice): [Matrix Q5, Q6B, Q6G]

  • At least weekly
  • Bi-weekly
  • Monthly
  • Less than monthly
  • Only at critical times in the file
  • Unable to assess

27. Is there documentation in the file that shows what the consultations with the client were about? (Check all that apply. If none apply, GO TO Q29.) [Matrix Q5, Q6B, Q6C, Q6G]

  • Identifying and assessing legal risk
  • The potential impact of legal risk
  • Ensuring consistent approach across government
  • Potential legal options
  • Potential litigation strategies
  • Seeking policy direction
  • Sharing information
  • Other
  • Unable to assess

28. (If you have identified any categories listed in Q27) What evidence is there in the file?

29. Did the PLS counsel consult with any of the groups listed below? Consultations can include oral/written updates or oral/written discussions of legal risk or possible strategies, options, and approaches to the file. [Matrix Q5, Q6B, Q6G]

  • Regional office within Justice
  • Other section in PLS
  • Other potentially affected government departments/agencies
  • Provincial/territorial representatives
  • Canadian judiciary
  • Other countries’ representatives/counsel
  • Unable to assess
  • Other

30. Approximately, how often did counsel consult with other PLS sections? (please check the most appropriate choice):

  • No consultations with other PLS sections – GO TO Q33 [Matrix Q5, Q6B, Q6G]
  • At least weekly
  • Bi-weekly
  • Monthly
  • Less than monthly
  • Only at critical times in the file
  • Unable to assess

31. Is there documentation in the file that shows what the consultations were about? (Check all that apply. If none apply, GO TO Q33.) [Matrix Q5, Q6B, Q6C, Q6G]

  • Identifying and assessing legal risk
  • The potential impact of legal risk
  • Ensuring consistent approach across government
  • Potential legal options
  • Potential litigation strategies
  • Seeking policy direction
  • Sharing information
  • Other
  • Unable to assess

32. (If you have identified any categories listed in Q31) What evidence is there in the file?

33. Did the PLS receive urgent requests for its services (i.e., requests needed within a short period of time)?

  • Yes
  • No
  • Unable to assess [Matrix Q6A]

34. On average, how long were you given to respond to the requests? [Matrix Q6A]

  • Same day
  • 1 day
  • 2–3 days
  • 4–10 days
  • 11–20 days
  • 21–30 days
  • More than 30 days
  • Unable to assess

35. Based on the documentation in the file, were there any requests for PLS services that were inconsistent with PLS roles and responsibilities? [Matrix Q11]

  • Yes
  • No
  • Unable to assess

36. (If yes to Q35) Who made the requests that were outside of the PLS roles and responsibilities? (Check all that apply) Matrix Q11]

  • Other Justice counsel
  • Other government department/agency
  • Other

37. Based on the documentation in the file, were any requests for PLS services overly broad? [Matrix Q11]

  • Yes
  • No
  • Unable to assess

37A. If yes, please explain.

38. (If yes to Q37) Who made the requests that were overly broad? (Check all that apply) Matrix Q11]

  • Other Justice counsel
  • Other government department/ agency
  • Other

39. Based on the documentation in the file, did PLS counsel identify issues that should be referred to other areas within PLS and/or within Justice? [Matrix Q11]

  • Yes
  • No
  • Unable to assess

40. (If yes to Q39) Did PLS counsel bring those issues to the attention of the client/DLSU. [Matrix Q11]

  • Yes
  • No
  • Unable to assess

41. (If yes to Q39) Were the issues referred to the other areas within PLS and/or within Justice? [Matrix Q11]

  • Yes, always
  • Yes, sometimes
  • No
  • Unable to assess

42. Did the PLS services include any of the following? (Check all that apply). [Matrix Q6A, Q6B, Q6C]

  • Explanation of law
  • Discussion of legal risks
  • Potential impact to client
  • Potential impact to Government of Canada
  • Policy options
  • Legal options
  • Litigation strategies
  • Unable to assess

43. What quality control processes are evident in the file documentation? (Check all that apply). [Matrix Q6A]

  • Peer review of written materials
  • Use of practice directives
  • Use of templates
  • Mentoring
  • Supervisor review of written materials
  • Other
  • Unable to assess

44. At what level was the advice approved? [Matrix Q6A]

  • Section head
  • Assistant Deputy Minister
  • Associate Deputy Minister
  • Deputy Minister
  • Director
  • Deputy Director
  • Other
  • Unable to assess

45. Who received the advice? [Matrix Q6B]

  • Minister
  • Deputy Minister
  • Associate Deputy Minister
  • Assistant Deputy Minister
  • Other portfolios
  • Regional offices
  • Privy Council Office
  • Unable to assess
  • Other

46. Based on the documentation in the file, was the advice provided by the PLS considered by other Justice counsel when developing the legal strategies and giving advice to the client? [Matrix Q6G, Q6K]

  • Yes, always
  • Yes, sometimes
  • No
  • Not applicable to this file
  • Unable to assess

47. (If yes to Q46) What evidence is there in the file? [Matrix Q6G, Q6K]

48. Based on the documentation in the file, was the advice provided by the PLS considered by the client in its decision making? [Matrix Q6G, Q6K]

  • Yes, always
  • Yes, sometimes
  • No
  • Unable to assess

49. (If yes to Q48) What evidence is there in the file? [Matrix Q6G, 6K]

50. Based on the documentation in the file, did any situations arise where the PLS advice differed from the advice of the lead counsel? [Matrix Q6A]

  • Yes
  • No
  • Not applicable to this file
  • Unable to assess

51. (If yes to Q50) Based on the documentation in the file, how were these situations resolved? (Check all that apply) [Matrix Q6A]

  • PLS advice provided to client
  • Lead counsel advice provided to client
  • Client provided both positions
  • Meeting of counsel was held to arrive at Justice position
  • Issue taken to the National Legal Advisory Committee for discussion
  • National Litigation Committee
  • Other
  • Unable to assess

52. Was the file brought to the attention of senior Justice officials/structures? [Matrix Q6I]

  • Yes
  • No
  • Unable to assess

53. (If yes to Q52) Which ones? (Check all that apply.) [Matrix Q6I]

  • Minister/Deputy Minister
  • Portfolio ADMs
  • Regional Litigation Committees
  • Regional Director Generals
  • National Litigation Committee
  • Team leader
  • National Legal Advisory Committee
  • Policy Committee
  • Other

54. Is there documentation in the file that shows that any of the following were used/prepared? [Matrix Q6I]

  • Risk assessment document
  • Contingency plan
  • Communication plan (should be part of contingency plan, but check to ensure it is)
  • Briefing notes
  • Early Warning Note for file

55. Was the file included in any of the following? [Matrix Q6I]

  • Early Warning Reports
  • Top 100 High Impact Report
  • Radar Screen
  • Scanning News
  • Justice Practice Group discussion

56. Is there evidence in the file of PLS reviewing or helping to prepare communication products for use domestically to communicate the Government of Canada’s position? [Matrix Q6C, Q6E]

  • Yes
  • No
  • Unable to assess
  • Not applicable

57. (If yes to Q56) Which of the below products are in the file? [Matrix Q6C, Q6E]

  • Q&As/Talking points
  • Briefing Notes
  • Press release
  • Backgrounder
  • Media lines
  • QP note
  • Other

Questions 58–63 are for files involving international issues

58. Is there evidence in the file of PLS reviewing or helping to prepare communication products for use internationally to communicate the Government of Canada’s position? [Matrix Q6C, Q6E]

  • Yes
  • No
  • Unable to assess
  • Not applicable

59. (If yes to Q58) Which of the products listed below are in the file? [Matrix Q6C, Q6E]

  • Q&As/Talking points
  • Briefing Notes
  • Press release
  • Backgrounder
  • Media lines
  • QP note
  • Other

60. What international fora did the PLS attend? [Matrix Q6D]

  • Not applicable – GO TO Q62

61. (If applicable) What was the nature of the PLS’s involvement in these fora? [Matrix Q6D]

62. Is there evidence in the file of efforts made by PLS counsel to make:

  • domestic stakeholders outside of the federal government aware of the Government of Canada’s rights and obligations under international instruments? [Matrix Q6E]
    • Yes
    • No
    • Unable to assess
    • Not applicable
  • other federal officials aware of the Government of Canada’s rights and obligations under international instruments? [Matrix Q6E]
    • Yes
    • No
    • Unable to assess
    • Not applicable

63. (If yes to Q62) What evidence is there in the files? [Matrix Q6E]

Risk assessment [Matrix Q6A]

64. PLS was not involved in assessing legal risk.

  • Not applicable (GO TO Q75)

65. Potential client impact:

  • Affects administration of justice/public confidence
  • Affects federal, provincial, or international relations, treaties, or agreements
  • Legal issues or events that may be controversial, attract significant national media attention, or involve Cabinet Ministers or prominent public figures
  • Limitations of federal jurisdiction
  • Major effect on fiscal resources of client or government
  • Major effect on human rights, personnel, access and privacy, gender, or diversity issues
  • Major effect on law/ regulations of client or government
  • Major effect on programs/ policies/initiatives of client or government
  • Major effect on relations with Aboriginal people, Métis
  • Major effect on the Charter or Constitution
  • Matter of national interest
  • Not applicable
  • Unable to assess
  • Other

66. Does the file include a discussion/indication of risk level with respect to the file as a whole or specific aspects of the file?

  • Yes
  • No (GO TO Q75)

67. Were equivalent levels of risk described consistently throughout the file?

  • Yes
  • No
  • Unable to assess

68. (If no to Q67) What language was used to describe the level of legal risk to clients? Please give verbatim examples that show the range of language used.

69. Based on your review of the file, was the highest legal risk level for this file:

  • Low
  • Medium
  • High
  • Can’t tell

70. At what stage in the case was the initial (or only) risk assessment done?

  • Post-pleadings
  • Post-discovery
  • Pre-scheduled trial date
  • After decision
  • After appeal filed
  • Can’t tell
  • Other, please specify

71. Was the level of legal risk reassessed?

  • Yes
  • No (GO TO Q75)
  • Can’t determine (GO TO Q75)

72. Why was the level of legal risk reassessed?

  • Further information provided by client
  • Further consultation with stakeholders
  • Evolution in jurisprudence
  • Policy modifications
  • Evolution of government priorities
  • Can’t tell
  • Other, please specify

73. If case was reassessed to a higher risk level, did any of the following occur after the reassessment?

  • Increased number of counsel on file
  • Assignment of senior counsel to file
  • (Litigation only) Consideration of dispute resolution process
  • (Litigation only) Use of dispute resolution process
  • Increased consultations
  • Increased reporting
  • Can’t tell
  • Other, please specify

74. At what stage in the case was the risk reassessed?

  • Post-pleadings
  • Post-discovery
  • Pre-scheduled trial date
  • After decision
  • After appeal filed
  • Can’t tell
  • Other, please specify

iCase information –Matrix Q7 and Q8]

75. How many hours did lead PLS counsel spend on the file? x hours

76. Indicate the number of additional PLS counsel on the file by seniority level and indicate the number of hours spent on the file:

For JLT: Please reflect counsel in DFAIT positions at the roughly equivalent LA level (e.g., FS-03 and CO-3s are equivalent to LA-2As; CO-2s are equivalent to LA-01).

Type # of counsel Hours for each counsel

  • LA0 Hours for 1: 2: 3: 4: 5:
  • LA1 Hours for 1: 2: 3: 4: 5:
  • LA2A Hours for 1: 2: 3: 4: 5:
  • LA2B Hours for 1: 2: 3: 4: 5:
  • LA3A Hours for 1: 2: 3: 4: 5:
  • LA3B Hours for 1: 2: 3: 4: 5:
  • LA3C Hours for 1: 2: 3: 4: 5:
  • Other Hours for 1: 2: 3: 4: 5:

77. Indicate the number of paralegals on the file and indicate the number of hours spent on the file by each paralegal:

  • Hours for 1: 2: 3: 4: 5:

Any additional comments (indicate applicablequestion, if appropriate):

Date modified: