Criminal Law Policy Function Evaluation

Appendix A: Evaluation Matrix

Evaluation of the Criminal Law Policy Function (including the support provided by the Research and Statistics Division)

Relevance (TB core issues 1–3)
Issues/Questions Indicators Data Sources Responsibility for Collecting
1. Do the activities of CLPS align with Department of Justice's strategic objectives?

Perception of whether CLPS activities align with the Department of Justice's strategic objectives and core expected results

  • Key informant interviews
  • Document review

Department of Justice Evaluation Division

2. Do the activities of CLPS align with federal priorities/policy commitments?

Perception of whether CLPS activities align with federal priorities

  • Key informant interviews
  • Document review

Department of Justice Evaluation Division

3. What need is the legal policy work of CLPS intended to address? Is there evidence of an ongoing need?

Legal trends/policy gaps/emerging issues (identified policy needs on issues based on case law, media scan, consultations, international fora)

  • Document review
  • Key informant interviews

CLPS

Department of Justice Evaluation Division

Perception of continued need for legal policy work

Key informant interviews

Department of Justice Evaluation Division

4. To what extent are the activities of CLPS appropriate to the federal government and a core federal role?
  • Constitutional and statutory authority for federal involvement
  • Constitutional and statutory authority for Department of Justice involvement

Document review

Department of Justice Evaluation Division

Activities of CLPS that fall outside of the role of the federal government under constitutional and statutory authority

Key informant interviews

Department of Justice Evaluation Division
Achievement of Expected Outcomes (TB core issue 4) - Immediate Outcomes
Issues/Questions Indicators Data Sources Responsibility for Collecting
5. To what extent has CLPS achieved its expected outcomes?

Enhanced knowledge and understanding of domestic and international criminal law issues:

  • Number and type of training sessions and other activities to promote education and awareness (i.e training for individual lawyers within the Dept/Section, public, CJ professionals)
  • Perceptions of effectiveness of training/educational sessions conducted by CLPS
  • Post-training surveys/briefings
  • Key informant interviews
  • Document review
Department of Justice Evaluation Division
  • Perceptions of how (1) proactive monitoring of the policy environment, (2) research, and (3) stakeholder engagement contributes to an enhanced knowledge and understanding of domestic criminal law issues and international instruments dealing with criminal law matters

Key informant interviews

Department of Justice Evaluation Division
  • Key stakeholder groups with whom CLPS teams consult/collaborate
  • Number and nature of meetings/consultations with stakeholders

Document review

CLPS
  • Perceptions of effectiveness of partnerships with stakeholders
  • Satisfaction of key groups with engagement approaches

Key informant interviews

Department of Justice Evaluation Division

Documented results of collaborative work

  • Document review
  • Key informant interviews
Department of Justice Evaluation Division

High- quality legal and policy advice on domestic and international criminal law issues:

  • Perceptions regarding quality of legal and policy advice (Consistency, Timeliness, Responsiveness, and Usefulness)
  • Perceptions of stakeholders on CLPS contribution to legal risk management
  • Use of research and statistics
  • Satisfaction of CLPS with RSD services
  • Number and nature of Research and Statistics Division (RSD) products that support CLPS' work
  • Usefulness /timeliness of RSD's research products and support
  • Nature and extent to which legal and policy advice is informed by stakeholder engagement
  • Nature of CLPS contribution to legislative drafting process
  • Key informant interviews
  • Case studies
Department of Justice Evaluation Division
  • Number and nature of corrective amendments (to legislation supported by CLPS)
  • Nature and extent of CLPS engagement in legal challenges to legislation or policy
  • Extent to which appropriate Sections (HRLS, Public International Law, CALS) are engaged at appropriate time
  • Effort by CLPS in developing legal arguments to address Charter challenges (hours spent conducting litigation support, type of activities)

Document review

CLPS
Achievement of Expected Outcomes (TB core issue 4) - Intermediate Outcomes
Issues/Questions Indicators Data Sources Responsibility for Collecting
5. To what extent has CLPS achieved its expected outcomes?

Contribution to the domestic and international criminal law framework:

  • Domestic legislation and policy developed by CLPS (bills tabled, legislation passed, Memoranda to Cabinet)
  • Types of assistance with implementing law reforms
  • Extent stakeholders believe they are supported in the implementation of new policy/law reforms
  • Perception that CLPS legal policy advice has contributed to domestic criminal law framework
  • Document review
  • Key informant interviews
  • Case studies
Department of Justice Evaluation Division
  • Perception that CLPS legal policy advice has been effective in facilitating the consideration of Canadian interests and the inclusion of Canadian approaches in international instruments dealing with criminal law matters
  • Key informant interviews
  • Case studies
Department of Justice Evaluation Division
  • Nature and extent of international criminal law policy work (international fora attended, negotiations where legal policy advice was provided)
  • Nature and extent of international peer reviews
  • Nature and extent of direct technical assistance internationally
  • Examples where CLPS policy work was incorporated in international instruments dealing with criminal law matters
  • Assistance with implementing international instruments
  • Document review
  • Key informant interviews
  • Case studies

CLPS

Department of Justice Evaluation Division

Government decision making is informed by legal and policy advice:

  • Perception that CLPS' legal and legal policy advice enables the government to make well-informed decisions regarding government legislation, private members bills and parliamentary reviews.
  • Observed use and nature of the use of legal and policy advice
  • Assessed degree of value for policy services and products
  • Demand for policy function services
  • Key informant interviews
  • Case studies
Department of Justice Evaluation Division
Achievement of Expected Outcomes (TB core issue 4) - Ultimate Outcome
Issues/Questions Indicators Data Sources Responsibility for Collecting
5. To what extent has CLPS achieved its expected outcomes?

A fair, relevant and accessible Canadian justice system:

Perception of CLPS contribution

Key informant interviews

Department of Justice Evaluation Division

A federal government that is supported by high-quality legal services:

Perception of CLPS contribution

Key informant interviews

Department of Justice Evaluation Division
6. Does CLPS have in place appropriate methods/systems for monitoring performance and reporting on outcomes?

Extent to which appropriate mechanisms are in place to monitor performance and report on outcomes

  • Document review
  • Key informant interviews
Department of Justice Evaluation Division
Efficiency and Economy (TB core issue 5)
Issues/Questions Indicators Data Sources Responsibility for Collecting
7. Could the work of CLPS be undertaken/conducted more efficiently and economically?
  • Evidence of strategies to achieve outputs and outcomes in the most cost-effective way
  • Assessment of the effectiveness of CLPS' organizational structure
  • Measures in place to manage efficiency
  • Suggestions for improvement in efficiency of performance
  • Alternative approaches to service delivery
  • Key informant interviews
  • Case studies
  • Document review
Department of Justice Evaluation Division
8. Has CLPS' resource utilization been appropriate, in relation to the resources allocated, activities and outputs produced and demands for services, by area of service delivery?
  • Assessment of resource allocation and utilization in relation to activities, outputs and results achieved
  • Document review
  • Key informant interviews
Department of Justice Evaluation Division
9. Are CLPS' activities sufficiently integrated and coordinated to support the achievement of results?
  • Level of coordination/integration of CLPS activities
  • Identified mechanisms/structures that facilitate information exchange internally and externally, and improvements that could strengthen collaboration
  • Key informant interviews
  • Document review
  • Case studies
Department of Justice Evaluation Division
10. Are there any best practices or lessons learned in the delivery of CLPS services?
  • Evidence of best practices and lessons learned
  • Key informant interviews
  • Case studies
  • Document review
Department of Justice Evaluation Division
11. Have the activities of CLPS led to any unintended or unanticipated impacts?
  • Instances of unintended impacts and their effects
  • Key informant interviews
  • Case studies
Department of Justice Evaluation Division
12. Is CLPS facing any resource challenges in terms of available funding, organizational structure/staffing, or internal support (training, technological, research) that are affecting its ability to conduct its policy work and achieve its expected outcomes?
  • Number of staff by category/team by year
  • Funding by year
  • Appropriate workload indicator by year – e.g., requests for legal policy advice, volume of documentation produced such as briefing notes, Memoranda to Cabinet, etc.)
  • Training/professional development by year

Document review

CLPS
  • Description of allocation and resource demands
  • Sufficiency of resources (human, financial, technical, competencies, training) to meet demand for services
  • Key informant interviews
  • Document review
Department of Justice Evaluation Division
Date modified: