Legislative Services Branch Evaluation

Appendix A: Evaluation Matrix

1. Relevance
Evaluation Issue Question Performance Indicator Measures Data Sources/ Methods
Continued Need for the LSB 1.1 To what extent is the LSB responsive to the legislative drafting needs of the Government of Canada? Extent to which the LSB is relevant to the legislative drafting needs of the Government of Canada

Comparison of ongoing Government of Canada (GoC) legislative drafting requirements to the types of services offered by the LSB

Perceived relevance of LSB services to the needs of the GoC

Document review

LSB departmental clients/ survey, interviews

Justice legal staff/ interviews

PCO, TBS policy makers/interviews

LSB staff/interviews, survey

LSB client case files/analysis

1.2 Has the demand for legislative services changed and how? Nature of/extent to which demand for legislative services has changed

Trends in client requests for legislative services by type of service

Changes in types of requests

New or special requests

LSB departmental clients/ survey, interviews

Justice legal staff/ interviews

PCO, TBS policy makers/interviews

LSB staff/interviews, survey

LSB client case files/analysis

LSB administrative records/analysis

Alignment with Government Priorities 1.3 To what extent are the activities of the LSB aligned with: 1. GoC priorities? and 2. strategic outcomes of Justice? Extent to which LSB activities are aligned with GoC priorities

Consistency between LSB activities and legislated responsibilities

Changes made to LSB activities to maintain alignment with changes to GoC priorities

Document review

LSB managers/ interviews

Justice legal staff/ interviews

PCO, TBS policy makers/interviews

Extent to which LSB activities are aligned with the strategic outcomes of Justice Consistency between LSB activities and Justice strategic outcomes

Document review

LSB managers/ interviews

Justice legal staff/ interviews

Alignment with Federal Roles and Responsibilities 1.4 To what extent do the activities of the LSB align with federal roles and responsibilities? Extent to which LSB activities are aligned with the federal government’s role and responsibilities.

Alignment of the responsibilities with the legislated authority of the GoC (and outside of P/T legislated authority)

Alignment of staff roles/responsibilities with the responsibilities of the LSB

Document review

Performance

Achievement of Expected Outcomes

2. Direct Outcomes
Evaluation Issue Question Performance Indicator Measures Data Sources/ Methods

Effectiveness

2.1 To what extent has the LSB contributed to the creation of legislation that is responsive to federal government policy directions? Responsiveness of the LSB to current GoC needs and policy directions

Ability to respond to needs of the GoC

Harmonization files completed

Policy developed related to legislative and regulatory processes (with PCO and TBS)

Streamlined regulations

Services provided to the PCO

LSB staff/interviews

PCO, TBS policy makers/interviews

LSB administrative records

LSB client case files/analysis

Case studies

Responsiveness of the LSB to current client needs

Ability to respond to requests/needs of Justice

Ability to respond to requests/needs of clients

Drafted legislation completed

Drafted regulations completed

LSB departmental clients/ interviews, survey

PCO, TBS policy makers/ interviews

Justice legal staff/ interviews

LSB staff/interviews, survey

LSB administrative records

LSB client case files/analysis

Case studies

Provision of legislative services, by type of services provided

Number/type/nature of legislative products created in response to federal government policy directions by type of service provided

  • drafting, harmonization, support to SRC, legal advice
  • advising on/developing federal policy regarding regulatory and drafting processes

LSB administrative records/ analysis

PCO, TBS policy makers/interviews

Extent to which LSB clients are satisfied with the responsiveness of LSB services, by type of service provided

Client satisfaction with responsiveness of LSB staff by type of service

  • drafting, harmonization, support to SRC, legal advice
  • advising on/developing federal policy regarding regulatory and drafting processes

LSB departmental clients/ interviews, survey

PCO, TBS policy makers/interviews

Case studies

Extent to which the LSB has contributed to the creation of legislative products that express intended federal government policy directions

Client experience working with LSB

Alignment of requests with services provided and results

Contribution of legislative products created to policy directions

LSB departmental clients/ interviews

LSB client case files/analysis

Case studies

2.2 To what extent has the LSB contributed to the creation of federal legislation that is accessible to Canadians? Extent to which federal legislation drafted is clear and consistent in both official languages, while respecting the civil law and common law in private matters

Type and adequacy of quality assurance protocols/procedures, etc. in place to ensure clarity and consistency of language, and civil and common law traditions

Capability of staff (e.g., credentials, training received) to draft legislation to meet bijural and bilingual requirements

Clarity of language and consistency between French and English

LSB staff/interviews, survey

LSB staff/PSES

Documentation/review

LSB client case files/analysis

LSB departmental clients/ interviews, survey

Other legal professionals/ interviews

Nature/extent of use of Justice Laws website

Justice Laws website usage

  • number of hits, repeat users, areas targeted
Website statistics/ analysis
Extent to which legislation is available/accessible in appropriate formats

Format in which federal legislative products are provided

Ease of accessing legislation in the formats available

Satisfaction with formats available

Publication/dissemination schedules/analysis

LSB departmental clients/ interviews, survey

Other legal professionals/interviews

2.3 To what extent has the LSB enhanced awareness/ understanding of federal legislation, legislative principles, processes and options by federal departments and agencies? Level of awareness/ understanding within the federal government of federal legislation, legislative principles, processes and options by target audience Footnote 66

Client awareness of relevant federal legislation, legislative principles, processes and options

Satisfaction with training received (content, clarity and relevance)

Training files/feedback results/analysis

LSB departmental clients/ interviews, survey

2.4 To what extent has the LSB enhanced its capacity (skills, tools, information) to deliver consistent federal legal and legislative products? Evidence of activities undertaken to enhance capacity in the LSB to deliver consistent products to its clients and to the GoC

Number/type/nature of quality improvement and capacity building efforts planned/undertaken by the LSB

Number/type/nature of capacity enhancing tools developed/applied (e.g., Charter check list)

Document review

LSB staff/interviews

Extent to which LSB staff perceives Branch capacity has been enhanced through internal training and knowledge management tools

Staff satisfaction with training received (content, relevance)

Staff satisfaction and use of materials designed to enhance consistency

Staff satisfaction with quality control processes/protocols within the LSB

Training files/feedback results/analysis

LSB staff/interviews, survey

PSES results (Q20)

Extent to which other recipients of LSB capacity building efforts perceive capacity is enhanced Client satisfaction with training received (content, relevance)

Training files/feedback results/analysis

Justice legal staff/ interviews

LSB departmental clients/interviews, survey

3. Intermediate Outcomes
Evaluation Issue Question Performance Indicator Measures Data Sources/ Methods
Effectiveness 3.1 To what extent has the LSB contributed to the creation of federal legislation that respects the Constitution and other legal requirements? Nature of/extent of LSB measures to ensure that legislative drafts respect the Constitution and other legal constraints

Number/type/nature of Charter challenges Footnote 67

Type and adequacy of quality assurance measures in place

LSB client case files/analysis

LSB staff/interviews, survey

Case Studies

3.2 To what extent has the LSB contributed to a bilingual and bijural federal legislative framework? Extent to which the bijuralism website is used

Justice Laws, Bijuralex.ca usage by type of user

  • number of hits, repeat users, areas targeted
Web site statistics/analysis
Extent to which LSB has contributed to a bilingual and bijural federal legislative framework Experiences/perceptions of committee members and federal government departments representatives

Document Footnote 68 review

LSB departmental clients/ interviews

PCO, TBS policy makers/interviews

Justice legal staff/ interviews

Case studies

3.3 To what extent has the LSB contributed to government departments and agencies being better able to manage their legal risks? Extent of LSB contribution to legal risk management

Number/percentage of files where legal risk is reported as high and nature of strategies used to manage/mitigate risk

Avoidance of legal risks

Perceptions of government departments and agencies on LSB contribution to legal risk management

Perceptions of Justice officials (e.g. DLSU) of LSB contribution to legal risk management

LSB administrative records/analysis

LSB client case files/analysis

LSB departmental clients/ interviews, survey

Justice legal staff/ interviews

LRM evaluation/analysis

Case studies

4. Ultimate Outcomes
Evaluation Issue Question Performance Indicator Measures Data Sources/ Methods
Effectiveness 4.1 To what extent has the LSB contributed to a federal government that is supported by effective and responsive legal services? Cumulative Achievement of direct and intermediate outcomes

Cumulative/contribution analysis

LSB staff/interviews

PCO, TBS policy makers/interviews

Justice legal staff/ interviews

4.2 To what extent has the LSB contributed to a fair, relevant and accessible justice system that reflects Canadian values? Cumulative Achievement of direct and intermediate outcomes

Cumulative/contribution analysis

LSB staff/interviews

PCO, TBS policy makers/interviews

Justice legal staff/ interviews

5. Demonstration of Efficiency and Economy
Evaluation Issue Question Performance Indicator Measures Data Sources/ Methods
Efficiency and Economy 5.1 Has the LSB’s resource utilization been appropriate, in relation to the resources allocated, activities and outputs produced and demands for services, by area of service delivery? Budget and work plan priorities and activities in relation to outputs delivered by service area

Cost of legal inputs for each service line/type of service

Resources used to produce request by type of request and by service line

Sufficiency (i.e., quality and quantity) of planned/allocated resources to support tasks required

Redundancy, duplication, surplus of resources

Analysis of planned versus actual inputs/resources used

Document review

LSB financial database/ analysis

LSB administrative records/ analysis

LSB staff/interviews, survey

Case studies

5.2 Is the LSB’s capacity to deliver services sufficient given current and future demand for services, by area of service delivery? Level of service in relation to demand for services (past, current, projected)

Volume of requests per staff

Volume of requests per service line

Ability to complete requests (within reasonable timelines)

Over or under-commitment of resources that compromised the quality, quantity, appropriateness or timeliness of required outputs

Future human resource needs (churn/vacancies)

Changes that have been made to increase the level of outputs produced with resources allocated

LSB administrative records/analysis

Documentation/review

LSB staff/survey, interviews

LSB staff/PSES

LSB departmental clients/ interviews, survey

5.3 Are the LSB’s activities sufficiently integrated and coordinated to support the achievement of results? Extent to which integration of LSB activities supports the achievement of expected results

Level of coordination/integration of activities within LSB, Justice and LSB client departments and agencies

Satisfaction with the level of coordination

LSB staff/ interviews/survey

Justice staff/interviews

5.4 To what extent has the LSB been able to use IT support and technology to improve efficiency and effectiveness of service delivery, by area of service delivery? Extent to and nature in which IT innovation has been developed/applied by service area

IT supports in place

Effects of IT on administrative practices/service delivery

LSB files/analysis

LSB staff/ interviews/survey

Extent to which LSB use of IT support and technology is consistent with or comparable to other jurisdictions

Comparison of LSB IT support and technology to that in other jurisdictions

Trends in use of IT support and technology in legislative services/ analysis

LSB staff/interviews

LSB files/analysis

Literature/environmental scan

5.5 Does the financial model Footnote 69 utilized by the LSB provide appropriate and sustainable funding for the LSB to meet current and future demand for legislative services? Sufficiency of A-base funding to discharge the Minister’s responsibilities Footnote 70

Resource coverage by A-base funding

Proportion of LSB work completed that is not cost recoverable (relative to amount of A-base funding)

Gap between financial inputs and resource requirements for outputs

LSB managers/interviews

LSB files/data

Financial documentation/ review

Trends in financial models for supporting/ funding legislative services

Examples from other Justice branches/services (e.g., Public Law Sector)

Examples from other jurisdictions that provide similar types of legal services within government

Comparison of alternative models with the LSB model (with respect to federal Canadian needs)

Literature/environmental scan

Financial documentation/review

LSB managers/ interviews

Finance staff/interviews

5.6 Are there alternative ways of delivering the LSB’s services by areas of service delivery? Trends in practices/ service delivery models

Examples from other jurisdictions

Comparison of alternative models with federal Canadian needs

Literature/environmental scan

LSB staff/interviews

Date modified: