Project Managers' Guide to Performance Measurement and Evaluation


Family Violence Initiative — Department of Justice Canada Logic Model

Policy Development
Activity / Outputs Reach / Client Groups Direct Outcomes Intermediate Outcomes Long Term Outcomes
  • Coordinated FV workplan

  • FV policy advice

  • FV policy information

  • FV policy products

  • Reports on consultations

  • Legal advice, litigation support and drafting advice

  • Canadian positions on international conventions related to family violence

  • National:

  • Minister of Justice (F)

  • Ministers of Justice / Attorneys General (PT)

  • FPT officials

  • FPT Justice community

  • FV Policy community

  • FV NGOs and community groups

  • International:

  • International bodies (e.g. United Nations)

  • Increased, coordinated FV criminal justice policy development across jurisdictions and internationally

  • FV-relevant legislation and policies

  • Canadian "position" developed in consultation with relevant client groups with respect to international conventions involving family violence issues

Implementation of FV criminal justice policy instruments across FPT jurisdictions and at international level

Improved responsiveness of the criminal justice system to address family violence.


Programs (PIF)
Activity / Outputs Reach / Client Groups Direct Outcomes Intermediate Outcomes Long Term Outcomes
  • Project needs

  • Funded projects [8]

  • Contracts

  • Partnerships & partnership contributions

  • Tested models

  • Strategies and tools

  • Project reports, evaluations, and products

  • FVI / DOJ Website updates in relation to family violence activities

  • Project activities workplans

  • Funding decisions

  • Report on funded activities

  • Funding application tools

  • Distribution plans

  • Performance reports

  • National:

  • Departmental staff

  • FPT officials

  • Municipal governments

  • FVI NGO's

  • Individuals and communities facing barriers to access to justice

  • General public

  • International:

  • International bodies (e.g. United Nations)

  • Canadian organizations affiliated to international bodies

  • Increased awareness and knowledge of family violence issues and effective solutions

  • Improved organizational capacities to address FV

  • Tested pilot projects, proven models, strategies, and tools to address FV

  • Contribution to policy and program development

  • Improved community capacities to address FV

Improved responsiveness of the criminal justice system to address family violence.


PLEI
Activity / Outputs Reach / Client Groups Direct Outcomes Intermediate Outcomes Long Term Outcomes
  • Funded projects [9]

  • Contracts

  • Partnerships & partnership contributions

  • Tested models

  • Strategies and tools

  • Project reports, evaluations, and products

  • Website updates

  • Project activities workplans

  • Funding decisions

  • Report on funded activities

  • Funding application tools

  • Distribution plans

  • PLEI organizations

  • Intermediary groups [10]

  • Clients facing barriers in accessing public legal education information on family violence

  • Criminal justice professionals

  • Increased ability of intermediary groups to advise / support clients facing barriers in accessing public legal education information on family violence

  • Increased access to public legal education information on family violence

  • Increased awareness / knowledge of legal issues in family violence situations for clients facing barriers in accessing public legal education information on family violence

  • Increased prevention of FV

  • Increased protection of FV victims / witnesses

  • Improved response to FV situations

  • Increased capacity of clients to make informed decisions about their particular situation

  • More engaged public and legal communities in matters related to family violence

Improved responsiveness of the criminal justice system to address family violence.


Research
Activity / Outputs Reach / Client Groups Direct Outcomes Intermediate Outcomes Long Term Outcomes
  • Annual DOJ research workplans

  • Contracts

  • Funding / partnership agreements

  • Research reports

  • Statistical reports

  • Policy-relevant information

  • Presentations

  • Publications

  • Other related information products (e.g. reference documents, summaries, briefing notes or bulletins)

  • Federal Minister of Justice

  • DOJ Family Violence Committee

  • FPT justice officials

  • FV policy community

  • Other relevant clients may include NGOs, general public, international bodies (e.g. United Nations)

  • Dissemination of research results to relevant client groups

  • Awareness of DOJ family violence research results by relevant client groups

  • Increased understanding of family violence issues

  • Increased knowledge on effective interventions to respond to FV

  • Informed policy analysis, development and decision making

Improved responsiveness of the criminal justice system to address family violence.


Family Violence Initiative — Department of Justice Canada Reporting Template
Activity Status DOJ FVI Budget Overview Partners & Contribution Client Groups Outputs Results-Direct Outcomes
  • Title of Activity or Project

  • Objective: To X…..[2 lines max]

  • Description: 2 lines max indicating type of project.

Indicate if underway or completed, and anticipated or actual completion date.

  • Indicate confirmed DOJ / FVI funding or contract value for each year of the project, if a multi-year project, e.g.

  •  

  • 99 / 00: $25K

  • 00 / 01: $15K

Confirmed Partners & contribution: to be provided in cases where DOJ is the LEAD department for the activity.

 

  • Who is influenced by the activity /

  • scope of reach.

  • List outputs e.g.

  • Workshop

  • Protocol

  • Reports,

  • Trained personnel

  • For Completed Activities

  • Refer to the FVI logic model direct outcomes.

  • Briefly describe the direct outcomes (5 line – 50 words); include any pertinent comments or analysis on the results & implications for future action, strategic direction.

  • For Activities Underway:

  • Briefly describe the anticipated direct outcomes. (5 line – 50 words);


Family Violence Initiative: Performance Measurement and Evaluation Strategy
Success / Achievement of Outputs and Outcomes

Policy Development
Outputs(SUMBAL-ON Monday)
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Coordinated FV workplan

  • FV policy advice

  • FV policy information

  • FV policy products

  • Reports on consultations

  • Legal advice, litigation support and drafting advice

  • Canadian positions on international FV initiatives

Annual FV workplan developed in consultation with Programs & Research Workplan Files / Document Analysis Policy – Legal Counsel Annual [11]
Participation in the development of coordinated federal approach to FV Meeting Agendas, minutes, Files / Content Analysis Policy – Legal Counsel Annual
Participation in the development of federal and DOJ policy initiatives related to family violence issues Meeting Agendas, minutes, Files / Content Analysis Policy – Legal Counsel Annual
Client groups provided policy advice, information and products by type and nature of advice Files & Products / Content Analysis Policy – Legal Counsel Annual
Consultation reports produced by issue, including groups / stakeholders consulted Files & Reports / Content Analysis Policy – Legal Counsel Annual
Client groups provided legal advice, litigation support and drafting advice by nature of advice Files / Content Analysis Policy – Legal Counsel Annual
Title / status of international FVI-related initiatives underway Files & Documents / Content Analysis Policy – Legal Counsel Annual
FVI DOJ role / representation on FVI-related international initiatives Files & Documents / Content Analysis Policy – Legal Counsel Annual

Direct Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Increased, coordinated FV criminal justice policy development across jurisdictions and internationally

  • FV-relevant legislation and policies

  • Canadian developments with respect to international FV issues

Level and nature of coordinated FV criminal justice policy development across Canada and internationally

  • Files & Documents / Content Analysis

  • Opinions, Documents, Literature, Web-based information

  • Scan of FV policy development activities across Canada (& internationally) [12]

Policy – Legal Counsel & Evaluation Division

  • Baseline Study

  • Year 3

Type / status of legislation

Files & Documents / Content Analysis

Policy – Legal Counsel

Annual

Type / status of developments (including negotiations) on international FV initiatives

Files & Documents / Content Analysis

Policy – Legal Counsel

Annual


Intermediate Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

FV implementation of criminal justice policy instruments across FPT jurisdictions and internationally

Extent / nature of implementation of criminal justice policy instruments at FPT and international levels

Follow-up FV Policy Scan

  • Policy – Legal Counsel

  • Evaluation Division

Year 5


Programs (PIF)
Outputs
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Project needs

  • Funded projects [13]

  • Contracts

  • Partnerships & partnership contributions

  • Project reports, evaluations, and products

  • Strategies and tools

  • FVI / DOJ Web site updates

  • Project activities workplans

  • Funding decisions

  • Report on funded activities

  • Funding application tools

  • Distribution plans

  • Performance reports

  • Workplan developed in collaboration with Policy and Research

  • #, type and status of funded project and contract products, including distribution plans

  • #, type of partnerships and value of partnership contributions by funded project

  • # / type of strategies and tools, [14] including # / type of trainings, workshops

  • #, type and status of DOJ FVI web site updates

  • # of visits or hits on DOJ FVI web site

  • #, nature and type of information downloads from DOJ FVI web site

  • Workplan Files / Document Analysis

  • GCIMs and Project Files, Reports and Evaluations / Analysis

  • Contract Files / .Analysis

  • Web site management data / Analysis

  • Programs – Project Manager

  • FCY Web site Manager

Annual


Direct Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

Improved organizational capacities to address FV

Organizational / systemic infrastructure to address FV within organizations that received DOJ FVI funding

  • GCIMs and Project Reports, Evaluations and Files / Analysis

  • Lessons Learned / Analysis

  • Programs - Project Managers

  • & Evaluation Division Project Manager

  • Annual

  • Lessons Learned Study Year 2

Level of knowledge, expertise, skills and resources to address FV within organizations that received DOJ FVI funding

  • GCIMs and Project Files / Analysis

  • Lessons Learned / Analysis

  • Programs - Project Managers

  • & Evaluation Division Project Manager

  • Annual

  • Lessons Learned Study Year 2

Extent and use of partnerships (formal and informal) within organizations that received DOJ FVI funding

  • GCIMs and Project Files / Analysis

  • Lessons Learned / Analysis

  • Programs - Project Managers

  • & Evaluation Division Project Manager

  • Annual

  • Lessons Learned Study Year 2

# and type of tested models and potential for effectiveness

GCIMS and Project Files, Reports and Evaluations

Programs [15] - Project Manager

Annual

Level of organizational awareness and knowledge of FV issues and effective solutions

  • GCIMs and Project Files / Analysis

  • Opinions /

  • Lessons Learned

Programs – Project Manager & Evaluation Division Project Manager

  • Annual

  • Lessons Learned Study Year 2


Intermediate Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

Improved community capacities to address family violence

Level of community capacity

  • GCIMS and Project Files, Project Reports and Evaluations

  • Opinions / Lessons Learned

Programs – Project Manager and Evaluation Division Project Manager

  • Annual

  • Lessons Learned Study Year 2

Enhanced implementation of tested pilot projects, proven models, strategies, and tools to address FV

Level / success of implementation of tested and proven mechanisms

  • GCIMS and project Files / . Project Reports and Evaluations / Analysis

  • Research Reports

Programs – Project Manager Research – Project Manager (?)

Annual

Contribution to policy and program development

Consideration of implementation results in policy and program development

Opinion / Interviews / Focus Groups or other feedback from relevant client groups

Programs – Project Manager

Annual


Public Legal Education Information (PLEI)
Activity / Outputs
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Funded projects [16]

  • Contracts

  • Partnerships & partnership contributions

  • Tested models

  • Strategies and tools

  • Project reports, evaluations, and products

  • Website updates

  • Project activities workplans

  • Funding decisions

  • Report on funded activities

  • Funding application tools

  • Distribution plans

  • #, type and status of funded PLEI projects and contract products

  • #,scope (e.g. to whom, where) distribution plans for PLEI Products

  • # / type of PLEI products distributed

  • GCIMs

  • Project files, Reports and Evaluations

PLEI – Project Manager

Annual


Direct Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Increased ability of intermediary groups to advise / support PLEI clients facing barriers in accessing public legal education information on family violence

  • Increased access to public legal education information on family violence

  • Increased awareness / knowledge of legal issues in family violence situations for PLEI clients facing barriers in accessing public legal education information on family violence

  • Level / type of demand for products

  • # of products distributed

  • Level / type / scope of access to PLEI information / services

  • Level of awareness / knowledge of legal issues in FV situations among PLEI clients facing barriers

  • # of request for PLEI requests for information on web site

  • GCIMS

  • Project files, reports and evaluations

  • Opinions / Lessons Learned

  • DOJ FVI web site

  • PLEI – Project Manager

  • Evaluation – Project Manager

  • FCY Web Manager

  • Annual

  • Year 2


Intermediate Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Increased prevention of FV

  • Increased protection of FV victims / witnesses

  • Improved response to FV situations

  • Increased capacity of individuals to make informed decisions about their particular situation

  • More engaged public and legal communities in matters related to family violence

  • Level, nature and scope of family violence

  • Level / nature of protection provided

  • Nature of response to FV

  • Level / nature of knowledge / skills of relevant clients in making decisions re FV

  • Extent of public and legal community engagement

  • GCIMs / Project Files

  • Lessons Learned Study Year 2

PLEI – Project Manager

  • Annual

  • Lessons Learned Study Year 2


Research
Outputs
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Annual DOJ research workplans

  • Contracts

  • Funding / partnership agreements

  • Research reports

  • Statistical reports

  • Policy-relevant information

  • Presentations

  • Publications

  • Other related information products (e.g. reference documents, summaries, briefing notes or bulletins)

  • DOJ FVI research plan developed in collaboration with Policy and Programs

  • #, type, subject matter and nature / complexity of research projects

  • Title / status of contracts, agreements

  • Title / status of products and related activities e.g. presentations, publications

  • Level of demand for information on a particular family violence issue

  • Workplan

  • RIMs

  • Project Files / Content Analysis

  • Product Requests / Analysis (please clarify how these are tracked)

Research – Project Manager

Annual


Direct Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

Dissemination of research results to relevant client groups

Relevant client groups to whom materials were disseminated by mechanism used (e.g. web site (intra and internet), targeted mail out)

  • Distribution / Dissemination Lists

  • Web site "hits" and "downloads" [17]

Research – Project Manager

Annual

Awareness of DOJ family violence research results by relevant client groups

Level of awareness of research outputs among relevant client groups

Client Survey, and / or Focus Group

  • Research – Project Manager

  • Evaluation Division – Project Manager

Year 3


Intermediate Outcomes
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

Increased understanding of family violence issues.

Level of understanding of the effectiveness of family violence issues among relevant client groups

Client Survey and / or Focus Groups

Research

Year 3

Increased knowledge on effectiveness of interventions to respond to FV

Level of knowledge on effective interventions among relevant client groups

Client Survey and / or Focus Groups

Research

Year 3

Informed policy analysis, development and decision making

Consideration of research and statistical products in the policy analysis, development and decision-making processes

Client Survey and / or Focus Groups

Research

Year 5


FVI LONG TERM OUTCOMES
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

To what extent and in what ways has the DOJ FVI contributed to improving the responsiveness of the criminal justice system to family violence?

  • Level / extent of influence / contribution

  • Extent / impact of legislative change

  • Tools, Strategies, Models developed, tested and implemented

  • Extent / impact of research on family violence issues

  • Relevant client group and key stakeholder Opinion / Survey

  • Year 3 Scan and Year 5 Follow up Scan

  • Statistical Reports / Analysis (Stats Can)

  • Performance Reports / Analysis

Evaluation Division – Project Manager

Year 5


INTERMEDIATE OUTCOMES
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement

To what extent and in what ways has the DOJ FVI contributed to the improved capacity of stakeholders and communities to respond to family violence issues?

Level / extent and nature of capacity improvements of stakeholders and communities to respond to family violence

  • Relevant client group and key stakeholder Opinion / Survey

  • Performance Reports / Analysis

Evaluation Division – Project Manager

Year 5

To what extent and in what ways has the DOJ FVI contributed to a better-informed public and legal community about family violence issues?

Level / extent and scope of information and knowledge dissemination

  • Relevant client group and key stakeholder Opinion / Survey

  • Performance Reports / Analysis

Evaluation Division – Project Manager

Year 5


IMPACTS AND EFFECTS
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Overall what has been the impact of DOJ FVI funding?

  • Have there been any unintended impacts (+ or - )?

  • Level of funding by area (policy, programs and research) and by FV issue, per year

  • Extent / level of impact by FVI issue

  • Evidence of unintended impacts (+ and -)

  • Financial Records (Allocations and Expenditures)

  • Relevant client group and key stakeholder Opinion / Survey

  • Lessons Learned Studies

  • Performance Reports / Analysis

Evaluation Division – Project Manager

Year 5


FVI IMPLEMENTATION AND DELIVERY
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • To what extent have the FVI components reached the relevant client groups?

  • To what extent as the FVI been implemented as intended?

  • Level / scope of reach to relevant client groups

  • Efficiency / effectiveness of FVI delivery mechanisms relative to resource allocation e.g. appropriateness of funding criteria, timeliness of funding process, project monitoring and reporting processes

  • Relevant client group and key stakeholder Opinion / Survey

  • Project / Contract Files

  • GCIMs analysis

  • Workplan analysis

  • Performance Reports / Analysis

  • JPIF Evaluation

Evaluation Division – Project Manager

Year 5


RELEVANCE
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Is there a continued need for the DOJ FVI?

  • To what extent does the DOJ FVI still reflect the objectives and priorities of the federal government?

  • What would be the impacts on the criminal justice system if the DOJ FVI ceased to exist?

  • What would be the impacts on other elements of the federal response to family violence if the DOJ FVI ceased to exist?

  • Extent / scope / impact of family violence on Canadian society

  • Evidence of priority in federal agenda

  • Evidence of priority and work load related to family violence within the criminal justice system

  • Relevant client group and key stakeholder Opinion / Survey

  • Document / Literature / Web site review

  • Needs assessment

  • Statistical review

  • Federal Priority statements / related plans & strategies

  • Speech(es) from the Throne

Evaluation Division – Project Manager

Year 5


COST EFFECTIVENESS AND ALTERNATIVES
Evaluation Issue / Performance Area Indicators Data Source / Methods Responsibility for Collection Timing / Frequency of Measurement
  • Are there more cost-effective approaches to achieving the DOJ FVI objectives?

  • Is the FVI the most appropriate means of achieving the intended objectives of the DOJ FVI?

  • Are there any other improvements that should be made to the design and delivery of the FVI?

  • Evidence of cost-effectiveness in relation to objectives achievement

  • Evidence of cost-effectiveness in relation to appropriate alternatives

  • Evidence of need for design and delivery improvements

  • Financial Records (Allocations and Expenditures) / Analysis

  • Relevant client group and key stakeholder Opinion / Survey

  • Performance Reports / Analysis

Evaluation Division – Project Manager

Year 5



  • [8] According to departmental funding priorities, criteria and processes of the Justice Partnership and Innovation Fund.

  • [9] According to departmental funding priorities, criteria and process of the Justice Partnership and Innovation Fund.

  • [10] Such as victim / witness assistance, social service organizations.

  • [11] March 31 of each year.

  • [12] Could also include a media analysis / scan.

  • [13] According to departmental funding priorities, criteria and processes of the Justice Partnership and Innovation Fund.

  • [14] See Appendix 2: Glossary of key terms.

  • [15] There is also a link to Research outputs related to the effectiveness of various approaches and models.

  • [16] According to departmental funding priorities, criteria and process of the Justice Partnership and Innovation Fund.

  • [17] Links between the DOJ FVI and the DOJ Research and Statistics components of the DOJ web site also need to be made as well as a means to determine how relevant client groups will be identified.


Date modified: