Final Evaluation Report

Executive Summary

Evaluation of the Second Action Plan of the National Plan

to Reduce Violence against Women and their Children (20102022)

KPMG was engaged by the Department of Social Services to undertake an evaluation of the Second Action Plan 2013-2016 – the second of four plans in the National Plan to Reduce Violence against Women and their Children 2010-2022 (the National Plan).

Scope

The scope of the evaluation of the Second Action Plan covers four main aspects:

• 
  evaluation of the work undertaken by governments against the five priorities and the 26 actions outlined in the Second Action Plan;
  
• 
  review of the extent to which National Plan flagship programs have contributed to the improvement of service responses for women who have experienced violence and in building the evidence base;
  
• 
  measurement of the contribution of the Second Action Plan in progressing the six National Outcomes of the National Plan; and
  
• 
  identification of emerging issues and areas requiring increased focus in the Third Action Plan.
  

Method

The evaluation has largely been a process based evaluation focusing on progress with the 26 actions that were outlined in the Second Action Plan. This is primarily due to the lack of an evidence base upon which the impact or effectiveness of particular programs or interventions can be assessed. Extensive consultation was undertaken as part of the evaluation with government and non-government stakeholders. A survey of service providers was also completed between November and December 2015. Details of the method for conducting the evaluation are outlined in Appendix 2 and a full list of stakeholders consulted and the survey instrument are contained in Appendix 3.

A summary of the key components of the evaluation method is outlined below.

The process evaluation has been based on an assessment of what was committed to by the Australian Government and state and territory governments and what was actually implemented under the Second Action Plan priorities and actions – progress has been assessed for each action under the Plan as follows:

	Complete – action completed and implemented by all jurisdictions
	Mostly complete – action implemented in most jurisdictions
	Partially complete – action underway and completed across some jurisdictions
	Partially complete – action underway and completed across a few jurisdictions
	Incomplete – action not fully progressed across most jurisdictions

The evaluation has also involved:

• 
  analysis of the impact or effectiveness of the priorities and actions based primarily on stakeholder views from the consultations and survey results (with ratings from not at all effective to extremely effective);
  
• 
  review of the usefulness and appropriateness of the evidence base already developed in assessing progress with achieving the outcomes and priority areas in the Plan; and
  
• 
  assessment of the contribution of the Second Action Plan to achievement of the higher level outcomes in the National Plan based on the indicators outlined in the Department of Social Services’ Evaluation Plan for the National Plan.
  

Progress and Effectiveness under the Second Action Plan

The first part of the evaluation considered the work undertaken by Governments in progressing the five priorities and the 26 actions that are outlined in the Second Action Plan.

Priority	Focus Area
1	Driving whole of community action to prevent violence (7 actions)
2	Understanding diverse experiences of violence (5 actions)
3	Supporting innovative services and integrated systems (7 actions)
4	Improving perpetrator interventions (4 actions)
5	Continuing to build the evidence base (3 actions)

Overall Assessment

The evaluation found that most of the priorities and actions have been implemented as intended across jurisdictions. Overall, the identified priority areas and action areas were also considered to be having a positive impact by stakeholders. Priority 5, continuing to build the evidence base, was viewed as being the most successful in terms of the progress with and effectiveness of the actions undertaken to date. This was followed by Priority 1, driving whole of community action, with most stakeholders considering that the various actions under this priority were effective in driving cultural and attitudinal change to prevent violence against women and children.

Priority 4, improving perpetrator interventions was the area that was considered to have been the least effective followed by Priority 2, understanding the diverse experiences of violence with most stakeholders indicating that not enough had been done to account for the needs of high risk groups like Aboriginal and Torres Strait Islander women, women from culturally and linguistically diverse backgrounds and women with a disability.

The Evidence Base

While there have been significant improvements in the collection of data on the incidence and prevalence of violence against women, some issues remain with the consistency and representativeness of the main data sources used to measure progress under the Plan and there are still differences between jurisdictions. However, the Australian Bureau of Statistics (ABS) has led the collection and dissemination of data on police incidents, assaults and homicides and is working with other parts of state and territory justice systems to improve data. The lack of data related to high risk groups including women with a disability and Aboriginal and Torres Strait Islander women is also seen as a major gap.

The National Community Attitudes towards Violence against Women Survey (NCAS) and the Personal Safety Survey (PSS) continue to be the major mechanisms by which data is collected respectively, on attitudes towards and the prevalence of family, domestic and sexual violence.

Evaluation of Flagship programs

As part of the evaluation of the Second Action Plan, a review was also undertaken of the extent to which National Plan flagship programs have contributed to the improvement of service responses for women who have experienced violence and in building the evidence base.

1.ANROWS

The evaluation found that Australia’s National Research Organisation Women’s Safety (ANROWS) has largely delivered on its stated mission and goals of delivering high quality innovative and relevant research in the area of violence and sexual assault against women and children. Stakeholders were generally positive about the work of ANROWS in terms of the quality and relevance of the research undertaken. The work on perpetrator interventions was seen as addressing a major gap and was particularly welcomed by service providers. The main issue raised was the need to ensure the research was provided in an easily digestible form with a number of stakeholders indicating the length of current documents was preventing them from being more read and utilised extensively.

2.1800RESPECT

An evaluation of 1800RESPECT was undertaken in 2013 by the Department of Social Services. It found continued growth in demand was putting pressure on the capacity of 1800RESPECT to respond and was also putting pressure on call answer times. This was addressed through the introduction of the first response triage function in August 2016.

An independent evaluation of 1800RESPECT will be completed in 2017.

3.Our Watch

KPMG conducted a social media analysis of the six media platforms currently operated by Our Watch including Facebook, Twitter, YouTube, Instagram, Linkedin and Google+. Overall, the analysis showed that Our Watch’s social media accounts are regularly updated and attract a high level of engagement from third party users.

Analysis of one of the social media platforms, in particular, Facebook, indicates that it has especially high third party engagement of 4.1 per cent which is impressive given that average engagement rate for Facebook is 0.5-1 per cent.

4.The Line

Similar results were seen for The Line social marketing campaign which is aimed specifically at promoting healthy and respectful relationships among young people. The campaign has made extensive use of social media and the social media analysis undertaken by KPMG demonstrates high levels of engagement on Facebook and Instagram compared to other entities like VicHealth and Lifeline.

Contribution of the Second Action Plan in progressing National Outcomes

5.Governance Arrangements

The National Evaluation Plan also sets out some key evaluation questions in relation to the governance arrangements for the National Plan. The evaluation of the Second Action Plan identified that:

• 
  while stakeholders had difficulty in attributing changes and impacts directly to the Second Action Plan, the directions being pursued by states and territories are broadly consistent with the directions outlined in the National Plan and that there have been benefits in adopting a national approach;
  
• 
  governance, advisory and communication mechanisms were generally considered by stakeholders to be operating effectively; and
  
• 
  there were numerous examples of work by governments and non-government organisations (NGOs) collaborating at the local level to address the issue of violence against women and children and creating linkages with civil society and other groups in particular sporting groups.
  

Contribution to High Level Outcomes in the National Plan

The evaluation found that the Second Action Plan has built on the foundations of the First Action Plan and made significant progress against the six National Outcomes. However, a long-term and sustained approach will be required to develop and implement evidence based policy and continue improvements in the sharing of information and quality of services offered to women and their children who have been impacted by violence.

6.National Outcome 1 – Communities are safe and free from violence

The latest NCAS 2013 results showed some improvement from the previous survey in 2009 in the community’s understanding of violence with the proportion of people who agree that controlling a person’s social life is a form of domestic violence increasing from 83 per cent in 2009 to 85 per cent in 2013. However, there has been a decline across other key indicators of community intolerance of violence against women. For example in 2013, 96 per cent of survey respondents thought that forcing a partner to have sex was a form of violence, a reduction of 1 per cent compared with results in 2009.

7.National Outcome 2 – Relationships are respectful

The most recent NCAS presented some poor indicators for increased awareness of respectful relationships education among young people. Up to 18 per cent of young people believe that men should be in charge of a relationship and there was a 4 per cent increase between 2009 and 2013 in the proportion of people who believe that a woman cannot be raped by someone she is in a relationship with.

8.National Outcome 3 – Aboriginal and Torres Strait Islander communities are strengthened

There is currently no data available on the proportion of Aboriginal and Torres Strait Islander women who consider that family violence and sexual assault are problems for their communities or the proportion of Aboriginal and Torres Strait Islander women who are able to have a say within their communities.

The 2008 National Aboriginal and Torres Strait Islander Social Survey (NATSISS) showed little improvement in the proportion of Aboriginal and Torres Strait Islander women who experience violence, with a decrease from 24 per cent in 2002 to 23 per cent in 2008. The 2014-15 NATSISS was released in 2016.

There is an urgent need for evaluations of trials or programs that have been implemented in some jurisdictions to assess overall effectiveness and help build the evidence base about what works in Aboriginal and Torres Strait Islander communities.

9.National Outcome 4 – Services meet the needs of women and their children experiencing violence

The evaluation revealed improvements in service integration across a number of jurisdictions. However, the needs of some cohorts of women are not being met as well as they could be including Aboriginal and Torres Strait Islander and CALD women and women with a disability. Of particular concern is the latest finding in the NCAS which indicates that there has been a reduction in the proportion of people who would know where to get help regarding a domestic violence problem, from 62 per cent in 2009 to 57 per cent in 2013.

10.

11.National Outcome 5 – Justice responses are effective

The most recent PSS indicated that of the sample of women interviewed, 41 per cent of women had experienced violence since the age of 15, 34 per cent had experienced physical violence and 19 per cent had experienced sexual violence. The most recent PSS also indicated 72 per cent of the women who experienced physical or sexual assault in the twelve months prior to the survey in 2012 did not report the incident to police. It is likely that a significant proportion of women are still concerned about taking their incident to the police and going through the court system.

12.National Outcome 6 – Perpetrators stop their violence and are held to account

The PSS indicated that in 2012 up to 65.1 per cent of the women surveyed, who had experienced partner violence since the age of 15, were the victim of more than one incident. A further 73.0 per cent of women who had experienced violence from a previous partner since the age of 15 had been assaulted more than once. This data indicates a high level of recidivism in violence against women in intimate relationships.

13.Future areas for focus

The evaluation has highlighted a number of areas for consideration in the development of the Third Action Plan where policy could be strengthened and government action be focused to achieve greater progress against the outcomes of the National Plan. These include:

• 
  maintaining the momentum around raised community awareness of the issue of violence against women and children and encouraging community, government, businesses and sporting organisations to continue working together;
  
• 
  ensuring that specialist and/or tailored services are available for women with special needs who experience violence for example Aboriginal and Torres Strait Islander women, women with a disability, and women from culturally diverse backgrounds;
  
• 
  promoting greater gender equality through programs that emphasise female leadership and empowerment, particularly in high risk groups such as Aboriginal and Torres Strait Islander communities and culturally and linguistically diverse communities;
  
• 
  ensuring that the social media engagement strategies for The Line and Our Watch maximise contact with the public, particularly with regard to integrating social media assets;
  
• 
  continuing to incorporate respectful relationships education into the school curriculum;
  
• 
  continuing efforts improve information sharing across sectors in particular between the courts, police and service providers including the national system for domestic and family violence apprehended violence orders;
  
• 
  considering the introduction of quality standards to complement the outcomes standards for perpetrator intervention programs;
  
• 
  considering the future development of policy and programs that focus on addressing the particular needs of women and their children who have been exposed to sexual violence; and
  
• 
  continuing to refine the evidence base to establish a base line against which success of future Action Plans can be considered.