Evaluation of the Encouraging Better Practice in Aged Care (ebpac) Initiative Final Report

3.7Comparison with Round 1 and Round 2

All projects in Round 1 and Round 2 focused on one thing – implementing evidence-based practice in residential aged care. In the current program, the mix of projects was more diverse, in terms of setting (residential and community care); emphasis (changing practices, training clinical leaders, developing resources to facilitate organisational change, nation-wide dissemination) and evidence base (in Round 1 and Round 2, there was a strong focus on implementing evidence-based clinical guidelines whereas in Round 1 and Round 2 there was a greater emphasis on building on what had been learnt or developed in previous projects).

There was much less data supporting an increase in the use of evidence in the current program than there was in either Round 1 or Round 2. Factors influencing the implementation of evidence based practice included establishing a common ground for change; seeing the benefits of change; support from managers, peers or someone in a designated position; and the ability to reconcile competing priorities in an environment of limited resources, all of which were found in Round 1 and Round 2. The main difference between the current round and the previous rounds (in terms of implementing evidence) was the pivotal role played by clients in whether evidence-based changes took place. The ability of staff to engage with clients was also a critical determinant of whether change took place or not, another difference compared to the earlier rounds.

4PROJECT IMPACT

As mentioned previously, the EBPAC program includes a diverse mix of projects, employing different methodologies to achieve a wide range of outcomes. In relation to the three levels of our evaluation framework (consumers, providers, system), each project can be categorised into those focusing at one ‘level’ and those working across all three levels as previously illustrated in Figure . To report on project impact we have again grouped the projects according to their main focus:

The four system-level projects which sought to develop models for training clinical leaders (CL2 and CC6) and facilitating organisational change (RC3 and CL1 ).

The five projects which aimed to implement evidence-based practice in community care (CC1, CC2, CC3, CC4 and CC5).

The two projects which primarily ran nation-wide workshops to disseminate current evidence in the areas of palliative care and skin care (RC1 and RC2).

This section provides a summary of project impacts at the consumer, provider and system level. Further details about individual projects are provided in the project summaries included in Appendix 1.

4.1Impact and outcomes on consumers

Many of the EBPAC projects incorporated practice changes targeted at consumers. Four projects did not include any practice changes in their project plan targeted specifically at consumers. However, two of these (CL2 and RC2) were able to demonstrate positive outcomes on consumers.

Consumer impacts and outcomes: Leadership and change projects

Table summarises the practice changes targeted at consumers for the leadership and organisational change projects. The outcomes reported in this table are synthesised from information reported by projects in their final reports, rather than reflecting the findings of the evaluators.

Table Consumer impacts and outcomes: Leadership and change projects

Project	Practice changes	Outcomes on consumers
CL1	Small-scale changes in each facility, as determined by each action learning team.	The project did not measure specific client outcomes
CL2	No direct implementation strategies targeted at clients	Qualitative comments from residents/clients reveal satisfaction with the changes in care services. Satisfaction surveys with both residents and clients showed no real changes in resident satisfaction but a significant increase in satisfaction with care services in the community
CC6	Nil (project limited to development and distribution of the resources)	N/A
RC3	Small-scale changes in each facility, as determined by the TOrCCh team.	The project did not measure specific client outcomes

For two of the leadership and change projects, the emphasis was on using project resources to initiate organisational change projects within aged care facilities. The RC3 project specifically targeted consumers in a walking program, a gardening program and the implementation of a palliative care trolley. In the case of CL1 the project targeted consumers including a focus on palliative care pathways, oral hydration, oral hygiene and falls prevention.

Neither of these projects measured client outcomes as part of their evaluation strategy. However, the key stakeholder interviews held with residential aged care managers suggested some positive outcomes for residents. For example the manager at an aged care facility in Western Australia reported receiving regular positive feedback from family/carers about the palliative care trolley (RC3). Another manager whose facility hosted a walking and gardening program commented that residents loved these two initiatives (RC3). He commented that prior to these two initiatives residents would ‘often isolate themselves in their rooms’. However, the residents slowly embraced the walking and gardening projects and as a result made new social connections and also improved their mobility at the same time.

A second leadership and change project (CL2) did not have any direct implementation strategies targeted at clients as part of their project plan. Rather, clinical mentors were trained to deliver specific workforce solutions in their facility or in community aged care. Projects included a focus on manual handling, wound management, pain management and managing dementia. An independent evaluation was carried out by Flinders University but evaluation results relating to consumers were mixed. A resident satisfaction survey revealed no significant change in satisfaction in facilities whilst a similar satisfaction survey carried out with community clients showed a significant increase in satisfaction with the care services they received.

The fourth leadership and change project (CC6) did not explicitly seek to impact directly on clients and therefore did not include any impact on consumers in their final report. The project did, however, include consumer input into the development of the methodology and resources developed by the project.

Consumer impacts and outcomes: Community care projects

Five of the six community care projects included practice changes targeted at consumers but the nature of change in practice was variable. These practice changes together with reported outcomes are highlighted in Table .

Table Consumer impacts and outcomes: Community care projects

Practice changes	Outcomes on consumers
CC1
Interventions by nurses and clients to improve wound management e.g. compression bandaging for leg ulcers. Interventions by community care aids and clients to improve prevention of pressure ulcers e.g. checking skin, use of skin care products	It is reported that both LUPP and SAP led to an increase in client knowledge of leg ulcers and skin health and the adoption of some of the recommended wound management behaviours
CC2
Interventions by care workers to promote client activity	Clients showed an increase in researcher-rated engagement, and a decrease in researcher-rated apathy, dysphoria and agitation
CC3
Play client’s favourite music for about 30 minutes while personal and home care services are being provided in the client’s home	Clients’ feedback revealed that they have re-found their love for music. The Mood Change analysis demonstrated that approximately 85% of participants experienced mood improvements as a result of the music intervention.
CC4
The aim is to increase client involvement in decision-making about their own care.	The quantitative data suggests that the CHOICES model had a statistically significant effect on the perceived quality of case management. Overall, clients felt more respected, informed, and appreciated the new financial arrangements.
CC5
Interventions by care workers and clients to improve daily oral hygiene e.g. use of fluoride toothpaste, brushing teeth regularly, relieving dry mouth,	The project demonstrated positive improvements in home care clients’ oral health related quality of life and wellbeing. Clients also felt more confident in their own efforts to improve their oral health.

Two of the projects (CC1 and CC5) both provided consumers with education and support for wound care and oral health. Another two (CC2 and CC4) provided information to consumers to promote physical activity and to introduce the concept of consumer directed care. The fifth project (CC3), trialled the use of music to complement existing home care services.

Four of the community projects (CC1, CC2, CC3 and CC5) involved specific interventions carried out by care workers. Each of these projects specifically measured consumer outcomes as part of their evaluation strategy. Project CC1, demonstrated an increase in client knowledge and management of chronic wounds. Four out of the five care workers interviewed by the evaluation also indicated that their clients were more proactive in managing their skin even after the project was completed. However, one care worker indicated that the project was not appropriate for use with clients with cognitive impairment, e.g. clients with dementia.

The CC2 project aimed to increase clients’ physical activity. The project evaluation methodology included semi-structured interviews with case managers and LEAP champions, questionnaires completed by care workers, and questionnaires and interviews with clients and family members. Whilst the evaluation results as presented in the projects final report were very general in nature, they did highlight an increase in client engagement. Interviews carried out by the evaluation team with care workers did not elicit any positive client outcomes.

The CC3 project focussed on music therapy. The project evaluation was able to show that client’s mood status increased as a result of the music intervention. A member of the evaluation team held a focus group with 13 care workers and their care managers. During this focus group several examples of positive client experiences were mentioned. Clients were described as ‘more talkative’, ‘happy’ and in one specific example the music intervention ‘took away the clients headaches’.

The CC5 project aimed to improve clients’ oral hygiene through interventions by care workers and the provision of relevant oral health resources. A detailed evaluation carried out by the project team demonstrated numerous benefits to clients. Overall the evaluation demonstrated positive improvements in clients’ oral health related quality of life and wellbeing and increased confidence in managing their own oral health. This was confirmed in care worker interviews carried out by the evaluation team. In one instance a care worker commented that:

One gentleman did say to me that he felt he could smile without holding his hand over his mouth; it definitely improved his quality of life.(CC5_2)

Consumer impacts and outcomes: National roll-out projects

Table summarises the practice changes targeted at consumers for the national roll-out projects.

Table Consumer impacts and Outcomes: National roll-out projects

Practice changes	Outcomes on consumers
RC1
No direct implementation strategies targeted at residents	After death audits reveal that more residents commenced an end of life pathway
RC2
No direct implementation strategies targeted at residents	Evaluation reports received from facilities demonstrated positive resident outcomes such as reduced prevalence of skin tears and wounds and improved skin integrity.

Little data was collected by RC1 relating to patient outcomes. The only reference to resident outcomes in the pre-implementation and post-implementation After Death Audit relates to length of hospital stay. The project final report indicates that there is no statistically significant difference between length of hospital stay between the pre and post audit.

Measuring impacts and outcomes on residents was also not included in the project evaluation plan for RC2. However, data relating to aged care clients, families or carers is implicitly included in many of the workshop participants’ project plan reports. According to the final report:

Overwhelmingly plans reported great improvements in skin integrity and reduction in the development of skin tears and pressure ulcers as a benefit to the aged care clients. When these did occur reported healing rates were shown to have improved a great deal compared to previous data (RC2 final report, p.22).

The most frequently reported CSI Project client and family outcomes in the 176 reports received include:

Reduced prevalence of skin tears - 32%

Improved skin integrity - 26%

Reduced prevalence of wounds - 18%

Increased implementation of skin moisturising - 18%

Reduced prevalence of pressure injuries - 13%

Shorter times to healing - 14%

Improved resident comfort - 8%

No data was able to be extracted from the surveys with workshop participants or the interviews with key stakeholders relating to resident outcomes.

Whilst four projects did not include any practice changes in their project plan targeted specifically at consumers, two were able to demonstrate positive outcomes on consumers (CL2 and RC2).

Overall, despite a mixture in the quality of the evaluation methods, the four community projects with specific interventions carried out by care workers produced the best evidence that consumer outcomes improved. Two of these, with a strong focus on prevention (CC1 and CC5), were able to provide the most comprehensive evidence that consumer outcomes improved following the provision of education and support.