Evaluation and Improvement
Evaluate the clinic to ensure you’re achieving your aims
Evaluating a nurse clinic
An evaluation is a systematic approach, using evidence-based criteria to measure how well a program is achieving its purpose and aims. How successful has the program been? What value has it contributed?
Defining a successful clinic involves being clear about what it is you are trying to achieve.
- How will you determine if your clinic is good?
- Who are you trying to help and in what ways?
- What are the criteria that you will use to judge success?
We suggest that you develop a clear definition of success and identify criteria that can help you judge whether your clinic is successful.
- Useful criteria could include:
- number of patients seen each week
- numbers of assessments conducted (e.g. health assessments)
- numbers of care plans reviewed.
Once you’ve identified these factors, you are ready to start.
The evaluation process
Use the following eight step approach to guide the evaluation
(adapted from Chambers & Wakley, 2005, as cited in Crow, J, Walker, L., Desboroough J., n.d.).
Reference:
Chambers, R., Wakley, G. 2005. Audit in primary care: demonstrating quality and outcomes (First edition), CRC Press https://www.crcpress.com/Clinical-Audit-in-Primary-Care-Demonstrating-Quality-and-Outcomes/Chambers-Wakley/p/book/9781857757095
Crow, J, Walker, L., Desboroough J. (n.d). Nurse Clinics in Australian General Practice. https://apna.asn.au/files/20121003_rsc_FINAL-Nurse-Clinics-in-Australian-General-Practice.pdf
Each step is outlined below
Step 1 – Define the objectives
Reasons for conducting an evaluation need to be clear. For example, the practice may want to evaluate the effectiveness of a nurse clinic to justify it continuing, or understand whether patients see any benefit in the care they receive. Objectives and target health outcomes should be identified during the planning stages.
Clear, realistic and easy to understand objectives need to be set that are Simple, Measurable, Achievable, Realistic and Timely (SMART).
Objectives might cover:
- whether agreed standards are being met
- monitoring patient consistency with recommended treatment or advice
- improving clinical effectiveness
- staff satisfaction, recruitment and retention.
Step 2 - Review the literature
Evaluation must be based on current, evidence-based practice and guidelines. A review of the literature is the best way to determine the most up-to-date evidence. The RACGP Clinical indicators for Australian general practice provide examples of measurable evidence-based objectives which can be used as specific criteria for measuring patients related to the care provided in a nurse clinic. Or benchmark against best practice guidelines: National Health and Medical Research Guidelines.
Step 3 - Design the evalution
The design of the evaluation will be dependent on the resources available, such as time, staff skills and funding. These will guide what information can be obtained and how it is collected and will help determine staff members’ roles in the evaluation process.
Designing the evaluation to collect a small amount of meaningful and clearly defined information, is better than obtaining general information that may prove difficult to analyse and interpret.
Different types of information will reflect different aspects of the way in which care is provided. For example:
- Evaluation of the structure of care includes examining things such as the office space, equipment and resources, and the clinical competence of staff [including knowledge and training].
- Evaluation of care processes refers to what is done to patients and how it is done. This includes how easy it is for patients to obtain an appointment with a nurse, the clinical protocols and guidelines used to guide care, patient recall and reminder systems, and waiting times.
- Health outcomes can be measured in terms of patient satisfaction and experience, clinical outcomes, and access to health care.
Step 4 – Collect the data
Deciding on the criteria which determines what success looks like, will be unique to each clinic’s own context. The answer to this question will guide the type of data to collect.
Further considerations include:
- Will data be retrospective, concurrent or prospective? For example, you might collect information before a program of care is introduced (retrospective]) and plan to collect information in the future (prospective) to compare patient outcomes before and after the program of care.
- How easy is it to collect the data? Will you have the time available to gather data that is hard to locate?
- You may also want to pilot the collection of information before you begin the evaluation cycle to ensure the information measures what it should, and the way it is collected works.
Be mindful of the ethical impacts of the evaluation. Similar methods are often used in research and evaluations to gather information. More information is available from the National Health and Medical Research Centre’s (NHMRC) National Statement on Ethical Conduct in Human Research and the RACGP National Standing Committee- Research (NSC-R).
Consider the security of patient information when collecting data for clinical audit.
Patient experience: Surveys, interviews and focus groups
There are a number of methods you can collect patients’ experiences. The Patient Enablement and Satisfaction Survey (PESS) measures patients’ perceptions of the following in terms of their experiences and outcomes of nursing care: affective support; provision of information; decisional control; technical competency; access; professionalism; time; overall satisfaction; and enablement.
A survey is a simple and cost-efficient method of gaining feedback from patients. Surveys can be either mailed to patients or provided directly by reception staff. The way in which surveys are provided to patients, and to which patients they are provided, needs to be discussed when designing the evaluation.
Other methods include interviews; either one on one, or by phone, or in focus groups. These provide qualitative information which, when considered together with information obtained through survey and clinical audit, can provide a valuable insight into patients and/or their families experience of the health care they have received.
Stakeholder perceptions
Understanding the level of acceptability and feasibility from the perspective of the nurse clinic stakeholders should be included in any nurse clinic evaluation. Information gathered from surveys and interviews can add understanding to aspects of the clinic which are working well and what areas may need to change.
Clinical audit
A clinical audit could include factors as clinic attendance (activity), crucial for a nurse clinic, pathology results (HbA1c, INR and cholesterol levels) and other measurements such as weight or BMI.
Practice software and data extraction tools play an important role in data collection
Quality Indicators in primary healthcare
| Practice-level indicators of safety and quality for primary health care
The Australian Commission on Safety and Quality in Health Care has developed 35 Practice-level indicators of safety and quality for primary health care (excluding general practice). |
| Clinical indicators for Australian general practice
The Royal Australian College of General Practitioners (RACGP) has developed a set of 16 clinical indicators for Australian general practice. |
| Clinical Governance in Community Health Services
The discussion paper by Australian Institute for Primary Care, La Trobe University explores possible clinical indicators that could be employed as part of an overall clinical governance framework in the community health sector. |
Step 5 - Analyse the data
Data analysis will depend on methods used to collect information. For example, the Patient Enablement and Satisfaction Survey (PESS) is scored according to instructions, while analysis of patient clinical audit data can be determined by the team.
It is important to remember that data should be analysed as an aggregate as opposed to individually. Given that evaluation is usually based on a variety of information using different methods of collection, data should be analysed in combination, to create a comprehensive picture.
6 – Discuss findings and develop an action plan
The findings of the evaluation and their implications should be discussed with the practice team. The potential for change and improvements, and how these can be implemented can then be determined.
Quality improvement requires more than an individual approach, as the greatest success is achieved if a team approach is taken to reforming and potentially redesigning clinical work structures and processes.
Review the Best-practice care building block for more information
Step 7 - Implement the action plan
Team meetings and meeting with individuals will assist in agreeing on everyone’s roles and responsibilities in relation to implementing an action plan for changes or improvements to health care delivery.
Changes might also need to be communicated to patients and other staff. Putting the action plan in writing and distributing it among staff for comment and feedback will also help.
Step 8 - Re-evaluate
As part of an ongoing review of the quality and safety of the nurse clinic, re-evaluation of the changes made from the initial evaluation is recommended.
To re-evaluate, you will need to consider what changes have been made, and how long it might take for these changes to make measurable impact on health outcomes or work structure.
Read the evaluation approaches used in the nurse clinic case studies
