BENCHMARKING AS A QUALITY ASSESSMENT TOOL
B NANCY LOVERIDGE describes the benefits and challenges associated with benchmarking
practice
Successive publications and consultation documents from the government have
placed quality related issues firmly on the agenda. The New NHS: Modern Dependable
focuses on the efficiency and quality of the NHS (DoH 1997). Another government
publication, Making a Difference, dedicated section seven to 'enhancing the
quality of care' (DoH 1999). This White Paper recognised the significance
and potential of nursing involvement in the process. The momentum has increased
further through the publication of NHS performance ratings (DoH 2001a), and
although designed not to focus on quality of clinical care, the inference
in the press exists.
My role as practice development adviser (PDA) has led me to examine issues
of quality in the clinical area of A&E where I am based. Clark et al
(2001) identified the PDA's role as one which is closely associated with
issues of quality, be it the individual practitioner, departmental or clinical
practice related. I have found this aspect of the role the most difficult,
the main issues being; what areas of practice to examine first, and the definition
or recognition of quality practice.
Within my trust, a busy district general hospital, quality is high priority,
which has been recognised in a three star rating allocated this year. To
ensure the focus upon quality continues, benchmarking has been adopted as
a quality assessment tool on the wards with an expectation to use it in A&E
in the near future.
The introduction and implementation of this quality assessment tool will
prove to be a key project in the coming months and an understanding of its
principles will facilitate the process. Therefore, the purpose of this paper
is to extract principle themes from the literature and provide a critical
analysis of the issues, the need for a quality agenda and the difficulties
which surround this. The concepts of benchmarking, with a focus on the efficacy
and benefits to A&E practice, related patient outcomes, and the potential
disadvantages of the process, will also be examined.
Due to benchmarking being undertaken on the wards, this has provided an opportunity
to evaluate its initial introduction to support implementation of the process
in A&E. To enable this critical analysis, a literature search was carried
out, methods used included; computerised searches of Cinahl and Medline,
manual search of journals and books, networking with nursing staff and managerial
personnel in my own and other A&E departments.
THE QUALITY AGENDA
Recently, due to a road traffic accident, I became a patient, and feeling
vulnerable, I focused on the need for quality care. Jackson (2001) reflects
this need, maintaining that the public has a right to quality assured care.
The impetus for ensuring quality care has not only been generated at local
level, but importantly from the government. The plethora of documents and
statements from this government appears continual since they were elected
in 1997, The New NHS (DoH 1997), A First Class Service (DoH 1998), and Making
a Difference (DoH 1999). These documents share a common theme, that of quality
with less emphasis on speed and delivery, and meeting quantitative targets.
Within my own trust, there is an emerging theme of quality, reflected in
a regular accreditation culture. My trust has often opted to be a 'pilot'
site for health quality service assessments and the Clinical Negligence Scheme
for Trusts (NHS Litigation Authority 2000). Each of these initiatives and
processes examine aspects of quality assurance, for example evidence of clinical
incident reporting and how practice has then improved. The fact that such
structures exist provides the evidence that quality is being addressed and
improved at trust level. The role of PDA is a further initiative to ensure
the agenda of quality is also focused at directorate level.
DEFINING QUALITY
Quality at the point of issue becomes a concept that does not lend itself
for definition. In quality related literature, attempts to define the concept
are rarely addressed. How then can we assure what we cannot define? The Oxford
dictionary gives two simple definitions, 'Quality...the degree of excellence
of a thing' and 'Quality Control...a system of maintaining standards in manufactured
products by testing a sample of the output against specification' (The Concise
Oxford Dictionary 1993).
This general, non-healthcare definition raises several issues, where quality
is viewed as the potential best and that this can be at different levels
or degrees. Although the definition of quality control is industry based,
the view that quality is judged on the end product reflects most of the quality
assessment tools in healthcare. Brocklehurst and Walshe (2000) echo this
with their definition of quality improvement as the achievement of agreed
requirements on a consistent basis, again this reflects quality as the end
product. I would question if this frequent assessment of the end product
to define quality prevents the understanding of the entire concept.
Redfern (1993), a director of a nursing research unit, highlights the inherent
difficulties. Quality can be viewed from three concepts; characteristics
of the entity, as the capacity or skill, and the achievement of a degree
of excellence. This offers a holistic perspective of the process, not a focus
on the achieved end product.
Within my trust, a frequently used quality assessment tool is audit, which
provides a respected key role in quality assurance. Examples of audits carried
out are; level of pain on discharge, overall waiting times, patients' satisfaction
of their attendance in the department and the number of missed fractures.
Rarely is the quality of the journey and specific aspects examined in a qualitative
manner, for example, exploring if practice and approaches were evidence based.
Within health care we tend to measure what is measurable rather than the
important (Anon 1998).
I acknowledge that audit is viewed as a production of quantitative data and
often reflects a final analysis of an intervention or process. An example
of this was an audit of pain management in my department which found that
34 per cent of patients left the department with moderate to severe pain
(Loveridge 2000). While this was an assessment of an end product it enabled
the department to explore a focused area of practice that needed to be addressed,
thereby improving quality in one area. However, I do acknowledge the potential
criticism from Anon (1998) that audits may provide quantitative data upon
isolated areas of practice, without considering the entirety of practice.
Indeed, after the audit of pain management we did have to take time to reexamine
the entire process to uncover where the problem lay. Redfern (1993) warns
of adopting a reductionist approach to the assurance of the quality of the
nursing care and of ignoring the holistic elements.
The project of benchmarking in A&E encouraged me to review the literature
to ascertain its applicability within the department. Ellis (2000) highlights
how this approach ensures the holistic nature of practice is examined and
is in complete contrast to a reductionist approach.
BENCHMARKING
Ellis (2000) clarifies the conception of benchmarking as from within the
industry; the aim being to collect data from a range of organisations to
develop a standard reference point. The approach to benchmarking in the health
service does differ from the industry concept. Pantall (2001) defines benchmarking
as 'the continuous, systematic search for, and implementation of, best practices
which lead to superior performance' (Pantall 2001). When the semantics of
this definition are explored, key themes emerge; first that benchmarking
relates to a structured approach to uncover innovative clinical practice.
Although not included in the definition, Pantall explains that the search
will occur outside your own clinical setting. This process of enquiry will
then be instrumental in a progressive improvement of practice.
While Pantall's statement identifies the ongoing process of searching for
good practices, there lacks, from this definition, the need to assess your
own practice first. Greenidge (1998) acknowledges the need to become aware
of key elements that formulate the body of a particular area of practice
in your own area and use this as a comparison. Greenidge (1998) defines this
approach as the achievement of improvement being based upon an external focus
on internal activities.
This need to self assess and then compare and further develop is outlined
in the Department of Health (2001b) benchmarking tool entitled Essence of
Care. This assessment tool was launched by the government in February 2001
and provides structure and advice to benchmarking, against what the government
felt were the eight crucial aspects of patient care. (Table 1)
Each of these areas of practice have been broken down to key factors with
corresponding benchmarks of best practice with a suggested approach. As Figure
1 identifies, the process of benchmarking is cyclical in nature, with a continuous
process of evaluation, comparison, action and reevaluation being central
to the concept. Pantall (2001) further clarifies the four different approaches
to benchmarking, which are primarily based upon the different areas that
can be accessed for comparison. (Table 2).
My trust has volunteered to be included in the pilot sites for this tool
with the understanding of its adoption as a major quality assessment tool
once established. A&E was not included in the first wave of benchmarking
as food and nutrition was the first area of practice identified. Not being
included in the first wave lead me to question whether the government's view
of crucial aspects of care did, in fact, reflect priorities in the acute
and transient field of A&E.
When the chosen aspects of care are examined, four of the eight have a low
priority in A&E, these being; personal and oral hygiene, principles of
self-care, food and nutrition and continence, bladder and bowel care. After
discussion with colleagues in my own and neighbouring departments, our priorities
would be approaches to trauma, triage, paediatrics and aspects of elderly
patient care.
There is a concern that, in adopting a broad based assessment tool, there
was a risk of using a tool that was trustwide applicable but not department
sensitive. This encouraged me to explore the relevance to A&E that this
particular tool had, and the evidence available to support its introduction.
My own literature search was extensive, but uncovered only ten pieces of
pure benchmarking literature, one each in 1998 and 2000, the remaining eight
in 2001. It is clear that the sudden increase in literature was due to political
impetus. Albarran (1995) urges the nursing profession to be politically aware,
otherwise its approach and concepts will be determined by non-nursing bodies.
A concern of the introduction of benchmarking is the major adoption by government
of a process with little critical evidence or research to support its implementation
nationally. It is crucial; to explore the evidence that supports using benchmarking
to address quality related issues and practice development in A&E.
BENEFITS AND EFFICACY
When the literature is explored to establish the efficacy of benchmarking
as a quality assessment tool, evidence does exist to support its role in
this field. Four pieces of literature give account of the process and principles
of benchmarking based upon actual experience, three being descriptive in
nature and the fourth research. Bland (2001) and Ellis (2000) both descriptive
articles, outline the process and journey undertaken by paediatric nurses
in the north west region of England in 1994. The third descriptive article
relates to the adoption of the process within an A&E department, again
in the north west region but not directly related to the paediatric-benchmarking
group as previously sited (Greenidge 1998). The fourth article relates to
a randomised controlled trial (RCT) in America focusing on 70 community physicians
(Kiefe et al 2001). When the three descriptive articles are critiqued, emerging
themes are identified regarding the efficacy and benefits of the tool. These
being the positive affects of the internal culture to effect practice development
which was based upon external practice, and using and generating sound clinical
evidence, which is department specific.
THE INTERNAL CULTURE
When the cyclical process of benchmarking, as described by the DoH (2001b)
is outlined, the introduction of an innovation through staff participation
is reflective of action research (See Fig. 1). Tarling and Crofts (2000)
describe action research as an approach that is participatory with the staff
involved in solving dilemmas in their own area, which produces quantitative
and qualitative data. Bland (2001) concedes that benchmarking reflects action
research due to the development and testing of innovation that is grounded
in the practitioners' local setting. Bland describes how these clinical developments
are not only captured and defined qualitatively but also given a numerical
score to aid monitoring of progress.
Due to the provision of quantitative data, each of the three descriptive
articles are able to identify the significant benefits of using benchmarking
as a quality assessment tool, demonstrating ongoing, improving standards
of care. Greenidge (1998), Ellis (2000) and Bland (2001) imply that this
success is due to the increased motivation and sense of ownership that is
generated by the use of benchmarking. To ensure the successful introduction
of benchmarking use of a rational-empirical approach followed by normative-re-educative
strategy as described by Wright (1998) ensures greater success. That is,
the idea is first 'sold' to the staff and the group then adopts the change
agent role, thereby activating the sense of ownership. Bland (2001) supports
the need for effective change management to increase effectiveness, implying
it must be inherent in the process to ensure success.
Evidence of the effectiveness and the benefits of benchmarking are further
supported by a randomised controlled trial (RCT) carried out by American
community physicians (Kiefe et al 2001). Although the applicability of the
findings to UK hospital based practice must be questioned, Kiefe et al 2001
demonstrate the benefits of benchmarking through improved patient outcomes.
The RCT consisted of randomly selecting 75 community physicians to either
an experimental group or comparison group. The experimental group were given
their performance ratings and related to performance achievements of the
other practices, while the comparison group were not. The experimental group
demonstrated significant improvements in standards of intervention, for example,
inoculation of influenza vaccine.
Kiefe et al (2001) identify the powerful effect of peer performance as a
motivator for change. I would question if the Hawthorne effect was instrumental
for the positive findings in this research. Knowing which interventions are
being monitored, scored and compared will focus adjustments of a practice.
However, this could be attributed to process of benchmarking in general,
that through special attention in one area, it is inevitable that practice
will improve.
A criticism of Kiefe et al (2001) research is that the development of overall
practice was not considered, but focus given to the number of interventions
carried out. Where by in the work carried out in the north west of Britain,
this aspect of overall practice development is central and imperative.
In summary, the implication to A&E clinical practice is that the application
of a benchmarking approach does offer a useful vehicle to manage change or
improvement of practice. The process accentuates a key benefit, the sense
of staff ownership. As a quality initiative, the structured process is in
contrast to a reductionist approach and is in fact holistic in nature. The
similarity of benchmarking to action research encourages the change agent
to monitor the process, recording any change in practice or issues raised
by the participating staff. This further ensures that quality is not viewed
as an end product but a continuous event.
USING AND GENERATING EVIDENCE
The Essence of Care manual (DoH 2001) provides scoring evidence for each
of the areas of practice and while this evidence is general to all areas,
where should A&E compare practice initially? The wards have chosen internal
benchmarking as described in Table 2; however, I would question the validity
for A&E to adopt the same approach. Based upon observation of clinical
practice, the cultures in the ward setting and A&E, while having similarities,
are different, making comparison difficult.
Examples of differences include the transient nature of patient flow; patients
are often undiagnosed on admission and are prioritised dependent on their
clinical condition. There is also a unique body of evidence relating to A&E
nursing which is specific to this area. However, I recognise potential benefits
of generic benchmarking, that is comparisons of approaches to practices between
A&E and other specialties -- medicine, surgery, paediatrics and orthopaedics.
Discussions with colleagues in each of these areas identify that many approaches
to practice, though unique to that area, could be used in each other's area,
for example acute pain management.
Bland (2001) discusses the use of different ranges of evidence highlighting
41 per cent of the underpinning evidence used to effect development within
his area during benchmarking, reflected level four, that of opinion and experience
(Table 3). Bland (2001) acknowledges the need to ensure, wherever possible,
that the evidence is sought from the highest level. The author reports that
only 12 per cent of the evidence was gained from levels 1 and 2.
While level 4 is deemed a low level of evidence (Pearson 2001), there is
a growing realisation of the importance of experiential evidence. Meerabeau
(1992), using Polanyi's (1958) work, describes knowledge embedded in practice
as tacit knowledge. Meerabeau (1992) acknowledges that refined tacit knowledge
is not reductionist in nature but takes account of the entity of practice
as an expert skill in that field. An advantage of benchmarking is an acknowledgement
of accessing this source of evidence. If an action research approach were
formally used, this would then generate evidence with a higher rate of credibility.
It should also be considered that the quantitative and qualitative data generated
is by design, open to comparison to other A&E settings thereby increasing
the generated evidence's reliability.
Clinical expertise can be buried in individual areas. Benchmarking can give
this untapped knowledge the recognition it deserves. I would further contend
if rigour is applied to the approach of benchmarking as reflected in action
research, this could impact on the perceived level of evidence it generates.
DISADVANTAGES OF BENCHMARKING
Uncovering disadvantages of benchmarking has proven difficult as the descriptive
and research literature is biased towards this concept, but difficulties
do surround this approach, these being time and lack of funding. There is
a need to ensure extra funding is sought to support time out of the clinical
setting and to support developing practice.
Ellis (2000) and Bland (2001) clarify time out of the clinical setting was
ensured; this I would contend would be necessary to affect levels of motivation.
An argument to support investment is that long-term, while the prime objective
is quality assurance a further benefit would be cost effective care. Blachett
et al (2001) warns against the perception of nurses' reputation of ignoring
financial and planning factors, and focusing purely on the clinical aspects.
COST CONTAINMENT
Aspling and Lagoe (1996), a hospital administrator and executive director
from the US, focus on the cost containment aspect generated by benchmarking.
They imply that benchmarking standardises practice and that effective practice
is also cost effective. I would support this, based upon observation of clinical
practice; we often continue certain approaches to care because of tradition.
If we eradicate or improve an aspect of practice and then provide evidence
of cost improvements, investment in benchmarking could be further assured.
A further concern of benchmarking as a quality assessment tool, as already
stated, is the lack of higher levels of evidence to underpin practice development.
The data and clinical evidence produced could be questioned for its reliability
and academic structure. The descriptive work of Ellis (2000) and Bland (2001)
identify the involvement of university lecturers and highlight the importance
of using this group. I would concur that this is an important element in
the process for the following reasons. First, this involvement makes inroads
in the theory/practice divide, which the government acknowledges in Making
a Difference (DoH 1999). Second, involvement from such academics may be used
in the search, critique and application of academic papers and research.
Finally, gaps in the theory and research required to support practice may
be taken further to generate clinically required research.
CONCLUSION
The agenda for quality assurance in the NHS has become a priority due to
the successive White Papers. The publication of Essence of Care (DoH 2001b)
offers one solution, that of benchmarking. This approach directly monitors
standards, and is a tool to assist in the development process. This approach
uses the practitioner responsible for the delivery of care that reflects
the ethos of Making a Difference (DoH 1999). Although this is a relatively
new and unproven approach in the NHS, initial reports of its success are
documented. Descriptive evidence and limited research highlights that clinical
practice is not only improved but also maintained. Also, if managed effectively,
benchmarking increases the sense of ownership and motivation in the workplace
thereby effecting observed practice improvement.
The sharing of ideas and networking with our colleagues is not new, but due
to a structured systematic approach the evidence generated is perceived credible.
The comparison of benchmarking to action research is a beneficial concept
and could assist in formalising the credibility of quality assessment. In
using university lecturers teamed with an action research approach to the
process, the depth of specific evidence could be addressed. Benchmarking
provides opportunities to address issues of poor practice and releases innovative
practice in the workplace. Nurses are keen to share their ideas, but often
lack the vehicle to do so.
The lack of clinical time to release practitioners to achieve such goals
must also be addressed or the opportunity will be lost. Nurses in the clinical
area must be prepared to provide evidence why investment in such a process
may be cost effective. The literature reflects the infancy of the quality
assessment tool, but being endorsed by government will .provide kudos to
the approach and results. As clinicians, it is important not to accept without
enquiry, issues surrounding new imposed concepts. However, when potential
benefits of a new approach become evident, as clinicians we must ensure that
its introduction is carefully, managed and supported. Finally, any generated
clinical evidence has the potential to be recognised as credible and used
to benefit patient care in the wider context.
This article has been subjected to double blind peer review
Table 1. Essence of care
Aspects of:
Principles of self care
Food and nutrition
Personal and oral hygiene
Continence, bladder and bowel care
Pressure ulcers
Record keeping
Safety of patients with mental health needs
Privacy and dignity (DoH 2001)
Table 2 - Four types of benchmarking
Internal Benchmarking - A comparison of internal operations within a single
NHS organisation.
Competitive Benchmarking - Specific comparisons are made between health organisations,
both public and private.
Functional Benchmarking -- Comparisons are made between similar functional
activities in health and non-health sector organisations.
Generic Benchmarking -- Comparisons are made of business function or processes
that are the same regardless of the business.
(Pentall 2000)
Table 3. Four levels of evidence to support the development of benchmarking
1. NHS Centre for Research and Dissemination or Cochrane Collaboration database
2(a). Large-scale, well-designed primary studies, randomised trials and other
controlled trials
2(b). Large-scale primary studies using other methodologies
3. Descriptive studies and reports (national and local standards, guidelines,
surveys)
4. Opinion and experience of respected authorities based on clinical experiences
and professional consensus
DIAGRAM: Figure 1. Phases of essence of care benchmarking
References
Albarran J (1995) Should nurses be politically aware. British Journal of
Nursing 4, 8, 461-465
Anon (1998) Managing Personal and Team Effectiveness. Book 5 Buckingham,
Open University
Aspling D, Lagoe R (1996) Benchmarking for clinical pathways in hospitals:
A summary of sources. Nursing Economics 14, 2, 92-97
Blachett M (2001) Understanding budgets Part 1 -- an introductory guide.
Nursing Management 8, 6, 23-26
Bland M (2001) North West Clinical Practice Benchmarking Group: Principles
processes and evaluation. Nursing times Research 6, 2, 581-593
Brocklehurst N, Walshe K (2000) Quality and the New NHS. Emergency Nurse
8, 2, 46-53
Clark A, Dooker J, Fowler J (2001) The Handbook of Practice Development Wiltshire,
Quay Books
Department of Health (1997) The New NHS, Modern, Dependable London, DoH
Department of Health (1998) A First Class Service. London, DoH
Department of Health (1999) Making a Difference: Strengthening the Nursing,
Midwifery and Health Visiting Contribution to Health and Health Care. London,
DoH.
Department of Health (2001a) NHS Performance Ratings London, DoH
Department of Health (2001b) Essence of Care: Patient focused Benchmarking
for Healthcare Practitioners London, DoH
Ellis J (2000) Sharing the Evidence: Clinical practice benchmarking to improve
continuously the quality of care. Journal of Advanced Nursing 3, 1, 215-225
Greenidge P (1998) Benchmarking -- from theory to practice. Emergency Nurse
5, 9, 22-27
Jackson S (2001) Believing in and recognising health care International Journal
of Healthcare Quality Assurance 14, 2, 47-48
Kiefe I et al (2001) Improving quality improvement using achievable benchmarking
for physician feedback. Journal of American Medical Association 285, 22,
2871-2879
Loveridge B N (2000) Ethical implications of achieving pain management through
advocacy. Emergency Nurse 8, 3, 16-21
NHS Litigation Authority (2000) Clinical Negligence Scheme for Trusts. London,
DoH
Pantall J (2001) Benchmarking in healthcare. Nursing times Research 6, 2,
568-580
Pearson L (2001) Providing the best for our patients: Evidence based practice.
The Nurse Practitioner 26, 3, 11-15
Polanyi M (1958) Personal Knowledge: Towards a post critical philosophy London,
Routledge and Kegal Paul cited in Meerabeau L (1992) Tacit Nursing Knowledge:
an untapped resource or a methodological headache? Journal of Advanced Nursing
17, 1, 108-112
Redfern S (1993) In pursuit of quality of nursing care. Journal of Clinical
Nursing. 2, 141-148
Richmond J et al (2001) Scoring for success. Professional Nurse 16, 8, 1266-1267
Tarting M, Crofts L (2000) The Essential Researcher's Handbook London, Bailliere
Tindall
The Concise Oxford Dictionary (1993) Oxford, Oxford University Press
Wright S (1998) Changing Nursing Practice Second edition. London, Edward
Arnold
~~~~~~~~
By B. Nancy Loveridge
B Nancy Loveridge RGN, BMed Sci(Hons), is Practice Development Adviser, Chesterfield
and North Derbyshire Royal Hospital NHS Trust
--------------------------------------------------------------------------------