Q Manage Health Care

Vol. 19, No. 3, pp. 211225

c 2010 Wolters Kluwer Health | Lippincott Williams & Wilkins


Assessing the Evidence of Six Sigma and Lean in

the Health Care Industry
Jami L. DelliFraine, PhD; James R. Langabeer II, PhD;
Ingrid M. Nembhard, PhD
Background: Popular quality improvement
tools such as Six Sigma and Lean Systems (SS/L)
claim to provide health care managers the
opportunity to improve health care quality
on the basis of sound methodology and data.
However, it is unclear whether these 2 quality
improvement tools actually improve health care
quality. Methods: The authors conducted a
comprehensive literature review to assess the
empirical evidence relating SS/L to improved
clinical outcomes, processes of care, and
financial performance of health care organizations.
Results: The authors identified 177 articles on
SS/L published in the last 10 years. However, only
34 of them reported any outcomes of the SS/L
projects studied, and less than one-third of these
articles included statistical analyses to test for
significant changes in outcomes. Conclusions: This
review demonstrates that there are significant gaps
in the SS/L health care quality improvement
literature and very weak evidence that SS/L improve
health care quality.

lthough the concept of evidence-based

medicine has been around for many
years and is widely accepted by health
care practitioners, evidence-based management in health care is still an evolving practice.
Evidence-based management is . . . the idea that
managers should adopt practices that scientific inquiry has shown to be effective.1(p216) Manufacturing industries have long used structured scientific
methods such as Total Quality Management, Zero Defects, Quality Circles, Continuous Quality Improvement (CQI), and Continuous Process Improvement
to reduce process variability and standardize outcomes, beginning with the use of statistical methods
for measuring and analyzing quality in the late 1930s.
Evidence-based management is particularly important in the area of quality improvement (QI) in health
care because of the need to develop and assess practices being used by managers of health care organizations to improve quality of care. With the development of information technology and new QI tools
over the last 20 years, health care managers now have
greater opportunities than their predecessors to actually practice evidence-based management and improve health care quality. Since the release of the Institute of Medicine report To Err Is Human, which
estimated that as many as 98 000 medical errors are
committed per year, there has been a strong push by
consumers, payers, and the federal government for

Author Affiliations: Division of Management, Policy, and

Community Health and Fleming Center for Healthcare
Management, University of Texas School of Public Health,
Houston (Drs DelliFraine and Langabeer II), and Yale University School of Medicine and Management, New Haven,
Connecticut (Dr Nembhard).

Key words: evidence-based management, health care,

Lean, quality, Six Sigma

Correspondence: Jami L. DelliFraine, PhD, MHA, Division

of Management, Policy, and Community Health, University
of Texas School of Public Health, 1200 Pressler, RAS E925,
Houston, TX 77030 (jami.l.dellifraine@uth.tmc.edu).

211
Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

212

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

health care organizations to increase accountability

and improve quality of care.2 For many health care
managers, this push for increased accountability and
quality has led to a re-emergence of evidence-based
management in health care.3
Although there are many QI tools used in health
care management today, Six Sigma and Lean (SS/L)
are 2 relatively new, but popular QI tools being used
in the health care industry.4,5 There have been numerous articles on the use of SS/L in health care settings.
In fact, over the last 10 years we found 177 articles
in relevant journals, and the trend in publications on
these topics is rising. The Figure presents the annual
number of citations for SS/L by year.
Both of these QI tools emphasize tracking data
and using quantitative methods to document QI and
progress toward a stated goal. Numerous books have
been written on SS/L alone, which today are the
leading QI tools in manufacturing industries. The
health care industry, however, has been slower to
adopt these methods, and the use of these tools did
not become popular until 1999, although rhetorical
evidence suggests that they are now being gradually diffused throughout hospitals on an increasing
basis.5 Yet, these new practices have been developed substantially without a theoretical foundation6
and the question of industry fit is the topic of debate for many physicians and administrators.7 However, some clinicians and managers are questioning
whether there is solid evidence that the use of SS/L is
positively associated with QI and results in improved
clinical outcomes. For example, QIs include shortening patients length of stay, reducing medication
errors, increasing on-time surgical starts, or simplifying the billing process.8,9 Some health care managers
and researchers might conclude that there is evidence
that SS/L are appropriate for the health care industry
and that there is evidence that these tools improve
quality and reduce costs. However, numerous management scholars have called named QI tools fads
and suggested that there is little real evidence that
these QI fads are beneficial.
Given the recurrence of evidence-based management in the current climate of quality and accountability in health care, it is important to examine the

tools health care managers use in order to achieve

quality. Six Sigma and Lean are 2 popular tools to
improve health care quality, including clinical outcomes, efficiency, and financial performance. Since
SS/L appear to be health care managements QI tools
du jour, it is important to examine the evidence that
these tools actually improve quality of care. Specifically, what is the evidence that SS/L tools improve
clinical outcomes, processes of care, and financial
performance in health care organizations? And if
there is evidence that SS/L improve clinical outcomes, processes of care, or financial performance,
how good is the evidence and where do we need further research? The purpose of this article was to examine the evidence of the effectiveness of SS/L QI
tools in the health care industry, identify gaps in the
health care management literature regarding these QI
tools, and identify practical recommendations and
conclusions from the SS/L literature for health care
managers.
In this study, we conducted a systematic review
of the research literature on SS/L using broader inclusion criteria and more recent dates for articles to
capture more and recent research. We assessed the
empirical evidence relating SS/L to improved clinical outcomes, processes of care, and financial performance of health care organizations. We also systematically assessed the level of evidence provided
by each study using Slavins classification system for
evidence-based practices.10 Thus, this study has 3
contributions: (1) it provides insights on the effectiveness of SS/L, important for managers selecting
among QI tools; (2) it contributes to the development
of evidence-based management and related research,
a growing field and concern; and (3) it informs health
care service researchers of the research opportunities
in this area.
We performed a structured review of peer-reviewed
literature that focused on the use and application of
SS/L in the health care industry. Our objectives were
to review and categorize (1) tools to improve clinical outcomes, processes of care, and financial performance in order to assess the main goals and objectives of SS/L in the health care industry and (2) the
strength of the research design and data reported in

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

213

Figure 1. Six Sigma and Lean diffusion in healthcare, articles over time.

these studies, hence assessing the evidence of effectiveness of SS/L on clinical outcomes, processes of
care, and financial performance. On the basis of these
findings, we evaluated the evidence of effectiveness
of SS/L in the health care industry and delineate the
need for future research.

SIX SIGMA AND LEAN

Six Sigma
Six Sigma was developed in the mid-1980s at
Motorola as an improvement concept that focused on
reduction of errors by establishing aggressive goals
for quality.11 Six Sigma measures quality in terms of
defect rates and sets a target error rate of no more
than 3.4 defects per million opportunities, or 6 standard deviations from the process mean. Six Sigmas
core philosophy focuses mainly on reducing variability. Output variability is reduced by implementing
a tightly controlled process. This is done through a
methodology that uses 5 basic processesdefining,
measuring, analyzing, improving, and controlling. In
short, a problem is defined, data are collected, and
statistical methods are used to determine sources of

variation and opportunities to improve. Processes

are then adjusted to remedy the problem, and data
are collected and analyzed multiple times to check
for improvement in error rates. Prior research suggests that many organizations must undergo a cultural change for the use of Six Sigma to result in performance improvement.12,13
Lean
Originally started at Toyota, and therefore also labeled the Toyota Production System, Lean tools
rely on creating standardized and stable processes in
order to provide the best quality services or products as efficiently as possible. Lean focuses on removing waste and unnecessary steps from processes.
Lean philosophy embraces a continuous improvement strategy that supports creating simple and direct pathways and eliminating loops or forks in a
system.14 In a process similar to Six Sigma, Lean QI
methodology defines an inefficient process, identifies waste within the process by delineating valueadded and nonvalue-added activities, improves the
process by creating standardized work, and uses standardized metrics to guide the work. Like Six Sigma,

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

214

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

Table 1
LEAN VERSUS SIX SIGMA EPISTEMOLOGY19
Dimension
Goals
Approach
Principal tool/method
Infrastructure
Methodology
Performance metrics

Six Sigma

Lean

Conformance to customer requirements,

elimination of defects (errors, rework)
Reduction of process variability
Statistical process control, run charts,
cause and effect diagrams
Through formalized structures, titles, and
roles
DMAIC (define, measure, analyze, improve,
control)
Quantifiable, cost of quality, mapped into
financial value

Lean requires cultural change to result in performance improvement.15

Six Sigma versus Lean
Although overarching goals are similar for both
tools, Lean focuses on doing the right things (valueadded activities) and Six Sigma focuses on doing
things right (with no errors). Lean also requires a
more traditional improvement methodology similar to Demings PDSA (plan, do, study, act) cycle
and relies on central individuals/roles such as sensei (master teacher) to diffuse new beliefs and promote cultural value shifts. In addition, in contrast
to Six Sigma, Lean focuses less on analytical techniques and error rates and more on process and cultural change. Despite these conceptual differences,
SS/L are sometimes used in conjunction with each
other.1618 They are viewed as complementary in process improvement projects, primarily because both
tools focus on eliminating waste and redundancy in
operational processes. Six Sigmas focus on statistical rigor and control of variation and Leans focus on
reduction of nonvalue-added activities both require
data collection and analysis to improve performance.
Table 1 presents the key characteristics of both
SS/L.

Remove nonvalue-added activities, eliminate

waste (errors, wait times)
Standardization, production flow leveling
Value stream mapping, Kanban, 5S
Cultural change, Sensei relationships
PDSA (plan, do, study, act)
Not consistent, often result in new metrics

METHODS
We conducted a structured systematic review of
articles on the use of SS/L in health care settings/
organizations that were published between 1999 and
June 2009. We selected 1999 as our start year because until 1999, the use of SS/L in health care did
not become widespread.5 We searched the research
literature using 5 electronic bibliographic databases:
PubMed, Medline, Proquest, Academic Search Complete, and Business Source Complete. We did not
limit our search to health services journals because
of the possibility that relevant articles may have appeared in business or general interest journals. We
searched the 5 databases using the following key
words and word combinations: (1) Six Sigma and
health and outcomes or finance or quality
or process or performance and (2) Lean and
health and outcomes or finance or quality
or process or performance. These key words and
phrases were chosen to capture as many articles related to the use of SS/L in health care as possible.
To ensure the comprehensiveness of our search, we
also extracted articles from the bibliographies of the
articles found in our primary search.
Our search yielded 177 studies related to SS/L
in the health care industry. The Figure presents the

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

215

Table 2
EMPIRICAL STUDIES EXAMINING SIX SIGMA AND LEAN

Authors

Quality
Improvement Research
Method
Design Data

Adams et al20

SS

Process, some
financial

Ben-Tovim
et al21

Process

Buck22

SS

Process

Bush et al23

SS

Process, some
financial

Chan and
Pharm24

SS

Process, some
financial

Chan et al25

SS

Process

Chassin26

SS

Process, some
financial

Chen et al27
Christianson
et al28

SS
SS

4
4

3
3

Process
Process, some
financial

Project
Type

Outcomes
OR turnaround
time, revenue
potential gained
Wait time in the ED,
compliance with
ED triage, medical
and surgical bed
admission, and
discharge
Medication errors,
laboratory errors
Wait times for new
obstetric visits,
patient time spent
in clinic, return
visits, number of
newpatient visits,
patient
satisfaction
Medication errors,
including drug
type and dose
Magnetic resonance
imaging
examination time
Medication errors,
laboratory
turnaround time,
bed availability,
revenue
enhancement,
revenue capture
X-ray film defects
ED throughput time,
medication errors,
clinic throughput,
Medicare
profitability

Setting

Evidence
Score
(lower
indicates)
stronger
evidence)

Hospital OR

Hospital ED and
inpatient units

Hospital pharmacy
and laboratory
Hospital-based
outpatient
obstetrics/
gynecology clinic

Hospital pharmacy

Radiology
department in a
hospital
Hospital

Hospital
Hospital

7
7

(continues)

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

216

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

Table 2
EMPIRICAL STUDIES EXAMINING SIX SIGMA AND LEAN (CONTINUED)

Authors

Quality
Improvement Research
Method
Design Data

Daniels29

SS

Dickson et al30 L

Does et al17

SS

Drenckpohl
et al31

SS

Elberfeld
et al32

SS

Eldridge
et al33
Esimai18

SS

SS/L

Fairbanks16
SS/L
Gamerdinger34 SS

4
4

1
3

Godin et al35

SS

Gorman et al36 SS

Griffin et al37

SS

Project
Type

Outcomes

Setting

Evidence
Score
(lower
indicates)
stronger
evidence)

Managed care
Claims processing,
company
claims
adjustment, held
claims
Process and
Patient satisfaction, Hospital ED
financial
expense per
patient, ED length
of stay, patient
volume
Process, some OR start time delay 13 hospitals in the
financial
Netherlands
Process
Breast milk
Hospital
administration
errors
4-hospital health
Process
Correct use of
system
aspirin,
beta-blockers, and
angiotensin
enzyme inhibitors
Process
Hand-hygiene
3 hospitals, ICUs
practices
Hospital pharmacy
Process, some Medication errors,
financial
including drug
type, duplicate
orders, and
incorrect dose
Process
OR throughput
Hospital
Hospital
Clinical, some Mortality and LOS
financial
for patients with
communityacquired
pneumonia
Process, some Stress test
Hospital, nuclear
financial
turnaround times
medicine
department
Process
Radiology report
Hospital
turnaround times
Clinical
Hospital-acquired
Hospital
pressure ulcers

Process and
financial

6
7

5
5

5
7

6
7
(continues)

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

217

Table 2
EMPIRICAL STUDIES EXAMINING SIX SIGMA AND LEAN (CONTINUED)

Authors

Quality
Improvement Research
Method
Design Data

Project
Type

Johnson et al38 SS

Process

Kang et al39

SS

Process

Kelly et al40

Process

LeBlanc et al41 SS

Process

Leslie et al42
L
Lloyd and
SS
Holsenback43
Morgan and
SS
Cooper44

4
4

2
3

Process
Financial and
process
Process

NelsonPeterson
and Leppa45
Neri et al46

Process

SS

Process, some
financial

Raab et al47

Process

Shukla et al48

SS

Clinical

Outcomes

Setting

Evidence
Score
(lower
indicates)
stronger
evidence)

Hospital
Patient wait times,
turnaround times,
LOS in multiple
areas of the
hospital
PACS system errors Hospital radiology
department
Ambulance bypass, Hospital ED
ED wait times
Hospital cardiac
Cardiac
catheterization
catheterization
laboratory
laboratory
throughput
OR turnaround time Hospital OR
Radiology costs
3-hospital system
Unavailable
medications, IV
starts, supply and
equipment
availability,
nursing
satisfaction
Nursing time spent
doing nonvalueadded activities
Blood product
defects and
inappropriate
utilization
Histopathology
laboratory
turnaround time
and work units/
full-time
equivalents
Sphincter
preservation in
rectal cancer

6
5
6

6
7

Hospital

Hospital telemetry
unit

Multihospital
health system

Hospital

Hospital surgical
unit

(continues)

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

218

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

Table 2
EMPIRICAL STUDIES EXAMINING SIX SIGMA AND LEAN (CONTINUED)

Authors

Quality
Improvement Research
Method
Design Data

Project
Type

van den
Heuvel
et al49

SS

Process

van den
Heuvel
et al9

SS

Process and
financial

Outcomes
Number of surgical
operations,
number of
admissions, LOS,
number of new
patient visits, and
others
Cost savings,
number of
patients receiving
IV antibiotics,
LOS

Setting

Evidence
Score
(lower
indicates)
stronger
evidence)

Hospital

Hospital

Abbreviations: ED , emergency department; IV, intravenous; L. Lean; LOS, length of stay; OR, operating room; SS, Six Sigma.

annual number of citations for SS/L by year. Of the

177 studies published during the study period, 70%
were related to Six Sigma, 23% were related to Lean,
and 7% were related to both SS/L. The first 2 authors
independently reviewed the abstracts of the 177 articles using predetermined inclusion/exclusion criteria and resolved disagreements through discussion
until they reached consensus. The criteria for article
inclusion were (1) article was published in a peerreviewed journal, (2) article described the implementation of a QI project utilizing SS/L in a health care organization as well the outcome or result of the project,
and (3) article reported on an empirical study that
used statistical methods to evaluate the effect of using SS/L to improve clinical processes or outcomes.
To make the search as inclusive as possible, we did
not exclude any articles on the basis of the type of
health care organization or QI project studied. Thus,
our review covers a wide range of organizations and
projects. However, because of our criteria, our review
excludes articles that described the use of SS/L as a
case study without providing performance data, pro-

vided only pre-SS/L performance data, simply discussed the potential use of SS/L to address an issue,
or commented only on the appropriateness of using
SS/L in health care organizations.
Using these criteria, we excluded 124 articles after
abstract review. The remaining 53 articles were independently reviewed by the first 2 authors using the
same criteria and process for resolving disagreements
used at the abstract stage. After article review, 19 additional articles were excluded. The 34 studies that
met our inclusion criteria are listed in Table 2.
Each of the 34 SS/L studies was reviewed to assess the quality of the study, including its research
design and adequacy of data collected and reported,
as well as the type of outcome/project reported (clinical, process, or financial). We assessed the strength
of the research design using Slavins criteria10 for scientific evidence. The ratings, in order of most rigorous to least rigorous, are 1 = randomized controlled trials, 2 = nonrandomized controlled trials, 3 = quasi-experiments with nonequivalent control groups, and 4 = observational studies with no

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

control group. Each article was assigned the rating

that reflected its study design. In addition, using
Slavins criteria,10 each article was assigned the rating that reflected the strength of the data collected and
reported. The ratings were 1 = article described the
sample, measures, outcome data, and statistical test
results; 2 = article described the sample, measure and
outcome data, and measures but did not adequately
conduct or report statistical data results; and 3 = article described the sample and measures but did not adequately report outcome data and statistical tests. We
summed the ratings for research design and data quality to create an overall evidence score for each study.
Evidence scores are reported in Table 2 along with
the focus of the SS/L projectclinical, process, or
financial. Lower scores indicate studies with stronger
research designs and data, indicating stronger evidence on the effectiveness of SS/L. We discuss the
most rigorous studies, those with evidence scores
lower than 7, in our Results section.

RESULTS
Level of evidence supporting SS/L
Of the 34 articles that met our review criteria, 26
focused on Six Sigma, 6 focused on Lean, and 2 focused on both SS/L. Evidence scores for these articles
ranged from 4 to 7, with 4 indicating stronger research
design, data presentation, and analysis than articles
scoring a 7. The average evidence score for the studies
included in the review was 6.1. Studies focusing on
Lean QI tools had an average evidence score of 5.7,
whereas studies focusing solely on Six Sigma tools
had an average evidence score of 6.2. Studies that incorporated both Lean and Six Sigma QI tools had an
average evidence score of 5. Nearly all studies in the
review were conducted in hospital settings, or multihospital systems. Only 1 study was conducted in a
managed care organization. The 34 studies that met
our criteria for review are listed in Table 2. Results of
these studies are grouped according to project focus
(improving clinical outcomes or processes of care),
and financial outcomes of these projects are reported
within these sections.

219

SS/L to improve clinical outcomes

There were only 3 studies that used SS/L to
improve clinical outcomes,34,37,48 and the average
evidence score of these studies was 6.3. Clinical
outcomes that were examined include mortality
for patients with community-acquired pneumonia,34
hospital-acquired pressure ulcers,37 and sphincter
preservation rates for patients with rectal cancer.48
Only 1 study48 among these 3 adequately reported
outcome results. Shukla et al48 used Six Sigma to
evaluate whether a new double-stapling surgical
technique improved sphincter preservation rates in
patients with rectal cancer. They found that sphincter preservation rates were statistically significantly
higher when using the new technique compared with
the old technique. Although the other studies also
reported improved outcomes (reduced mortality, reduced pressure ulcer rate, and improved cardiac
medication administration) from implementing Six
Sigma, the studies either did not report the outcome
data in detail or did not report statistics demonstrating that the improved outcomes were a significant improvement over the preintervention period. For example, Griffin et al37 provide a graphical presentation
of the data that shows a marked decrease in pressure
ulcers after a Six Sigma QI implementation, but without statistical analyses one does not know whether
this decrease was statistically significant. One article reviewed reported a cost savings of more than
$100 000 annually by reducing length of stay.34
SS/L to improve processes of care
Eight articles focused on using SS/L to improve
processes of care; these varied considerably in
breadth of the projects and depth of supporting data
and statistics on outcomes. These articles contained
descriptions of SS/L projects that were implemented
throughout several areas in a hospital or health system (such as the hospital pharmacy, hospital laboratory, and emergency department), and therefore reported outcomes for process improvement projects
in multiple areas of the hospital of health care
system9,21,22,26,28,38,44,49 All of the 8 articles reported
positive results for the process improvement projects,

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

220

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

indicating that SS/L was effective at improving processes of care. Although broad in scope, these articles
often did not report in-depth statistics on the outcomes of the projects that were implemented. The
average evidence score of these broad-scope articles
was 6.75 out of 7, with 7 indicating the lowest possible level of evidence. However, 2 articles presented a
rich description of the process improvement projects
and the measures used to monitor the outcomes of
the projects.26,28
Ten articles included in our review focused on using SS/L to improve multiple processes in 1 area
of the hospital, health care system, or organization
(such as number of return visits, patient satisfaction, and wait times for new patient visits in an obstetrics/gynecology clinic).16,18,23,24,29,30,40,43,45 The
average evidence score for these articles was 5.7,
indicating that articles that focused more narrowly
on more targeted areas were able to present stronger
in-depth evidence that SS/L can improve processes
of care, although the evidence was still relatively
weak. Other articles had an even narrower focus on
only 1 process improvement project with only 1 or
2 outcomes in 1 area of a hospital or health system, such as magnetic resonance imaging examination time.17,20,25,27,31,33,35,36,39,41,42,47 The average evidence score for these articles was 5.75, again suggesting that articles with a narrow focus present stronger
in-depth evidence that SS/L can improve processes
of care.
The articles in our review contained some
common types of projects for improving processes of care. Common process improvement
projects included improving operating room (OR)
throughput,9,16,17,20,42 improving emergency department (ED) throughput,21,28,30,38,40 reducing medication errors,18,22,24,26,28 reducing patient wait
times,21,23,28,38 reducing other turnaround times (not
OR or ED),23,25,26,28,29,35,36,38,41,47 reducing other errors (nonmedication),27,39 and following best practices of care.9,31,32,33,38,46
OR throughput
In total, there were 5 articles that focused on
using SS/L to improve OR throughput,9,16,17,20,42,49

although 3 articles appear to report results on

the same hospital system and implementation
projects.9,17,49 There were several factors identified that impacted OR throughput, including room
cleanup, unclear staff assignments, and complete
case charts,20,42 OR planning, scheduling, and anesthesia technique,17 and physician arrival time.9,16 In
all of these examples, the use of Six Sigma or Lean
was used to target these factors in order to reduce
start-time delays and decrease turnaround time between surgical cases. In some cases, improved OR
throughput resulted in overall increased surgical volumes for the hospitals in the study.17,20 Fairbanks
et al16 and Adams et al20 provide especially detailed
descriptions and statistical analyses of the problems
slowing OR throughput, the solutions implemented,
and the supporting statistical evidence that the SS/L
interventions were effective. Although not all of the
studies on OR throughput examined cost savings
associated with improved throughput, 1 study estimated the cost savings to range from $350 000 to
$500 000 annually,17 while another study estimated
potential revenue gained to be $617 000 annually.20
ED throughput
There were 5 articles that focused on using SS/L to
improve ED throughput,21,28,30,38,40 and all of these
articles found that SS/L was effective in improving
ED throughput. Outcomes examined in these articles include ED workload or volume,21,28,30,40 patient
wait time,40 ambulance diversion,38,40 patient walkout rate,21,28,38 length of stay,21,28,30 and patient satisfaction, even though satisfaction is not technically
a part of throughput.28,30 There were several factors
identified that impacted ED throughput, including
appropriate triage of patients,21 turnaround time of
other services such as laboratory and radiology,28
lack of available beds,40 registration and discharge
processes,21,28,30 and staffing patterns.21,30,40 In these
articles, the use of Six Sigma or Lean was used to
target these factors in order to improve ED throughput. Three articles focused solely on the application
of Lean to improve ED throughput.21,30,40 All 3 articles found that after the Lean implementation, patients length of stay or wait time decreased while ED

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

workload increased significantly. None of the articles

examining the use of SS/L to improve ED throughput
estimated cost savings or revenue generated from the
implementation.
Medication errors
Reducing medication errors was another common
application of SS/L.18,22,24,26,28 The authors of these
articles identified several factors that led to medication errors in hospital settings. Although one
article focused on the pharmacy as the primary
source of origin for medication errors,24 other articles focused on medication errors originating from
several sources, including physicians, nurses, and
pharmacists.18,22,28 Medication errors happened in
numerous ways, including dispensing the wrong
medication, dispensing the wrong dosage, dispensing a contraindicated medication, duplicate order entries, order entry failure or missing order entry, dispensing medication to the wrong patient, dispensing
the wrong frequency, and dispensing the medication
in the wrong route (oral instead of intravenous). In all
the articles except 1,28 the implementation of an SS/L
program was effective at reducing medication errors.
In the case of Christianson et al,28 the Six Sigma initiative was not implemented because of lack of participant support. Esimai18 provides good analytical
evidence that SS/L can be effectively implemented to
reduce medication errors and estimates that the hospital saved $1.32 million in labor costs as a result of
implementing the Six Sigma program, although it is
not clear how the cost savings were estimated. None
of the other articles estimated cost savings from using
SS/L to reduce medication errors.
Patient wait times
Another set of SS/L articles was focused on
improving health system delivery efficiency
and service by reducing patient wait times,23,38
reducing other turnaround times (not OR or
ED),23,25,26,28,29,35,36,38,41,47 or reducing other errors (nonmedication).27,39 SS/L tools were used
to reduce turnaround times in clinical areas such
as the cardiac catheterization laboratory,26,38,41
radiology
department,25,36,38
pathology
and

221

laboratory departments,26,47 nuclear medicine

department,35 medical/surgical units,26 and an
obstetrics/gynecology clinic,23 as well as nonclinical
areas such as environmental services,38 billing
compliance,38 and claims processing.29 Two articles
focused on using SS/L to reduce errors in radiology systems or machines.27,39 These systems and
machine errors include picture archiving and communication system (PACS) radiology system errors39
and defects in radiography films.27 Although all 12
of the articles report reduced turnaround times and
patient wait times, only 4 report statistically significant reductions in patient waiting and turnaround
times.23,25,35,47 One article compares turnaround
times to a nonequivalent control group; out of 34
studies, this was the only study that had a comparison group, and the authors estimate an increased
revenue of 73% for the obstetrics/gynecology clinic
in the first 6 months.23 Other estimated cost savings
and revenue generated ranged from $34 000 saved
from reduced staffing,35 a 49% decrease in claims
costs,29 an increased revenue of $18 000,25 and an
increased revenue of more than $5 million (using
SS/L for multiple system-wide projects).26
Processes for patient care
In 6 articles, SS/L were used to implement and
improve best practices of patient care.9,31,32,33,38,46
Although these articles did not measure clinical
outcomes, most of the processes targeted for improvement can have a direct impact on patient outcomes.
For example, best practices that were targeted for implementation or improvement using SS/L include the
correct use of aspirin, beta-blockers, and angiotensin
enzyme inhibitors32 ; the reduction of breast milk administration errors31 ; the reduction of patients receiving intravenous antibiotics9 ; the reduction of fatal falls, increased standardized medication use, and
the reduction of hypoglycemic episodes38 ; the implementation of the Centers for Disease Control and Prevention Guideline for Hand Hygiene in Health Care
Settings33 ; and the reduction of inappropriate blood
product utilization.46 Although all of these articles
report an increase in the use of the best practices targeted and a reduction in any associated errors, only

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

222

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

Eldridge et al33 demonstrated a statistically significant improvement in best practices of care for the correct use of hand hygiene practices. Although many of
these process improvement tools were implemented
to improve patient care and not necessarily to save
money, several studies report cost savings after using SS/L to improve practices. Specifically, 1 study
reported cost savings in excess of $75 000 by switching from intravenous to oral antibiotics,9 and another
study reported saving $121 000 by reducing inappropriate blood product utilization.46

DISCUSSION
The purpose of this article was to conduct a comprehensive literature review to assess the empirical evidence that the use of SS/L improves clinical
outcomes, processes of care, and financial performance of health care organizations. Our review
shows that the evidence that these QI tools improve
health care quality is relatively sparse, both in the
number of studies and in the diffuse nature of the
studies. We identified 34 empirical research articles
on SS/L published between 1999 and 2009. Although
this number of studies is nearly double the number of
studies (n = 18) found in an earlier review50 that did
not include the 2 most recent years, this is a small
number of studies relative to the research base for
other QI tools. For example, Shortell et al51 identified 55 studies in their review of the research evidence for CQI in health care, using similar inclusion
criteria to ours over a shorter time frame (5 years vs
our 9 years). Nevertheless, several conclusions about
the current evidence of effectiveness of SS/L QI programs in health care settings can be drawn from our
review.
First, we found that the level of evidence supporting a positive relationship between the use of SS/L
and performance improvement was weak. With an
average evidence score of 6.1 on a scale for which
7 indicates the lowest level of scientific evidence,
there is little rigorous evidence validating the effectiveness of SS/L for QI. Only 11 studies in our review conducted any statistical analyses to test for
a statically significant improvement after the SS/L

intervention. The evidence that SS/L improves processes of care is slightly stronger than the evidence
that SS/L improves clinical outcomes. Financial outcomes were measured in 13 articles, but 12 of these
articles focused on improving processes of care and
only 1 focused on improving clinical outcomes. The
overall evidence for these studies is relatively weak
but would be stronger with better study designs. Although several articles reported data and results, only
1 article used a nonequivalent control group design.
No articles used randomized controlled trials. Therefore, the breakdown in evidence largely occurs in the
weak study designs of the articles.
Second, most studies focused on SS/L to improve
processes of care, while few studies focused on SS/L
to improve clinical outcomes. This may be because
clinical outcomes are often more difficult to measure.
However, given the link between processes of care
and clinical outcomes, it stands to reason that the
demonstrated effectiveness of SS/L to improve processes of care translates into effectiveness to improve
clinical outcomes as well. For example, Eldridge et
al33 demonstrated a statistically significant improvement in best practices of care for the correct use of
hand hygiene practices. The correct use of hand hygiene practices is directly linked to a decrease in
the number of nosocomial infections acquired by patients in the hospital. Therefore, we might hypothesize that the incidence of nosocomial infections
diminished in the hospitals they studied. Unfortunately, they did not examine this outcome. Likewise,
several articles demonstrated the use of SS/L to reduce the number of medication administration errors. Medication administration errors can be directly
linked to adverse events, but none of these studies examined the number of adverse events in patients.
Our review suggests that there is a trade-off between breadth and depth in SS/L research. Studies
of the implementation of SS/L in several areas of
a health care system reported results from multiple
projects9,26,28,49 but did not report in-depth statistical findings about individual projects and hence had
lower evidence scores. In contrast, other articles focused on 1 specific project in 1 area of a hospital
or health care system and reported detailed results

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

of the project and had higher evidence scores. This

trade-off likely exists because of space limitations in
most journals, and authors reporting results from a
very large study may sacrifice detail to include all
aspects of the study. However, a better understanding of the effectiveness of SS/L would be provided
by additional detailed studies focusing on specific areas that demonstrate statistically significant improvements after SS/L implementation. This would help
provide evidence that the changes were due to SS/L
and not other factors. In addition to more detailed
analyses, future research should include stronger research designs to examine the relationship between
SS/L and QI. Only 1 study in this review had a comparison group, whereas the other studies research designs were pre-post comparisons. Without comparison groups, it is difficult to tell if clinical and process
improvements were due to the SS/L tools or other
explanations.
Third, more studies need to report the costeffectiveness of SS/L QI tools. Although 41% of the
studies we reviewed contained some information on
the cost savings or revenue generated after implementing SS/L, cost savings and revenue generation
were not the primary focus of the studies. Although
improving health care processes and quality is certainly the main purpose of most SS/L QI tools, studies
should carefully consider the value (benefit relative
to cost) of implementing these tools as well. No studies discussed the costs of training and/or hiring personnel for these QI tools, or the indirect costs of
labor spent in implementing these tools. Several articles we reviewed reported financial savings and described SS/L as potential cost-saving mechanisms,
but managers in health care systems would benefit
from detailed studies describing actual costs of implementation. Consultant fees and training in SS/L is
not inexpensive, and in some instances the costs of
training and implementation may outweigh the benefits, especially if there are less expensive QI options.
Fourth, we found limited literature on the failures
of SS/L. Only 1 study in our review described an
instance where SS/L was attempted but not implemented because of lack of support,28 and no studies
reported negative results from the SS/L implemen-

223

tations. This may be because SS/L usually results

in desired performance improvement. Alternatively,
the absence of articles reporting no or negative findings may reflect a publication bias. If the latter is the
case, the current literature may overstate the effectiveness of SS/L and therefore, may be misleading for
health care managers considering implementing one
or both of these QI tools in their organizations. Given
the adage that we learn more from our failures than
from our successes, studies on the failures of SS/L
to improve processes of care would serve as valuable
teaching lessons.

CONCLUSION
This review demonstrates that there are significant
gaps in the SS/L health care QI literature and very
weak evidence that SS/L improve health care quality. We conclude that the literature provides some
guidance on potential applications of SS/L to managers wishing to implement SS/L in their own health
care organizations and caution that the literature contains few examples of failures or partial successes of
SS/L projects, or a full scope of costs. Although there
certainly is potential for QI and cost savings in implementing SS/L in health care settings, more research
with rigorous designs and detailed statistical reporting is needed to bolster the evidence base for the use
of these QI tools.
In conclusion, SS/L are 2 popular tools currently
being used to improve clinical outcomes, efficiency,
and financial performance. However, the current focus on evidence-based management to improve quality in health care cautions that current trends in
evidence-based management are largely based on
conceptual arguments that evidence-based management improves best practices because there is little empirical research that evidence-based management demonstrates effectiveness.52,53 The findings
from this review largely support this argument, implying that while there is little solid evidence of the
effectiveness of SS/L, managers may be using these
tools as a means to achieve best practices and improve quality of care.

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

224

QU A LITY MA N A G EM EN T

IN

HEA LTH CA RE /VO LU M E 19, ISSU E 3, JU LYSEPTEM BER 2010

REFERENCES
1. McDaniel R, Lanham HJ. Evidence as a tool for managerial
action: a complex adaptive systems view. Health Care Manag
Rev. 2009;34(3):216-218.
2. Institute of Medicine. To Err Is Human: Building a Safer
Health System. Washington, DC: Institute of Medicine National Academy Press; 2000.
3. Rundall TG, Martelli PP, Arroyo L, et al. The informed
decisions toolbox: tools for knowledge transfer and performance improvement. J Healthc Manag. 2007;52(5):325343.
4. DelliFraine JL, Langabeer JR. An institutional perspective on
quality initiatives: evidence beyond manufacturing. Int J Inf
Syst Change Manag. 2009;4(1):3-14.
5. Pande PS, Neuman RP, Cavanagh RR. The Six Sigma Way.
New York, NY: McGraw-Hill; 2000.
6. Linderman K, Schroeder R, Zaheer S, Choo A. Six Sigma:
a goal-theoretic perspective. J Oper Manag. 2003;21(2):193203.
7. Kassirer JP. Doctor discontent. N Engl J Med. 1998;339:15431545.
8. de Koning H, Verver J, van den Heuvel J, Bisgaard S, Does R.
Lean Six Sigma in healthcare. J Healthc Qual. 2006;28(2):4-11.
9. Van den Heuvel J, Does RJ, Bogers AJ, Berg M. Implementing Six Sigma in the Netherlands. J Qual Patient Saf.
2006;32(7):393-399.
10. Slavin RE. Best evidence synthesis: an intelligent alternative to
meta-analysis. J Clin Epidemiol. 1995;48(1):9-18.
11. Harry M, Schroeder R. Six Sigma. New York, NY: Doubleday
Publishers; 2000.
12. Antony J, Banuelas R. Key ingredients for the effective
implementation of Six Sigma program. Meas Bus Excell.
2002;6(4):20-27.
13. Banuelas Coronado R, Antony J. Critical success factors for the
successful implementation of Six Sigma projects in organizations. The TQM Mag. 2002;14(2):92-99.
14. Printezis A, Gopalakrishnan M. Current pulse: can a production system reduce medical errors in health care? Qual Manag
Health Care. 2007;16(3):226.
15. Liker J, Meier D. The Toyota Way. New York, NY: McGrawHill; 2006.
16. Fairbanks C. Using Six Sigma and Lean methodologies to improve OR throughput. AORN J. 2007;86(1):73-82.
17. Does RJ, Vermaat TM, Verver JP, Bisgaard S, van den Heuvel J.
Reducing start time delays in operating rooms. J Qual Technol.
2009;41(1):95-109.
18. Esimai G. Lean Six Sigma reduces medication errors. Qual
Prog. 2005;38(4):51-57.
19. Langabeer JR, DelliFraine JL, Heineke J, Abbass I. Determinants of goal attainment and performance outcomes from quality improvement initiatives in the healthcare industry. Oper
Manag Res. 2009;2(1):1327.
20. Adams R, Warner P, Hubbard B, Goulding T. Decreasing
turnaround time between general surgery cases. J Nurs Adm.
2004;34(3):140-148.
21. Ben-Tovim DI, Bassham JE, Bennett DM, et al. Redesigning

22.
23.

24.

25.

26.
27.

28.

29.
30.

31.

32.

33.

34.
35.
36.

37.

38.

39.

40.

care at the Flinders Medical Centre: clinical process redesign

using lean thinking. Med J Aus. 2008;188(6):S27-S31.
Buck C. Application of Six Sigma to reduce medical errors.
Qual Cong ASQs Annu Qual Congress Proc. 2001;739-742.
Bush SH, Lao MR, Simmons KL, Goode JH, Cunningham SA,
Calhoun BC. Patient access and clinical efficiency improvement in a resident hospital-based womens medicine center
clinic. Am J Manag Care. 2007;13:686-690.
Chan ALF, Pharm BS. Use of Six Sigma to improve pharmacist dispensing errors at an outpatient clinic. Am J Med Qual.
2004;19(3):128-131.
Chan WP, Chiu W, Chen W, Lin M, Chu B. Applying Six Sigma
methodology to maximize magnetic resonance imaging capacity in a hospital. Int J Healthc Tech Manag. 2005;6(3):321330.
Chassin R. The Six Sigma initiative at Mount Sinai Medical
Center. Mount Sinai J Med. 2008;75:45-52.
Chen Y, Lin J, Chang C. Taiwan: improving radiography
through application of Six Sigma techniques. J Healthc Qual.
2005;27(3):44-52.
Christianson JB, Warrick LH, Howard R, Vollum J. Deploying
Six Sigma in a health care system as a work in progress. J Qual
Patient Saf. 2005;31(11):603-613.
Daniels SE. Six Sigma at Cigna. Qual Prog. 2007;40(5):43-48.
Dickson EW, Singh S, Cheung DS, Wyatt CC, Nugent AS. Application of Lean manufacturing techniques in the emergency
department. J Emer Med. 2008;11(108):1-6.
Drenckpohl D, Bowers L, Cooper H. Use of the Six Sigma
methodology to reduce incidence of breast milk administration
errors in the NICU. Neonatal Netw. 2007;26(3):161-166.
Elberfeld A, Goodman K, Van Kooy M. Using the Six Sigma
approach to meet quality standards for cardiac medication administration. J Clin Outcomes Manag. 2004;11(8):510-516.
Eldridge NE, Woods SS, Bonello RS, et al. Using the Six Sigma
process to implement the Centers for Disease Control and Prevention guideline for hand hygiene in 4 intensive care units. J
Gen Int Med. 2006;21:S35-S42.
Gamerdinger K. Six Sigma CAP initiative reduces mortalities.
Healthc Benchmarks Qual Imp. 2008;4-6.
Godin E, Raven D, Sweetapple C, Del Giudice FR. Faster test
results. Qual Prog. 2004;37(1):33-39.
Gorman A, Donnell L, Hepp H, Mack T. Improving communication and documentation concerning preliminary and final radiology reports. J Healthc Qual. 2007. http://www.nahq.
org/journal/ce/article.html?article id=276. Accessed July 6,
2009.
Griffin B, Cooper H, Horack C, Klyber M, Schimmelpfenning
D. Best-practice protocols: reducing harm from pressure ulcers. Nurs Manag. 2007;29-32.
Johnson T, Currie G, Keill P, Corwin SJ, Pardes H, Cooper MR.
New York-Presbyterian Hospital: translating innovation into
practice. J Qual Patient Saf. 2005;31(10):554-560.
Kang JO, Kim MH, Hong SE, Jung JH, Song MJ. The application
of the Six Sigma program for the quality management of the
PACS. Am J Roentgenol. 2005;185:1361-1365.
Kelly A, Bryant M, Cox L, Jolley D. Improving emergency department efficiency by patient streaming to outcomes-based
teams. Aus Health Rev. 2007;31(1):16-21.

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Assessing the Evidence of Six Sigma and Lean

41. LeBlanc F, McLaughlin S, Freedman J, Sager R, Weissman M. A

Six Sigma approach to maximizing productivity in the cardiac
cath lab. J Cardiovasc Manag. 2004;19-24.
42. Leslie M, Hagood C, Royer A, Reece CP, Maloney S. Using Lean methods to improve OR turnover times. AORN J.
2006;84(5):849-855.
43. Lloyd DH, Holsenback JE. The use of Six Sigma in health care
operations: application and opportunity. Acad Health Care
Manag J. 2006;2:41-49.
44. Morgan SP, Cooper C. Shoulder work intensity with Six Sigma.
Nurs Manag. 2004;35(3):28-32.
45. Nelson-Peterson D, Leppa CJ. Creating an environment for
caring using Lean principles of the Virginia Mason production
system. J Nurs Adm. 2007;37(6):287-294.
46. Neri RA, Mason CE, Demko LA. Application of Six Sigma/CAP
methodology: controlling blood-product utilization and costs.
J Healthc Manag. 2008;53(3):183-196.
47. Raab SS, Grzybicki DM, Stewart WR, Turcsanyi BD, Mahood
LK, Becich MJ. Effect of Lean method implementation in the
histopathology section of an anatomical pathology laboratory.
J Clin Path. 2008;61:1193-1199.

225

48. Shukla PJ, Barreto SG, Nadkarni MS. Application of Six Sigma
towards improving surgical outcomes. Hepatogastroenterology. 2008;55:311-314.
49. Van den Heuvel J, Bogers AJ, Does RJ, van Dijk SL, Berg M.
Quality management: does it pay off? Qual Manag Health
Care. 2006;15(3):137-149.
50. Vest J, Gamm L. A critical review of the research literature on
Six Sigma, Lean and StuderGroups Hardwiring Excellence in
the United States: the need to demonstrate and communicate
the effectiveness of transformation strategies in healthcare. Implement Sci. 2009;4(35):1-9.
51. Shortell SM, Bennett CL, Byck GR. Assessing the impact of
continuous quality improvement on clinical practice: what
it will take to accelerate progress? Milbank Q. 1998;76:593624.
52. Issel M. Evidence on all sides. Health Care Manag Rev.
2009;34(3):205.
53. Arndt M, Bigelow B. Evidence-based management in health
care organizations: a cautionary note. Health Care Manag Rev.
2009;34(3):206-213.

Copyright 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.