Watson Health 100 Top Hospitals Study 2019: 26th Edition - March 4, 2019

Watson Health
100 Top Hospitals Study

2019
26th edition | March 4, 2019

IBM Watson Health™
75 Binney Street
Cambridge, MA 02142
800-525-9083
ibm.com/watsonhealth
Watson Health 100 Top Hospitals Study, 2019; 26th edition
© 2019 IBM Watson Health. All rights reserved. IBM, the IBM logo, ibm.com,
Watson Health, and 100 Top Hospitals are trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide. Other product and service
names might be trademarks of IBM or other companies.
Printed and bound in the United States of America.
The information contained in this publication is intended to serve as a guide for general
comparisons and evaluations, but not as the sole basis upon which any specific
conduct is to be recommended or undertaken.
The reader bears sole risk and responsibility for any analysis, interpretation, or
conclusion based on the information contained in this publication, and IBM shall not
be responsible for any errors, misstatements, inaccuracies, or omissions contained
herein. No part of this publication may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopying, recording, or by
any information storage and retrieval system, without permission in writing from
IBM Watson Health.
ISBN: 978-1-57372-474-6
Introduction
Contents
Welcome to the 26th edition of the
03 Introduction Watson Health 100 Top Hospitals® study
07 2018 100 Top Hospitals from IBM Watson Health™.
award winners
13 2018 Everest Award winners For over 25 years, the 100 Top Hospitals program
19 Findings has been producing annual, quantitative studies
35 Methodology designed to shine a light on the nation’s highest
51 Appendix A performing hospitals and health systems.
53 Appendix B
55 Appendix C: The 2019 study of US hospitals began with the
Methodology details same goal that has driven each study since the
beginning of the 100 Top Hospitals program:
To identify top performers and deliver insights
that may help all healthcare organizations
better focus their improvement initiatives on
achieving consistent, balanced, and sustainable
high performance.
Illuminating achievement for a

value-based world
Our research is based on clinical, operational, and
patient perception-of-care measures that form
a balanced scorecard. For over 25 years, the
hospitals achieving excellence on our scorecard
inherently set attainable benchmarks for others in
the industry to aspire to over time.
Providing these measures of successful

performance may be especially important today as
the healthcare landscape continues to evolve from
fee-for-service toward value-based care models.
3
100 Top Hospitals winners
consistently set industry By finding ways to take balanced performance
benchmarks for measures to the next level, the winners of our 100 Top
like 30-day readmissions, Hospitals award are identifying opportunities to
mortality rates, patient deliver healthcare value to patients, communities,
experience, and profit and payers. The performance levels achieved
margins. by these hospitals may motivate their peers to
use data, analytics, and benchmarks to close
performance gaps.
Hospitals do not apply for our 100 Top Hospitals

selection process, and award winners do not pay
to market their honor.
Delivering a transparent assessment

To maintain the 100 Top Hospitals study’s integrity
and avoid bias, we use public data sources and
explain the methodologies we use to calculate
outcome metrics. This supports inclusion of
hospitals across the country and facilitates
consistency of definitions and data.
Our national balanced scorecard, based on Norton

and Kaplan’s concept1, is the foundation of our
research. It is comprised of key measures of
hospital performance: inpatient and extended care
quality, operational efficiency, financial health, and
customer experience. The composite score derived
from these measures reflects excellence in hospital
care, management, and leadership.
In addition, to support consideration of different

types of hospitals, the 100 Top Hospitals study
categorizes the nation’s hospitals into five groups:
major teaching, teaching, large community,
medium community, and small community
hospitals. This produces benchmarks that are
comparable and action-driving across each
organizational type. This is important because
each kind of hospital has its own set of challenges
and opportunities.
4 IBM Watson Health

Yielding a measure of leadership excellence –– Provided faster emergency care
Since 1993, the 100 Top Hospitals program –– Kept expenses low, both in-hospital and
has also sought to shed light on the efficacy through the aftercare process
of innovative leaders. The methodology is
aimed at identifying leaders who can transform –– Scored higher on patient ratings of their overall
an organization by pinpointing improvement hospital experience
opportunities and adjusting goals for key
performance domains. We believe that higher Our study projections also indicate that if the
composite scores on the balanced scorecard benchmarks of performance established by our
typically indicate more effective leadership and a 2019 winners were achieved by all hospitals in the
consistent delivery of value. US, the following would be true:
–– More than 103,000 additional lives could be
The leadership of today’s hospitals, including saved in-hospital
the board, executive team, and medical staff
leadership, is responsible for ensuring all facets of –– Over 38,000 additional patients could be
a hospital are performing at similarly high levels complication-free
in both the short and long term. The 100 Top –– Over $8.2 billion in inpatient costs could
Hospitals study and analytics provide a view of that be saved
enterprise performance alignment.
––The typical patient could be released from
Comparing the performance of our the hospital a half a day sooner and would
2019 winners to nonwinners have 12 percent fewer expenses related to
the complete episode of care than the median
Using the measures presented in our national
patient in the US
balanced scorecard, this year’s 100 Top Hospitals
study revealed significant differences between –– Over 155,000 fewer discharged patients would
award winners and their nonwinning peers. be readmitted within 30 days
–– Patients would spend 17 minutes less in
Our study’s highest-performing hospitals:
hospital emergency rooms per visit
–– Had lower inpatient mortality, considering
patient severity This analysis is based on applying the difference
between study winners and nonwinners to
–– Had fewer patient complications
Medicare patient counts. If the same standards
–– Delivered care that resulted in fewer HAIs were applied to all inpatients, the impact would be
even greater.
–– Had lower 30-day mortality and 30-day
readmission rates
For more details about this study’s findings and the
–– Sent patients home sooner achievements of the 100 Top Hospitals, please see
the Findings section of this document.
5
Welcoming your input In addition to the major studies, customized
The 100 Top Hospitals program works to ensure analyses are also available from the 100 Top
that the measures and methodologies used in our Hospitals program, including custom benchmark
studies are fair, consistent, and meaningful. We reports. Our reports are designed to help
continually test the validity of our performance healthcare executives understand how their
measures and data sources. In addition, as part of organizational performance compares to peers
our internal performance improvement process, we within health systems, states, and markets.
welcome comments about our study from health
system, hospital, and physician executives. To 100 Top Hospitals program reports offer a
submit comments, visit 100tophospitals.com. two-dimensional view of both performance
improvement over time, applying the most
current methodologies across all years of data
Showcasing the versatility of the to produce trends, as well as the most current
100 Top Hospitals program year performance.
The 100 Top Hospitals research is one of three

You can read more about these studies, order
major annual studies of the Watson Health
customized reports, and view lists of all winners by
100 Top Hospitals program. To increase
visiting 100tophospitals.com.
understanding of trends in specific areas of the
healthcare industry, the program includes:
–– 100 Top Hospitals and Everest Award studies About IBM Watson Health
Research that annually recognizes the 100 Each day, professionals throughout the health
top-rated hospitals in the nation based on ecosystem make powerful progress toward a
a proprietary, balanced scorecard of overall healthier future. At IBM Watson Health, we help
organizational performance, and identifies them remove obstacles, optimize efforts, and
those hospitals that also excel at long- reveal new insights to support the people they
term rates of improvement in addition to serve. Working across the landscape, from payers
performance and providers to governments and life sciences,
–– 50 Top Cardiovascular Hospitals study we bring together deep health expertise; proven
An annual study introduced in 1999 that innovation; and the power of artificial intelligence
identifies hospitals demonstrating the highest to enable our customers to uncover, connect, and
performance in hospital cardiovascular act as they work to solve health challenges for
services for four important patient groups: people everywhere.
heart attack, heart failure, coronary artery
bypass graft and percutaneous coronary For more information, visit ibm.com/watsonhealth.
intervention
–– 15 Top Health Systems study
An annual study introduced in 2009 that
provides an objective measure of health
system performance overall and offers insight
into the ability of a system’s member hospitals
to deliver consistent top performance across
the communities they serve, all based on our
national health system scorecard
6 IBM Watson Health

Note that the order of hospitals in the following
2019 tables does not reflect performance rating.
Hospitals are ordered alphabetically. For full details
100 Top Hospitals on these peer groups and the process we used to
select the winning benchmark hospitals*, see the
award winners Methodology section of this document.
The Watson Health 100 Top Hospitals® program

is pleased to present the 2019 Watson Health
100 Top Hospitals.
Major teaching hospitals*

Hospitals Location Medicare ID Total year(s) won
Advocate Illinois Masonic Medical Center Chicago, IL 140182 8
Ascension Providence Hospital Southfield, MI 230019 11
Banner - University Medical Center Phoenix Phoenix, AZ 030002 2
Cedars-Sinai Medical Center Los Angeles, CA 050625 3
Garden City Hospital Garden City, MI 230244 2
Mayo Clinic Hospital Jacksonville, FL 100151 2
Mount Sinai Medical Center Miami Beach, FL 100034 2
NorthShore University HealthSystem Evanston, IL 140010 20
Saint Francis Hospital and Medical Center Hartford, CT 070002 8
Spectrum Health Hospitals Grand Rapids, MI 230038 10
St. Joseph Mercy Hospital Ann Arbor, MI 230156 10
St. Luke's University Hospital - Bethlehem Bethlehem, PA 390049 7
The Miriam Hospital Providence, RI 410012 1
UCHealth University of Colorado Hospital Aurora, CO 060024 6
University of Utah Hospital Salt Lake City, UT 460009 2
* Everest Award winners are in bold type.
* To see a full list of our award winners through the years, visit https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=40019540USEN&.
7
Teaching hospitals*
Abbott Northwestern Hospital Minneapolis, MN 240057 3
Aspirus Wausau Hospital Wausau, WI 520030 7
Brandon Regional Hospital Brandon, FL 100243 7
BSA Health System Amarillo, TX 450231 6
CHRISTUS St. Michael Health System Texarkana, TX 450801 3
Good Samaritan Hospital Cincinnati, OH 360134 6
Lakeland Medical Center St. Joseph, MI 230021 2
Mercy Hospital St. Louis St. Louis, MO 260020 7
Monmouth Medical Center Long Branch, NJ 310075 1
Morton Plant Hospital Clearwater, FL 100127 7
Mount Carmel St. Ann's Westerville, OH 360012 2
Park Nicollet Methodist Hospital St. Louis Park, MN 240053 5
Parkview Regional Medical Center Fort Wayne, IN 150021 4
PIH Health Hospital - Whittier Whittier, CA 050169 5
Riverside Medical Center Kankakee, IL 140186 10
Rose Medical Center Denver, CO 060032 12
Sentara Leigh Hospital Norfolk, VA 490046 5
Sky Ridge Medical Center Lone Tree, CO 060112 2
SSM Health St. Mary's Hospital - Madison Madison, WI 520083 6
St. Luke's Hospital Cedar Rapids, IA 160045 8
St. Mark's Hospital Salt Lake City, UT 460047 6
Sycamore Medical Center Miamisburg, OH 360239 10
UCHealth Poudre Valley Hospital Fort Collins, CO 060010 13
Utah Valley Hospital Provo, UT 460001 1
West Penn Hospital Pittsburgh, PA 390090 5
8 IBM Watson Health

Large community hospitals*
Advocate Sherman Hospital Elgin, IL 140030 2
Banner Del E. Webb Medical Center Sun City West, AZ 030093 1
Baylor Scott & White Medical Center - Grapevine Grapevine, TX 450563 1
Hoag Hospital Newport Beach Newport Beach, CA 050224 4
IU Health Bloomington Hospital Bloomington, IN 150051 1
Mease Countryside Hospital Safety Harbor, FL 100265 11
Memorial Hermann Memorial City Medical Center Houston, TX 450610 8
Mercy Health - Anderson Hospital Cincinnati, OH 360001 12
Mercy Health - St. Rita's Medical Center Lima, OH 360066 4
Mercy Hospital Coon Rapids, MN 240115 7
Mercy Hospital Oklahoma City Oklahoma City, OK 370013 4
Northwestern Medicine Central DuPage Hospital Winfield, IL 140242 10
Sarasota Memorial Hospital Sarasota, FL 100087 4
Scripps Memorial Hospital La Jolla La Jolla, CA 050324 4
St. Clair Hospital Pittsburgh, PA 390228 5
St. David's Medical Center Austin, TX 450431 10
St. Joseph's Hospital Tampa, FL 100075 3
Texas Health Harris Methodist Hospital Southwest Fort Worth Fort Worth, TX 450779 4
University of Maryland St. Joseph Medical Center Towson, MD 210063 1
WellStar West Georgia Medical Center LaGrange, GA 110016 3
9
Medium community hospitals*
AdventHealth Wesley Chapel Wesley Chapel, FL 100319 2
Dupont Hospital Fort Wayne, IN 150150 5
East Cooper Medical Center Mt. Pleasant, SC 420089 1
East Liverpool City Hospital East Liverpool, OH 360096 2
Garden Grove Hospital Medical Center Garden Grove, CA 050230 5
IU Health North Hospital Carmel, IN 150161 2
IU Health West Hospital Avon, IN 150158 1
Logan Regional Hospital Logan, UT 460015 9
Memorial Hermann Katy Hospital Katy, TX 450847 3
Mercy Health - Clermont Hospital Batavia, OH 360236 10
Mercy Hospital Northwest Arkansas Rogers, AR 040010 1
Mercy Medical Center Cedar Rapids, IA 160079 7
Montclair Hospital Medical Center Montclair, CA 050758 4
Mountain View Hospital Payson, UT 460013 3
Northwestern Medicine Delnor Hospital Geneva, IL 140211 1
St. Luke's Anderson Campus Easton, PA 390326 1
St. Vincent's Medical Center Clay County Middleburg, FL 100321 1
UCHealth Medical Center of the Rockies Loveland, CO 060119 3
West Valley Medical Center Caldwell, ID 130014 6
Wooster Community Hospital Wooster, OH 360036 5
*Everest Award winners are in bold type.
10 IBM Watson Health

Small community hospitals*
Alta View Hospital Sandy, UT 460044 6
Aurora Medical Center Two Rivers, WI 520034 3
Brigham City Community Hospital Brigham City, UT 460017 5
Buffalo Hospital Buffalo, MN 240076 5
Cedar City Hospital Cedar City, UT 460007 8
Hill Country Memorial Hospital Fredericksburg, TX 450604 8
Lakeview Hospital Bountiful, UT 460042 9
Lone Peak Hospital Draper, UT 460060 1
Marshfield Medical Center Rice Lake, WI 520011 4
Nanticoke Memorial Hospital Seaford, DE 080006 1
Parkview Noble Hospital Kendallville, IN 150146 2
Parkview Whitley Hospital Columbia City, IN 150101 1
Piedmont Mountainside Hospital Jasper, GA 110225 1
San Dimas Community Hospital San Dimas, CA 050588 3
Seton Medical Center Harker Heights Harker Heights, TX 670080 1
Southern Tennessee Regional Health System Lawrenceburg, TN 440175 2
Spectrum Health Zeeland Community Hospital Zeeland, MI 230003 5
St. John Owasso Hospital Owasso, OK 370227 3
St. Luke's Hospital - Quakertown Quakertown, PA 390035 2
Stillwater Medical Center Stillwater, OK 370049 2
*Everest Award winners are in bold type.
11
This award recognizes the boards, executives, and
2019 Everest medical staff leaders who developed and executed
the strategies that drove the highest rates of
Award winners improvement, resulting in the highest performance
in the US at the end of five years.
The Watson Health 100 Top Hospitals® Everest
Award honors hospitals that have both the highest The Everest Award winners are a special group
current performance and the fastest long-term of the 100 Top Hospitals award winners that, in
improvement in the years of data analyzed. addition to achieving benchmark status for one
year, have simultaneously set national benchmarks
for the fastest long-term improvement on our
national balanced scorecard. In 2019, only 15
organizations achieved this level of performance.
The 2019 Everest Award winners

IBM Watson Health™ is pleased to present
the winners of the 2019 100 Top Hospitals
Everest Award.
2019 Everest Award winners

Advocate Sherman Hospital Elgin, IL 140030 2
CHRISTUS St. Michael Health System Texarkana, TX 450801 1
East Liverpool City Hospital East Liverpool, OH 360096 2
Garden City Hospital Garden City, MI 230244 1
IU Health Bloomington Hospital Bloomington, IN 150051 1
Parkview Regional Medical Center Fort Wayne, IN 150021 1
Parkview Whitley Hospital Columbia City, IN 150101 1
Rose Medical Center Denver, CO 060032 1
Sentara Leigh Hospital Norfolk, VA 490046 2
St. Joseph Mercy Hospital Ann Arbor, MI 230156 3
St. Joseph's Hospital Tampa, FL 100075 2
St. Mark's Hospital Salt Lake City, UT 460047 1
Stillwater Medical Center Stillwater, OK 370049 1
UCHealth University of Colorado Hospital Aurora, CO 060024 1
Utah Valley Hospital Provo, UT 460001 1
13
The value of the Everest Award measures to the –– What incentives do we need to implement
healthcare industry for management to achieve the desired
Leaders facing the challenges of a rapidly changing improvement more quickly?
healthcare environment may benefit from unbiased –– Will the investments we are considering help
intelligence that provides objective insights into us achieve improvement goals?
complex organizational performance. Those
insights may also help leaders balance short- and –– Can we quantify the long- and short-term
long-term goals to drive continuous gains in increases in value our hospital has provided to
performance and value. our community?
Transparency may present hospital boards and

CEOs with a public challenge to increase the value How we select the Everest Award winners
of core services to their communities. Providing Winners of the 100 Top Hospitals Everest
value is characteristically not a one-time event; it Award set national benchmarks for both fastest
is a continuous process of increasing worth over rate of improvement and highest current year
time. The goal of the 100 Top Hospitals program performance on the study’s balanced scorecard.
is to provide information that can help inform
the leadership decisions that guide hospitals to Everest Award winners are selected from among
achieve those objectives. the new 100 Top Hospitals award winners. The
national award and the Everest Award are based on
We believe the greatest value can be achieved a set of measures that reflect performance across
when leaders integrate knowledge of their the whole organization.
organization’s performance compared to
national benchmarks with information on rates Our methodology for selecting the Everest Award
of improvement compared to peers. In this way, winners can be summarized in three main steps:
leaders can determine the effectiveness of long-
1. Selecting the annual 100 Top Hospitals award
term strategies that led to current performance and
winners using our objective methodology*
understand where to act to course-correct.
based on publicly available data and a
balanced scorecard of performance measures
Our research is designed to help boards and CEOs
using the most current data available (2017 at
better answer questions such as:
the time of this study)
–– Did our long-term strategies result in a stronger
2. Using our five-year (2013 - 2017) trending
hospital across all performance areas?
methodology to select the 100 hospitals
–– Did our strategies drive improvement in some that have shown the fastest, most consistent
areas but inadvertently cause deteriorating improvement rates on the same balanced
performance in others? scorecard of performance measures
–– What strategies will help us increase the rate of 3. Identifying those hospitals that ranked in the
improvement in the right areas to come closer top 100 on both lists; these hospitals are the
to national performance levels? Everest Award winners
* For full details on how the 100 Top Hospitals winners are selected, see the Methodology section of this document.

Combining these two methodologies yields a select For this year’s study, after excluding hospitals
group of Everest Award winners. The number with insufficient, missing, or invalid data, along
of winners will vary every year, based solely on with hospitals that would skew study results (for
performance in the two dimensions. example, specialty hospitals), we had a database
study group of 2,752 hospitals.
Comparison groups
Because bed size and teaching status have an
Top Everest Most improved effect on the types of patients a hospital treats
performance, Award performance, and the scope of services it provides, we assigned
current year winners five years each hospital in the study database to one of
five comparison groups according to its size and
teaching status (for definitions of each group, see
the Methodology section of this document):
–– Major teaching hospitals
––Teaching hospitals
Data sources
As with all 100 Top Hospitals studies, our –– Large community hospitals
methodology is designed to be objective, and –– Medium community hospitals
all data comes from public sources. We build a
database of short-term, acute care, nonfederal US –– Small community hospitals
hospitals that treat a broad spectrum of patients.
The primary data sources are the Medicare Provider To support evaluating hospitals fairly and
Analysis and Review (MEDPAR) patient claims data comparing them to like hospitals, we use these
set, the Centers for Medicare & Medicaid Services comparison groups for all scoring and ranking to
Hospital Compare hospital performance data set, uncover winners. For more information on how we
and the Hospital Cost Report Information System build the database, see the Methodology section.
Medicare Cost Report file. We use the most recent
five years of data available for trending and the
most current year for selection of winners*.
Residency program information, used in classifying

teaching hospitals, is from the Accreditation
Council for Graduate Medical Education (AMA-
accredited programs) and the American
Osteopathic Association.
* Hospital inpatient mortality and complications are based on two years of data combined for each study year data point. See the Performance Measures section of this
document for details.
15
Performance measures present-on-admission (POA) data in our proprietary
Both the 100 Top Hospitals and the Everest Awards risk models. POA coding became available in the
are based on a set of measures that, taken together, 2009 MEDPAR data set.
are designed to assess balanced performance
across the organization, reflecting the leadership For the inpatient mortality and complications
effectiveness of board members, management, and (clinical measures with low frequency of
medical and nursing staff. These measures fall into occurrence), we combine two years of data for
five domains of performance: inpatient outcomes, each study year to stabilize results. This year, we
extended outcomes, operational efficiency, combined data sets as follows:
financial health, and patient experience. –– Study year 2017 = 2017 and 2016 MEDPAR
data sets
The 10 measures used to select the 2019
winners are: –– Study year 2016 = 2016 and 2015 MEDPAR
data sets
1. Risk-adjusted inpatient mortality index
–– Study year 2015 = 2015 and 2014 MEDPAR
2. Risk-adjusted complications index data sets
3. Mean healthcare-associated infection index –– Study year 2014 = 2014 and 2013 MEDPAR
4. Mean 30-day risk-adjusted mortality rate data sets
(includes acute myocardial infarction [AMI]), –– Study year 2013 = 2013 and 2012 MEDPAR
heart failure [HF], pneumonia, chronic data sets
obstructive pulmonary disease [COPD],
and stroke) For specific data periods used for each measure,
5. Mean 30-day risk-adjusted readmission rate see page 47 of the Methodology section.
(includes AMI, HF, pneumonia, THA/TKA,
COPD, and stroke)
6. Severity-adjusted average length of stay
7. Mean emergency department throughput
(in minutes)
8. Case mix- and wage-adjusted inpatient
expense per discharge
9. Adjusted operating profit margin
10. Hospital Consumer Assessment of
Healthcare Providers and Systems score
(overall hospital performance)
For full details, including calculation and scoring

methods, see the Methodology section. We use

Ranking and five-year trend summary
To select the 100 Top Hospitals award winners,
we rank hospitals on current year performance
on each of the study measures relative to other
hospitals in their comparison group. We then sum
each hospital’s performance-measure rankings and
re-rank them, overall, to arrive at a final rank for
the hospital. The hospitals with the best final ranks
in each comparison group are selected as the 100
Top Hospitals award winners. See the Methodology
section for details on the ranking methodology,
including measures, weighting, and selection of
100 Top Hospitals winners.
Separately, for every hospital in the study, we

calculate a t-statistic that measures five-year
performance improvement for each of the included
performance measures. This statistic measures the
direction and magnitude of change in performance,
and the statistical significance of that change. We
rank hospitals on the basis of their performance
improvement t-statistic on each of the study
measures relative to other hospitals in their
comparison group. We then sum each hospital’s
performance-measure rankings and re-rank them
overall, to arrive at a final rank for the hospital.
The hospitals with the best final rank in each
comparison group are selected as the performance
improvement benchmark hospitals. See the
Methodology section for details on trending,
including measure weighting.
As our final step, we find those hospitals that are

identified as benchmarks on both lists. These
hospitals are the Everest Award winners.
17
–– Over 155,000 fewer discharged patients
Findings would be readmitted within 30 days
–– Patients would spend 17 minutes less in
The Watson Health 100 Top Hospitals® study hospital emergency rooms per visit
shines a light on the top-performing hospitals in
the country. According to publicly available data We based this analysis on the Medicare patients
and our transparent methodologies, these industry included in this study. If the same standards were
leaders appear to have successfully negotiated applied to all inpatients, the impact would be
the fine line between running highly effective even greater.
operations and being innovative and forward-
thinking in ways that grow their organizations over Note: All currency amounts listed in this 100 Top
the short and long term. Hospitals study are in US dollars.
Year after year, the public data we have gathered

for the 100 Top Hospitals studies has provided How the winning hospitals compared to
numerous examples of benchmark hospitals’ their peers
clinical, financial and operational excellence and In this section, we show how the 100 Top Hospitals
affirmed the validity and stability of this approach performed within their comparison groups
to performance measurement2–28. (major teaching, teaching, large community,
medium community, and small community
The study is more than a list of accomplishments; it hospitals), compared with nonwinning peers. For
is a method US hospital and health system leaders performance measure details and definitions of
can use to help guide their own performance each comparison group, see the Methodology
improvement initiatives. By highlighting what the section of this document.
highest-performing leaders around the country are
doing well, we create aspirational benchmarks for Note: In Tables 1 through 6, data for the 100 Top
the rest of the industry. Hospitals award winners is labeled “Benchmark,”
and data for all hospitals, excluding award winners,
Based on comparisons between the 100 Top is labeled “Peer group.” In columns labeled
Hospitals study winners and a peer group of similar “Benchmark compared with peer group,” we
hospitals that were not winners, we found that calculated the actual and percentage difference
if all hospitals performed at the level of this between the benchmark hospital scores and the
year’s winners: peer group scores.
–– Over 103,000 additional lives could be saved
in-hospital 100 Top Hospitals had better survival rates*
–– Over 38,000 additional patients could be –– Overall, the winners had 24% fewer deaths
complication-free than expected (0.76 index), considering patient
severity, while their nonwinning peers had 1%
–– Over $8.2 billion in inpatient costs could more deaths than would be expected (1.01
be saved index) (Table 1)
––The typical patient could be released from –– Small community hospitals had the most
the hospital a half day sooner and would dramatic difference between winners and
have 12% fewer expenses related to the nonwinners; the winning small hospital median
complete episode of care than the median mortality rate was 47% lower than nonwinning
patient in the US peers (Table 6)
* Risk-adjusted measures are normalized by comparison group, so results cannot be compared across comparison groups.
19
–– Medium-sized community hospitals also had –– Overall, nationally, there were 35% fewer
a significantly lower median mortality index infections than expected at winning hospitals
values than nonwinning peer hospitals, with a (0.65 standardized infection ratio [SIR]
29.5% lower mortality index (Table 5) median), compared to 19% fewer infections
at peer nonwinning hospitals (0.81 SIR
100 Top Hospitals had fewer patient complications* median)*** (Table 1)
–– Overall, patients at the winning hospitals had –– On the HAI composite index, medium
23% fewer complications than expected (0.77 community hospitals showed the widest
index), considering patient severity, while difference between winning benchmark
their nonwinning peers had only 5% fewer hospital performance and nonwinners,
complications than expected (0.95 index)*** with the winning median HAI composite
(Table 1) index 30% lower than the median value of
nonwinners (0.51 and 0.73 median SIR values,
–– For complications, as with inpatient mortality,
respectively) (Table 5)
small community hospitals had the most
dramatic difference between winners and ––The winners among major teaching hospitals
nonwinners; the winning small hospital median had 19% fewer infections than expected (0.81
observed-to-expected ratio of complications SIR median), while their nonwinning major
was 41.5% lower than nonwinning peers’ index teaching peers had only 7% fewer infections
value (0.54 versus 0.92) (Table 6) than expected (0.93 SIR median) (Table 2)
100 Top Hospitals had fewer healthcare- 100 Top Hospitals had lower 30-day mortality
associated infections and readmission rates
Healthcare-associated infections (HAIs)**, Several patient groups are included in the 30-
captures information about the quality of inpatient day mortality and readmission extended care
care. Based on nation-wide data availability, we composite metrics. The mean 30-day mortality
built a composite measure of HAI performance rate includes heart attack (AMI), heart failure (HF),
at the hospital level, considering up to six HAIs, pneumonia, chronic obstructive pulmonary disease
depending on assigned comparison group. (The (COPD), and stroke patient groups. The mean 30-
HAI measure is not ranked for small community day readmission rate includes AMI, HF, pneumonia,
hospitals in the 2019 study.) The six reported HAIs total hip arthroplasty and/or total knee arthroplasty
are: methicillin-resistant staphylococcus aureus (THA/TKA), COPD, and stroke patient groups.
(MRSA-bloodstream), central line-associated blood
–– Mean 30-day mortality and readmission rates
stream infections, catheter-associated urinary tract
were lower at the winning hospitals than
infections, clostridium difficile (C.diff), surgical site
nonwinning hospitals, across all comparison
infections (SSIs) following colon surgery, and SSIs
groups (by 0.6 and 0.4 percentage points,
following an abdominal hysterectomy.
respectively) (Table 1)
** As developed by the National Healthcare Safety Network and reported by the Centers for Medicare & Medicaid Services (CMS) in the public Hospital Compare data set.
*** Mortality, complications and HAI index values are calculated using a subset of hospitals from which the measures are developed, which is why there will be instances
where both peer and bench indexes are below 1.0 .

–– Major teaching hospital winners continued ––The most dramatic difference in emergency
to demonstrate the best 30-day mortality department (ED) service delivery times
performance among all hospital comparison between winning hospitals and their peers
groups, with a median rate at 11.7%) (Table 2) occurred in the major teaching category where
there was 69.3 minutes less time-to-service;
–– Small community hospital winners again had
winning teaching hospitals followed right
the best 30-day readmission performance
behind with 41.5 minutes less time-to-service
among all comparison groups (14.3%)
(Tables 2 and 3)
(Table 6)
–– However, major teaching hospitals had the
–– Major teaching hospital winners outperformed
longest throughput times of all comparison
nonwinners on 30-day readmissions by the
groups, at 246.5 minutes for winners and
greatest margin (0.7 percentage points)
315.8 minutes for nonwinners (Tables 2
(Table 2)
through 6)
Patients treated at 100 Top Hospitals returned –– Small community hospitals had the shortest
home sooner* throughput times of all comparison groups for
both winning and nonwinning hospitals (163
–– Overall, winning hospitals had a median
and 182.5 minutes, respectively) (Table 6)
severity-adjusted average length of stay (LOS)
that was a 0.5 day shorter than peers (Table 1)
100 Top Hospitals had lower inpatient expenses
–– Both the winning small- and medium-sized
––The findings show that overall, and in all
community hospitals had a large difference
comparison groups, the winning hospital
from the non-winners in average LOS, with
median for case mix- and wage-adjusted
a median average LOS of 0.7 days shorter
inpatient expense per discharge was lower
(Tables 5 and 6)
than the median for nonwinner peers this
–– Among major teaching hospitals, there was year (Tables 1 through 6)
also a large difference between winners and
–– Medium community hospital winners and
nonwinners on median average LOS, at 4.3
nonwinners had the lowest case mix- and
days versus 4.9 days (a 13.0% difference)
wage-adjusted inpatient expense per
(Table 2)
discharge than any other comparison
group with expenses at $5,894 and $6,742,
Patients spent less time in 100 Top Hospitals
respectively (Table 5)
emergency departments
––The largest difference in expenses between
–– Overall, winning hospitals had shorter median
and winning and non-winning hospitals was
wait times for emergency services** than their
found in the small hospital comparison group
peers, by 8% (Table 1)
with a difference of $1,340 (Table 6)
** Includes median time from ED arrival to ED departure for admitted patients and median time from ED arrival to ED departure for non-admitted patients.
21
100 Top Hospitals were more profitable
–– Overall, winning hospitals had a median
operating profit margin that was 11.9
percentage points higher than nonwinning
hospitals (15.6% versus 3.8%) (Table 1)
–– Profitability difference was the most dramatic
in the medium community hospital group,
where winners had operating profit margins
that were 17.1 percentage points higher than
nonwinners (Table 5)
–– Medium hospital winners also had the largest
median operating profit margin of any winning
group at 21.5% (Table 5)
–– In contrast, small community hospital winners
had the lowest median operating profit margin
of any winning group at 12.9% (Table 6)
Patients rated 100 Top Hospitals higher than

peer hospitals
–– Patients treated at the 100 Top Hospitals
reported a better overall hospital experience
than those treated in peer hospitals, with
a 3.0% higher median Hospital Consumer
Assessment of Healthcare Providers and
Systems (HCAHPS) overall rating score
(Table 1)
––The winning small community hospitals had
the highest median HCAHPS score of all
comparison groups, at 273 versus 265 for
nonwinners (maximum score is 300) (Table 6)
–– Large community hospital winners had the
biggest performance difference over peers
(4.0 % higher HCAHPS scores) among all
comparison groups (Table 4)

Table 1. National performance comparisons (all hospitals in study)
Domain Performance measure Medians Benchmark compared with peer group
Benchmark
hospitals Peer hospitals Percent
(winners) (nonwinners) Difference difference Comments
Clinical Inpatient Mortality Index 1

0.76 1.01 -0.25 -24.9% Lower mortality
Outcomes
Complications Index 1
0.77 0.95 -0.18 -18.7% Fewer complications
HAI Index 2
0.65 0.81 -0.2 -19.3% Fewer infections
Extended 30-Day Mortality Rate3 12.0 12.7 -0.6 n/a6 Lower 30-day mortality
Outcomes
30-Day Readmission Rate3 14.5 14.9 -0.4 n/a6 Fewer 30-day readmissions
Operational Average Length of Stay 1
4.3 4.7 -0.5 -10.1% Shorter stays
Efficiency
ED Throughput Measure 4
196.8 214.0 -17.3 -8.1% Less time to service
Inpatient Expense per Discharge 5
$6,150 $6,980 -$830 -11.9% Lower inpatient cost
Financial
Operating Profit Margin5 15.6 3.8 11.9 n/a6 Higher profitability
Health
Patient Hospital Consumer Assessment of 271.0 263.0 8.0 3.0% Better patient experience
Experience Healthcare Providers and Systems
(HCAHPS) score4
1
Mortality, complications and average length of stay based on Present on Admission (POA)-enabled risk models applied to MedPAR 2016 and 2017 data (ALOS 2017 only).
2
Healthcare-Associated Infections (HAI) data from CMS Hospital Compare Jan 1, 2017 - Dec 31, 2017 data set (excluding Small Community Hospitals).
3
30-day rates from CMS Hospital Compare July 1, 2014-June 30, 2017 data set.
4
ED measure and HCAHPS data from CMS Hospital Compare Jan 1, 2017-Dec 31, 2017 data set.
5
Inpatient expense and operating profit margin data from CMS Hospital Cost Report Information System (HCRIS) data file, 2017.
6
We do not calculate percent difference for this measure because it is already a percent value.
23
Table 2. Major teaching hospital performance comparisons
Benchmark Peer hospitals Difference Percent How winning benchmark
hospitals (nonwinners) difference hospitals outperformed
(winners) nonwinning peer hospitals
Clinical Inpatient Mortality Index1 0.82 1.01 -0.19 -19.0% Lower mortality
Outcomes
HAI Index 2
Outcomes
4.3 4.9 -0.6 -13.0% Shorter stays
Efficiency
$6,761 $8,027 -$1,267 -15.8% Lower inpatient cost
Financial Operating Profit Margin 5
13.1 2.6 10.6 n/a 6
Higher profitability
Health
Patient HCAHPS Score4 270.0 263.0 7.0 2.7% Better patient experience
Experience
1
2
3
4
5
6
Table 3. Teaching hospital performance comparisons

Outcomes
HAI Index2 0.62 0.84 -0.2 -26.5% Fewer infections
Extended 30-Day Mortality Rate 3
12.1 12.7 -0.6 n/a 6
Lower 30-day mortality
Outcomes
30-Day Readmission Rate 3
14.5 15.0 -0.5 n/a 6
Fewer 30-day readmissions
4.4 4.9 -0.5 -11.2% Shorter stays
Efficiency
Inpatient Expense per Discharge5 $6,152 $6,797 -$645 -9.5% Lower inpatient cost
Financial Operating Profit Margin5 15.4 5.1 10.3 n/a6 Higher profitability
Health
Experience
1
2
3
4
5
6

Table 4. Large community hospital performance comparisons
Outcomes
HAI Index 2
Outcomes
4.6 5.0 -0.4 -8.3% Shorter stays
Efficiency
$6,231 $6,776 -$544 -8.0% Lower inpatient cost
16.4 6.9 9.6 n/a 6
Health
Experience
1
2
3
4
5
6
Table 5. Medium community hospital performance comparisons

Outcomes
HAI Index2 0.51 0.73 -0.2 -30.0% Fewer infections
Extended 30-Day Mortality Rate 3
12.1 12.7 -0.6 n/a 6
Lower 30-day mortality
Outcomes
30-Day Readmission Rate 3
14.8 15.0 -0.3 n/a 6
Fewer 30-day readmissions
4.2 4.9 -0.7 -13.9% Shorter stays
Efficiency
Inpatient Expense per Discharge5 $5,894 $6,742 -$848 -12.6% Lower inpatient cost
Financial Operating Profit Margin5 21.5 4.4 17.1 n/a6 Higher profitability
Health
Experience
1
2
3
4
5
6
25
Table 6. Small community hospital comparisons
Outcomes
HAI Index 2
n/a n/a n/a n/a n/a
Outcomes
4.2 4.9 -0.7 -13.6% Shorter stays
Efficiency
$6,039 $7,379 -$1,340 -18.2% Lower inpatient cost
12.9 1.8 11.0 n/a 6
Health
Experience
1
2
3
4
5
6
US map and states by region This analysis allows us to observe geographic

The US maps featured in Figures 1 and 2 patterns in performance. Among our observations:
provide a visual representation of the variability ––The Midwest continues to be the frontrunner
in performance across the country for the current in percentage of states in the top two
and previous studies (2019 and 2018). Additionally, performance quintiles versus other regions,
Table 7 shows each state’s rank quintile although with a slight decrease in this lead
performance, grouped by geographic region, for this year (83.3% of states in 2019 versus
the current and previous year studies. 91.7% in 2018)
To produce this data, we calculated the 100 Top ––The Northeast continues to show the poorest
Hospitals measures at the state level*, ranked each performance overall, by a large margin in both
measure, then weighted and summed the ranks to years, with 66.7% of its states in the bottom
produce an overall state performance score. States two quintiles in 2019 and 77.8% in 2018
were ranked from best to worst on the overall score, ––The South continues to show the same pattern
and the results are reported as rank quintiles. as last year with the majority of its states in
the bottom two quintiles (47.1% in 2019 and
52.9% in 2018)
* Each state measure is calculated from the acute care hospital data for that state (short-term, general acute care hospitals; critical access hospitals; and cardiac, orthopedic,
and women’s hospitals) with valid data for the included measures. Inpatient mortality, complications, and average LOS are aggregated from MEDPAR patient record data. HAIs,
30-day mortality rates, and 30-day readmission rates are aggregated from the numerator and denominator data for each hospital. Inpatient expense per discharge, operating
profit margin, and HCAHPS scores are hospital values weighted by the number of acute discharges at each hospital. Mean ED throughput is calculated by averaging the median
minutes of member hospitals to produce the unweighted mean minutes for each ED measure, then averaging the two ED measures to produce the state-level unweighted ED
throughput measure. For expense, profit, and HCAHPS, a mean weighted value is calculated for each state by summing the weighted hospital values and dividing by the sum of the
weights. To calculate the state overall score, individual measure ranks are weighted, using the same measure rank weights as in the 100 Top Hospitals study, then summed.

Figure 1. State-level performance comparisons, 2019 study
WA
ME
MT
ND
MN VT
OR NH
MA
ID WI NY
SD RI
MI CT
WY
PA
IA NJ
NE
OH MD
NV IN DE
UT IL
WV
CO VA
DC (red)
KS MO
CA KY
NC
TN
AZ OK
SC
NM AR
MS AL GA
100 Top Hospitals performance
2019 study state-level rankings
TX LA
Quintile 1 - Best
FL
Quintile 2
HI
Quintile 3
AK
Quintile 4
Quintile 5 - Worst
State data note: The 2019 state findings were based on the 100 Top Hospitals measure methodologies, using 2016 and 2017 MEDPAR data
(combined) for inpatient mortality and complications; July 1, 2014- June 30, 2017, for 30-day rates, and 2017 data for all other measures.
27
Figure 2. State-level performance comparisons, 2018 study
WA
ME
MT
ND
MN VT
OR NH
MA
ID WI NY
SD RI
MI CT
WY
PA
IA
NJ
NE
OH
NV IN DE
UT IL
WV MD
CO VA
DC (red)
KS MO
CA KY
NC
TN
AZ OK
SC
NM AR
MS AL GA
2019 study state-level rankings
TX LA
Quintile 1 - Best
FL
Quintile 2
HI
Quintile 3
AK
Quintile 4
Quintile 5 - Worst
State data note: The 2018 state findings were based on the 100 Top Hospitals measure methodologies, using 2015 and 2016 MEDPAR data
(combined) for inpatient mortality and complications; July 1, 2013- June 30, 2016, for 30-day rates, and 2016 data for all other measures.

Table 7. 100 Top Hospitals two-year state-level performance comparisons
ME
Northeast Midwest South West
Current
VT study Previous study Current study Previous study Current study Previous study Current study Previous study
NH
NY Connecticut
MA Connecticut Illinois Illinois Alabama Alabama Alaska Alaska
RI
CT
Maine Maine Indiana Indiana Arkansas Arkansas Arizona Arizona
PA Massachusetts Massachusetts Iowa Iowa Delaware Delaware California California
NJ
New Hampshire
DE
New Hampshire Kansas Kansas District of District of Colorado Colorado
MD
Columbia Columbia
VA
DC (red)
New Jersey New Jersey Michigan Michigan Florida Florida Hawaii Hawaii
NC New York New York Minnesota Minnesota Georgia Georgia Idaho Idaho
Pennsylvania Pennsylvania Missouri Missouri Kentucky Kentucky Montana Montana
SC
Rhode Island Rhode Island Nebraska Nebraska Louisiana Louisiana Nevada Nevada
Vermont Vermont
North Dakota North Dakota Maryland Maryland New Mexico New Mexico
2018 study state-level rankings Ohio Ohio Mississippi Mississippi Oregon Oregon
Quintile 1 - Best South Dakota South Dakota North Carolina North Carolina Utah Utah
FL Wisconsin Wisconsin Oklahoma Oklahoma Washington Washington

Quintile 2
South Carolina South Carolina Wyoming Wyoming
Quintile 3
Tennessee Tennessee
Quintile 4 Texas Texas
Virginia Virginia
Quintile 5 - Worst
West Virginia West Virginia
29
Performance improvement over time:
All hospitals
By studying the direction of performance change of
all hospitals in our study (winners and nonwinners),
we can see that US hospitals have not been able
to improve performance much across the entire
balanced scorecard of performance measures
(Table 8).
Notably, only one metric, 30-day readmissions,

had fewer than 75% of in-study hospitals in the
category of “no statistically significant change in
performance.”
Table 8. Direction of performance change for all hospitals in study, 2013 - 2017
Performance measure Significantly improving No statistically significant Significantly declining
performance change in performance performance
Count of Percentage of Count of Percentage of Count of Percentage of
hospitals1 hospitals2 hospitals1 hospitals2 hospitalsa1 hospitals2
Risk-adjusted inpatient mortality index 83 3.1% 2,374 88.1% 238 8.8%

Risk-adjusted complication index 198 7.3% 2,402 89.1% 95 3.5%
30-day mortality rate 33 1.2% 2,405 89.2% 257 9.5%
30-day readmission rate 1510 56.0% 1,165 43.2% 20 0.7%
Severity-adjusted average LOS 476 17.7% 2,050 76.1% 169 6.3%
ED throughput (minutes) 183 6.8% 2,050 76.1% 462 17.1%
Adjusted inpatient expense per discharge 71 2.6% 2,177 81.2% 434 16.2%
Operating profit margin 185 6.9% 2,336 87.0% 165 6.1%
HCAHPS score 286 10.6% 2,287 84.9% 121 4.5%
1. Count refers to the number of in-study hospitals whose performance fell into the highlighted category on the measure.
Note: Total number of hospitals included in the analysis will vary by measure due to exclusion of interquartile range outlier data points.
Inpatient expense and profit are affected. Some in-study hospitals had too few data points remaining to calculate trend.
2. Percent is of total in-study hospitals across all peer groups.
However, over the years we studied (2013 through

2017), many hospitals have been able to raise the
performance bar on many clinical and operational
measures (see green column in Table 8):
–– Over half (56%) of hospitals improved their 30-
day readmission rates, reiterating the attention
these measures continue to receive in payment
system incentives and penalties
–– Only 3.1% of hospitals improved their inpatient
mortality (88% had no change), while the great
majority of hospitals (nearly 90%) also had no

change in their 30-day mortality rates care. The data source for the calculations are the
Hospital OPPS and Physician part B claims and
–– Nearly 18% of hospitals made strides in
only apply to Medicare patients treated in hospital
improving average LOS, but during the same
outpatient departments. Lower percentages
period, 6.3% showed worsening length of stay
indicate more efficient use of medical imaging,
–– While 16% of the hospitals studied exhibited thus reducing the exposure of patients to
an increase in inpatient expense per unnecessary contrast material and/or radiation.
discharge between 2013 and 2017 (declining The data time period for these measures is
performance), it is worth noting that 81% of July 1, 2016 – June 30, 2017.
hospitals held inpatient operating expenses
steady (important in the volatile landscape for Process of care measure
healthcare, as 100 Top Hospitals financial data Another new information only measure in the 2019
is not adjusted for inflation) study is the process of care measure for severe
sepsis and septic shock. In the July 2018 Hospital
–– HCAHPS patient experience showed the third-
Compare release, CMS started publishing hospital
highest proportion of hospitals improving,
performance rates of appropriate care given to
with 10.6% of hospitals nationwide seeing
patients with severe sepsis or septic shock. Higher
an increase in how patients rate their care
percentages indicate greater care compliance with
experience overall
regard to hospitals meeting recommended sepsis
treatment guidelines. The data time period for this
Test metrics: Reported for information only
new measure is calendar year 2017.
Every year, we evaluate the 100 Top Hospitals
study and explore whether new measures would 30-day all-cause hospital-wide
enhance the value of the analysis we provide. readmission measure
For this 2019 study, we are testing several We are continuing to publish the hospital-wide
new performance measures that update basic 30-day readmission measure, which CMS is
standards of inpatient care and expand the publicly reporting in the Hospital Compare data set
balanced scorecard across the continuum of care. to provide an overall readmission comparison, for
information only. However, we rank on a composite
If you would like to provide feedback on score based on the publicly available individual
the following proposed measures, email patient groups. The data period for the hospital-
100tophospitals@us.ibm.com. wide readmission measure is July 1, 2016 -
June 30, 2017.
Outpatient imaging efficiency 30-day episode-of-care payment measures
New to the study this year are the inclusion of We are continuing to publish risk-standardized
three outpatient imaging measures from the payments associated with 30-day episode-of-care
CMS Hospital Compare data set: follow-up measures for three patient groups that are now
mammogram, ultrasound, or MRI of the breast being published by CMS in the Hospital Compare
within 45 days after a screening mammogram; data set. These measures capture differences in
outpatient CT scans of the abdomen that were services and supplies provided to patients who
“combination” (double) scans; and outpatient have been diagnosed with AMI, HF, or pneumonia.
CT scans of the chest that were “combination” According to the CMS definition of these measures,
(double) scans. According to CMS, the purpose of they are the sum of payments made for care and
these measures is to promote high-quality, efficient supplies starting the day the patient enters the
hospital and for the next 30 days. In our study, the
data period for these measures is the same as
for the other 30-day metrics for specific patient
31
conditions: three years, combined (July 1, 2014 -
June 30, 2017). 90-day episode-of-care payment measure
Another measure recently made available in the
Excess days in acute care measures Hospital Compare data set is the 90-day episode-
The newest set of measures available from CMS of-care payment metric for primary, elective
in the Hospital Compare data set are the EDAC THA/TKA. Like the other 30-day episode-of-
measures for AMI and HF, and just released this care payment measures, CMS calculates risk-
year, Pneumonia. CMS defines “excess days” as standardized payments associated with a 90-day
the difference between a hospital’s average days in episode of care, compared to an “average” hospital
acute care and expected days, based on an average nationally. The measure summarizes payments
hospital nationally. Days in acute care include for patients across multiple care settings, services,
days spent in an ED, a hospital observation unit, and supplies during the 90-day period, which
or a hospital inpatient unit for 30 days following starts on the day of admission. The data period for
a hospitalization. The data period in our study for this measure combines three years, April 1, 2014 -
these measures is the same as for the other 30-day March 31, 2017.
metrics for specific patient conditions: three years,
combined (July 1, 2014 - June 30, 2017).
Table 9. National Performance Comparisons (All Classes)

Medians Benchmark compared with peer group
Performance measure Winning Nonwinning Difference Percent How benchmark hospitals

benchmark peer group of difference outperformed peer group
hospitals US hospitals
Mammography Follow-up Rate1 7.9 7.9 0.0 n/a3 fewer follow up procedures
Abdomen CT Use of Contrast Material Rate 1
5.5 5.9 -0.4 n/a3
fewer double scans
Thorax CT Use of Contrast Material Rate1 0.5 0.6 -0.1 n/a3 fewer double scans
Appropriate Care for Sepsis Percent2 54.0 49.0 5.0 n/a3 greater care compliance
1. Outpatient measures from CMS Hospital Compare July 1, 2016 - June 30, 2017 data set.
2. Core measures from CMS Hospital Compare Jan 1, 2017 - Dec 31, 2017 data set.
3. We do not calculate percent difference for this measure because it is already a percent value.
Table 10. National performance comparisons (all classes) — information-only metrics

Medians Benchmark compared with peer group
Performance measure Winning Nonwinning Difference Percent How benchmark hospitals

benchmark peer group of difference outperformed peer group
hospitals US hospitals
30-Day Hospital-Wide Readmission Rate1 14.9 15.3 -0.4 n/a5 fewer 30-day readmissions
30-Day AMI Episode Payment2 $23,726 $23,890 -164.5 -0.7% lower episode cost
30-Day Heart Failure Episode Payment2 $16,815 $16,604 $211 1.3% higher episode cost
30-Day Pneumonia Episode Payment 2
$17,874 $17,502 $372 2.1% higher episode cost
90-Day THA/TKA Episode Payment 3
$21,454 $21,732 -$278 -1.3% lower episode cost
90-Day THA/TKA Complications Rate 3
2.4 2.6 -0.2 n/a5
fewer complications
30-Day AMI Excess Days in Acute Care 4
-5.8 5.4 -11.2 -206% fewer days in acute care
30-Day Heart Failure Excess Days in Acute Care 4
-9.1 5.9 -15.0 -254% fewer days in acute care
30-Day Pneumonia Excess Days in Acute Care4 -6.0 7.2 -13.2 -183% fewer days in acute care
1. 30-Day hospital-wide readmission rate from CMS Hospital compare July 1, 2016 - June 30, 2017 data set.
2. 30-day episode payment metrics from CMS Hospital Compare July 1, 2014 - June 30, 2017 data set.
3. 90-Day THA/TKA payment and complication rate from CMS Hospital Compare April 1, 2014 - March 31, 2017 data set
4. 30-Day excess days in acute care metrics from CMS Hospital Compare July 1, 2014 - June 30, 2017 data set.
5. We do not calculate percent difference for this measure because it is already a percent value.

Critical access hospitals
90-day complication measure Two years ago, as part of the 2017 15 Top Health
Along with the THA/TKA 90-day payment measure Systems study, we included results for critical
recently made available in Hospital Compare access hospitals (CAHs) that were affiliated with a
data, CMS is also publishing a THA/TKA 90-day profiled health system on the performance and rate
complication measure. This measure calculates of improvement matrix graphs in the health system
a risk-standardized complication rate for elective, report. Health systems have expressed interest in
primary THA/TKA procedures using the occurrence the inclusion of their CAHs to gain a more complete
of one or more of the below complications within picture of their whole system performance.
the specified timeframes. The data period for this
measure combines three years, April 1, 2014 - Last year, we expanded the publication of CAH
March 31, 2017. results by including our findings for CAH national
–– AMI, pneumonia, or sepsis/septicemia/shock benchmark and peer group performance in the
during or within seven days of index admission 2018 100 Top Hospitals national study. Again, this
year in the 2019 national study, we are publishing
–– Surgical site bleeding, pulmonary embolism, the findings for the CAH comparison group. Since
or death during or within 30 days of index this information is presented for information only,
admission CAH winners are not being selected, as we are still
–– Mechanical complication or periprosthetic joint evaluating the usefulness of the publicly available
infection/wound infection during or within 90 data. We welcome feedback from healthcare
days of index admission leaders on the new CAH profile and reports that are
now available.
See the CMS website for measure methodology29.
CAHs with valid data for all of the six measures
Tables 9 and 10 shows the national performance of listed below were included in the analysis. A
benchmark and peer hospitals on the test metrics. total of 679 of 1,342 CAHs available in MEDPAR
Key findings include: meet this criterion. Standard 100 Top Hospitals
methodologies were applied in developing the
–– Winners outperformed peers on 30-day all- metrics and in analyzing CAH performance. See
cause hospital-wide readmissions, with a Appendix C for details on methodologies.
risk-standardized rate that was 0.4 percentage
points better than nonwinning peers (14.9% –– Risk-adjusted inpatient mortality
versus 15.3%) –– Risk-adjusted complications
–– Winners also performed better onall three of –– Pneumonia 30-day mortality
the EDAC measures, AMI, HF and pneumonia,
with median values that were 11.2, 15.0 –– Pneumonia 30-day readmissions
and 13.2days less than the peer medians, –– Severity-adjusted average LOS
respectively
–– Operating profit margin
–– On two measures, HF and pneumonia 30-day
payment, benchmark hospitals had higher total We selected the 20 top-ranked CAHs to be our
episode amounts than peers (a 1.3% higher HF benchmark group. This group outperformed
amount and a 2.1% higher pneumonia median national peers on each included measure.
total Medicare payment amount)
––The most notable difference between
benchmark CAHs compared to the peer
group was in operating profit margin (12.5
percentage points higher)
33
–– Benchmark CAHs were also strong in risk-
adjusted inpatient mortality, with 56% fewer
deaths than expected, compared to peer
hospitals with as many deaths as expected
(median index values of 0.44 and 1.00,
respectively)
–– For risk-adjusted complications, benchmark
CAHs had an index value 43.9% lower than
peers, while both had median values below 1,
reflecting fewer complications than expected
–– Average LOS was also 19.3% shorter at
benchmark CAHs, where patients left the
hospital almost a full day sooner than in
peer hospitals (2.8 days versus 3.5 days,
respectively)
–– Pneumonia 30-day rates were better at
benchmark facilities; the biggest difference
found was in the 30-day mortality rate (14.9%
versus 15.8%)
Table 11. National performance comparisons — critical access hospitals

Performance measure Medians Benchmark compared with peer group
Benchmark Peer group of Difference Percent How benchmark hospitals
hospitals US CAHs difference outperformed peer group
Mortality Index¹ 0.44 1.00 -0.6 -56.1% lower mortality

Complications Index¹ 0.42 0.74 -0.3 -43.9% fewer complications
30-Day Pneumonia Mortality Rate 2
14.9 15.8 -0.9 n/a 4
lower 30-day mortality
30-Day Pneumonia Readmission Rate 2
15.6 16.3 -0.7 n/a 4
fewer 30-day readmissions
Average Length of Stay¹ 2.8 3.5 -0.7 -19.3% shorter stays
Operating Profit Margin 3
12.6 0.1 12.5 n/a4 higher profitability
1. Mortality, complications and average length of stay based on Present on Admission (POA)-enabled risk models applied to MedPAR 2016 and 2017 data (ALOS 2017 only).
2. 30-day rates from CMS Hospital Compare July 1, 2014-June 30, 2017 data set.
3. Operating profit margin data from CMS Hospital Cost Report Information System (HCRIS) data file, 2017.
4. We do not calculate percent difference for these measures because they are already a percent value.

Note: This section details the methods used to
Methodology determine the 100 Top Hospitals award winners.
For details on the methods used to select the
Watson Health 100 Top Hospitals® is a quantitative Everest Award winners, see the Everest Awards
study that annually identifies 100 US hospitals with section of this document.
the highest achievement on a balanced scorecard.
Building the database of hospitals
The 100 Top Hospitals scorecard, based on Norton
and Kaplan’s1 concept, consists of 10 measures The publicly available data used for this study
distributed across five domains (inpatient primarily come from:
outcomes, extended outcomes, operational –– Medicare Provider Analysis and Review
efficiency, financial health, and patient experience) (MEDPAR) data set
and uses only publicly available data. The hospitals
with the highest ranking on a composite score –– Medicare Hospital Cost Reports (all-payer)
of the 10 measures are the highest-achieving –– Centers for Medicare & Medicaid Services
hospitals in the study. (CMS) Hospital Compare data sets
This 100 Top Hospitals study includes only short- We use MEDPAR patient-level demographic,
term, nonfederal, acute care US hospitals that treat diagnosis, and procedure information to calculate
a broad spectrum of patients. inpatient mortality, complications, and length
of stay (LOS). The MEDPAR data set contains
The main steps we take in selecting the 100 Top information on the approximately 15 million
Hospitals are: Medicare patients discharged annually from US
–– Building the database of hospitals, including acute care hospitals. In this study, we used the
special selection and exclusion criteria most recent two federal fiscal years of MEDPAR
data available (2016 and 2017), which include
–– Classifying hospitals into comparison groups Medicare Advantage (HMO) encounters*, to identify
by size and teaching status current performance and to select the winning
–– Scoring hospitals on a balanced scorecard of hospitals. To be included in the study, a hospital
10 performance measures across five domains must have the two most current years of data
available, with valid present-on-admission (POA)
–– Determining 100 Top Hospitals by ranking coding. Hospitals that file Medicare claims jointly
hospitals relative to their comparison groups with other hospitals under one provider number
were analyzed as one organization. Six years of
The following section is intended to be an overview MEDPAR data were used to develop the study trend
of these steps. To request more detailed information database (2012-2017).
on any of the study methodologies outlined here,
email us at 100tophospitals@us.ibm.com or call The 100 Top Hospitals program has used the
800-525-9083. MEDPAR database for many years. We believe it to
be an accurate and reliable source for the types of
high-level analyses performed in this study.
* The MEDPAR data years quoted in 100 Top Hospitals research are federal fiscal years (FFYs), a year that begins on October 1 of each calendar year and ends on September 30
of the following calendar year. FFYs are identified by the year in which they end (for example, FFY 2017 begins October 1, 2016, and ends September 30, 2017). Data for all CMS
Hospital Compare measures is provided in calendar years, except the 30-day rates. CMS publishes the 30-day rates as three-year combined data values. We label these data
points based on the end date of each data set. For example, July 1, 2014 - June 30, 2017, is named “2017.”
35
Note: To identify the Everest Award winners, we Note: Due to the lack of updated data for Medicare
also reviewed the most recent five years of data, Spend per Beneficiary (MSPB) in the CMS Hospital
2013 through 2017, to study the rate of change Compare data set, that measure was dropped from
in performance through the years. To read more the ranked metrics this year. However, last year’s
about the Everest Award methodology, see the performance and improvement graphs will be
special Everest Award section of this document. published in the reports for informational purposes.
For specific data sources for each performance
measure, see the table on page 47. We also used residency program information
to classify hospitals. This comes from the
We use Medicare Cost Reports to create our 100 Accreditation Council for Graduate Medical
Top Hospitals database, which contains hospital- Education (ACGME) and the American Osteopathic
specific demographic information and hospital- Association (AOA)*.
specific, all-payer revenue and expense data. The
Medicare Cost Report is filed annually by every US Risk- and severity-adjustment models
hospital that participates in the Medicare program. The IBM Watson Health™ proprietary risk- and
Hospitals are required to submit cost reports to severity-adjustment models for inpatient mortality,
receive reimbursement from Medicare. It should complications, and LOS have been recalibrated for
be noted that the Medicare Cost Report includes this study release using FFY 2015 data available in
all hospital costs, not just costs associated with the all-payer Watson Health’s Projected Inpatient
Medicare beneficiaries. Database (PIDB). The PIDB is one of the largest
US inpatient, all- payer databases of its kind,
The Medicare Cost Report promotes comparability containing approximately 23 million inpatient
of costs and efficiency among hospitals in reporting. discharges annually, obtained from approximately
We used hospital 2017 cost reports published in 5,000 hospitals, which comprise more than 65%
the federal Healthcare Cost Report Information of the nonfederal US market. Watson Health risk-
System (HCRIS) 2018 third-quarter data set for and severity-adjustment models take advantage of
this study. If we did not have a complete 2017 cost available POA coding that is reported in all-payer
report for a hospital, we excluded the hospital from data. Only patient conditions that are present on
the study. admission are used to determine the probability of
death, complications, or the expected LOS.
In this study, we used CMS Hospital Compare data
sets published in the third quarter of 2018 for The recalibrated models were used in producing
healthcare-associated infection (HAI) measures, the risk-adjusted inpatient mortality and
30-day mortality rates, 30-day readmission rates, complications indexes, based on two years of
emergency department (ED) throughput measures, MEDPAR data (2016 and 2017). The severity-
and Hospital Consumer Assessment of Healthcare adjusted LOS was produced based on MEDPAR
Providers and Systems (HCAHPS) patient 2017 data.
experience-of-care data. We used the 2017 data
point to identify current performance and to select
the winning hospitals. Five data points,
2013 through 2017, were used to develop the
study trend database.
* We obtain AMA graduate medical education program data directly from the ACGME. This year’s study is based on the ACGME files for 2016/2017 hospital residency programs.
AOA residency information is collected from the AOA website (opportunities.osteopathic.org). In addition, we consult online information about graduate medical education
programs from the Fellowship and Residency Electronic Interactive Database Access (FREIDA) and hospital websites to confirm program participation.

Present-on-admission coding adjustments –– Specialty hospitals (that is, critical access*,
From 2010 through 2017, we have observed children’s, women’s, psychiatric, substance
a rise in the number of principal diagnosis and abuse, rehabilitation, cardiac, orthopedic, heart,
secondary diagnosis codes that do not have a valid cancer, and long-term acute care)
POA indicator code in the MEDPAR data files. Since
–– Federally owned hospitals
2011, an invalid code of “0” has been appearing.
This phenomenon has led to an artificial rise in –– Non-US hospitals (such as those in Puerto Rico,
the number of complications that appear to be Guam, and the US Virgin Islands)
occurring during the hospital stay. See Appendix C
–– Hospitals with fewer than 25 acute care beds
for details.
–– Hospitals with fewer than 100 Medicare patient
To correct for this bias, we adjusted MEDPAR discharges in FFY 2017
record processing through our mortality and
–– Hospitals with Medicare average LOS longer
complications risk models, and LOS severity-
than 25 days in FFY 2017
adjustment model, as follows:
–– Hospitals with no reported Medicare patient
1. Original, valid (Y, N, U, W, or 1) POA codes
deaths in FFY 2017
assigned to diagnoses were retained
–– Hospitals for which a 2017 Medicare Cost
2. Where a POA code of “0” appeared, we took
Report was not available
the next four steps:
–– Hospitals with a 2017 Medicare Cost Report
a. We treated all diagnosis codes on the
that was not for a 12-month reporting period
CMS exempt list as “exempt,” regardless of
POA coding –– Hospitals that had fewer than 60% of patient
records with valid POA codes
b. We treated all principal diagnoses as
“present on admission” –– Hospitals missing data required to calculate
performance measures
c. We treated secondary diagnoses where the
POA code “Y” or “W” appeared more than
In addition, specific patient records were
50% of the time in Watson Health’s all-
also excluded:
payer database, as “present on admission”
–– Patients who were discharged to another
d. All others were treated as “not present”
short-term facility (this is done to avoid
double-counting)
Hospital exclusions
After building the database, a total of 3,156 –– Patients who were not at least 65 years old
short-term, general, acute care US hospitals were
–– Rehabilitation, psychiatric, and substance
available in the MEDPAR 2017 data file. This was
abuse patients
our starting population, prior to applying hospital
exclusions to avoid skewing study results. Excluded –– Patients with stays shorter than one day
from the study were:
After all exclusions were applied, 2,752 hospitals
were included in the study.
* In the 2019 study, critical access hospitals (CAHs) that had valid data for six measures were included in a separate analysis to provide national benchmark performance
comparisons for them. See page 33 for details on the CAH analysis.
37
Classifying hospitals into comparison groups Major teaching hospitals
Bed size, teaching status, and extent of residency/ There are three ways to qualify:
fellowship program involvement can affect 1. 400 or more acute care beds in service, plus a
the types of patients a hospital treats and the resident*-per-bed ratio of at least 0.25, plus
scope of services it provides. When analyzing
the performance of an individual hospital, it is –– Sponsorship of at least 10 GME programs, or
important to evaluate it against other similar –– Involvement in at least 20 programs overall
hospitals. To address this, we assigned each
hospital to one of five comparison groups, 2. Involvement in at least 30 GME programs
according to its size and teaching status. overall (regardless of bed size or resident*-
per-bed ratio)
Our classification methodology draws a distinction 3. A resident*-per-bed ratio of at least 0.60
between major teaching hospitals and teaching (regardless of bed size or GME program
hospitals by reviewing the number and type of involvement)
teaching programs, and by accounting for level of
involvement in physician education and research Teaching hospitals
through evidence of program sponsorship versus
simple participation. This methodology de- –– 200 or more acute care beds in service, and
emphasizes the role of bed size and focuses more –– Either a resident*-per-bed ratio of at least
on teaching program involvement. Using this 0.03 or involvement in at least three GME
approach, we seek to measure both the depth and programs overall
breadth of teaching involvement and recognize
teaching hospitals’ tendencies to reduce beds and Large community hospitals
concentrate on tertiary care.
–– 250 or more acute care beds in service, and
Our formula for defining the teaching comparison –– Not classified as a teaching hospital per
groups includes each hospital’s bed size, definitions above
residents*-to-acute-care-beds ratio, and
involvement in graduate medical education (GME) Medium community hospitals
programs accredited by either the ACGME or the
AOA. The definition includes both the number of –– 100 to 249 acute care beds in service, and
programs and type (sponsorship or participation) –– Not classified as a teaching hospital per
of GME program involvement. In this study, AOA definitions above
residency program involvement is treated as being
equivalent to ACGME program sponsorship. Small community hospitals
The five comparison groups and their parameters –– 25 to 99 acute care beds in service, and
are as follows: –– Not classified as a teaching hospital per
definitions above
Note: Again this year, we are publishing results for

the critical access hospitals (CAHs) comparison
group. These hospitals are not included in the 100
Top Hospitals analyses or selection of winners.
We are providing norms and benchmarks for six
measures, for information only. See page 33
for details.
* We include interns, residents, and fellows reported in full-time employees (FTEs) on the hospital cost report.

Scoring hospitals on weighted Operational efficiency
performance measures
6. Severity-adjusted average LOS
Evolution of performance measures
We use a balanced scorecard approach, based on 7. Mean ED throughput measure
public data, to select the measures we believe to 8. Case mix- and wage-adjusted inpatient
be most useful for boards, CEOs, and other leaders expense per discharge
in the current hospital operating environment. In
addition, we continually review trends in the Financial health
healthcare market, to identify the need for, and
availability of, new performance measurement 9. Adjusted operating profit margin
approaches. We welcome feedback from hospital
and system executives on the usefulness of our Patient experience
measures and our approach. 10. HCAHPS score (overall hospital performance)
As the healthcare industry has changed, our Following is the rationale for the selection of our
methods have evolved. Our current measures are balanced scorecard domains and the measures
centered on five main components of hospital used for each.
performance: inpatient outcomes, extended
outcomes, operational efficiency, financial health, Inpatient outcomes
and patient experience. Our measures of inpatient outcomes include
three measures: risk-adjusted mortality index,
The 10 measures included in the 2019 study, by risk-adjusted complications index, and mean
performance domain, are: healthcare-associated infection index. These
measures show us how the hospital is performing
Inpatient outcomes on what we consider to be the most basic and
1. Risk-adjusted inpatient mortality index essential care standards (survival, error-free care
and avoidance of infections) while treating patients
2. Risk-adjusted complications index in the hospital.
3. Mean HAI index
Extended outcomes
Extended outcomes The extended outcomes measures (30-day
mortality rates for AMI, HF, pneumonia, COPD, and
4. Mean 30-day risk-adjusted mortality rate stroke patients; and 30-day readmission rates for
(includes acute myocardial infarction [AMI], AMI, HF, pneumonia, THA/TKA, COPD, and stroke
heart failure [HF], pneumonia, chronic patients) help us understand how the hospital’s
obstructive pulmonary disease [COPD], patients are faring over a longer period29. These
and stroke) measures are part of the CMS Hospital Value-
5. Mean 30-day risk-adjusted readmission Based Purchasing Program and are reported upon
rate (includes AMI, HF, pneumonia, total hip widely in the industry. Hospitals with lower values
and knee arthroplasty [THA/TKA], COPD, appear to be providing or coordinating the care
and stroke) continuum with better medium-term results for
these conditions.
39
As hospitals become more interested in contracting data (calendar year 2017) for this measure.
for population health management, we believe that Instead of using last year’s data (CY 2016), we
understanding outcomes beyond the walls of the opted to drop it from the ranking, but still provide
acute care setting is imperative. We are committed the performance and improvement graphs for
to adding new metrics that assess performance informational purposes in the report, using last
along the continuum of care as they become year’s data.
publicly available.
Financial health
Operational efficiency Currently, we have one measure of hospital
The operational efficiency domain includes financial health: adjusted operating profit margin.
severity-adjusted average LOS, ED throughput, The operating profit margin is a measure of
and inpatient expense per discharge. Average management’s ability to operate within current
LOS serves as a proxy for clinical efficiency in financial constraints and provides an indicator of
an inpatient setting, while the ED throughput the hospital’s financial health. We adjust operating
measures focus on process efficiency profit margin for net related organization expense,
in one of the most important access points to as reported on the hospital cost report, to provide a
hospital care. more accurate measure of a hospital’s profitability.
See Appendix C for details on the calculation of
Average LOS is adjusted to increase the validity of this measure.
comparisons across the hospital industry. We use
a Watson Health proprietary severity-adjustment Previous studies included measures of hospital
model to determine expected LOS at the patient liquidity and asset management. We retired these
level. Patient-level observed and expected LOS measures as more and more hospitals became
values are used to calculate the hospital-level, part of a health systems. Health system accounting
severity-adjusted, average LOS. practices often recognize hospitals as units of
the system, with no cash or investment assets of
For ED throughput, we use the mean of the their own. Moreover, hospitals in health systems
reported median minutes for two critical processes: are often reported as having no debt in their own
median time from ED arrival to ED departure for name. Using public data, there is no effective way
admitted patients, and median time from ED arrival to accurately measure liquidity or other balance
to ED departure for non-admitted patients. sheet-related measures of financial health.
We adjust inpatient expense, as reported on the Patient experience

hospital cost report, for patient severity (Medicare We believe that a measure of patient perception
case mix index) and area wage levels (CMS of care (the patient “experience”) is crucial to the
area wage index applied to labor cost). These balanced scorecard concept. Understanding how
adjustments allow us to more accurately compare patients perceive the care a hospital provides, and
hospitals with different levels of patient severity how that perception compares with perceptions
operating in varying cost-of-living environments. of patients in peer hospitals, is an important
See Appendix C for details on the calculation of step a hospital can take in pursuing performance
this measure. excellence. For this reason, we calculate an
HCAHPS score, based on patient perception-of-
This year the MSPB index was dropped from care data from the HCAHPS patient survey. In this
the included ranked metrics. The CMS Hospital study, the HCAHPS score is based on the HCAHPS
Compare data set used for this study (third overall hospital rating question only.
quarter 2018 release) did not contain updated

A comprehensive, balanced view
Through the combined measures described
above, we hope to provide a balanced picture
of overall hospital performance, which can
reflect leadership’s ability to consistently
improve performance over time and sustain high
performance, once achieved. Full details about
each of these performance measures are included
on the following pages.
Performance measures
Risk-adjusted inpatient mortality index

Why we include this element Calculation Comments Favorable
values are
Patient survival is a universally accepted For this study, we calculated an index We rank hospitals on the difference Lower
measure of hospital quality. The lower value based on the number of actual between observed and expected deaths,
the mortality index, the greater the in-hospital deaths in 2016 and 2017, expressed in normalized standard
survival of the patients in the hospital, divided by the number expected, given deviation units (z-score)30,31. Hospitals
considering what would be expected the risk of death for each patient. We use with the fewest deaths, relative to the
based on patient characteristics. While our proprietary risk-adjusted mortality number expected, after accounting for
all hospitals have patient deaths, this index model to determine expected standard binomial variability, receive the
measure can show where deaths did not deaths. This model is designed to most favorable scores. We use two years
occur but were expected, or the reverse, predict the likelihood of a patient’s death of MEDPAR data (for this study, 2016
given the patient’s condition. based on patient-level characteristics and 2017) to reduce the influence of
(age, sex, presence of complicating chance fluctuation.
diagnoses, and other characteristics).
We normalize the expected value based The MEDPAR data set includes
on the observed and expected deaths for both Medicare fee-for-service claims
each comparison group. We calculate a and Medicare Advantage (HMO)
normalized index based on the observed encounter records.
and normalized expected deaths, and
patient count. Hospitals with observed values
statistically worse than expected
Palliative care patients (Z515/V66.7) are (99% confidence), and whose values
included in the risk model. POA coding are above the high trim point (75th
is used in the risk model to identify percentile of statistical outliers), are
pre-existing conditions for accurate not eligible to be named benchmark
assessment of patient severity. Do not hospitals. For more details, see
resuscitate (DNR) patients (Z66/V49.86) Appendix C.
are excluded. Post-discharge deaths
are excluded. For more information, see
Appendix C.
The reference value for this index is

1.00; a value of 1.15 indicates 15%
more deaths occurred than were
predicted, and a value of 0.85 indicates
15% fewer deaths than predicted.
41
Risk-adjusted complications index
values are
Keeping patients free from potentially We calculate an index value based on We rank hospitals on the difference Lower
avoidable complications is an important the number of cases with complications between the observed and expected
goal for all healthcare providers. A lower (for this study, in 2016 and 2017), number of patients with complications,
complications index indicates fewer divided by the number expected, given expressed in normalized standard
patients with complications, considering the risk of complications for each deviation units (z-score). We used two
what would be expected based on patient. We use our proprietary expected years of MEDPAR data (for this study,
patient characteristics. Like the mortality complications risk index models to 2016 and 2017) to reduce the influence
index, this measure can show where determine expected complications. of chance fluctuation.
complications did not occur but were These models account for patient-level
expected, or the reverse, given the characteristics (age, sex, principal The MEDPAR data set includes both
patient’s condition. diagnosis, comorbid conditions, and Medicare fee-for-service claims
other characteristics). Complication and Medicare Advantage (HMO)
rates are calculated from normative encounter records.
data for two patient risk groups:
medical and surgical. We normalize the Hospitals with observed values
expected value based on the observed statistically worse than expected
and expected complications for each (99% confidence), and whose values
comparison group. are above the high trim point (75th
percentile of statistical outliers), are
POA coding is used in the risk model not eligible to be named benchmark
to identify pre-existing conditions hospitals.
for accurate assessment of patient
severity and to distinguish them
from complications occurring during
hospitalization. For more details, see
Appendix C.
The reference value for this index is

1.00; a value of 1.15 indicates 15%
more complications occurred than
were predicted, and a value of 0.85
indicates 15% fewer complications
than predicted.

Mean healthcare-associated infection index
values are
Because there is a public interest in For this study, CMS Hospital Compare We rank hospitals on the mean Lower
tracking and preventing healthcare- data for calendar year 2017 was normalized HAI z-score, by
associated infections (HAIs), we included. Hospitals complete the comparison group.
now use the HAI data reported by required surveillance and report HAI
CMS to analyze hospital performance occurrences, and the count of patient For reporting, we calculate the mean of
and provide national benchmarks in days associated with each HAI metric, the CMS-reported SIR for the included
this area. through the US Centers for Disease HAI metrics.
Control and Prevention’s National
Healthcare Safety Network (NHSN), The CMS Hospital Compare HAI data set
which in turn reports data to CMS. includes hospital-reported HAIs for all
inpatients.
To calculate a standardized infection
ratio (SIR) for reporting HAI incidence,
expected values are developed by
the NHSN using probability models
constructed from NHSN baseline data,
which represents a standard population.
We normalize each expected value
based on the observed and expected
HAIs for each comparison group.
We use the observed, normalized

expected values and associated days to
calculate a normalized z-score for each
HAI metric. For each comparison group,
the composite HAI measure is the
mean of the individual HAI normalized
z-scores included for that group. See
Appendix C for methodology details.
Mean 30-day risk-adjusted mortality rate (AMI, HF, pneumonia, COPD, and stroke patients)
values are
30-day mortality rates are a widely Data is from the CMS Hospital Compare We rank hospitals by comparison group, Lower
accepted measure of the effectiveness data set. CMS calculates a 30-day based on the mean rate for included
of hospital care. They allow us to look mortality rate (all-cause deaths within 30-day mortality measures (AMI, HF,
beyond immediate inpatient outcomes 30 days of admission, per 100 patients) pneumonia, COPD, and stroke).
and understand how the care the for each patient condition using three
hospital provided to inpatients with years of MEDPAR data, combined. For The CMS Hospital Compare data for
these conditions may have contributed this study, we included data for the July 30-day mortality is based on Medicare
to their longer-term survival. Because 1, 2014, through June 30, 2017, data fee-for-service claims only. For more
these measures are part of the CMS set. CMS does not calculate rates for information, see Appendix C.
Hospital Value-Based Purchasing hospitals where the number of cases
Program, they are being watched is too small (less than 25). In these
closely in the industry. In addition, cases, we substitute the comparison
tracking these measures may help group-specific median rate for the
hospitals identify patients at risk for affected 30-day mortality measure. For
post-discharge problems and target more information about this data, see
improvements in discharge planning and Appendix C.
aftercare processes. Hospitals that score
well may be better prepared for a pay- We calculate the arithmetic mean of the
for-performance structure. included 30-day mortality rates (AMI,
HF, pneumonia, COPD, and stroke).
43
Mean 30-day risk-adjusted readmission rate (AMI, HF, pneumonia, THA/TKA, COPD, and stroke patients)
values are
30-day readmission rates are a widely Data is from the CMS Hospital We rank hospitals by comparison group, Lower
accepted measure of the effectiveness Compare data set. CMS calculates a based on the mean rate for included
of hospital care. They allow us to 30-day readmission rate (all-cause 30-day readmission measures
understand how the care the hospital readmissions within 30 days of (AMI, HF, pneumonia, THA/TKA, COPD,
provided to inpatients with these discharge, per 100 patients) for each and stroke).
conditions may have contributed to patient condition using three years
issues with their post-discharge medical of MEDPAR data, combined. For this
stability and recovery. study, we included data for the July
1, 2014, through June 30, 2017, data
These measures are being watched set. CMS does not calculate rates for
closely in the industry. Tracking these hospitals where the number of cases is
measures may help hospitals identify too small (less than 25). In these cases,
patients at risk for post-discharge we substitute the comparison group-
problems if discharged too soon, as specific median rate for the affected
well as target improvements in 30-day readmission measure. For
discharge planning and aftercare more information about this data, see
processes. Hospitals that score well Appendix C.
may be better prepared for a pay-for-
performance structure. We calculate the arithmetic mean of
the included 30-day readmission rates
(AMI, HF, pneumonia, THA/TKA, COPD,
and stroke).
Severity-adjusted length of stay

values are
A lower severity-adjusted average For this study, we used 2017 MEDPAR We rank hospitals on their severity- Lower
length of stay (LOS) generally indicates data for this measure. We calculate an adjusted average LOS. We severity-
more efficient consumption of hospital LOS index value by dividing the actual adjust average LOS to factor out
resources and reduced risk to patients. LOS by the normalized expected LOS. differences attributable to the varying
Expected LOS adjusts for difference severity of illness of patients at
in severity of illness using a linear each hospital.
regression model. We normalize the
expected values based on the observed
and expected LOS of the hospitals in
each comparison group. Each hospital
LOS index is converted to an average
LOS in days by multiplying by the in-
study population grand mean LOS. See
Appendix C for more information.
POA coding is used in the risk model

to identify pre-existing conditions for
accurate assessment of patient severity.
For more details, see Appendix C.

Mean emergency department throughput measure
values are
The emergency department (ED) is an Data is from the CMS Hospital Compare The mean ED throughput metric is the Lower
important access point to healthcare data set. CMS publishes the median ranked measure.
for many people. A key factor in minutes for each throughput measure,
evaluating ED performance is process by calendar year (for this study, 2017). CMS requires hospitals to submit a
throughput, which is a measurement We include two of the published sample of ED visit wait times.
of the timeliness with which patients measures in our composite: median
receive treatment and are either time from ED arrival to ED departure For more details, see Appendix C.
admitted or discharged. Timely ED for admitted patients, and median time
processes impact both care quality and from ED arrival to ED departure for non-
the quality of the patient experience. admitted patients.
We calculate the unweighted mean of

the two metrics.
Case mix- and wage-adjusted inpatient expense per discharge

values are
This measure helps to determine This measure uses Medicare Cost Adjusted inpatient expense per Lower
how efficiently a hospital cares for its Report data for hospital cost reports (for discharge measures the hospital’s
patients. Low values indicate lower costs this study, reports ending in calendar average cost of delivering inpatient care
and thus better efficiency. year 2017). We calculate the inpatient on a per-unit basis.
expense per discharge measure by
aggregating the cost center-level The hospital’s CMS-assigned case
inpatient expense from the hospital cost mix index adjusts inpatient expense
report and dividing by the total acute to account for differences in patient
inpatient discharges, adjusted for case complexity. The CMS area wage index
mix and area wage indexes. is applied to labor cost only and
accounts for geographic differences
Inpatient expense for each department in cost of living.
is calculated from fully allocated cost
using the ratio of inpatient charges to We rank hospitals on their adjusted
total charges. For inpatient nursing units, inpatient expense per discharge.
this will always be 100% of the fully
allocated cost. For departments with Hospitals with extreme outlier values
inpatient and outpatient services, the for this measure are not eligible to be
ratio will vary. named benchmark hospitals.
Non-reimbursable and special purpose

cost centers are omitted, as these have
no charges for patient care.
See Appendix C for detailed calculations

and the Medicare Cost Report locations
(worksheet, line, and column) for each
calculation element.
45
Adjusted operating profit margin
values are
Operating profit margin is one of the This measure uses Medicare Cost We adjust hospital operating expense Higher
most straightforward measures of Report data for hospital cost reports (for for net related organization expense to
a hospital’s financial health. It is a this study, reports ending in calendar obtain a true picture of the operating
measure of the amount of income a year 2017). We calculate the adjusted costs. Net related organization expense
hospital is taking in versus its expenses. operating profit margin by determining includes the net of costs covered
the difference between a hospital’s total by the hospital on behalf of another
operating revenue and total operating organization and costs covered by
expense, expressed as a percentage another organization on behalf of
of its total operating revenue, adjusted the hospital.
for net related organization expense.
Total operating revenue is the sum of We rank hospitals on their adjusted
net patient revenue plus other operating operating profit margin.
revenue. Total operating expense is
the sum of operating expense and net Hospitals with extreme outlier values
related organization expense. for this measure are not eligible to be
named benchmark hospitals.
See Appendix C for detailed calculations
and the Medicare Cost Report locations
(worksheet, line, and column) for each
calculation element.
Hospital Consumer Assessment of Healthcare Providers and Systems score (overall hospital rating)
values are
We believe that including a measure of Data is from the CMS Hospital Compare We rank hospitals based on the Higher
patient assessment/perception of care data set. For this study, we included weighted percent sum or HCAHPS score.
is crucial to the balanced scorecard the HCAHPS results for calendar year The highest possible HCAHPS score is
concept. How patients perceive the care 2017. We use the HCAHPS survey 300 (100% of patients rate the hospital
a hospital provides has a direct effect instrument question, “How do patients high). The lowest HCAHPS score is 100
on its ability to remain competitive in rate the hospital, overall?” to score (100% of patients rate the hospital low).
the marketplace. hospitals. Patient responses fall into
three categories, and the number of See Appendix C for full details.
patients in each category is reported as
a percent: HCAHPS data is survey data, based on
either a sample of hospital inpatients or
–– Patients who gave a rating of 6 or
all inpatients. The data set contains the
lower (low)
question scoring of survey respondents.
–– Patients who gave a rating of 7 or 8
(medium)
–– Patients who gave a rating of 9 or 10
(high)
For each answer category, we assign

a weight as follows: 3 equals high or
good performance, 2 equals medium or
average performance, and 1 equals low
or poor performance. We then calculate
a weighted score for each hospital
by multiplying the HCAHPS answer
percent by the category weight. For each
hospital, we sum the weighted percent
values for the three answer categories.
The result is the HCAHPS score.

Data sources and periods
Performance measure Current performance Five-year trend performance
(100 Top Hospitals award selection)
Risk-Adjusted Inpatient Mortality Index MEDPAR Federal Fiscal Year (FFY) 2016 and MEDPAR Federal Fiscal Year (FFY) 2012-2017*
2017*
Risk-Adjusted Complications Index MEDPAR FFY 2016 and 2017* MEDPAR FFY 2012-2017*
Mean Healthcare Associated Infection CMS Hospital Compare CY 2017 Trend not available
Index**
Mean 30-Day Mortality Rate (AMI, Heart CMS Hospital Compare Jul 1, 2014-Jun 30, CMS Hospital Compare: Three-year datasets
Failure, Pneumonia, COPD, Stroke) 2017 ending Jun 30 in 2013, 2014, 2015, 2016, 2017
Mean 30-Day Readmission Rate (AMI, Heart CMS Hospital Compare Jul 1, 2014-Jun 30, CMS Hospital Compare: Three-year datasets
Failure, Pneumonia, THA/TKA, COPD, Stroke) 2017 ending Jun 30 in 2013, 2014, 2015, 2016, 2017
Severity-Adjusted Average Length of Stay MEDPAR FFY 2017 MEDPAR FFY 2013-2017
Mean Emergency Department Throughput CMS Hospital Compare Calendar Year (CY) CMS Hospital Compare FFY 2013, CY 2014, CY
Measure 2017 2015, CY 2016, CY 2017 (CY2013 not published)
Inpatient Expense per Discharge (Case Mix- HCRIS Medicare Cost reports ending in 2017 HCRIS Medicare Cost reports ending in 2013-2017
and Wage-Adjusted)
Adjusted Operating Profit Margin HCRIS Medicare Cost reports ending in 2017 HCRIS Medicare Cost reports ending in 2013-2017
HCAHPS Score (Overall Hospital Rating) CMS Hospital Compare CY 2017 CMS Hospital Compare CY 2013-2017
* Two years of data is combined for each study year data point.
** The HAI measure is not included in the small community hospital group ranked metrics.
47
Determining the 100 Top Hospitals A hospital is winner-excluded if both of the
Eliminating outliers following conditions apply:
Within each of the five hospital comparison groups, 1. Observed value is higher than expected and
we rank hospitals based on their performance on the difference is statistically significant with
each of the measures relative to other hospitals in 99% confidence. When a hospital’s observed
their group. Prior to ranking, we use three methods value is 30 or greater, we use the approximate
of identifying hospitals that were performance binomial confidence interval methodology.
outliers. These hospitals are not eligible to be When a hospital’s observed value is less
named winners. than 30, we use the exact mid-p binomial
confidence interval methodology. If the
Interquartile range methodology hospital’s low confidence interval index value
We use the interquartile range methodology to is greater than or equal to 1.0, the hospital is
identify hospitals with extreme outlier values for statistically worse than expected with
the following measures: 99% confidence.
–– Case mix- and wage-adjusted inpatient 2. We calculate the 75th percentile index value
expense per discharge (high or low outliers) for mortality and complications, including
–– Adjusted operating profit margin (high and data only for hospitals that meet condition 1.
low outliers) These values are used as the high trim points
for those hospitals. Hospitals with mortality
This is done to avoid the possibility of hospitals or complications index values above the
with a high probability of having erroneous cost respective trim points are winner-excluded.
report data being declared winners.
Hospitals with a negative operating profit margin
For more information on the interquartile range We identify hospitals with a negative adjusted
methodology, see Appendix C. operating profit margin as outliers. This is done
because we do not want hospitals that fail to
Mortality and complications outliers meet this basic financial responsibility to be
For mortality and complications, which have declared winners.
observed and expected values, we identify
hospitals with performance that is statistically Ranking
worse than expected. Hospitals that are worse than Within the five hospital comparison groups, we
expected are excluded from consideration when rank hospitals on the basis of their performance on
we select the study winners. This is done because each of the performance measures independently,
we do not want hospitals that have poor clinical relative to other hospitals in their comparison
outcomes to be declared winners. group. Each performance measure is assigned a
weight for use in overall ranking (see table below).
Each hospital’s weighted performance measure
ranks are summed to arrive at a total score for the
hospital. The hospitals are then ranked based on
their total scores, and the hospitals with the best
overall rankings in each comparison group are
selected as the winners.

Table 12. Ranked performance measures and weights
Measure Weight Small community hospital weight
Risk-adjusted inpatient mortality index 1 1.25
Risk-adjusted complications index 1 1.25
Mean HAI index* 1 n/a
Mean 30-day mortality rate (AMI, HF, pneumonia, COPD, stroke) 1 1.25
Mean 30-day readmission rate (AMI, HF, pneumonia, THA/TKA, COPD, stroke) 1 1.25
Severity-adjusted average LOS 1 1
Mean ED throughput measure 1 1
Inpatient expense per discharge (case mix- and wage-adjusted) 1 1
Adjusted operating profit margin 1 1
HCAHPS score (overall hospital rating) 1 1
* HAI metrics are not ranked for small community hospitals. For this comparison group only, 2017 weights for inpatient mortality,
complications, 30-day mortality, and 30-day readmission ranks were increased to 1.25 to balance quality and operational group weights.
This study hospital population includes:
Table 13. Study populations by comparison group

Comparison group Number of winners Number of nonwinners Total hospitals in study
Major teaching hospitals 15 202 217
Teaching hospitals 25 463 488
Large community hospitals 20 270 290
Medium community hospitals 20 894 914
Small community hospitals 20 823 843
All hospitals 100 2,652 2,752
49
Appendix A
Ohio 8 15
Oklahoma 3 3
Oregon 0 1
Distribution of winners by state and region Pennsylvania 5 6
Rhode Island 1 0
Winners by state South Carolina 1 2
State Number of winners South Dakota 0 0
Current study Previous study Tennessee 1 1
Alabama 0 0 Texas 9 9
Alaska 0 0 Utah 10 3
Arizona 2 3 Vermont 0 0
Arkansas 1 0 Virginia 1 2
California 7 6 Washington 0 0
Colorado 5 6 West Virginia 0 0
Connecticut 1 0 Wisconsin 4 4
Delaware 1 0 Wyoming 0 0
District of Columbia 0 0
Florida 9 6
Winners by region
Georgia 2 0
US Census region Number of winners
Hawaii 0 0
Current study Previous study
Idaho 1 3
Northeast 8 6
Illinois 6 8
Midwest 38 43
Indiana 7 5
South 29 28
Iowa 2 2
West 25 23
Kansas 0 2
Kentucky 0 0
Louisiana 0 3
Maine 0 0
Maryland 1 0*
Massachusetts 0 0
Michigan 6 4
Minnesota 4 1
Mississippi 0 0
Missouri 1 2
Montana 0 1
Nebraska 0 0
Nevada 0 0
New Hampshire 0 0
New Jersey 1 0
New Mexico 0 0
New York 0 0
North Carolina 0 2
North Dakota 0 0
* Maryland hospitals were winner-excluded due to missing Medicare spend per beneficiary (MSPB) measure.
51
Appendix B
States included in each US Census region
US census regions
Northeast Midwest South West
Connecticut Illinois Alabama Alaska
Maine Indiana Arkansas Arizona
Massachusetts Iowa Delaware California
New Hampshire Kansas District of Columbia Colorado
New Jersey Michigan Florida Hawaii
New York Minnesota Georgia Idaho
Pennsylvania Missouri Kentucky Montana
Rhode Island Nebraska Louisiana Nevada
Vermont North Dakota Maryland New Mexico
Ohio Mississippi Oregon
South Dakota North Carolina Utah
Wisconsin Oklahoma Washington
South Carolina Wyoming
Tennessee
Texas
Virginia
West Virginia
53
Appendix C: Normative database development
Watson Health constructed a normative database
Methodology details of case-level data from its Projected Inpatient

Database (PIDB), a national all-payer database
containing more than 23 million all-payer
IBM Watson Health™ makes normative discharges annually. This data is obtained from
comparisons of mortality and complications approximately 5,000 hospitals, representing over
rates by using patient-level data to control for 65% of all discharges from short-term, general,
case mix and severity differences. We do this by nonfederal hospitals in the US. PIDB discharges are
evaluating ICD-9-CM diagnosis and procedure statistically weighted to represent the universe of
codes to adjust for severity within clinical case mix short-term, general, nonfederal hospitals in the US.
groupings. Conceptually, we group patients with Demographic and clinical data are also included:
similar characteristics (that is, age, sex, principal age, sex, and LOS; clinical groupings (Medicare
diagnosis, procedures performed, admission type, Severity Diagnosis Related Groups, or MS-DRGs),
and comorbid conditions) to produce expected, or ICD-9-CM and ICD-10-CM principal and secondary
normative, comparisons. Through testing, we have diagnoses and procedures; present-on-admission
found that this methodology produces normative (POA) coding; admission source and type; and
comparisons using readily available administrative discharge status. For this study, risk models were
data, eliminating the need for additional data recalibrated using federal fiscal year (FFY) 2015
collection32–36. all-payer data.
To support the transition from ICD-9-CM to

ICD-10-CM, our risk- and severity-adjustment Use of present-on-admission data
models have been modified to use the Agency for
Healthcare Research and Quality (AHRQ) Clinical Under the Deficit Reduction Act of 2005, as of
Classifications Software (CCS)37 categories for risk FFY 2008, hospitals receive reduced payments for
assignment. CCS categories are defined in both cases with certain conditions, such as falls, surgical
coding languages with the intent of being able to site infections, and pressure ulcers, which were
accurately compare ICD-9 categories with not present at the time of the patient’s admission
ICD-10 categories. Calibrating our models using but occurred during hospitalization. The Centers for
CCS categories provides the flexibility to accept Medicare & Medicaid Services (CMS) now requires
and process patient record data in either ICD-9 or all Inpatient Prospective Payment System (IPPS)
ICD-10 coding formats and produces consistent hospitals to document whether a patient has these
results in risk and severity adjustment. and other conditions when admitted. The Watson
Health proprietary risk- and severity- adjustment
The CCS-based approach applies to all models for inpatient mortality, complications, and
100 Top Hospitals program proprietary models LOS use POA data reported in the all-payer data to
that use code-based rate tables, which include identify conditions that were present on admission
the Risk-Adjustment Mortality Index, Expected and distinguish them from complications that
Complication Risk Index, and Expected Resource occurred while the patient was in the hospital. Our
Demand (PFD/ERD) Length of Stay (LOS) models models develop expected values based only on
used in this study. conditions that were present on admission.
55
In addition to considering the POA indicator To correct for this bias, we adjusted MEDPAR record
codes in calibration of our risk- and severity- processing through our mortality, complications,
adjustment models, we have adjusted for missing/ and LOS models as follows:
invalid POA coding found in the Medicare Provider
1. Original, valid (Y, N, U, W, or 1) POA codes
Analysis and Review (MEDPAR) data files. After
assigned to diagnoses were retained
2010, we have observed a significantly higher
percentage of principal diagnosis and secondary 2. Where a POA code of “0” appeared, we took
diagnosis codes that do not have a valid POA the next four steps:
indicator code in the MEDPAR data files. Since
a. We treated all diagnosis codes on the
2011, an invalid code of “0” has been appearing.
CMS exempt list as “exempt,” regardless of
This phenomenon has led to an artificial rise in
POA coding
the number of conditions that appear to be
occurring during the hospital stay, as invalid POA b. We treated all principal diagnoses as
codes are treated as “not present” by POA-enabled “present on admission”
risk models.
c. We treated secondary diagnoses where the
POA code “Y” or “W” appeared more than
50% of the time in Watson Health’s all-
payer database, as “present on admission”
d. All others were treated as “not present”
Percentage of diagnosis codes with POA indicator code of “0” by MEDPAR year
2010 2011 2012 2013 2014 2015 2016 2017
Principal diagnosis 0.00% 4.26% 4.68% 4.37% 3.40% 4.99% 2.45% 3.96%
Secondary diagnosis 0.00% 15.05% 19.74% 22.10% 21.58% 23.36% 21.64% 24.11%

Methods for identifying patient severity Hospice versus palliative
Without adjusting for differences in patient severity, care patients
comparing outcomes among hospitals does not
present an accurate picture of performance. To Separately licensed hospice unit
make normative comparisons of hospital outcomes, patient records are not included in
we must adjust raw data to accommodate MEDPAR data. They have a separate
differences that result from the variety and severity billing type and separate provider
numbers. In addition, patients
of admitted cases.
receiving hospice treatment in acute
care beds are billed under hospice,
Risk-adjusted inpatient mortality index models
not the hospital, and would not be in
Watson Health has developed an inpatient the MEDPAR data file.
mortality risk model that can be applied to coded
patient claims data to estimate the expected Inpatients coded as palliative care
probability of death occurring, given various (V66.7) (Z515) are included in the
patient-related factors. The mortality risk model study. Over the past few years, the
used in this study is calibrated for patients age 65 number of patients coded as palliative
and older. Additionally, in response to the transition care has increased significantly, and
to ICD-10-CM, diagnosis and procedure codes (and our risk models have been calibrated
the interactions among them) have been mapped to produce expected values for
to the AHRQ CCS for assignment of risk instead these patients.
of using the individual diagnosis, procedure, and
interaction effects.
We exclude long-term care, psychiatric, substance

abuse, rehabilitation, and federally owned or
controlled facilities. In addition, we exclude certain
patient records from the data set: psychiatric;
substance abuse; unclassified cases (MS-DRGs
945, 946, and 999); cases in which patient age was
less than 65 years; and cases in which a patient
transferred to another short-term, acute care
hospital. Palliative care patients (Z515; V66.7)
are included in the mortality risk model, which is
calibrated to estimate probability of death for these
patients. The Watson Health mortality risk model
excludes records with “do not resuscitate” (DNR)
(Z66; V49.86) orders that are coded as present
on admission.
Note: We are no longer able to exclude all

rehabilitation patients as we have done in the past.
This is because the ICD-10-CM coding system
does not identify rehabilitation patients. We can
only exclude those patients coded as being in
a Prospective Payment System (PPS)-exempt
hospital rehabilitation unit (provtype = R or T).
57
Excluding records that are DNR status at admission Expected complications rate index models
is supported by the literature. A recent peer- Watson Health has developed a complications
reviewed publication stated: “Inclusion of DNR risk model that can be applied to coded patient
patients within mortality studies likely skews those claims data to estimate the expected probability
analyses, falsely indicating failed resuscitative of a complication occurring, given various
efforts rather than humane decisions to limit care patient-related factors. We exclude long-term
after injury”38. Our rationale is straightforward: If care, psychiatric, substance abuse, rehabilitation,
a patient is admitted DNR (POA), then typically no and federally owned or controlled facilities. In
heroic efforts would be made to save that patient if addition, we exclude certain patient records
they began to fail. Without the POA DNR exclusion, from the data set: psychiatric; substance abuse;
if a given hospital has a higher proportion of POA unclassified cases (MS-DRGs 945, 946, and
DNR patients that it is not attempting to save from 999); cases in which patient age was less than 65
death compared to an otherwise similar hospital years; and cases in which a patient transferred to
that is not admitting as high a proportion of such another short-term, acute care hospital. Palliative
patients, the first hospital would look lower- care patients (Z515; V66.7) are included in the
performing compared to the second through no complications risk model, which is calibrated to
fault of its own. The difference would be driven by estimate probability of complications for
the proportion of POA DNR patients. these patients.
A standard logistic regression model is used to Note: We are no longer able to exclude all
estimate the risk of mortality for each patient. This rehabilitation patients as we have done in the past.
is done by weighting the patient records of the This is because the ICD-10-CM coding system does
hospital by the logistic regression coefficients not identify rehabilitation patients. We can only
associated with the corresponding terms in the exclude those patients coded as being in a PPS-
model and the intercept term. This produces exempt hospital rehabilitation unit (provtype =
the expected probability of an outcome for R or T).
each eligible patient (numerator) based on the
experience of the norm for patients with similar Risk-adjusted complications refer to outcomes that
characteristics (for example, age, clinical grouping, may be of concern when they occur at a greater-
and severity of illness)32–36. This model accounts than-expected rate among groups of patients,
for only patient conditions that are present on possibly reflecting systemic quality-of-care
admission when calculating risk. Additionally, in issues. The Watson Health complications model
response to the transition to ICD-10-CM, diagnosis uses clinical qualifiers to identify complications
and procedure codes, and the interactions among that have occurred in the inpatient setting. The
them, have been mapped to AHRQ CCS categories complications used in the model are listed on the
for assignment of risk instead of using the following page.
individual diagnosis, procedure, and interaction
effects. See discussion under the methods for
identifying patient severity above.
Staff physicians at Watson Health suggested

clinical patient characteristics that were
incorporated into the proprietary models. After
assigning the predicted probability of the outcome
for each patient, the patient-level data can then be
aggregated across a variety of groupings, including
health system, hospital, service line, or MS-DRG
classification.

Complication Patient group
Postoperative complications relating to urinary tract Surgical only
Postoperative complications relating to respiratory system except pneumonia Surgical only
Gastrointestinal (GI) complications following procedure Surgical only
Infection following injection/infusion All patients
Decubitus ulcer All patients
Postoperative septicemia, abscess, and wound infection Surgical, including cardiac
Aspiration pneumonia Surgical only
Tracheostomy complications All patients
Complications of cardiac, vascular and hemodialysis devices Surgical, including cardiac
Nervous system complications from devices/complications of nervous system devices Surgical only
Complications of genitourinary devices Surgical only
Complications of orthopedic devices Surgical only
Complications of other and unspecified devices, implants, and grafts Surgical only
Other surgical complications Surgical, including cardiac
Miscellaneous complications All patients
Cardio-respiratory arrest, shock, or failure Surgical only
Postoperative complications relating to nervous system Surgical only
Postoperative acute myocardial infarction (AMI) Surgical only
Postoperative cardiac abnormalities except AMI Surgical only
Procedure-related perforation or laceration All patients
Postoperative physiologic and metabolic derangements Surgical, including cardiac
Postoperative coma or stupor Surgical, including cardiac
Postoperative pneumonia Surgical, including cardiac
Pulmonary embolism All patients
Venous thrombosis All patients
Hemorrhage, hematoma, or seroma complicating a procedure All patients
Postprocedure complications of other body systems All patients
Complications of transplanted organ (excludes skin and cornea) Surgical only
Disruption of operative wound Surgical only
Complications relating to anesthetic agents and central nervous system (CNS) depressants Surgical, including cardiac
Complications relating to antibiotics All patients
Complications relating to other anti-infective drugs All patients
Complications relating to antineoplastic and immunosuppressive drugs All patients
Complications relating to anticoagulants and drugs affecting clotting factors All patients
Complications relating to narcotics and related analgesics All patients
Complications relating to non-narcotic analgesics All patients
Complications relating to anticonvulsants and antiparkinsonism drugs All patients
Complications relating to sedatives and hypnotics All patients
Complications relating to psychotropic agents All patients
Complications relating to CNS stimulants and drugs affecting the autonomic nervous system All patients
Complications relating to drugs affecting cardiac rhythm regulation All patients
Complications relating to cardiotonic glycosides (digoxin) and drugs of similar action All patients
Complications relating to other drugs affecting the cardiovascular system All patients
Complications relating to antiasthmatic drugs All patients
Complications relating to other medications (includes hormones, insulin, iron, and oxytocic agents) All patients
59
A standard regression model is used to estimate Examples:
the risk of experiencing a complication for
each patient. This is done by weighting the 10 events observed ÷ 10 events expected = 1.0:
patient records of the hospital by the regression The observed number of events is equal to the
coefficients associated with the corresponding expected number of events based on the
terms in the prediction models and intercept term. normative experience
This method produces the expected probability
of a complication for each patient based on the 10 events observed ÷ 5 events expected = 2.0:
experience of the norm for patients with similar The observed number of events is twice the
characteristics. After assigning the predicted expected number of events based on the
probability of a complication for each patient in normative experience
each risk group, it is then possible to aggregate the
patient-level data across a variety of groupings39–42, 10 events observed ÷ 25 events expected = 0.4:
including health system, hospital, service line, The observed number of events is 60% lower
or MS-DRG classification. This model accounts than the expected number of events based on the
for only patient conditions that are present on normative experience
admission when calculating risk. Additionally,
in response to the transition to ICD-10-CM, Therefore, an index value of 1.0 indicates no
diagnosis and procedure codes, and the difference between observed and expected
interactions among them, have been mapped to outcome occurrence. An index value greater than
AHRQ CCS categories for assignment of risk instead 1.0 indicates an excess in the observed number
of using the individual diagnosis, procedure, and of events relative to the expected based on the
interaction effects. normative experience. An index value of less than
1.0 indicates fewer events observed than would
Index interpretation be expected based on the normative experience.
An outcome index is a ratio of an observed An additional interpretation is that the difference
number of outcomes to an expected number of between 1.0 and the index is the percentage
outcomes in a population. This index is used to difference in the number of events relative to the
make normative comparisons and is standardized norm. In other words, an index of 1.05 indicates
in that the expected number of events is based 5% more outcomes, and an index of 0.90 indicates
on the occurrence of the event in a normative 10% fewer outcomes than expected based on
population. The normative population used to the experience of the norm. The index can be
calculate expected numbers of events is selected calculated across a variety of groupings (for
to be similar to the comparison population with example, hospital or service line).
respect to relevant characteristics, including age,
sex, region, and case mix.
The index is the number of observed events divided

by the number of expected events and can be
calculated for outcomes that involve counts of
occurrences (for example, deaths or complications).
Interpretation of the index relates the experience
of the comparison population relative to a specified
event to the expected experience based on the
normative population.

Healthcare-associated infections To enable reporting of a hospital’s general
Healthcare-associated infections (HAIs), as performance level on the HAI measures overall,
developed by the National Healthcare Safety we calculate a composite HAI measure for each
Network* (NHSN) and reported by CMS in the hospital. Each facility’s composite HAI measure
public Hospital Compare data set, capture new considers only the HAIs included for its designated
information about the quality of inpatient care. 100 Top Hospitals® comparison group, as indicated
Tracking and intervening to reduce infection rates in the table below. Since not all hospitals report
for methicillin-resistant staphylococcus aureus data for all six HAIs, we vary the number of
(MRSA), central line-associated blood stream included HAI measures based on data availability
infections (CLABSI), catheter-associated urinary in each comparison group.
tract infection (CAUTI), clostridium difficile colitis
(C.diff), and other problematic infections must be HAIs by compare group
reported to CMS. New public data will allow the Compare group Included HAIs Minimum
development of national benchmarks for use by required
hospital leadership to affect change. Major teaching HAI-1, HAI-2, HAI-3, HAI-4, 4
HAI-5, HAI-6
Teaching HAI-1, HAI-2, HAI-3, HAI-5, 4
HAI measures HAI-6
HAI-1 CLABSI in ICUs and select wards
Large community HAI-1, HAI-2, HAI-3, HAI-5, 4
HAI-2 CAUTI in intensive care units (ICUs) and select wards HAI-6
HAI-3 Surgical site infection (SSI): colon Medium community HAI-1, HAI-2, HAI-6 1
HAI-4 Surgical site infection from abdominal hysterectomy Small community Not ranked n/a
(SSI: hysterectomy)
HAI-5 Methicillin-resistant staphylococcus aureus (MRSA) blood
laboratory-identified events (bloodstream infections) In addition to the SIR values for each HAI, CMS
HAI-6 C.diff laboratory-identified events (intestinal infections) publishes the observed and expected values, as
well as a population count (days or procedures),
which varies by measure**. We normalize the
The HAI measures are reported as risk-
individual hospital expected values for each HAI
adjusted standardized infection ratios (SIRs)
by multiplying them by the ratio of the observed to
using probability models and normative data
expected values for their comparison group for
sets maintained by a branch of the Centers for
that HAI.
Disease Control and Prevention (CDC), the NHSN.
Along with reporting SIR data to CMS, NHSN is
We calculate a normalized z-score for each HAI,
responsible for administering HAI surveillance
for each hospital, using the observed, normalized
procedures and reporting specifications, along with
expected and count. We did not calculate a z-score
producing software and training programs for all
for an individual HAI if CMS did not report a SIR
participating hospitals. Its underlying methodology
value for that measure in the Hospital Compare
details for building the SIR are documented and
data set.
updated annually in a reference guide posted at the
CDC website43.
* See blog.eoscu.com/blog/what-is-the-national-healthcare-safety-network for more information.

** CLABSI: device days, CAUTI: urinary catheter days, SSI colon: procedures, SSI hysterectomy: procedures, MRSA: patient days, C.diff: patient days.
61
Data note relating to the To develop a composite HAI measure, we believe it
July 2016 Hospital Compare is not appropriate to simply “roll up” observed and
performance period expected values across the different HAIs because
(July 1, 2012 - June 30, the overall observed to expected ratio would be
2015): weighted by the rates for each HAI, which could be
quite different, and the HAIs are also likely to be
The pneumonia measure distributed differently from hospital to hospital. For
cohort was expanded to these reasons, we calculate an unweighted mean
include principal discharge of the normalized z-scores as the composite HAI
codes for sepsis and measure used for ranking hospitals.
aspiration pneumonia. This
resulted in a significant For reporting, we calculate an unweighted mean
increase in pneumonia 30- of the CMS SIRs for each hospital. If no value was
day mortality rates nationally, available for a measure, the composite measure
beginning with the 2015 data represents the mean of available measures, as long
year. as the hospital had the minimum required number
of HAIs for its comparison group. For each HAI, the
SIR can be viewed as a unitless measure that is
essentially a percent difference; that is, observed
to expected ratio minus 1 x 100 = percent
difference, which is unbiased by differences in the
rates by HAI or distributions of HAIs by hospital. It
is methodologically appropriate to ask: What is the
average (mean) percent difference between my
observed rates of HAIs and the expected rates of
those HAIs?
30-day risk-adjusted mortality rates and 30-day

risk-adjusted readmission rates
This study currently includes two extended
outcome measures (30-day mortality and 30-day
readmissions), as developed by CMS and published
in the Hospital Compare data set. CMS is reporting
three-year rolling data periods, with the most
current data set being July 1, 2014 - June 30,
2017. The Hospital Compare website and database
were created by CMS, the US Department of Health
and Human Services, and other members of the
Hospital Quality Alliance. The data on the website
comes from hospitals that have agreed to submit
quality information that will be made public. Both
measures used in this study have been endorsed
by the National Quality Forum (NQF).

CMS calculates the 30-day mortality and 30-day Length-of-stay methodologies
readmission rates from Medicare enrollment Watson Health has developed a severity-adjusted
and claims records using statistical modeling resource demand model that can be applied to
techniques that adjust for patient-level risk coded patient claims data to estimate the expected
factors and account for the clustering of patients LOS, given various patient-related factors44. We
within hospitals. Only Medicare fee-for-service exclude long-term care, psychiatric, substance
records are included. We are including 30-day abuse, rehabilitation, and federally owned or
mortality rates for acute myocardial infarction controlled facilities. In addition, we exclude certain
(AMI), heart failure (HF), pneumonia, chronic patient records from the data set: psychiatric;
obstructive pulmonary disease (COPD), and stroke substance abuse; unclassified cases (MS-DRGs
patients, and 30-day readmission rates for AMI, HF, 945, 946, and 999); cases in which patient age was
pneumonia, elective total hip and knee arthroplasty less than 65 years; and cases in which a patient
(THA/TKA), COPD, and stroke patients. was transferred to another short-term, acute care
hospital. Palliative care patients (Z515; V66.7) are
The individual CMS mortality models estimate included in the LOS model, which is calibrated to
hospital-specific, risk-standardized, all-cause predict expected LOS for these patients.
30-day mortality rates for patients hospitalized
with a principal diagnosis of AMI, HF, pneumonia, Note: We are no longer able to exclude all
COPD, or stroke. All-cause mortality is defined rehabilitation patients, as we have done in the
as death from any cause within 30 days after the past, because the ICD-10-CM coding system
admission date, regardless of whether the patient does not identify rehabilitation patients. We can
dies while still in the hospital or after discharge. only exclude those patients coded as being in a
PPS-exempt hospital rehabilitation unit (provtype
The individual CMS readmission models estimate = R or T).
hospital-specific, risk-standardized, all-cause
30-day readmission rates for patients discharged Our severity-adjusted resource demand model
alive to a non-acute care setting with a principal allows us to produce risk-adjusted performance
diagnosis of AMI, HF, pneumonia, THA/TKA, COPD, comparisons on LOS between or across subgroups
or stroke. Patients may have been readmitted of inpatients. These patient groupings can be
back to the same hospital, to a different hospital, based on factors such as clinical groupings,
or to another acute care facility. They may have hospitals, product lines, geographic regions, and
been readmitted for the same condition as their physicians. This regression model adjusts for
recent hospital stay or for a different reason differences in diagnosis type and illness severity,
(CMS has indicated this is to discourage hospitals based on ICD-9-CM coding. It also adjusts for
from coding similar readmissions as different patient age, gender, and admission status. Its
readmissions)30. All readmissions that occur 30 associated LOS weights allow group comparisons
days after discharge to a non-acute care setting on a national level and in a specific market area.
are included, with a few exceptions. CMS does not In response to the transition to ICD-10-CM,
count planned admissions (obstetrical delivery, diagnosis, procedure, and interaction codes
transplant surgery, maintenance chemotherapy, have been mapped to AHRQ CCS categories for
rehabilitation, and non-acute admissions for a severity assignment instead of using the individual
procedure) as readmissions. diagnosis, procedure, and interaction effects.
63
POA coding allows us to estimate appropriate Inpatient expense per discharge and operating
adjustments to LOS weights based on pre-existing profit margin measure calculations
conditions. Complications that occurred during the For this study, we used hospital-reported data
hospital stay are not considered in the model. We from 2017 Medicare cost reports available in the
calculate expected values from model coefficients Hospital Cost Report Information System 2018
that are normalized to the clinical group and third-quarter data file to calculate the inpatient
transformed from log scale. expense per discharge and operating profit margin
measures. Below you will find our calculations
and the cost report locations (worksheet, line, and
Emergency department throughput measure column) of data elements for these measures. The
We have included two emergency department line and column references are the standard based
(ED) throughput measures from the CMS Hospital on CMS Form 2552-10.
Compare data set. The hospital ED is an access
point to healthcare for many people. A key factor in Case mix- and wage-adjusted inpatient expense
evaluating ED performance is process “throughput,” per discharge
measures of timeliness with which patients are [((0.62 × acute inpatient expense ÷ CMS wage index)
seen by a provider, receive treatment, and either + 0.38 × acute inpatient expense)
are admitted or discharged. Timely ED processes ÷ acute inpatient discharges]
may impact both care quality and the quality of the ÷ Medicare case mix index
patient experience. We chose to include measures
that define two ED processes: median time from ED acute inpatient expense = inpatient expense −
arrival to ED departure for admitted patients, and subprovider expense − nursery expense − skilled
median time from ED arrival to ED departure for nursing facility expense − intermediate-care
non-admitted patients. facility expense − other long-term care facility
expense − cost centers without revenue (for
For this study’s measure, we used 2017 data from example, organ procurement, outpatient therapy,
CMS Hospital Compare. Hospitals are required and other capital-related costs)
to have reported both ED measures or they are
excluded from the study. Our ranked metric is the inpatient expense = sum over all departments
calculated mean of the two included measures. [(inpatient department charges
÷ department charges) × department cost]
Hospitals participating in the CMS Inpatient
Quality Reporting and Outpatient Quality Reporting Individual element locations in the Medicare
Programs report data for any eligible adult ED Cost Report:
patients, including Medicare patients, Medicare
–– Acute inpatient discharges — worksheet S-3,
managed care patients, and non-Medicare patients.
line 14, column 15
Submitted data can be for all eligible patients or a
sample of patients, following CMS sampling rules. –– Inpatient department (cost center) elements
–– Fully allocated cost — worksheet C, part 1,
ED throughput measures
column 1; if missing, use worksheet B, part 1,
ED-1b Average time patients spent in the ED before they were
admitted to the hospital as an inpatient column 26
OP-18b Average time patients spent in the ED before being ––Total charges — worksheet C, part 1,
sent home
column 8
–– Inpatient charges — worksheet C, part 1,
column 6

–– Medicare case mix index — Federal Register: When the reported value is less than 80% of the
CMS IPPS FFY 2017 Final Rule table 2 (cost reported related organization expense, we subtract
report end dates in 2017 Q1, Q2, Q3) or IPPS the G-2 expense additions from total operating
FFY2018, table 2 (cost report end dates in expense. When the G-2 expense additions is any
2017 Q4) other value, we back-out related organization
expense from total operating expense. In this study,
–– CMS wage index — CMS Federal Register: CMS
2.4% of in-study hospitals received this correction.
IPPS FFY 2017 (cost report end dates in 2017
Q1, Q2, Q3) or IPPS FFY2018, table 2 (cost
Hospital Consumer Assessment of Healthcare
report end dates in 2017 Q4)
Providers and Systems overall hospital rating
Adjusted operating profit margin To measure patient perception of care, this

[(net patient revenue + other operating revenue − study uses the Hospital Consumer Assessment
(total operating expense + net related organization of Healthcare Providers and Systems (HCAHPS)
expense)) ÷ (net patient revenue + other operating patient survey. HCAHPS is a standardized survey
revenue)] × 100 instrument and data collection methodology for
measuring patients’ perspectives on their hospital
other operating revenue = [total other income − care. HCAHPS is a core set of questions that can be
other income: (for example, contributions and combined with customized, hospital-specific items
donations) − other income from investments] to produce information that complements the
data hospitals currently collect to support internal
Individual element locations in the Medicare customer service and quality-related activities.
Cost Report:
HCAHPS was developed through a partnership
–– Net patient revenue — worksheet G-3, line 3, between CMS and AHRQ that had three
column 1 broad goals:
––Total other income — worksheet G-3, line 25, –– Produce comparable data on patients’
column 1 perspectives of care that allow objective and
–– Other income: contributions, donations, etc. — meaningful comparisons among hospitals on
worksheet G-3, line 6, column 1 topics that may be important to consumers
–– Other income from investments — worksheet –– Encourage public reporting of the survey
G-3, line 7, column 1 results to create incentives for hospitals to
improve quality of care
––Total operating expense — worksheet G-3, line
4, column 1 –– Enhance public accountability in healthcare
by increasing the transparency of the quality
–– Related organization expense — worksheet A-8, of hospital care provided in return for the
line 12, column 2 public investment
Note: When a hospital has already reported the net The HCAHPS survey has been endorsed by the
related organization expense in its total operating NQF and the Hospital Quality Alliance. The federal
expense, we subtract it back out to avoid double- government’s Office of Management and Budget
counting. This issue is identified on worksheet G-2 has approved the national implementation of
expense additions, lines 30 through 35 (including HCAHPS for public reporting purposes.
sublines) where titles contain references to “home
office,” “related organization,” “shared services,” Voluntary collection of HCAHPS data for public
“system assessment,” “corporate allocation,” or reporting began in October 2006. The first public
“internal allocation.” reporting of HCAHPS results, which encompassed
eligible discharges from October 2006 through
65
June 2007, occurred in March 2008. HCAHPS Performance measure normalization
results are posted on the Hospital Compare The inpatient mortality, complications, and LOS
website, found at medicare.gov/hospitalcompare. measures are normalized based on the in-study
A downloadable version of HCAHPS results population, by comparison group, to provide
is available. a more easily interpreted comparison among
hospitals. To address the impact of bed size and
The HCAHPS data is adjusted by CMS for both teaching status, including extent of residency
survey mode (phone, web, or mail survey) and program involvement, and compare hospitals to
the patient mix at the discharging facility, since other like hospitals, we assign each hospital in
respondents randomized to the phone mode tend the study to one of five comparison groups (major
to provide more positive evaluations about their teaching, teaching, large community, medium
care experience than those randomized to the community, and small community hospitals).
mail survey mode. Details on this adjustment’s Detailed descriptions of the hospital comparison
parameters are available for all facilities with each groups can be found in the Methodology section of
quarterly update, at hcahpsonline.org. the 100 Top Hospitals study.
Although we report hospital performance on all For the mortality and complications measures,
HCAHPS questions, only performance on the we base our ranking on the difference between
overall hospital rating question, “How do patients observed and expected events, expressed in
rate the hospital, overall?” is used to rank hospital standard deviation units (z-scores) that have
performance. Patient responses fall into three been normalized. We normalize the individual
categories, and the number of patients in each hospital expected values by multiplying them by
category is reported as a percent: the ratio of the observed to expected values for
–– Patients who gave a rating of 6 or lower (low) their comparison group. We then calculate the
normalized z-score based on the observed and
–– Patients who gave a rating of 7 or 8 (medium) normalized expected values and the patient count.
–– Patients who gave a rating of 9 or 10 (high)
For the HAI measures, we base our ranking on
For each answer category, we assign a weight as the unweighted mean of the normalized z-scores
follows: 3 equals high or good performance, 2 for the included HAIs. Included HAIs vary by
equals medium or average performance, and 1 comparison group. See page 61 for details. We
equals low or poor performance. We then calculate normalize the individual hospital expected values
a weighted score for each hospital by multiplying for each HAI by multiplying them by the ratio of the
the HCAHPS answer percent by the category observed to expected values for their comparison
weight. For each hospital, we sum the weighted group for that HAI. We calculated a normalized
percent values for the three answer categories. z-score for each HAI, for each hospital, using the
Hospitals are then ranked by this weighted percent observed, normalized expected and count.
sum. The highest possible HCAHPS score is 300
(100% of patients rate the hospital high). The For the LOS measure, we base our ranking on the
lowest possible HCAHPS score is 100 (100% of normalized, severity-adjusted LOS index expressed
patients rate the hospital low). in days. This index is the ratio of the observed and
the normalized expected values for each hospital.
We normalize the individual hospital’s expected
values by multiplying them by the ratio of the
observed to expected values for its comparison
group. The hospital’s normalized index is then
calculated by dividing the hospital’s observed value

by its normalized expected value. We convert this Both the current and trend profiles are
normalized index into days by multiplying by the internally consistent. They each provide relevant
average LOS of all in-study hospitals (grand comparisons of a profiled hospital’s performance
mean LOS). versus peers and national benchmarks.
Differences between current and trend profiles Interquartile range methodology

Normalization For each measure, we calculate an interquartile
The 2017 values on the current and trend range (IQR) based on data for all in-study hospitals.
graphs will not match for inpatient mortality, Two outlier points (trim points) are set for each
complications, or average LOS. This is because measure: one upper limit and one lower limit.
we use different norm factors to normalize the
expected values. A value (X) is considered an outlier if either of the
following is true:
–– Current profile: We combine in-study hospitals’
data for only the most current study year X > = upper-limit outlier point
to calculate each comparison group norm
X < = lower-limit outlier point
factor (observed/expected). Note: The current
study year was comprised of 2016 and 2017
The procedure for calculating the IQR and outlier
MEDPAR data for inpatient morality and
points is as follows:
complications, and 2017 data only for
average LOS. –– Determine the first quartile (Q1). This is the
25th percentile value of all records in the
––Trend profile: We combine in-study hospitals’
population.
data for all five study years to calculate each
comparison group norm factor. –– Determine the third quartile (Q3). This is the
75th percentile value of all records in the
In-study hospital counts population.
There are fewer in-study hospitals in the trend
–– Calculate the IQR by subtracting Q1 from Q3
profile than the current profile because some
(IQR = Q3 – Q1).
hospitals do not have enough data points for one or
more measures to calculate trend, so they –– Calculate the upper- and lower-limit trim
are excluded. points for inpatient expense per discharge:
–– Additional impact on average LOS calculation: –– Upper-limit = Q3 + (3.0 × IQR)
The observed/normalized expected LOS index
–– Lower-limit = Q1 – (3.0 × IQR)
for each hospital is converted into an average
LOS in days by multiplying it by the mean –– Calculate the upper- and lower-limit trim
average LOS for all in-study hospitals (sum points for operating profit margin:
observed LOS/in-study hospital count). The
–– Upper limit = Q3 + (2.0 × IQR)
grand mean average LOS will be different in
current and trend profiles when there are –– Lower limit = Q1 – (2.0 × IQR)
different numbers of in-study hospitals.
Data points that are outside the IQR limits are
considered extreme outliers and are excluded.
67
Why we have not calculated percent change in
specific instances
Percent change is a meaningless statistic when
the underlying quantity can be positive, negative,
or zero. The actual change may mean something,
but dividing it by a number that may be zero or of
the opposite sign does not convey any meaningful
information because the amount of change is not
proportional to its previous value.
We also do not report percent change when the

metrics are already percentages. In these cases,
we report the simple difference between the two
percentage values.
Protecting patient privacy

We do not report any individual hospital data that is
based on 11 or fewer patients, as required by CMS.
This affects the following measures:
–– Risk-adjusted inpatient mortality index
–– Risk-adjusted complications index
–– 30-day mortality rates for AMI, HF, pneumonia,
COPD, and stroke (CMS does not report a rate
when count is less than 25)
–– 30-day readmission rates for AMI, HF,
pneumonia, THA/TKA, COPD, and stroke
(CMS does not report a rate when count is less
than 25)
–– Average LOS

Footnotes 9. Foster DA, Chenoweth J. Comparison 17. Foster DA. Trends in Patient Safety
of Baldrige Award Applicants and Adverse Outcomes and 100 Top
1. Kaplan RS, Norton DP. The Balanced Recipients With Peer Hospitals on a Hospitals Performance, 2000-2005.
Scorecard: Measures That Drive National Balanced Scorecard. Ann Arbor, Ann Arbor, MI: Center for Healthcare
Performance. Harvard Bus Rev, MI: Center for Healthcare Improvement, Improvement, Watson Health (Truven
Jan–Feb 1992. Watson Health (Truven Health Analytics). Health Analytics). March 2008.
October 2011.
2. Young J. Trends in Acute Myocardial 18. Shook J, Young J. Inpatient and
Infarction Incidence, Detection, and 10. Young J. Outpatient Care Standards Outpatient Growth by Service Line:
Treatment. Ann Arbor, MI: Center for Followed More Closely at Top- 2006 100 Top Hospitals: Performance
Healthcare Improvement, Watson Performing Hospitals. Ann Arbor, MI: Improvement Leaders Versus Peer
Health (Truven Health Analytics). Center for Healthcare Improvement, Hospitals. Ann Arbor, MI: Center for
November 2014. Watson Health (Truven Health Analytics). Healthcare Improvement, Watson
March 2011. Health (Truven Health Analytics).
3. Chenoweth J, Foster D, Shook August 2007.
J. Linking CEO Compensation to 11. Young J. Hospitals Increase
Organization-Wide Performance. Ann Cardiovascular Core Measure 19. Chenoweth J, Safavi K. Leadership
Arbor, MI: Center for Healthcare Compliance. Ann Arbor, MI: Center Strategies for Reaching Top Performance
Improvement, Watson Health (Truven for Healthcare Improvement, Watson Faster. J Healthc Tech. January 2007.
Health Analytics). October 2014. Health (Truven Health Analytics). HCT Project Volume 4.
November 2010.
4. Foster DA. CMS Readmission 20. McDonagh KJ. Hospital Governing
Penalties: Estimating the Impact of 12. Foster DA. Top Cardiovascular Boards: A Study of Their Effectiveness in
Socioeconomics and Race. Ann Arbor, Care Means Greater Clinical and Relation to Organizational Performance.
MI: Center for Healthcare Improvement, Financial Value. Ann Arbor, MI: Center Healthc Manag. 2006 Nov–Dec; 51(6).
Watson Health (Truven Health Analytics). for Healthcare Improvement, Watson
September 2014. Health (Truven Health Analytics). 21. Griffith JR, Alexander JA, Foster
November 2009. DA. Is Anybody Managing the Store?
5. Foster DA, Young J, Heller S. National Trends in Hospital Performance.
Community Need Associated With 13. Lee DW, Foster DA. The association Healthc Manag. 2006 Nov-Dec; 51(6):
30-Day Readmissions. Ann Arbor, MI: between hospital outcomes and 392-405; discussion 405-6.
Center for Healthcare Improvement, diagnostic imaging: early findings. J Am
Watson Health (Truven Health Analytics). Coll Radiol. 2009 Nov; 6(11): 780-5. 22. Bass K, Foster DA, Chenoweth J.
June 2014. Study Results—Proving Measurable
14. Foster DA. HCAHPS 2008: Leadership and Engagement Impact on
6. Shook J, Chenoweth J. 100 Top Comparison Results for 100 Top Quality, CMS Invitational Conference on
Hospitals CEO Insights: Adoption Rates Hospitals Winners versus Nonwinners. Leadership and Quality. Sept. 28, 2006.
of Select Baldrige Award Practices Ann Arbor, MI: Center for Healthcare
and Processes. Ann Arbor, MI: Center Improvement, Watson Health (Truven 23. Chenoweth J, Foster DA, Waibel
for Healthcare Improvement, Watson Health Analytics). August 2008. BC. Best Practices in Board Oversight of
Health (Truven Health Analytics). Quality. The Governance Institute.
October 2012. 15. Foster DA. Risk-Adjusted Mortality June 2006.
Index Methodology. Ann Arbor, MI:
7. Foster DA. Hospital System Center for Healthcare Improvement, 24. Kroch E, Vaughn T, Koepke M,
Membership and Performance. Ann Watson Health (Truven Health Analytics). Roman S, Foster DA, Sinha S, Levey S.
Arbor, MI: Center for Healthcare July 2008. Hospital Boards and Quality Dashboards.
Improvement, Watson Health (Truven J Patient Safety. 2(1): 10-19,
Health Analytics). May 2012. 16. Bonis PA, Pickens GT, Rind DM, March 2006.
Foster DA. Association of a clinical
8. HIMSS Analytics, Watson Health knowledge support system with 25. Health Research and Educational
(Truven Health Analytics). 2012 HIMSS improved patient safety, reduced Trust and Prybil, L. Governance in
Analytics Report: Quality and Safety complications and shorter length of stay High-Performing Organizations: A
Linked to Advanced Information among Medicare beneficiaries in acute Comparative Study of Governing Boards
Technology Enabled Processes. Chicago, care hospitals in the United States. Int J in Not-for-Profit Hospitals. Chicago:
IL: HIMSS Analytics. April 2012. Med Inform. 2008 Nov; 77(11): 745-53. HRET in Partnership with AHA. 2005.
Epub 2008 Jun 19.

26. Cejka Search and Solucient, LLC. 35. DesHarnais SI, et al. Measuring 43. NHSN Standardized Infection Ratio
2005 Hospital CEO Leadership Survey. Hospital Performance: The Development (SIR): A Guide to the SIR, National
and Validation of Risk-Adjusted Indexes Healthcare Safety Network. Centers
27. Griffith JR, Knutzen SR, Alexander of Mortality, Readmissions, and for Disease Control and Prevention,
JA. Structural versus Outcomes Complications. Med Car. 28, no. 12 National Center for Emerging and
Measures in Hospitals: A Comparison (Dec 1990): 1127-1141. Zoonotic Infectious Diseases, Division
of Joint Commission and Medicare of Healthcare Quality Promotion. July
Outcomes Scores in Hospitals. Qual 36. Iezzoni LI, et al. Chronic Conditions 2017. Available at cdc.gov/nhsn/pdfs/
Manag Health Care. 2002; 10(2): 29-38. and Risk of In-Hospital Death. Health ps-analysis-resources/nhsn-sir-guide.
Serv Res. 29, no. 4 (Oct 1994): 435-460. pdf. Last accessed Jan. 23, 2018.
28. Griffith JR, Alexander JA, Jelinek
RC. Measuring Comparative Hospital 37. Elixhauser A, Steiner C, Palmer 44. Foster, D. Model-Based Resource
Performance. Healthc Manag. 2002 L. Clinical Classifications Software Demand Adjustment Methodology.
Jan-Feb; 47(1). (CCS), 2014. US Agency for Healthcare Watson Health (Truven Health Analytics).
Research and Quality. Available via July 2012.
29. See the CMS website at hcup-us.ahrq.gov/toolssoftware/ccs/
cms.gov/Medicare/Quality-Initiatives- ccs.jsp. 45. AHRQ Medicare spending
Patient-Assessment-Instruments/ per beneficiary (MSPB) measure
HospitalQualityInits/ 38. Adams S, Cotton B, Wade C, Kozar R, summary: Cost to Medicare of services
OutcomeMeasures.html. Dipasupil E, Podbielski J, Gill B, Duke J, performed by hospitals and other
Adams P, Holcomb J. Do Not Resuscitate healthcare providers during a MSPB
30. Iezzoni L, Ash A, Shwartz M, Daley (DNR) Status, Not Age, Affects episode. October 2015. Available via
J, Hughes J, Mackiernan Y. Judging Outcomes After Injury: An Evaluation qualitymeasures.ahrq.gov/summaries.
Hospitals by Severity-Adjusted Mortality of 15,227 Consecutive Trauma Patients.
Rates: The Influence of the Severity- J Trauma Acute Care Surg. 2013 May;
Adjusted Method. Am J Public Health. 74(5): 1327–1330.
1996; 86(10): 1379-1387.
39. Iezzoni LI, et al: Identifying
31. Iezzoni L, Shwartz M, Ash A, Hughes Complications of Care Using
J, Daley J, Mackiernan Y. Using severity- Administrative Data. Med Care. 32, no.7
adjusted stroke mortality rates to judge (Jul 1994): 700-715.
hospitals. Int J Qual Health C. 1995;
7(2): 81-94. 40. Iezzoni LI, et al. Using Administrative
Data to Screen Hospitals for High
32. DesHarnais SI, McMahon LF Jr, Complication Rates. Inquiry. 31, no. 1
Wroblewski RT. Measuring Outcomes (Spring 1994): 40-55.
of Hospital Care Using Multiple Risk-
Adjusted Indexes. Health Services 41. Iezzoni LI. Assessing Quality Using
Research, 26, no. 4 (Oct 1991): Administrative Data. Ann Intern Med.
425-445. 127, no. 8 (Oct 1997): 666-674.
33. DesHarnais SI, et al. The Risk- 42. Weingart SN, et al. Use of
Adjusted Mortality Index: A New Administrative Data to Find Substandard
Measure of Hospital Performance. Care: Validation of the Complications
Medical Care. 26, no. 12 (December Screening Program. Med Care. 38, no. 8
1988): 1129-1148. (Aug 2000): 796-806.
34. DesHarnais SI, et al. Risk-Adjusted

Quality Outcome Measures: Indexes
for Benchmarking Rates of Mortality,
Complications, and Readmissions. Qual
Manag Health Care. 5 (Winter 1997):
80-87.
71
About IBM Watson Health™ IBM, the IBM logo, ibm.com, and
Each day, professionals make powerful Watson Health are trademarks of
progress toward a healthier future. At International Business Machines
IBM Watson Health, we help remove Corp., registered in many jurisdictions
obstacles, optimize their efforts, and worldwide. Other product and service
For more information reveal powerful new insights so they names might be trademarks of IBM or
can transform health for the people other companies.
Visit 100tophospitals.com, call they serve. Working across the
800-525-9083 option 4, or send an landscape, from payers and providers to A current list of IBM trademarks is
government and life sciences, we bring available on the web at “Copyright and
email to 100tophospitals@us.ibm.com. together deep health expertise, proven trademark information” at:
innovation, and the power of artificial ibm.com/legal/copytrade.shtml.
intelligence to enable our clients
to uncover, connect, and act on the This document is current as of the initial
insights that advance their work — and date of publication and may be changed
change the world. by IBM at any time. Not all offerings
are available in every country in which
© Copyright IBM Corporation 2019 IBM operates.
IBM Corporation
Software Group The information in this document is
Route 100 provided “as is” without any warranty,
Somers, NY 10589 express or implied, including without
ibm.com/watsonhealth any warranties of merchantability,
800-525-9083 fitness for a particular purpose and
any warranty or condition of non-
Produced in the United States of infringement.
America February 2019
IBM products are warranted
according to the terms and conditions
of the agreements under which they
are provided.
TOP 0219

Watson Health 100 Top Hospitals Study 2019: 26th Edition - March 4, 2019

Uploaded by

Document Information

Original Title

Copyright

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Watson Health 100 Top Hospitals Study 2019: 26th Edition - March 4, 2019

Uploaded by

Copyright:

Watson Health

100 Top Hospitals Study

26th edition | March 4, 2019

Watson Health 100 Top Hospitals Study, 2019; 26th edition

Printed and bound in the United States of America.

Illuminating achievement for a

Providing these measures of successful

Hospitals do not apply for our 100 Top Hospitals

Delivering a transparent assessment

Our national balanced scorecard, based on Norton

In addition, to support consideration of different

4 IBM Watson Health

The 100 Top Hospitals research is one of three

6 IBM Watson Health

The Watson Health 100 Top Hospitals® program

Major teaching hospitals*

8 IBM Watson Health

10 IBM Watson Health

The 2019 Everest Award winners

2019 Everest Award winners

Transparency may present hospital boards and

14 IBM Watson Health

Residency program information, used in classifying

For full details, including calculation and scoring

16 IBM Watson Health

Separately, for every hospital in the study, we

As our final step, we find those hospitals that are

Year after year, the public data we have gathered

20 IBM Watson Health

Patients rated 100 Top Hospitals higher than

22 IBM Watson Health

Clinical Inpatient Mortality Index 1

Table 3. Teaching hospital performance comparisons

24 IBM Watson Health

Table 5. Medium community hospital performance comparisons

US map and states by region This analysis allows us to observe geographic

26 IBM Watson Health

28 IBM Watson Health

FL Wisconsin Wisconsin Oklahoma Oklahoma Washington Washington

Notably, only one metric, 30-day readmissions,

Risk-adjusted inpatient mortality index 83 3.1% 2,374 88.1% 238 8.8%

However, over the years we studied (2013 through

30 IBM Watson Health

Table 9. National Performance Comparisons (All Classes)

Performance measure Winning Nonwinning Difference Percent How benchmark hospitals

Table 10. National performance comparisons (all classes) — information-only metrics

Performance measure Winning Nonwinning Difference Percent How benchmark hospitals

32 IBM Watson Health

Table 11. National performance comparisons — critical access hospitals

Mortality Index¹ 0.44 1.00 -0.6 -56.1% lower mortality

34 IBM Watson Health

36 IBM Watson Health

Note: Again this year, we are publishing results for

38 IBM Watson Health

We adjust inpatient expense, as reported on the Patient experience

40 IBM Watson Health

Risk-adjusted inpatient mortality index

The reference value for this index is

The reference value for this index is

42 IBM Watson Health

We use the observed, normalized

Severity-adjusted length of stay

POA coding is used in the risk model

44 IBM Watson Health

We calculate the unweighted mean of

Case mix- and wage-adjusted inpatient expense per discharge

* See blog.eoscu.com/blog/what-is-the-national-healthcare-safety-network for more information.