Quality assurance in cardiac surgery

D. M. THYS, M.D.

General principles
Quality assurance (QA) programs utilizing total quality management (TQM) concepts are widely used to improve medical services [1, 2, 3, 4, 5, 6, 7]. Quality management employs three basic, closely related activities: quality planning, quality control and quality improvement. Quality planning involves developing definitions of quality as applied to the customer, designing products and services to meet customer needs, and designing processes capable of producing these products and services. Quality control involves developing and applying methods for assuring that processes work as they are designed. Quality improvement focuses on improving the level of performance of key processes. Continuous quality improvement maintains periodic reviews of the QA program to meet changing customer needs and improve the organizations efficiency. Any discussion of quality assurance must, therefore, by necessity begin with a definition of quality. Indeed, it would be impossible to review quality assurance and quality improvement without first defining what one tries to assess or improve. Defining quality, however, is not easy and it has different connotations for different segments of the health care universe (Fig. 1). In this review, several quality assurance initiatives in cardiac surgery will be discussed. They will include large governmental initiatives, as well, as local attempts to assess and improve the quality of cardiac surgery and anesthesia.

The New York state initiative

The program
In 1989, the Department of Health of New York State began a program to assess the quality of cardiac surgery in New York State. All hospitals providing
Correspondence to: Department of Anesthesiology, St. Lukes-Roosevelt Hospital Center, Columbia University, IIII Amsterdam ave, New York, N.Y., 10025.

cardiac surgery were invited to prospectively submit clinical data on all cardiac surgical patients, on a voluntary basis. The system is known as the Cardiac Surgery Reporting System (CSRS). The main purposes of the registry are: a- to provide information to hospitals that will assist them in assessing the quality of cardiac surgical care at their own institution and allow to make improvements, where appropriate, b- to assist the Department of Health in its quality improvement activities, and c- to give consumers information that allows them to make an informed selection of their providers of cardiac surgery. The CSRS has predominantly focused on coronary artery bypass surgery.

Public Patient Health Insurers

QUALITY
Health Care organization Health Profession
Fig. 1. - Quality has different meanings for the various sectors of health care.

Accreditation Organization

For each cardiac surgical patient, the hospital submits a 2 page questionnaire (Fig. 2a & 2b) to the Department of Health with information on the patients preoperative condition, procedural information including the operating surgeons identity, and the patients outcome. Using this information, the CSRS has developed statistical models that determine the relation between preoperative risk factors and morbidity and mortality, and predict the probability of adverse outcomes given the presence of various risk factors. The CSRS also compares the outcomes of the participating hospitals and surgeons by comparing predicted mortality rates with actual mortality. All hospitals that perform CABG surgery in New York State have elected to participate in the program. They receive data from CSRS on a regular basis that outline their actual and risk-adjusted mortality rates. Each hospital also receives a list of preoperative risk factors for CABG surgery that are significantly related to inpatient mortality. For a given patient, surgeons can check which risk factors are present and calculate a predicted probability of inhospital mortality on the basis of the New York State experience. CSRS also provides hospitals with surgeon-specific information such as the number of CABG procedures, the number of deaths, crude mortality rate, and riskadjusted mortality rate.

Fig. 2a. - The New York State Department of Health Cardiac Surgery Reporting form.

Fig. 2b. - The New York State Department of Health Cardiac Surgery Reporting form.

General effects of the program

Several reports have described the changes in outcomes of CABG surgery performed in New York State before and after implementation of the CSRS program. In 1994, Hannan et al. reported that for the 57,187 patients undergoing isolated CABG surgery in N.Y. between 1989 and 1992, the actual mortality rate decreased from 3.52% in 1989 to 2.78% in 1992 [8]. Risk-adjusted mortality decreased from 4.17% in 1989 to 2.45% in 1992. The most recent figures indicate that for the 19,283 patients who underwent CABG in 1995, both the actual mortality rate and risk-adjusted mortality rates were identical at 2.52%. It has been hypothesized that the reductions in actual and risk-adjusted mortalities in New York State were, at least in part, due to the implementation of this program. Comparable longitudinal risk-adjusted data have, however, not been published for other populations making it difficult to assess whether improved outcomes in New York State are different than general trends.

Lay press releases

In 1990, the Department of Health released the mortality data per hospital to the lay press for the first time. Since then, all the major newspapers in New York State have published the mortality figures for CABG surgery, per hospital on an annual basis. In a report in the New England Journal of Medicine, the current and past commissioners of the Department of Health have opined that the program produces reliable and valid measures of quality [9]. They believe that cardiac surgery programs have used this information to improve the outcomes of their patients and that the public release of the data has galvanized physicians and hospitals to seize these opportunities to improve.

Surgeon-specific data
While the CSRS reporting system includes information on individual surgeons, initially these data were not released to the lay press. In 1990, the newspaper Newsday sued the Department of Health under the states Freedom of Information law to gain access to the results of individual surgeons. The department lost the law suit and the data were released to the newspaper which published them in December 1991. Not unexpectedly, physicians reacted angrily and asked that the data be blinded as to the individual surgeons. The Department of Health refused the blinding, but agreed to average the data over 3 years and to only release data from surgeons who had performed at least 200 operations in the same hospital for that time period. This averaging was highly justified since it had been shown that changes in ranking were extensive over short periods of time [10]. In one year, 46 % of the surgeons had moved from one half of the ranked list to the other half. One important aspect of surgeon-specific information was related to the role of surgeon volume on mortality. Hannan et al. [11] observed that over a 4year period (1989-1992), low volume surgeons (<50 operations/year) experienced a greater decrease in risk-adjusted mortality (60%) than highvolume surgeons (>150 operations/year). The percentage of patients undergoing surgery by low-volume surgeons decreased from 7.6% in 1989 to 5.7% in 1992.

Table I
(Modified from reference [11])

Year 1989 1990 1991 1992

Risk-adjusted mortality Low-volume surgeons 7.94* 5.72* 3.56 3.20 High-volume surgeons 3.57 3.03 2.57 2.36

The authors attributed this dramatic improvement in results for lowvolume surgeons on the out of state exodus of low-volume surgeons with high risk-adjusted mortality, the better performance of surgeons who were new to the system, and the performance of surgeons who were not consistently lowvolume surgeons.

Hospital-specific responses
The St. Peters Hospital in Albany, N.Y., is a 447-bed community hospital that began a cardiac surgical program in 1989 [12]. In 1991, St. Peters had a high actual mortality (4.6%), but low expected mortality (2.1%) resulting in a risk-adjusted mortality of 6.6% while the New York State average was 3.08%. The surgeons initial response was an intense mortality and morbidity review. Repeated reviews, however, all yielded the same conclusion: almost all the deaths had occurred in high-risk patients and no obvious errors could be found in their care. Having agreed that this was not just a surgeons problem, a comprehensive review of the database was undertaken. It was found that the mortality was largely concentrated in emergency cases, which represented 10% of the surgical volume (mortality of 26% versus statewide average of 7.7%). The mortality for elective cases was found to be 1.2%, slightly below the statewide average. Further analysis revealed that death in emergency patients were concentrated in those patients who had a myocardial infarction within 6 hours before the operation, or who were in shock or hemodynamically unstable before entering the operating room. As a result of the review several changes were made to the cardiac surgical care of the emergency patients such as more frequent use of preoperative intra-aortic balloon pumping, standardized cardioplegia administration, and others. Since then, overall mortality has decreased to 1.5% and has consistently remained below the expected mortality rate. Other hospital-specific consequences of the publication of the outcomes have included the suspension of a cardiac surgery program until a new chief of cardiac surgery could be recruited, retirement from cardiac surgery of surgeons with unsatisfactory performance, reassignment of patients to surgeons with better performance and limitations of surgical activities to specific areas of cardiothoracic surgery (e.g. pediatric or thoracic surgery).

Other issues
Predictive value

For risk-adjusted mortality to be a valid indicator of quality of care, the mathematical model must be able to quantify accurately baseline differences in case mix amongst health care providers. Green et al. have found that the ability of the CSRS model to do this was limited [10]. Predicted mortality rates assigned to surgeons by the model explained only a small portion of the variance in mortality and for hospitals the predictive value was negligible. The power of the model to predict outcomes for individual patients was also low. It has been argued, however, that the CSRS main function was not to predict outcomes, but to help physicians and hospitals to improve their performance [9].
Limitations of the system

It has been claimed that to avoid poor ranking, some surgeons and hospitals have declined to operate on the highest-risk patients. Analysis of the data for the period of 1990 to 1992, however, indicates that the number of high risk patients being operated on in New York State had increased by 73% [9]. In addition, the data have consistently demonstrated that some hospitals with the highest-risk patients had the lowest risk-adjusted mortality in the state. Omouigi et al., have suggested that some of the reduction in mortality over time was attributed to migration of the highest-risk patients out of state, rather than to improvements in care [13]. The authors compared mortality figures for CABG at the Cleveland Clinic before and after 1989 (Fig. 3). They noted that before 1989, there was no difference in mortality among patients referred to the Cleveland Clinic from different parts of the world. After 1989, however, the mortality for New York referrals was significantly higher than for other referral groups. In addition, the expected mortality for patients referred from New York State was significantly higher than for referrals from other states and than for patients operated upon in New York State. The authors conclusions have been disputed and the referral patterns have been attributed to long-standing referral patterns [9]. Another limitation of the CSRS system is that it is predominantly confined to outcomes data with little regard for process information. As a result, it is impossible to undertake the systematic exploration of factors that are responsible for the differences in outcomes for the various providers. A particularly striking comment, is the absence of any information on anesthesia.
The role of anesthesia

The CSRS database is not designed to include information on anesthesia. It is, therefore, impossible to evaluate whether anesthesia factors play any role in the outcomes of cardiac surgery at various institutions. In an attempt to partially address this issue, Reich et al. [14] have studied hemodynamic

variables in 2149 patients undergoing coronary artery bypass surgery at 2 New York State institutions and have assessed the association between hemodynamics and postoperative outcomes. Their findings have demonstrated the prognostic significance of intraoperative hemodynamic abnormalities, including data from pulmonary artery catheterization. The study did not allow one to determine, however, whether interventions to control these variables

MORTALITY
Referrals to Cleveland Clinic
6 5 4 3 2 1 0
NY Ohio Other states Other countries

1980-88

1989-93

Fig. 3. - Mortality of patients referred to the Cleveland Clinic before and after 1989. (Modified from reference [13]).

EXPECTED MORTALITY
7 6 5 4 3 2 1 0 1989 1990 1991 1992 1993
NY NY Referrals US Referrals

Fig. 4. - Expected mortality for CABG patients operated in New York State and for patients referred to the Cleveland Clinic. (Modified from reference [13]).

10

would improve outcome.

The Northern New England initiative

The Northern New England Cardiovascular Disease Study group is a voluntary research consortium that is composed of clinicians, scientists, and hospital administrators [15]. They represent 5 hospitals in Maine, New Hampshire, and Vermont where CABG is performed. In 1990, they initiated a 3-part intervention in an attempt to reduce hospital mortality associated with cardiac surgery. The intervention consisted of: 1) feedback of outcome data, 2) training in continuous quality improvement techniques, and 3) site visits to other medical centers. During the post-intervention outcome, they observed 6488 consecutive cases of CABG surgery and noted 74 fewer deaths than expected (Fig. 5). All participants reported changes in the technical aspects of care that resulted as a consequence of the intervention. The changes were substantive and temporally associated with the reduction in mortality rate.

7 6 5 4 3 2 1 0 Expected Observed
n = 6,638

MORTALITY
Pre-intervention Post-intervention
n = 6,488

Intervention
n = 1,969

Fig. 5. - Expected and observed mortality for CABG surgery in Northern New England. ( M o d i f i e d from reference [15]).

Unfortunately, the design of the study did not allow one to isolate the specific causal factors leading to the improvement of care.

Quality assurance cardiac anesthesia

Quality assurance in anesthesia
In the United States, the major organization that is responsible for the evaluation of quality assurance activities is the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO). It publishes standards of

7/87 10/87 1/88 4/88 7/88 10/88 1/89 4/89 7/89 10/89 1/90 4/90 7/90 10/90 1/91 4/91 7/91 10/91 1/92 4/92 7/92 10/92 1/93 4/93

11

care on a yearly basis and every 3-years evaluates compliance with the standards during site visits at the healthcare organizations. In 1990, JCAHO published a list of 13 anesthesia indicators that were considered useful for internal hospital quality assurance activities. Table II
JCAHO Anesthesia Care Indicators 1. CNS complication (within 24 hrs of anesthesia) 2. Peripheral neurologic (within 48 hrs of anesthesia) 3. Cardiac arrest (within 24 hrs of anesthesia) 4. Acute MI (within 48 hrs of anesthesia) 5. Respiratory arrest (within 24 hrs of anesthesia) 6. Unplanned admission 7. Unplanned ICU admission 8. Mortality (within 48 hrs of anesthesia) 9. Pulmonary edema (within 24 hrs of anesthesia) 10. Aspiration pneumonitis (within 48 hrs of anesthesia) 11. Postural headache (within 4 days of anesthesia) 12. Dental injury 13. Ocular injury

The occurrence of these complications must be tabulated and reported to the hospital Quality Improvement committee on a monthly basis. The JCAHO expects that these indicators be tracked for individual anesthesiologists and that appropriate actions be taken if any particular anesthesiologist surfaces as an outlier. The problem with this approach is that, fortunately, most of these events occur infrequently and that, as a result, it s difficult to find deviations for individual anesthesiologists that are statistically significant. Another limitation of the anesthesia indicators is that they provide no information on the patients perception of quality. While there are numerous problems in the assessment of quality from the patients perspective, it appears nonetheless essential to try to incorporate such a pespective [16]. In 1995, the Department of Anesthesiology of St Lukes Roosevelt Hospital Center began a patient survey study to assess the patients satisfaction with several aspects of their anesthetic care. On a monthly basis, approximately 400 patients (20% of total) are sent a survey card inquiring about their perception of the anesthesiologists time commitment, concern with patients worries, and courtesy. A question is also included concerning pain management as it is wellestablished that patients are most concerned and often dissatisfied with postoperative pain management (Fig. 6). To date, the response rate has been approximately 25 % and the overall

12

satisfaction scores for the department have been high (Fig. 7). The most interesting aspect of this effort, however, are the specific comments that patients convey. They provide great insight in some of the process factors of care and allow one to intervene once specific problems are identified. The other benefit is

Your anesthesiology care is important to us

Wont you please take a few moments of your time to complete this survey and let us know weve done? Return postage has been provided. (Su Cuidado Anesthesico es importante para nosotros Por favor tome el tiempo para Ilenar este cuestionario. El jiro postal esta incluido). On a scale from 1 - 10, please rate the following. (Del 1 al 10 por favor indique lo siguiente). 1) Amount of time your anesthesiologist spent with you (El tiempo su anesthesiologo compartio con usted). (Unsatisfactory) 1 2 3 4 5 6 7 8 9 10 (Excellent) (No Satisfecho) (Excellente) 2) Anesthesiologists concern for your questions/worries (El concierno acerca sus preguntas/preocupaciones). (Unsatisfactory) 1 2 3 4 5 6 7 8 9 10 (Excellent) 3) Courtesy and attentiveness of your anesthesiologist (Cortesia y attencion de su anesthesiologo). (Unsatisfactory) 1 2 3 4 5 6 7 8 9 10 (Excellent) 4) The effectiveness of your pain management after surgery (El manejo de su dolor depues de la cirujia). (Unsatisfactory) 1 2 3 4 5 6 7 8 9 10 (Excellent) 5) Your overall satisfaction with your anesthesia care (Su satisfaction, en si, del cuidado anesthesico). (Unsatisfactory) 1 2 3 4 5 6 7 8 9 10 (Excellent) Comments about your anesthesia experience (good or bad) ____________________________________ ___________________________________________________________________________________ Patient Name (Optional) ___________________ Phone Number (Optional) ____________________ If you would like to speak with someone in the Department, please call us at (212)523-2865.

Fig. 6. - Survey instrument used by the Department of Anesthesiology of St. Lukes-Roosevelt Hospital Center to assess patient satisfaction with Anesthesia services.
Department of Anesthesiology St. Luke's-Roosevelt Hospital Center

Patient Satisfaction 1997 1 Qrtr 2 Qrtr 3 Qrtr

4 Qrtr

10 8 6 4 2 0
Time Concern Courtesy Pain Mgmt Overall
Fig. 7. - Quarterly patient satisfaction scores for 1997.

13

Department of Anesthesiology St. Luke's-Roosevelt Hospital Center

Overall Satisfaction
10 8 6 4 2 0 1 3 5 7 9

(> 25 Records/Anesthesiologist)

Mean Score
2 SD

Individual Anesthesiologists

11 13 15 17 19 21 23 25 27 29 31

Fig. 8. - Overall satisfaction scores for individual anesthesiologists for whom more than 25 surveys were returned. The dark bar represents the anesthesiologist who deviated more than 2 standard deviations from the mean.

that with an increasing database, it becomes possible to identify anesthesiologists who are statistical outliers (Fig. 8). Such individuals are given feedback and counseled on techniques to improve their interactions with patients.

Special considerations in cardiac anesthesia

One of the aspects of cardiac anesthesia that is most amenable to quality assurance, is intraoperative echocardiography [17]. When applying TQM concepts to perioperative TEE, the terms products and customers need to be employed in their broadest meaning. The major product of perioperative TEE is information, which is provided to customers. These customers can include other anesthesiologists, surgeons, cardiologists, other physicians, the patient or the manufacturers of echocardiographic equipment. Which process of a TEE program is the current key process that may benefit from improvement will vary from program to program. The following outline of processes was developed to assist in focusing on possible key processes within echocardiography programs. Table III
Key processes of intraoperative echocardiography 1. Indications for performing TEE 2. Technical aspects of performing and recording the examination 3. Application of examination findings to physiologic condition 4. Documentation 5. Equipment 6. Professional communications

14

7. Education 8. Billing

The application of Quality Management to an echocardiographic service need not be restricted to the areas mentioned above. Periodic updating of the QA program as the TEE service evolves will facilitate meeting the goals originally set forth and defining new goals to meet clinical needs.

Conclusions
Quality assurance can focus on many aspects of cardiac care. Many programs and studies have, however, limited their emphasis on mortality and morbidity data. It is only through a better understanding of the processes that lead to untoward outcomes that improvements can be made. Unfortunately, the processes associated with cardiac anesthesia have received very little attention in the overall assessment of cardiac care. Anesthesiologists have begun to assume greater responsibilities in the determination of cardiac care through their involvement in perioperative echocardiography. A program for quality assurance in cardiac anesthesia should, therefore, include the key process of perioperative echocardiography.

REFERENCES
[1] Berwick D.M., Godfrey A.B., Roessner J. - Curing Health Care: New Strategies for Quality Improvement. A Report on the National Demonstration Project on Quality Improvement in Health Care. Josie-Bass Publishers, San Francisco, 1990. [2] Berwick D.M. - Continuous improvement as an ideal in health care. N. Engl. J. Med., 1989, 320, 53-56. [3] Deming W.E. - Out of the Crisis. Cambridge, MA: Massachusetts Institute of Technology, 1986. [4] Feigenbaum A.V. - Total Quality Control. New York, McGraw Hill, 1983. [5] Kritchevsky S.B., Simmons B.P. - Continuous quality improvement. Concepts and applications for physician care. JAMA, 1991, 266, 1817-1823. [6] Laffel G., Blumenthal D. - The case for using industrial quality management science in health care organizations. JAMA, 1989, 272, 2869-2873. [7] Wenzel R.P. - Beyond total quality management. Clinical performance and quality health care, 1993, 1, 43-48. [8] Hannan E.L. et al. - Improving outcomes of coronary artery bypass surgery in New York State. JAMA, 1994, 271, 761-766. [9] Chassin M. et al. - Benefits and hazards of reporting medical outcomes publicly. N. Engl. J. Med., 1996, 334, 394-398. [10] Green J. et al. - Report cards on cardiac surgeons: Assessing New York States

15

approach. N. Engl. J. Med., 1995, 332, 1229-1232. [11] Hannan E.L. et al. - The decline in coronary artery bypass graft surgery mortality in New York State. The role of surgeon volume. JAMA, 1995, 273, 209-213. [12] Dziuban S.W. et al. - How a New York cardiac surgery program uses outcomes data. Ann. Thorac. Surg., 1994, 58, 1871-1876. [13] Omouigi N.A. et al. - Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes. Circulation, 1996, 93, 27-33. [14] Reich D.L. et al. - Intraoperative hemodynamic predictors of mortality, stroke and myocardial infarction following coronary artery bypass surgery. (In press). [15] OConnor G.T. et al. - A regional intervention to improve hospital mortality associated with coronary artery bypass graft surgery. JAMA, 1996, 275, 841-846. [16] Cleary P.D. et al. - Health Care Quality: Incorporating consumers perspectives. JAMA 1997, 278, 1608-1612. [17] Thys D.M. et al. - Practice guidelines for perioperative transesophageal echocardiography. Anesthesiology, 1996, 84, 986-1006.

16