Accepted Manuscript

Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and

systemic inflammatory response syndrome (SIRS) criteria for predicting mortality
in hospitalized patients with suspected infection: A meta-analysis of observational
studies

Souvik Maitra, MD, DNB, EDIC, Assistant Professor, Anirban Som, MD, Attending
Consultant, Sulagna Bhattacharjee, MD, DNB, Senior Resident

PII: S1198-743X(18)30294-5
DOI: 10.1016/j.cmi.2018.03.032
Reference: CMI 1263

To appear in: Clinical Microbiology and Infection

Received Date: 20 January 2018

Revised Date: 13 March 2018
Accepted Date: 20 March 2018

Please cite this article as: Maitra S, Som A, Bhattacharjee S, Accuracy of quick Sequential Organ
Failure Assessment (qSOFA) score and systemic inflammatory response syndrome (SIRS) criteria for
predicting mortality in hospitalized patients with suspected infection: A meta-analysis of observational
studies, Clinical Microbiology and Infection (2018), doi: 10.1016/j.cmi.2018.03.032.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to
our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and all
legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and systemic

inflammatory response syndrome (SIRS) criteria for predicting mortality in hospitalized patients

with suspected infection: A meta-analysis of observational studies

Subtitle: Predictive accuracy of qSOFA: A meta-analysis

PT
Souvik Maitra MD, DNB, EDIC1, Anirban Som MD2, Sulagna Bhattacharjee MD, DNB3
1 Assistant Professor, Department of Anaesthesiology, Pain Medicine & Critical Care, All India
Institute of Medical Sciences, New Delhi, India

RI
2 Attending Consultant, Department of Anaesthesiology, MAMBS, Max Super Specialty
Hospital, Saket, New Delhi, India

SC
3 Senior Resident, Department of Anaesthesiology, Pain Medicine & Critical Care, All India
Institute of Medical Sciences, New Delhi, India

U
Correspondence to:

Dr Anirban Som, MD
AN
Attending Consultant,
Department of Anaesthesiology, MAMBS,
Max Super Specialty Hospital
M

Press Enclave Road, Saket

New Delhi- 110017, India
Mobile: +91- 8010089986
D

Email: anirbansm@gmail.com
Key Words: Sepsis, qSOFA, SIRS, prognosis, mortality
TE
C EP
AC

1
 ACCEPTED MANUSCRIPT

Abstract
Objective: To identify sensitivity, specificity and predictive accuracy of quick sequential organ

failure assessment (qSOFA) score and systemic inflammatory response syndrome (SIRS) criteria

to predict in-hospital mortality in hospitalized patients with suspected infection.

PT
Methods: This meta-analysis followed MOOSE consensus statement for conducting and

reporting the results of systematic review. PubMed & EMBASE were searched for the

RI
observational studies which reported predictive utility of qSOFA score for predicting mortality in

SC
patients with suspected or proven infection with the following search words: ‘qSOFA’, ‘q-SOFA’,

‘quick- SOFA’, ‘Quick Sequential Organ Failure Assessment’, ‘quick SOFA’. Sensitivity,

U
specificity, area under receiver operating characteristic curves (ROC) with 95% confidence

interval of qSOFA and SIRS criteria for predicting in-hospital mortality was collected for each
AN
study and a 2x2 table was created for each study.

Results: Data of 406802 patients from 45 observational studies have been included in this meta-
M

analysis. Pooled sensitivity (95% CI) and specificity (95% CI) of qSOFA≥2 for predicting
D

mortality in patients who are not in intensive care unit (ICU) is 0.48(0.41- 0.55) and 0.83(0.78-
TE

0.87) respectively. Pooled sensitivity (95% CI) of qSOFA ≥2 for predicting mortality in patients

(both ICU and non- ICU setting) with suspected infection is 0.56(0.47- 0.65) and pooled
EP

specificity (95% CI) is 0.78(0.71-0.83).

Conclusion: qSOFA has been found to be a poorly sensitive predictive marker for in-hospital
C

mortality in hospitalized patients with suspected infection. It is reasonable to recommend

AC

developing another scoring system with higher sensitivity to identify high-risk patients with

infection.

2
 ACCEPTED MANUSCRIPT

Sepsis and its consequences are a serious burden to the health care and society1. A meta-analysis

published in 2016 found that hospital mortality was 17% and 26% for patients with sepsis and

severe sepsis respectively during the years 1997- 20152. It is well known that early recognition

and timely therapy are the keys to successful outcome in patients with sepsis and septic shock3.

PT
A consensus definition of sepsis was developed in 19924 and again re-examined in 20015.

However, a need for a new definition of sepsis is well recognized, as the understanding of

RI
pathophysiology of sepsis has changed6. The third international definition of sepsis (Sepsis-3)

SC
coined the term quick sequential organ failure assessment (qSOFA) incorporating altered

mentation, systolic blood pressure of 100 mm Hg or less, and respiratory rate of 22/min or

U
greater, which provides simple bedside criteria to identify adult patients with suspected infection

who are likely to have poor outcomes7. This definition also states that patients who are out of the
AN
hospital, in the hospital ward or in the emergency department, tend to have a poor outcome when

any two of the above criteria are present in them. Seymour et al8 reported that presence of any
M

two of the qSOFA criteria predicts in-hospital mortality with an area under the receiver operating
D

characteristics curve (AUROC) of 0.81. This predictive value was similar to the full sequential
TE

organ failure assessment score (SOFA) in the out side intensive care unit (ICU) setting. On the

other hand, the systemic inflammatory response syndrome (SIRS) criteria were inferior to the
EP

SOFA score for predicting mortality in ICU patients with infection.8 Criteria in the qSOFA model

are simple, easy to use and could be used for making the practicing physician alert about the
C

possible poor outcome in such patients. However, to be used as a screening tool for identifying
AC

patients who are at risk of death from infection, qSOFA is expected to be highly sensitive for

detecting ‘at risk’ patients.

Although both the derivation and validation cohorts of qSOFA criteria were from the United

States, since publication of the Sepsis-3 definition, a large number of studies from different

regions of the world have evaluated this score for prognostication in patients with suspected

3
 ACCEPTED MANUSCRIPT

infection. Results of the individual studies reported a wide range of predictive accuracy of

qSOFA.

In this systematic review and meta-analysis, we have assessed the pooled predictive value of

qSOFA for in-hospital mortality in patients with suspected infection.

PT
Methods

This meta-analysis followed the consensus statement of MOOSE (Meta-analysis of observational

RI
studies in epidemiology) group in conducting and reporting of its results9.

SC
Data source

Two authors (AS & SM) independently searched PubMed EMBASE databases for the studies

U
which reported predictive utility of qSOFA score for predicting mortality in patients with

suspected or proven infection. Following search words were used to identify potentially eligible
AN
studies: ‘qSOFA’, ‘qSOFA sepsis’, ‘q-SOFA’, ‘quick- SOFA’, ‘Quick Sequential Organ Failure

Assessment’, ‘quickSOFA’. The details of Medline search strategy have been provided in the
M

online supplementary material. The initial search was conducted on 30th October 2017 and a
D

subsequent search was conducted on 6th March 2018. We included all studies available till that
TE

date. Observational studies where qSOFA was reported in a manner that it was possible to create

a 2x2 table have been included in this meta-analysis. Initially a pooled analysis has been done
EP

incorporating all the studies where predictive value of qSOFA has been reported. After that,

another pooled analysis has been done incorporating studies, which reported both qSOFA and
C

SIRS criteria for predicting mortality in such patients. References of the studies were also
AC

manually searched for potentially eligible studies. We did not search for the ongoing trials and

authors were not contacted for unpublished data. Studies for which full text was not available in

English have not been included in this review.

4
 ACCEPTED MANUSCRIPT

Study eligibility

All prospective and retrospective studies that assessed the predictive accuracy of qSOFA for

predicting in-hospital or 30-day or 28-day mortality in adult patients having suspected or proven

infection and reported enough information to formulate a 2x2 contingency table for further

PT
analysis have been included in this meta-analysis.

Study Selection

RI
Two authors (SM and AS) independently read the abstract of the potentially eligible trials. Both

SC
of them selected eligible trials as per inclusion criteria. When a difference in opinion was

encountered, the opinion of the third author (SB) was considered to be final.

U
Data Collection

We collected the required data from the full text of the trials. Initially all data were tabulated in
AN
Microsoft Excel™ spreadsheets (Microsoft Corp., Redmond, WA). AS and SB extracted data

from the eligible studies and SM crosschecked those data independently. Statistical analyses were
M

independently performed by AS and SM.

D

Data Items
TE

The following data were collected from each study: name of the first author, year of publication,

total number of patients studied, patient population, place of study (emergency department or
EP

intensive care unit), sample size, time point of evaluation, cut off points for qSOFA and SIRS,

sensitivity, specificity, area under receiver operating characteristic curves (AUROC) with 95%
C

confidence interval (CI) for predicting in-hospital mortality. Where in-hospital mortality has not
AC

been reported, 30- day or 28-day mortality has been noted. We also collected data of qSOFA in

non- ICU patients from the studies where it has been reported. In the context of this meta-analysis

about predictive accuracy, sensitivity was defined as the percentage of patients with a positive

index test who died during hospitalization or within 28 or 30 days; while specificity was used to

describe the percentage of patients with a negative index test who survived the period of

hospitalization or after 28 or 30 days.

5
 ACCEPTED MANUSCRIPT

Quality assessment

Methodological qualities of the included studies have been systematically evaluated by QUADAS

2 guidelines10 and summarized in RevMan 5.3 (Review Manager (RevMan) [Computer program].

Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).

PT
Statistical analysis

A 2x2 table with the four core values, i.e., the numbers of true positives, false positives, true

RI
negatives and false negatives was constructed for all the included studies. Studies, which did not

SC
report these four values, relevant values were calculated from the reported sensitivity, specificity,

and predictive values in Revman 5.3 (Review Manager (RevMan) [Computer program]. Version

U
5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014). Initially a

bivariate random effects regression model for diagnostic meta-analysis was used to obtain
AN
weighted summary estimates of the sensitivity and specificity of qSOFA and SIRS criteria. We

calculated summary estimates of performance statistics, which included sensitivity, specificity,

M

positive and negative likelihood ratios with the midas or metandi command in Stata (STATA SE
D

12.0, STATA Corp, College Station, TX, USA). Pooled diagnostic odds ratio (OR) was also
TE

calculated by random effects model by metandi command. A subgroup analysis was planned to

directly compare the predictive accuracies of qSOFA and SIRS from studies that reported the
EP

required data related to both these criteria.

Quantification of heterogeneity11 was not done by I2 value, since this parameter is not useful in
C

analysis of accuracy studies. Therefore, reasons for heterogeneity among different studies were
AC

searched for using regression models with following covariates: sample size, clinical setting,

prevalence of disease, age of the patients, cut-off value of the index test. Publication bias was

tested by visual inspection of funnel plot and Deeks’ regression model12, although they are known

to have low power.

6
 ACCEPTED MANUSCRIPT

Results

Search result and characteristics of the included studies

PubMed & EMBASE search as per predefined search words revealed 256 articles and 250 articles

respectively, and references from the relevant articles revealed 57 articles. After removal of

PT
duplications, 112 unique articles were assessed from abstract and full text for possible inclusion

in this meta-analysis. Finally, 56 articles fulfilled our inclusion criteria; whereas a total of 11

RI
articles13-23 were excluded, the details of which have been shown in the flow diagram illustrating

different stages of database searching and study selection as per PRISMA24 (Preferred Reporting

SC
Items for Systematic Reviews and Meta-analyses) checklist in figure 1.

U
Methodological qualities of the included studies have been depicted in e figure 1. As can be seen

from the diagram, most studies have a high risk of bias in patient selection because of the
AN
retrospective nature of their analyses, while an uncertain element of bias is introduced by the risk

of treatment paradox in all of them. There is no concern regarding the performance or

M

applicability of the index tests, whereas the reference standard, i.e. mortality, is beyond any risk
D

of bias. In contrast, the selected patient populations do raise questions about applicability in a few
TE

studies.

Data of 406,802 patients from 45 observational studies8, 24-67 have been included in this meta-
EP

analysis where predictive ability of qSOFA for in-hospital/ 30-day mortality has been reported.

Primarily we collected data of in-hospital mortality; whereas 30-day or 28-day mortality was
C

considered an alternative to in-hospital mortality where it was not reported.

AC

In 38 studies24-26,28-30,32-34,36-56, 58-61, 64-67

data were available from non-ICU patients regarding

predictive accuracy of qSOFA. Among these, in 13 studies, predictive ability of both qSOFA and

SIRS criteria for in-hospital mortality has been reported. Individual study characteristics have

been provided in e table 1 [online supplement]. All the studies have been published between 2016

and 2018. We considered any study, which had been designed or commenced before the

publication of Sepsis-3 definition7 (February 2016) as a retrospective cohort study. Only four

7
 ACCEPTED MANUSCRIPT

studies included here are prospective in design32,45,59, 64. Reported predictive accuracy of qSOFA

for predicting in-hospital mortality/30-day mortality in the included studies has been depicted in e

Table 2 [online supplement].

Predictive validity of qSOFA in patients not in ICU

PT
Data were available of 199705 patients from 38 studies regarding predictive accuracy of qSOFA

to predict in-hospital mortality. Pooled sensitivity (95% CI) and specificity (95% CI) of

RI
qSOFA≥2 for predicting mortality in non-ICU patients are 0.48(0.41- 0.55) and 0.83(0.78-

SC
0.87) respectively. A summary ROC curve has been provided in figure 2 and AUC with 95% CI

was found to be 0.71(0.67- 0.75). Forests plot for sensitivity and specificity at study level and

U
pooled analysis level has been provided in e figure 2.
AN
A meta-regression analysis did not find any relation between baseline risks of in-hospital

mortality (p=0.05) and sample size (p=0.58). However, mean age of the included patients has

been found to be a significant co-variate for predictive accuracy of qSOFA (I2=99%, p<0.01, e
M

figure 3). When the probability of death from suspected infection is 20%, a qSOFA ≥2 increases
D

possibility of death to 41% [Fagan’s nomogram, e figure 4]. No evidence of publication bias was
TE

found in Deek’s regression model (p=0.76) [e figure 5], although it is important to stress that this

measure has low reliability12.

EP

When patients from both ICU and non- ICU settings are included in the meta-analysis, pooled

sensitivity (95% CI) of qSOFA ≥2 for predicting mortality in patients with suspected infection is
C

0.56(0.47- 0.65) and pooled specificity (95% CI) is 0.78(0.71-0.83). A summary ROC curve
AC

has been provided in e figure 6 and AUC with 95% CI was found to be 0.73(0.69- 0.77).

Predictive validity of qSOFA versus SIRS criteria

Twenty-one studies reported predictive accuracy of both qSOFA and SIRS criteria for predicting

in-hospital mortality. Pooled sensitivity (95% CI) of SIRS ≥2 for predicting mortality in

hospitalized patients with suspected infection is 0.82(0.73- 0.89) and pooled specificity (95%

8
 ACCEPTED MANUSCRIPT

CI) is 0.36(0.24- 0.50) [figure 3]. Diagnostic OR (95% CI) of SIRS is 2.59 (2.09- 3.22) with a

positive LR (95% CI) of 1.30 (1.10- 1.50) for predicting in-hospital mortality. A summary of

predictive accuracy of qSOFA and SIRS has been provided in table 1.

Considering the studies where both qSOFA and SIRS were reported, pooled sensitivity (95% CI)

PT
of qSOFA ≥2 for predicting mortality in hospitalized patients with suspected infection is 0.60

(0.45- 0.73) and pooled specificity (95% CI) is 0.76 (0.64- 0.84) [figure 4]. In all summary ROC

RI
curves we have found a large area of 95% prediction interval, which is a measure of between-

SC
study heterogeneity. So that, the prediction interval can predict the sensitivity and specificity of

future studies.

U
Discussion AN
This meta-analysis of observational studies illustrates that qSOFA is a less sensitive predictive

marker than SIRS criteria for in-hospital mortality in hospitalized patients with suspected
M

infection. However, qSOFA is more specific than SIRS criteria in this regard. The most important

strength of this meta-analysis is its large number of patients’ database from various geographical
D

regions of the world.

TE

Global burden of sepsis is increasing worldwide with estimated load up to 560 cases per 100000

persons per year2. It is also the single most expensive health care condition in the United States
EP

accounting for 5.2% of total health care expenditure69. The Third International Consensus

Definitions Task Force7 defined sepsis as a “life-threatening organ dysfunction due to a

C

dysregulated host response to infection” on the basis of updated knowledge about the
AC

pathophysiology of sepsis. As early diagnosis and management are the keys to successful

outcome in sepsis patients, Third International Consensus Definitions Task Force developed an

easy and reproducible criteria termed qSOFA to identify patients with suspected infection who

would require escalation of therapy and monitoring. Seymour et al8 found that qSOFA had a

predictive accuracy equivalent to that of the full SOFA score for in-hospital mortality in non-

9
 ACCEPTED MANUSCRIPT

ICU patients. However, in patients who are in ICU, full SOFA score had better predictive validity

than qSOFA7.

Interestingly, this meta-analysis has found that qSOFA has a poorer sensitivity for predicting

mortality even in hospitalized patients who are not in ICU. As most of the ICU patients receive

PT
mechanical respiratory support and/or vasopressors, qSOFA is expected to be of little actual

clinical value in this setting. We have found an AUC of summary ROC curve of 0.72 and

RI
included studies in this meta-analysis reported a wide range of AUCROC of 0.55 to 0.82. But

SC
from a clinical point of view, AUCROC is of little value in this scenario, because the aim of

formulating qSOFA was to develop it as a screening tool. As a screening tool for identification of

high-risk patients, it is desirable to have at least a high sensitivity70. We have found that pooled

U
specificity of qSOFA is 0.79 for all patients, which is reasonably good.
AN
This meta-analysis included a large number of patients from different settings and geographical

areas; different hospitals have variable quality of care that would obviously have affected
M

patients’ ultimate outcome. This has been statistically reflected in this meta-analysis in terms of a
D

significant amount of heterogeneity. We have also found that age of the included patients is a
TE

statistically significant reason for heterogeneity.

As qSOFA is a simple criterion and does not include any laboratory investigation, its widespread
EP

use in different settings including emergency departments and pre-hospital areas is feasible.

Although blood lactate is a well-known prognostic marker in patients with sepsis71; benefits of
C

adding blood lactate to the original qSOFA is of questionable value8,33. The current Sepsis-3
AC

definition recommends that qSOFA can be used to identify patients who are at risk of death or

prolonged ICU stay72. However, this meta-analysis did not find any evidence that it can be used

to identify patients who are at risk of in-hospital death, since the overall sensitivity is only 0.54.

Being a predictive tool designed to predict mortality, the score should have a reasonably high

level of sensitivity, which qSOFA evidently lacks. As qSOFA is not a diagnostic criterion to

identify patients with sepsis, its clinical utility is questionable.

10
 ACCEPTED MANUSCRIPT

On the contrary, the SIRS criteria, which have been criticized for a long time, have been found to

be very non-specific for predicting mortality in our analysis42. The SIRS criteria, which were

originally developed in 19924, have been found to be a predictor of mortality in various infectious

and non- infectious conditions such as trauma73 and acute myocardial infraction74. Presence of

PT
SIRS criteria denotes activation of innate immunity system irrespective of its cause and both

infectious and non-infectious triggers can activate immunologic response. As there is a wide

RI
range of activators of the innate immunity system, the SIRS criteria are known to be non-specific

SC
as marker of sepsis.

U
Limitations

The most important limitation of our analysis is the presence of large amount of heterogeneity in
AN
this meta- analysis. Though we have conducted a meta-regression analysis, we could not identify

any factor associated with heterogeneity except for mean age of the patient cohorts. It’s obvious
M

that as we have included a large number of patients from different settings, a significant amount
D

of heterogeneity will always exist. In the absence of individual patient data, it was impossible to
TE

adjust the various pre existing clinical risk factors of death in these patients. Most of the included

studies in this meta-analysis are retrospective reviews of patients’ database and they are not
EP

specifically designed to identify accuracy of qSOFA.

We analyzed predictive accuracy of qSOFA and SIRS criteria from different dataset. A meta-
C

regression analysis could have taken into account of qSOFA and SIRS from same study.
AC

Conclusion

In this meta-analysis of observational studies, qSOFA has been found to be a poorly sensitive

predictive marker for in-hospital mortality in hospitalized patients with suspected infection. As

the original aim of developing qSOFA score was to identify patients who are at risk of death or

11
 ACCEPTED MANUSCRIPT

prolonged ICU stay, it is reasonable to recommend developing another scoring system with

higher sensitivity.

Author Contributions:
Drs Souvik Maitra and Anirban Som had full access to all of the data in the study and take

PT
responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: Souvik Maitra and Anirban Som.
Drafting of the manuscript: Sulagna Bhattacharjee and Souvik Maitra

RI
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Souvik Maitra and Anirban Som
Obtained funding: Not applicable

SC
Administrative, technical, or material support: Literature search was done at BB Dikshit
Library, All India Institute of Medical Sciences, New Delhi, India
Study supervision: Souvik Maitra
Conflict of interests: None

U
Source of support: Nil
AN
Figure legends
M

Figure 1: Flow diagram showing stages of database search and study selection

Figure 2: Summary ROC curve (bivariate model) for pooled analysis of sensitivity and specificity
D

of qSOFA≥2 for predicting in-hospital mortality [Data from non-ICU patients included here]
TE

Figure 3: Summary ROC curve (bivariate model) for pooled analysis of sensitivity and specificity

of SIRS≥2 for predicting in-hospital mortality

EP

Figure 4: Summary ROC curve (bivariate model) for pooled analysis of sensitivity and specificity

of qSOFA≥2 for predicting in-hospital mortality [Data from the studies where predictive accuracy
C

of SIRS criterion was also reported]

AC

Table legends

Table 1: Characteristics of q-SOFA and SIRS criteria for predicting mortality in hospitalized

patients with suspected infection (Hierarchical summary statistics)

12
 ACCEPTED MANUSCRIPT

PT
RI
References:
1. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in

SC
patients with sepsis from 2 independent cohorts. JAMA. 2014;312:90-2.

2. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, et al;

U
International Forum of Acute Care Trialists. Assessment of Global Incidence and
AN
Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir

Crit Care Med. 2016;193:259-72.

M

3. Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, et al. Duration of

hypotension before initiation of effective antimicrobial therapy is the critical

D

determinant of survival in human septic shock. Crit Care Med. 2006;34:1589-96.

TE

4. American College of Chest Physicians/Society of Critical Care Medicine Consensus

Conference: definitions for sepsis and organ failure and guidelines for the use of
EP

innovative therapies in sepsis. Crit Care Med. 1992;20:864-74.

5. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al; International
C

Sepsis Definitions Conference. 2001 SCCM/ESICM/ACCP/ATS/SIS International

AC

Sepsis Definitions Conference. Intensive Care Med. 2003;29:530-8.

6. Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change.

Lancet. 2013;381:774-5.

13
 ACCEPTED MANUSCRIPT

7. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al.

The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).

JAMA. 2016;315:801-10.

8. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al.

PT
Assessment of Clinical Criteria for Sepsis: For the Third International Consensus

Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:762-74.

RI
9. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-

SC
analysis of observational studies in epidemiology: a proposal for reporting. Meta-

analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA.

U
2000;283:2008-12.

10. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al;
AN
QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic

accuracy studies. Ann Intern Med. 2011;155:529-36.

M

11. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med.
D

2002;21:1539-58.
TE

12. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other

sample size effects in systematic reviews of diagnostic test accuracy was assessed. J
EP

Clin Epidemiol. 2005;58:882-93.

13. Wang S, Li T, Li Y, Zhang J, Dai X. [Predictive value of four different scoring systems
C

for septic patient's outcome: a retrospective analysis with 311 patients]. Zhonghua Wei
AC

Zhong Bing Ji Jiu Yi Xue. 2017;29:133-138.

14. Wang Y, Wang D, Fu J, Liu Y. [Predictive value of SOFA, qSOFA score and traditional

evaluation index on sepsis prognosis]. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue.

2017;29:700-704.

14
 ACCEPTED MANUSCRIPT

15. Siddiqui S, Chua M, Kumaresh V, Choo R. A comparison of pre ICU admission SIRS,

EWS and q SOFA scores for predicting mortality and length of stay in ICU. J Crit Care.

2017;41:191-193.

16. April MD, Aguirre J, Tannenbaum LI, Moore T, Pingree A, Thaxton RE, et al. Sepsis

PT
Clinical Criteria in Emergency Department Patients Admitted to an Intensive Care Unit:

An External Validation Study of Quick Sequential Organ Failure Assessment. J Emerg

RI
Med. 2017;52:622-631.

SC
17. Innocenti F, Tozzi C, Donnini C, De Villa E, Counts A, Zanobetti M, et al. SOFA score

in septic patients: incremental prognostic value over age, comorbidities and parameters

U
of sepsis severity. Intern Emerg Med. 2017 Feb 10. [Epub ahead of print]

18. Dorsett M, Kroll M, Smith CS, Asaro P, Liang SY, Moy HP. qSOFA Has Poor
AN
Sensitivity for Prehospital Identification of Severe Sepsis and Septic Shock. Prehosp

Emerg Care. 2017;21:489-497.

M

19. Spoto S, Cella E, de Cesaris M, Locorriere L, Mazzaroppi S, Noble E, et al.

D

Procalcitonin and Mr-Proadrenomedulin Combination with Sofa and Qsofa Scores for
TE

Sepsis Diagnosis and Prognosis: A Diagnostic Algorithm. Shock. 2017 Oct 11. [Epub

ahead of print]
EP

20. Patidar KR, Shaw J, Acharya C, Thacker LR, White MB, Ganapathy D, et al. No

Association Between Quick Sequential Organ Failure Assessment and Outcomes of

C

Patients With Cirrhosis and Infections. Clin Gastroenterol Hepatol. 2017;15:1803-1804.

AC

21. Sager R, Wirz Y, Amin D, Amin A, Hausfater P, Huber A, et al. Are admission

procalcitonin levels universal mortality predictors across different medical emergency

patient populations? Results from the multi-national, prospective, observational

TRIAGE study. Clin Chem Lab Med. 2017;55:1873-1880.

15
 ACCEPTED MANUSCRIPT

22. Jawa RS, Vosswinkel JA, McCormack JE, Huang EC, Thode HC Jr, Shapiro MJ, Singer

AJ. Risk assessment of the blunt trauma victim: The role of the quick Sequential Organ

Failure Assessment Score (qSOFA). Am J Surg. 2017;214:397-401.

23. Kwak H, Suh GJ, Kim T, Kwon WY, Kim KS, Jung YS, Ko JI, Shin SM. Prognostic

PT
performance of Emergency Severity Index (ESI) combined with qSOFA score. Am J

Emerg Med. 2018 Jan 31.[Epub ahead of print]

RI
24. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The

SC
PRISMA statement for reporting systematic reviews and meta-analyses of studies that

evaluate health care interventions: explanation and elaboration. PLoS Med.

U
2009;6:e1000100.

25. Donnelly JP, Safford MM, Shapiro NI, Baddley JW, Wang HE. Application of the
AN
Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a

retrospective population-based cohort study. Lancet Infect Dis. 2017;17:661-670.

M

26. Chen YX, Wang JY, Guo SB. Use of CRB-65 and quick Sepsis-related Organ Failure
D

Assessment to predict site of care and mortality in pneumonia patients in the emergency
TE

department: a retrospective study. Crit Care. 2016;20:167.

27. Wang JY, Chen YX, Guo SB, Mei X, Yang P. Predictive performance of quick Sepsis-
EP

related Organ Failure Assessment for mortality and ICU admission in patients with

infection at the ED. Am J Emerg Med. 2016;34:1788-93.

C

28. Ho KM, Lan NS. Combining quick Sequential Organ Failure Assessment with plasma
AC

lactate concentration is comparable to standard Sequential Organ Failure Assessment

score in predicting mortality of patients with and without suspected infection. J Crit

Care. 2017;38:1-5.

29. Giamarellos-Bourboulis EJ, Tsaganos T, Tsangaris I, Lada M, Routsi C, Sinapidis D, et

al; Hellenic Sepsis Study Group. Validation of the new Sepsis-3 definitions: proposal

for improvement in early risk identification. Clin Microbiol Infect. 2017;23:104-109.

16
 ACCEPTED MANUSCRIPT

30. Williams JM, Greenslade JH, McKenzie JV, Chu K, Brown AFT, Lipman J. Systemic

Inflammatory Response Syndrome, Quick Sequential Organ Function Assessment, and

Organ Dysfunction: Insights From a Prospective Database of ED Patients With

Infection. Chest. 2017;151:586-596.

PT
31. Singer AJ, Ng J, Thode HC Jr, Spiegel R, Weingart S. Quick SOFA Scores Predict

Mortality in Adult Emergency Department Patients With and Without Suspected

RI
Infection. Ann Emerg Med. 2017;69:475-479.

SC
32. Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, et al; Australian

and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes and Resource

U
Evaluation (CORE). Prognostic Accuracy of the SOFA Score, SIRS Criteria, and

qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection
AN
Admitted to the Intensive Care Unit. JAMA. 2017;317:290-300.

33. Freund Y, Lemachatti N, Krastinova E, Van Laer M, Claessens YE, Avondo A, et al;
M

French Society of Emergency Medicine Collaborators Group. Prognostic Accuracy of

D

Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection
TE

Presenting to the Emergency Department. JAMA. 2017;317:301-308.

34. Burnham JP, Kollef MH. qSOFA score: Predictive validity in Enterobacteriaceae
EP

bloodstream infections. J Crit Care. 2017;43:143-147.

35. Piano S, Bartoletti M, Tonon M, Baldassarre M, Chies G, Romano A, et al. Assessment

C

of Sepsis-3 criteria and quick SOFA in patients with cirrhosis and bacterial infections.
AC

Gut. 2017 Aug 31. [Epub ahead of print]

36. Umemura Y, Ogura H, Gando S, Kushimoto S, Saitoh D, Mayumi T, et al. JAAM

Sepsis Registry (JAAMSR) Study Group. Assessment of mortality by qSOFA in

patients with sepsis outside ICU: A post hoc subgroup analysis by the Japanese

Association for Acute Medicine Sepsis Registry Study Group. J Infect Chemother.

2017;23:757-762.

17
 ACCEPTED MANUSCRIPT

37. Amland RC, Sutariya BB. Quick Sequential [Sepsis-Related] Organ Failure Assessment

(qSOFA) and St. John Sepsis Surveillance Agent to Detect Patients at Risk of Sepsis:

An Observational Cohort Study. Am J Med Qual. 2017:1062860617692034.

38. Kim MW, Lim JY, Oh SH. Mortality prediction using serum biomarkers and various

PT
clinical risk scales in community-acquired pneumonia. Scand J Clin Lab Invest.

2017;77:486-492.

RI
39. Park HK, Kim WY, Kim MC, Jung W, Ko BS. Quick sequential organ failure

SC
assessment compared to systemic inflammatory response syndrome for predicting sepsis

in emergency department. J Crit Care. 2017;42:12-17.

U
40. Ranzani OT, Prina E, Menéndez R, Ceccato A, Cilloniz C, Méndez R, et al. New Sepsis

Definition (Sepsis-3) and Community-acquired Pneumonia Mortality: A Validation and

AN
Clinical Decision-making Study. Am J Respir Crit Care Med. 2017;196:1287-1297

41. Askim Å, Moser F, Gustad LT, Stene H, Gundersen M, Åsvold BO, et al. Poor
M

performance of quick-SOFA (qSOFA) score in predicting severe sepsis and mortality -

D

a prospective study of patients admitted with infection to the emergency department.

TE

Scand J Trauma Resusc Emerg Med. 2017;25:56.

42. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick Sepsis-
EP

related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and

Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside
C

the Intensive Care Unit. Am J Respir Crit Care Med. 2017;195:906-911.

AC

43. de Groot B, Stolwijk F, Warmerdam M, Lucke JA, Singh GK, Abbas M, et al. The most

commonly used disease severity scores are inappropriate for risk stratification of older

emergency department sepsis patients: an observational multi-centre study. Scand J

Trauma Resusc Emerg Med. 2017;25:91.

44. Moskowitz A, Patel PV, Grossestreuer AV, Chase M, Shapiro NI, Berg K, et al; Center

for Resuscitation Science. Quick Sequential Organ Failure Assessment and Systemic

18
 ACCEPTED MANUSCRIPT

Inflammatory Response Syndrome Criteria as Predictors of Critical Care Intervention

Among Patients With Suspected Infection. Crit Care Med. 2017;45:1813-1819.

45. Huson MAM, Katete C, Chunda L, Ngoma J, Wallrauch C, Heller T, van der Poll T, et

al. Application of the qSOFA score to predict mortality in patients with suspected

PT
infection in a resource-limited setting in Malawi. Infection. 2017;45:893-896.

46. Haydar S, Spanier M, Weems P, Wood S, Strout T. Comparison of QSOFA score and

RI
SIRS criteria as screening mechanisms for emergency department sepsis. Am J Emerg

SC
Med. 2017;35:1730-1733.

47. Jouffroy R, Saade A, Carpentier A, Ellouze S, Philippe P, Idialisoa R, et al. Triage of

U
Septic Patients Using qSOFA Criteria at the SAMU Regulation: A Retrospective

Analysis. Prehosp Emerg Care. 2018;22:84-90.

AN
48. Hwang SY, Jo IJ, Lee SU, Lee TR, Yoon H, Cha WC, et al. Low Accuracy of Positive

qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During
M

the Early Period After Emergency Department Presentation. Ann Emerg Med.
D

2018;71:1-9
TE

49. Kim M, Ahn S, Kim WY, Sohn CH, Seo DW, Lee YS, et al. Predictive performance of

the quick Sequential Organ Failure Assessment score as a screening tool for sepsis,
EP

mortality, and intensive care unit admission in patients with febrile neutropenia. Support

Care Cancer. 2017;25:1557-1562.

C

50. Quinten VM, van Meurs M, Wolffensperger AE, Ter Maaten JC, Ligtenberg JJM.
AC

Sepsis patients in the emergency department: stratification using the Clinical Impression

Score, Predisposition, Infection, Response and Organ dysfunction score or quick

Sequential Organ Failure Assessment score? Eur J Emerg Med. 2017 May 8. [Epub

ahead of print]

19
 ACCEPTED MANUSCRIPT

51. Rannikko J, Syrjänen J, Seiskari T, Aittoniemi J, Huttunen R. Sepsis-related mortality

in 497 cases with blood culture-positive sepsis in an emergency department. Int J Infect

Dis. 2017;58:52-57.

52. González Del Castillo J, Julian-Jiménez A, González-Martínez F, Álvarez-Manzanares

PT
J, Piñera P, Navarro-Bustos C, et al; representing the Infectious Disease Group of the

Spanish Emergency Medicine Society (see addendum). Prognostic accuracy of SIRS

RI
criteria, qSOFA score and GYM score for 30-day-mortality in older non-severely

SC
dependent infected patients attended in the emergency department. Eur J Clin Microbiol

Infect Dis. 2017;36:2361-2369.

U
53. Forward E, Konecny P, Burston J, Adhikari S, Doolan H, Jensen T. Predictive validity

of the qSOFA criteria for sepsis in non-ICU inpatients. Intensive Care Med.
AN
2017;43:945-946.

54. Henning DJ, Puskarich MA, Self WH, Howell MD, Donnino MW, Yealy DM, et al. An
M

Emergency Department Validation of the SEP-3 Sepsis and Septic Shock Definitions
D

and Comparison With 1992 Consensus Definitions. Ann Emerg Med. 2017;70:544-
TE

552.e5.

55. Huson MA, Kalkman R, Grobusch MP, van der Poll T. Predictive value of the qSOFA
EP

score in patients with suspected infection in a resource limited setting in Gabon. Travel

Med Infect Dis. 2017;15:76-77.

C

56. Moore CC, Hazard R, Saulters KJ, Ainsworth J, Adakun SA, Amir A, et al. Derivation
AC

and validation of a universal vital assessment (UVA) score: a tool for predicting

mortality in adult hospitalised patients in sub-Saharan Africa. BMJ Glob Health.

2017;2:e000344.

57. Peake SL, Delaney A, Bailey M, Bellomo R; ARISE Investigators. Potential Impact of

the 2016 Consensus Definitions of Sepsis and Septic Shock on Future Sepsis Research.

Ann Emerg Med. 2017;70:553-561.

20
 ACCEPTED MANUSCRIPT

58. Guirgis FW, Puskarich MA, Smotherman C, Sterling SA, Gautam S, Moore FA, Jones

AE. Development of a Simple Sequential Organ Failure Assessment Score for Risk

Assessment of Emergency Department Patients With Sepsis. J Intensive Care Med.

2017 Jan 1:885066617741284.[Epub ahead of print]

PT
59. Szakmany T, Pugh R, Kopczynska M, Lundin RM, Sharif B, Morgan P, et al; Welsh

Digital Data Collection Platform collaborators. Defining sepsis on the wards: results of

RI
a multi-centre point-prevalence study comparing two sepsis definitions. Anaesthesia.

SC
2018;73:195-204.

60. Goulden R, Hoyle MC, Monis J, Railton D, Riley V, Martin P, Martina R, Nsutebu E.

U
qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in

emergency admissions treated as sepsis. Emerg Med J. 2018 Feb 21.[Epub ahead of
AN
print]

61. Müller M, Guignard V, Schefold JC, Leichtle AB, Exadaktylos AK, Pfortmueller CA.
M

Utility of quick sepsis-related organ failure assessment (qSOFA) to predict outcome in

D

patients with pneumonia. PLoS One. 2017;12:e0188913.

TE

62. Costa RT, Nassar AP Jr, Caruso P. Accuracy of SOFA, qSOFA, and SIRS scores for

mortality in cancer patients admitted to an intensive care unit with suspected infection. J
EP

Crit Care. 2018 Jan 4;45:52-57.[Epub ahead of print]

63. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison of the performance

C

of SOFA, qSOFA and SIRS for predicting mortality and organ failure among sepsis
AC

patients admitted to the intensive care unit in a middle-income country. J Crit Care.

2018;44:156-160.

64. Estella A, Gamazo-Del Rio J, Álvarez-Manzanares J, Julián-Jiménez A, González Del

Castillo J. Prognostic accuracy of qsofa according to the site of infection in older patient

attended in Emergency Department. Eur J Intern Med. 2018 Jan 4. [Epub ahead of print]

21
 ACCEPTED MANUSCRIPT

65. Fukushima H, Kobayashi M, Kawano K, Morimoto S. Performance of qSOFA and

SOFA for predicting mortality in patients with acute pyelonephritis associated with

upper urinary tract calculi. J Urol. 2017 Dec 29.[Epub ahead of print]

66. Asai N, Watanabe H, Shiota A, Kato H, Sakanashi D, Hagihara M, Koizumi Y,

PT
Yamagishi Y, Suematsu H, Mikamo H. Could qSOFA and SOFA score be correctly

estimating the severity of healthcare-associated pneumonia? J Infect Chemother.

RI
2018;24:228-231.

SC
67. LeGuen M, Ballueer Y, McKay R, Eastwood G, Bellomo R, Jones D; Austin Health

RRT qSOFA investigators. Frequency and significance of qSOFA criteria during adult

U
rapid response team reviews: A prospective cohort study. Resuscitation. 2018;122:13-

18.
AN
68. Finkelsztein EJ, Jones DS, Ma KC, Pabón MA, Delgado T, Nakahira K, et al.
M

Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with

suspicion of sepsis outside the intensive care unit. Crit Care. 2017;21:73.
D

69. Torio CM, Andrews RM. National inpatient hospital costs: the most expensive
TE

conditions by payer, 2011. Statistical Brief #160. Healthcare Cost and Utilization

Project (HCUP) Statistical Briefs. August 2013.

EP

70. Goetzinger KR, Odibo AO. Statistical analysis and interpretation of prenatal diagnostic

imaging studies, Part 1: evaluating the efficiency of screening and diagnostic tests. J
C

Ultrasound Med. 2011;30:1121-7.

AC

71. Filho RR, Rocha LL, Corrêa TD, Pessoa CM, Colombo G, Assuncao MS. Blood

Lactate Levels Cutoff and Mortality Prediction in Sepsis-Time for a Reappraisal? a

Retrospective Cohort Study. Shock. 2016;46:480-485.

72. Shankar-Hari M, Phillips GS, Levy ML, Seymour CW, Liu VX, Deutschman CS, et al;

Sepsis Definitions Task Force. Developing a New Definition and Assessing New

22
 ACCEPTED MANUSCRIPT

Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for

Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:775-87.

73. Jacome T, Tatum D. Systemic Inflammatory Response Syndrome (SIRS) Score

Independently Predicts Poor Outcome in Isolated Traumatic Brain Injury. Neurocrit

PT
Care. 2018;28:110-116.

RI
74. Huang WC, Chou RH, Chang CC, Hsu CY, Ku YC, Huang HF, et al. Systemic

Inflammatory Response Syndrome is an Independent Predictor of One-Year Mortality

SC
in Patients with Acute Myocardial Infarction. Acta Cardiol Sin. 2017;33:477-485

U
AN
M
D
TE
C EP
AC

23
 ACCEPTED MANUSCRIPT

Table 1: Characteristics of q-SOFA and SIRS criteria for predicting mortality in hospitalized

patients with suspected infection (Hierarchical summary statistics)

Performance q-SOFA≥2
SOFA≥2 q-SOFA≥2
SOFA≥2a q-SOFA≥2
SOFA≥2b SIRS≥2
SIRS≥2

PT
of the score (n=406,802) (n=207,708) (n=350,164) (n=348,980)

Sensitivity 0.56(0.47- 0.48(0.41- 0.60(0.45- 0.82(0.73- 0.89)

RI
0.65) 0.55) 0.73)

SC
Specificity 0.78(0.71- 0.83(0.78- 0.76(0.64- 0.36(0.24- 0.50)

0.83) 0.87) 0.84)

U
Diagnostic 2.5(2.1-2.9) 4.36(3.72- 4.63(3.57- 2.59 (2.09- 3.22)
AN
Odds ratio
5.13) 6.00)

AUC of 0.73(0.69- 0.71(0.67- 0.74(0.70- 0.67(0.63- 0.71)

M

summary ROC
0.77) 0.75) 0.78)

Likelihood 2.50(2.15- 2.8(2.4- 3.2) 2.45(1.97- 1.30 (1.10- 1.50)

D

ratio (+)
2.92) 3.05)
TE

Likelihood 0.57(0.49- 0.63(0.58- 0.53(0.42- 0.49 (0.42- 0.58)

ratio (-)
0.65) 0.69) 0.67)
EP

a Data from only non-ICU patients have been included in this analysis
C

b Studies where both prognostic accuracies of q-SOFA and SIRS have been reported from
AC

same cohort have been included in this pooled analysis

 ACCEPTED MANUSCRIPT

Identification

Records identified through database Additional records identified through

searching references of the primary search

PT
(n = 506) (n=57)

RI
Records after duplicates removed
(n =112)

SC
Screening

U
Records screened Records excluded
(n =112) (n = 56)
AN
M
Full-text articles assessed n=11 full-text articles
for eligibility excluded because,
Eligibility

(n = 56) n =2 for non availability of

D

English language full text

n=7 for in-adequate data to
constitute 2x2 table
TE

n=2 for patient population

Studies included in
not being clinically suspected
qualitative synthesis
or proven infection
(n = 45)
EP
Included

Studies included in
C

quantitative synthesis
(meta-analysis)
AC

(n = 45)
 ACCEPTED MANUSCRIPT

PT
RI
SC
SROC with Prediction & Confidence Contours
1.0

U 18
AN
27

29
M

36
17 28
30
D

2 25
TE

22
8 38
Sensitivity

23
3
9
10
EP

2611
0.5 4
12
37

35
14
C

5 19
16
31
3320
AC

15
34 Observed Data
24
Summary Operating Point
6 SENS = 0.48 [0.41 − 0.55]
13 32
SPEC = 0.83 [0.78 − 0.87]
21
SROC Curve
AUC = 0.71 [0.67 − 0.75]
71 95% Confidence Contour

95% Prediction Contour

0.0
1.0 0.5 0.0
Specificity
 ACCEPTED MANUSCRIPT

PT
RI
SC
1.0 20

U
11
16 23
7
AN
9

14 15
10
18
M

6
5
4
17 13
D

19
1 12
TE
Sensitivity

EP

8
0.5
C
AC

Observed Data

Summary Operating Point

SENS = 0.82 [0.73 − 0.89]
SPEC = 0.36 [0.24 − 0.50]
SROC Curve
21 AUC = 0.67 [0.63 − 0.71]

95% Confidence Contour

95% Prediction Contour

0.0
1.0 0.5 0.0
Specificity
 ACCEPTED MANUSCRIPT

PT
RI
SC
1.0 20

U 11 19
AN
3
M

164
D

2 14
7
TE
Sensitivity

EP

15 8
0.5 5
9
C

6 12
10
18
AC

21 Observed Data
13
Summary Operating Point
17 SENS = 0.60 [0.46 − 0.73]
SPEC = 0.76 [0.64 − 0.84]
SROC Curve
AUC = 0.74 [0.70 − 0.78]
1 95% Confidence Contour

95% Prediction Contour

0.0
1.0 0.5 0.0
Specificity