You are on page 1of 5

J Infect Chemother 24 (2018) 887e891

Contents lists available at ScienceDirect

Journal of Infection and Chemotherapy


journal homepage: http://www.elsevier.com/locate/jic

Original Article

Evaluation of three automated Treponema pallidum antibody assays


for syphilis screening
Chang-sheng Xia, Zhi-hong Yue, Hui Wang*
Department of Clinical Laboratory, Peking University People's Hospital, No.11 Xizhimen South Street, Beijing, 100044, China

a r t i c l e i n f o a b s t r a c t

Article history: The accuracy of the test is critical for the syphilis serology diagnosis. This study aims to evaluate the
Received 15 March 2018 values of the Elecsys syphilis assay, the Architect syphilis assay, and the Mindray syphilis assay, as
Received in revised form syphilis screening tests for pregnant women and patients with syphilis or other diseases. A reverse
23 June 2018
algorithm was used for the syphilis serology diagnosis. Serum samples (n ¼ 584) were tested with three
Accepted 26 July 2018
Available online 7 September 2018
automated screening assays. All reactive sera by one, two, or three screening assays were further
analyzed with the tolulized red unheated serum test (TRUST). Inconsistent results were confirmed by the
Treponema pallidum particle agglutination assay (TPPA). The final patient diagnosis was made according
Keywords:
Treponema pallidum
to the results of syphilis serology, clinical evidence, and past medical history. The sensitivity, specificity,
Syphilis accuracy, and kappa value of each assay were as follows: for the Elecsys syphilis assay, 100.0%, 98.5%,
Tolulized red unheated serum test (TRUST) 98.6%, and 0.927, respectively; for the Architect syphilis assay: 100.0%, 94.5%, 95.0%, and 0.770; and for
Chemiluminescent microparticle the Mindray syphilis assay: 100.0%, 97.0%, 97.3%, and 0.862. The McNemar test showed that there were
immunoassay (CMIA) significant differences in the performance between the Elecsys syphilis assay and the Architect syphilis
Electrochemiluminescence immunoassay assay (P < 0.001), and between the Mindray syphilis assay and the Architect syphilis assay (P ¼ 0.001).
(ECLIA) Our study demonstrated that three automated Treponema pallidum antibody assays generally
Treponema pallidum particle agglutination
showed high sensitivities and specificities, and so, they are suitable for use in screening for syphilis.
test (TPPA)
The performances of the Elecsys syphilis assay and the Mindray syphilis assay are superior to Architect
syphilis assay.
© 2018 Japanese Society of Chemotherapy and The Japanese Association for Infectious Diseases.
Published by Elsevier Ltd. All rights reserved.

1. Introduction tests include: the traditional algorithm and the reverse algorithm
[2]. The traditional algorithm for syphilis begins with a non-
Syphilis is a chronic and complex sexually transmitted disease treponemal test, such as the rapid plasma reagin (RPR), tolulized
caused by Treponema pallidum. It is transmitted mainly through red unheated serum test (TRUST) or Venereal Disease Research
sexual contact or during pregnancy from a mother to her fetus. Laboratory (VDRL) test. Then, reactive samples are confirmed by
Syphilis remains a global public health problem. In 2015, about 45.4 using one of several treponemal tests. This algorithm performs well
million people were infected with syphilis throughout the world [1]. in identifying patients with active syphilis, while reducing the
If left untreated, syphilis can cause serious complications. false-positive rate in the low-prevalence population [3]. Recently,
The accuracy of diagnostic tests is critical for the successful treat- the availability of automated treponemal chemiluminescence
ment of syphilis. It is hard to culture Treponema pallidum in vitro in immunoassay (CIA) has led more and more laboratories to use a
the routine laboratory. Serological tests, which include treponemal reverse algorithm in which a treponemal CIA is performed first,
and nontreponemal tests, are commonly used as the mainstay for followed by detecting of the reactive specimen with a quantitative
diagnosing syphilis and for monitoring the treatment. The two RPR or other nontreponemal test. If test results are discordant, the
different approaches for the diagnosis of syphilis with serological specimen should be tested reflexively using a second and different
treponemal assay [4]. The fluorescent treponemal antibody ab-
sorption (FTA-ABS) test has long been considered as a gold standard
treponemal test and is still used by some laboratories [5,6].
* Corresponding author.
E-mail addresses: xiachangsheng@bjmu.edu.cn (C.-s. Xia), yue_zhihong@pkuph. However, the FTA-ABS test has probably lower sensitivity and lower
edu.cn (Z.-h. Yue), wanghui@pkuph.edu.cn (H. Wang). specificity than other treponemal tests [7]. The US CDC

https://doi.org/10.1016/j.jiac.2018.07.017
1341-321X/© 2018 Japanese Society of Chemotherapy and The Japanese Association for Infectious Diseases. Published by Elsevier Ltd. All rights reserved.

Downloaded for hendsun Goh (hendsunh@ymail.com) at Universitas Tarumanagara from ClinicalKey.com by Elsevier on October 22, 2018.
For personal use only. No other uses without permission. Copyright ©2018. Elsevier Inc. All rights reserved.
888 C.-s. Xia et al. / J Infect Chemother 24 (2018) 887e891

recommends use of the Treponema pallidum particle agglutination conducted to evaluate the agreement between each syphilis
assay (TPPA) test rather than the FTA-ABS test to confirm discordant screening assay and the final patient diagnosis. The McNemar
treponemal screening results [8]. The reverse algorithm has been Chi-square test was performed to analyze the qualitative data
reported to have higher diagnostic efficacy, with sensitivities comparison between each syphilis screening assay. The optimal
ranging from 99.38% to 99.85% as compared with a 75.81% cutoff value of each automated assay cutoff index (COI, namely
sensitivity of the traditional algorithm [9]. S/CO) for syphilis diagnosis were determined by the receiver
The Elecsys syphilis assay using an electrochemiluminescence operating characteristic (ROC) and comparisons of ROC curves were
immunoassay (ECLIA) analyzer and the Architect syphilis assay performed by the methodology of DeLong et al. [10]. P value < 0.05
using a chemiluminescent magnetic microparticle immunoassay was considered statistically significant.
(CMIA) analyzer are two common methods for syphilis screening.
The Mindray syphilis assay, which also uses CMIA instrument, was 3. Results
launched in China recently. These automated Treponema pallidum
antibodies immunoassays that have been developed use either 3.1. Syphilis serology diagnosis with the reverse algorithm
whole cells or recombinant antigens, such as TpN15-TpN17-TpN47, and final patient diagnosis
derived from the Nichols strain of Treponema pallidum, to detect IgG,
IgM, or total immunoglobulins. Now, CIA assays including ECLIA and The reverse algorithm results and final patient diagnosis are
CMIA, are commonly used to detect Treponema pallidum antibodies illustrated in Fig. 1. Serum samples (n ¼ 584) were tested with three
for syphilis screening. However, evaluation of the performances of screening assays. Overall, 493 samples were nonreactive by all
these various immunoassays is lacking. three screening assays. Of 91 reactive samples by CMIA/ECLIA,
The aim of this study was to evaluate the performances of 23.1% (21/91) were TRUST reactive and 76.9% (70/91) were TRUST
three automated Treponema pallidum antibody immunoassays for nonreactive. Among 70 TRUST nonreactive samples, 34 were TPPA
syphilis screening. reactive, 2 were TPPA indeterminate, and 34 were TPPA nonreac-
tive. For the 34 TPPA nonreactive samples and 2 TPPA indetermi-
2. Materials and methods nate samples, the details are shown in Table 1. A total of 57 patients
were diagnosed with syphilis based on the results of syphilis
2.1. Samples serology, clinical evidence, and past medical history. Laboratory
results and final patient diagnosis for 57 syphilis patients are
This prospective study involved a total of 584 serum samples shown in Table 2.
(one sample per patient) collected from the Peking University Peo-
ple's Hospital from December 2016 to February 2017. These samples
3.2. Evaluation of three automated Treponema pallidum antibody
included those from 124 pregnant women, 452 patients with various
assays based on the final patient diagnosis
diseases, and 8 patients with suspected current syphilis. The median
age of all patients was 42 years (range, 20e93 years). Of all patients,
The sensitivity, specificity, accuracy, and kappa value of each
214 were male and 370 were female. The study was approved by the
assay based on the final patient diagnosis, were as follows: for the
Institutional Review Board of Peking University People's Hospital.

2.2. Laboratory analysis and clinical diagnosis

The reverse algorithm was used for the diagnosis of syphilis


serology. Samples were firstly tested with three kinds of Treponema
pallidum antibody immunoassays. These assays included the Elec-
sys syphilis assay (Roche, Mannheim, Germany), the Architect
syphilis assay (Abbott, Wiesbaden, Germany), and the Mindray
syphilis assay (Mindray, Shenzhen, China). All reactive sera by one,
two or three CIA assays were further analyzed with the semi-
quantitative nontreponemal assay, TRUST (Rongsheng, Shanghai,
China). Discordant results that were reactive to CIA but nonreactive
to TRUST were confirmed by the TPPA (Fujirebio, Tokyo, Japan) test.
Testing was performed following the manufacturer's instructions in
the Department of Clinical Laboratory at Peking University of
People's Hospital. When the testing cannot be completed in one
day, the serum samples were stored at 20  C until the next testing
started. The final patient diagnosis was made based on the results
of syphilis serology, clinical evidence, and past medical history.
Simply, patients were diagnosed with syphilis if the samples were
CIA reactive/TRUST reactive or CIA reactive/TRUST nonreactive/
TPPA reactive. Moreover, patients who recently or previously
infected Treponema pallidum were also diagnosed with syphilis.

2.3. Statistical analysis

Statistical analyses were performed using MedCalc statistical


software version 11.4.2 (D JINN) and SPSS 16.0 statistical software Fig. 1. Syphilis serology diagnosis with the reverse algorithm and final patient
(SPSS, Inc., Chicago, IL). The sensitivity, specificity, and accuracy of diagnosis. Abbreviations: TRUST, tolulized red unheated serum test; TPPA, Treponema
each syphilis screening assay were calculated. The kappa test was pallidum particle agglutination.

Downloaded for hendsun Goh (hendsunh@ymail.com) at Universitas Tarumanagara from ClinicalKey.com by Elsevier on October 22, 2018.
For personal use only. No other uses without permission. Copyright ©2018. Elsevier Inc. All rights reserved.
C.-s. Xia et al. / J Infect Chemother 24 (2018) 887e891 889

Table 1
Details of 36 samples inconsistent between CIA and TPPA.

No. of samples Test reactivity Final diagnosis

Elecsys syphilis Architect syphilis Mindray syphilis TPPA

2 Reactive Reactive Reactive Indeterminate Syphilis#


3 Reactive Reactive Reactive Nonreactive No syphilis
1 Reactive Nonreactive Reactive Nonreactive No syphilis
4 Reactive Nonreactive Nonreactive Nonreactive No syphilis
12 Nonreactive Reactive Reactive Nonreactive No syphilis
14 Nonreactive Reactive Nonreactive Nonreactive No syphilis

Abbreviations: CIA, chemiluminescence immunoassays; TPPA, Treponema pallidum particle agglutination. #, previously treated syphilis.

Table 2
Laboratory results and the final patient diagnosis for 57 syphilis patients.

No. of patients Syphilis stage Laboratory results

3 Primary All three automated screening assays reactive/TRUST reactive


5 Secondary All three automated screening assays reactive/TRUST reactive
1 Tertiary All three automated screening assays reactive/TRUST reactive
5 Latent All three automated screening assays reactive/TRUST reactive
7 Unknown All three automated screening assays reactive/TRUST reactive
2 Unknown All three automated screening assays reactive/TRUST nonreactive/TPPA indeterminate
34 Unknown All three automated screening assays reactive/TRUST nonreactive/TPPA reactive

Elecsys syphilis assay, 100.0%, 98.5%, 98.6%, and 0.927, respectively; interval (CI) of 0.994e1.000 (P < 0.001). The AUC for the Architect
for the Architect syphilis assay, 100.0%, 94.5%, 95.0%, and 0.770, syphilis assay was 0.998 (95% CI: 0.990e1.000, P < 0.001). The AUC
respectively; and for the Mindray syphilis assay, 100.0%, 97.0%, for the Mindray syphilis assay was 1.000 (95% CI: 0.994e1.000,
97.3%, and 0.862, respectively (Table 3). P < 0.001) (Fig. 2). Comparisons of ROC curves revealed no signifi-
cant differences in the AUC between the Elecsys syphilis assay and
the Architect syphilis assay (P ¼ 0.057), between the Elecsys syphilis
3.3. Comparisons of three automated Treponema pallidum
assay and the Mindray syphilis assay (P ¼ 1.000), and between the
antibody assays
Architect syphilis assay and the Mindray syphilis assay (P ¼ 0.057).
The McNemar test (Table 4) showed that there were significant
differences in the performance between the Elecsys syphilis assay
4. Discussion
and the Architect syphilis assay (P < 0.001), and between the
Architect syphilis assay and the Mindray syphilis assay (P ¼ 0.001).
The reverse algorithm has been found to show superior per-
The performances for the Elecsys syphilis assay and the Mindray
formance compared to the traditional algorithm for syphilis diag-
syphilis assay were not significantly different (P ¼ 0.077).
nosis [4,11,12]. Simcic's study showed that the traditional algorithm
had a missed serodiagnosis rate of 30.0% [11]. A recent study
3.4. ROC analysis of three automated Treponema pallidum antibody reported that the traditional algorithm failed to detect 81.4%
assays for syphilis diagnosis (48/59) reactive specimens [12]. In the present study, we use a
reverse algorithm for syphilis serology diagnosis. For 57 samples
The optimal cutoff values of COI were as follows: for the Elecsys from patients with final syphilis diagnosis, only 21 were TRUST
syphilis assay, 5.9; for the Architect syphilis assay, 1.8; and for the reactive, and 36 were TRUST nonreactive. Our study showed the
Mindray syphilis assay, 2.5. The corresponding sensitivity and traditional algorithm had a missed diagnosis rate of 63.2% (36/57).
specificity values of each assay were 100.0% and 100.0%, 100.0% and The US CDC reported similar results [4].
97.3%, 100.0% and 100.0%, respectively. The area under the curve The availability of automatable and high-throughput trepo-
(AUC) for the Elecsys syphilis assay was 1.000, with a 95% confidence nemal CIA assays has led more and more laboratories to adopt

Table 3
Evaluation of three automated Treponema pallidum antibody assays based on the final patient diagnosis.

Assay and results Final diagnosis %Sensitivity %Specificity %Accuracy Kappa value
NO. of samples (95%CI) (95%CI) (95%CI) (95%CI)

positive negative

Elecsys 100.0% 98.5% 98.6% 0.927


Reactive 57 8 (93.9%e100.0%) (97.0%e99.3%) (97.7%e99.6%) (0.877e0.977)
Nonreactive 0 519
Architect 100.0% 94.5% 95.0% 0.770
Reactive 57 29 (93.9%e100.0%) (92.2%e96.3%) (93.3%e96.8%) (0.691e0.850)
Nonreactive 0 498
Mindray 100.0% 97.0% 97.3% 0.862
Reactive 57 16 (93.9%e100.0%) (95.1%e98.3%) (95.9%e98.6%) (0.796e0.928)
Nonreactive 0 511

Abbreviations: CI, confidence interval.

Downloaded for hendsun Goh (hendsunh@ymail.com) at Universitas Tarumanagara from ClinicalKey.com by Elsevier on October 22, 2018.
For personal use only. No other uses without permission. Copyright ©2018. Elsevier Inc. All rights reserved.
890 C.-s. Xia et al. / J Infect Chemother 24 (2018) 887e891

Table 4
Comparisons of three automated Treponema pallidum antibody assays.

Assay and results compared to the Elecsys Architect Mindray McNemar test
final diagnosis
NO. of samples P value

Agreement Disagreement Agreement Disagreement Agreement Disagreement

Elecsys < 0.001


Agreement 550 26
Disagreement 5 3
Architect 0.001
Agreement 554 1
Disagreement 14 15
Mindray 0.077
Agreement 564 4
Disagreement 12 4

treponemal CIA assay as the first-line test for the reverse algorithm showed that TPPA had perfect specificity and TPPA results in
[13e17]. In the present study, we evaluated the performances of conjunction with clinical assessment helped guide the management
three automated CIA assays based on the final patient diagnosis. of patients with CIA reactive, TRUST nonreactive serology. CIA
We found that the sensitivity, specificity, accuracy, and kappa reactive, TRUST nonreactive, TPPA nonreactive serology mostly
values of comparable (Table 3), leading us to conclude these auto- represented a false-positive CIA in our study population. Therefore,
mated Treponema pallidum antibody assays generally show high it is important to perform a thorough review of a patient's clinical
sensitivities and specificities, remarkable accuracies, and substan- information for interpreting the results of syphilis serology.
tial agreements. However, each automated assay has false-positive When we compared the performances of the three automated
results. So, these automated assays are acceptable for the screening Treponema pallidum antibody assays, we used the McNemar test
of syphilis and reactive samples should be tested by other assays (Table 4) firstly. It indicated that the performances of the Elecsys
following the reverse algorithm. syphilis assay and the Mindray syphilis assay were superior to
Many laboratories use TPPA as a confirmation test for samples Architect syphilis assay. Secondly, we established the ROC of each
with discordant screening reactive and RPR nonreactive results assay (Fig. 2). These optimal cutoff values of COI were higher than
[2,8,14,17e20]. However, some laboratories reported that some those (COI, 1.0) recommended by the manufactures. Our study
samples that were screening reactive and TPPA nonreactive were revealed that the samples with higher COI values were more
Western blots reactive and/or FTA-ABS reactive [2,19,20]. In our likely to be true-positive. Comparisons of ROC curves showed that
study, we made a final patient diagnosis according to the results of the performances of three automated Treponema pallidum anti-
syphilis serology, clinical evidence, and past medical history. Among body assays were not significantly different. Our study showed
70 TRUST nonreactive samples, 34 TPPA reactive samples were true- that the manufactures probably chose lower COI to increase
positive, 2 TPPA indeterminate samples were also true-positive, and sensitivity and decrease specificity for these syphilis screening
34 TPPA nonreactive samples was true-negative. The present study tests.

Fig. 2. Receiver operator characteristic (ROC) curves of three automated Treponema pallidum antibody assays for syphilis diagnosis. The area under the curve was 1.000 for the
Elecsys syphilis assay, 0.998 for the Architect syphilis assay, and 1.000 for the Mindray syphilis assay.

Downloaded for hendsun Goh (hendsunh@ymail.com) at Universitas Tarumanagara from ClinicalKey.com by Elsevier on October 22, 2018.
For personal use only. No other uses without permission. Copyright ©2018. Elsevier Inc. All rights reserved.
C.-s. Xia et al. / J Infect Chemother 24 (2018) 887e891 891

This study has several limitations. First, it is a single-center [3] Pope V. Use of treponemal tests to screen for syphilis. Infect Med 2004;21:
399e402.
study. Second, the final patient diagnosis based on the syphilis
[4] Syphilis testing algorithms using treponemal tests for initial screening–four
serology results, clinical evidence, and past medical history, may be laboratories, New York city, 2005-2006. MMWR Morb Mortal Wkly Rep
not absolutely right. 2008;57:872e5.
In summary, our study demonstrated that three automated [5] Binnicker MJ, Jespersen DJ, Rollins LO. Treponema-specific tests for serodi-
agnosis of syphilis: comparative evaluation of seven assays. J Clin Microbiol
Treponema pallidum antibody assays generally showed high sensi- 2011;49:1313e7.
tivities and specificities, remarkable accuracies, and substantial [6] Park BG, Yoon JG, Rim JH, Lee A, Kim HS. Comparison of six automated
agreements, and so, they are suitable for use in screening for syphilis. treponema-specific antibody assays. J Clin Microbiol 2016;54:163e7.
[7] Marangoni A, Sambri V, Storni E, D'Antuono A, Negosanti M, Cevenini R.
In addition, the performances of the Elecsys syphilis assay and the Treponema pallidum surface immunofluorescence assay for serologic diag-
Mindray syphilis assay are superior to Architect syphilis assay. nosis of syphilis. Clin Diagn Lab Immunol 2000;7:417e21.
[8] Discordant results from reverse sequence syphilis screening–five laboratories,
United States, 2006-2010. MMWR Morb Mortal Wkly Rep 2011;60:133e7.
Funding [9] Tong ML, Lin LR, Liu LL, Zhang HL, Huang SJ, Chen YY, et al. Analysis of 3 al-
gorithms for syphilis serodiagnosis and implications for clinical management.
This work was supported by the National Development and Clin Infect Dis 2014;58:1116e24.
[10] DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or
Reform Commission of China (grant number 2127000068). more correlated receiver operating characteristic curves: a nonparametric
approach. Biometrics 1988;44:837e45.
Authorship statement [11] Simcic S, Potocnik M. Serological diagnosis of syphilis: a comparison of
different diagnostic methods. Acta Dermatovenerol Alpina Pannonica Adria-
tica 2015;24:17e20.
All authors meet the ICMJE authorship criteria. Xia CS per- [12] Nah EH, Cho S, Kim S, Cho HI, Chai JY. Comparison of traditional and reverse
formed the laboratory tests and drafted the first versions of the syphilis screening algorithms in medical health checkups. Ann Lab Med
manuscript. Yue ZH reviewed the patients’ medical records and 2017;37:511e5.
[13] Kremastinou J, Polymerou V, Lavranos D, Aranda Arrufat A, Harwood J, Mar-
designed the figure in the manuscript. Wang H edited the manu- tinez Lorenzo MJ, et al. Evaluation of elecsys syphilis assay for routine and
script and provided intellectual inputs. All authors approved the blood screening and detection of early infection. J Clin Microbiol 2016;54:
final version of the manuscript. 2330e6.
[14] Li L, Cai B, Tao C, Wang L. Performance evaluation of clia for treponema pal-
lidum specific antibodies detection in comparison with elisa. J Clin Lab Anal
Conflict of interest 2016;30:216e22.
[15] Zhou J, Liang Y, Zhang J, Cui L. The analyzation and clinical evaluation of eclia
and cmia in the detection of treponema pallidum. Medicine (Baltim) 2017;96:
None. e7139.
[16] Sommese L, Sabia C, Esposito A, Iannone C, Montesano ML, Napoli C. Com-
Acknowledgments parison of performance of two treponema pallidum automated chemilumi-
nescent immunoassays in blood donors. Infect Dis (Lond) 2016;48:483e7.
[17] Saw S, Zhao H, Tan P, Saw B, Sethi S. Evaluation of the automated advia
We want to thank Chun-hong Fan, Hong Liu, Ai-min Zhang, and centaur(r) xp syphilis assay for serological testing. Diagn Microbiol Infect Dis
Li-xin Huang for their work on this study. 2017;88:7e11.
[18] Li Z, Feng Z, Liu P, Yan C. Screening for antibodies against treponema pallidum
with chemiluminescent microparticle immunoassay: analysis of discordant
References serology results and clinical characterization. Ann Clin Biochem 2016;53:
588e92.
[1] Global, regional, and national incidence, prevalence, and years lived with [19] Zhiyan L, Meiling W, Ping L, Jinhua D, Zhenlin Y, Zhenru F. Consistency between
disability for 310 diseases and injuries, 1990-2015: a systematic analysis for treponema pallidum particle agglutination assay and architect chemilumi-
the global burden of disease study 2015. Lancet 2016;388:1545e602. nescent microparticle immunoassay and characterization of inconsistent
[2] Park IU, Chow JM, Bolan G, Stanley M, Shieh J, Schapiro JM. Screening for samples. J Clin Lab Anal 2015;29:281e4.
syphilis with the treponemal immunoassay: analysis of discordant serology [20] Lee K, Park H, Roh EY, Shin S, Park KU, Park MH, et al. Characterization of sera
results and implications for clinical management. J Infect Dis 2011;204: with discordant results from reverse sequence screening for syphilis. BioMed
1297e304. Res Int 2013;2013:269347.

Downloaded for hendsun Goh (hendsunh@ymail.com) at Universitas Tarumanagara from ClinicalKey.com by Elsevier on October 22, 2018.
For personal use only. No other uses without permission. Copyright ©2018. Elsevier Inc. All rights reserved.

You might also like