Original Research
Evaluation of Newer Risk Markers for Coronary Heart Disease
Risk Classification
A Cohort Study
Maryam Kavousi, MD, MSc; Suzette Elias-Smale, MD, PhD; Joost H.W. Rutten, MD, PhD; Maarten J.G. Leening, MD, MSc;
Rozemarijn Vliegenthart, MD, PhD; Germaine C. Verwoert, MSc; Gabriel P. Krestin, MD, PhD; Matthijs Oudkerk, MD, PhD;
Moniek P.M. de Maat, PhD; Frank W.G. Leebeek, PhD; Francesco U.S. Mattace-Raso, MD, PhD; Jan Lindemans, PhD;
Albert Hofman, MD, PhD; Ewout W. Steyerberg, PhD; Aad van der Lugt, MD, PhD; Anton H. van den Meiracker, MD, PhD;
and Jacqueline C.M. Witteman, PhD
Background: Whether newer risk markers for coronary heart dis-
ease (CHD) improve CHD risk prediction remains unclear.
Objective: To assess whether newer risk markers for CHD risk
prediction and stratification improve Framingham risk score (FRS)
predictions.
Design: Prospective population-based study.
Setting: The Rotterdam Study, Rotterdam, the Netherlands.
Participants: 5933 asymptomatic, community-dwelling participants
(mean age, 69.1 years [SD, 8.5]).
Measurements: Traditional CHD risk factors used in the FRS (age,
sex, systolic blood pressure, treatment of hypertension, total and
high-density lipoprotein cholesterol levels, smoking, and diabetes)
and newer CHD risk factors (N-terminal fragment of prohormone
B-type natriuretic peptide levels, von Willebrand factor antigen lev-
els, fibrinogen levels, chronic kidney disease, leukocyte count,
C-reactive protein levels, homocysteine levels, uric acid levels, cor-
onary artery calcium [CAC] scores, carotid intimamedia thickness,
peripheral arterial disease, and pulse wave velocity).
Results: Adding CAC scores to the FRS improved the accuracy of
risk predictions (c-statistic increase, 0.05 [95% CI, 0.02 to 0.06];
C linical decision making for detection, management,
and prevention of coronary heart disease (CHD) relies
on accurate risk assessment. The Framingham risk score
(FRS) is the most commonly used CHD risk prediction
instrument in clinical settings (1) and constitutes the ba-
sis for the Adult Treatment Panel III guidelines for
cholesterol-lowering therapy (2). Since validation of the
FRS, many coronary risk factors, also called risk markers,
have been identified. Efforts are ongoing to assess the in-
See also:
Print
Editors Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
Editorial comment. . . . . . . . . . . . . . . . . . . . . . . . . . 468
Web-Only
Appendix
Appendix Tables
Appendix Figure
Conversion of graphics into slides
438 2012 American College of Physicians
Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015
Annals of Internal Medicine
net reclassification index, 19.3% overall [39.3% in those at inter-
mediate risk, by FRS]). Levels of N-terminal fragment of prohor-
mone B-type natriuretic peptide also improved risk predictions but
to a lesser extent (c-statistic increase, 0.02 [CI, 0.01 to 0.04]; net
reclassification index, 7.6% overall [33.0% in those at intermediate
risk, by FRS]). Improvements in predictions with other newer mark-
ers were marginal.
Limitation: The findings may not be generalizable to younger or
nonwhite populations.
Conclusion: Among 12 CHD risk markers, improvements in FRS
predictions were most statistically and clinically significant with the
addition of CAC scores. Further investigation is needed to assess
whether risk refinements using CAC scores lead to a meaningful
change in clinical outcome. Whether to use CAC score screening as
a more routine test for risk prediction requires full consideration of
the financial and clinical costs of performing versus not performing
the test for both persons and health systems.
Primary Funding Source: Netherlands Organization for Health Re-
search and Development (ZonMw)
Ann Intern Med. 2012;156:438-444. www.annals.org
For author affiliations, see end of text.
crement in risk prediction accuracy, if any, that these
newer risk markers contribute to the FRS and other stan-
dard risk-scoring systems (315).
Recent guidelines recommend that several measures be
used for assessing the increment in risk prediction accuracy
offered by newer risk markers (16), but few studies have im-
plemented those recommendations and none has used them
to compare multiple markers within the same cohort (5 8,
10, 11, 1315). Use of only some of the recommended as-
sessment methods may provide misleading impressions
of the clinical utility of novel markers in CHD risk
prediction (17, 18).
Therefore, we sought to compare the change in the
accuracy of risk predictions when newer risk markers rep-
resentative of various pathophysiologic pathways, includ-
ing several subclinical measures of atherosclerosis, were
added to the established clinical risk predictors. To that
end, we implemented the recent methods recommended
for assessment of risk prediction models (19, 20). These
assessments were performed in a community-dwelling pop-
ulation and in a subpopulation at intermediate risk for
 Improvement of CHD Risk Classification
CHD, in whom an increase in accuracy of risk predictions
may be most clinically relevant.
METHODS
Study Population
The study was embedded within the Rotterdam Study,
a prospective population-based cohort of persons aged 55
years or older in the municipality of Rotterdam, the Neth-
erlands. The rationale and design of the study have been
described elsewhere (21). The baseline examination was
completed between 1990 and 1993 (Rotterdam Study-I).
In 1999, the cohort was extended to include inhabitants
who reached the age of 55 years after the baseline exami-
nation and persons aged 55 years or older who migrated
into the research area (Rotterdam Study-II). Baseline par-
ticipation for all RS cycles was 72% (14 926 of 20 744).
The present study used data obtained from 6498 partici-
pants at the third examination of the original cohort (Rot-
terdam Study-I) (1997 to 1999) and the first examination
of the extended cohort (Rotterdam Study-II) (2000 to
2001). After excluding 565 participants with a history of
CHD (defined as clinically manifest myocardial infarction,
coronary artery bypass grafting, or percutaneous translumi-
nal coronary angioplasty), we had data from 5933 asymp-
tomatic participants. C-reactive protein (CRP) (n 3029)
and coronary artery calcium (CAC) score (n 3678) mea-
surements were available for a smaller group. The study
was approved by the Medical Ethics Committee of Eras-
mus University, Rotterdam, the Netherlands, and all par-
ticipants gave written informed consent.
Risk Factors
We studied traditional cardiovascular risk factors, such
as age, sex, body mass index, systolic blood pressure, treat-
ment of hypertension, total and high-density lipoprotein
cholesterol levels, use of lipid-lowering medication, smok-
ing, and diabetes mellitus.
We also studied newer risk factors, such as levels of
N-terminal fragment of prohormone B-type natriuretic
peptide (NT-proBNP), von Willebrand factor antigen lev-
els, fibrinogen levels, chronic kidney disease (CKD) (esti-
mated glomerular filtration rate 60 mL/min per 1.73
m2), leukocyte count, CRP levels, homocysteine levels, uric
acid levels, CAC scores, carotid intimamedia thickness
(cIMT), peripheral arterial disease, and pulse wave velocity.
The Appendix (available at www.annals.org) describes
the methods for measuring all of the risk factors.
Clinical Outcomes
We obtained information on study outcomes from
general practitioners and from letters and discharge reports
from medical specialists. Events were classified by study
physicians. Incident CHD was defined as a definite non-
fatal or fatal myocardial infarction or death due to CHD.
Definite and possible fatal CHD were coded by using the
definitions applied within the Cardiovascular Health Study
www.annals.org
Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015
Original Research
Context
Newer coronary risk factors strongly associated with car-
diovascular events are not necessarily useful for distin-
guishing between individuals who will and will not have
events.
Contribution
In this comparison of what 12 newer coronary risk factors
contribute to risk distinctions using standard risk factors,
only coronary artery calcium score and possibly N-terminal
fragment of prohormone B-type natriuretic peptide led to
clinically meaningful improvements.
Caution
The findings derive from a white European population.
Implication
Coronary artery calcium measurement may be the only
newer coronary risk factor to add meaningful information
to standard risks, such as smoking and diabetes. But mea-
suring coronary artery calcium is expensive and exposes
individuals to radiation, so its use for coronary risk assess-
ment requires further evaluation.
The Editors
(22) and Atherosclerosis Risk in Communities Study (23).
Only first CHD events were included in the analyses; 20
participants were lost to follow-up.
Statistical Analysis
We assessed the independent relationship of each
marker to CHD incidence by using Cox proportional haz-
ards models. We transformed markers with highly skewed
distributions to the natural logarithmic scale, and for CAC
score we used the natural logarithm of (CAC1) to handle
CAC scores of zero. To determine the functional form
used for each marker, we examined restricted cubic spline
transformations for continuous predictors (24) and used
the likelihood ratio test to examine the linearity assump-
tion. If appropriate, we chose the simplest form, usually a
log-linear term. All models met the proportional hazards
assumption as evaluated by the cox.zph function in R,
version 2.10.1 (R Foundation for Statistical Computing,
Vienna, Austria). Multivariable-adjusted hazard ratios
(HRs) were calculated for comparison of the highest versus
the lowest quartile (as the reference) for each marker.
Because median follow-up in the cohort was 6.8 years
and most CHD risk prediction instruments, including the
FRS, predict 10-year CHD risk, we used a parametric
Weibull proportional hazards regression model to estimate
10-year CHD risk from data available over a shorter
follow-up period for each person. We first fit a Weibull
prediction model to our data on the basis of the variables
used in the FRS (age, sex, systolic blood pressure, treat-
ment of hypertension, total and high-density lipoprotein
cholesterol levels, current smoking, and diabetes mellitus)
20 March 2012 Annals of Internal Medicine Volume 156 Number 6 439
 Original Research Improvement of CHD Risk Classification

base model, calculated by 100 bootstrap repetitions (24,

Table 1. Characteristics of the Study Population (n 5933)
25); and using the net reclassification improvement (NRI)
with the newer marker as suggested by Steyerberg and Pen-
Variable Value* Missing
Values, cina for survival data (26). The c-statistic is a measure of
% discrimination (the ability to distinguish between 2 per-
Characteristic sons, 1 with and 1 without a CHD event), and NRI spec-
Age, y 69.1 (8.5) 0.0
Women, n (%) 3525 (59.4) 0.0 ifies the amount of correct reclassification of estimated (not
Systolic blood pressure, 143.3 (21.3) 0.6 actual) events and nonevents to 10 years. The NRI esti-
mm Hg
Diastolic blood pressure, 77.0 (11.2) 0.6 mates were based on the reclassification tables classifying
mm Hg
Antihypertensive treatment, 1392 (23.5) 5.0 participants in 10-year CHD risk categories of low
n (%) (10%), intermediate (10% to 20%), and high (20%).
Body mass index, kg/m2 27.0 (4.0) 1.9
Total cholesterol level We then repeated all of the analyses for men and women
mmol/L 5.8 (1.0) 1.2 separately.
mg/dL 225.6 (37.6) 1.2
HDL cholesterol level Information on some markers and covariables was
mmol/L 1.4 (0.4) 2.0
mg/dL 54.0 (15.1) 2.0 missing in up to 13% of participants. We performed mul-
Triglyceride level tiple imputations of the missing values by using the Hmisc
mmol/L 1.5 (0.8) 0.0
mg/dL 136.5 (70.3) 0.0 library of R (R-library: Hmisc, function: aregImpute). All
Lipid-lowering medication, 605 (10.2) 5.0
n (%)
analyses were performed with R, version 2.10.1. A 2-sided
Glucose level P value of less than 0.05 denoted statistical significance.
mmol/L 5.9 (1.5) 1.2
mg/dL 107.1 (27.6) 1.2 Role of the Funding Source
Diabetes mellitus, n (%) 767 (12.9) 1.2
Current smoking, n (%) 1037 (17.5) 1.5 The Rotterdam Study is funded by Erasmus Medical
Creatinine level
mol/L 76.0 (66.0, 87.0) 0.0 Center and Erasmus University, Rotterdam; the Research
mg/dL 0.9 (0.7, 1.0) 0.0 Institute for Diseases in the Elderly; the Ministry of Edu-
Novel risk markers cation, Culture and Science; the Ministry for Health, Wel-
NT-proBNP level, pmol/L 9.5 (5.1, 18.1) 0.0 fare and Sports; the European Commission (DG XII); and
vWF antigen level 1.8
IU/mL 1.2 (0.9, 1.6) the Municipality of Rotterdam. The current study is sup-
mIU/L 11.9 105 (9.2 105, 15.8 105)
Fibrinogen level, mol/L 11.2 (9.7, 12.9) 1.9 ported by the Netherlands Heart Foundation, Netherlands
eGFR, mL/min per 1.73 m2 76.1 (67.0, 86.9) 0.0 Organization for Scientific Research (NWO), Netherlands
CKD, n (%) 725 (12.2) 0.0
Leukocyte count, 6.8 (1.9) 1.2 Organization for Health Research and Development
109 cells/L
CRP level, mg/L 2.3 (1.2, 4.4) NA
(ZonMw), and Netherlands Consortium for Healthy Age-
Homocysteine level, 13.5 (11.4, 16.6) 12.7 ing, with additional funding from Vereniging Trustfonds
mol/L
Uric acid level, mol/L 300.0 (260.0, 360.0) 0.0 Erasmus Universiteit Rotterdam. The funding sources had
CAC score 65.8 (4.4, 322.8) NA no influence on study design; collection, analysis, or inter-
cIMT, mm 1.0 (0.9, 1.1) 5.1
ABI 1.1 (0.2) 5.0 pretation of the data or approval of the manuscript.
PAD, n (%) 830 (14) 5.0
PWV, m/s 12.6 (10.9, 14.8) 10.8

ABI ankle-brachial index; CAC coronary artery calcium; cIMT carotid

intimamedia thickness; CKD chronic kidney disease; CRP C-reactive pro-
tein; eGFR estimated glomerular filtration rate; HDL high-density lipopro-
tein; NA not applicable; NT-proBNP N-terminal fragment of prohormone
B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave
velocity; vWF von Willebrand factor.
* Values are means (SDs) or medians (25th, 75th percentiles), unless otherwise
reported. Median is reported in cases of skewed distribution.
Percentage of individuals with missing values for the particular variable, which
was imputed for the analyses.
For CRP (n 3029) and CAC score (n 3678), no imputation was done and
the analyses for these 2 markers were done with the actual numbers.
(1). We refer to this model as the base model. We then
developed 12 new models, each containing the FRS vari-
ables with the addition of 1 of the 12 newer risk markers
(referred to as the newer marker model). The analyses
including CAC score were additionally adjusted for the
type of computed tomography scanner.
We compared the base model with the newer marker
models by using the likelihood ratio chi-square test for
global model fit; using the difference in the optimism-
corrected c-statistic between each newer model and the
440 20 March 2012 Annals of Internal Medicine Volume 156 Number 6
RESULTS
Mean participant age was 69.1 years (SD, 8.5), and
59.4% were women. Table 1 reports the baseline values
for traditional and newer risk markers. Demographic
characteristics or risk factor values did not differ be-
tween the overall population and the subpopulation
with CRP levels and CAC scores (data not shown). Dur-
ing a median follow-up of 6.8 years (25th, 75th percen-
tiles: 5.8, 8.1 years), 347 first CHD events, including
190 nonfatal myocardial infarctions and 157 CHD
deaths, occurred (Table 2).
Appendix Table 1 (available at www.annals.org) de-
tails HRs for incident CHD for traditional risk factors in
the base model, as well as the performance measures for the
base model (global model fit and c-statistic). The Figure
illustrates adjusted HRs for comparisons of the highest ver-
sus lowest quartiles for each newer marker. Newer risk
markers that were significantly associated with incident
CHD after adjustment for traditional risk factors were
NT-proBNP levels (HR, 2.5 [95% CI, 1.7 to 3.6]), fibrin-
www.annals.org

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Improvement of CHD Risk Classification Original Research

ogen levels (HR, 1.4 [CI, 1.1 to 2.0]), CKD (HR, 1.4 [CI,
Table 2. Incidence of CHD Events in the Total Population
1.1 to 1.8]), leukocyte count (HR, 1.8 [CI, 1.3 to 2.5]),
(n 5933) During a Median Follow-up of 6.8 Years
CRP levels (HR, 1.6 [CI, 1.0 to 2.5]), homocysteine levels
(HR, 1.4 [CI, 1.0 to 2.0]), CAC scores (HR, 6.2 [CI, 3.4
Event Total Events, n Events per 1000
to 11.5]), cIMT (HR, 1.6 [CI, 1.1 to 2.3]), and peripheral Person-Years, n
arterial disease (HR, 1.3 [CI, 1.0 to 1.7]). The Appendix Nonfatal MI 190 4.80
Figure (available at www.annals.org) illustrates the ad- Fatal CHD 157 3.97
justed HRs (95% CIs) for each newer marker, by sex. Definite fatal MI 30 0.76
Definite fatal CHD 88 2.22
Appendix Table 2 (available at www.annals.org) de- Possible fatal CHD 39 0.99
tails measures of improvement in model fit, for the overall
population and by sex, when each newer risk marker is CHD coronary heart disease; MI myocardial infarction.
added to the base model. In the overall population, statis-
tically significant improvements were observed for NT-
gorized at intermediate risk for a CHD event on the basis
proBNP levels, fibrinogen levels, CKD, leukocyte count,
of the base model. In this subpopulation, NRI was highest
CRP levels, homocysteine levels, CAC scores, cIMT, and
for CAC score (39.3% [CI, 26.8% to 51.7%]), followed
peripheral arterial disease.
by NT-proBNP level (33.0% [CI, 23.4% to 42.6%]). All
Table 3 summarizes the change in the c-statistic and
other markers resulted in NRIs of 10% or less in this sub-
the overall NRI when each newer risk marker is added to
population. Appendix Tables 4 and 5 (available at www
the base model. The maximum change in the c-statistic
.annals.org) detail these results, by sex. With some excep-
was observed for CAC score (increase, 0.05 [CI, 0.02 to
tions, NRIs tended to be higher for men than for women,
0.06]), followed by NT-proBNP level (increase, 0.02 [CI,
corresponding to the stronger associations (HRs) between
0.01 to 0.04]). The highest overall net percentage of per-
the newer risk markers and CHD events in men. Appendix
sons correctly reclassified was also observed for CAC score
Table 6 (available at www.annals.org) details CHD inci-
(NRI, 19.3% [CI, 12.5% to 26.2%]), with a smaller NRI
dence in the subpopulation with the CAC score. The com-
for NT-proBNP (7.6% [CI, 2.8% to 12.5%]). Changes in
plete reclassification table for CAC score is presented in
c-statistics and overall NRIs in total population were oth-
Appendix Table 7 (available at www.annals.org).
erwise negligible or absent for every newer marker. The
results for the c-statistic changes and NRI, by sex, are pro-
vided in Appendix Table 3 (available at www.annals.org). DISCUSSION
Table 4 summarizes the overall NRI when newer risk Among 12 newer CHD risk markers, CAC score pro-
markers are added to the base model for the persons cate- vided the best improvement in CHD risk prediction and

Figure. Multivariable-adjusted HR for incident coronary heart disease.

Marker HR (95% CI)

NT-proBNP 2.5 (1.73.6)

vWF antigen 1.2 (0.91.6)
Fibrinogen 1.4 (1.12.0)
CKD 1.4 (1.11.8)
Leukocyte count 1.8 (1.32.5)
CRP 1.6 (1.02.5)
Homocysteine 1.4 (1.02.0)
Uric acid 0.9 (0.71.3)
CAC score 6.2 (3.411.5)
cIMT 1.6 (1.12.3)
PAD 1.3 (1.01.7)
PWV 1.2 (0.81.8)

0.75 1.0 1.5 6.0

HR (95% CI)

CAC coronary artery calcium; cIMT carotid intimamedia thickness; CKD chronic kidney disease; CRP C-reactive protein; HR hazard
ratio; NT-proBNP N-terminal fragment of prohormone B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave velocity;
vWF von Willebrand factor.
www.annals.org 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 441

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Original Research Improvement of CHD Risk Classification

proBNP compared with other serum biomarkers may cor-

Table 3. Discriminative Ability of Risk Markers
respond to its position in the later stages of the disease
process. Specifically, an increase in BNP levels is viewed by
Marker Change in c-Statistic NRI (95% CI) for Total
(95% CI)* Population, % some as a response to age-related, subclinical alterations in
NT-proBNP 0.02 (0.01 to 0.04) 7.6 (2.8 to 12.5)
cardiac structure or function (33), so this biomarker may
vWF antigen 0.00 (0.00 to 0.00) 0.4 (1.7 to 2.5) be more useful for CHD risk prediction at older ages.
Fibrinogen 0.00 (0.00 to 0.01) 2.9 (0.2 to 6.0) However, the 95% CIs for NRI with NT-proBNP overlap
CKD 0.00 (0.00 to 0.00) 2.7 (0.2 to 5.7)
Leukocyte count 0.01 (0.00 to 0.02) 1.5 (1.5 to 4.6)
with those surrounding NRIs from other markers, making
CRP 0.00 (0.01 to 0.00) 2.0 (2.3 to 6.4) it difficult to conclude that NT-proBNP is superior to
Homocysteine 0.00 (0.00 to 0.00) 0.3 (3.0 to 2.3) other markers. Nevertheless, our results suggest a potential
Uric acid 0.00 (0.00 to 0.00) 0.8 (0.5 to 2.1)
CAC score 0.05 (0.02 to 0.06) 19.3 (12.5 to 26.2)
role of NT-proBNP for inclusion in CHD risk prediction
cIMT 0.00 (0.00 to 0.00) 1.6 (1.1 to 4.4) instruments.
PAD 0.00 (0.00 to 0.00) 0.6 (1.8 to 2.9) Although other biomarkers, such as fibrinogen levels,
PWV 0.00 (0.00 to 0.00) 0.00 (2.1 to 2.1)
CKD, leukocyte count, CRP levels, and homocysteine lev-
CAC coronary artery calcium; cIMT carotid intimamedia thickness; els, were independently associated with the risk for later
CKD chronic kidney disease; CRP C-reactive protein; NRI net reclassi- coronary events, their incremental value beyond traditional
fication improvement; NT-proBNP N-terminal fragment of prohormone
B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave risk factors was marginal. Addition of these biomarkers to
velocity; vWF von Willebrand factor. the base model yielded NRIs in the intermediate-risk
* Increment in the c-statistic in the extended model (with conventional risk factors
and the newer risk marker) vs. the base model (with conventional risk factors group of 10% or less, with lower confidence bounds below
alone). The c-statistic is corrected for overoptimism by using 100 bootstrap reclassification thresholds that would probably be consid-
repetitions.
Percentage of NRI (95% CI) for the extended model (with conventional risk ered clinically useful.
factors and the newer risk marker) vs. the base model (with conventional risk Strengths of the current study include its large sample
factors alone), using risk categories of 10%, 10%20%, and 20%. It is esti-
mated as [(number of events reclassified higher minus number of events reclassified size, comparison of multiple markers that were all mea-
lower)]/number of events [(number of nonevents reclassified lower minus num- sured with standardized methods, use of hard end points to
ber of nonevents reclassified higher/number of nonevents)].
The analyses for CRP (n 3029) and CAC score (n 3678) were performed avoid misclassification bias, and implementation of various
in a smaller group. statistical measures for assessment of risk-scoring models.

Table 4. Coronary Heart Disease Risk Reclassification for

stratification above the FRS, as measured by an increase in the Framingham Intermediate-Risk Group After Extending
the c-statistic and clinically significant reclassification in the Model With Each Marker
the overall population and in persons categorized as inter-
mediate risk by traditional cardiovascular risk factors. The Extended Model Persons With Persons NRI (95% CI), %
finding is consistent with those of previous studies (3, 4, Event, %* Without
Event, %*
27).
Improvements in CHD risk prediction with other Up Down Up Down
newer risk markers, including cIMT, ankle brachial index, FRS NT-proBNP 29.0 16.4 9.8 30.2 33.0 (23.4 to 42.6)
and pulse wave velocity, which have been shown to be FRS vWF antigen 4.4 3.9 2.0 5.5 4.0 (0.2 to 8.1)
FRS fibrinogen 10.6 4.8 5.9 10.3 10.2 (4.5 to 15.9)
strong predictors of CHD in other studies (6, 9, 28 30), FRS CKD 10.0 3.2 5.4 8.4 9.8 (4.4 to 15.1)
were modest. The better performance of CAC score com- FRS leukocyte 11.5 6.4 6.0 10.2 9.3 (3.2 to 15.4)
pared with other vascular measures of atherosclerosis prob- count
FRS CRP 12.3 10.4 4.9 12.2 9.2 (0.2 to 18.0)
ably reflects the disparity in contribution of various vascu- FRS homocysteine 7.6 7.8 3.7 8.6 4.7 (0.9 to 10.3)
lar beds in the disease process. However, because of FRS uric acid 0.8 0.0 0.5 2.3 2.6 (1.0 to 4.2)
variations across studies in the number of risk categories FRS CAC score 37.0 13.0 18.7 34.0 39.3 (26.8 to 51.7)
FRS cIMT 6.0 4.2 4.3 7.1 4.6 (0.1 to 9.3)
and thresholds and in clinical outcomes of interest, it is FRS PAD 6.8 2.1 4.2 6.8 7.3 (2.9 to 11.7)
difficult to make direct comparisons of our findings with FRS PWV 4.4 2.8 2.1 3.7 3.2 (0.6 to 7.1)
those of other population studies.
CAC coronary artery calcium; cIMT carotid intimamedia thickness;
A relatively new risk marker, NT-proBNP, has been CKD chronic kidney disease; CRP C-reactive protein; FRS Framingham
shown to be a strong predictor of coronary events and risk score; NRI net reclassification improvement; NT-proBNP N-terminal
fragment of prohormone B-type natriuretic peptide; PAD peripheral arterial
death (11, 13, 31). Among serum biomarkers, NT- disease; PWV pulse wave velocity; vWF von Willebrand factor.
proBNP was most associated with CHD events and led to * Percentages of persons with or without an event within the intermediate-risk
category who move to the high- or low-risk category after extension of the base
the greatest NRI both overall and among participants cat- model with the specific marker.
egorized as being at intermediate risk by traditional cardio- Percentage of NRI (95% CI) for the intermediate-risk category after extension of
the base model with the specific marker. It is estimated as [(number of events
vascular risk factors. Because elevation of various bio- reclassified higher minus number of events reclassified lower/number of events)]
marker levels correlates with the various phases of the [(number of nonevents reclassified lower minus number of nonevents reclassified
higher/number of nonevents)].
atherogenesis cascade (32), we speculate that the greater The analyses for CRP (n 3029) and CAC score (n 3678) were performed
CHD risk prediction and reclassification with NT- in a smaller group.

442 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 www.annals.org

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Improvement of CHD Risk Classification
There are also limitations. Our cohort comprised
white participants aged 55 years or older; therefore, the
generalizability of our findings to younger and nonwhite
populations remains uncertain. Also, due to the differ-
ence in the age range of our population compared with the
Framingham population, we refit a model on the basis of
the Framingham variables in our own population instead
of using the original FRS algorithm. This implies that our
results may differ from the settings that directly apply pub-
lished versions of the FRS or its modifications. We fit the
Weibull model to estimate 10-year predicted risk for CHD
from our actual median follow-up of 6.8 years. That ex-
trapolation cannot be validated. Moreover, CAC score
measurements in our study were performed with 2 types of
scanners (electron beam and multidetector computed to-
mography). However, analyses using CAC score were ad-
justed for the type of scanner, and our results are similar to
our previous findings using the data from only our electron
beam computed tomography cohort (27). Measurements
of CRP levels and CAC score were available only in a
subpopulation; however, general characteristics of that sub-
population did not materially differ from those of the
larger population. Finally, we performed many statistical
analyses to compare risk markers, and some of our statisti-
cally significant findings may have occurred by chance.
However, our results show a consistent pattern for CAC
score across statistical methods.
In summary, in this large population-based study, im-
provements in CHD risk prediction and reclassification
with CAC score were statistically and clinically significant.
Increments in accuracy with other newer risk markers were
less significant. Further investigation is still needed to assess
whether risk refinements using CAC score lead to a mean-
ingful change in clinical outcome. Moreover, whether to
include any newer test in a risk prediction algorithm re-
quires full consideration of the financial and clinical costs
of performing versus not performing the new test for both
people and health systems. For CAC score in particular,
exposing potentially healthy populations to radiation in a
screening program requires careful considerations of the
balance of risks and benefits.
From Erasmus University Medical Center, Rotterdam, and University
Medical Center Groningen, Groningen, the Netherlands.
Acknowledgment: The investigators are grateful to the participants and
staff from the Rotterdam Study, participating general practitioners, and
pharmacists.
Financial Support: The Rotterdam Study is funded by Erasmus Univer-
sity Medical Center and Erasmus University, Rotterdam, the Nether-
lands; the Research Institute for Diseases in the Elderly; the Ministry of
Education, Culture and Science; the Ministry for Health, Welfare and
Sports; the European Commission (DG XII); and the Municipality of
Rotterdam. This study was supported by grants from the Netherlands
Heart Foundation (2003B179 [Dr. Witteman], 2007B159 [Dr.
Leebeek]), the Netherlands Organization for Scientific Research
(NWO), the Netherlands Organization for Health Research and Devel-
www.annals.org
Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015
Original Research
opment (ZonMw) (Vici grant 918.76.619), and the Netherlands Con-
sortium for Healthy Ageing. The authors received additional funding
from Vereniging Trustfonds Erasmus Universiteit Rotterdam.
Potential Conflicts of Interest: None disclosed. Forms can be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNumM11
-0450.
Reproducible Research Statement: Study protocol and data set: Not
available. Statistical code: Available from Dr. Witteman (e-mail, j.witteman
@erasmusmc.nl).
Requests for Single Reprints: Jacqueline C.M. Witteman, PhD, De-
partment of Epidemiology, Erasmus University Medical Center, Dr.
Molewaterplein 50, 3015 GE, Rotterdam, the Netherlands; e-mail, j.witteman
@erasmusmc.nl.
Current author addresses and author contributions are available at
www.annals.org.
References
1. Wilson PW, DAgostino RB, Levy D, Belanger AM, Silbershatz H, Kannel
WB. Prediction of coronary heart disease using risk factor categories. Circulation.
1998;97:1837-47. [PMID: 9603539]
2. National Cholesterol Education Program (NCEP) Expert Panel on Detec-
tion, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult
Treatment Panel III). Third Report of the National Cholesterol Education Pro-
gram (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High
Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation.
2002;106:3143-421. [PMID: 12485966]
3. Polonsky TS, McClelland RL, Jorgensen NW, Bild DE, Burke GL, Guerci
AD, et al. Coronary artery calcium score and risk classification for coronary heart
disease prediction. JAMA. 2010;303:1610-6. [PMID: 20424251]
4. Erbel R, Mohlenkamp S, Moebus S, Schmermund A, Lehmann N, Stang A,
et al; Heinz Nixdorf Recall Study Investigative Group. Coronary risk stratifica-
tion, discrimination, and reclassification improvement based on quantification of
subclinical coronary atherosclerosis: the Heinz Nixdorf Recall study. J Am Coll
Cardiol. 2010;56:1397-406. [PMID: 20946997]
5. Folsom AR, Chambless LE, Ballantyne CM, Coresh J, Heiss G, Wu KK,
et al. An assessment of incremental coronary risk prediction using C-reactive
protein and other novel risk markers: the atherosclerosis risk in communities
study. Arch Intern Med. 2006;166:1368-73. [PMID: 16832001]
6. Folsom AR, Kronmal RA, Detrano RC, OLeary DH, Bild DE, Bluemke
DA, et al. Coronary artery calcification compared with carotid intima-media
thickness in the prediction of cardiovascular disease incidence: the Multi-Ethnic
Study of Atherosclerosis (MESA). Arch Intern Med. 2008;168:1333-9. [PMID:
18574091]
7. Kim HC, Greenland P, Rossouw JE, Manson JE, Cochrane BB, Lasser NL,
et al. Multimarker prediction of coronary heart disease risk: the Womens Health
Initiative. J Am Coll Cardiol. 2010;55:2080-91. [PMID: 20447530]
8. Melander O, Newton-Cheh C, Almgren P, Hedblad B, Berglund G, Eng-
strom G, et al. Novel and conventional biomarkers for prediction of incident
cardiovascular events in the community. JAMA. 2009;302:49-57. [PMID:
19567439]
9. Nambi V, Chambless L, Folsom AR, He M, Hu Y, Mosley T, et al. Carotid
intima-media thickness and presence or absence of plaque improves prediction of
coronary heart disease risk: the ARIC (Atherosclerosis Risk In Communities)
study. J Am Coll Cardiol. 2010;55:1600-7. [PMID: 20378078]
10. Rana JS, Cote M, Despres JP, Sandhu MS, Talmud PJ, Ninio E, et al.
Inflammatory biomarkers and the prediction of coronary events among people at
intermediate risk: the EPIC-Norfolk prospective population study. Heart. 2009;
95:1682-7. [PMID: 19587389]
11. Wang TJ, Gona P, Larson MG, Tofler GH, Levy D, Newton-Cheh C,
et al. Multiple biomarkers for the prediction of first major cardiovascular events
and death. N Engl J Med. 2006;355:2631-9. [PMID: 17182988]
12. Wilson PW, Pencina M, Jacques P, Selhub J, DAgostino R Sr, ODonnell
20 March 2012 Annals of Internal Medicine Volume 156 Number 6 443
 Original Research Improvement of CHD Risk Classification
CJ. C-reactive protein and reclassification of cardiovascular risk in the Framing-
ham Heart Study. Circ Cardiovasc Qual Outcomes. 2008;1:92-7. [PMID:
20031795]
13. Tzoulaki I, Murray GD, Lee AJ, Rumley A, Lowe GD, Fowkes FG. Rela-
tive value of inflammatory, hemostatic, and rheological factors for incident myo-
cardial infarction and stroke: the Edinburgh Artery Study. Circulation. 2007;115:
2119-27. [PMID: 17404162]
14. Woodward M, Rumley A, Welsh P, MacMahon S, Lowe G. A comparison
of the associations between seven hemostatic or inflammatory variables and cor-
onary heart disease. J Thromb Haemost. 2007;5:1795-800. [PMID: 17723116]
15. Zethelius B, Berglund L, Sundstrom J, Ingelsson E, Basu S, Larsson A,
et al. Use of multiple biomarkers to improve the prediction of death from car-
diovascular causes. N Engl J Med. 2008;358:2107-16. [PMID: 18480203]
16. Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind
MS, et al; American Heart Association Expert Panel on Subclinical Atheroscle-
rotic Diseases and Emerging Risk Factors and the Stroke Council. Criteria for
evaluation of novel markers of cardiovascular risk: a scientific statement from
the American Heart Association. Circulation. 2009;119:2408-16. [PMID:
19364974]
17. Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved
prediction beyond the Framingham risk score. JAMA. 2009;302:2345-52.
[PMID: 19952321]
18. Helfand M, Buckley DI, Freeman M, Fu R, Rogers K, Fleming C, et al.
Emerging risk factors for coronary heart disease: a summary of systematic reviews
conducted for the U.S. Preventive Services Task Force. Ann Intern Med. 2009;
151:496-507. [PMID: 19805772]
19. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski
N, et al. Assessing the performance of prediction models: a framework for tradi-
tional and novel measures. Epidemiology. 2010;21:128-38. [PMID: 20010215]
20. Pencina MJ, DAgostino RB Sr, Steyerberg EW. Extensions of net reclassi-
fication improvement calculations to measure usefulness of new biomarkers. Stat
Med. 2011;30:11-21. [PMID: 21204120]
21. Hofman A, van Duijn CM, Franco OH, Ikram MA, Janssen HL, Klaver
CC, et al. The Rotterdam Study: 2012 objectives and design update. Eur J
Epidemiol. 2011;26:657-86. [PMID: 21877163]
22. Pearte CA, Furberg CD, OMeara ES, Psaty BM, Kuller L, Powe NR, et al.
Characteristics and baseline clinical predictors of future fatal versus nonfatal cor-
onary heart disease events in older adults: the Cardiovascular Health Study. Cir-
ACP CHAPTER MEETINGS
For information on upcoming ACP chapter meetings, including scientific
programs and registration forms, please visit www.acponline.org
/meetings/chapter.
444 20 March 2012 Annals of Internal Medicine Volume 156 Number 6
Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015
culation. 2006;113:2177-85. [PMID: 16651468]
23. White AD, Folsom AR, Chambless LE, Sharret AR, Yang K, Conwill D,
et al. Community surveillance of coronary heart disease in the Atherosclerosis
Risk in Communities (ARIC) Study: methods and initial two years experience.
J Clin Epidemiol. 1996;49:223-33. [PMID: 8606324]
24. Harrell FE Jr. Regression Modeling Strategies: With Application to Linear
Models, Logistic Regression, and Survival Analysis. New York: Springer Publish-
ing; 2001.
25. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Devel-
opment, Validation, and Updating. New York: Springer Publishing; 2009.
26. Steyerberg EW, Pencina MJ. Reclassification calculations for persons with
incomplete follow-up [Letter]. Ann Intern Med. 2010;152:195-6. [PMID:
20124243]
27. Elias-Smale SE, Proenca RV, Koller MT, Kavousi M, van Rooij FJ, Hunink
MG, et al. Coronary calcium score improves classification of coronary heart
disease risk in the elderly: the Rotterdam study. J Am Coll Cardiol. 2010;56:
1407-14. [PMID: 20946998]
28. Fowkes FG, Murray GD, Butcher I, Heald CL, Lee RJ, Chambless LE,
et al; Ankle Brachial Index Collaboration. Ankle brachial index combined with
Framingham Risk Score to predict cardiovascular events and mortality: a meta-
analysis. JAMA. 2008;300:197-208. [PMID: 18612117]
29. Mattace-Raso FU, van der Cammen TJ, Hofman A, van Popele NM, Bos
ML, Schalekamp MA, et al. Arterial stiffness and risk of coronary heart disease
and stroke: the Rotterdam Study. Circulation. 2006;113:657-63. [PMID:
16461838]
30. Mitchell GF, Hwang SJ, Vasan RS, Larson MG, Pencina MJ, Hamburg
NM, et al. Arterial stiffness and cardiovascular events: the Framingham Heart
Study. Circulation. 2010;121:505-11. [PMID: 20083680]
31. Rutten JH, Mattace-Raso FU, Steyerberg EW, Lindemans J, Hofman A,
Wieberdink RG, et al. Amino-terminal pro-B-type natriuretic peptide improves
cardiovascular and cerebrovascular risk prediction in the population: the Rotter-
dam study. Hypertension. 2010;55:785-91. [PMID: 20083731]
32. Vasan RS. Biomarkers of cardiovascular disease: molecular basis and practical
considerations. Circulation. 2006;113:2335-62. [PMID: 16702488]
33. Redfield MM, Rodeheffer RJ, Jacobsen SJ, Mahoney DW, Bailey KR,
Burnett JC Jr. Plasma brain natriuretic peptide concentration: impact of age and
gender. J Am Coll Cardiol. 2002;40:976-82. [PMID: 12225726]
www.annals.org

Current Author Addresses: Drs. Kavousi, Elias-Smale, Leening, Verwo-
ert, Hofman, and Witteman: Department of Epidemiology, Erasmus
University Medical Center, Dr. Molewaterplein 50, 3015 GE, Rotter-
dam, the Netherlands.
Drs. Rutten, Mattace-Raso, and van den Meiracker: Department of In-
ternal Medicine, Erasmus University Medical Center, Dr. Molewater-
plein 50, 3015 GE, Rotterdam, the Netherlands.
Dr. Vliegenthart: Department of Radiology, University Medical Center
Groningen, Hanzeplein 1, 9713 GZ, Groningen, the Netherlands.
Drs. Krestin and van der Lugt: Department of Radiology, Erasmus Uni-
versity Medical Center, Dr. Molewaterplein 50, 3015 GE, Rotterdam,
the Netherlands.
Dr. Oudkerk: Center for Medical ImagingNorth East Netherlands,
Department of Radiology, University Medical Center Groningen, Han-
zeplein 1, 9713 GZ, Groningen, the Netherlands.
Drs. de Maat and Leebeek: Department of Hematology, Erasmus Uni-
versity Medical Center, Dr. Molewaterplein 50, 3015 GE, Rotterdam,
the Netherlands.
Dr. Lindemans: Department of Clinical Chemistry, Erasmus University
Medical Center, Dr. Molewaterplein 50, 3015 GE, Rotterdam, the
Netherlands.
Dr. Steyerberg: Department of Public Health, Erasmus University
Medical Center, Dr. Molewaterplein 50, 3015 GE, Rotterdam, the
Netherlands.
Author Contributions: Conception and design: M. Kavousi, M. Oud-
kerk, M.P.M. de Maat, E.W. Steyerberg, J.C.M. Witteman.
Analysis and interpretation of the data: M. Kavousi, J.H.W. Rutten, R.
Vliegenthart, M. Oudkerk, E.W. Steyerberg, A. van der Lugt, A.H. van
den Meiracker.
Drafting of the article: M. Kavousi, F.U.S. Mattace-Raso, A. van der
Lugt, J.C.M. Witteman.
Critical revision of the article for important intellectual content: M.
Kavousi, S. Elias-Smale, J.H.W. Rutten, M.J.G. Leening, R. Vliegent-
hart, G.P. Krestin, M. Oudkerk, M.P.M. de Maat, F.W.G. Leebeek,
F.U.S. Mattace-Raso, E.W. Steyerberg, A. van der Lugt, A.H. van den
Meiracker, J.C.M. Witteman.
Final approval of the article: M. Kavousi, S. Elias-Smale, J.H.W. Rutten,
R. Vliegenthart, G.C. Verwoert, M. Oudkerk, M.P.M. de Maat, F.W.G.
Leebeek, J. Lindemans, A. Hofman, E.W. Steyerberg, A. van der Lugt,
J.C.M. Witteman.
Provision of study materials or patients: S. Elias-Smale, J.C.M.
Witteman.
Statistical expertise: M. Kavousi, J.H.W. Rutten, E.W. Steyerberg.
Obtaining of funding: A. van der Lugt, J.C.M. Witteman.
Administrative, technical, or logistic support: M.J.G. Leening, M. Oud-
kerk, F.U.S. Mattace-Raso.
Collection and assembly of data: S. Elias-Smale, M.J.G. Leening, R.
Vliegenthart, G.C. Verwoert, M. Oudkerk, M.P.M. de Maat, F.W.G.
Leebeek, J. Lindemans, A. van der Lugt.
34. Kottgen A, Glazer NL, Dehghan A, Hwang SJ, Katz R, Li M, et al.
Multiple loci associated with indices of renal function and chronic kidney disease.
Nat Genet. 2009;41:712-7. [PMID: 19430482]
35. Fox CS, Larson MG, Leip EP, Culleton B, Wilson PW, Levy D. Predictors
of new-onset kidney disease in a community-based population. JAMA. 2004;291:
844-50. [PMID: 14970063]
36. Levey AS, Bosch JP, Lewis JB, Greene T, Rogers N, Roth D. A more
accurate method to estimate glomerular filtration rate from serum creatinine: a
new prediction equation. Modification of Diet in Renal Disease Study Group.
Ann Intern Med. 1999;130:461-70. [PMID: 10075613]
37. Levey AS, Coresh J, Balk E, Kausz AT, Levin A, Steffes MW, et al; Na-
tional Kidney Foundation. National Kidney Foundation practice guidelines for
chronic kidney disease: evaluation, classification, and stratification. Ann Intern
Med. 2003;139:137-47. [PMID: 12859163]
W-132 20 March 2012 Annals of Internal Medicine Volume 156 Number 6
Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015
Annals of Internal Medicine
38. Coresh J, Astor BC, McQuillan G, Kusek J, Greene T, Van Lente F, et al.
Calibration and random variation of the serum creatinine assay as critical ele-
ments of using equations to estimate glomerular filtration rate. Am J Kidney Dis.
2002;39:920-9. [PMID: 11979335]
39. Odink AE, van der Lugt A, Hofman A, Hunink MG, Breteler MM, Kres-
tin GP, et al. Risk factors for coronary, aortic arch and carotid calcification; The
Rotterdam Study. J Hum Hypertens. 2010;24:86-92. [PMID: 19494836]
40. Vliegenthart R, Oudkerk M, Hofman A, Oei HH, van Dijck W, van Rooij
FJ, et al. Coronary calcification improves cardiovascular risk prediction in the
elderly. Circulation. 2005;112:572-7. [PMID: 16009800]
41. Bots ML, Hoes AW, Koudstaal PJ, Hofman A, Grobbee DE. Common
carotid intima-media thickness and risk of stroke and myocardial infarction: the
Rotterdam Study. Circulation. 1997;96:1432-7. [PMID: 9315528]
42. Meijer WT, Hoes AW, Rutgers D, Bots ML, Hofman A, Grobbee DE.
Peripheral arterial disease in the elderly: The Rotterdam Study. Arterioscler
Thromb Vasc Biol. 1998;18:185-92. [PMID: 9484982]
43. Mohler ER 3rd. Peripheral arterial disease: identification and implications.
Arch Intern Med. 2003;163:2306-14. [PMID: 14581250]
APPENDIX: METHODS
Cardiovascular Risk Factors
Information on medical history, medication use, and smok-
ing behavior was collected by using a computerized question-
naire. Anthropometric measures were obtained during the visit at
the research center. Blood pressure was measured at the right
brachial artery with a random-zero sphygmomanometer with the
participant in sitting position, and the mean of 2 consecutive
measurements was used. Serum glucose and serum total and
high-density lipoprotein cholesterol levels were measured with
standard laboratory techniques. Diabetes mellitus was diagnosed
on the basis of a fasting plasma glucose level of 7.0 mmol/L or
higher (126.1 mg/dL), or use of antidiabetic medication.
Measurement of Markers
Levels of NT-proBNP were measured by using a commer-
cially available electrochemiluminescence immunoassay system
(Elecsys proBNP; F Hoffman-La Roche, Basel, Switzerland) on
an Elecsys 2010 analyzer. Levels of von Willebrand factor antigen
were measured with an in-house enzyme-linked immunosorbent
assay with polyclonal rabbit antihuman von Willebrand factor
antibodies and horseradishperoxidase conjugated antihuman
von Willebrand factor antibodies (DakoCytomation, Glostrup,
Denmark) for catching and tagging, respectively. Fibrinogen lev-
els were derived from the clotting curve of the prothrombin time
assay using Thromborel S (Behringwerke, Marburg, Germany) as
a reagent on an automated coagulation analyzer (Sysmex CA-500
Series Systems, Siemens, Breda, the Netherlands). Serum creati-
nine levels were measured by using an enzymatic assay (Roche
Diagnostics, Mannheim, Germany), which was calibrated by iso-
tope dilution mass spectrometry. Because creatinine measure-
ments can vary across laboratories, we first calibrated our mea-
sures. For this purpose, mean creatinine values from the
Rotterdam Study, by sex-specific age groups (60, 60 to 69, and
70 years), were aligned with the corresponding corrected means
from the NHANES III (Third National Health and Nutrition
Examination Survey) participants, as described previously (34).
The NHANES III creatinine measures were calibrated to the
Cleveland Clinic Laboratory, requiring a correction factor of 0.23
mg/dL (20.3 mol/L) (35). The glomerular filtration rate was
estimated by the abbreviated modification of diet in renal disease
www.annals.org
 equation (36) as recommended by the National Kidney Founda-
tion (37). Estimated glomerular filtration rate (in mL/min per
1.73 m2) 186 [serum creatinine (in mg/dL)1.154
age0.203 0.742 (if female) 1.210 (if black)] (38). Chronic
kidney disease was defined as estimated glomerular filtration rate
of less than 60 mL/min per 1.73 m2. Leukocyte count was as-
sessed with an automated blood cell counter (Coulter Counter
T660; Beckman Coulter, Fullerton, California) using Beckman
reagents. Levels of CRP were measured by using a nephelometric
method with the rate near infrared particle immunoassay (Im-
mage Immunochemistry System; Beckman Coulter). We mea-
sured total homocysteine levels using a fluorescence polarization
immunoassay on an IMx analyzer (Abbott Laboratories, Abbott
Park, Illinois). Serum uric acid concentration was assessed with a
Kone Diagnostica reagent kit and a Kone autoanalyzer (KONE
Corporation, Espoo, Finland). Coronary artery calcium was mea-
sured in the epicardial coronary arteries on scans obtained with
either a C-150 electron beam computed tomography scanner
(Imatron, South San Francisco, California) (n 2035) or a 16-
or 64-slice multidetector computed tomography scanner (Soma-
tom Sensation 16 or 64; Siemens, Forchheim, Germany) (n
1643) and quantified by using the Agatston method (39, 40).
The latter scanner was more recently developed. Ultrasonography
of the left and right carotid arteries was performed with a 7.5
MHz linear array transducer (ATL UltraMark IV; Advanced
Technology Laboratories, Bethel, Washington). The maximal
cIMT, summarized as the mean of the maximal measurements
from the near and far walls on both the left and right sides, was
used for analysis (41). The ratio of the systolic blood pressure at
the ankle to the systolic blood pressure at the arm (that is, ankle-
brachial index [ABI]) was calculated for each leg. The lowest ABI
in either leg was used in the analysis (42). Values of ABI greater
than 1.4 were excluded because high ABI may represent a differ-
ent underlying pathology related to calcified, noncompressible
arterial vessels (43). Peripheral arterial disease was defined as ABI
values of 0.9 or less, compared with the ABI values above 0.9 as
the reference category, and used in the analyses. Pulse wave ve-
locity was assessed with an automatic device (Complior; Artech
Medical, Pantin, France) that measures the time delay between
the rapid upstroke of the feet of simultaneously recorded pulse
waves in the carotid and femoral arteries. Pulse wave velocity was
calculated as the ratio of the distance between the recording sites
in the carotid and femoral arteries to the foot-to-foot time delay
and expressed in meters per second (29).

Appendix Table 1. Parameter Estimates and Performance

Measures of the Base Model

Parameter HR (95% CI)

Age 1.08 (1.061.09)
Male sex 1.95 (1.552.46)
Systolic blood pressure 1.01 (1.001.01)
Treatment for hypertension 1.21 (0.951.53)
Total cholesterol 1.23 (1.111.37)
HDL cholesterol 0.35 (0.250.49)
Diabetes 1.35 (1.031.77)
Current smoking 1.38 (1.051.82)

Model performance measures

Model likelihood chi-square 230.49
Model c-statistic (95% CI)* 0.73 (0.710.75)

HDL high-density lipoprotein; HR hazard ratio.

* The c-statistic is corrected for overoptimism by using 100 bootstrap repetitions.

www.annals.org 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 W-133

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Appendix Figure. Multivariable-adjusted HR for incident coronary heart disease in men (top ) and women (bottom).

Marker in Men HR (95% CI)

NT-proBNP 2.4 (1.53.9)

vWF antigen 1.3 (0.92.0)
Fibrinogen 1.8 (1.12.8)
CKD 1.8 (1.22.6)
Leukocyte count 2.2 (1.43.3)
Homocysteine 1.5 (1.02.3)
Uric acid 0.9 (0.61.3)
CAC score 7.8 (3.318.4)
cIMT 1.6 (0.92.6)
PAD 1.6 (1.12.2)
PWV 1.5 (0.92.4)

0.75 1.0 1.5 8.0

HR (95% CI)

Marker in Women HR (95% CI)

NT-proBNP 2.6 (1.54.5)

vWF antigen 1.0 (0.71.6)
Fibrinogen 1.1 (0.71.8)
CKD 1.1 (0.81.6)
Leukocyte count 1.6 (0.92.6)
Homocysteine 1.4 (0.82.3)
Uric acid 1.1 (0.71.9)
CAC score 4.8 (2.011.7)
cIMT 1.5 (0.92.6)
PAD 1.1 (0.71.6)
PWV 1.0 (0.51.9)

0.75 1.0 1.5 5.0

HR (95% CI)

CAC coronary artery calcium; cIMT carotid intimamedia thickness; CKD chronic kidney disease; HR hazard ratio; NT-proBNP
N-terminal fragment of prohormone B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave velocity; vWF von Willebrand
factor.

W-134 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 www.annals.org

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Appendix Table 2. Global Model-Fit Statistics, for Total Appendix Table 4. Coronary Heart Disease Risk Reclassifi-
Population and by Sex cation for Men in the Framingham Intermediate-Risk Group
After Extending the Model With Each Marker
Extended Model Likelihood Ratio Test Statistic
(Chi-Square)* Extended Model Men With Men NRI (95% CI), %
Event, %* Without
Total Men Women Event, %*
Population
FRS NT-proBNP 58.7 35.6 25.7 Up Down Up Down
FRS vWF antigen 2.2 2.3 0.1 FRS NT-proBNP 29.4 15.9 10.1 32.9 36.3 (23.9 to 48.8)
FRS fibrinogen 7.2 11.4 0.1 FRS vWF antigen 7.6 4.8 4.5 8.5 6.8 (0.1 to 13.4)
FRS CKD 5.4 7.8 0.3 FRS fibrinogen 19.1 6.9 7.9 17.5 21.8 (12.3 to 31.2)
FRS leukocyte count 14.1 19.8 0.8 FRS CKD 14.6 5.7 7.4 8.6 10.1 (1.8 to 18.4)
FRS CRP 4.8 NA NA FRS leukocyte count 19.8 11.9 9.0 19.4 18.3 (7.9 to 28.7)
FRS homocysteine 5.3 5.9 0.7 FRS homocysteine 13.8 5.7 5.6 11.1 13.6 (5.5 to 21.8)
FRS uric acid 0.2 0.1 0.0 FRS uric acid 1.3 0.0 0.9 2.1 2.5 (0.1 to 4.9)
FRS CAC score 60.9 41.2 22.6 FRS CAC score 43.9 11.0 19.5 37.5 50.9 (33.7 to 68.1)
FRS cIMT 5.2 1.8 3.6 FRS cIMT 8.0 3.7 3.3 6.7 7.7 (1.4 to 14.0)
FRS PAD 3.7 5.6 0.1 FRS PAD 10.7 6.1 6.9 8.8 6.5 (1.1 to 14.1)
FRS PWV 1.2 5.2 0.9 FRS PWV 9.9 3.7 5.3 12.3 13.2 (6.2 to 20.2)

CAC coronary artery calcium; cIMT carotid intimamedia thickness;
CKD chronic kidney disease; CRP C-reactive protein; FRS Framingham
risk score; NA not applicable; NT-proBNP N-terminal fragment of prohor-
mone B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse
wave velocity; vWF von Willebrand factor.
* The increase in model fit after extending the base model (including conventional
risk factors) with each new marker.
Statistically significant improvement in model fit (P 0.05) as evaluated by
likelihood ratio chi-square.
The analyses for CRP (n 3029) and CAC score (n 3678) were performed
in a smaller group. In the case of CRP, the power was not enough to perform the
sex-specific analyses.
CAC coronary artery calcium; cIMT carotid intimamedia thickness;
CKD chronic kidney disease; FRS Framingham risk score; NRI net
reclassification improvement; NT-proBNP N-terminal fragment of prohormone
B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave
velocity; vWF von Willebrand factor.
* Percentages of men with or without an event within the intermediate-risk cate-
gory who move to a higher or lower risk category after extension of the base model
with the specific marker.
Percentage of NRI (95% CI) for the intermediate-risk category after extension of
the base model with the specific marker. It is estimated as ([number of events
reclassified higher minus number of events reclassified lower]/number of events)
([number of nonevents reclassified lower minus number of nonevents reclassified
higher]/number of nonevents).
The analysis for CAC score (n 3678) was performed in a smaller group. In the
case of C-reactive protein (n 3029), the power was not enough to perform the
sex-specific analyses.

Appendix Table 3. Discriminative Ability of Risk Markers, by Sex

Marker Men Women

Change in c-Statistic Total NRI Change in c-Statistic Total NRI

(95% CI)* (95% CI), % (95% CI)* (95% CI), %
NT-proBNP 0.03 (0.01 to 0.05) 9.9 (2.8 to 17.1) 0.02 (0.01 to 0.04) 8.4 (1.8 to 15.1)
vWF antigen 0.00 (0.01 to 0.00) 0.5 (3.5 to 4.6) 0.00 (0.00 to 0.00) 1.6 (0.8 to 3.9)
Fibrinogen 0.01 (0.00 to 0.02) 6.1 (0.2 to 12.1) 0.00 (0.00 to 0.00) 2.2 (0.1 to 4.6)
CKD 0.00 (0.01 to 0.01) 0.03 (4.8 to 4.9) 0.00 (0.00 to 0.00) 1.8 (0.9 to 4.6)
Leukocyte count 0.03 (0.00 to 0.04) 5.6 (0.4 to 11.6) 0.00 (0.00 to 0.00) 1.1 (3.5 to 1.4)
Homocysteine 0.00 (0.00 to 0.01) 3.2 (1.7 to 8.0) 0.00 (0.00 to 0.00) 0.4 (2.6 to 3.3)
Uric acid 0.00 (0.01 to 0.00) 0.5 (0.9 to 1.8) 0.00 (0.00 to 0.00) 0.2 (1.2 to 1.6)
CAC score 0.06 (0.03 to 0.09) 24.1 (14.4 to 33.8) 0.05 (0.03 to 0.07) 13.4 (3.9 to 22.9)
cIMT 0.00 (0.01 to 0.01) 2.2 (1.7 to 6.1) 0.00 (0.00 to 0.00) 2.3 (1.9 to 6.5)
PAD 0.01 (0.00 to 0.01) 1.6 (6.5 to 3.3) 0.00 (0.00 to 0.00) 0.9 (2.7 to 1.0)
PWV 0.01 (0.00 to 0.01) 1.7 (2.6 to 6.1) 0.00 (0.00 to 0.00) 1.4 (0.5 to 3.4)

CAC coronary artery calcium; cIMT carotid intimamedia thickness; CKD chronic kidney disease; NRI net reclassification improvement; NT-proBNP
N-terminal fragment of prohormone B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave velocity; vWF von Willebrand factor.
* Increment in c-statistic in the extended model (with conventional risk factors and the newer risk marker) vs. the base model (with conventional risk factors alone). The
c-statistic is corrected for overoptimism by using 100 bootstrap repetitions.
Percentage of NRI (95% CI) for the extended model (with conventional risk factors and the newer risk marker) vs. the base model (with conventional risk factors alone),
using risk categories of 10%, 10%20%, and 20%. It is estimated as ([number of events reclassified higher minus number of events reclassified lower]/number of
events) ([number of nonevents reclassified lower minus number of nonevents reclassified higher]/number of nonevents).
The analysis for CAC score (n 3678) was performed in a smaller group. In the case of C-reactive protein (n 3029), the power was not enough to perform the
sex-specific analyses.

www.annals.org 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 W-135

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015

 Appendix Table 5. Coronary Heart Disease Risk Reclassifi- Appendix Table 6. Incidence of CHD Events in the
cation for Women in the Framingham Intermediate-Risk Subpopulation With CAC Score Measurements (n 3678)
Group After Extending the Model With Each Marker During a Median Follow-up of 5.5 Years

Extended Model Women Women NRI (95% CI), % Event Total Events per
With Without Events, n 1000
Event, %* Event, %* Person-Years, n
Nonfatal MI 132 5.79
Up Down Up Down
Fatal CHD 78 3.42
FRS NT-proBNP 29.8 18.1 13.0 29.1 27.8 (12.8 to 42.8) Definite fatal MI 12 0.53
FRS vWF antigen 3.6 3.0 0.6 2.3 2.3 (3.1 to 7.8) Definite fatal CHD 42 1.84
FRS fibrinogen 1.7 1.7 0.7 2.1 1.4 (2.6 to 5.5) Possible fatal CHD 24 1.05
FRS CKD 2.1 1.6 2.1 2.8 1.2 (3.2 to 5.6)
FRS leukocyte count 2.3 3.1 2.0 4.4 1.6 (3.7 to 6.8) CAC coronary artery calcium; CHD coronary heart disease; MI myocar-
FRS homocysteine 0.0 5.0 2.3 5.5 1.8 (7.0 to 3.4) dial infarction.
FRS uric acid 0.0 1.5 0.0 0.9 0.6 (3.3 to 2.1)
FRS CAC score 24.3 19.8 14.3 35.0 25.2 (6.4 to 44.0)
FRS cIMT 12.4 4.7 5.1 11.1 13.7 (4.6 to 22.9)
FRS PAD 0.0 1.6 1.2 1.6 1.2 (4.2 to 1.7)
FRS PWV 1.7 1.6 2.4 6.1 3.8 (0.8 to 8.3)

CAC coronary artery calcium; cIMT carotid intimamedia thickness;

CKD chronic kidney disease; FRS Framingham risk score; NRI net
reclassification improvement; NT-proBNP N-terminal fragment of prohormone
B-type natriuretic peptide; PAD peripheral arterial disease; PWV pulse wave
velocity; vWF von Willebrand factor.
* Percentages of women with or without an event within the intermediate risk
category who move to a higher or lower risk category after extension of the base
model with the specific marker.
Percentage of NRI (95% CI) for the intermediate-risk category after extension of
the base model with the specific marker. It is estimated as ([number of events
reclassified higher minus number of events reclassified lower]/number of events)
([number of nonevents reclassified lower minus number of nonevents reclassified
higher]/number of nonevents).
The analysis for CAC score (n 3678) was performed in a smaller group. In the
case of C-reactive protein (n 3029), the power was not enough to perform the
sex-specific analyses.

Appendix Table 7. Coronary Heart Disease Risk Reclassification After Extending the Base Model With CAC Score

Base Model Base Model CAC Score Reclassified as Reclassified as

Higher Risk* Lower Risk*
<10% 10%20% >20% Total
<10%
Persons with event 71 50 4 125 54 (43.2) NA
Persons without event 2015 315 16 2346 331 (14.1) NA
Total persons, n 2086 365 20 2471 385 NA
Observed risk (95% CI) 0.03 (0.020.05) 0.14 (0.100.19) 0.21 (0.060.60)

10%20%
Persons with event 19 75 55 149 55 (37.0) 19 (13.0)
Persons without event 262 364 144 770 144 (18.7) 262 (34.0)
Total persons, n 281 439 199 919 199 281
Observed risk (95% CI) 0.07 (0.040.12) 0.17 (0.130.22) 0.28 (0.200.37)

>20%
Persons with event 0 9 62 71 NA 9 (14.5)
Persons without event 17 60 140 217 NA 77 (41.4)
Total persons, n 17 69 202 288 NA 86
Observed risk (95% CI) 0.00 (0.000.00) 0.13 (0.060.27) 0.31 (0.230.40)

Totals
Persons with event 90 134 121 345 109 (31.6) 28 (8.1)
Persons without event 2294 739 300 3333 475 (14.3) 339 (10.1)
Total persons, n 2384 873 421 3678 584 367

CAC coronary artery calcium; NA not applicable.

* Numbers (percentages) of persons with or without an event who are reclassified to a higher or lower risk category after the base model is extended with CAC score. They
are based on the results from the parametric Weibull proportional hazards regression model, estimating the individual 10-y predicted risk for coronary heart disease from the
available median follow-up of 5.5 y.

W-136 20 March 2012 Annals of Internal Medicine Volume 156 Number 6 www.annals.org

Downloaded From: http://annals.org/ by a University of Ottawa User on 01/02/2015