EBM-Diagnostic Testing

Nur Samsu

Division of Nephrology & Hypertension Department of Internal Medicine

Dr. Saiful Anwar Hospital FK UB
Malang 2014

Evolution of Diagnostic Standard

Diagnosis historically

Diagnosis today

Evidence-Based
Physical Diagnosis
essential here

Dermatologic diagnosis
Cellulitis, Zoster, Psoriasis

Neurologic diagnosis
Parkinsons disease, Bells palsy,
Amyotropic lateral scleosis

Musculoskeletal diagnosis
Cardiologic diagnosis

Bedside diagnosis
Technology

Pericarditis,
Mitral valve prolaps

Opthalmologic diagnosis
Psychiatric diagnosis

The Diagnostic Process

Working diagnosis- pretest probability

With each new finding/test we move from the

pre-test probability to a new post-test probability

Clinicians estimate probability of disease using

probabilistic, prognostic and pragmatic
approaches

Compare disease probabilities to two

thresholds

Test and Treatment Thresholds in

the Diagnostic process
Test
threshold

Zone of Uncertainty
(probability diagnosis)

0%
Probability below
test threshold;
No testing warranted

Treatment
threshold

100%
Probability between test
and treatment threshold;
further testing required

Probability above treatment

threshold; testing completed;
treatment commences

Introduction
What

is EBM?

"...the

conscientious, explicit, and judicious use

of current best evidence in making decisions
about the care of individual patients."

Integrated with clinical expertise

expertise in performing the history and physical examination
knowledge of the patient, the family, and the community
which creates a context for therapeutic decision-making
a relationship with the patient informed by his or her beliefs
and values
practical knowledge of the availability of resources in the
community

Introduction

Introduction
"Without

clinical expertise, practice

risks becoming tyrannized by external
evidence, for even excellent external
evidence may be inapplicable to or
inappropriate for an individual patient.
Without current best external evidence,
practice risks becoming rapidly out of
date, to the detriment of patients."

Diagnostic test
Diagnosis

sen, spec
Screening high prevalence,
significance morbidity/mortality, efective
treatment +, early treatment
Treatment

follow up, drug monitoring

Epidemiology survey (prevalence of
disease)

Diagnostic and Screening

Diagnostic

Screening

Detect disease
at all stage

Detect early
stage of disease

In Clinical Practice

Diagnostic Results
may be in error

Diagnostic test
Probability,

not certainty

Certain test vs gold standard

Medicine is a science of
uncertainty and an art of
probability

William Osler

Gold Standard
The

best available diagnostic test

To compare with other diagnostic test
High sensitivity & specificity
No component of the tested test

Diagnostic tests
Start thinking about
what youre going to do with the results of the
diagnostic test, and
whether doing the test will help your patients

Diagnostic test

standard
More convenience
Easy and simple
Cheaper
Early diagnosis

Disease: detection and prediction

Disease

detection

Diagnostic

tests - present
Cross-sectional
Disease

prediction

Prognostic
Cohort.

tests future

Test and Treatment Thresholds

Traditional 2 x 2 Tables
Disease
Present

Absent

positive

True-positive
(a)

False-positive
(b)

negative

False-negative
(c)

True-negative
(d)

a+c

b+d

Test result

Properties of test
Sensitivity
Specificity
Predictive

value of positive test

Predictive value of negative test
Likelihood ratio of positive test
Likelihood ratio of negative test
Pretest probability and odds
Posttest probability and odds
ROC curve and area

Gold Standard for DX

Disease (+)

Disease (+)
No Disease

a
c

No Disease

+ PV =
a/(a+b)

- PV =
d/(c+d)

Accuracy = (a + d) / N

Sensitivity

Specificity

Prevalence = (a + c) / N

a / (a+c)

d / (b+d)

Sensitivity & Specificity

Test

with sensitivity ~100% .."SnNOut

~ NPV ~ post test probability of +ve

Test

with specificity ~ 100% .... "SpPIn

~ PPV ~ post test probability of -ve

Limitation of Sensitivity and

specificity
if a patient have positive test ---

disease + ?
If

a patient have negative test

---disease - ?

Must know :
Prevalence/pre-test
PPV

or LR

probability and

Predictive value of test

Predictive

value of positive test (PPV)

Predictive

value of negative test (NPV)

2 x 2 Tables
Sensitivity

= 90%
Specificity = 90%
PPV = 90%
NPV = 90%

Disease

Present Absent

positive

90

10

negative

10

90

Test
result

2 x 2 Tables

Sensitivity = 90%

Specificity = 90%

PPV = 8.3% (90/1090)

NPV = 99.9% (9000/9010)

Prevalence = 1%

Disease

Present

Absent

positive

90

1000

negative

10

9000

Test
result

Clinical Setting
Specialist referral hospital
(High prevelance)

Primary care
(Low prevalence)

Disease
absent
present

Disease
absent
present

Test +

50

10

Test +

50

100

Test -

100

Test -

1000

Sensitivity = 50/55 = 91%

Specificity = 100/110 = 91%

Sensitivity = 50/55 = 91%

Specificity = 1000/1100 = 91%

Prevalence = 55/165 = 33%

Prevalence = 55/1155 = 3%

PPV = 50/60 = 83%

NPV = 100/105 = 95%

PPV = 50/150 = 33%

NPV = 1000/1005 = 99,5%

Nomogram for Interpreting

Diagnostic Test Results
Hospital setting

Primary care setting

Likelihood ratio (LR)

LR= Probability of result in diseased people
Probability of result in non-dis. people

LRs describe how much the odds change after

applying the results of a test.

Likelihood ratios
LR

positive (LR+)
= sensitivity/false-positive
= sensitivity/(1-specificity)
The higher ratio (>1) , the better the test

LR

negative (LR )
= false-negative/specificity
= (1-sensitivity)/specificity
The smaller ratio (close to 0), the better the test

no effect of prevalence on likelihood ratios

Likelihood ratios
Disease

LR+ = [a/(a+c)]/[b/(b+d)
LR- = [c/(a+c)/d(b+d)]
Present

Absent

positive

True-positive
(a)

False-positive
(b)

negative

False-negative
(c)

True-negative
(d)

a+c

b+d

Test result

2 x 2 Tables

90% with disease have a

positive test

Disease

15% without disease have a

positive test
Someone with a Pos test is 6X
more likely to have the
disease as not (90%/15%)
Likelihood Ratio +

Sens /(1- Spec)

Test
result

90

15

105

10

85

95

2 x 2 Tables

10% with disease have a

negative test

Disease

85% without disease have a

negative test
Someone with a Neg test is
less likely to have the disease
by 1/8.5 X (10%/85%)
Likelihood Ratio

(1-sens)/spec

Test
result

90

15

105

10

85

95

100

100

200

Probability versus Odds

Odds

of disease does not equal probability

of disease
Example: 10 patients 3 have anemia and 7
do not
Probability

of having anemia is 3/10 = 30% =

Prevalence = Pretest probability
Odds of having anemia are 3:7 or 0.43

Probabilities and Odds

Can

be calculated back and forth

Odds of a:b will give probability of a/(a+b)

Probability of X% will give Odds of X/(100-X)

Start with
Pre-test probability
(Prevalence)

Pre-test Odds
Apply test
Post-test Odds
Post-test probability

Probability of having disease

Pre-test

probability

= prevalence of disease
pre-test odds of disease
Post-test

probability

=probability of disease if test is positive

post-test odds of disease if test is positive
= Pre-test odds of disease

LR+

= Post-test Odds/(1 + Post-test Odds)

Calculation of post-test probability

Post-test

probability

post-test odds of disease if test is positive

= Pre-test odds LR+
(Bayes Theorem)
Example: if LR+ = 8
- Prevalence = 20% pre-test odds = 20/80 = 1:4
- Post-test odds = 1:4 x 8 = 8:4
- Post-test probability = 8/(8+4) = 66.6 %

Pre-Test
Probability

Pre-Test
Odds

Odds = Probability/1-probability

LR (+) = Sensitivity
1Specificity

Post-Test
Odds

Post-Test
Probability

Probability = Odds/1 + Odds

LR (-) = 1 Sensitivity
Specificity

Calculation of post-test probability of disease

1.
2.
3.

4.
5.

Find the symptom, disease

and test of interest
Estimate the probability of
disease before testing
Convert the probability of
disease to an odds of
disease
Multiply the pre-test odds
by the likelihood ratio
Covert the post-test odds
to a post-test probability

1.

Hypothyroidism: TSH level

2.

Pretest probability = 20%

Odds of hypothyroidism
20:80 = 1:4
4. LR+ for TSH in
hypothyroidism = 99
1:4 x 99 = 99:4
5. 99/(99+4) = 96%
3.

PRE-TEST ODDS
GOLD STANDARD
Total

YES

a b

10

a+b

NO

NO

c d

89

90

c+d

TEST

YES

Total

96

100

a + c

b + d

a+b+c+d

In the sample as a whole, the odds of having the disease are 4 to

96 or 4%
Prevalence / (1 prevalence) or total case / total non-case

POST-TEST ODDS
GOLD STANDARD
Total

YES

a b

10

a+b

NO

NO

c d

89

90

c+d

TEST

YES

Total

96

100

a + c

b + d

a+b+c+d

In those who score positive on the test, the odds of

having the disease are 3 to 7 or 43% (the POST-TEST
ODDS)

POST-TEST ODDS
YES

NO

Total

YES

10

a+b

NO

89

90

c+d

TEST

GOLD STANDARD

Total

96

100

a + c

b + d

a+b+c+d

c
a d
b

In those who score negative on the test, the odds of

having the disease are 1 to 89 or approximately 1%

Choice of a cut-off point

for continuous results
Consider the implications of the two possible errors:

If falsepositive results must be avoided (such as the

test result being used to determine whether a patient
undergoes dangerous surgery), then the cutoff point
might be set to maximize the test's specificity

If falsenegative results must be avoided (as with

screening for neonatal phenylketonuria), then the
cutoff should be set to ensure a high test sensitivity

Continuous diagnostic test results

Diagnostic Threshold

specificity=94%

sensitivity=94%

Non-diseased
normal

TN

FN

Diseased
diseased

FP

TP

Choice of a cut-off point

for continuous results
Using

receiver operator characteristic

(ROC) curves:
Selects

several cut-off points, and determines the

sensitivity and specificity at each point

Then,

graphs sensitivity (truepositive rate) as a

function of 1specificity (falsepositive rate)

Usually,

the best cut-off point is where the

ROC curve "turns the corner

Critical Appraisal of
Diagnostic Study

Tools for Critical Appraisal

EBM simplified approach:
Is

the evidence valid?

Is

the evidence Important?

Does

the evidence Apply to our patient?

V
I
A

Guideline for Diagnostic Tests Evidence

Valid?

Is this evidence about the accuracy of

a diagnostic test valid?
Important? [Does this (valid) evidence
demonstrate an important ability of this test t
o accurately distinguish patients who do and
dont have a specific disorder?]
Apply? [Can I apply this valid, important
diagnostic test to a specific patient?]

Critical appraisal
- Valid
- Important
- Applicable

Methods
Results
Discussion

Critical Appraisal - Diagnosis

Is

the evidence valid?

Was

there an independent, blinded

comparison with a gold standard?
Was the test evaluated in an
appropriate spectrum of patients?
Was the reference standard applied
regardless of the test result?
If NO, stop here

Critical Appraisal - Diagnosis

Is

this valid test important?

Distinguish

between patients with and

those without the disease
Two by two tables
Sensitivity and Specificity
SnNOut
SpPIn

ROC

curves
Likelihood Ratio

Can I apply this valid, important

diagnostic test to a specific patient?
1. Is the diagnostic test available, affordable, and
accurate in our setting?
2. Can we generate a clinically sensible estimate of our
patients pre-test probability?
3. Will the resulting post-test probabilities affect our
management and help our patient?

 Will the post-test probability affect management ?

Movement
Patient

above treatment threshold

willing to undergo testing

Case

75-year-old woman with a Hb 10, MCV was

80 on routine checkup, a negative history
and physical except OA, and on no meds
likely to suppress her marrow or cause a
bleed
Her probability of iron deficiency was 50%
You want to avoid doing a bone marrow and
order serum ferritin to diagnose IDA

Case

P: In an elderly symptomless woman with mild

anemia
I: how useful is serum ferritin
C:
O: in diagnosing iron deficiency anemia
T(ype of question): Diagnosis
T(ype of study): Prospective Cohort
*Diagnosis of Iron Deficiency Anemia in the
Elderly (Guyatt, et al. Am J Med, 1990;
(88):205-209

Three Main Questions

Validity-Is this evidence about the

accuracy of a diagnostic test valid?

Results-Does this evidence show that this

test can adequately distinguish patients
who do and do not have the disorder?

Applicability-How can I apply this valid,

accurate diagnostic test to a specific
patient?

T
e
s
t

Positive
<45
Negative
>45

Iron Deficiency Anemia

Present Absent
70
15
a b
c d
15
135
a+c b+d

Totals

85

150

a+b+c+d
235

Low ferritin (<45) in diagnosing IDA

Prevalence (study pre-test probability)
= 85/235= 36%
Sensitivity = 70/85
= 82%
Specificity = 135/150
= 90%

Low ferritin (<45) in diagnosing IDA

L.R.+

= sensitivity/(1-specificity)
= 82%/10% = 8.2

L.R. -

= (1-sens)/spec
= 18%/90% = 0.2

Simplifying LR Calculations
Bone Marrow:
iron deficient

Bone Marrow:
normal iron

> 46

70
15

15
135

Totals

85

150

Test Results:
< 45

Calculating LR at 45 cut point

Bone Marrow:
iron deficient

Bone Marrow:
normal iron

Likelihood
Ratios

< 45

70
70/85=0.824

15
15/150=0.1

0.824/0.1=
8.24

> 46

15
15/85=0.176

135
135/150=0.9

0.176/0.9=
0.196

85

150

Test Results:

Totals

Pre-test Prob = 36%

LR+ = 8.2
LR- =

0.2

Pre-test Prob = 50%

LR+ = 8.2
LR- =

0.2

Treatment Thresholds
No Tx

ZONE OF UNCERTAINTY

Tx

x
0%

10%

90%
100%

90% Probability of
Fe def Anemia when
Ferritin is <45

LRs for 4 levels of Serum Ferritin

Ferritin

Fe def #

Not Fe def

L.R.

<18

47

41.47

>18<45

23

13

3.12

>45<100

27

0.46

>100

108

0.13

Total

85

150

Calculating LRs
Bone Marrow: Bone Marrow:
normal iron
iron deficient
Test Results:
< 18

Likelihood
Ratios

47
47/85=0.553

2
2/150=0.013

0.553/0.013=
42.5

19-45

23
23/85=0.271

13
0.271/0.087=
13/150=0.087
3.11

46-100

7
7/85=0.082

27
27/150=0.18

0.082/0.18=
0.456

>100

8
8/85=0.094

108
108/150=0.72

0.094/0.72=
0.131

Totals

85

150

Thank you