Patient-oriented

Outcome Measures
ACSM 2014 Lecture

The DASH to the
perfect clinical tool.
James MacDonald, M.D., M.P.H., FACSM
1
What Well Talk About
Background.
History and Context.
Whats the Purpose?
Language Matters.
Specific Examples of Measures.
Selected Survey of Studies Using
Measures.
2
Doing It matters
We do it
Researchers do it
Payors do it
Patients want to do it too

In modern medicine, its how you prove your
patients are getting better.
3
In medicalese
Outcomes management is a technology
of patient experience designed to help
patients, payers, and providers make
rational medical care-related choices
based on better insight into the effect of
these choices on the patients life.

Paul Ellwood, 1988 Shattuck Lecture
4
Its Really a Part of EBM
Integration of 3 elements:
High quality clinical research
Pooled data, readily accesisble
Greater reliance on standards/guidelines
Clinicians own expertise
Ability to interpret when to use above
Patient input and values
Measure and incorporate




5
A bit of history
Epidemiologic shift: 20
th
century
1
st
valid/reproducible
questionnaires: 1949
EBM (Prof. Cochrane): 1972
RAND SF-36: 1988
A Technology of Patient
Experience: 1988
Shattuck Lecture, Paul Ellwood
PCORI established: 2010

6
Patient-Centered Outcomes
Research Institute
7
Important Functions of Scales
Comparison of functional status
Of one patient over time
Inter-patient comparison
Allowing patients to self-evaluate
Evaluating effectiveness of various treatment
modalities
8
Simple Stuff
9
Graded Symptom Scale in
Concussions
J Sport Rehabil. 2008 Feb;17(1):11-20.
Healthy youth are reliable in
reporting symptoms on a graded
symptom scale.
Found excellent reliability in middle school
athletes on the PCSS
However, difference in patient/parent

10
Old School

11 Steiner and Micheli Bracing in
Spondylolysis 1985
New School
0 - 10
Micheli Functional Scale
0 - 5
0 - 4
0 - 3
0 - 3
A hierarchy of numbers
Scales can be
Nominal
e.g. 1 = male, 2 = female
Ordinal
e.g. 1 = mild pain, 2 = moderate pain, 3 = severe pain
Interval*
e.g. temperature (10C, 20C, etc.)
Ratio*
e.g. weights
13
*only scales that can be summated
Words Matter
RELIABILITY
VALIDITY
RESPONSIVENESS
GENERALIZABILITY



An Ideal Rating System has proven reliability, validity, and
responsiveness and can be applied with the knowledge that the
results will be precise, accurate, and detect meaningful change. It
should also be generalizable.
14
Reliability
What is it?
Intra-observer Reliability
Inter-observer Reliability
How can it be measured
Correlation coefficients
Pearsons
Spearmans
ICC
Cronbachs
15
Validity
What is it?
Face Validity
Criterion Validity
Construct Validity
How can it be measured?
Correlation coefficients

16
Responsiveness
Also known as sensitivity to change
The ability of a rating scale to detect
clinical change.
17
Minimal Clinically Important
Difference (MCID)
a.k.a. The minimal clinically significant
difference
Smallest difference in score in domain of
interest which patients perceive as
beneficial
Anterior Knee Pain Scale (AKPS) or
Kujala Scale 3 studies variously found 7
14 is MCID (scale 0 to 100)
VAS (0 to 100mm) is 18mm
18
Ceiling and Floor effects
19

Micheli Functional Scale
Generalizability
One cannot assume that instruments can
be used in every population
orModerate pain may be different in
Sweden than in Ethiopia
Need to do testing in different populations
Cross-cultural differences
Gender differences
Age differences (think PEDI SCAT3)




20
Button and Pinney 2004

21
Which one to choose?
22

or.What is the ideal test?
Reliable, valid, responsive
Pertinent to and tested on your population
Culturally and linguistically appropriate
What are you looking at?
Health Related Quality of Life (HRQOL)
Functional status
Disease state (e.g. osteoarthritis)
Body part (e.g. hip)
Who fills it out?
and.
23
Public vs. Private ($)
Public
SPADI
AIMS2
Oswestry Disability Index
DASH
Private
SF-36
POSNA Pediatric MSK
Functional Health
Questionnaire
Pediatric Quality of Life
Inventory*
24
1. USER AGREEMENT

User is required to complete and sign a User Agreement in which specific
conditions requested by the Authors are detailed.

2. ACCESS FEES

AUTHORS ROYALTY FEES:

Authors' royalty fees are requested according to the study design
and context of use of the questionnaire:
- In academic research and individual clinical practice: free of charge

- In commercial studies (1) involving "for-profit" organizations and
academic studies funded by commercial companies: 500 * per
protocol/application plus an additional

- l 500 * per available language version (original version and translated
versions)
Other specific conditions requested by the Authors are detailed in the User
Agreement.

25 Mapi Research Institute Lyon,
France
From the American College of
Rheumatology

Measures should
Be relevant to and/or used in rheumatology research
and/or clinical practice.
Be in the public domain (do not require purchase)
Require very little or easily attainable equipment to
administer.
Not require special training or certification to
administer.
Not be not biologically based (e.g., radiographic
grading systems).

26
General Types of Scales
Quality of Life
e.g. SF 36
Region Specific
e.g. DASH
Disease Specific
e.g. AIMS2, WOMAC
Mixed
27
Mixed Instruments
Harris Hip Score used in Hip OA,
outcomes of total hip arthroplasty, etc.
Mix of subjective and objective criteria
http://www.orthopaedicscore.com/scorepa
ges/harris_hip_score.html
28

29
Clinical Outcomes In
Sports Related Concussions



30 Journal of Sport Rehabilitation
2011 20 46-60
Specific Scales
HRQOL
Pediatric Quality of Life
Inventory
SF-36

FHS
HIT-6 (Headache Impact
Test)
POSNA Pediatric MSK
Functional Health
WOMAC
Oswestry Low Back Pain
Scale
DASH
Micheli Functional Scale

31
Health Related Qualify of Life
HRQoL: what is it?
Who does it?
http://www.healthypeople.gov/2020/about/Qo
LWBabout.aspx (CDC)
http://www.nihpromis.org/ (NIH)


32
HRQL: defining it
Health-related quality of life is the value
assigned to duration of life as modified by
the impairments, functional states,
perceptions, and social opportunities that
are influenced by disease, injury,
treatment or policy.
..the ultimate goal of all health care is to
improve, restore, and preserve HRQL
33
SF 36
34
Pediatric Quality of Life Inventory
Used increasingly in HRQOL assessments
in a variety of pedi studies
Parent reports cannot be substituted
for child self-reports
65 translations
Different modules
Validated on children and adolescents:
Cronbachs alpha 0.83
35
DASH (Disability Arm, Shoulder, Hand )
DASH has 30 questions
QuickDASH has 11 questions
Both offer a Sports/Performing Arts
Module which is 4 questions
Free! Find it at
http://www.dash.iwh.on.ca/index.htm
Evaluates symptoms and patients ability
to perform ADLs/Sport-specific activity


36
ODI
37
ODI: other versions

38
Oswestry Disability Index

39
Spine Patient Outcomes
Research Trial (SPORT)
To assess efficacy of surgery for lumbar intervertebral
disk herniation

40 JAMA 2006
HRQOL in collegiate athletes
with history of concussion

41
SF-36
Two indices used
SF-36 and HIT-6
Kuehl et al. CJSM 2010
42
SF-36

43
HIT-6
44
Quality of Life in Adolescents
with Mild Asthma
13-18 year olds
HRQoL lower in adolescent athletes with
EIB in comparison with peers without this
disorder
Previous studies have demonstrated
treatment of moderate to severe asthma
improves HRQoL
Argues for doing the same in EIB
45

46
JAMA 5/21/14
102 community volunteers, age > 50 with
radiographic evidence of hip OA and VAS
pain > 4cm at baseline
12 weeks of PT vs. sham
Outcome measures
VAS pain
WOMAC OA Index (physical function, hip
specific)
Hip OA Outsome scale
No difference!!!
47
Micheli Functional Scale

48 d'Hemecourt et al. CJSM 2012
Validity testing of MFS
49
Spearmans correlation Coefficient
showing high correlation between MFS and ODI
(r
s
= 0.821, p < 0.0001)
Summary
History and Current Use
Why it matters
Psychometric Terminology
Reliability, validity, responsiveness,
generalizability, floor/ceiling effects
Survey of Instruments
Examples in MSK/
Sports Med Literature



Surv
50
References
A Meta-Analysis of Outcome Rating Scales in Foot and Ankle Surgery: Is
there a Valid, Reliable, and Responsive System? Button and Pinney, 2004
Foot and Ankle International 25 (8): 521 525.
Measuring Clinically Important Changes With Patient-Oriented
Questionnaires, Liang et al. 2002 Med Care 40 (suppl): II-45-II-51
A Systematic Review of Faces Scales for the Self-report of Pain Intensity in
Children Tomlinson et al., 2010 Pediatrics 126: e1168-e1198
Clinical Outcomes Assessment for the Management of Sport-Related
Concussion, Mcleod and Register-Mihalik 2011 Journal of Sports
Rehabilitation 2011 20 46 20.
The New Subjective Medicine: Taking the Patients Point of View on
Health Care and Health, Sullivan 2003 Social Science and Medicine 1595
1604
http://moon.ouhsc.edu University of Oklahoma
Effect of Physical Therapy on Pain and Function in Patients with Hip
Osteoarthritis: A Randomized Clinical Trial, Bennell et al. 2014. JAMA
311:
This talk will be available at cjsmblog.com

51
Thanks!
52