Letters to the Editor

Letters to the Editor

More on How USMLE Step
1 Scores Are Challenging
Academic Medicine
To the Editor: Prober and
colleagues1 make a strong, rational
argument for not using United States
Medical Licensing Examination
(USMLE) Step 1 scores to screen
graduate medical education (GME)
applicants. The USMLE was designed
to certify minimum competency for
medical licensure. Unfortunately,
residency program directors are
inappropriately using USMLE scores
to screen applicants. The authors
recommend development of systems to
collect competency-based evidence that
predicts success in GME. We would like
to suggest a solution to institute such
a change.
First, the National Board of Medical
Examiners should stop reporting
three-digit scores for the USMLE Steps
and provide only pass/fail scores and
a list of areas of strength and areas for
improvement based on performance.
To promote habits of lifelong
learning, we should require students
to address these recommended areas
for improvement and reflect on the
efforts they make to improve. Program
directors would use this evidence
of efforts towards lifelong learning
instead of USMLE scores when
reviewing applications.
The mission of medical education is
to improve the health of the nations
population. Failure to achieve
this mission can jeopardize GME
funding and the entire medical
education structure. It is accepted
that assessment drives learning and
should inform curriculum design.
Thus our second suggestion is that
major consideration be given by GME
programs to evidence of successful
participation in a population health
initiative during medical school. This
will require evidence of working in
interprofessional teams, using data for
quality improvement and knowledge
of population health. This requirement
will drive medical schools to help
students participate in innovative
projects for population health and
use systems for 360-degree feedback

Academic Medicine, Vol. 91, No. 5 / May 2016

to assess students in a broad range of

competencies. This process will generate
the type of data needed by program
directors and will complement the
evidence for lifelong learning described
above.
These goals are actually quite feasible. At
the Cleveland Clinic Lerner College of
Medicine, we have had a No Tests, No
Grades philosophy since 2004. Students
learn in collaborative small groups and
receive formative feedback from peers
and faculty, which they use to strive
for constant improvement and write a
reflective essay on their progress each year.2
A system such as this could be modified to
implement the proposals described above.
Disclosures: None reported.
Neil B. Mehta, MBBS, MS
Assistant dean of education informatics and
technology, Cleveland Clinic Lerner College of
Medicine at Case Western Reserve University,
Cleveland, Ohio; mehtan@ccf.org.

Alan Hull, MD, PhD

Associate dean of curricular affairs, Cleveland Clinic
Lerner College of Medicine at Case Western Reserve
University, Cleveland, Ohio.

James Young, MD
Executive dean, Cleveland Clinic Lerner College
of Medicine at Case Western Reserve University,
Cleveland, Ohio.

References
1 Prober CG, Kolars JC, First LR, Melnick DE.
A plea to reassess the role of United States
Medical Licensing Examination Step 1 scores in
residency selection. Acad Med. 2016;91:1215.
2 Dannefer EF, Henson LC. The portfolio
approach to competency-based assessment
at the Cleveland Clinic Lerner College of
Medicine. Acad Med. 2007;82:493502.

To the Editor: United States

Medical Licensing Exam (USMLE)
Step 1 scores are increasingly being
used to screen residency applicants.
Recently, this misuse of the exam
was highlighted by Prober and
colleagues1 in Academic Medicine.
The authors discussed unintended
consequences of such behavior on
medical students, including greater
stress and anxiety, distortions of
medical school curriculum, changes
in specialty choice, and increased
financial and opportunity costs of

test preparation. There has been a

proliferation of free and commercial
educational resources geared toward
standardized test preparation, which
prompts the question of whether these
third parties are disrupting traditional
medical education and possibly even
supplanting elements of institutionderived and delivered curricula.
Developing a universal preclinical
curriculum to prepare students for Step
1 could enable schools and students to
face this challenge more effectively.
We surveyed 599 medical students across
the country to explore their perceptions
of Step 1 content coverage by the formal
curriculum at their medical schools
and how those perceptions influenced
their methods and prioritization of
study. Students reported that Step 1
content coverage in the medical school
curriculum was 17.6% less than they
expected on average (a statistically
significant finding). As 40% of students
believed their curriculum lacked
overlap with Step 1 content, students
overwhelmingly turned to review books
and question banks (98%), as well as
to other online preparation products
(79%). Reliance on these third-party
resources prompted a large proportion
of students to shift their focus away from
their medical schools core curriculum
(35%), and, by self-report, their
performance in their schools curriculum
suffered (20%).
The medical student practice of
supplementing and supplanting local
institutional curricula with thirdparty materials may represent a trend
toward decentralization of curricular
content. This observation offers further
justification for the academic medicine
community to reflect on the influence of
the USMLEs on medical school curricula
and student behavior. Does it stifle
innovation and individualization within
medical schools and instead promote the
growth of third-party learning products
that are exempt from oversight and
unregulated by the Liaison Committee on
Medical Education?
When we presented our results at
Learn Serve Lead 2014, attendees were
intrigued by a universal preclinical
curriculum as an innovation (and an

609

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Letters to the Editor

even more radical disruption) to wrest

back control. Implementing a universal
preclinical curriculum could redefine
the requirements for entering medical
school and could even call into question
the necessity of delivering the preStep
1 curriculum in a brick-and-mortar
setting. Students could decide whether to
take courses offered by medical schools,
enroll in courses at other academic
institutions, or study the material on their
own (as they, in part, do now). Perhaps
Step 1 will eventually become the
entrance examination for medical school
admission.
It is essential that medical school curricula
provide greater scope and nuance than
that of the Step 1 blueprint. Consideration
of a universal preclinical curriculum could
help us focus the medical schools primary
role in establishing and implementing
the teaching and evaluation of essential
knowledge, attitudes, and skills required
of students as they are trained to become
competent, entrustable physicians, not
master test takers.
Disclosures: The data reported in this letter were
previously presented at Learn Serve Lead 2014:
The AAMC Annual Meeting, November 9, 2014,
Chicago, Illinois.
Daniel A. London, MD, MS
Orthopaedic surgery resident, Icahn School of
Medicine at Mount Sinai, New York, New York.
He was a medical student, Cleveland Clinic Lerner
College of Medicine of Case Western Reserve
University, Cleveland, Ohio, when this project was
undertaken; daniel.london@mountsinai.org.

Regina Kwon
Medical student, University of Colorado School of
Medicine, Aurora, Colorado.

Anupama Atluru
Medical student, University of Texas Southwestern
Medical School, Dallas, Texas.

Katie Maurer, PhD

MD/PhD student, New York University School of
Medicine, New York, New York.

Ron Ben-Ari, MD
Vice chair for educational affairs, Department of
Medicine, associate dean for continuing medical
education and professional development, and
assistant dean for curriculum, Keck School of
Medicine of University of Southern California, Los
Angeles, California.

Pamela B. Schaff, MD
Associate professor of clinical pediatrics and family
medicine, associate dean for curriculum, and
director, Program in Medical Humanities, Arts, and
Ethics, Keck School of Medicine of University of
Southern California, Los Angeles, California.

610

Reference
1 Prober CG, Kolars JC, First LR, Melnick
DE. A plea to reassess the role of United
States Medical Licensing Examination Step
1 scores in residency selection. Acad Med.
2016;91:1215.

In Reply to Mehta et al and to

London et al: We appreciate the letters
in response to our recent Commentary.1
We hope that our Commentary and this
exchange continue to stimulate others to
consider strategies that enhance medical
education while clarifying the role of
standardized testing in the holistic review
of applicants for residency. A broader
national conversation that includes those
responsible for undergraduate medical
education, graduate medical education,
and regulatory standards governing
medical education and licensure is
critical for meaningful and sustainable
education reform.
Mehta and colleagues suggest abandoning
the United States Medical Licensing
Examination (USMLE) three-digit scores
in favor of a pass/fail score. We outlined
in our Commentary the limitations of
this approach, but it should be recognized
that individual medical schools can
choose to suppress the specific USMLE
scores of their graduates in the Electronic
Residency Application Service. Mehta
and colleagues also suggest that evidence
of medical student engagement in
population health initiatives should be a
factor considered in residency applicant
selection. This is a good example of
program directors deciding what specific
characteristics of applicants are most
critical to success in their programs,
guiding their selection accordingly.
London and colleagues provide
interesting medical student survey data
that underscore the preponderance of
students who rely upon third-party
resources to prepare for their USMLE
Step 1 examination. It has been suggested
that medical students have three
curricula: one that we are teaching, one
that prepares them for their clinical
clerkships, and a third that is relevant to
Step 1. On behalf of optimizing learning,
we must assume responsibility for
harmonizing these curricula.
London and colleagues suggestion of a
universal preclinical curriculum strongly
resonates. Relevant to this point, we
note that the content specifications

for the USMLE are developed using

such a national consensus to which
faculty from most U.S. medical schools
contribute. Furthermore, in a prior
Commentary one of us (C.G.P.)
proposed the creation of a medical
school collaborative, charged with the
identification of material that would
represent a consensus opinion on
the core content of the curriculum.2
We have initiated this process in one
content area (microbiology) and a fivemedical-school collaborative. If this
were extended across the full curriculum
and all medical schools, students would
be assured that they are being tested on
what is being taught, and they would
not feel the need to rely on third-party
sources to prepare for the USMLE.
Disclosures: None reported.
Charles G. Prober, MD
Senior associate dean for medical education and
professor of pediatrics, Microbiology & Immunology,
Stanford School of Medicine, Stanford, California;
cprober@stanford.edu.

Joseph C. Kolars, MD
Senior associate dean for education and global
initiatives, University of Michigan Medical School,
Ann Arbor, Michigan.

Lewis R. First, MD
Professor and chair, Department of Pediatrics,
University of Vermont College of Medicine,
Burlington, Vermont, and past chair, National Board
of Medical Examiners, Philadelphia, Pennsylvania.

Donald E. Melnick, MD
President and chief executive officer, National Board
of Medical Examiners, Philadelphia, Pennsylvania.

References
1 Prober CG, Kolars JC, First LR, Melnick DE.
A plea to reassess the role of United States
Medical Licensing Examination Step 1 scores
in residency selection. Acad Med. 2016;91:14.
2 Prober CG, Khan S. Medical education
reimagined: A call to action. Acad Med.
2013;88:14071410.

Social Media and Storytelling

in Medicine: Probing Deeper
To the Editor: We read with interest
the article Sounding Off on Social
Media: The Ethics of Patient Storytelling
in the Modern Era.1 The authors address
a timely issuehealth professionals
increasing use of social mediaand
present three scenarios involving the
sharing of patient stories on social media
to examine the considerable ethical
issues that arise from this modern form
of digital storytelling. Like the authors,

Academic Medicine, Vol. 91, No. 5 / May 2016

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.