597058

research-article2015
AJMXXX10.1177/1062860615597058American Journal of Medical QualityGupte et al

Article
American Journal of Medical Quality

Together We Learn: Analyzing the

1­–11
© The Author(s) 2015
Reprints and permissions:
Interprofessional Internal Medicine sagepub.com/journalsPermissions.nav
DOI: 10.1177/1062860615597058

Residents’ and Master of Public Health ajmq.sagepub.com

Students’ Quality Improvement Education

Experience

Gouri Gupte, MHA, PhD1, Craig Noronha, MD1, Michal Horný, MSc1,
Karin Sloan, MD1, and Winnie Suen, MD, MSc2

Abstract
Although the value of interprofessional collaborative education has been promoted, it is unclear how teams of clinical
and nonclinical learners perceive this experience. The authors studied an interprofessional quality improvement (QI)
curriculum implemented in 2013 integrating internal medicine residents (n = 90) and Master of Public Health (MPH)
students (n = 33) at an urban safety net academic medical center. Pre and post curriculum surveys assessed attitudes
toward QI and interprofessional education and team performance. Resident attitudes toward learning and engaging
in QI work improved at the end of the curriculum. Overall, MPH students demonstrated significantly more positive
attitudes about interprofessional learning and work than residents. They also agreed more strongly than residents that
patients would benefit if residents and public health students worked together. As health care organizations evolve to
become more integrated, it is crucial that interprofessional educational opportunities be developed and evaluated to
help encourage a culture of collaboration among health care providers.

Keywords
quality improvement, interprofessional, residents, evaluation

Effective collaborations among clinicians, support staff,
administrators, and patients are integral to making
health care patient-centered, safe, timely, equitable,
effective, and efficient.1 Creating opportunities for
trainees to learn and work together may foster an under-
standing of the importance of an interdisciplinary
approach and produce better working relationships.2 In
2011, the Interprofessional Education Collaborative
expert panel (IPEC) identified the following core com-
petencies for interprofessional collaborative practice:
values/ethics for interprofessional practice, roles/
responsibilities, interprofessional communication, and
teams and teamwork.3 The report encouraged integrating
interprofessionalism practice into curricula and assessing
achievement of these competencies.
The IPEC report focuses on systems thinking and col-
laborative approaches that align with the Accreditation
Council for Graduate Medical Education competencies
of practice-based learning and improvement and systems-
based practice, and the new Next Accreditation System
requirement of working in multidisciplinary teams to
solve patient safety problems.3-5 Although the value of
interprofessional collaborative education has been pro-
moted, the approach has rarely been implemented in resi-
dency curricula, especially focusing on nonclinical health
professionals as colearners.6-10 It is unclear how teams of
clinical and nonclinical learners would perceive their
experiences with interprofessional education and collab-
oration as each type of learner may have different
approaches to and perspectives on problem solving and
learning.10 Existing limited research has shown that some
health professionals do have positive attitudes toward
interprofessional learning.6
An interprofessional quality improvement (QI) curric-
ulum was implemented in the 2012-2013 academic year at
1
Boston University, Boston, MA
2
Virginia Commonwealth University, Washington, DC
Corresponding Author:
Gouri Gupte, MHA, PhD, Boston University School of Public Health,
Room 264, 715 Albany Street, Boston, MA 02118.
Email: gourig@bu.edu

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

2 American Journal of Medical Quality

Table 1.  List of Final Quality Improvement (QI) Projects Completed by Internal Medicine Residents and Masters in Public
Health Students During the QI Curriculum.

Intensive care unit (ICU)

•• Improving ICU family meetings
•• Implementation of a spontaneous breathing trial protocol in the ICU
•• Improving time to antibiotic delivery in sepsis
Inpatient
•• Improving inpatient hypoglycemia prevention
•• Improving antiemetics for prevention of chemotherapy-induced nausea and vomiting in inpatients receiving chemotherapy
•• Improving cellulitis management
•• Improving blood transfusion management
•• Improving the early patient discharge process
Transitions
•• Improving laboratory follow-up by providers
Primary care/outpatient
•• Improving the resident continuity clinic experience in the primary care clinic
•• Improving diabetes management point of care laboratory testing in a primary care clinic
•• Improving diabetes and cholesterol management in a primary care clinic
•• Improving primary care clinic postdischarge follow-up
Outpatient
•• Improving provider access to behavioral health services for patients
•• Improving the smoking cessation program
•• Increasing hepatitis C testing in the suboxone program

Boston Medical Center (BMC), an urban safety net aca-
demic medical center in Boston, Massachusetts. The cur-
riculum involved internal medicine residents working on
projects with Master of Public Health (MPH) students,
who functioned as QI consultants. The curriculum was
based on the pilot implemented during the 2011-2012 aca-
demic year, which successfully improved resident QI
knowledge and promoted positive attitudes toward QI.7
Based on feedback received from students and residents,
major changes such as improving the time allocation for
projects, providing sponsorship for the projects, and eval-
uating the interprofessional experience were made to the
2012-2013 curriculum. This article describes the updated
curriculum structure and results of the interprofessional
experience of the 2012-2013 academic year curriculum.
Methods
Setting
Members of the Boston University School of Medicine’s
(BUSM) Department of Medicine and Boston University
School of Public Health’s (BUSPH) Department of Health
Policy and Management collaborated to conduct the sec-
ond iteration of the interprofessional QI curriculum.
Participants
Ninety BMC second postgraduate year (PGY-2) and third
postgraduate year (PGY-3) internal medicine residents
and 33 MPH students enrolled in the Lean Management
and Operations Management in Healthcare courses at
BUSPH participated in the QI curriculum. Four faculty
members—a BUSM clinician experienced with QI and
medical education, a BUSPH faculty member with exper-
tise in QI methodologies and an instructor for the Lean
and Operations Management Courses, the director of
clinical quality at BUSM, and an associate program
director in the internal medicine residency program—
collaborated to direct the curriculum. In addition, 16
BUSM faculty members and staff participated as spon-
sors for the chosen QI projects (Table 1).
Planning
Curriculum planning started 6 months prior to the first
session. The course directors applied the driver diagram
tool, used primarily to guide change in QI implementa-
tion work, to map out curricular improvements (see
online Appendix A, available at http://ajmq.sagepub.
com/supplemental).11 The pre-curriculum planning steps
involved identifying clinical sponsors to champion proj-
ects in clinical areas and to link QI teams with relevant
stakeholders, as well as soliciting interest and commit-
ment from MPH students to participate in the residents’
QI curriculum. Students who elected to participate used
the QI project as their mandatory course assignment.
They then completed BMC’s human resources require-
ments. One month prior to the start of the curriculum,
course faculty provided the residents and MPH students

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

Gupte et al 3

Table 2.  Content Delivered During the Didactic Sessions Held for the Quality Improvement (QI) Curriculum Implemented at
Boston Medical Center for Residents.

Learning Goals Session Activities Suggested Homework

Session 1
  Introduction to QI •• Explain the hospital QI •• Introductions and survey •• Engage clinical staff on
curriculum process approach of plan- project
do-study-act (PDSA)
  Gap Analysis/ •• Develop a process map •• Brief overview of the QI •• Develop current process
Identifying Wastes in process approach map, identification of wastes,
the System obtain available data
  •• Develop a fishbone/5 whys •• Describe curriculum, project,  
diagram and deliverables
  •• Identify wastes in the system •• Process mapping, fishbone  
diagram, identification of
waste
  •• Identify team roles •• Team communication/work  
plan discussion
Session 2
  Data Analysis •• Use data analysis tools in QI •• Paper puppet making activity •• Develop root cause analysis/
work (eg, Pareto, control to demonstrate use of data fishbone diagram
charts, histograms) analysis tools
  •• Team meeting •• Start PDSA
Session 3
  Challenges in QI work •• Identify reasons why QI •• Discussion of challenges that •• Complete PDSA
projects could fail arise in QI
  •• Successful project showcase •• Complete Presentation
  •• Team meeting  
Session 4
 Presentations •• Coherently and effectively •• 10-minute presentation,  
present project to audience 3-minute question session for
each group in front of peers,
staff, and judges
  •• Curriculum evaluation  

with a list of 28 projects to select from according to their
personal or career interests. Sixteen teams, each consist-
ing of 1 to 2 MPH students and 4 to 6 residents, were then
formed based on individual preferences (Table 1).
Curriculum Structure and Content
The 16-week QI curriculum (January-May 2013) was
incorporated into the 3-week inpatient 1-week ambula-
tory (3+1) residency schedule model, in which QI session
time was provided for a half day during the ambulatory
week. The QI curriculum (Table 2) ran for 4 months with
each group meeting for 90 minutes once per month.
Given this 3 + 1 residency schedule, residents were
divided equally into 4 groups and further subdivided into
project teams.7 Half of the session time provided just-in-
time learning on QI topics, while the remainder was used
for project work. The teams created their own meeting
and project work schedules outside of these sessions. The
groups also were provided with another hour for project
work during the ambulatory weeks. Sponsors were
invited to attend all sessions, although many were unable
to attend because of scheduling conflicts.
Concurrently, the MPH students received in-depth
training on various tools that are important in operations
management, project planning, as well as QI tools and
methods during their week-long Lean Management
course prior to the start of the resident-MPH QI curricu-
lum and their semester-long Operations Management
course. Similar to the pilot curriculum, MPH students
then served as QI consultants and were required to attend
all 4 QI sessions with the residents.7
Data Collection and Analysis
A 38-question pre-curriculum survey adapted from 3
instruments—Quality Improvement Knowledge
Application Tool (QIKAT), the Tess et al14 QI assessment,
and the Readiness for Interprofessional Learning Scale
(RIPLS)—was administered in Session 1 to evaluate QI
content knowledge and attitudes (see online Appendix B,
available at http://ajmq.sagepub.com/supplemental).12-15

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

4 American Journal of Medical Quality
The survey included 5 demographic questions and 33
five-point Likert scale questions (1 = strongly disagree; 5
= strongly agree). The post survey provided in Session 4
included 3 demographic questions, the 33 questions from
the QIKAT and RIPLS, and 18 six-point Likert scale ques-
tions (1 = none of the time; 6 = all of the time) from the
Team Performance Scale.16 The RIPLS and Team
Performance Scale were administered to both MPH stu-
dents and residents, while the condensed version of the
QIKAT survey was administered only to the residents.
Demographic information was examined using descrip-
tive statistics and differences between the subpopulations
were assessed by a χ2 test. Prior to the response analysis,
the internal consistency and reliability of the 3 question-
naires was verified using the Cronbach α analysis. The
attitudes toward interprofessional education questionnaire
contained several questions with opposite scoring and
meaning from the remaining items; therefore, the
responses to these questions were reverse coded for the
purposes of the Cronbach α analysis. Differences in
responses to individual questions were assessed by the
Wilcoxon rank sum test. Because of a very low proportion
of males among MPH students, a robustness check of the
results was performed by rerunning all analyses for
females only. All analyses were generated using SAS soft-
ware, version 9.3 (SAS Institute Inc., Cary, North
Carolina). Figures were generated using the statistical
software R, version 3.2.0. Finally, free-text answers found
in the comments section of the survey were analyzed and
common themes were derived. The BMC Institutional
Review Board deemed this project exempt from review.
Results
Survey
In the pre-curriculum survey, the attitudes toward QI edu-
cation and interprofessional education questionnaires
showed good internal consistency (Cronbach α of .825
and .894, respectively). The Team Performance Survey
Pre-Curriculum
EssPre
MPH Students 0% 0%
p < .0001
Residents 3% 10%
Response Very non-essential
ConfiPre
How confident are you that you can make a change to improve health care in a local setting
MPH Students 0% 32%
p = .0002
Residents 24% 48%
Response
Figure 1.  Survey results comparing Master of Public Health (MPH) students’ and residents’ attitudes toward interprofessional education.
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
questionnaire showed an excellent internal consistency
with a Cronbach α of .974.
Participants
A total of 123 individuals (90 residents and 33 MPH stu-
dents) participated in the 2012-2013 academic year cur-
riculum. In all, 67.5% of the participants (34 PGY-2
residents, 28 PGY-3 residents, and 21 MPH students)
completed the pre-curriculum survey, and 66.7% of the
participants (34 PGY-2 residents, 27 PGY-3 residents,
and 21 MPH students) completed the post-curriculum
survey. There were approximately equal numbers of
males and females among residents (30 females, 32
males), but substantially more females than males among
MPH students (17 females, 2 males, 2 did not disclose
their sex). More residents (61%, n = 38) than MPH stu-
dents (29%, n = 6) had prior exposure to QI (eg, personal
improvement project, workplace initiative, other experi-
ence that used formal improvement principles and meth-
ods) (P = .04). However, a similar share of residents
(29%, n = 18) and MPH students (29%, n = 6) had prior
formal training in QI techniques (P = .73).
Assessment of Attitudes Toward
Interprofessional Education
The responses of MPH students and residents both before
and after the curriculum are summarized in Figure 1.
MPH students demonstrated significantly more positive
attitudes toward interprofessional learning and work than
residents for all questions before the curriculum and in
almost all questions after the curriculum. MPH students
agreed more strongly than residents both before and after
the curriculum that patients would benefit if medical resi-
dents and public health students worked together, that
shared learning would increase understanding of clinical
problems, that communication skills should be learned
with other medical residents and public health students,
Post-Curriculum
How essential do you consider quality improvement in your future professional work EssPost
100% 5% 0% 95%
p = .36
87% 0% 98%
Non-essential Neither essential nor non-essential Essential Very essential
ConfiPost
68% 0% 40% 60%
p = .75
27% 17% 29% 54%
Not confident at all Non confident Reasonably confident Confident Very confident
(continued)
Gupte et al 5

Pre-Curriculum Post-Curriculum
Learning
ProfPrewith other professionals will make me a more effective member of a medical and public health team
ProfPost

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p < .0001

Residents 0% 11% 89% 0% 2% 98%

Patients would ultimately benefit if medical residents and public health students worked together
BenefPre BenefPost

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p = .0001

Residents 2% 15% 84% 5% 9% 86%

SharedPre
Shared learning SharedPost
with other medical residents and public health students will increase my ability to understand clinical problems

MPH Students 0% 5% 95% 0% 0% 100%

p = .0001 p = .0001
Residents 5% 11% 84% 2% 20% 78%

CommPre CommPost
Communications skills should be learned with other medical residents and public health students

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p < .0001

Residents 2% 21% 77% 7% 88%

100 50 0
TeamPre Team-working50
skills are vital for all100 100and public health students
medical residents 50 to learnTeamPost 50 100
Percentage Percentage
MPH Students 0% 0% 100% 0% 95%

p < .0001 p < .0001

Residents Response
0% Strongly disagree Disagree
6% Neutral Agree Strongly agree
94% Response
0% Strongly disagree Disagree Agree Strongly agree
98%

LimitPre Shared learning will help me to understand my own professional limitations LimitPost

MPH Students 5% 0% 95% 0% 0% 100%

p < .0001 p < .0001

Residents 0% 16% 84% 0% 11% 89%

FinishPre
Learning between medical residents FinishPost
and public health students before finishing residency/school would improve working relationships after finishing residency/school

MPH Students 0% 10% 90% 0% 10% 90%

p < .0001 p < .0001

Residents 2% 31% 68% 0% 23% 77%

Shared learning will help me think positively about other medical residents and public health professionals
PositPre PositPost

MPH Students 0% 10% 90% 10% 0% 90%

p < .0001 p < .0001

Residents 3% 16% 80% 0% 20% 80%

SmallPre SmallPost
For small-group learning to work, students/medical residents need to respect and trust each other

MPH Students 0% 0% 100% 10% 90%

p < .0001 p = .01

Residents 2% 2% 97% 5% 95%

WastePre WastePost
I don't want to waste time learning with other medical residents and public health students / professionals

MPH Students 95% 0% 5% 100% 0% 0%

p < .0001 p < .0001

Residents 73% 13% 15% 71% 25% 4%

100 50 0
NecesPre 50
It is not necessary 100
for medical residents 100health students to50
and public 0
learn together NecesPost 50 100
Percentage Percentage
MPH Students 90% 10% 0% 95% 5% 0%

p < .0001 p < .0001

Residents Response
76% Strongly disagree Disagree
13% Neutral Agree Strongly agree
11% Response Strongly disagree Disagree
29% Neutral Agree Strongly agree
5%

Clinical problemClinicPre ClinicPost

solving can only be learned effectively with medical residents/ students / professionals from my own school / organization

MPH Students 95% 5% 0% 14% 10%

p < .0001 p = .16

Residents 66% 21% 13% 11% 7%

Comm1Pre
Shared learning with Comm1Post
other medical residents and public health students will help me to communicate better with patients and other professionals

MPH Students 0% 0% 100% 5% 0% 95%

p < .0001 p < .0001

Residents 6% 13% 81% 5% 20% 75%

WelcoPre
I would welcome the opportunity to work on small group projects with other medical residents and public health WelcoPost
students / professionals

MPH Students 0% 0% 100% 5% 0% 95%

p < .0001 p < .0001

Residents 3% 15% 82% 2% 41% 57%

GenerPre
I would welcome the opportunity GenerPost
to share some generic lectures, tutorials or workshops with other medical residents and public health students / professionals

MPH Students 0% 5% 95% 0% 5% 95%

p < .0001 p < .0001

Residents 3% 16% 81% 7% 38% 55%

ClariPreShared learning and practice will help me clarify the nature of patients' or clients' problemsClariPost

MPH Students 0% 5% 95% 0% 10% 90%

p < .0001 p < .0001

Residents 2% 23% 76% 7% 27% 66%

100 50 BetterPre
0 Shared learning
50 before and after100
residency will help
100me become a better
50 team worker BetterPost
0 50 100
Percentage Percentage
MPH Students 0% 0% 0% 0% 0% 0%

Residents Response
2% Strongly disagree Disagree
16% Neutral Agree Strongly agree
82% Response
0% Strongly disagree Disagree
18% Neutral Agree Strongly agree
82%

UnsurePre I am not sure what my professional role will be / is UnsurePost

MPH Students 0% 0% 0% 0% 0% 0%

Residents 44% 34% 23% 77% 16% 7%

I have to acquire much more knowledge and skill than other students / professionals in my own faculty
AcquirePre / organization
AcquirePost

MPH Students 0% 0% 0% 0% 0% 0%

Residents 21% 52% 26% 53% 35% 13%

100 50 0 50 100 100 50 0 50 100

Percentage Percentage

Response Strongly disagree Disagree Response

Neutral Strongly
Agree disagree Disagree Response
Strongly agree Neutral Agree disagree
Strongly Strongly agree
Disagree Neutral Agree Strongly agree

Figure 1. (continued)
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
6 American Journal of Medical Quality
and that shared learning would help them better under-
stand their own professional limitations (P < .01 for all).
The results were robust to gender differences. The next
assessment focused on how attitudes toward interprofes-
sional education changed after the curriculum for the
samples of MPH students and residents separately. The
findings were that attitudes improved significantly among
residents while there were essentially no significant
changes in the attitudes of the MPH students.
Assessment of Residents’ Attitudes Toward QI
in General and QI in the Residency Program
The pre- and post-curriculum QIKAT questionnaires
showed that the QI curriculum and projects did signifi-
cantly improve residents’ attitudes toward QI work and
the residency program’s focus on QI (Figure 2)—although
the results of improvement in attitudes assessment were
more pronounced among PGY2 residents. After the cur-
riculum, residents felt more strongly that QI is an impor-
tant educational topic (P < .01) and that they play a role
in designing QI changes in the hospital and/or clinic (P <
.01). The results were robust to gender differences.
On further analyzing differences in responses by sex
prior to the curriculum, it was found that female residents
felt more strongly than male residents (P < .01) that their
ideas to improve patient care were sought and used con-
structively by hospital and/or clinic leaders, but this
difference was no longer observed after the curriculum
(P = .18). Understanding one’s own role within the mul-
tidisciplinary team generally improved throughout the QI
curriculum (P < .01) for both males and females.
However, male residents demonstrated stronger attitudes
than female residents consistently before (P = .029) and
after (P = .031) the QI curriculum.
Overall, there were no significant differences in atti-
tudes toward QI education between PGY-2 and PGY-3
residents before the curriculum; however, PGY-2 residents
had more positive feelings about how the residency was
preparing them to deliver high-quality care (medianPGY-2 =
strongly agree/agree; medianPGY-3 = agree; P = .046) and
to think analytically in clinical situations (medianPGY-2 =
strongly agree; medianPGY-3 = agree; P = .001) than PGY-3
residents after the QI curriculum.
Assessment of Team Performance
MPH students evaluated team performance post curricu-
lum slightly better than residents (Figure 3); however,
these differences were no longer significant when the 2
data samples were restricted to females only. In general,
both MPH students and residents rated team performance
relatively high, with median responses ≥5 on a 6-point
scale.
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
Reflections by Residents and MPH Students
An open-ended question in the post survey invited resi-
dents and MPH students to comment about the curriculum
experience. Residents requested more time for the projects
(n = 20), to start the curriculum earlier in the semester (n =
4), and reported individual team member issues (n = 3).
Residents also commented on the difficulty of prioritizing
and balancing clinical work while working as a team
because of busy schedules (n = 2). MPH students made
comments about difficulties communicating and contact-
ing team members, as residents were not responsive to
e-mails when clinically busy with challenging work sched-
ules and clinical priorities (n = 11). One MPH student com-
mented that the experience was “worthwhile, with initial
‘growing pains’ while working with the residents that later
normalized.” MPH students reported concerns of not being
respected during their teamwork experience (n = 13). One
resident made a comment that was representative of others
regarding working with MPH students:
QI is very important to me and I truly believe that it enhances
medical education/clinical training. . . . Additionally, while
gaining perspective from MPH students is valuable, I feel it
didn’t help much in this situation because both sides (MPH
& MD) were on two different wavelengths with two different
goals of implementation, which really didn’t come to a
common ground.
Discussion
Accrediting bodies have encouraged interprofessional
teamwork and collaboration in medicine to improve
care delivery processes and outcomes; to reduce staff
turnover, absenteeism, and burnout; to increase staff
motivation; and to reduce staff conflict.5,17-20 A QI cur-
riculum created at BMC with residents and MPH stu-
dents promoted an interprofessional learning and
working environment while teaching QI tools and sys-
tems thinking. Surprisingly, the findings show glaring
differences between MPH students and residents with
regard to interprofessional engagement and collabora-
tion. MPH students had more favorable attitudes than
residents toward working in teams, understanding clini-
cal problems, and improving patient care. Additionally,
MPH students were more interested in improving their
communication and understanding of their professional
limitations through these shared learning activities than
residents. Gender-based differences were observed, with
women participants placing significantly more impor-
tance on QI knowledge and interprofessional work than
male colleagues from the residency program.
Many of the previously published interprofessional QI
curricula included only clinical professionals working in
teams.14,21,22 A survey focusing on attitudes of faculty
Gupte et al 7

I feel that quality improvement is an important educational topic

Pre-Curriculum 2% 12% 87%

p = .001
Post-Curriculum 0% 0% 100%

I play a role in designing quality improvement changes in the hospital and/or clinic

Pre-Curriculum 30% 33% 37%

p = .0003
Post-Curriculum 8% 21% 71%

I play a role in implementing quality improvement changes in the hospital and/or clinic

Pre-Curriculum 20% 17% 63%

p = .13
Post-Curriculum 8% 13% 79%

I want to learn more about how to improve quality in a hospital/clinic

Pre-Curriculum 3% 15% 82%

p = .23
Post-Curriculum 0% 8% 92%

My ideas to improve patient care are sought and used constructively by hospital and/or clinic leaders

Pre-Curriculum 25% 42% 32%

p = .17
Post-Curriculum 17% 36% 47%

I feel the education I receive about patient safety and health care quality is relevant to the clinical care I deliver

Pre-Curriculum 2% 13% 85%

p = .029
Post-Curriculum 0% 2% 98%

100 50 0 to practice medicine safely

I feel residency is preparing me 50 100
Percentage
Pre-Curriculum 2% 3% 95%

p = .004
Post-Curriculum 0%
Response Strongly disagree Disagree
0%
Neutral Agree Strongly agree 100%

I feel residency encourages me to analyze my own practice and make changes based on what I learn

Pre-Curriculum 0% 18% 82%

p = .007
Post-Curriculum 0% 6% 94%

I feel I understand my role within the multidisciplinary team (physicians, nurses, social workers, case managers, PT, and OT) caring for patients on the medical floors

Pre-Curriculum 0% 12% 88%

p = .007
Post-Curriculum 0% 0% 100%

I feel residency is preparing me to deliver high-quality care to my patients

Pre-Curriculum 2% 7% 92%

p = .007
Post-Curriculum 0% 0% 100%

I feel residency is preparing me to think analytically in clinical situations

Pre-Curriculum 0% 2% 98%

p = .046
Post-Curriculum 0% 0% 100%

I feel our residency encourages me to learn with other professionals and my peers

Pre-Curriculum 2% 5% 93%

p = .35
Post-Curriculum 0% 2% 98%

100 50 0 errors and waste in healthcare

I feel comfortable understanding 50 100
Percentage
Pre-Curriculum 8% 30% 62%

p < .0001
Post-Curriculum 2%
Response Strongly disagree Disagree
4%
Neutral Agree Strongly agree 94%

I am satisfied overall with the quality of care I deliver

Pre-Curriculum 3% 13% 83%

p = .13
Post-Curriculum 2% 4% 94%

100 50 0 50 100
Percentage

Response Strongly disagree Disagree Neutral Agree Strongly agree

Figure 2.  Survey comparing residents’ attitudes toward quality improvement (QI) in general and QI in the residency program
before and after the quality improvement curriculum.

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

8 American Journal of Medical Quality

All team members made an effort to participate in discussions

MPH Students 21% 79%

p = .46
Residents 17% 83%

When team members had different opinions, each member explained his or her point of view

MPH Students 11% 89%

p = .29
Residents 2% 98%

Team members encouraged one another to express their opinions and thoughts

MPH Students 11% 89%

p = .24
Residents 5% 95%

Team members shared and received criticism without making it person

MPH Students 0% 100%

p = .20
Residents 0% 100%

Different points of view were respected by team members

MPH Students 0% 100%

p = .053
Residents 0% 100%

Often members helped a fellow team member to be understood by paraphrasing what he or she was saying

MPH Students 11% 89%

p = .03
Residents 7% 93%

100 50 for problem solving (such as brainstorming)

My team used several techniques 0 with each team member50
presenting his or her best ideas 100
Percentage
MPH Students 5% 95%
p = .009
Residents 2% Response None of the time Very rarely Rarely Occasionally Very frequently All of the time 98%

Team members worked to come up with solutions that satisfied all members

MPH Students 0% 100%

p = .01
Residents 0% 100%

All team members consistently paid attention during group discussions

MPH Students 11% 89%

p = .04
Residents 14% 86%

My team actively elicited multiple points of view before deciding on a final answer

MPH Students 5% 95%

p = .23
Residents 0% 100%

Team members listened to each other when someone expressed a concern about individual or team performance

MPH Students 0% 100%

p = .18
Residents 5% 95%

Team members willingly participated in all relevant aspects of the team

MPH Students 11% 89%

p = .05
Residents 14% 86%

100 50 members resolved differences of0 opinion by openly speaking their mind
Team 50 100
Percentage
MPH Students 0% 100%
p = .10
Residents 10% Response None of the time Very rarely Rarely Occasionally Very frequently All of the time 90%

Team members used feedback about individual or team performance to help the team be more effective

MPH Students 16% 84%

p = .04
Residents 18% 82%

Team members seemed attentive to what other team members were saying when they spoke

MPH Students 5% 95%

p = .11
Residents 7% 93%

My team resolved many conflicts by compromising between team members, with each one giving a little

MPH Students 5% 95%

p = .16
Residents 8% 92%

Members who had different opinions explained their point of view to the team

MPH Students 11% 89%

p = .16
Residents 5% 95%

Team members were recognized when something they said helped the team reach a good decision

MPH Students 0% 100%

p = .12
Residents 2% 98%

100 50 0 50 100
Percentage

Response None of the time Very rarely Rarely Occasionally Very frequently All of the time

Figure 3.  Survey results comparing Master of Public Health (MPH) students’ and residents’ perceptions of team performance
post curriculum. Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
Gupte et al 9
members from various health care professions toward
interprofessional education had reported that, overall,
medical faculty had less interest in working with indi-
viduals from other health professions who are not usually
in their immediate clinical team, such as pharmacists or
social workers.23 The present study at BMC showed simi-
lar results for collaborations between clinical and non-
clinical professionals. The authors partly subscribe to the
theory that professional cultures may explain the differ-
ences between attitudes toward interprofessional team-
work and education. Traditionally, different paradigms
drive different disciplines. Physician training has focused
more on teaching learners to be team leaders and to work
with other clinical professionals, to be able to think inde-
pendently about the situation and derive treatment plans
that may be applied in a more paternalistic and authoritar-
ian manner.9,24,25 Traditionally, MPH students as well as
pharmacists and social workers have learned and worked
on projects that involve more collaborative practices.26-28
Although current training and competencies have been
developed to encourage interprofessional teamwork and
relationship building in medical programs, culture change
and historical hierarchies will take time to evolve and be
ingrained into practice.
Another potential explanation for the differences in
attitudes toward interprofessional work between MPH
students and residents may be attributable to how distinct
professions perceive and interpret issues based on their
cognitive map.9,29,30 These cognitive maps are likely cre-
ated because instructors teaching different professions
may create curricula, objectives, and competencies in iso-
lation of each other. As a result “. . . quite literally, two
opposing ‘disciplinarians’ can look at the same thing and
not see the same thing . . .”29(p. 35) Therefore, students not
only learn the skills and values of their profession but also
assume the occupational identity commonly referred to as
“professionalization.”29,31,32 Identifying the differences in
cognitive maps and professional perceptions and raising
them for discussion during the interprofessional curricu-
lum introduction may help learners better understand their
team experience, roles, and potential challenges.29
The observed gender differences wherein females
had a more positive attitude toward teamwork and col-
laboration corresponded to already established find-
ings.23,33 Various studies have suggested that there are
important differences in the way the 2 genders interact
and adopt new methodologies. Women are known to be
more sensitive to relationships and work harder to main-
tain a team environment.34,35 In addition, women tend to
show less skepticism toward new interventions or meth-
odologies.34 Consciously assigning and evaluating a
certain gender mix on the teams in future iterations of
the curriculum may be helpful in determining how this
characteristic could influence overall attitudes toward
interprofessionalism.
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
Strengths of the Program
One of the strengths of this curriculum is the unique col-
laboration between medicine and public health providers,
both as learners and as instructors. The curriculum was
made possible because BUSPH was physically next to
the hospital where the internal medicine residents were
training. In addition, the residency program director,
chair of medicine, internal medicine faculty, School of
Public Health faculty, and members of the hospital qual-
ity department were supportive of the curriculum. This
allowed for collaborations across schools, departments,
and the hospital.
Another strength is in the design of the program. The
QI curriculum was structured in a way to incorporate
into the 3 + 1 residency scheduling system for all resi-
dents and matched the semester schedule of the MPH
students taking either Operations Management or Lean
Management courses. The program also was designed to
promote active and reflective learning styles based on
principles of renowned education frameworks such as
Bloom’s taxonomy, Knowles adult learning theory, and
the Kirkpatrick framework.36-38
The evaluation of interprofessional attitudes of the
clinical and nonclinical learners toward the curriculum is
a strength of the curriculum. There are tools to evaluate
QI knowledge and tools to assess interprofessionalism,
but it is unique to combine these 2 parts into an evalua-
tion of a curriculum that encompasses both.
Challenges for the Program
One of the major challenges for the curriculum was the
incorporation of more project and meeting time for resi-
dents who were trying to balance other clinical and
research obligations. The lack of protected time outside
the ambulatory week may have affected both residents’
and MPH students’ attitudes toward interprofessional
work. However, the results may also be a true reflection
of real life where many clinicians have to participate in
QI work as part of their clinical expectations and mainte-
nance of certification requirements. Many physicians
refer to it as a “second unpaid job.”
Another challenge the participants faced was defining
clear roles for each other, as well as goals and objectives
for the project. Because residents were frequently clini-
cally busy for several weeks at a time during their inpa-
tient rotation, the MPH students often had to carry the
team and hold the project together. Because of a lack of
effective communication at times, residents and MPH
students ultimately focused on different parts of the proj-
ect, leading to disjointed end products that required addi-
tional time to be made cohesive.
The team structure may affect the effectiveness of the
teamwork and outcomes of the project. This curriculum
10 American Journal of Medical Quality
included both PGY-2 and PGY-3 residents. Based on the
evaluation results, the authors believe that the PGY-3
residents may have been less invested in the curriculum
and could have negatively affected the attitudes of the
group. The relatively large size of each group (ie, 4-6
residents and 1-2 MPH students) also may have reduced
the sense of teamwork and individual responsibility of
team members.
The perceived and actual role of the sponsors also
affected the interprofessional team experience. In some
cases, the sponsor’s expectations differed from the resi-
dent-MPH team’s expectations, which again led to a frag-
mented project product. However, the authors have
observed that some projects have led to ongoing work
and improved processes in patient care thanks to the addi-
tion of faculty sponsors to the curriculum.
This study has some limitations. First, although there
was a large cohort of residents and MPH students, the gen-
eralizability of the findings is limited because the partici-
pants represent only a single cohort, specialty, department,
and institution. Second, despite the strong validity of the
constructs in the questionnaire, there is room for improve-
ment in future versions (eg, controlling for the colearning
environment, considering the context of daily work).
Future of This Curriculum
As part of future steps, the authors plan to develop a com-
prehensive 3-year evaluative strategy using psychometric
assessment tools and 360-degree feedback mechanisms
from main stakeholders of the curriculum inclusive of resi-
dents, public health students, and participating faculty and
sponsors of the QI projects; to build in more training for
team formation and consideration of gender differences; to
create more partnerships with other professions such as
nursing and engineering; and to involve medical students
to provide residents with the opportunity to examine QI
from multiple perspectives. Last, the authors plan to evolve
the content to the next level through introduction of the
concepts of human factors and system errors.
Conclusion
As health care organizations evolve to become more inte-
grated, it is crucial that interprofessional educational
opportunities be developed and evaluated to help encour-
age a culture of collaboration among health care provid-
ers. The approach needs to be more interdisciplinary, not
just multidisciplinary. Future studies of interprofessional
curricula also should investigate the effect of gender mix
on the teams and consider collaborative learning early in
the medical education process to foster improved attitudes
toward interprofessionalism. National School of Public
Health and School of Medicine committees both
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
recommend that schools actively seek opportunities for
collaboration in education to increase the number of grad-
uates who will collaborate across disciplines.26,39 The sus-
tainability of these kinds of interprofessional curricula can
be made possible with institutional support and strategi-
cally progressive medical education programs that under-
stand the upcoming needs of health care. QI is a “team
sport”; therefore, these curricula can be used as a good
starting point for current residents. However, the authors
encourage that these assessments and strategies for more
collaboration start very early in the training programs.
Authors’ Note
Prior presentation: Gupte G, Suen W, Sloan K, Noronha C. An
interprofessional quality improvement curriculum for internal
medicine residents and masters of public health students: the
lean way; Association of University Programs in Health
Administration Annual Meeting; San Antonio, Texas; 2014.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The authors received the following financial support for the
research, authorship, and/or publication of this article: Education
Pilot Grant, Department of Medicine, Boston University, July 1,
2012 to June 30, 2013.
References
1. Institute of Medicine. Crossing the Quality Chasm: A New
Health System for the 21st Century. Washington, DC:
National Academies Press; 2001.
2. Lerner S, Magrane D, Friedman E. Teaching teamwork in
medical education. Mt Sinai J Med. 2009;76:318-329.
3. Interprofessional Education Collaborative Expert Panel. Core
Competencies for Interprofessional Collaborative Practice:
Report of an Expert Panel. Washington, DC: Interprofessional
Education Collaborative Expert Panel; 2011.
4. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME
accreditation system—rationale and benefits. N Engl J
Med. 2012;366:1051-1056.
5. Accreditation Council of Graduate Medical Education.
Next accreditation system. http://www.acgme.org/acg-
meweb/tabid/435/ProgramandInstitutionalAccreditation/
NextAccreditationSystem.aspx. Accessed November 14,
2014.
6. Giordano C, Umland E, Lyons KJ. Attitudes of faculty
and students in medicine and the health professions toward
interprofessional education. J Allied Health. 2012;41(1):
21-25.
7. Weigel C, Suen W, Gupte G. Using lean methodology to
teach quality improvement to internal medicine residents at
a safety net hospital. Am J Med Qual. 2013;28:392-399.
Gupte et al 11
8. Reid R, Bruce D, Allstaff K, McLernon D. Validating the
Readiness for Interprofessional Learning Scale (RIPLS)
in the postgraduate context: are health care professionals
ready for IPL? Med Educ. 2006;40:415-422.
9. Hall P. Interprofessional teamwork: professional cultures
as barriers. J Interprof Care. 2005;19(suppl 1):188-196.
10. Payler J, Meyer E, Humphris D. Pedagogy for interprofes-
sional education—what do we know and how can we eval-
uate it? Learning in Health and Social Care. 2008;7:64-78.
11. Langley GJ, Moen R, Nolan KM, Nolan TW, Norman
CL, Provost LP. The Improvement Guide: A Practical
Approach to Enhancing Organizational Performance. San
Francisco, CA: Jossey-Bass; 2009.
12. Morrison L, Headrick L, Ogrinc G, Foster T. The quality
improvement knowledge application tool: an instrument
to assess knowledge application in practice-based learning
and improvement. Paper presented at: Society of General
Internal Medicine meeting; Vancouver, British Columbia,
Canada; May 7, 2003.
13. Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and
assessing resident competence in practice-based learning and
improvement. J Gen Intern Med. 2004;19(5 pt 2):496-500.
14. Tess AV, Yang JJ, Smith CC, Fawcett CM, Bates CK,
Reynolds EE. Combining clinical microsystems and an
experiential quality improvement curriculum to improve
residency education in internal medicine. Acad Med.
2009;84:326-334.
15. Parsell G, Bligh J. The development of a questionnaire to
assess the readiness of health care students for interprofes-
sional learning (RIPLS). Med Educ. 1999;33:95-100.
16. Thompson BM, Levine RE, Kennedy F, et al. Evaluating
the quality of learning-team processes in medical educa-
tion: development and validation of a new measure. Acad
Med. 2009;84(10 suppl):S124-S127.
17. Zwarenstein M, Goldman J, Reeves S. Interprofessional
collaboration: effects of practice-based interventions on
professional practice and healthcare outcomes. Cochrane
Database Syst Rev. 2009;(3):CD000072.
18. Coleman MT, Roberts K, Wulff D, van Zyl R, Newton K.
Interprofessional ambulatory primary care practice-based
educational program. J Interprof Care. 2008;22(1):69-84.
19. McPherson K, Headrick L, Moss F. Working and learning
together: good quality care depends on it, but how can we
achieve it? Qual Health Care. 2001;10(suppl 2):ii46-ii53.
20. Varkey P, Reller MK, Smith A, Ponto J, Osborn M. An
experiential interdisciplinary quality improvement educa-
tion initiative. Am J Med Qual. 2006;21:317-322.
21. Varkey P, Karlapudi SP. Lessons learned from a 5-year
experience with a 4-week experiential quality improve-
ment curriculum in a preventive medicine fellowship. J
Gen Intern Med. 2009;1:93-99.
22. Wong BM, Etchells EE, Kuper A, Levinson W, Shojania
KG. Teaching quality improvement and patient safety
to trainees: a systematic review. Acad Med. 2010;85:
1425-1439.
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
23. Curran VR, Sharpe D, Forristall J, Flynn K. Attitudes of
health sciences students towards interprofessional team-
work and education. Learning in Health and Social Care.
2008;7:146-156.
24. Robinson M, Cottrell D. Health professionals in multi-dis-
ciplinary and multi-agency teams: changing professional
practice. J Interprof Care. 2005;19:547-560.
25. Reese DJ, Sontag MA. Successful interprofessional collabora-
tion on the hospice team. Health Soc Work. 2001;26:167-175.
26. Maeshiro R, Johnson I, Koo D, et al. Medical education
for a healthier population: reflections on the Flexner Report
from a public health perspective. Acad Med. 2010;85:
211-219.
27. Varda D, Shoup JA, Miller S. A systematic review of col-
laboration and network research in the public affairs litera-
ture: implications for public health practice and research.
Am J Public Health. 2012;102:564-571.
28. Horsburgh M, Lamdin R, Williamson E. Multiprofessional
learning: the attitudes of medical, nursing and pharmacy
students to shared learning. Med Educ. 2001;35:876-883.
29. Petrie HG. Do you see what I see? The epistemology of inter-
disciplinary inquiry. J Aesthetic Educ. 1976;10(1):29-43.
30. Itano JK, Williams J, Deaton MD, Oishi N. Impact of a
student interdisciplinary oncology team project. J Cancer
Educ. 1991;6:219-226.
31. Hall P, Weaver L. Interdisciplinary education and teamwork:
a long and winding road. Med Educ. 2001;35:867-875.
32. Loseke DR, Cahill SE. Actors in search of a character:
student social workers’ quest for professional identity.
Symbolic Interaction. 1986;9:245-258.
33. Hertweck M, Hawkins S, Bednarek M, Goreczny A,
Schreiber J, Sterrett S. Attitudes toward interprofes-
sional education: comparing physician assistant and other
health care professions students. J Physician Assist Educ.
2012;23(2):8-15.
34. Wilhelmsson M, Ponzer S, Dahlgren LO, Timpka T,
Faresjö T. Are female students in general and nurs-
ing students more ready for teamwork and interprofes-
sional collaboration in healthcare? BMC Med Educ.
2011;11(1):15.
35. Fox MF. Women, science, and academia: graduate educa-
tion and careers. Gender Society. 2001;15:654-666.
36. Bloom BS, Krathwohl DR. Taxonomy of Educational
Objectives: The Classification of Educational Goals.
Handbook I: Cognitive Domain. New York, NY: David
McKay; 1956.
37. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training
Programs: The Four Levels. San Francisco, CA: Berrett-
Koehler; 2006.
38. Knowles MS. The Modern Practice of Adult Education. Vol
41. New York, NY: New York Association Press; 1970.
39. Gebbie K, Rosenstock L, Hernandez LM. Who Will Keep
the Public Healthy? Educating Public Health Professionals
for the 21st Century. Washington, DC: National Academies
Press; 2003.