1

Engineering student problem solving

processes and text-to-diagram translations



Nikita Dawe (996562116)












Supervisor: Professor Susan McCahan


MIE498 Full-year Thesis: Engineering Education
Department of Mechanical and Industrial Engineering
University of Toronto
April 2014




2









i. Abstract
This engineering education thesis investigates problem solving processes in the context of student
learning styles, with a focus on text-to-diagram translations. An exploratory study was conducted in
which participants performed a think-aloud problem solving activity; written work was analyzed for
accuracy, and verbal transcripts were coded and analyzed qualitatively. Results indicate that students have
not been taught explicit problem solving processes or representational tools, and students with very strong
visual or verbal learning preferences will perform less accurate translations. These findings have
implications for engineering educators and students and should be investigated further to inform the
development of educational interventions.


3

ii. Acknowledgements
I would like to thank Professor Susan McCahan for providing me with the opportunity to pursue an
engineering education thesis on a topic that intrigued me. I am grateful for the guidance and feedback
offered throughout this project.
I would also like to thank the students who volunteered to participate in the study; their willingness to
find time for the research activity between academic and extracurricular commitments is appreciated. It is
not always easy to verbalize internal thoughts and reasoning, but the think-aloud material provided useful
data for analysis and interpretation.
Thank you to Matt Strohack in the Engineering Communication Program for reviewing my thesis when it
was very much a work-in-progress.
Finally, I am grateful to friends, peers, and mentors for their support and motivation.


4

Table of Contents
i. Abstract ..........................................................................................................................................2
ii. Acknowledgements.........................................................................................................................3
List of tables ......................................................................................................................................7
1 Introduction ................................................................................................................................8
2 Literature Review ........................................................................................................................9
2.1 Engineering Education and Problem Solving .........................................................................9
2.1.1 Problem Solving ...........................................................................................................9
2.2 Problem Solving Models ....................................................................................................10
2.2.1 The text to diagram to symbol (TDS) algorithm ............................................................10
2.2.2 The Integrated Problem Solving (IPS) Model ...............................................................11
2.3 Learning Styles and Teaching Practices ...............................................................................12
2.3.1 Student Learning Styles ..............................................................................................12
2.4 Learning Style Categorization Tools ...................................................................................13
2.4.1 The Kolb Learning Style Inventory (LSI) .....................................................................14
2.4.2 The Myers-Briggs Type Indicator ................................................................................14
2.4.3 The Index of Learning Styles (ILS) ..............................................................................15
2.5 Learning Styles and Problem Solving Tasks ........................................................................15
3 Objectives.................................................................................................................................16
3.1 Research Questions ............................................................................................................16
3.1.1 Are text-to-diagram translations a major source of difficulty during problem solving?.....16
3.1.2 Do expressive blocks emerge in problem solving processes? .........................................18
3.1.3 What motivates students to perform text to diagram translations? ..................................18
3.1.4 Are internal representations bound across semiotic systems? .........................................18
3.1.5 Can problem solving processes and behaviour be connected to general behaviours
associated with different student learning style profiles? ..............................................................19
4 Methods....................................................................................................................................20
4.1 Research Activity ..............................................................................................................20
4.1.1 Methodology ..............................................................................................................20
4.1.2 Rationale....................................................................................................................21
4.2 Data Analysis Methodology................................................................................................23
4.2.1 Data Collected............................................................................................................23
4.2.2 Coding of Transcripts .................................................................................................24
5

4.2.3 Coding of Written Material .........................................................................................25
4.2.4 Quantitative Analysis..................................................................................................25
5 Analysis and Interpretation of Results.........................................................................................26
5.1 Are text-to-diagram translations a major source of difficulty during problem solving? ............26
5.1.1 Did participants perform diagramming processes quickly and early? ..............................26
5.1.2 Did participants produce accurate diagrams? ................................................................28
5.1.3 Did participants verbalize accurate representations of the problems? ..............................30
5.1.4 Were participants unsure how to represent elements of the text in their diagrams? ..........31
5.1.5 What visual vocabulary patterns emerged in participants' diagrams? Did participants
exhibit diagramming conventions? .............................................................................................34
5.1.6 Did participants have difficulty representing elements of the text in their diagrams due to
conceptual prior knowledge issues? ............................................................................................35
5.2 What motivated participants to perform text-to-diagram translations?....................................38
5.2.1 Interpretation of results ...............................................................................................38
5.3 Are internal representations bound across semiotic systems? ................................................40
5.3.1 Interpretation of results: Problem 1 ..............................................................................40
5.3.2 Interpretation of results: Problem 2 ..............................................................................40
5.4 Did expressive blocks emerge? ...........................................................................................41
5.4.1 Problem 1 ..................................................................................................................41
5.4.2 Interpretation of results ...............................................................................................41
5.4.3 Problem 2 ..................................................................................................................42
5.4.4 Interpretation of results ...............................................................................................42
5.5 Summary of Results ...........................................................................................................43
5.6 Connecting Interpretations to Learning Style Preferences .....................................................44
5.6.1 Balanced Visual-Verbal Preferences ............................................................................44
5.6.2 Visual Preferences ......................................................................................................45
5.6.3 Verbal Preferences......................................................................................................45
5.6.4 Strong Visual-Verbal Preferences ................................................................................46
5.6.5 Sequential-Global Preferences .....................................................................................46
6 Conclusions ..............................................................................................................................47
6.1 Research Question Outcomes .............................................................................................47
6.1.1 What motivated participants to perform text to diagram translations? .............................47
6.1.2 Did participants perform diagramming processes early and quickly? ..............................47
6.1.3 Did participants produce accurate diagrams? ................................................................47
6

6.1.4 Did participants verbalize accurate representations of the problems? ..............................47
6.1.5 Were participants unsure of how to represent elements of the text in their diagrams? ......48
6.1.6 What visual vocabulary patterns emerged in participants' diagrams? Did participants
exhibit diagramming conventions? .............................................................................................48
6.1.7 Did participants have difficulty representing elements of the text in their diagrams due to
conceptual prior knowledge issues? ............................................................................................48
6.1.8 Are internal representations bound across semiotic systems? .........................................48
6.1.9 Did expressive blocks emerge? ....................................................................................48
6.1.10 Are text-to-diagram translations a major source of difficulty during problem solving?.....49
6.2 Revisiting the Thesis Objectives .........................................................................................49
6.3 Implications.......................................................................................................................50
6.4 Limitations and Future Work ..............................................................................................51
7 Works Cited..............................................................................................................................54
8 Bibliography .............................................................................................................................55
Appendices
Appendix A: Problem Solving Activity Problems ............................................................................ 56
Appendix B: Recruitment Email..................................................................................................... 57
Appendix C: Participant Information and Learning Style Profiles .................................................... 58
Appendix D: Terminology ............................................................................................................ 59
Appendix E: Iterative Coding Terms .............................................................................................. 60
Appendix F: TDS and IPS Coding of Transcripts ............................................................................ 61
Appendix G: Problem Solutions ..................................................................................................... 62
Appendix H: Rubrics for Analysis of Diagram Accuracy ................................................................. 64
Appendix I: Participant Worksheets ............................................................................................... 65



7

List of tables
Table 1: Problem 1 timing data ..........................................................................................................26
Table 2: Problem 2 timing data ..........................................................................................................27
Table 3: Analysis of written material (P1) ..........................................................................................28
Table 4: Analysis of written material (P2) ..........................................................................................29
Table 5: Accuracy of internal representations (P1) ..............................................................................30
Table 6: Accuracy of internal representations (P2) ..............................................................................31
Table 7: Evidence of prior knowledge (P1).........................................................................................36
Table 8: Evidence of prior knowledge (P2).........................................................................................36
Table 9: Expressive blocks (P1) .........................................................................................................41
Table 10: Expressive blocks (P2) .......................................................................................................42
Table 11: Summary of results ............................................................................................................43
Table 12: Participant Information......................................................................................................58
Table 13: Learning Style Profiles.......................................................................................................58
Table 14: Problem 1 rubric ................................................................................................................64
Table 15: Problem 2 rubric ................................................................................................................64



8

1 Introduction
"If engineering problem solving is moving from recognition to framing and then synthesis and relies on
the TDS [text-diagram-symbol] representations to construct meaning, how is that taught, or is it taught?"
(McCracken & Newstetter, 2001).
Early undergraduate engineering courses often focus on mathematical fundamentals but students are not
explicitly guided in the important steps of understanding a problem statement and translating elements
into useful diagrams for analysis. These problem solving deficiencies impede performance in coursework
and real-world activities.
McCracken and Newstetter (2001) developed the Text-Diagram-Symbol (TDS) algorithm defining three
phases and associated semiotic systems that are involved in ideal problem solving processes. The
Integrated Problem Solving (IPS) model describes more specific processes performed during each phase,
as well as episodes in which prior knowledge may be recalled or mapped. This thesis investigates student
problem solving and focuses on:
1. understanding implicit problem solving processes
2. investigating deficiencies in text-to-diagram translations
3. exploring any capabilities and deficiencies in terms of student learning profiles
The study is of a preliminary, exploratory nature due to the project time frame and the researcher's limited
research experience. It begins with a review of engineering education literature in Chapter 2. Project
objectives and research questions that stemmed from the literature review are presented in Chapter 3.
Chapter 4 describes the research activity and methods used to analyse data. Analysis and interpretation of
problem solving processes is presented in Chapter 5. This chapter includes the connection of findings to
student learning styles in Section 5.5. Chapter 6 includes a discussion of implications, limitations, and
future work.
9

2 Literature Review
The literature review began with the review of key articles on the subjects of engineering problem
solving. Wendy Newstetter and Thomas Litzinger were identified as two experts in the field, so the next
step was to obtain more of their related work. Searches were also performed to find works by co-authors
of Newstetter and Litzinger. After gaining a foundational understanding of the research topics, literature
from engineering education publications including the International Journal of Engineering Education
(IJEE) and American Society for Education (ASEE) was identified and reviewed. As the study
progressed, related topics including visual representation and engineering education practices were also
investigated. The Felder-Silverman Index of Learning Styles (ILS) was introduced to the researcher at the
beginning of the project. Articles validating and incorporating the ILS were reviewed and other learning
style categorization tools were studied and tested. The complete literature review follows.
2.1 Engineering Education and Problem Solving
Engineering education research is a broad field with a general goal of improving the quality of engineers
by advancing educational methods. Within the area of learning mechanisms, one competency that
researchers are investigating is engineering problem solving.
2.1.1 Problem Solving
Basic problem solving is an important engineering skill that is developed in first year university courses.
A traditional textbook problem will describe a system and any external factors, then ask students to solve
for a particular outcome or scenario. Typical problems cover statics, dynamics, material flow, current
flow, and abstractions of other real-world situations.
Engineering students demonstrate mixed capabilities in problem solving. Student approaches to solving
engineering problems are frequently based on prior knowledge or memory of examples presented in
lecture. They often begin using mathematical formulae to obtain a solution without ensuring a full
10

understanding of the scenario at hand. In comparison, experts tend to translate the given verbal
information into a visual model of the complete system before moving on to mathematical calculations
(McCracken & Newstetter, 2001). This aids in the correct formulation of calculations that take into
account all relevant information in interpretation of the problem.
Suboptimal problem solving techniques do not always prevent students from obtaining a correct solution.
For simple and familiar problems, it is possible to obtain a correct or sensible solution without employing
a visual translation. Students with strong mental imagery or visualization abilities may be able to identify
suitable calculations and formulate a solution without producing a diagram on paper. While it is
understood that there are numerous methods that can lead to a solution, the creation of a visual
representation motivates the deeper understanding of complex problems that extend beyond student
knowledge (McCracken & Newstetter, 2001). The optimal problem solving approach depends on the type
of problem and desired solution, student abilities and preferences, time constraints, and various other
factors. Researchers have defined optimal problem solving techniques in problem solving models.
2.2 Problem Solving Models
Problem solving models are developed to define effective approaches to problem solving and can also be
used as a point of comparison when attempting to identify why students do not succeed in solving a
problem. General problem solving models that focus on the higher level thought processes that lead to a
solution can be applied to simple problem solving tasks where all necessary information is provided, and
can also be carried over to more complex problems where students must make assumptions or define the
problem themselves. Two relevant problem solving models are the Text-Diagram-Symbol algorithm and
the Integrated Problem Solving Model.
2.2.1 The text to diagram to symbol (TDS) algorithm
McCracken and Newstetter (2001) break down the engineering problem solving process into three
iterative phases:
11

1. Problem recognition: understanding facts and data (textual)
2. Problem framing: simplification with assumptions and hypotheses (diagrammatic)
3. Problem synthesis: validation via mathematical modeling (symbolic)
They have formalized a common algorithm employed by novices and especially experts to progress
through these stages. The text to diagram to symbol (TDS) algorithm is a community-sanctioned practice
involving transformations of information between three different representational systems. The problem
statement (text) is translated into a sketch (diagram) which is then represented by mathematical formulae
(symbols) to generate a numerical solution. This process requires comprehension and generative skills
spanning across the three semiotic forms or languages verbal, visual, and symbolic. Novices face
challenges in eliciting meaning from the individual languages and their interactions since they are not
comfortable with the associated grammatical and semantic conventions. Teaching interventions are
needed to guide students through the problem solving translations and improve their understanding of
text, diagrams, and symbols so they can obtain the information required to solve engineering problems. In
order to develop teaching interventions, researchers need to understand the cognitive processes involved
in each problem solving phase. The need for greater understanding of student cognitive processes was a
motivational factor in the development of this research thesis.
2.2.2 The Integrated Problem Solving (IPS) Model
The Integrated Problem Solving model (Litzinger, Van Meter, Wright & Kulikowich, 2006) drew from
the TDS model to map more detailed cognitive processes involved in problem solving. This model
defines ideal analysis skills of prior knowledge, problem solving processes, and translations between text,
diagrams, and symbols. These skills are expected to be used in combination to effectively solve
engineering problems. The purpose of the IPS model is to act as a tool to find areas of difficulty in student
problem solving; student processes can be analyzed in relation to the IPS model to identify skill
deficiencies.
12

IPS developers tested the model on a small group of four students to confirm its validity as a model of
problem solving and to identify student skill deficiencies. The students were tasked with constructing free
body diagrams based on a written description and an illustration. Analysis of their actions and spoken
thoughts showed that a lack of prior knowledge in the subject domain was the primary reason they had
trouble producing a correct free body diagram. This may be due to the nature of the task; if an image of
the scenario had not been provided along with the problem statement, it is likely that the students may
have found it more challenging to translate the verbal information into diagram form. This thesis will
expand on the IPS work to investigate student processes when problems do not prescribe the use of
diagrams.
2.3 Learning Styles and Teaching Practices
IPS developers noted that the model could be used to compare deficiencies between different groups of
students. Engineering education studies frequently categorize students by discipline, gender, and
academic level. Another mode of categorization is by learning style preferences.
2.3.1 Student Learning Styles
Educators recognize that students learn by individual methods based on their cognitive processes, inherent
strengths and weaknesses, and learning experience. These characteristics influence the way students
absorb, retain, and apply knowledge. The traditional methods of engineering course delivery, such as
presenting material in a lecture environment, are not effective for all students. Likewise, testing methods
do not always enable students to demonstrate the extent of their knowledge and ability. In the interest of
providing engineering students with the best possible educational experience and preparing them for
activities beyond the classroom, researchers are continuously investigating existing processes and
developing new methods for the delivery of undergraduate engineering education.
Engineering teaching methods also vary with instructor preferences. Some instructors are motivated to
vary their activities to engage students while others are unwilling to implement new methods and expect
13

students to adapt to their existing preferences. Instructors sometimes criticize students for not fully
applying themselves to their studies, while students feel that lectures and other course activities are not an
effective use of their time. Researchers need to intervene to help identify a satisfactory learning
environment for students and instructors.
If engineering education specialists can develop effective, well-defined teaching processes that suit
student preferences then instructors may be more willing to change the way they deliver courses. This
should increase participation rates as students will then regard the education process in a more positive
light. Instructors will benefit as student engagement will make their jobs more enjoyable, and students
will benefit as the quality and efficiency of their learning will improve. The first step towards this
improvement requires an understanding of student learning styles.
2.4 Learning Style Categorization Tools
Student learning practitioners have created and adopted various tests to categorize student learning
preferences. These include the Kolb Learning Style Inventory, Myers-Briggs Type Indicator, and
Inventory of Learning Styles (ILS) tool. These tools identify where students fall on particular continuums
in terms of their preferred methods of absorbing and understanding information, retaining information,
generating ideas, and solving problems. The categorization tools are based on psychological theory and
scientific studies.
Despite the extensive research that has been performed, there is still some debate regarding the validity of
aligning teaching styles with learning style preferences (Newcombe & Stieff, 2012). There are other
internal and external factors that influence student learning and it is challenging to prove that changes to
teaching styles significantly affect the quality of learning. Nonetheless, it is still important to consider the
audience when determining the content and style of information delivery. Similarly, it is well-accepted
that traditional assignments and testing methods are not always the best tools for the application and
14

evaluation of engineering material. By understanding how students think, engineering education
researchers can begin to identify better ways of helping students to grasp challenging technical concepts.
2.4.1 The Kolb Learning Style Inventory (LSI)
The Kolb LSI (Kolb, 1999) asks users to complete twelve sentences by ranking how well four different
phrases describe their way of learning. The resulting scores denote an individual's reliance on the four
learning modes of Concrete Experience, Reflective Observation, Abstract Conceptualism, and Active
Experimentation. Concrete Experience and Abstract Conceptualism relate to how an individual takes in
experience; Active Experimentation and Reflective Observation are different ways of handling
experience. The ideal learner would employ a balance of all four modes.
A comparison between the two types of scores places someone in one quadrant of the Kolb learning style
type grid. The quadrants distinguish between Diverging, Assimilating, Converging, and Accommodating
learners. These learning styles are useful when considering the higher level methods used to deliver
course content to students and to test their knowledge. The Kolb LSI is not so useful for the purpose of
investigating student techniques in the problem solving process.
2.4.2 The Myers-Briggs Type Indicator
The Myers-Briggs Type Indicator (MBTI) produces scores based on a multiple-choice survey. It reports
preferences on four dichotomies and categorizes people into one of sixteen types. The extraversion-
introversion (E-I) dichotomy defines where individuals prefer to focus their attention and whether they
gain energy from external or internal sources. The sensing-intuition (S-N) dichotomy describes how
individuals take in information. The thinking-feeling (T-F) dichotomy differentiates typical methods of
decision making. The judging-perceiving (J-P) dichotomy indicates how an individual is oriented to the
world around them. None of the MBTI dichotomies would suggest specific differences in areas of
problem solving performance that this project is focused on. Extraverted participants may be more
forthcoming with their thought processes in a think-aloud activity. Judging participants may be more
15

methodical in their problem solving approaches. This information was intended for use in the initial
selection of participants.
2.4.3 The Index of Learning Styles (ILS)
The ILS (Felder & Soloman) is a categorization tool created specifically for students in technical
programs. It was reworked to its current functionality by Felder and Soloman, based on the initial work of
Felder and Silverman. Expansive studies have validated the ILS as an effective model and identified
common preferences for groups of engineering students as well as students in other programs (Litzinger,
Lee, Wise & Felder, 2005). A questionnaire consisting of 44 prompts, each with two possible responses,
determines a student learning profile over four dimensions: active-reflective, sensing-intuitive, visual-
verbal, and sequential-global. Results denote a very strong, moderate, or balanced preference for one
category in each dimension. Students exhibiting a very strong preference may struggle when a learning
environment does not support that preference. Students with a mild preference will generally have a better
learning experience if the preference is supported. This thesis project compares student problem solving
across the visual-verbal and sequential-global dimensions.
2.5 Learning Styles and Problem Solving Tasks
As discussed above, problem solving requires students to integrate a range of skills to work from problem
statement to solution. Depending on a student's learning style preferences, unique capabilities may benefit
problem solving while associated deficiencies may hinder the problem solving process. In terms of the
ILS, one could expect to observe different approaches and sources of difficulty associated with sequential
versus global, and visual versus verbal students.
16

3 Objectives
The primary objective of this thesis project is to investigate the processes implicit in engineering student
problem solving, with a focus on translations from text to diagram. This objective is motivated by the
engineering education community's present understanding of problem solving methods and deficiencies,
and integrates the categorization of students by learning style preferences. Studies in these areas could
contribute to the design and delivery of undergraduate engineering courses by clarifying which problem
solving skills students need to be taught more explicitly.
3.1 Research Questions
Research questions centred about the three facets of the primary thesis objective were posed to guide the
design of the study methodology and broken down into more specific questions to focus data analysis.
3.1.1 Are text-to-diagram translations a major source of difficulty during problem
solving?
This overarching question has been investigated in previous engineering student problem solving studies.
If translations from text to diagram are a source of difficulty, students may not be successful in the
analytical problem synthesis phases of problem solving. To analyse the data in this study for evidence of
difficulty in text-to-diagram translations, the question was broken down into several sub-questions
addressing written problem solving processes as well as verbalized cognitive processes.
3.1.1.1 Do students perform diagramming processes early and quickly?
If translations from text to diagram are not a major source of difficulty during problem solving, students
should be able to perform these processes quickly before proceeding to mathematical processes.
3.1.1.2 Do students produce accurate diagrams?
For diagramming processes to be useful to the overall problem solving process, students need to produce
diagrams that are accurate representations of the problem statement. If translations from text to diagram
17

are a source of difficulty in problem solving, students may not construct accurate diagrams. In order to
analyse students' diagrammatic representation capabilities, this question can be further broken down:
Do students represent and connect elements of the problem in diagram form?
Do students omit elements of the problem in their diagrams?
Do students include irrelevant information in their diagrams?
3.1.1.3 Do students verbalize accurate representations of the problems?
Students might hold an accurate representation internally, but misrepresent the problem as they perform
translations from text to diagram if this process is a source of difficulty during problem solving.
3.1.1.4 Are students unsure of how to represent elements of the text in their diagrams?
The engineering education community recognizes that undergraduate students are not always taught
diagramming techniques or conventions (Litzinger, Van Meter, Wright & Kulkowich, 2006). This
research question seeks to investigate whether students experience uncertainty as they perform
translations from text to diagram form.
3.1.1.5 What visual vocabulary patterns emerge in student diagrams? Do students
follow diagramming conventions?
Students who have been taught or developed consistent visual vocabularies may have less difficulty
performing translations from text to diagram during problem solving. This question investigates whether
students show evidence of standard representations for common problem elements.
3.1.1.6 Do students have difficulty representing elements of textual problems in their
diagrams due to conceptual prior knowledge issues?
In the IPS model, phases of the TDS algorithm are composed of problem solving processes and the prior
knowledge necessary to perform processes (Litzinger, Van Meter, Wright & Kulkowich, 2006). Research
suggests that inadequate prior knowledge is a greater cause of student problem solving deficiencies than
poor problem solving processes. Prior knowledge includes pattern recognition, determination of a
18

problem's deeper structure, the retrieval of analogical problems, and mapping of mathematical
expressions and knowledge.
3.1.2 Do expressive blocks emerge in problem solving processes?
In their discussion of creativity in problem solving, Wankat and Oreovicz define an expressive block as
an inappropriate language path (Wankat & Oreovicz, 1993). Solving a problem without the appropriate
diagram or based on an inaccurate diagram is a form of expressive block, as the problem is not suitably
represented for translation to symbolic synthesis processes. If text-to-diagram translations are a source of
difficulty, the diagrams produced may act as expressive blocks.
3.1.3 What motivates students to perform text to diagram translations?
Participants were not explicitly instructed to create diagrams in the problem solving activity. The TDS
algorithm describes an ideal problem solving method, and the IPS model was only confirmed to represent
student problem solving behaviour based on a small study that instructed participants to create diagrams
(Litzinger, Van Meter, Wright & Kulikowich, 2006). This research question investigates whether students
use diagrams in their typical problem solving processes, or are motivated to do so by the nature of the
problem or challenges during non-diagrammatic problem solving processes.
3.1.4 Are internal representations bound across semiotic systems?
One cognitive factor that could have an influence on problem solving is the nature of internal
representations. This research question investigates whether information represented using one semiotic
system is tied to its representations in other semiotic systems, or is not inherently bound. If internal
representations are bound across semiotic systems, then a diagrammatic process might provoke symbolic
representations of the same information.
19

3.1.5 Can problem solving processes and behaviour be connected to general
behaviours associated with different student learning style profiles?
For the comparison of problem solving behaviours, students can be categorized by different demographic
factors. In this study, students are categorized by ILS learning style profiles to investigate whether
capabilities and deficiencies have any relation to learning preferences.

20

4 Methods
An exploratory study was conducted to investigate engineering student problem solving processes,
focusing on text-to-diagram translations. A small group of five student participants performed a problem
solving activity and their performance was analyzed for emerging patterns based on written material and
audio recordings collected during the activity. The study's research questions structured the analysis
methodology.
4.1 Research Activity
The exploratory study involved a think-aloud problem solving activity in which participants verbalized
cognitive processes as they solved two physics problems using pencil and paper.
4.1.1 Methodology
After participant recruitment and selection, participants were asked to complete the ILS survey and report
results to the researcher. The problem solving activities were performed in small meeting rooms on the
University of Toronto campus. The researcher met with participants individually. Five fourth-year
engineering students completed the study. These students were peers of the researcher, but this
relationship should not have affected their performances.
Two problems from Serway and Jewett's physics textbook (2008) were used for the problem solving
activity. The first problem (P1) was a speed and acceleration scenario while the second (P2) involved
basic trigonometry. The original diagram that accompanied P2 was not provided to participants. Both
problems required minimal expert knowledge. The problems are provided in Appendix A: Problem
Solving Activity Problems. Participants were provided with an 8.5 x 11 in worksheet containing the two
problem statements, and a pencil. They were not provided with calculators, formulae, or other reference
material. They were instructed to solve the problems as they would in an exam, and to verbalize the
internal thoughts they had as they solved the problems. The researcher performed an example activity to
ensure participants understood the think-aloud technique.
21

A Dell laptop microphone was used to record verbal protocols with Audacity 1.3 software.
4.1.2 Rationale
The experiment closely followed the methods that were used to analyze student performance upon the
development of the IPS model (Litzinger, Van Meter, Wright & Kulikowich, 2006). Decisions were made
to optimize the quality of information gained by the research, given the time and resources available.
4.1.2.1 Ethics Review
The study was proposed to and accepted by an ethics review committee before it was conducted.
4.1.2.2 Recruitment
Students previously enrolled in APS443: Leadership and Leading in Groups and Organizations were
asked to volunteer to participate in the study because they completed a MBTI assessment in October
2013. A recruitment email was distributed (see Appendix B: Recruitment Email). Participants were peers
of the student author.
4.1.2.3 Choice of Participants
Participants were University of Toronto Faculty of Applied Science and Engineering students in their
fourth years of undergraduate study. Students enrolled in a leadership course were asked to volunteer to
participate in the study because they completed a MBTI assessment in October 2013. MBTI profiles were
to be used to select participants with different learning styles if there was a large pool of volunteers. Six
volunteers responded and five were able to complete the study, so MBTI profiles were not used for
selection.
The review of literature provides evidence that students struggle with problem solving at numerous points
in the process, including the text to diagram translation necessary to represent the problem scenario
visually. By selecting upper year students for the study, the researcher hoped to identify deficiencies that
have not been addressed after three or four years of problem solving experience at the university level.
22

Upper year students who demonstrate competent translation and representation skills were expected to
provide insight that could contribute to the development of problem solving teaching interventions.
4.1.2.4 Choice of Learning Style Categorization Tool
The ILS was selected for its visual-verbal and sequential-global dimensions. The questionnaire is freely
available online and validated for students in technical programs. Participants completed the ILS in their
own time and reported their results to the researcher.
4.1.2.5 Choice of Problems to Solve
Problem 1 was chosen on the assumption that some students may attempt to solve it without the use of
diagrams. The researcher intended to investigate emerging patterns in the accuracy of solutions reached
with or without diagrams.
In the textbook, there is an image of the fountain presented with Problem 2. The image was not provided
to participants so that they would be more likely to construct their own diagrams. Even if participants
were unable to recall specific trigonometric equations, their general approach should have provided useful
information about the text-to-diagram translation.
Both problems require minimal expert knowledge. They demand the application of basic mathematical
concepts that students apply to more complex problems in their upper year engineering courses. It was
anticipated that due to these choices, the problem solving activity would provide information primarily
about student processes and text-diagram-symbol translations rather than specific technical knowledge.
4.1.2.6 Think-aloud Protocol
Think-aloud protocols are enlisted in psychology studies and user testing activities to obtain inf ormation
that cannot be observed from participant actions or measurable variables. Participants are encouraged to
express their immediate thoughts to overcome this challenge. A number of engineering education studies
related to problem solving have used think-aloud protocols to understand student challenges in the same
way this research attempts to do (Litzinger, Van Meter, Wright & Kulikowich, 2006).
23

4.2 Data Analysis Methodology
Due to the small-scale, preliminary nature of the study, it was not possible to perform thorough
quantitative analysis or implement rigorous qualitative data analysis methods. Detailed research
questions were developed relating to the study objectives of understanding cognitive aspects of student
problem solving processes and analysing text-to-diagram translations. These questions were motivated by
the literature reviewed before development of the research methodology, particularly the Text to Diagram
to Symbol (TDS) algorithm and Integrated Problem Solving (IPS) model. Observation of participants'
actions during the data gathering activities confirmed that students incorporated diagrams in their problem
solving procedures and included components of the TDS and ILS models. Since the data appeared to
align with the existing frameworks, the research questions were deemed applicable for exploratory
analysis.
4.2.1 Data Collected
For each participant, the following data was collected in the course of the learning styles and problem
solving investigation:
Preliminary and demographic information (see Appendix C: Participant Information and
Learning Style Profiles):
o Age, gender, engineering discipline
o Myers-Briggs Type Indicator type as determined in APS443 course activity
Index of Learning Styles profile: ILS preference scale scores (see Appendix C: Participant
Information and Learning Style Profiles):
o Active-reflective score
o Sensing-intuitive score
o Visual-verbal score
o Sequential-global score
Problem solving activity material
24

o Audio recordings of verbal protocols
o Participant's written solutions and work (see Appendix I: Participant Worksheets)
o Observational notes
4.2.2 Coding of Transcripts
Audio recordings of participant commentaries ranged in length from 6 minutes to 25 minutes. Audio
recordings were transcribed into digital text on the same day a participant completed the problem solving
activity. After all transcripts were available, they were broken down into "segment utterances" of single
complete thoughts. Pauses, transition words, and shifts in thoughts were used to identify separations
(Litzinger, Van Meter, Wright, & Kulikowich, 2006). Time-stamps and observational notes were
incorporated at this stage, including details of the diagrams sketched and symbolic information written in
association with each segment utterance.
The single thoughts and actions were first coded by a free, iterative process (Carberry, McKenna &
Dalrymple, 2012). The researcher began identifying applicable coding terms during the problem solving
activities and audio transcription. Transcripts were coded and additional terms were added in the process.
This coding was then revised to combine terms that were too narrow and replace terms that were too
broad. The selection of iterative coding terms is listed in Appendix E: Iterative Coding Terms.
The majority of research questions related to the three semiotic systems (textual, diagrammatic, symbolic)
or the three phases (problem representation, problem framing, problem synthesis) of the TDS and IPS
models. To investigate the research questions and explore characteristics of the data, a thematic analysis
of transcript materials was implemented. Transcripts were inspected and segment utterances were
grouped based on indicators of TDS and IPS concepts. The following four separate coding systems were
used for grouping:
1. Primary semiotic system in use; textual, diagrammatic, or symbolic
2. Episodes of translations between semiotic systems
25

3. Phases of problem solving; problem representation, problem framing, or problem synthesis
4. Episodes of the IPS model problem solving components
Details of the coding terms are listed in Appendix F: TDS and IPS Coding of Transcripts.
4.2.3 Coding of Written Material
Written material was treated as an independent artefact for coding, so corresponding transcript content
was not considered. A rubric was created to analyze the accuracy of diagrams by identifying elements
from the problem statements that could be represented in diagram form. This rubric was informed by
detailed solutions to the problems as provided by the source textbook (Serway & Jewett, 2008). These
solutions are presented in Appendix G: Problem Solutions. In addition to the correct information that
could be represented or omitted, the researcher performed an informal risk analysis to identify errors and
irrelevant information students might represent. Numerical values such as velocity and height were
considered represented if they were written in close proximity to diagrams as participants constructed the
diagrams. The complete rubric is provided in Appendix H: Rubrics for Analysis of Diagram Accuracy.
The correct symbolic solutions were also obtained from the textbook. This analysis method was selected
to increase the credibility of analysis by incorporating the impartial perspective of an external source.
4.2.4 Quantitative Analysis
Data was analysed quantitatively when possible. It was not feasible to perform frequency counts of
different types of problem solving processes as participants did not translate consistent quantities of
problem information in one action or in association with one verbal protocol statement. The accuracy of
the information comprehended, translated, or represented also varied. To enable preliminary quantitative
analysis, the timing of participant processes was analysed, and written material was analysed using a
rubric as described above.

26

5 Analysis and Interpretation of Results
This section reports and interprets patterns that emerged in the analysis of student problem solving
processes.
5.1 Are text-to-diagram translations a major source of difficulty during
problem solving?
This research question was investigated through the following questions and analysis.
5.1.1 Did participants perform diagramming processes quickly and early?
To determine whether participants performed diagramming processes early and quickly in their problem
solving processes, transcripts were reviewed to identify the beginning and completion of diagramming,
discounting minor additions some participants made later in the process. The timing of these events was
compared to the overall timeframes of participants' problem solving activities. Transcript material that
preceded the main diagramming processes was inspected to identify trends in participants' actions that
occurred before diagramming.
5.1.1.1 Problem 1
The analysis of timing for Problem 1 is presented in Table 1: Problem 1 timing data.
Table 1: Problem 1 timing data
P1 S01 S02 S03 S04 S05 Average
Pre-diagramming time
(s) 15.00 50.00 80.00 70.00 165.00 76.00
Diagramming time (s) 20.00 110.00 40.00 60.00 330.00 112.00
Total time (s) 135.00 330.00 480.00 720.00 1365.00 606.00
% Pre-diagramming 11% 15% 17% 10% 12% 13%
% Diagramming 15% 33% 8% 8% 24% 18%

5.1.1.2 Problem 2
The analysis of timing for Problem 2 is presented in Table 2: Problem 2 timing data.
27

Table 2: Problem 2 timing data
P2 S01 S02 S03 S04 S05 Average
Pre-diagramming time
(s) 30.00 5.00 15.00 30.00 10.00 18.00
Diagramming time (s) 15.00 145.00 75.00 65.00 290.00 118.00
Total time (s) 145.00 435.00 720.00 225.00 345.00 374.00
% Pre-diagramming 21% 1% 2% 13% 3% 5%
% Diagramming 10% 33% 10% 29% 84% 32%

5.1.1.3 Interpretation of results
For Problem 1, the time to begin diagramming ranged from 15 seconds to almost 3 minutes. On average,
participants spent 18% of their time diagramming. For Problem 2, participants took no longer than 30
seconds to begin diagramming after they began to read the problem statement. On average, they spent
32% of their time diagramming. Participants may have spent a larger portion of time diagramming in
Problem 2 because problem synthesis involved less complicated mathematical analysis than Problem 1.
S01, S02, and S03 were consistent in the proportion of time focused on diagramming for each problem,
while S04 and S05 spent smaller proportions of their total times focused on diagramming for Problem 1
than Problem 2. These participants both struggled to understand the scenario described in Problem 2.
S01 completed diagramming at least twice as fast as the other subjects, and also provided his final
solutions in less time. S05 focused on Problem 1 diagramming for triple the time of the next slowest
participant. Her diagramming was performed in two stages of similar lengths, and accounted for a quarter
of the total time spent on the problem. S05 also spent significantly more time focused on diagramming for
Problem 2, although her overall time was faster than two other participants'.
There were no time limits, and some participants persevered through problems with prompting, so these
values are not suitable for rigorous analysis. The evidence does suggest that students tend to perform
diagramming processes early, but may not complete diagramming processes quickly. Based on transcript
content, participants showed a tendency to begin diagramming before performing thorough symbolic
analysis, although S05 tried to remember Problem 1 equations before constructing a complete diagram.
28

Some students iterated between diagramming and mathematical analysis while others moved more
sequentially from diagramming to mathematical analysis.
5.1.2 Did participants produce accurate diagrams?
The rubrics provided in Appendix H: Rubrics for Analysis of Diagram Accuracy were used to determine
the accuracy of participants' diagrams. The rubrics evaluated final, completed diagrams, so included
revisions that participants made later in their problem solving activities.
5.1.2.1 Problem 1
Analysis of the accuracy of written material for Problem 1 is presented in Table 3: Analysis of written
material (P1).
Table 3: Analysis of written material (P1)
Problem 1 S01 S02 S03 S04 S05
Represent or Omit
v(car) = 45m/s; constant Represented (Textual) (Textual) (Textual) Represented
trooper starts from v = 0 No No No No Represented
1s wait time No Represented (Textual) No Represented
a(trooper) = 3m/s
2
Represented (Textual) (Textual) (Textual) Represented
t = ? No No Represented (Textual) Represented
Connect
billboard = origin (d = 0) Represented Represented Represented Represented Represented
car gets ahead of trooper No Represented Represented No Represented
d(car) = d(trooper) No Represented Represented Represented Represented
I rrelevant 2D diagram,
Equations
1.5t^2 - 45t - 45 = 0 t 45 = 3t^2 3t^2 - 45t -
45 = 0
3t^2 - 45t -
45
3t^2 = 45t t^2 - 15t - 15
= 0
# Diagrams 3 1 2 1 1 2
Events A, B,
C
A-B A-B, B-C A-B-C A-C A-C, A-B-C
# Elements Represented 8 3 6 7 5 8

5.1.2.2 Interpretation of results
Participants did not generally produce accurate diagrammatic representations of Problem 1. Of the eight
identified elements from the problem statement that could have been represented, all participants except
29

S05 omitted one element or more. Participants also represented some elements in text form in close
proximity to the diagram rather than using the diagrammatic semiotic system. The number of diagrams
did not determine the accuracy of a participant's overall representation of the problem; S03 represented 7
elements in one diagram, while S02 represented only 6 elements in two diagrams.
The textbook indicated that the three events described in the problem statement should be represented in
three separate diagrams. Participants produced one or two diagrams, showing evidence that students have
a tendency to attempt to represent a combination of two or three events in one diagram.
5.1.2.3 Problem 2
Analysis of the accuracy of written material for Problem 2 is presented in Table 4: Analysis of written
material (P2).
Table 4: Analysis of written material (P2)
Problem 2 S01 S02 S03 S04 S05
Represent or Omit
circular pool Represented Represented Represented Represented Represented
fountain at centre Represented Represented Represented Represented Represented
C = 15m Represented Represented Represented (Textual) (Textual)
= 55 Represented Represented Represented Represented Represented
h = ? No Represented Represented Represented
("x")
Represented
Connect
C r (top view) Represented Represented Represented Represented Represented
, r h (side view) No Error Represented Represented Represented
I rrelevant Other s,
protractors
Person,
protractor

h = (15/2)tan55 cos55 2.5 = h =
15/cos55
h =
(r)tan55; C
= 2r
x =
[(tan55)2]/15
h = tan55 2;
r = 2
# Diagrams 2 1 5 2 1 1
Views top,
side
top top, side top, side top, side top, side
# Elements
Represented
7 5 7 7 7 7

30

5.1.2.4 Interpretation of results
Participants showed a strong tendency to represent almost all relevant information. S01 did not construct
a diagram of the side view of the fountain, and did not represent the height (h) dimension that the problem
asked participants to solve for. S02 included irrelevant angles in her diagram, and S03 included a stick
figure to represent the person described in the problem.
Overall, participants tended to produce more accurate diagrams for Problem 2 than Problem 1. Participant
S01 produced the least accurate diagrams for both Problems. The four other participants produced
diagrams that were mostly accurate, although they combined textual and diagrammatic semiotic systems
to represent problem elements.
5.1.3 Did participants verbalize accurate representations of the problems?
To determine whether participants held accurate internal representations of the problems they solved
during the research activity, problem representation components of transcripts, as identified by the
Problem Framing (F) group of the IPS phase coding methodology, were examined for emerging trends
and compared against the accuracy of participants' diagrams.
5.1.3.1 Problem 1
Analysis of the accuracy of participants' internal representations of Problem 1, as determined from verbal
and diagrammatic representations, is presented in Table 5: Accuracy of internal representations (P1).
Table 5: Accuracy of internal representations (P1)
P1 S01 S02 S03 S04 S05
1.5t^2 - 45t - 45 = 0 t 45 = 3t^2 3t^2 - 45t -
45 = 0
3t^2 - 45t -
45
3t^2 = 45t t^2 - 15t - 15
= 0
# Elements Verbal
8 6 7 8 6 8
# Elements Diagram
8 3 6 7 5 8

31

5.1.3.2 Problem 2
Analysis of the accuracy of participants' internal representations of Problem 2, as determined from verbal
and diagrammatic representations, is presented in Table 6: Accuracy of internal representations (P2).
Table 6: Accuracy of internal representations (P2)
P2 S01 S02 S03 S04 S05
h = (15/2)tan55 cos55 2.5 = h =
15/cos55
h =
(r)tan55; C
= 2r
x =
[(tan55)2]/15
h = tan55 2;
r = 2
# Elements Verbal 7 6 7 7 7 7
# Elements Diagram 7 5 7 7 7 7

5.1.3.3 Interpretation of results
In all cases where participants did not represent the complete problem in their diagrams, they provided
verbal evidence of additional elements held in internal representations. This evidence often emerged at
later stages in the problem solving activity as participants performed problem synthesis processes and
revisited the problem statement to obtain information or clarify their understanding of the problem.
This data aligns with the engineering education community's impression that translations from text to
diagram are a major source of difficulty in student problem solving; students are unable to represent their
understanding of the problem using the diagrammatic system. It also suggests that students move on to
problem framing and problem synthesis phases before fully understanding the problem, or use these
phases to develop their understanding. As their understanding becomes more complete, students do not
necessarily revise their diagrams to represent additional elements of the problem.
5.1.4 Were participants unsure how to represent elements of the text in their diagrams?
To investigate whether students may produce inaccurate or incomplete diagrams because they are not sure
how to represent elements of the problem, transcripts were examined for evidence of uncertainty or
confusion during diagramming processes. This evidence was then examined for patterns in these
expressions and in consideration of the accuracy of participants' diagrams.
32

5.1.4.1 Results
S01 did not express uncertainty or confusion during diagrammatic processes yet produced less accurate
diagrams than all other participants for both problems. S03 did not express uncertainty or confusion
during diagrammatic processes either and represented 7/8 Problem 1 elements and all seven Problem 2
elements. S04 did not appear unsure during diagramming and represented 5/8 Problem 1 elements and all
seven Problem 2 elements.
S02 expressed uncertainty or confusion on five occasions during diagramming processes. S02's Problem 1
diagram represented 6/8 problem elements and her Problem 2 diagram represented all seven problem
elements; however, she was assisted with prompts from the researcher after she showed indications of
being stuck.
5.1.4.2 Interpretation of results: Problem 1
S02 hesitated and showed indications of confusion while determining how to represent the transition
between event A and event B for Problem 1: " so the it's one second after what I have now this is
embarrassing. " S02 represented this element of the problem in her first of two diagrams by adding an
arrow and writing "1s" above the arrow. This element was not represented by two other participants, and
represented only in textual form by a third subject. S05 did not verbalize confusion as to how to represent
the transition, but did revise components of her second diagram to clarify its representation, as shown in
Figure 1: S05 P1 diagram 2/2. This data suggests that students were unsure how to represent a transition
in which one body changed its motion while the other continued its same pattern of movement.

Figure 1: S05 P1 diagram 2/2
33


S02 constructed a second diagram for Problem 1 after reviewing the problem statement at the end of her
problem solving process, stating "actually when I think about it that means it's not the right answer so I
would actually write that's when the distance is equal - to overtake it you need like a millisecond more."
None of the other participants considered this detail, presumably because they have adopted the
convention of simplifying problems of this sort; the textbook solution did not discuss the distinction
between overtaking and drawing level with the car.
S05 was unsure about the path of the vehicles in Problem 1, initially believing that they were travelling
along different paths on a two dimensional plane: "so when this gets here he starts trying to catch him
going this way." She retained this impression as she continued her diagramming processes which caused
her to spend significantly more time diagramming that any other participant. Eventually she chose to
assume motion was occurring in one dimension.
5.1.4.3 Interpretation of results: Problem 2
Two participants were unsure how to represent the relative locations of the person and fountain in
Problem 2. S02 did not understand the term "angle of elevation" and thought it was related to the curve of
the fountain stream. She spent time visualizing and sketching before the researcher determined that she
needed prompting. S05 was unsure whether she had represented the fountain correctly: "I don't know if
the fountain is like this - I'm confused about what he is measuring here." She was also confused by the
"angle of elevation" and did not assume that the fountain was vertical.
S02 constructed a three dimensional diagram for Problem 2, and indicated that it was more difficult to
represent information in that format, stating "but how does this angle help me get this length that's what
I don't understand [] well actually I don't like three-d stuff." Three other participants (S03, S04, S05)
constructed diagrams in which different views were overlaid but did not exhibit uncertainty as they added
34

representations of problem elements. For these participants, the only representation common to both
views was the radius, whereas S02 was representing relations between different angles.
This data suggests that participants were unsure how to represent Problem 2 diagrammatically because
they struggled to understand elements of the statement and elicit suitable assumptions to be represented in
problem framing. Students were sure of their representational methods for elements of the problem that
they understood correctly.
5.1.5 What visual vocabulary patterns emerged in participants' diagrams? Did
participants exhibit diagramming conventions?
Written material was examined and transcripts were reviewed to clarify the intended meaning of visual
components, in order to identify patterns in visual vocabulary.
5.1.5.1 Problem 1
The participants that constructed more than one diagram did not clearly represent how they were related.
Participants used lines and arrows to represent a number of different elements in their diagrams:
S01 used a single-line arrow () to represent velocity and a double-line arrow () to represent
acceleration. A vertical dashed line represented the origin.
S02 used single-line arrows () to represent three different types of information; the event of the
trooper setting out, the trooper's acceleration, and distances travelled. A vertical line represented the
billboard.
S03 used single-line arrows () to represent the car's path and the trooper's acceleration, and a
dimension line () to represent the equal distance.
S04 used a horizontal line with extension lines to represent the equal distance, a vertical line to
represent the billboard, and a vertical line to represent the endpoint.
S05 used dimension and extension lines to represent distance travelled and common positions , and
single-line arrows to represent direction, velocity, and accelerations.
35

Participants used different levels of detail to represent the vehicles.
5.1.5.2 Problem 2
All participants except S04 drew a circle to represent the fountain pool. S04 drew an oval instead,
combining the anticipated top and side views in one perspective diagram.
S04 and S05 overlaid views in one diagram. Neither S02 nor S03 indicated how their diagram of the
circle was related to their triangle diagram(s).
Triangle geometry:
Of the four subjects who constructed a triangle, S02 and S05 used the conventional symbol for a right
angle in the right angle triangle.
All four included a segment curve at the vertex of the angle of elevation.
S04 included extension lines on the fountain height edge but not the radius dimension.
S05 did not use extension lines to represent relevant dimensions in this problem although she did in
Problem 1.
5.1.5.3 Interpretation of results
Participants did not show evidence of a consistent visual vocabulary as a group or within their own
diagrams. The meanings of alphanumeric symbols associated with diagrams were also inconsistent.
Participants combined the use of variable symbols (e.g. , x) and values (15m) within single diagrams.
The simplicity of the problems may explain why technical diagramming conventions were not generally
demonstrated.
5.1.6 Did participants have difficulty representing elements of the text in their diagrams
due to conceptual prior knowledge issues?
To investigate the extent of participants' prior knowledge in the problem subject domains, evidence of
prior knowledge identified in transcripts by the IPS coding methodology was examined for general
patterns as well as trends in relation to the accuracy of participants' diagrams and solutions.
36

5.1.6.1 Problem 1
Analysis of the evidence of prior knowledge in Problem 1 is presented in Table 7: Evidence of prior
knowledge (P1).
Table 7: Evidence of prior knowledge (P1)
P1 S01 S02 S03 S04 S05
Pattern recognition (RK1) Y Y Y N Y
Determine deep structure (RK2) Y Y Y Y Y
Prior knowledge mapped (FK1) N N Y N Y
Analogical problems retrieved (FK2) Y N N N N
Mathematical expression (SK1) Y Y Y Y Y
Prior Knowledge 5 4 3 4 3 4
1.5t^2 - 45t - 45 = 0 t 45 = 3t^2 3t^2 - 45t -
45 = 0
3t^2 - 45t -
45
3t^2 = 45t t^2 - 15t - 15
= 0
# Elements Verbal 8 6 7 8 6 8
# Elements Diagram 8 3 6 7 5 8

5.1.6.2 Problem 2
Analysis of the evidence of prior knowledge in Problem 2 is presented in Table 8: Evidence of prior
knowledge (P2).
Table 8: Evidence of prior knowledge (P2)
P2 S01 S02 S03 S04 S05
Pattern recognition (RK1) Y Y Y Y Y
Determine deep structure (RK2) N Y Y Y Y
Prior knowledge mapped (FK1) Y Y N N N
Analogical problems retrieved (FK2) N N N N N
Mathematical expression (SK1) Y Y Y Y Y
Prior Knowledge 5 3 4 3 3 3
h = (15/2)tan55 cos55 2.5 = h =
15/cos55
h =
(r)tan55; C
= 2r
x =
[(tan55)2]/15
h = tan55 2;
r = 2
# Elements Verbal 7 6 7 7 7 7
# Elements Diagram 7 5 7 7 7 7

37

5.1.6.3 Interpretation
According to the transcript data, prior knowledge was primarily embedded in problem representation
(textual) and problem synthesis (symbolic) stages. The prior knowledge components of pattern
recognition and determination of deeper structure, that should occur before students move on to problem
representation and problem synthesis, emerged throughout participants' processes and were not restricted
to earlier stages. This suggests that students have a tendency to not recognize patterns or gain a complete
understanding of the problem from interpretation of the problem statement alone, but continue to
recognize patterns and deepen their understanding of the problem during diagrammatic framing and
symbolic synthesis processes.
Patterns of mathematical expressions emerged earlier in participants' problem solving processes. This
prior knowledge is modelled to be associated with the problem synthesis stage of an effective problem
solving process, however participants verbalized or wrote down mathematical expressions during any of
the three stages. This tendency suggests that certain characteristics of the problems provoked associated
mathematical expressions before students had determined how they would use the expressions or whether
the expressions were relevant. The emergence of mathematical expressions also suggests that additional
implicit components of prior knowledge, such as pattern recognition and the retrieval of analogical
problems, were present in participants' processes but were not expressed in their verbal protocols.
Participant data indicated a strong tendency for students not to retrieve analogical problems during
problem framing, as this knowledge was only recognized during S01's problem solving process for
Problem 1. There is evidence that misconceptions in prior knowledge can produce errors in problem
solving: "underlying errors are passed on to the mental model that is constructed to solve a problem"
(Litzinger, Van Meter, Wright & Kulikowich, 2006). S01 did not produce a correct solution to either
problem, and it might be worthwhile to investigate whether analogical problems can interfere with
problem solving processes if they cause students to make the wrong assumptions about the problem to be
solved.
38

Prior knowledge is embedded in the problem solving processes of the IPS model, so the identification of
additional implicit knowledge may display different patterns. The diagramming methods students used
were presumably based on their experience with similar problems in the past; however the practices were
so ingrained they did not identify the knowledge informing their diagramming processes. Furthermore,
knowledge associated with pattern recognition and determination of the deep structure of the problem
could have been mapped in diagrammatic representations, but not identified since it was embedded in the
process
5.2 What motivated participants to perform text-to-diagram translations?
To identify any factors that may have motivated participants to perform text-to-diagram translations,
transcripts were reviewed to identify the beginning of diagramming, and the preceding content was
examined for meaningful patterns. Content after the completion of diagramming was also reviewed to
identify any emerging trends in participants' tendencies to refer back to the diagrams they had
constructed.
5.2.1 Interpretation of results
S01 and S04 did not express specific motivations before initiating diagramming processes. S01 simply
stated that he would draw a diagram as his next step after reading the problem statements, indicating that
diagramming is a part of his typical problem solving procedure. S04 also proceeded to diagramming after
reading the problem statements, without indicating any specific motivations for doing so. Her phrasing in
Problem 2, "first I am going to draw a picture of the fountain," could be considered evidence that she
typically performs diagramming processes after reading the problem statement.
S02, S03, and S05 expressed a need to construct diagrams to aid the problem solving process, and
indicated that their diagramming practices are motivated by characteristics of a problem. S02 began
analysing Problem 1 before deciding to construct a diagram: "How long does it take her to overtake the
car?" "okayso the distance" "okay I'm gonna draw it." She appeared to move directly to the
39

symbolic semiotic system but then realized that she needed to use the diagrammatic system to clarify
elements of the problem statement. S03 and S05 demonstrated similar patterns. S02 acted differently after
she read the Problem 2 text, stating "okay I need to draw this," and beginning diagramming processes
before considering problem synthesis.
S03 also demonstrated a tendency to construct diagrams reactively rather than methodically. Near the
beginning of her Problem 2 activity she said, "I'm not even finished reading this but I'm going to go ahead
and draw this circle, because [] for some reason this seems more concrete to me." As she completed her
first diagram she appeared to realize it did not represent all elements of the problem: "he's holding the
protractor like this [] to get the angle between him and the top of the fountain" "oh, I'm going to
draw a different perspective." S05 also provided evidence that she is motivated to construct diagrams at
different stages depending on the problem, stating during Problem 2, "I find these ones I might usually
draw straight away, if it starts off with a visual thing [] whereas if it was just like cars travelling, I need
to figure out the numbers first."
S01 did not represent problems diagrammatically with as much accuracy as other participants. He referred
back to his Problem 1 diagram once after its completion, and did not show evidence of referring back to
his Problem 2 diagram. It is possible that he has adopted the practice of constructing diagrams, but does
not use them to inform problem synthesis processes, at least not in the case of this research activity. S04
also produced a less complete diagram for Problem 1, but represented all elements of Problem 2. She
referred back to her diagrams more frequently that S01.
S02, S03, and S05 produced more complete diagrams overall, and also produced more correct solutions to
the problem. All three of these participants revised their diagrams as their understanding of the problem
became more complete, and they showed a strong tendency to refer back to their diagrams during
problem synthesis. These trends suggest that S02, S03, and S05 are motivated to construct diagrams when
40

they believe they need them to solve a problem successfully, whereas S01 and S04 are motivated by the
general problem solving procedure they often follow.
5.3 Are internal representations bound across semiotic systems?
To investigate informational ties across semiotic systems, diagramming processes identified in the
research activity transcripts according to the TDS coding methodology were examined for evidence of
symbolic or mathematical language and meaningful patterns in its appearance.
5.3.1 Interpretation of results: Problem 1
Three participants (S02, S03, S04) demonstrated an association between the visual representations of
bodies in motion, and the analytical technique of equating mathematical expressions. As they constructed
diagrams, they were already dividing their attention between diagrammatical representations and
symbolic representations:
S02: drawing arrows, "I need to equate them"
S03: "x-amount of distance past the billboard"
S04: "that distance travelled will be the same so both are x"
S02 and S05 interrupted diagramming to retrieve mathematical expressions even though they had not
completely determined which unknowns they would be using equations to solve for:
S02: pauses before completing P2 diagram; "okay well acceleration equals velocity times
distanceno"
S05: "I didn't get there yet because I stopped to remember the formula. I was trying to draw the car"
5.3.2 Interpretation of results: Problem 2
Diagramming geometric representations in Problem 2 provoked symbolic representations of trigonometric
functions. S01 and S02 both used cosine instead of tangent in their final solutions for the height of the
fountain. S01 explained that he selected cosine based on the location of the radius length: "a little
rhyming thing I remember was cos x, sine y like the way you pronounce it is how I remember it." The
41

textual or diagrammatic representations of the problem and S01's recognition that it was a trigonometry
problem seemed to provoke a jump to symbolic calculations without the construction of a diagrammatic
side view. S02 used "h" to represent the height of the fountain. Despite the construction of multiple side
views that represented elements in the correct orientation, trigonometry equations in symbolic form
appeared to override diagrammatic and symbolic representations of "height" with representations of
"hypotenuse". S02 verbalized "therefore hypotenuse equals" while writing "h = ", and was unaware of her
error even as she presented her final solution by writing "the fountain is X m high. "
5.4 Did expressive blocks emerge?
Participants' final symbolic solutions were identified and compared to the correct solutions provided by
the textbook (Serway & Jewett, 2008). The correctness of solutions was examined in relation to the
accuracy of participants' diagrams for the emergence of expressive blocks.
5.4.1 Problem 1
The accuracy of participants' diagrams and their final solutions are presented in Table 9: Expressive
blocks (P1).
Table 9: Expressive blocks (P1)
Problem 1 S01 S02 S03 S04 S05
# Diagrams 3 1 2 1 1 2
Events A, B,
C
A-B A-B, B-C A-B-C A-C A-C, A-B-C
# Elements Represented 8 3 6 7 5 8
1.5t^2 - 45t - 45 = 0 t 45 = 3t^2 3t^2 - 45t -
45 = 0
3t^2 - 45t -
45
3t^2 = 45t t^2 - 15t - 15
= 0

5.4.2 Interpretation of results
None of the study participants reached the correct solution to Problem 1. S02, S03, and S05 used an
incorrect formula to calculate distance travelled by the trooper but were otherwise correct in their
solutions; they produced more accurate diagrams, representing 6/8, 7/8, and 8/8 problem statement
elements, respectively.
42

S01 and S04 constructed less accurate diagrams, representing 3/8 and 5/8 problem statement elements,
respectively. They produced solutions that were less correct as they did not account for the 1 second
transition between events A and B and the distance the car travelled in that time. These elements were
omitted from both of their diagrams. The three other subjects who did not make this error represented the
information in their diagrams.
This data indicates that expressive blocks did emerge as barriers to the correct solution of Problem 1. S01
and S04 reached incorrect solutions after constructing incomplete diagrams. S02, S03, and S05 reached
more correct solutions after constructing more accurate diagrams. This information could also be
interpreted to mean that S01 and S04 were unable to construct accurate diagrams because they had a
poorer understanding or internal representations of the problem, and their incorrect solutions were a result
of their incomplete understanding rather than expressive blocks resulting from inaccurate diagrams.
5.4.3 Problem 2
The accuracy of participants' diagrams and their final solutions are presented in Table 10: Expressive
blocks (P2).
Table 10: Expressive blocks (P2)
Problem 2 S01 S02 S03 S04 S05
# Diagrams 2 1 5 2 1 1
Views top,
side
top top, side top, side top, side top, side
# Elements Represented 7 5 7 7 7 7
h = (15/2)tan55 cos55 2.5 = h =
15/cos55
h =
(r)tan55;
C = 2r
x =
[(tan55)2]/15
h = tan55 2;
r = 2

5.4.4 Interpretation of results
Excluding approximations and errors during algebraic manipulation, three participants (S03, S04, and
S05) reached the correct solution to Problem 2. They had produced accurate diagrams representing all
seven elements identified.
43

The other two participants (S01 and S02) incorrectly used cosine as the trigonometric function to
calculate the height of the fountain. S01 represented 5/7 elements of the problem and constructed only a
top view of the fountain. The incorrect solution can be connected to his omission of a diagrammatic
representation that indicates the height dimension is opposite the angle of elevation. S02 represented all
seven elements but misidentified "h" as hypotenuse. She also used the circumference measurement in
place of the radius in her second diagram (despite identifying their relation) and carried this
representation through to her solution.
This data indicates that expressive blocks may have emerged as barriers to the correct solution of Problem
2. S01's incomplete diagrammatic representation and S02's incorrect diagrammatic representation
contributed to errors during problem synthesis. If the incorrect solutions were the result of expressive
blocks, these blocks can be connected to difficulties the students had in performing text-to-diagram
translations.
5.5 Summary of Results
Table 11: Summary of results provides a summary of the results discussed above.
Table 11: Summary of results
S01 S02 S03 S04 S05
ILS Seq-Glo GLO 9 GLO 7 GLO 9 SEQ 3 GLO 3
ILS Vis-Vrb VIS 7 VRB 5 VIS 3 VRB 9 VRB 1
Problem 1
1.5t^2 - 45t - 45 = 0 t 45 = 3t^2 3t^2 - 45t -
45 = 0
3t^2 - 45t -
45
3t^2 = 45t t^2 - 15t - 15
= 0
Total time (s) 135.00 330.00 480.00 720.00 1365.00
% before begin diagramming 11% 15% 17% 10% 12%
% diagramming 15% 33% 8% 8% 24%
Pre-diagramming time (s) 15 50 80.00 70.00 165.00
Diagramming time (s) 20 110.00 40.00 60.00 330.00
Number of pauses 0 5 0 4 6
Number of assists 1 2 7 10 12
# Elements Diagram 8 3 6 7 5 8
# Elements Verbal 8 6 7 8 6 8
# Diagrams 3 1 2 1 1 2
44

Events A, B,
C
A-B A-B, B-C A-B-C A-C A-C, A-B-C
Expressive blocks Y N N Y N
Uncertainty or confusion N Y N N Y
Prior Knowledge 5 4 3 4 3 4

Problem 2
h = (15/2)tan55 cos55 2.5 = h =
15/cos55
h =
(r)tan55; C
= 2r
x =
[(tan55)2]/15
h = tan55 2;
r = 2
Total time (s) 145 435 720 225 345
% Pre-diagramming 21% 1% 2% 13% 3%
% Diagramming 10% 33% 10% 29% 84%
Pre-diagramming time (s) 30.00 5.00 15.00 30.00 10.00
Diagramming time (s) 15.00 145.00 75.00 65.00 290.00
Number of pauses 0 8 0 0 1
Number of assists 0 7 0 2 3
# Elements Diagram 7 5 7 7 7 7
# Elements Verbal 7 6 7 7 7 7
Expressive blocks Y Y N N N
Uncertainty or confusion N Y N N Y
Prior Knowledge 5 3 4 3 3 3

General
Motivation Procedure Need Need Procedure Need
Bound across systems Y Y Y Y Y

5.6 Connecting Interpretations to Learning Style Preferences
Data analysis identified some potential correlations between problem solving behaviours and visual-
verbal learning style preferences, as described below. Research limitations and future steps that could be
performed to obtain more informative data are discussed in Section 6.4.
5.6.1 Balanced Visual-Verbal Preferences
S03 and S05 have balanced visual-verbal preferences. They solved both problems most correctly, and
represented problem statements most accurately in diagrammatic form as well as in verbalizations of their
internal representations. This pattern suggests that individuals with balanced visual-verbal preferences
may demonstrate greater text-to-diagram translation capabilities as they are comfortable comprehending
45

and generating information in both semiotic forms. S03 and S05 spent longer than at least two other
participants in solving each problem, and produced worksheets heavy in textual, diagrammatic, and
symbolic information.
5.6.2 Visual Preferences
S01 is the participant with the strongest visual preference. He represented 3/8 Problem 1 elements in one
diagram and 5/7 Problem 2 elements in a top view diagram. He showed little evidence of referring back to
his diagrams. If S01 does have a strong visual preference as his ILS score indicates, he may have relied
primarily on visualizations as he performed the problem solving activities. He visualized the fountain
height without constructing a diagram of the side view. S01 did not solve either problem correctly,
suggesting that his visualization techniques were ineffective, or he did not comprehend all of the
information in the text-based problem statements. However he did not verbalize uncertainty or confusion
during his processes. S01 was the only participant who did not write any complete words on his
worksheet his written material was only in diagrammatic and symbolic form. All other participants
recorded or represented some quantity of information in textual form.
S03 has the next strongest visual preference, and S01 and S03 share a very strong global preference.
Analysis did not identify meaningful similarities between S01 and S02's problem solving behaviours. The
only notable observation is that S03 demonstrated the second lowest use of written textual information.
S03 also stated that she needed to read problem statements in her head: "if I read it out loud I won't know
what I'm reading."
5.6.3 Verbal Preferences
S04 is the participant with the strongest verbal preference. She represented 5/8 Problem 1 elements in one
diagram and all seven Problem 2 elements in one perspective diagram. She did not produce a correct
solution for Problem 1, but was correct in Problem 2 except for one error in an algebraic manipulation
step.
46

S02 has a moderate verbal preference, but analysis did not identify meaningful similarities between S02
and S04's problem solving behaviours. S02 produced an incorrect solution for Problem 2 despite
representing all elements in her diagram. She made errors during problem synthesis when the symbolic
association between "h" and "hypotenuse" interfered with her representation of "height" as "h" in her
diagrams.
5.6.4 Strong Visual-Verbal Preferences
S01 (VIS7) and S04 (VRB9) have the most polarized preferences, and demonstrated some similar
patterns in their problem solving processes. They both indicated that their diagramming processes were
motivated by standard procedures whereas the other participants constructed diagrams when they
identified the need to do so. S01 may have adopted this mechanism to overcome deficiencies in
understanding textual problems which could make it more difficult to assess when diagramming is
needed. S04 may have adopted this mechanism to overcome deficiencies in internal visualizations.
Both S01 and S04 failed to represent the same key Problem 1 element ("car gets ahead of trooper") in
their diagrams, and produced identical incorrect symbolic solutions.
5.6.5 Sequential-Global Preferences
S04's preference is balanced and oriented slightly towards sequential, which corresponds with her
procedural motivation for constructing diagrams and performing other problem solving processes. Her
written material appears to be the most structured and neat.
S01, S02, and S03 have moderate to very strong global preferences but did not show evidence of
similarities in problem solving behaviour. S05 is balanced, with a slight orientation towards global; her
total activity time was significantly longer than other subjects. Inspection of her verbal transcript suggests
she desired to understand the problem scenarios entirely, but did not have structured methods for doing
so.
47

6 Conclusions
This chapter presents outcomes from the study, a discussion of potential implications for problem solving
teaching interventions, and limitations of the study that could be addressed in future work.
6.1 Research Question Outcomes
This section summarizes findings through the structure of the research questions.
6.1.1 What motivated participants to perform text to diagram translations?
A portion of students appeared to construct diagrams as an inherent stage of their problem solving
process, while the other participants represented the text in diagram form when they determined
diagrammatic representations would aid their problem solving processes.
6.1.2 Did participants perform diagramming processes early and quickly?
Participants displayed a strong tendency to perform text-to-diagram translations early in their problem
solving processes. A typical process involved reading some or all of the problem statement, annotating or
writing down information, then diagramming. Participants showed variation in the efficiency of their
diagramming processes. They often had to return to the problem statement to obtain omitted information
or clarify their understanding of the verbal information.
6.1.3 Did participants produce accurate diagrams?
The majority of participants omitted elements in the diagrams they produced for the first problem but they
generally produced accurate diagrams to represent the second problem. Some participants supplemented
incomplete diagrams with textual representations.
6.1.4 Did participants verbalize accurate representations of the problems?
Participants exhibited a tendency to verbalize representations of the problems that were more accurate
than their diagrammatic representations.
48

6.1.5 Were participants unsure of how to represent elements of the text in their
diagrams?
Some participants indicated that they were not sure how to represent elements in their diagrams. This
tendency was most evident when participants were trying to represent the relationships between a series
of events.
6.1.6 What visual vocabulary patterns emerged in participants' diagrams? Did
participants exhibit diagramming conventions?
Participants demonstrated inconsistent and unique visual vocabularies. Some participants employed the
same visual to signify several separate elements of a diagram. Technical visual notations including
dimension and extension lines, and angle notations were produced by some participants.
6.1.7 Did participants have difficulty representing elements of the text in their diagrams
due to conceptual prior knowledge issues?
The research methodology only elicited a limited understanding of the prior knowledge participants
referred to during translations from text to diagram. The problems only required knowledge within
participants' realms of experience, and participants showed a tendency to recognize prior knowledge in
problem representation and problem synthesis phases.
6.1.8 Are internal representations bound across semiotic systems?
There was a strong tendency for participants to write or verbalize symbolic representations during
diagramming, indicating that internal representations are bound across semiotic systems. In some cases
these associations were advantageous while in others they disadvantaged students.
6.1.9 Did expressive blocks emerge?
Correlations suggested that inaccurate diagrams may have been expressive blocks against successful
problem solving. The data could did not identify or eliminate other potential sources of solution errors.
49

6.1.10 Are text-to-diagram translations a major source of difficulty during problem
solving?
Based on the research and analysis methods and the findings that emerged, it appears that translations
from text to diagram are one major source of difficulty during problem solving. Students struggle to
represent problems diagrammatically, and produce incorrect solutions based on their inaccurate diagrams
which act as expressive blocks. The engineering education community shares this belief but also
recognizes other sources of difficulty.
6.2 Revisiting the Thesis Objectives
The first objective of this thesis was to understand implicit problem solving processes performed by
engineering students. The research activity and analysis provided some insight into the cognitive
processes of a small group of students, and these results can contribute to further studies that attempt to
develop an understanding of problem solving processes in the broader engineering student body.
The second objective was to investigate deficiencies in text-to-diagram translations. Results suggest that
students are unable to produce accurate diagrams because they have not explicitly been taught translation
processes or methods of representation, and if they have, they do not know how to use diagrams to their
full advantage in mathematical analysis. Again, these findings are limited by the research and analysis
methodologies of this study, but they do identify relevant areas for future studies.
The final objective was to explore problem solving capabilities and deficiencies in terms of student
learning profiles. Interpreting the results in terms of participants' ILS profiles, it seems that students with
a very strong visual preference or very strong verbal preference will be least capable at text-to-diagram
translations. Students with a sequential preference may be more likely to adopt suggested problem solving
processes.
50

6.3 Implications
This study focused on well-defined analytical problems related to concepts fourth-year engineering
students are familiar with. The three stages of the TDS algorithm and IPS model; understanding, framing,
and synthesis, are also applicable to ill-defined problems that require students to comprehend ambiguous
or incomplete information and frame their assumptions through the text-to-diagram translation process.
These are the types of problems engineers will be solving as they move beyond the well-defined problems
of their foundational years. If they are unable to competently perform translations, inaccuracies in the
diagrammatic representations of ill-defined problems may lead students with strong analytical skills to
correctly solve the wrong problem. In addition to mathematical fundamentals, engineering students need
to be taught fundamental problem solving processes that can be extended to open-ended, real-world
problems. This research project has identified preliminary implications for any teaching interventions
being developed to address student problem solving deficiencies.
Some students with a sequential learning preference might translate from text to diagram without
understanding the problem or how diagrams will help them solve it, potentially creating inaccurate or
uninformative diagrams. As stated in the literature, "many students see diagrams as merely fulfilling a
component of the assignment rather than as an essential cognitive tool" (Waller, LeDoux & Newstetter,
2013). Findings from this project suggest that students with sequential learning profiles may express such
attitudes, and ILS studies have shown engineering students to be more sequential than other populations.
Teaching interventions are needed to help a large population of engineering students understand that
diagramming is often necessary and useful.
Interpretation of the research data identified a tendency for translations from text to diagram to be a more
prominent source of difficulty for engineering students with strong visual or verbal preferences. Students
with strong visual learning profiles require teaching interventions that enable them to become more
proficient at comprehending problem statements (problem representation). Students with strong verbal
learning profiles require interventions that will enhance their ability to represent information
51

diagrammatically (problem framing). These contrasting needs pose a challenge for educators as generic
teaching interventions may not fully address these deficiencies in students' problem solving techniques.
The most challenging translation in this study involved the representation of bodies travelling at different
rates and altering their movements in a sequence of related events. Study participants struggled to capture
the motion in static diagrams even though they held accurate internal representations. It appears that
engineering students need to be explicitly taught how to identify important events and represent these
individual stages as well as their relationships.
Study participants were inconsistent in their symbolic notations on diagrams and as a group did not
demonstrate a shared visual vocabulary. One criterion for effective engineering diagrams is "the
notational system used to reference elements in the diagram should be consistent and meaningful"
(Waller, LeDoux & Newstetter, 2013). The results of this study suggest that students must be taught why
and how they should use consistent symbolic notation as they translate problem statements to useful
diagrams. Providing students with a set of standard visual vocabulary would also be an effective means of
improving translations from text to diagram, as students not have to expend cognitive effort determining
how to represent elements.
6.4 Limitations and Future Work
This thesis project was self-directed work by an undergraduate engineering student. The researcher is a
novice when it comes to scholarly research and has limited knowledge in the fields of engineering
education and cognitive psychology. This status meant that the research was chosen to be very specific in
scope. External perspectives were not consulted to ensure consistency or correctness in the transcript
codifying process, and results and interpretations generated from qualitative analysis methods represent
the biases of the researcher.
The thesis course spanned an eight-month academic year. The time-commitment for background research
and manual coding restricted the number of participants to a small quantity. Ideally the study would
52

incorporate a large number of participants in each learning style category to allow for a more meaningful
investigation. It is also important to remember that students alter their actions in response to particular
tasks and do not always demonstrate their preferred learning style.
To extend the study and obtain more definitive results, the following recommendations are suggested for
future work:
Increase the quantity of participants and variety of problems to solve.
Include participants with different levels of experience, e.g. first-year students, fourth-year
students, graduate students, professors.
Select problems that participants have addressed more recently to eliminate the anxiety that arose
in this study due to problem unfamiliarity.
Alter the problem solving activity structure to be more like a test environment; enforce time
limits, do not prompt or assist participants, and ensure participants are motivated to persevere
with problems.
Provide support materials such as calculators or basic formulae, or instruct participants to focus
on the construction of diagrams.
Record video footage to capture specific problem solving actions.
Incorporate eye tracking technology to analyse focus of attention.
Have additional researchers participate in an iterative coding process that ensures grouping is
consistent and agreed upon.
Investigate a smaller selection of research questions in more detail.
Address research questions that were beyond the scope of this study:
o Do students perform translations iteratively?
o Do students detect and resolve errors?
o Do students demonstrate expert problem solving behaviour?
o What are the cognitive requirements of the different problem solving processes?
53

Include a survey about general problem solving processes and attitudes towards diagramming in
problem solving.

54

7 Works Cited
Carberry, A. R., McKenna, A. F. & Dalrymple, O. O., "Eliciting students' interpretations of
engineering representations." 2012 ASEE Annual Conference. (2013): n. page. Web.
<http://www.asee.org/public/conferences/8/papers/3218/view>.
Felder, R. M., & Soloman, B. A. (n.d.). Index of Learning Styles (ILS). Retrieved from
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSpage.html
Kolb, D. A. (1999). The kolb learning style inventory. (Version 3 ed.). Experience Based Learning
Systems, Inc.
Litzinger, T. A., Lee, S. H., Wise, J. C., & Felder, R. M. (2005). "A study of the reliability and validity of
the Felder-Soloman Index of Learning Styles." Proceedings of the 2005 ASEE Annual
Conference.
Litzinger, T., Van Meter, P., Wright, M., and Kulikowich, J., "A cognitive study of modeling during
problem-solving: An Integrated Problem Solving model," ASEE Annual Conf. and Exp.,
Chicago, IL, June 2006.
McCracken, W. M., & Newstetter, W. C. (2001). "Text to diagram to symbol: Representational
transformations in problem-solving. " FIE 2001 Reno 31st Annual Frontiers in Education
Conference, F2G-13.
Newcombe, N. S., & Stieff, M. (2012). "Six myths about spatial thinking. " International Journal of
Science Education, 34(6), 955-971. doi: 10.1080/09500693.2011.588728
Serway, R. A., & Jewett, J. W. (2008). Physics for scientists and engineers. (7 ed., Vol. 1). Stamford, CT:
Cengage Learning.
Waller, A. A., LeDoux, J. M., & Newstetter, W. C. (2013). "What makes an effective engineering
diagram? a comparative study of novices and experts." 120th ASEE Annual Conference &
Exposition, Retrieved from http://www.asee.org/public/conferences/20/papers/6903/view
Wankat, Phillip C and Frank S Oreovicz. Teaching Engineering. Purdue University, 1993. 19 02 2014.
<https://engineering.purdue.edu/ChE/AboutUs/Publications/TeachingEng>.

55

8 Bibliography

Anderson, E. E., & Taraban, R. (2008). "Comparison of recent engineering problem-solving models."
ICEE 2008, Retrieved from
http://www.ineer.org/Events/ICEE2008/full_papers/full_paper119.pdf
Case, J. M., & Light, G. (2011). "Emerging research methodologies in engineering education
research." Journal of Engineering Education, 100(1), 186-210. doi: 10.1002/j. 2168-
9830.2011.tb00008.x
Felder, R. M. and Silverman L. K. (1988) "Learning and Teaching Styles in Engineering Education,"
Journal of Engineering Education, 78(7), 674-681.
Felder, R. M., & Soloman, B. A. (n.d.). Learning styles and strategies. Retrieved from
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/styles.htm
Johri, A., Roth, W., & Olds, B. M. (2013). "The role of representations in engineering practices: Taking
a turn towards inscriptions. " Journal of Engineering Education, 102(1), 2-19. doi:
10.1002/jee.20005
Jonassen, D., Strobel, J., & Beng Lee, C. (2006). "Everyday problem solving in engineering: Lessons for
engineering educators." Journal of Engineering Education, 95 (2), 139-151.
Kaplan, R. M., D.P. Sacuzzo, (2009, 7th ed.). Psychological testing: Principles, applications, and issues.
Belmont, CA: Wadsworth.
Lumsdaine, M. & Lumsdaine, E. (1995). "Thinking preferences of engineering students: Implications for
curriculum restructuring. " Journal of Engineering Education, 84(2), 193-204.
Malgorzata, S. Z. & Waalen, J. K. (2002). "The effect of individual learning styles on student outcomes in
technology-enabled education. " Global Journal of Engineering Education, 6(1), 3543.
McKenna, A. F., & Agogino, A.M. (2004). Supporting mechanical reasoning with a representational-rich
learning environment. Journal of Engineering Education, 93 (2), 97-104.
Taraban, R. C., & Anderson, E. (2011) "Using paper-and-pencil solutions to assess problem solving
skill. " Journal of Engineering Education, 100(3), 498-519.
Wai, J., Lubinski, D., and Benbow, C. P., (2009), "Spatial ability for STEM domains: Aligning over 50
years of cumulative psychological knowledge solidifies its importance. " Journal of Educational
Psychology, 101: 817-835.


56

Appendix A: Problem Solving Activity Problems

Problem 1 (Serway & Jewett, 2008):
A car traveling at a constant speed of 45.0 m/s passes a trooper on a motorcycle hidden behind a
billboard. One second after the speeding car passes the billboard, the trooper sets out from the billboard
to catch the car, accelerating at a constant rate of 3.00 m/s2. How long does it take her to overtake the
car?

Problem 2 (Serway & Jewett, 2008):
A high fountain of water is located at the centre of a circular pool. Not wishing to get his feet wet, a
student walks around the pool and measures its circumference to be 15.0 m. Next, the student stands at
the edge of the pool and uses a protractor to gauge the angle of elevation of the top of the fountain to be
55.0 degrees. How high is the fountain?


57

Appendix B: Recruitment Email

Dear classmates,
Please help me complete my engineering education thesis by volunteering as a research
participant. Participation will only take a couple of hours of your time and may help you with
future coursework. I am investigating the processes students with different learning styles use to
solve textbook-style engineering problems. I am selecting up to six participants from this course
since we have already determined our MBTI personality types. If you are willing to participate,
please read on.
Attached is the Informed Consent form for my study which contains full procedural details as
well as useful contact information. If you are selected to participate, you will be asked to
complete a short learning style survey online. In early January, I will then have you solve two
first year textbook problems in an individual interview. I will be analyzing your written work and
an audio recording of the problem solving activity.
Please read over the Informed Consent attachment carefully and return a signed copy to me in
person. Please respond to this email with the following contact and demographic information:
Name:
Email:
Telephone
Age:
Engineering discipline:
Gender:
MBTI:

Thank you very much,
Nikita Dawe

58

Appendix C: Participant Information and Learning Style Profiles

Table 12: Participant Information
Age Gender Discipline MBTI
22 M Civil Engineering INFP
21 F Chemical Engineering INTP
22 F Industrial Engineering ENTJ
22 F Materials Science and Engineering ESTJ
23 F Industrial Engineering ESFJ

Table 13: Learning Style Profiles
Subject Act-Ref Sen-Int Vis-Vrb Seq-Glo
S01 ACT 3 SEN 5 VIS 7 GLO 9
S02 ACT 1 INT 5 VRB 5 GLO 7
S03 ACT 1 INT 11 VIS 3 GLO 9
S04 ACT 3 SEN 7 VRB 9 SEQ 3
S05 ACT 7 SEN 1 VRB 1 GLO 3
1-3: Balanced
5-7: Moderate preference; may learn more easily if the preference is supported
9-11: Very strong preference; may experience difficulty if preference not supported

Engineering students are typically more sequential, sensing, and visual than non-engineering students
(Litzinger, Wise, & Felder, 2005). None of the participants demonstrate all three typical engineering
student preferences. The research did not seek to investigate a representative sample of the engineering
population.


59

Appendix D: Terminology
Semiotic systems: T (verbal/textual), D (diagrammatic), S (symbolic)
TDS algorithm: Text to Diagram to Symbol algorithm
IPS model: Integrated Problem Solving model
TDS and IPS phases: R (representation), F (framing), S (synthesis)
IPS components: P (problem solving), K (knowledge-driven)
Problems: P1 (trooper and speeding car), P2 (fountain height)
Problem 1 events: A (car passes trooper), B (trooper sets out), C (trooper overtakes car)
Subjects/Participants: S01-S05

60

Appendix E: Iterative Coding Terms
reading problem
annotating problem
understanding problem
assuming
sketching
writing symbols
deriving
guessing equation
calculating
reviewing symbols
reviewing sketch
process
presenting solution
thinking
remembering
forgotten
confusion
negativity
researcher question
researcher hint

61

Appendix F: TDS and IPS Coding of Transcripts
1. Semiotic System 3. Phase 4a. Problem Solving
Processes (P)
4b. Prior Knowledge (K)
Verbal (T) Problem Representation (R) Set subgoals (P2) Pattern recognition (K1)
holds a verbal
representation, engages in
data gathering
Planning (P3) Determine deep structure
(K2)
Diagrammatic (D) Problem Framing (F) Execution of plans (P1) Prior knowledge mapped
(K1)
generates hypotheses and
makes assumptions about
the problem and draws an
external diagram to depict
these understandings
Mapping givens (P2) Analogical problems
retrieved (K2)
Mapping knowledge (P3)
Monitoring (P4)
Evaluation of final diagram
(P5)

Symbolic (S) Problem Synthesis (S) Execution of plans (P1) Mathematical expression
(K1)
translates the diagram into
a set of mathematical
equations
Mapping givens (P2)
Mapping knowledge (P3)
Monitoring (P4)
Evaluation of final solution
(P5)



62


Appendix G: Problem Solutions
Problem 1 solution (Serway & Jewett, 2008):



63

Problem 2 solution (Serway & Jewett, 2008):


64

Appendix H: Rubrics for Analysis of Diagram Accuracy

Table 14: Problem 1 rubric
Problem 1
Represent or Omit
v(car) = 45m/s; constant Error: v(car) 45m/s, v(car) 0, v(trooper) = 45m/s
trooper starts from v = 0 Error: trooper starts from v 0
1s wait time Error: wait time 1s
a(trooper) = 3ms-2 Error: a(trooper) 3ms-2, wrong variable = 3ms-2
t = ? Error: solve for different measurement
Connect
billboard = origin (d = 0) Error: origin elsewhere
car gets ahead of trooper Error: trooper accelerates from location of car
d(car) = d(trooper) Error: distances not equal
Irrelevant Irrelevant information included in diagram(s)
Ideal Solution:
Solution 1.5t^2 - 45t - 45 = 0
# Diagrams 3
Events A, B, C
# Elements Represented 8

Table 15: Problem 2 rubric
Problem 2
Represent or Omit
circular pool Error: wrong shape
fountain at centre Error: fountain elsewhere, fountain not vertical
C = 15m Error: C 15m, wrong var = 15m
= 55 Error: 55, wrong var = 55
h = ? Error: solve for different measurement
Connect
C r (top view) Error: radius not determined
, r h (side view) Error: height not determined
Irrelevant Irrelevant information included in diagram(s)
Ideal Solution:
Solution h = (15/2)tan55
# Diagrams 2
Views top, side
# Elements Represented 7


65

Appendix I: Participant Worksheets

S01:


66

S02:


67

S03:


68

S04:


69

S05: