QUANTITATIVE VS.

QUALITATIVE RESEARCH
Methodological Critique

MARCH 15, 2015

SUSAN BEELEY
ETEC 500: Research Methodology in Education
1 Methodological Critique #1

Introduction

This Methodological Critique will consider the articles Can Instructional and Emotional

Support in the First-Grade Classroom Make a Difference for Children at Risk of School Failure?

(Hamre & Pianta, 2005) as an example of Quantitative work, and Developing Teacher

Epistemological Sophistication About Multicultural Curriulum: A Case Study (Sleeter, 2009) as

an example of Qualitative work.

The former article uses the experimental method to take a rigorously scientific look at the

question of whether strong instructional and emotional support during Grade 1 can help to

decrease the risk of school failure for at risk students. Using rather intensive statistical analysis

of standardised instruments the authors were able to draw the conclusion that such instructional

and emotional support does allow at risk students to achieve at a similar level to their low risk

counterparts.

The later article uses the case study of one second year teacher who was carefully

selected to have her artifacts analysed to determine whether encouraging novice teachers to take

a more complex approach to consideration of the curriculum might inform their practice and

pedagogy. The data came in the form of journals, papers, notes, and interviews and was

analysed using a rubric to determine that in this case the new teacher was able to think more

complexly about curriculum and to challenge the institution of education in her curriculum

delivery.

Part 1: Descriptive Analysis and Critique

I will be addressing the suitability of the methods selected for looking at the problems of

interest in the above articles separately, but will interchange information from the two articles to
2 Methodological Critique #1

examine the major differences between the quantitative and qualitative approach across key

dimensions later in this critique.

The method used by Hamre and Pianta (2005) was a natural experiment. This method

was selected because the authors were hoping to establish a cause and effect relationship

whereby high quality intellectual and emotional support during grade one allowed at risk

students to achieve as well as their low risk peers. The independent variable in this case is the

quality of intellectual and emotional support. If there was some prior evidence to suggest that

high quality support would help at risk students achieve, it would be unethical to condemn any of

the at risk students to a low quality support group. For this reason, taking advantage of pre-

existing data from the NICHD study to identify those at risk, observing to determine the quality

support available in the 827 classrooms, and then testing the students on a variety of dimensions

during grade one provided the required information without the independent variable being

unethically manipulated.

Sleeter (2009) opted to carry out a case study. The reason behind this choice was that

only a case study is expected to catch the complexity of a single case (Stake, 2000). Ann was

selected as she met the criteria for someone who had made big changes and therefore could teach

others a lot through her journey. A critique of case studies is that they lack external validity. It

this case, however, it is acceptable to sacrifice the external validity for what can be learned from

this best-case scenario. Further, triangulation of several sources of data from Ann was carried

out to ensure internal validity as Ann moved through the stages towards becoming

accomplished at thinking complexly about multicultural curriculum, thereby supporting the

idea that under the right conditions a new teacher can be taught to think critically about an

institutional approach to curriculum in order to adapt their delivery.

3 Methodological Critique #1

Differences between the quantitative example and the qualitative example can be seen by the use

of the following dimensions:

Description of research problem

In the quantitative study the research problem was clearly defined and the relationship

that they were searching for was specific and detailed. Previous work had been used to support a

belief that it would be found that high quality support would have a positive outcome for at risk

students. The qualitative study took a much more organic approach. The author used previous

studies not to back an expectation of outcome, but to justify a curiosity about a particular

circumstance.

Selection and Assignment of Participants

In the quantitative study the participants were drawn from a huge, national sample.

Those who were missing data or who did not fit criteria of being at risk were excluded, and at

risk was very thoroughly defined. Standardised instruments whose validity and reliability had

already been tested and confirmed by others were used to measure the criteria determined to

contribute to a student being categorised as at risk and also to measure child outcomes. The end

result was a large sample size which is good for both reliability and statistical significance, key

features of a good scientific experiment.

For the qualitative study the criterion applied to choose Ann as the case study were not

about selecting someone who was likely to respond in a certain way, but in selecting someone

who was likely to respond in a very notable way. She was selected from a larger study because

the author wanted a new teacher who was open to learning, had not had much experience with

multicultural education in the past but found herself in a diverse classroom and as such has
4 Methodological Critique #1

developed an interest in multicultural education. This allowed the author to track her

development from a novice thinker, who accepted the legitimacy of external authorities to

determine curriculum (Sleeter, 2009) to an accomplished thinker who would pay heed to

alternative views.

Data Collection and Analysis and Procedures and Instruments

For the quantitative research the data gathered was largely based on standardised

instruments with proven reliability and validity. All of these instruments produced quantitative

results directly so there was little chance of subjectivity in the interpretation of results. All of the

data collected this way was subjected to statistical analysis and was therefore used as supporting

evidence for the claims of the authors.

In the qualitative research data took to form of papers, journal writings, notes on

observation and interviews. A rubric was developed to allow for classification of each artifact as

showing evidence of novice, developing or accomplished thinking complexity. Though

subjectivity could pose a problem in the analysis of such artifacts, triangulation was used to

ensure reliability and Ann was also asked to complete the rubric, the results of her analysis was

similar to the authors.

Reporting of Literature

The literature used to support the quantitative research was largely used to provide a

background to justify the cause effect relationship being proposed between the quality of

instruction and support and the achievement of at risk students. There was an abundance of

literature referenced in this paper and as much as speaking to the theories being investigated they
5 Methodological Critique #1

spoke to the legitimacy of the instruments being used to further confirm the validity and

reliability of the data collected.

In contrast, the literature cited in the qualitative study was used more often as a platform

from which the authors curiosity developed. It seemed to provide a theoretical basis upon

which the author developed her own work rather than offering pre-existing support for her ideas.

Reporting of Conclusions

The conclusions drawn in these two studies differed greatly in terms of purpose. The

conclusions of the quantitative study seemed to be aimed very directly at policy makers. They

did highlight the areas of weakness in their studies to provide areas of further research, but it is

still an area that is mainly concerned with a subset of students, those deemed as at risk. The

conclusions of the quantitative study, however, were very much aimed at teachers. The

suggestions incorporated in the conclusion were very much about teaching practice and activities

that would help individual teachers develop their own complex thinking skills. Additionally,

these activities could be applied outside of the domain of the paper itself.

Part 2: Methodologies for Your Future Research

My bias towards quantitative research has been longstanding. As a science teacher who

also has a degree in psychology statistical analysis of numerical data holds the truth. Or does it?

This activity has certainly made me reassess previous beliefs. From the initial stages of reading

these articles I found the Sleeter (2009) article much nicer to read. It was written for teachers to

read and learn from and you do not need an advanced degree in statistics to make sense of what

it is presenting. The advantage of qualitative data for new areas of discovery is clear, it presents

an open pathway of discovery in new directions. Where there is pre-existing research, however,
6 Methodological Critique #1

some quantitative data would still help to strengthen a point of view, but only when it is made

accessible to the audience for whom the research is relevant. The two approaches are most

effectively used to support one another and the applicability of one over the other is not a case of

absolute but perhaps one of the stage we are at in our research.

7 Methodological Critique #1

References:

Hamre, B. K. & Pianta, R. C. (2005). Can Instructional and Emotional Support in the First-

Grade Classroom Make a Difference for Children at Risk of School Failure? Child

Development, 76, p. 949-967

Sleeter, C., (2009). Developing Teacher Epistemological Sophistication About Multicultural

Curriculum: A Case Study. Action in Teacher Education, v. 31 no.1, p. 3-13.

Stake, R. E. (2000). The Art of Case Study Research. Thousand Oaks, CA: Sage