Measurement

The choice of a cooperative play-based intervention provided a naturalistic group setting

for elementary school students and enabled the collection of data within this setting through
systematic observation. The objective was to observe the use of pro-social skills in real contexts
while students were engaged with one another in play.
Observation forms (Appendix D) used interval recording, i.e. the observer recorded an
occurrence of the behavior if the behavior happened at any time during the interval. Interval
recording data are presented as a percentage of the occurrences of the target behavior.




During the 1,3,5,7, & 9
th
sessions, the researcher filmed the participants playing a 10 minute
game. For interval recording, this 10-minute game was broken into five 2-minute intervals. The
researcher reviewed these videos and charted the target behaviors. Target behaviors have been
defined in this study and consist of three categories: self-regulating, cooperation, and
communication. Each category is measured by specific behaviors. Self-Regulation = accepting
consequences, demonstrating patience, adapting, using self-control through considered action.
Cooperation = following instructions, helping others, accepting help, problem solving.
Communication = listening, making a suggestion, describing thinking/needs/experience, offering
encouragement. The researcher viewed each video three times. During a single viewing,
behavior occurrence was charted for all students in the group for a single category (i.e. first video
viewing = self- regulation behavior tracking, second video viewing = cooperating behavior
tracking, and third video viewing = communication behavior tracking).
An additional data collection tool used in this study was a behavior rating scale
(Appendix G) completed by the students primary classroom teacher. Teachers completed the
questionnaire for the first time in the week prior to the first session of the intervention. The
teachers completed the questionnaire for the second time two weeks after completion of the
intervention. This questionnaire used a 4-point Likert scale and the researcher considered the
scale points to be ordinal in the sense that higher numbers represented higher values, but that the
intervals between numbers were not necessarily equal. Instead of using terms such as strongly
agree or strongly disagree, the questionnaire provided example statements for 1-4 to measure
the teachers professional opinion of the students pro-social skill competence. Questions
included: How often does the student appear to act impulsively? How well is the student able to
manage his/her disappointment or frustration? How well is the student able to follow rules?
How well does the student get along with peers? How difficult is it for the student to participate
in group activities? Pro-social skills overlap and influence each other and are difficult to
represent with a rating scale consisting of so few questions. However, the behavior rating scale
questionnaire was specifically created to have few items in order to increase compliance with
form completion. Items on the teacher questionnaire were selected based on research of the
Skillstreaming Model for Elementary School Children (McGinnis, 2012) and the ProSocial
Behavior Questionnaire (Weir & Duveen, 1981).

Data Analysis
Data was analyzed along two pathways. The first considered information about pro-
social behavior gathered from the observation forms. The second considered information about
teacher perception of students pro-social competence gathered from the teacher rating scales.
First, we will consider the methods for analysis of observation form data. These methods
investigated whether students had demonstrated a change over time in their use of the targeted
pro-social skills. The null hypothesis is: the use of pro-social skills does not increase in
subsequent cooperative games. The alternative hypothesis is: the use of pro-social skills
increases in subsequent cooperative games.
A one-way ANOVA was used to determine if there was a significant difference in the
occurrence of pro-social behavior between the games. This data reveals whether there has been a
change over time in pro-social behavior that would not be attributed to chance at the 95%
confidence level. Data was analyzed for each individual student separately, with the exception of
three students who did not have sufficient observation data due to absences which resulted in
participation in 2 or less games. Data was additionally analyzed for all students as one group,
which provided a somewhat larger sample size and more data points to provide a more accurate
representation. Finally, a t-test comparing the t-stat values for each game provide a picture of
how pro-social behavior differed from game to game. This helps to determine where the
increases occur and that it is not just one data set (or one game) that accounts for the increase in
pro-social behavior occurrence (Appendix H).
The researcher also considered Group 1 and Group 2s average percentage of pro-social
behavior occurrence for each game, based on the observation form data. These averages can be
reviewed to gain information about which specific behaviors were effected by which games and
what was the trend over time. Linear regression was used to analyze the graphs results. This
provides a picture of how the behaviors vary from game to game and how the frequency of pro-
social behaviors is changing over time.
The second focus for data analysis was to investigate how the pro-social behavior of
participating students was perceived by their primary classroom teachers pre-intervention and
post-intervention. The null hypothesis is: there is no increase in the teachers rating of student
pro-social skill competency between repeated measures. The alternative hypothesis is: there is
an increase in the teachers rating of student pro-social skill competency between repeated
measures. The first analysis of the teacher rating scales was to determine whether the variance
of the pre and post test are the same using an F-test two-sample for variances (Appendix H). The
next step was to run a two-sample t-test assuming equal variance (Appendix H).




Findings
Observation Form Data
Results from the Anova data for each individual student indicated that we cannot reject
the null hypothesis. In other words, when considering the data for each participating student
separately, it cannot be disproven that the use of pro-social skills does not increase significantly
in subsequent cooperative games. There is the exception of one student who did show a
significant increase.

F(4,55)=1.068916 F critical for F(4,55) = 2.539689
F(4,55)=1.057692

F(4,55)=1.926432

F(4,55)=1.483307

F(4,55)=1.597237

F(4,55)=3.320157

F(3,44)=1.480186 F critical for F(3,44) = 2.816466
F(4,55)=1.450176

F(2,33)=3.198413 F critical for F(2,33) = 3.284918
F(4,55)=1.2241

Figure 1 - Anova Data for Each Student

However, when the Anova data is considered for all the students as a group, we do see a
significant increase in the use of pro-social skills in subsequent games and are able to reject the
null hypothesis and accept the alternative hypothesis. When considering the whole group, the
use of pro-social skills increases in subsequent cooperative games.



ANOVA

Source of Variation SS df MS F P-value F crit
Between Groups 102.3505 4 25.58763 8.357729 1.49E-06 2.387878
Within Groups 1711.408 559 3.061553


Total 1813.759 563
Figure 2 - Anova Data for all students grouped together to form one sample

Using the average percentage of behavior occurrence for Group 1 and Group 2 an
analysis of the frequency of student pro-social behaviors across games was considered. In this
analysis, each behavior was considered separately so that information could be gained regarding
which particular behaviors were most utilized during a given game and which behaviors showed
the most increase in occurrence across games during the intervention. This data is represented in
figures 8 through 13 (Appendix H). Many of the behaviors show a large standard deviation
between data points. This indicates that behaviors varied much between games. With a low
linear regression value these behaviors are not following a predictable change. However, the
majority of the behaviors do show an increase overall in the average percentage of occurrence.
The linear regression results are seen in table 2 (Appendix H). From this data, it can be deduced
that the four behaviors for the first grade group that can best be represented linearly are:
adapting, listening, following instructions, and helping others. By analyzing the figures, it can
also be seen that three behaviors (adapting, helping others, and accepting help) all increased
tremendously from 20-76%, 16-96% and 8-92% respectively. The behavior that showed the
least amount of improvement for the first grade group was problem solving.
In the third and fourth grade group, it can easily be seen in the data that the students
behavior in the last game differed significantly from previous games. There are several possible
mitigating factors, an obvious one being student excitement about the last game and the end of
the intervention. Students were also permitted to choose which game would be played in this
last session. Because of the issues surrounding this game, this data set was excluded from the
analysis. For the group comprised of third and fourth graders, we can deduce from the data that
only two behaviors can best be represented linearly: listening and making a suggestion.
However, by analyzing the figures we can see that other behaviors increased tremendously.
Adapting, helping others, and accepting help increased from 40-70%, 28-75% and 4-75%
respectively. The behavior that showed the least improvement in the group of third and fourth
graders was Describing thinking, needs, and experience.
Considering Group 1 and Group 2 as a whole, listening and following instructions were
the two behaviors with the most frequent occurrence. The three behaviors that both groups
displayed the least were: problem solving, describing thinking/needs/experience and offering
encouragement.
Figure 3 and 4 show the bar graph representation of the average percentage of times the
students displayed a behavior in each of the three skill categories identified for the present study:
self-regulation, communication, and cooperation. For the first grade group, the pro-social
behaviors in the area of cooperation show the largest increase. During the first game played
together, these students used cooperative skills an average of 29% of the time. During the last
game played together, these students used cooperative skills an average 78% of the time. For the
third and fourth grade group, the pro-social behaviors in the area of self-regulation show the
largest increase. In the first game played together, these students used self-regulatory skills an
average of 40% of the time. During the fourth game these students played together, they used
self-regulatory skills an average of 72.5% of the time.


Figure 3 - First Grade Group

0
10
20
30
40
50
60
70
80
90
Amoeba
Tag
Stepping
Stones
Moonball Road
Hazards
Stepping
Stones
2/18/2014 2/25/2014 3/4/2014 3/11/2014 3/18/2014
Average of Communications
Average of Cooperation
Average of Self Regulating

Figure 4 - Second Grade Group


Teacher Rating Scale Data
Next, findings for the data from the teacher rating scales demonstrated that we cannot
reject the null hypothesis because t Stat < t Critical, 1.244083585 < 1.980272249, and p > a,
0.215934425 > 0.05 (Appendix H). In other words, it cannot be disproven that there is no
significant increase in the teachers rating of student pro-social skill competency between
repeated measures.
However, looking at Figure 6, we can see that for most students the mean score was
slightly larger for the post test. It is also interesting to look at the pre and post test mean score by
question in Figure 7. As a group, students were rated somewhat higher for every question, with
Question 3 showing the most increase. Question 3 = How often does the student appear to act
impulsively?

0
10
20
30
40
50
60
70
80
Amoeba
Tag
Stepping
Stones
Carry the
Moon
Road
Hazards
Amoeba
Tag
2/18/2014 2/25/2014 3/4/2014 3/11/2014 3/18/2014
Average of Communications
Average of Cooperation
Average of Self Regulating

Figure 5 - Student Mean Score Pre and Post Test

Figure 6 - Mean Score by Question Pre and Post Teacher Rating Scale


Discussion
The purpose of this study was to investigate the effects of a cooperative play-based
intervention on the use of social emotional skills by elementary school children who have
exhibited behavioral problems. The intervention was specifically designed to pair small group
cooperative games with experiential learning, which utilized strategies such as video review, role
play, and story-boarding. Three broad categories of pro-social skill were defined and measured:
0
0.5
1
1.5
2
2.5
3
3.5
Before
After
0
0.5
1
1.5
2
2.5
3
Question
1
Question
2
Question
3
Question
4
Question
5
Before
After
self-regulation, cooperation and communication. Specific observable behaviors within each
category were tracked and analyzed for trends and changes in frequency of use. Additionally,
teacher perception of participating students pro-social skill competence was also measured pre
and post intervention and considered in this study.
The results of the intervention have been positive. Although statistically significant
increases in the frequency of pro-social behaviors across games was not shown when considering
individual student data, when data was considered for all students as a group, statistically
significant increases were seen. We can conclude that the larger sample provided more data
points and therefore a bigger picture of what is happening over time during the intervention.
Over time, as students played more games together and participated in social emotional
experiential learning sessions, they began (as a group) to more frequently display behaviors
associated with pro-social skills. Specifically, both the first grade group and the third/fourth
grade group showed encouraging increases in their use of behaviors that demonstrated they were
adapting, helping others and accepting help as they played together. An interesting finding was
that the first grade group showed the least gains in problem solving, whereas for the third/fourth
grade group the least gain was seen in describing thinking/needs/experience. These results
suggest that the younger students need more guidance and modeling in problem solving skills.
The third/fourth grade students need more guidance in communicating and perhaps identifying
their thinking/needs/experience during social interactions. It is useful to look at which games
pulled most significantly from these skills. Carry the Moon and Road Hazards appear to be good
games for working on describing thinking/needs/experience. Carry the Moon and Stepping
Stones appear to be the games that required the most problem solving, however the percentages
for this behavior stayed relatively low throughout the intervention, which indicates the need for
more research and perhaps different games to encourage this behavior in students.
Another interesting finding was that listening and following instructions were
consistently high for both groups. Often, students with who exhibit behavior problems at school
are said to lack these particular skills. The results from this intervention offer initial support for
the possibility that this is not in fact where the skill deficits lie. Future research could explore
how behaviors such as problem solving and describing thinking/needs/experience might impact
classroom teachers perceptions about how well students are able to listen and follow
instructions.
Teacher perceptions were used in this study, however the researcher encountered several
methodological problems in interpreting the results, foremost of which is the validity of the
scale. The rating scale cannot be interpreted to offer assessement of students in distinct areas,
but can perhaps offer evidence of the teachers overall perception of the students pro-social skill
competence. The work of Spitzberg and Dillard (2002) suggests that pro-social communicative
behavior is highly important to teachers perception of social competence in students. The work
of Weir and Duveen (1981) provides evidence that different types of pro-social behaviors are
positively related to each other. The fact that the teachers ratings between pre and post test did
not vary significantly can be evidence that the scale provides an accurate picture of the teachers
perception, since we would not expect significant variance in so short a time period.
There are many limitations to highlight in this study. Foremost, the present study was not
long enough, nor did it have a sufficient sample size to adequately investigate the research
question. An additionally significant limitation was the amount of time the researcher/facilitator
spent with students during the intervention. Fifteen minute sessions do not provide sufficient
time to implement the planned games and instruction with fidelity. However, this was the
maximum time permitted for the sessions in the students school day schedule. This is a
frequently encountered barrier to social emotional learning interventions in the school building.
It is difficult to balance the need for students to be present in the classroom for core curriculum
instruction with the need for some students to receive targeted social emotional instruction,
which research suggests would better enable them to access the classroom core curriculum.
Providing social emotional interventions during recess, as in the present study, is one common
solution which prevents students from missing classroom instruction. However, as demonstrated
in this study, significant outcomes are unlikely to occur within a few weeks or months of
intervention. Therefore, in addition to the need for empirically validated interventions for
increasing social competence in students, there is another key research need. We need evidence
to guide decisions about how to provide these interventions in a school day. An important future
research question would explore whether pulling students from their classroom during the day to
provide these interventions does in fact disrupt academic instruction and student learning or
whether the benefits of those interventions might perhaps translate into greater academic success
despite missed class time.
A strength of the present study is its contribution to the evidence that cooperative play
can be an effective pedagogical tool for pro-social skill education. Future studies might explore
how cooperative play could be integrated throughout students school days, for example on the
playground or woven in to interdisciplinary classroom activities. Another strength of the study
is the data gained from the observations, which helps to create a picture of how pro-social
behaviors are employed by students during the various cooperative games. We can see which
behaviors are most responsive to the intervention. By also identifying the behaviors that are least
responsive, we can make recommendations for future work in these areas.
In conclusion, the experience of this intervention and research taught me a great deal
about the importance of helping children practice pro-social behaviors in a natural social setting.
I observed shared regulation at work as students played the cooperative games together and I saw
how students developed both pro-social skills and relationships through play. I also observed the
way play generated an enthusiasm for learning social emotional skills and motivated students to
participate. The results of the study reiterate for me the importance of keeping kids with
behavior problems in settings where they can play with socially adept peers. Just as the different
types of pro-social behaviors appeared to be positively related to each other, individual students
seemed to follow the group trend in frequency of pro-social skill usage.