Designing Video for Massive Open Online-Education

Research on the effects of collaborative tasks framing the use of video

Carmen Zahn, Karsten Krauskopf, Jonas Kiener & Friedrich W. Hesse

video source: https://www.coursera.org/course/inf4oec

 Theory Background:
Conceptual challenges from a learner-centered perspective

Experimental Study:
Effects emerging from different tasks framing the usage of online-video (tools) on student collaboration and learning

Discussion:
Future Research relating to MOOCs

Carmen Zahn

10.02.2014

Conceptual challenges in MOOC-Designspaces

Instruction E.g. Lecture, readings, videos, interactivity Content E.g. Domain, Modularization, Pacing

Assessment Community E.g. In-video quizzes, homework, E.g. Discussion board, social group projects media, video chat, text chat
(Taxonomy: General Features of Moocs, Schneider, 2013)

Carmen Zahn

10.02.2014

Design challenge # 1
Instruction E.g. Lecture, readings, videos, interactivity Content E.g. Domain, Modularization, Pacing

(Taxonomy: General Features of Moocs, Schneider, 2013)

Design instructional video/video lectures that support individual understanding

Carmen Zahn

10.02.2014

Design Challenge # 2

Assessment Community E.g. In-video quizzes, homework, E.g. Discussion board, social (Taxonomy: General Features of Moocs, Schneider, 2013) group projects media, video chat, text chat

Connect video materials to meaningful learning experiences and the community = Effective (collaborative) tasks & assessment

Carmen Zahn

10.02.2014

Related research on video-based learning

Showing video is not the most efficient way to activate learners` mental effort especially problematic for learning of complex topics (e.g., compared to text, Salomon, 1984)

Digital video tools (e.g. highlighting, editing functions, annotation tools, hypervideo) successfully enable learning activities necessary for learning of complex topics (e.g.,
Schwan & Riempp, 2004; Spiro, et al. 2007; Zahn, Pea, et al., 2005)

Embedding video in collaborative and creative tasks is a powerful strategy to stimulate and enhance learning from video (e.g., Schwartz & Hartmann, 2007; Zahn et al., 2013)

More related research in a nutshell

Video collaboration tools (e.g., hypervideo with discussion tool, webspaces for coediting and sharing comments, etc.) improve learning of complex topics (e.g., Goldman, 2004; Zahn, Pea, Hesse & Rosen, 2010) not only in lab, but in real learning scenarios of school-based and university education (Pea et al., 2004; Pea et al., 2006; Stahl, Zahn, Finke, 2005; Zahn, Krauskopf, Hesse & Pea, 2010) Extra-support for social interactions when learning with video tools increases success
(e.g., Zahn, Krauskopf, Hesse & Pea, 2012)

 Theory Background:
Conceptual challenges from a learner-centered perspective

Experimental Study:
Effects emerging from different tasks framing the usage of online-video (tools) on student collaboration and learning

Discussion:
Future Research relating to MOOCs

Carmen Zahn

10.02.2014

Goals of the study

Test if specific video tools can be helpful for collaborative online-learning in a complex domain like history

Initial exploration of the question Which effects can we expect from different tasks framing the use of those video tools on collaborative online-learning and outcome?

Experimental Learning Environment

WebDIVER TM by Roy Pea, Stanford University

(Pea et al., 2004)

10

Task and Experimental Design

Students used WebDiverTM for discussion and multimedia assignments Online-Lesson: German History (Berlin Airlift) Video & Materials: Original Newsreel (historical source) & Textbook information Learning Goal: Integration of content knowledge and an understanding of the propagandistic functions of the newsreel = Evaluation of historical source of evidence
(e.g., Lindsay et al. 2013)

Experimental Variation: Discussion vs. design tasks framing video use

Group 1:analyse and comment on the video within an online-discussion Group 2:analyse and comment on video in order to design a hypertext-like product, so that other student learners can come to a good understanding of the newsreel (multimedia design)

11

Measures for Collaboration and Learning

MC-Tests, analyses of WebDiverTM panels & comments and screen-videos
Level 1: Cognitive learning outcome Variable History content knowledge acquisition Measure Factual Knowledge Test Picture Recognition Test

2: Surface level effects on Performance, collaboration and collaboration and learning learning - quantity

Number of panels created in partnership Number of comments Length of comments Collaboration index

3: Deeper level effects on Performance, collaboration and collaboration and learning learning - quality

Number of panels referring to details Number of utterances in comments addressing historical content Number of utterances in comments addressing filmic style Number of utterances in comments integrating aspects of historical content and filmic style

12

Level 1: Cognitive Outcome

Design-condition (n = 19 dyads) Discussion-condition (n = 17 dyads) Total t-Test (N = 36 dyads) Effect size

Indicator

SD

SD

SD

t(34)

Factual knowledge Picture recognition

33.4

2.5

34.0

1.7

33.7

2.1

-0.85

.40

25.7

1.3

24.9

1.4

25.3

1.3

1.79

.08

0.7

In sum: Similar cognitive outcome from both tasks, marginal on picture recognition, medium effect size (Cohen, 1988)

13

Level 2: Surface Level Collaborative Activity

Design-condition (n = 19 dyads) Discussion-condition (n = 17 dyads) Total t-Test (N = 36 dyads) M 32 518.7 SD 9.6 225.9 t(10) -2.5 -2.4 p .02** .02** d 0.8 0.8 Effect size

M Number of comments Length of comments (in words) 28 426.7

SD 8.7 161.1

M 36.1 610.6

SD 10.6 290.6

n = 6 dyads Collaboration index Dives created in partnership 12.3 4.2 10.6 4.2

n = 6 dyads 33.6 12.0 21.9 6.9

n = 12 dyads 22.6 8.1 16.3 5.6 -2.08 -2.26 .07+ .05* 0.7 1.4

In sum: Discussion group = **More collaborative activities than design group

14

Level 3: Deeper Level Collaborative Activity

Design-condition Discussion-condition (n = 4 dyads) M SD Total t-Test (n = 6 dyads) Indicator Utterances integrating historic and filmic aspects M SD (n = 10 dyads) M SD t(8) p d Effect size

5.2

1.7

3.3

1.2

4.3

1.5

2.23

.05*

1.3

Utterances addressing historical content

5.8

3.1

4.6

1.7

5.2

2.4

0.87

.41

Utterances addressing filmic style

12.7

5.2

18.1

2.9

15.4

4.1

-2.24

.05*

1.3

n = 19 Number of details 2.2 2.8 0.6

n = 17 1.5 1.4

N = 36 2.2 116a .16

In sum: Design group *stronger in discussing an integrative view of historic content and filmic style (source evaluation!), while discussion group focuses on one aspect
15

Summary

Under the surface of apparently similar cognitive outcomes in multiple choice tests asking for factual knowledge, fine-grained differences in important other aspects of online learning became explicit:

While the discussion task stimulated significantly more collaborative surface activity than the design task

the design task stimulated for more knowledge intensive elaboration than the discussion task, such as an integrative view on video content and style

(Please remember the learning goals in our domain of history!)

16

Implications of the Results of this Study

If we know that tasks can influence what we see in an online-video Teachers can decide according to their specific teaching goals which task is to prefer (discussion or multimedia design assignments)

We might only speculate about how such differences (obtained in an ad-hoc experiment, not MOOC) might add up in longer-lasting courses (e.g., tasks over several weeks time) and then extend to cognitive outcomes (e.g., multiple choice quizzes).

17

General Implications from Video-Research

Recommendations beyond design of instructional videos

Video editing and annotation tools provided with video lectures and video sources make video a true working medium which students can use actively and collaboratively for learning.

Think carefully about integration of video tools and tasks.

Research on MOOC video effects is definitely needed!

18

 Thank you for your attention!

Carmen Zahn

10.02.2014

19

 Theory Background:
Conceptual challenges from a learner-centered perspective

Experimental Study:
Effects emerging from different tasks framing the usage of online-video (tools) on student collaboration and learning

Discussion:
Future Research relating to MOOCs

Carmen Zahn

10.02.2014

20