Redefining Educational Evaluation

Pagtakhan 1

GUIDELINES FOR EVALUATING IE PROGRAMS: EVALUATING THE SCHOOL AND THE STUDENT From what we gather now, we know that we would like to create an evaluation mechanism that is not laden with existing positivist, western-capitalist or factory output models. We would like to create an evaluation mechanism that gathers less quantitative data and more qualitative data on our students journey of learning. We would like to give students more agency in their development and to provide an evaluation program that allows their voices to be heard while they are assessed in their learning. We want an evaluation mechanism that provides a framework to mindfully follow their development of understanding in the concepts we teach them. This framework does not limit itself to predetermined outcomes because it acknowledges that each student is unique. Instead the evaluation mechanisms in place would come together as one larger narrative with the student as the main character in his/her development. What we would have at the end of a unit of instruction is a story of a person/student, who was presented with learning challenges; responded to them and has grown in conceptual development; has been presented with meaningful learning activities to challenge their thinking; has found a resolution or new view into the world as a result of having gone through this journey.

For that reason, the evaluation model I propose for evaluating an IE program is based on the same elements found in a story. Stories or narratives are key to IE because, according to Egan, our lives have a place in larger narratives (1997, p. 60). A story structure of evaluation will remind evaluators that the student is part of a bigger picture: he or she is a daughter, a brother, a baseball player, a friend to someone, along with being a character in their classroom. Because we are in the middest of them we cannot ascribe determinate meaning to them (Egan, 1997, p. 63). And because we are in the mind

Redefining Educational Evaluation

Pagtakhan 2

set of evaluating them according to curricular outcomes, limit their potential for thinking. But if we adopt a story structure to evaluation, we revise our conception of the student as a learner to mean something more; they are human beings, with belong to families, interact in communities and will be stewards of our complicated world. The student is not fictional, as characters in stories often are. But a story structure evaluation scheme does not limit our ability to attach emotions and invest in their development. In fact, I would argue that it liberates us from existing scientific models which have forced us to distance ourselves our subjects. That is the biggest humanizing factor that is missing in evaluation today. When we read novels, we invest into the characters, buy into their journey and are emotionally compelled to turn the page to see how they deal with whatever situation they are presented with. Why wouldnt we want evaluation of our students to be viewed this way? The students in our classrooms are vibrant, changing people who should not be reduced to student numbers and test scores in record books. Those numbers do not have human hopes, fears, and passions (Egan, 1997) and give no insights into a students journey in meaning-making.

A narrative-based evaluation is structured so that teachers, parents and other stakeholders are reminded that learning is on-going process in developing the character of this student. I mean character in every sense of the word their traits as a learner and young adult who is developing their moral and ethical orientation. In essence, when students finish a unit or a grade level, they are only finished one chapter of their lives. The ending of those units or grade levels only represent a resolution at that moment. However, as more challenges, events or meaning-making opportunities are presented to them, they start a new chapter. The model also reminds us that the student comes to us with something already. They come with tools, with histories, having completed other chapters before we came to meet them. They come with a point of view and cognitive tools, events and the learning

Redefining Educational Evaluation

Pagtakhan 3

activities we present them with should be constantly shaping that point of view until they again are presented with a formidable challenge which forces them to rethink and reshape themselves again.

Figure 1: Freytags Pyramid used in English 8 Short Story Units

Redefining Educational Evaluation Figure 2: Freytags Diagram Explained

Pagtakhan 4

That progression I have just described is mirrored after a Freytags pyramid for understanding short stories. The figure above illustrates the plot structure, a concept taught in the BC English 8 curriculum, and often employed when teachers begin their short story units. How we will modify this diagram to an IE framework is in the upcoming section of this paper. The diagram starts with an introduction where we meet characters and establish a setting, some kind of context for which to place our initial

Redefining Educational Evaluation

Pagtakhan 5

understanding of the story. Throughout a short story there are twists and turns in the plot which we call the rising action. It is here we learn more deeply about our character-student, how they face challenges, what skills they possess and how they have employed these cognitive tools to shape their thinking. They are met with twists and turns because the IE program should be presenting them with cognitive tools from the appropriate planning framework to reconceptualise their views. As students respond to those twists and turns, teachers using this evaluation framework can come to understand, how these students are making meaning from the content. During the rising action, the conflict in the story becomes apparent. And as the story intensifies, we reach a climax or turning point which forces the character to respond in some deep and meaningful way that changes them and resolves that conflict. The conclusion marks only the end of this aspect of the story.

Redefining Educational Evaluation

Pagtakhan 6

Figure 3: Freytags Diagram as an Evaluation Framework, (See Appendix 1 for clearer image)

This plot diagram draws many reference points to the way we teach according to the kinds of understanding, employing Egans cognitive tools. For instance, at the heart of every IE lesson is a story that contains a hero or heroic quality in which we embed the curricular content we are teaching. Just as the IE teacher looks for the heroic in the subject matter we teach, s/he needs to view our students in the same way. We should ask how our teaching and lessons encourage students to transcend current understandings and norms. In this way we are teaching questioning skills and inspiring them to be

Redefining Educational Evaluation

Pagtakhan 7

agents in their own learning, seeking out new understanding of the things around them. The rising action in the plot is where those opportunities should present themselves. It is here where teachers should be tapping into students cognitive tools in order to draw out deeper understandings. Our lessons should set them up so that they are met with crisis. We are using crisis as Dr. Natalia Gajdamaschko frequently uses the term in describing Lev Vygotskys theories in education. Crisis is the point where the students concepts are challenged, where their current understandings have an opportunity to be tested, enriched and transformed into higher level concepts (N. Gajdamaschko, personal communication, October 28, 2011). Upon meeting that crisis, the student has acquired new knowledge and this has changed them in some way. Regurgitating facts on tests and asking students to write essays that do not ask critical questions does not allow our students to develop into thinking beings. In fact it has the complete opposite effect, sending subtle messages of compliance and passivity. But when we present our students with meaningful crisiss or engage them in lessons that employ the cognitive tools to meet learning challenges head on, we allow them the opportunity to learn and resolve the new knowledge with what they understood before. They grow and have new tools for conceptualizing curricular concepts. In this way, they have reached a resolution to this chapter of learning. However, we know and accept that there are other chapters awaiting them with new crisiss to spur their development. Do stories ever really end after we read them? Egan would likely argue that yes, they do. He even states it explicitly in The Educated Mind that *t+he crucial feature of stories is that they end (Kermode, 1966 in Egan, 1997, p.63). (Thanks for that, Kieran) But I would argue that they end in some physical capacity, but the message we take from them, or whatever way we were affected by them, does not. As a teacher, I think frequently of what my former students are doing, who they are now, and who they are becoming. Not knowing is part of the mystery of teaching. This mystery is what keeps educators curious, invested, and committed to professional development. For some of us, this is unsettling. But if we look at students as people who we have taught how to think

Redefining Educational Evaluation

Pagtakhan 8

with their cognitive tools, we can trust that they are able to making meaning outside of our classrooms in whatever capacity they have chosen to. We can look with optimism that they are writing additional chapters to their story.

A narrative structure allows us to employ a specific principle of IE, specifically the use of cognitive tools to make meaning. And if a story structure is our new evaluation mechanism to guide our evaluation of IE in schools, new questions need to be asked of our students, teachers and administrators. Listed here are just some of these questions if we were to base our assessment on a student with a Romantic Understanding. That being said, assessment questions on Mythic and Philosophic have also been included because it is the nature of an IE framework to incorporate tools from other frameworks to allow for as many meaning-making opportunities for the student.

Redefining Educational Evaluation

Components of a Story as the principles to Evaluate IE Introduction Hero/heroic quality setting Questions to ask students in an IE program:

Pagtakhan 9
Questions to ask teachers in an IE program:

Questions to ask Administrators

Performance Indicators the Ministry of Education should be looking to assess whether students are doing well:
Do students possess, affective/emotional orientation, able to shape factual content into story shape Can students locate binary opposites and then determine how the content mediates between the two opposites. To what extent has the student transcended beyond the basic understanding of the concept to something that is wonderful, or heroic? To what extent has each student gained distance and perspective? To what degree did their shifts in thinking change? What did those shifts look like? Are students able to locate the wonderful or mysterious about the topic? How does the student understand the extremes and limits of this topic to be? To what extent are students able to provide a vivid mental picture of what they have learned? Can the student detect the metanarrative and give substantial reasoning backed by evidence for their thinking?

Who were you when you entered this course/subject area? What is heroic about what you are studying? Can you relate some aspect of the curricular content to your life? Explain.

What is heroic about the subject I am teaching? What heroic qualities can I model? What is the cultural context in which I am teaching?

What heroes do we want our students to identify with? Should certain heroic qualities be promoted over other ones? What school-wide activities can we create to give students an opportunity to act out on their heroic qualities?

Organization of the plot IE framework/ cognitive tools used for rising actions to occur in students conflict is presented

Your teacher has presented you with some learning challenges/conflicts. Which one did you learn the most from? What are some of the extremes and limits of this topic? What binary opposites have you derived from the course material? Regarding vivid mental imagery: describe a scene that best describes what you have learned. What would it be like to be a contemporary of this topic? Would you buy into it? Why or why not? Is there an underlying message in the topics you have studied? A larger story, perhaps? What emotional connections have you made to the material in this course? How would you personify the content? Use like or as to describe what you have learned What lessons do we learn about people or human nature from the topic? How have your values and morals been shaped by this content?

What meaningful activities can I engage students in so that they employ a range of cognitive tools? How can I expose students to enough cognitive tools to prepare them for the upcoming crisis? Which cognitive tools would best bring out a conflict for the student to resolve? How can my worksheets be geared towards collections and sets that do not just turn into copying from the textbook and busy work? How can the students play or engage in games with the material?

Can administrators rewrite or rethink their school mandates in terms of plot development or characterization of a student? What is the larger story they want for their school? If it is a success story, then define success? What genres does their school story fall into? (ex. Universitypreparatory) Are we prepared to change the language of our assessment to this story-scheme? Are administrators willing to reconceptualise learning as a developmental process the unfolds similarly to a story unfolding Are administrators willing to give teachers license to reshape that story each year as student stories differ each year How can we support teachers and students when the emotion is the priority? What kind of professional development can we provide?

Characterization appealing to the human emotion

Who is this student that approaches me in this classroom? What is their story? What chapters would we like to add to their on-going narrative?

What value has the student learned for him/herself in a wider context? What is the moral has the student learned the story/curriculum?

Redefining Educational Evaluation

Components of a Story as the principles to Evaluate IE Conflict person vs. person person vs. him/herself person vs. environment Questions to ask students in an IE program:

Pagtakhan 10
Questions to ask teachers in an IE program:

Questions to ask Administrators

Performance Indicators the Ministry of Education should be looking to assess whether students are doing well:
What conflicts can the students detect within certain ideas? How proficient are they at recognizing or seeking out anomalies? Are students aware of larger metanarrative? How do they articulate it?

How is the curricular content a reflection of the various types of conflicts How do the characters or people of the topic find justice? How do the people of the topic revolt against certain norms?

What conflicts can we present to the student that will result in meaningful learning?

Are we prepared to move out of traditional grade level groupings when students stories/chapters require that? Are we prepared to see more active students who may want to get more involved in contentious issues in society? What kind of clubs or school groups can we create to foster their sense of agency? How can we support teachers in report card writing so that it is an informative and feasible exercise?

Resolution falling action conclusion

What tools do you have for thinking? How do you understand the content of the course?

How has our student reconciled the content with themselves? How have they changed? Have we moved the student from mythic to romantic or romantic to philosophic? What scientific concepts has the student acquired from the course? What tools do they have for thinking/ What skills have they acquired?

What types of representations of students learning are acceptable representations of their understanding of the curricular concept?

Conventions literal artistic scientific concepts

What skills have you acquired? Essaywriting, effective oral communication. Which scientific concepts will you take forward in future studies?

How can our report cards be designed to reflect the students acquisition of the curricular concepts as well as an-ongoing process. How often can we report and how?

What research skills does the student have What oral and written communication skills should the student have at this point in their curricular development